We get a lot of CSVs and have to do transforms on them before putting them into our database. Sometimes it’s a lot of JOINed data that takes a long time to finish, and I want to make sure I’m not going to lock the table beyond a production day window.
So I spend an extra 15 minutes writing some code into a PL/PgSQL function to track the rate of INSERTs in some looped-over SELECTs. I finally work out all the CASTing issues and other typos and stupidities on my part, run the function, and begin to watch the numbers come up on the RAISE NOTICE messages…and then the imports done within 3 seconds. Truly time well spent.
>_<
