March 22, 2006
I have started using the Konesans Checksum task from sqlis.com. It is a great task and I'm grateful that it is around.
I have two very wide flat files and I need to compare them to see all of the Insert Delete and Updates that are described. I have found that when calculating a checksum over a lot of columns, there is a significant performance hit.
Instead of calculating a checksum across 170 columns and then comparing the two checksums, I have found that using a conditional split that compares each column individually is a much faster way to get through the data. You might have to split your comparison up into multiple outputs on your conditional split, but if you union or even merge them back together afterwords, you'll be very happy with the performance.
March 22, 2006
I have a functionality request for the SSIS team.
I have come across a situation where a flat file was sorted by one column but I would like to add a secondary sort to the data pipe. I think it would be great if the sort task was clever enough to just add an additional sort column instead of having to wait to buffer the entire pipe until releasing any data downstream.
If we know that there is already one sort column on the data pipe, the sort task should be able to release data down the pipe as soon as all of that particular member of the primary sort was in the sort task buffer.
March 20, 2006
Neat freak? I am.
I like to make sure that all my boxes in my SSIS packages all line up.
If you feel the compulsion to line them up, go into the BI Studio and click on View\Toolbars and choose Alignment.
This toolbar will allow you to make all the boxes the same size, align to sides or to the center, and space them equally.