Konesans Checksum

I have started using the Konesans Checksum task from sqlis.com.  It is a great task and I'm grateful that it is around.

I have two very wide flat files and I need to compare them to see all of the Insert Delete and Updates that are described.  I have found that when calculating a checksum over a lot of columns, there is a significant performance hit.

Instead of calculating a checksum across 170 columns and then comparing the two checksums, I have found that using a conditional split that compares each column individually is a much faster way to get through the data.  You might have to split your comparison up into multiple outputs on your conditional split, but if you union or even merge them back together afterwords, you'll be very happy with the performance.

Good Luck!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: