Mar 2009

Data Comparisons 0.90 Beta released

by Steve Wortham
I spent this weekend working on a few things I wanted to have done in Data Comparisons before it went from Alpha to Beta.

In the alpha version the software would look for a common primary key between the two tables you're comparing. If it can't find a primary key of the same name in both tables, it simply won't allow you to perform the comparison. That's obviously a huge limitation. So I've made the beta a bit more flexible. It'll automatically select the first key it finds in a table you select, but it'll also allow you to select a column to key the comparison off of. It'll warn you if the column you select isn't guaranteed to be unique, but it'll still allow you to perform the comparison. I think this allows the flexibility that it desperately needed.

The next thing I wanted to fix was the lack of multi-threading on the comparison itself. So I've added multi-threading to the "Results" screen where you'll see a progress bar as it does its thing. You can also click "Cancel" to stop the comparison if it's taking too long.

While I was at it I took a deeper look into the code to see if I could find any obvious bottlenecks. I happened to find a huge inefficiency where I wasn't making full use of the primary keys in the comparison. I fixed that and increased performance by at least 5x. I've done some performance tuning like this in a few places already but pretty soon I'm going to start getting a little more scientific with my measurements. I'll try to see what techniques are the fastest, use the least memory, etc. Maybe I'm weird, but I love that kind of work; getting every last ounce of performance out of my applications.

Anyway, I think that sums up what's new in the beta. There are still UI improvements I'd like to make, but overall I'm pleased with the way its working now. All I need now are more people using it. =)