Performance problem with 1 million rows representing 1000x1000 combinations

[jo...@theabrahams.ca]

I’m running into performance problems with Wizard 2 with my dataset that has about a million rows, representing pairwise combinations of about 1000 elements.  (They are origin-to-destination travel attributes for 1000 “zones”).  In particular, if I try to join datasets using the pair of categories (representing the origin destination pair) Wizard stops responding. It also is very slow to respond if I try to filter on either the origin column or the destination column, or both.

I have a fast computer (iMac Pro 14 core/28 threads) and lots of ram (64GB).

I’m wondering the best way to try to solve the problem?  I could anonymize the data and upload it somewhere I guess.  

-- 

John Abraham
HBA Specto Incorporated
j...@hbaspecto.com
403-232-1060