I need to load approximately 225K contact records into Dyanmics. I need to
check if the contact exists via exact match of First, Middle, and Last Name
as well as address. If contact exists, update the record with additional
information. If it is a new contact insert it. However, the destination
system has well over a million contacts.
The first attempt was to use a cached lookup against those fields from
FilteredContact but that was overwhelmingly painful because of all the data
which needs to be cached and the package failed a few times due to memory
shortage. My second attempt was to join them in the OLE DB Source, but that
query is painful as well.
What is my best path here? Could I utilize the DetectDuplicate
functionality of the CozyRoc destination and then route those failed
records into a another CRM destination set to update?