Dedupe Deep Dive article and video

Below are some of the takeaways of the deep dive into the Dedupe module.

Tactics for Varying Scenario Sizes

Medium Batches

Modify the merge batch size by going to DemandTools V (DTV) > Settings Gear Icon> General Settings > Change the “Merge” field.

• You can increase to up to 200 merges at a time. *If your environment includes a lot of custom Apex code, modifying the merge batch size may present conflicts and throw errors.
• Requires trial and error to find the batch size that works best for you.

Large Batches

A large batch size is considered to be about 1.5 to 2.0 million records, but this depends on your environment.

Filtering:

• By state, assuming most states are populated and accurate.
  • Run pass with all records after doing all subsets to catch those that were missed.
• Account / Company Name “Starts with” a,b,c,d,e etc
  • Run a pass with all records after doing all subsets to catch those that were missed.
• Last Name “starts with” a,b,c,d,e etc
  • Run a pass with all records after doing all subsets to catch those that were missed.

Other things to consider

There are no hard and fast limits to how many records can be evaluated for duplicates, however, if you do run into any problems, please note the following:

• To minimize API timeouts while attempting to download the requested records, increase the CRM timeout (in minutes) setting in DemandTools V (DTV) > Settings Gear Icon > General Settings > Change CRM timeout to up to 10 minutes.
• To avoid local memory issues finding duplicates or applying a winner rule, have a minimum of 8 GB of RAM installed on your PC/Mac. We highly recommend at least 16 GB of RAM or higher.

Comparison Type Examples

Email Plus Address Match

• Firstname.Lastname@example.com
• Firstname.Lastname+modifier@example.com
• All are Will@example.com
• will+welcome1@example.com
• will+welcome2@example.com
• will+welcome3@example.com

Domain

Any information to the right of the @ symbol of an email address.

• Alardiere@example.com and Anthony.Lardiere@example.com would match.

For website, it returns everything after the http:// and/or www.

• www.validity.com and validity.com would match
• www.shop.validity.com and www.live.validity.com would not match. The system sees this as “shop.com” and “live.com”

Relaxed NA Phone Match

Leaves only the 7 primary digits of the phone number.

• +1 879 555 1212 ext 500
• (879) 555 1212
• 1.879.555.1212 ext 408

Relaxed Domain

Matches top level domain:

• www.validity.com
• validity.com
• http://www.validity.com
• validity.co.uk
• www.validity.com/downloads
• john.sample@validity.com

Numeric

Compares numeric values, periods, and a leading hyphen (negative sign).

• Apartment #31 and Apt #31 will match as 31.

Negative numbers will not match with positive numbers.

• -34 and 34 will not match.

Add Alpha Clean algorithm if you would like to also ignore periods. * This is a great option for phone numbers.

Relaxed Street Address Match

• 123 NW Pavillion Ave
• 123 Pavillion St
• 123 Pavillion Avenue
• 123 Pavillion Rd.

Regular Expression

Use Expression [0-9]+ to only match on the number in the address field.

• 123 S Main St and 123 Mane St would match.

Comparison Type Tips

Country Match

• Long Name
• 2-digit ISO
• 3-digit ISO
• Numeric ISO

Normalized Postal Code

• Matches USPS 5- and 9-digit zip codes

First Name Matches

• First Name Comparison Type
• First XX Letters or First XX Words can be used if some of the first name fields contain middle initials or middle names.

Last Name Matches

• Alpha Clean for Smith-Jones and Smith Jones
• Fuzzy matching for spelling errors

Company/Account Name

• Cleaned Account Name (strips prefixes and suffixes, and replaces values of your choice)
• Add Transposed Values to catch University of NC and NC, University of
• Add Alpha Clean for punctuation differences

City Matches

• Use Alpha Clean to match St. Charles to St Charles
• Add Fuzzy Matching to catch spelling errors

Phone Number Matches

• Do not use Relaxed NA Phone Match when it is the only field you’re matching on as it will match to only the main 7 digits of the phone number.

Using Numeric Comparison Type with Alpha Clean Algorithm

Matching phone numbers without special characters and spaces

Examples:

• +1-879-555-1234
• 1-879-555-1234
• 1.879.555.1234
• 1 (879) 555-1234
• 1 879 555 1234