Dedupe

Dedupe enables you to identify and merge large quantities of duplicate records in both standard and custom CRM objects. 

• Choose from a variety of pre-built deduplication scenarios or create your own
• Use a broad array of comparison types to more easily identify duplicate records  
• Deduplicate previously identified duplicates from an external file
• Determine master records based on your company's logic or use pre-defined master rules 
• Control field values on winning records using robust field rule functionality 

Starting a Dedupe Scenario

To begin a new scenario, select Dedupe from the module list.

To start from an existing scenario, select a Dedupe scenario from My Scenarios on your Dashboard. 

Your source, object, and record selection criteria will already be set for you. You can keep all settings or change them as you go.

Step 1: Select Records

Select your source file or database under Select Source. CRM Object will use the organization your instance of DemandTools is linked to, while Import File will prompt you to upload a CSV file.

• For large jobs, check the box beside Use the Bulk API. This option groups your records into larger batches, so it uses fewer API calls. Learn more about the Bulk API option .

Under the Object to Dedupe dropdown menu, select the object you want to find duplicates of. (You'll define what you consider to be a duplicate is in Step 2.)

In the Record Selection Criteria section, choose Add Selection Condition to narrow down the accounts included in your search. Learn more about Record Selection Criteria.

• In this example, set the Billing City to start with P.

Fields to Show on Preview is an optional menu that lets you select the fields shown on your results page in step 3. If you don't use this menu, your results will include a subset of fields related to the dedupe object you selected, record criteria, and match conditions.

To move to step 2, choose Select Matching Fields.

Step 2: Match Records

In this step, you define what you consider to be duplicate records. Under the Matching Account Field dropdown, select the field you want to match on to find duplicates.

Select Comparison Type next, whether exact or more relaxed. A few examples include:

• Original Value: Use the original string value to compare.
• Cleaned Account Name: Use built-in cleanse rules to match similar names based on common punctuation, abbreviations, and common business prefixes. For example, The ABC Company and ABC Co. would be seen as matching account names.   These rules are customizable to meet your organization's specific needs.  To customize go to Settings - Manage Cleanse Rules. 
• Acronym: Use the first letter from each word in the field value.
• First X Letters: Use only part of the field value by specifying the number of characters to use.
• First X Words:
• Domain: Strip out all components of the field value except the domain portion.
• Address Cleanse: Normalize the address to ensure better matching between similar address field values.

A comprehensive list of comparison types including use cases and examples can be found here.

Four checkboxes let you adjust the logic for your search.

• Blank Values: Look for positive matches even when field values are blank for a given field. Please note that at least one field must have a non-blank match for a positive match to occur.
• Fuzzy Matching: Analyze words based on their phonetic value, removing vowels and analyzing the remaining consonants.
• Transposed Values: Analyze field values by reordering multiple words within each field value.
• Alpha Clean: Analyze field values by using only characters that are one of the 26 characters of the English alphabet, numbers 0-9, space, and ampersand.

• When you select this option and add one or more selection conditions, the results grid will only return duplicate groups where at least one of the records matches the criteria.
  • For example, you can filter to return all duplicates of an account where at least one record in the duplicate set has an external account number.
• You will still need to apply a Winner Rule to pick the desired record based on the criteria, but there will be fewer matches to evaluate, resulting in a quicker application of the Winner Rule.

• Matching on multiple fields acts as an AND operation. For example, if you define three field names, a record has to match on all three fields to be considered a duplicate.

To move to step 3, select Manage Merge.

• As Dedupe prepares your results, you can see how many records are being processed. If the results set is too large, you can cancel, then go back and narrow your search further.
• Moving to step 3 does not commit changes.

Step 3: Merge Management

In this step, you can review your duplicates, designated by brackets grouping records together, and select a winner for each bracket.



For small sets of duplicates, you can choose a winner manually. Select the trophy icon under the Winner column for the record you want to keep.

For larger sets, automate the process with Winner Rules. Set your first rule, such as Most Recently Modified Record, and select Apply Winner Rules. Dedupe will use that rule to pick the winner record from each deduplicate group.

• Selecting the Add another Rule option lets you apply additional rules sequentially to catch more records.
• Use the Manage Rules menu to create new or edit existing rules.

Setting your Merge Options tells Dedupe how to merge your winning records into your CRM.

• Prefix Records: For non-winning records, add a prefix instead of deleting.
• Task Creation: Track merge activity by creating a summary task or email, or track merges individually by creating a completed task per merge.
• Ownership Changes: Enable assignment rules to change the ownership of accounts, leads, or cases, based on any active assignment or territory during update or insert operations.
• Child Object Selection: Select which child objects will be included in your merge. Learn more about Child Object Selection.

Select the Use field rules checkbox to override specific fields from your Winner record with field values from the other records inside each duplicate group that you want to merge into the winning record.

• For example, you can choose to update a specific field such as Name with the newest record, even if all other fields are updated with Most Recently Modified according to your Winner Rule.

The Export Grid Data option lets you download your results before (or instead of) merging records into your CRM.

To finish, select Merge selected records for [x] duplicate groups. This action will commit changes to your CRM.

In select circumstances, you can unmerge recent Dedupe scenarios from the Run History dashboard.

• Learn more about Run History.

If you want to re-run or schedule this scenario again in the future, open the Scenarios bar slider and select Save As, then name your scenario and select Save.