Dedupe enables you to identify and merge large quantities of duplicate records in both standard and custom CRM objects.
- Choose from a variety of pre-built deduplication scenarios or create your own
- Use a broad array of comparison types to more easily identify duplicate records
- Determine master records based on your company's logic or use pre-defined winner rules
- Control field values on winning records using robust field rule functionality
Dedupe is available as a 14-day trial to all subscribers. After 14 days, you can become a DemandTools Elements Edition customer and unlock full access to Dedupe, Verify, Schedule, and more.
Starting a Dedupe Scenario
To begin a new scenario, select Dedupe from the left navigation bar.
To start from an existing scenario, select a Dedupe scenario from the Scenarios table on your Dashboard. Your source, object, and record selection criteria will already be set for you. You can keep all settings or change them as you go.
Step 1: Select Records
Under the Object to Dedupe dropdown menu, select the object you want to find duplicates of. (You'll define what you consider to be a duplicate is in Step 2.) For example, select Account.
- For large jobs, check the box beside Bulk API. This option groups your records into larger batches, so it uses fewer API calls. Learn more about the Bulk API option.
In the Record Selection Criteria section, choose Add Selection Condition to narrow down the accounts included in your search. Learn more about Record Selection Criteria.
- In this example, set the Billing City to start with P.
Fields to Show on Preview is an optional menu that lets you select the fields shown on your results page in step 3. If you don't use this menu, your results will include a subset of fields related to the dedupe object you selected, record criteria, and match conditions.
To move to step 2, choose Select Matching Fields.
Step 2: Match Records
In this step, you define what you consider to be duplicate records. Under the Matching Account Field dropdown, select the field you want to match on to find duplicates.
Select your Comparison Type next, whether exact or more relaxed. A few examples include:
- Original Value: Use the original string value to compare.
- Cleaned Account Name: Use built-in cleanse rules to match similar names based on common punctuation, abbreviations, and common business prefixes. For example, The ABC Company and ABC Co. would be seen as matching account names. These rules are customizable to meet your organization's specific needs. To customize go to Settings - Manage Cleanse Rules.
- Acronym: Use the first letter from each word in the field value.
- First X Letters: Use only part of the field value by specifying the number of characters to use.
- First X Words: Use only part of the field value by specifying the number of words to use.
- Domain: Strip out all components of the field value except the domain portion.
- Address Cleanse: Normalize the address to ensure better matching between similar address field values.
A comprehensive list of comparison types including use cases and examples can be found here.
Four checkboxes let you adjust the logic for your search.
- Blank Values: Look for positive matches even when field values are blank for a given field. Please note that at least one field must have a non-blank match for a positive match to occur.
- Fuzzy Matching: Analyze words based on their phonetic value, removing vowels and analyzing the remaining consonants.
- Transposed Values: Analyze field values by reordering multiple words within each field value.
- Alpha Clean: Analyze field values by using only characters that are one of the 26 characters of the English alphabet, numbers 0-9, space, and ampersand.
While Record Selection Criteria narrows down which records get pulled to begin with, Filter Match Results applies filtering after the record pull is complete, allowing you to narrow down the results of your search based on the supplied criteria.
- When you select this option and add one or more selection conditions, the results grid will only return duplicate groups where at least one of the records matches the criteria.
- For example, you can filter to return all duplicates of an account where at least one record in the duplicate set has an external account number.
- You will still need to apply a Winner Rule to pick the desired record based on the criteria, but there will be fewer matches to evaluate, resulting in a quicker application of the Winner Rule.
You can continue to narrow down and refine your deduplicate matches by adding more fields. To add more fields, select Add Field to Compare.
- Matching on multiple fields acts as an AND operation. For example, if you define three field names, a record has to match on all three fields to be considered a duplicate.
To move to step 3, select Manage Merge.
- As Dedupe prepares your results, you can see how many records are being processed. If the results set is too large, you can cancel, then go back and narrow your search further.
- Moving to step 3 does not commit changes.
Step 3: Merge Management
In this step, you can review your duplicates, designated by brackets grouping records together, and select a winner for each bracket.
For small sets of duplicates, you can choose a winner manually. Select the trophy icon under the Winner column for the record you want to keep.
For larger sets, automate the process with Winner Rules. Set your first rule, such as Most Recently Modified Record, and select Apply Winner Rules. Dedupe will use that rule to pick the winner record from each deduplicate group.
- Selecting the Add another Rule option lets you apply additional rules sequentially to catch more records.
- Use the Manage Rules menu to create new or edit existing rules.
Setting your Merge Options tells Dedupe how to merge your winning records into your CRM.
- Prefix Records: For non-winning records, add a prefix instead of deleting.
- Task Creation: Track merge activity by creating a CRM task or email report, or track merges individually by creating a completed task per merge.
- Ownership Changes: Enable assignment rules to change the ownership of accounts, leads, or cases, based on any active assignment or territory during update or insert operations.
Select the Use field rules checkbox to override specific fields from your Winner record with field values from the other records inside each duplicate group that you want to merge into the winning record.
- For example, you can choose to update a specific field such as Name with the newest record, even if all other fields are updated with Most Recently Modified according to your Winner Rule.
The Export Grid Data option lets you download your results before (or instead of) merging records into your CRM.
To finish, select Merge selected records for [x] duplicate groups. This action will commit changes to your CRM.
Dedupe Errors
If you encounter an error during the dedupe merge process, you will see a banner like this:
To see which records encountered the error and what it is, you can reveal a hidden column called Error Message. To reveal this column:
Go to the Review Duplicates grid and open any hamburger menu beside a column label. We'll open the one beside the Rule Applied label.
With this menu open, select the grid icon to the right of the hamburger icon.
Scroll to the bottom of the column list and find Error Message. Check the box beside this column label.
A new column will be added to the far right side of the Review Duplicates grid. Here you can view the error messages sent to DemandTools by the target CRM. Only the rows that encountered an error will show an error message.
Because the target CRM provides the error messages, DemandTools does not maintain a comprehensive list of possible error. Contact your CRM if you have questions about an error you receive.
Saving Your Scenario
If you want to re-run or schedule this scenario again in the future, open the Scenarios bar slider and select Save As, then name your scenario and select Save.
