DupeBlocker Scenario Keys

An internal table of duplicate keys needs to be maintained for all Scenarios to function properly.

• When DupeBlocker is initially installed, keys will need to be built for all deployed scenarios
  • This includes the default scenarios that ship with DupeBlocker and any new scenarios created from scratch
• When a new record is inserted/updated in Salesforce, keys are automatically added/updated, including batch inserts/updates
• Keys will need to be rebuilt occasionally to be kept up-to-date
• Keys require Salesforce storage
  • The amount of storage needed will depend on the size of the source data and the number of scenarios created
  • More information on calculating/checking the amount of storage for the Duplicate Keys can be found in HERE.
• Undeploying or deleting a scenario DOES NOT delete the keys for that scenario
  • If keys have been built for a deployed scenario and the scenario is later deleted or undeployed, the old keys will continue to use storage
  • To delete "orphaned keys" select Rebuild All Keys from within a scenario

Since keys are built/updated every time a record is inserted/updated, changes to matching (rules, object filters, account cleaning list changes etc.) can be tested WITHOUT rebuilding all keys by simply updating the existing record that should be matched, and then inserting a new "matching" record.  Once all changes have been tested, THEN all the keys can be rebuilt for all existing records.

Selecting Desired Key Builder:  Original vs. Batch Apex

To choose a particular option access the DupeBlocker Settings -> General Settings tab

Note:  Organizations using Salesforce Professional Edition should use Original ONLY as Batch Apex will NOT work.


 
1.  Key Builder Type:  Select the desired method for building keys

• Original:  Uses API calls to build keys but runs faster than batch
  • Uses approximately 1 API call per 1800 records
  • Need to keep the key building window OPEN until the build is complete
    • Closing the window cancels the key building process
• Batch Apex:  Does NOT use API calls, but runs slower than original
  • Key building window CAN BE CLOSED and the job will run in the background
  • An email will be sent to the user that initiated the rebuild when the key build is complete
  • Batch key builder progress can be monitored from Setup -> Administration Setup -> Monitoring -> Apex Jobs

Note:  The difference in speed between the 2 key building options will vary.  For batch, the speed will be affected by API server load and how Salesforce prioritizes batches.  For original, the speed will be affected by the chosen browser and if keys are being built for a single scenario or multiple scenarios.  Original tends to run faster than batch. 

2.  Key Builder Batch Size:  Defaults to 500 for Original and 200 for Batch

• Default batch sizes were chosen to increase key building performance and avoid possible errors
• It may be necessary to lower the batch size if governor limit warnings are received while building keys

Rebuild Needed

 A check mark is displayed in the scenario detail indicating when a rebuild is needed.  The date of the last rebuild is also displayed.



Actions That Require A Rebuild

• Scenario Changes
  • Scenario is initially deployed/redeployed
  • Scenario Rules (matching criteria) are updated, deleted or added
  • A Scenario "object" Filter is added or updated (not required when adding a "current user" filter)
• General Changes
  • Anytime changes are made to Clean Account Name Settings, FirstName, State or Country Mappings in DupeBlocker Settings
    • ONLY scenarios that included the related mapping type in scenario rules will be flagged as needing an update
  • Re-enabling DupeBlocker
    • More information on disabling/enabling DupeBlocker can be found HERE
    • Upgrading to a new version
    • Importing Settings

Building Keys

Keys can be built for all scenarios at one time, or on a scenario by scenario basis.

Note:  Scenarios need to be marked as Deployed in order to build keys.

Building Keys from a DupeBlocker Scenarios View
                                                              

• To rebuild all scenario keys, click on the DupeBlocker Scenarios tab, select the View for * All * scenarios, check all scenarios, and click Rebuild Scenario Keys
• To rebuild keys for an individual scenario or selected group of scenarios, simply check the box next to the specific scenario(s)

Building Keys From the Scenario Details Page

• Rebuild Scenario Keys:  Builds/Rebuilds keys for this particular Scenario
• Rebuild All Keys:  Builds/Rebuilds keys for ALL deployed Scenarios
  • If this option is selected, ALL existing keys will be deleted and new keys created ONLY for all currently deployed DupeBlocker Scenarios.

If the storage usage for keys seems to be high based on the calculations provided here , then it is recommended to Rebuild All Keys.

Using Original Key Builder

Status can be monitored real-time during the build


 
There are 3 parts to the key building process:

1.    Deleting existing (old) keys
2.    Caching object data
3.    Building new keys

If keys are being built for multiple objects at once (i.e. checked multiple scenarios in the scenario view or Rebuild All Keys selected), the process will be repeated for each object type.

As per the message when the key building process is initiated, DO NOT LEAVE THIS PAGE UNTIL KEY BUILDING HAS COMPLETED OR THE PROCESS WILL BE INTERRUPTED.  This tab should not be closed during the rebuild.  Opening a new tab to login to the same Salesforce instance will also terminate the build.  Login on another browser or use the "Batch Apex" key builder option below to avoid this issue.

• Do not let the PC go into sleep or hibernate mode while building keys
• Only one key building process can be initiated at once
• The key builder will display an error if another instance of the key builder is currently building keys

Using the Batch Apex Key Builder

A message appears alerting the user that keys are being built

 

• The window CAN BE CLOSED
• Email sent to the user that initiated the rebuild when complete
• Batch key builder progress can be monitored in 2 ways:
  • 1.  From Setup -> Administration Setup -> Monitoring -> Apex Jobs
  • 2.  From the DupeBlocker User Interface - Status pop-up will appear when first initiated or when clicking on Status from the red notification message shown when accessing scenarios when a Batch Apex build is in process

 

How Long Will it Take?

The amount of time needed to build keys depends on the number of deployed scenarios, the amount of objects in each table, and chosen key building method "Original" or "Apex Batch".  If keys need to be rebuilt for multiple scenarios it is faster to build all keys at once vs. one scenario at a time.  More information on choosing the desired key building method can be found HERE.

Times will vary based on the actual internet speed, Salesforce API traffic, and the internet browser used.  With the Original builder using Mozilla, Chrome, or Safari completes key builds about 10% faster than Internet Explorer.

• Keys are built approximately at the rate of 12-15k/minute using the Original method.  Batch Apex can be up to 50% slower than Original but this can vary greatly based on API factors.

Example 1:
350,000 Accounts
3 deployed Account scenarios
1,050,000 total keys
Original:  70-88 minutes (approx 1.25-1.5 hours)
Batch Apex:  140-176 minutes (approx 2.5-3 hrs)

Example 2:
500,000 Contacts
1,000,000 Leads
2 deployed Lead to Contact Scenarios
3,000,000 total keys (1,500,000 * 2)
Original:  200-250 minutes (approx 3.5 - 4hrs)  
Batch Apex: 400-500 minutes (approx 7-8hrs)

Times noted above are ESTIMATES only!