Navigation:  Using CGX - Step-By-Step > Dupe Record Options

Duplicate Record Options & Criteria

When exporting records (or updating an output dataset), there are essentially 4 recordset categories available in terms of which records from a contact folder (or GAL/GDG) will be processed.

 

 

 

(1) PROCESS OPTION

 

PROCESS ALL

 

Default option and indicates that all records found in the selected input sources will be processed according to all other selection options regardless of whether or not these are duplicates. Key Fields and Sort Sequence do not apply in this case.

 

PROCESS DISTINCT

 

Distinct means that one contact record is completely unique for all the fields selected in the Export Field Map and that no duplicates exist. In short, this is the reverse of duplicates in that only contacts that have no duplicates will be processed.

 

Note, if Creation Date and/or Last Modification Date fields are also selected for output, all records will be considered Distinct since it is virtually impossible to have 2 contacts with the exact same Creation and/or Modification Date/Time stamps which include seconds. This is also true if fields such as <Entry ID>, <Size> etc  are selected.

 

In general, only fields that are relevant and required should be selected for export if the Process Distinct option is selected. Selecting <All Standard> fields combined with <Process Distinct> serves no purpose.

 

Duplicate Key Fields and Sort Sequence do not apply for this option since everything is determined from the actual fields to be exported.

 

PROCESS UNIQUE ONLY (FIRST IN SORT SEQ)

 

This indicates that only the first record found in a duplicate set will be processed (either output to file or used to update a target dataset depending on options). This option requires one or more Duplicate Key Fields to be specified and a sort option to be selected.

 

EXPORT DUPES ONLY

 

Indicates that only duplicate contacts will be exported to the selected output file. This option is not available when the Export Processing option is set to <UPDATE> a target dataset.

 

(2) DUPLICATE KEY FIELDS

 

Duplicate key fields are required only when using the "Process Unique" option otherwise you cannot enter any Duplicate Key fields. Any field found in the selected contact record/message class combination are available for selection and do not need to be included in the Export Field Map.

 

Contents of all fields specified must be found to be equal to be considered a duplicate.

 

(3) SORT SEQUENCE

 

There are 4 available Sort Sequence options which are::

 

 

1) Last Modification Date - Descending Order (newest to oldest)

2) Last Modification Date - Ascending Order (oldest to newest)

3) Creation Date - Descending Order (newest to oldest)

4) Creation Date - ascending Order (oldest to newest)

 

The default selection is LastMod <Dsc>

 

 

DUPLICATES & MULTIPLE FOLDER PROCESSING

 

Processing of all records is done on an <Aggregate> basis which means that what duplicate processing options are selected, these will be applied to the records contained in all selected contact folders.

 

If you want the processing options to apply to each individual contact folder selected, then you only need to assign a unique OwnerID to each contact folder and include the OwnerID field as a selected export field (for Process Distinct option) or as a duplicate key field (for Process Unique Field).