Mapping (Algorithm frameworks)

A Mapping algorithm allows you to state what fictitious values will replace original data. It maps original data values to masked values pre-populated in a lookup table through the Continuous Compliance Engine user interface. There will be no collisions in the masked data, because it always matches the same input to the same output. For example, David will always become Raj and Melissa will always become Jasmine. The algorithm checks whether an input has already been mapped; if so, the algorithm changes the data to its designated output.

The Mapping algorithm can be used on any set of values, of any length, but you must know how many values are being masked. To that end, provide at least the same amount of values as there are unique values being masking; though, more is acceptable. For example, if there are 10,000 unique values in the column being masked, you must give the Mapping algorithm at least 10,000 values.

The Mapping algorithm can be configured for mappings managed locally on the Continuous Compliance Engine or remotely on a user-managed PostgreSQL database. Remote mapping should be used for those who want to manage the storage allocated for mappings or share the same mappings from this algorithm across multiple Continuous Compliance Engines. More information about remote mapping can be found in the Remote mapping page.

Continuous Compliance Engine 6.0.9.0 and earlier: When you use a Mapping algorithm, you cannot mask more than one table at a time. You must mask tables serially.

Continuous Compliance Engine 6.0.10.0 and later: A single Mapping algorithm can have multiple jobs running concurrently.

Tokenization/Reidentification

Mapping algorithms can be used with Tokenization and Reidentification jobs. However, if ignoreCharacters are configured for the algorithm, Tokenization/Reidentification cannot be used.

Sync

Mapping algorithms can be synced in one of two ways:

Syncing a locally managed Mapping algorithm: This can be done to effectively make a copy of an algorithm from one Continuous Compliance Engine to another. In addition to syncing the algorithm, the mappings must be manually exported from the source engine and imported into the target engine. Once this is complete, the two algorithms (on the source and target) will have the same names and initial set of mappings (at the time of sync), but will function as two separate algorithms. That is to say, adding new mappings on the source will not have any impact on the algorithm on the target.
Syncing a remotely managed Mapping algorithm: This can be done to share the same Mapping algorithm across Continuous Compliance Engines. In this case, once synced, the algorithm on the source and target(s) would point to the SAME remote mapping database. This would mean that adding/removing/manipulating the mappings would affect the algorithm on all engines that use it.

More information on Sync can be found in the Introduction (Managing multiple engines for masking) page.

Creating a mapping algorithm via the UI

In the upper right-hand corner of the Algorithm tab, click Add Algorithm.
Select Mapping.
The Create Mapping Algorithm window will appear.
Enter an Algorithm Name.
1. This MUST be unique.
Enter a Description.
Select whether or not the mappings will live locally or remotely by toggling the Local Mapping Store checkbox appropriately. If using a local mapping store, proceed to step 9.
1. For more information about remote mapping stores, visit the Remote mapping page.
Specify Host/IP, Port, Mapping Database, and Schema of the remote database.
Enter any remaining connection parameters in a properties file specified by the Mapping Connection Properties field.
To ignore specific characters, enter one or more characters in the Ignore Character List box. Separate values with a comma.
To ignore the comma character (,), select the Ignore comma (,) checkbox.
When you are finished, click Save.

Before you can use the algorithm by specifying it in a profiling job, you must add it to a domain. If you are not using the Continuous Compliance Engine Profiler to create your inventory, you do not need to associate the algorithm with a domain.

For information on creating Mapping algorithms through the API, visit the Mapping (API client) page.

Managing mappings via UI

Regardless of where the mappings reside (local or remote), the management process is the same. Use the UI to perform options like import/export, delete, or reset mappings.

These tasks can only be performed by a user with sufficient privileges per each task, as follows:

Export mappings
- admin privileges required.
- A passphrase is required, meaning exports will be encrypted.
- Due to the encryption, it will not be possible to see the allocated mappings.
Import mappings
- algorithm:update privileges required.
Delete mappings
- algorithm:update privileges required.
Reset mappings
- algorithm:update privileges required.

Navigate to the Edit Mapping Algorithm page and select Manage Mappings. At the top there are two statistics provided for the mappings:

Total Mappings is the number of mapping outputs that exist for this algorithm.
Available Mappings is the number of mappings that have not yet been assigned to an input value.

When a job using the Mapping algorithm runs, the mappings are loaded into memory. This means that enough memory must be provided to the job to load the mappings. A Mapping algorithm with 2GB worth of mappings will require a job with a larger configured XMX than what is needed for a Mapping algorithm with 2MB worth of mappings.

In addition to mapping statistics, the import/export, delete, or reset mappings actions can be performed.

Delete mappings

This action will delete all input/output combinations and effectively start this algorithm fresh. For this option to take effect, select the Delete Mappings action, then click Delete.

Export mappings

This action will export all mappings into a file that can then be used to seed another mapping algorithm or exist as a backup list of established mappings. For security purposes, a passphrase is required to encrypt the file on export.

To export mappings, select the Export Mappings action and provide a passphrase, then click Export.

Once the export file has been generated, a Click here to Download File link will appear, which will download the file to the designated downloads folder.

If you wish to decrypt the exported file from the command line, run the following command:

openssl enc -aes-128-cbc -a -d -pass stdin -pbkdf2 -iter 100000 -md SHA256 -in PATH_TO_EXPORT_FILE

Import mappings

This action will add mappings to the mapping algorithm. Mappings can be provided in two different formats; PLAINTEXT and CSV.

PLAINTEXT

A PLAINTEXT mapping file can ONLY provide mapping outputs (i.e.: values you want to mask to). The file must have NO header. Make sure there are no spaces or returns at the end of the last line in the file.

The following is a sample PLAINTEXT mapping file. Notice that there is no header and only a list of values.

CODE

Smallville
Clarkville
Farmville
Townville
Cityname
Citytown
Towneaster

CSV

A CSV mapping file can provide both mapping inputs and outputs. That is, you can determine beforehand what you want your mappings to be. The CSV file must have only two columns – input and output. The first line of the file must be the header input,output. Make sure there are no spaces or returns at the end of the last line in the file.

The following is a sample CSV mapping file.

CODE

input,output
New York,Smallville
Boston,Clarkville
San Francisco,Townville
"",Cityname
"",Citytown
"",Towneast

An input value does not have to be specified, but an output value must be specified for a line to be considered valid. Invalid lines are silently ignored.

Once a File Type is selected, choose the mapping file in the Import Mappings/Outputs field.

If providing a previously exported mapping file that has been encrypted with a passphrase, select the CSV file type, provide the unaltered encrypted file, and provide a passphrase.

When the appropriate selections have been made, click Import.

Any duplicate values provided will be silently ignored.

Reset mappings

This action will delete all inputs for provided mappings, giving you a mapping algorithm with as many outputs as you had before, but with all of them available for assignment the next time the mapping algorithm is used.