Character Replacement (Algorithm frameworks)

The character replacement framework allows creation of rules to replace specific characters in a string with other characters. The framework must wrap another string algorithm, which defaults to dlpx-core:Alpha Numeric. Character replacement is often used as part of a chained string algorithm, either before or after masking. Other features are removal of accented characters and ensuring that output of the algorithm is changed from the input. At this time there is no UI or default instance for the character replacement framework.

Replacement of characters is specified in a list of JSON rules. There is no enforced limit to the number of rules that can be created.

Creating a Character Replacement algorithm via API:

{
    "stringAlgorithm": {
        "name": "dlpx-core:CM Alpha-Numeric"
    },
    "replacementRules": [
        {
            "inputFilteredCharacters": null,
            "inputReplacementCharacter": null,
            "outputFilteredCharacters": ["1","2","3","4","5","6","7","8","9","0"],
            "outputReplacementCharacter": "*",
            "filteredCharacters": ["a","e","i","o","u"],
            "replacementCharacter": "+"
        }
    ],
    "requireInputChange": false,
    "filterAccents": "NONE"
}

Filter pair: Two JSON keys which together describe the list of characters to filter, and the character they should be replaced with.

1. Define stringAlgorithm - Object - Required

   • Define name - String - Required

     • This must be a valid string algorithm in the Delphix engine. The string algorithm should always be chosen, but it defaults to dlpx-core:CM Alpha Numeric.

2. Define requireInputChange - Boolean - Required

   • Whether to ensure the input to the algorithm is different than the output. Throws an exception if set to true and the input and output match.

3. Define filterAccents - Enum - Required

   • The accents from characters like "é" or "ñ" can be stripped and decomposed to the root character like "e" or "n". This can be done before character replacement and chained algorithm masking, after, neither or both.

     • Accepted values: INPUT, OUTPUT, NONE, BOTH

4. Define replacementRules - List - Required

   • Contains a list ofJSON objects with filter pairs of characters to replace and their replacements. For each rule object you must define a filter pair for at least one of input, output or the unqualified filteredCharacters which apply to both:

     • inputFilteredCharacters - List - Optional (If another filter pair defined)

       • Characters to filter from initial input.

     • inputReplacementCharacter - String - Optional (If another filter pair defined)

       • Single character to replace those from inputFilteredCharacters with.

     • outputFilteredCharacters - List - Optional (If another filter pair defined)

       • Characters to filter from output of chained string algorithm.

     • outputReplacementCharacter - String - Optional (If another filter pair defined)

       • Single character to replace those from outputFilteredCharacters with.

     • filteredCharacters - List - Optional (If another filter pair defined)

       • Characters to filter from input and output.

     • replacementCharacter - String - Optional (If another filter pair defined)

       • Single character to replace those from filteredCharacters with.

Validation

Validation is done to ensure that:

• Each list of filtered characters has a corresponding replacement character (is a complete filter pair)

• Each replacement rule has at least one filter pair

• The replacement character is not present in the filtered characters

• At least one replacement rule is defined

• The stringAlgorithm key refers to a valid chained string algorithm

Examples

Examples of output vary widely because they depend on the chained string algorithm chosen. If we keep the default chained algorithm of dlpx-core:Alpha Numeric, it will never change punctuation marks and will allow us to create examples of punctuation filtering. Consider the following configuration:

{
    "stringAlgorithm": {
        "name": "dlpx-core:CM Alpha-Numeric"
    },
    "replacementRules": [
        {
            "inputFilteredCharacters": null,
            "inputReplacementCharacter": null,
            "outputFilteredCharacters": null,
            "outputReplacementCharacter": null,
            "filteredCharacters": ["@","#","$","%","&","+","?"],
            "replacementCharacter": "."
        }
    ],
    "requireInputChange": false,
    "filterAccents": "BOTH"
}

Potential masked results with the same keyed dlpx-core:Alpha Numeric instance as follows:

• “1+2+3+4+5+6+7+8+9+0" can become "0.0.0.4.8.3.7.7.7.4“

• "1+2&3?4+5&6?7+8&9?0" can become "0.0.0.4.8.3.7.7.7.4"

With filter accents, alphanumeric characters with accents can be masked to unaccented characters:

• "áéíóú" can become "bkxqw"

• "ÂÊÎÔÛ" can become "BKXQW"

In this example configuration altering the rule to filter on input, output or both makes no difference. Changing the filter point could produce different results with an algorithm that could potentially change, add or remove punctuation marks and accented characters.