Making an algorithm configurable

To integrate an algorithm as a configurable framework within the Continuous Compliance Engine, specific requirements must be met:

Method requirement: The algorithm class should have the getAllowFurtherInstances method implemented to return true.
Data member annotation: The algorithm class must include one or more public data members. These members are crucial for configuration and must be annotated with @JsonProperty from the com.fasterxml.jackson.annotation.JsonProperty package. This annotation enables the data members to be recognized and processed as configurable parameters.

Additional discretionary notes

JSON handling and schema consistency: Most JSON processing tasks are handled by the Masking Plugin API instead of the plugins directly. This ensures uniformity in JSON document and schema interpretation. For each algorithm marked as configurable, the SDK or Continuous Compliance Engine inspects the class annotations to identify which parameters can be configured.
Instance creation and JSON application: When creating a new instance of the algorithm, the system tries to apply the user-provided JSON configuration to the algorithm class object. This process includes a level of validation, ensuring that the provided JSON aligns with the expected schema as inferred from the annotated fields. Note that there are certain limitations in this validation, as detailed in the corresponding section below.

Example configurable algorithm explained

The concept of configurability can be illustrated using one of the sample algorithms from the SDK as an example, StringRedaction.java in this case:

CODE

package sample.masking.algorithm.redaction;
...

public class StringRedaction implements MaskingAlgorithm<String> {
    private String name = "StringRedaction";

    @JsonProperty(value = "redactionCharacter", required = true)
    public String redactionCharacter = "specified";

    @Override
    public String getName() {
        return name;
    }

    @Override
    public Collection<MaskingComponent> getDefaultInstances() {
        StringRedaction instanceX = new StringRedaction();
        instanceX.name = "Redaction X";
        instanceX.redactionCharacter = "X";
        return Arrays.asList(instanceX);
    }

    @Override
    public boolean getAllowFurtherInstances() {
        return true;
    }

    @Override
    public String getDescription() {
        return String.format(
                "Redact String by overwriting with '%s' character", redactionCharacter);
    }

    @Override
    public String mask(@Nullable String input) throws MaskingException {
        if (input == null) {
            return null;
        }
        StringBuilder returnVal = new StringBuilder();

        for (int i = 0; i < input.length(); i++) {
            returnVal.append(redactionCharacter);
        }
        return returnVal.toString();
    }

    @Override
    public void validate() throws ComponentConfigurationException {
        if (redactionCharacter == null || redactionCharacter.length() != 1) {
            throw new ComponentConfigurationException(
                    "redactionCharacter must be a single character");
        }
    }
}

This algorithm is designed for straightforward redaction of input strings, with a unique feature allowing the redaction character to be customized. Here's how it functions:

Configurable parameters: The class includes a public field named redactionCharacter, marked with the @JsonProperty annotation. This field can be configured with different values, and a default value is set to ensure that the getDescription method provides an appropriate description in both the framework and individual instance contexts.
Default instance specification: The getDefaultInstances method in the class defines a default instance of this algorithm. This is achieved by returning a list of objects configured with 'X' as their redaction character. The API framework then converts these object configurations into JSON, which it stores for future use when an instance of the “Redaction X” algorithm is requested.
Enabling additional instances: Setting the getAllowFurtherInstances method to return true allows the creation of additional instances of this algorithm. This capability is particularly useful once the plugin is integrated into the Continuous Compliance Engine using the Masking API through an API client.
Configuration validation: The class implements a validate method to ensure that the configuration values provided are appropriate. Specifically, for the redactionCharacter field, the validation restricts the string length to a single character, ensuring the redaction functionality works as intended.

Frameworks, instances, and configuration injection

When used as a framework, the algorithm class is instantiated and used without any configuration injection. In the example above, that means that the getDescription method will return, "redact String by overwriting with 'specified' character" when the algorithm framework is evaluated. Similarly, getName will return "StringRedaction", the name of the framework.

When a runnable algorithm instance is needed, the algorithm class is instantiated, and all saved configuration is injected before any methods are called. This configuration is gathered in one of two ways:

For statically provided instances embedded in the plugin, the configurable fields of each object returned by the getDefaultInstances method are serialized to JSON and saved. Again, only the values of public fields marked with the @JsonProperty annotation are extracted this way.
When the user creates a new algorithm instance using the Masking Web API, the contents of the algorithmExtension field of the POST or PUT request is validated and saved for future injection whenever that particular algorithm instance is needed in the future.

In the above example, when algorithm instance "Redaction X" is created, the saved values will be injected, so redactionCharacter will have the value 'X'.

Validation of configuration values

The major JSON handling libraries, for performance reasons, conduct minimal validation during object deserialization. This means that many @JsonProperty annotation aspects are not strictly enforced. For instance, even if a property is flagged as required, an object can still be deserialized successfully without that property in the input JSON.

While there are libraries to enhance JSON validation within the framework, implementing them complicates the distinction between validations handled by the API framework and those required in the component's validate method. Consequently, the framework currently performs only essential input validation. It falls upon plugin authors to thoroughly validate all aspects of an object's configuration in their validate method implementation, particularly on the presence of required fields (i.e. non-empty and non-null values).

Despite this enforcement lapse of @JsonProperty annotation properties, it is important not to disregard them. These properties are included in the auto-generated schema for each framework, accessible through the SDK's maskApp and the algorithm/framework endpoint in the masking API client. Their visibility in these schemas might be helpful for future UI development.

Default interface implementations

The masking API defines default implementations of getDefaultInstances and getAllowFurtherInstances as follows:

CODE

default Collection<ComponentInstanceDescription> getDefaultInstances() {
        return Collections.singletonList(this));
    }

default boolean getAllowFurtherInstances() {
        return getDefaultInstances() == null || getDefaultInstances().isEmpty();
    }

This means that if neither of these methods is overridden by the masking algorithm class, a single instance capturing whatever default values exist for configurable fields is created by default.

Only algorithm classes that define getAllowFurtherInstances to return true appear as Algorithm Frameworks on the Continuous Compliance Engine.

Build dependencies for configurable algorithms

When the maskScript init sub-command is used to create a new project, the initial build files may not include the dependencies required for the Jackson @JsonProperty annotation. This can be corrected by adding this line to proj_root/gradle.properties:

CODE

jacksonVer=2.15.2

The Jackson version is 2.15.2 at the time of authoring, though, be sure to use the latest stable version to remain up-to-date with security fixes in the library.

And this line to the dependencies* section at the end of proj_root***/build.gradle:

CODE

compileOnly ('com.fasterxml.jackson.core:jackson-annotations:' + jacksonVer)

The set of Jackson annotations tested and supported for use in algorithm plugin classes are:

@JsonProperty
@JsonPropertyDescription
@JsonFormat (Useful in specifying formats for Date fields)