Migrating algorithms

Overview

As Delphix continues to make continuous improvement to the algorithms included with the Masking Engine, some algorithm frameworks will have multiple versions available simultaneously. New API paths have been added to allow migration of existing algorithm instances from old frameworks to new ones. The migration mechanism creates a new algorithm with the same configuration as the existing algorithm, allowing the behavior and performance of the migrated algorithm to be evaluated before adoption of the new algorithm for production use.

In this release, the following algorithm migrations are available:

FROM: algorithmType=MAPPING TO: algorithmType=COMPONENT, pluginName=dlpx-core, frameworkName=Mapping

The algorithm usage APIs can be used to conveniently transition usage from the old to the new algorithm instance created by the migration mechanism.

Listing available migrations

The following API endpoint returns a list of result objects describing each possible migration. One object is returned for every algorithm on the engine that can be migrated:

CODE

algorithm   GET algorithm/migration

Each object in the response contains the name of the algorithm that can be migrated, as well as the frameworkId of the framework that the migrated algorithm would use.

Migrating algorithms to new frameworks

The following API endpoint creates a new algorithm named newAlgorithmName (from the API query parameters), by migrating from the algorithm named in the query path:

CODE

algorithm   POST /algorithms/{algorithmName}/migration

This endpoint requires the following option in the query:

newAlgorithmName (required, no default)

String The name of the new algorithm to be created by the migration.

This response from the API is an AsyncTask object that can be used to check the status and result of the migration.

Migration of algorithms with a large amount of state (ex. a mapping algorithm with many mappings) can take several minutes or longer to complete. The engine's info.log will contain log messages indicating that the migration operation is making progress. Mapping algorithm migration is estimated to take approximately 3 minutes per 1,000,000 mapping values associated with the source algorithm.

Also, only one algorithm migration can be ran at one time.

Examples

Listing available migrations:

REQUEST

CODE

curl -X GET --header 'Accept: application/json' --header 'Authorization: 3d2d6f53-4b1a-42b5-b4c0-33ec3d66082f'
'http://masking-engine.example.com/masking/api/v5.1.10/algorithms/migration'

RESPONSE

CODE

{
  "availableMigrations": [
    {
      "algorithmName": "alg_J24QXMN3",
      "frameworkId": 13
    }
  ]
}

Migrating a mapping algorithm from the legacy framework to the new framework:

REQUEST

CODE

curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json'
--header 'Authorization: 3d2d6f53-4b1a-42b5-b4c0-33ec3d66082f'
'http://masking-engine.example.com/masking/api/v5.1.10/algorithms/alg_J24QXMN3/migration?newAlgorithmName=new_J24QXMN3'

RESPONSE

CODE

{
  "asyncTaskId": 29,
  "operation": "ALGORITHM_MIGRATE",
  "reference": "alg_J24QXMN3",
  "status": "WAITING",
  "cancellable": false
}