Continuous Compliance API client
Introduction
The launch of API v5 on the Delphix Continuous Compliance Engine marks a significant advancement, enabling scripting and automation capabilities. This development offers exciting new possibilities for Continuous Compliance users. This page provides a high-level overview of API v5's features and includes some useful links to help you get started.
REST
API v5 is a RESTful API. REST stands for REpresentational State Transfer. A REST API will allow you to access and manipulate a textual representation of objects and resources using a predefined set of operations to accomplish various tasks.
JSON
API v5 uses JSON (JavaScript Object Notation) to ingest and return representations of the various objects used throughout various operations. JSON is a standard format and, as such, has many tools available to help with creating and parsing the request and response payloads, respectively.
Here are some UNIX tools that can be used to parse JSON – Parsing JSON with UNIX tools. That being said, this is only the tip of the iceberg when it comes to JSON parsing and the reader is encouraged to use their method of choice.
API client
The various operations and objects used to interact with API v5 are defined in a specification document. This allows us to utilize various tooling to ingest that specification to generate documentation and an API Client, which can be used to generate cURL commands for all operations. To see how to log into the API client and for some starter recipes, please check out API Cookbook document.
To access the API client on your Continuous Compliance Engine, go to
http://myEngine.myDomain.com/masking/api-client
.To access the API client documentation without an engine, please refer to the static HTML representations here in the expanded list below:
API documentation for version 27.0.0.0 and ABOVE can be found using the links below, where the * version coincides with this version of documentation:
Supported features
API v5 is in active development but does not currently support all features that are accessible in the GUI. The list of supported features will expand over the course of subsequent releases.
For a full list of supported APIs, the best place to look is the API client on your Continuous Compliance Engine.
Application settings APIs
General group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
general | EnableMonitorRowCount | Boolean | Controls whether a job displays the total number of rows that are being masked. Setting this to false reduces the startup time of all jobs. | true |
PasswordTimeSpan | Integer [0, ∞) | The number of hours a user is locked out for before they can attempt to log in again. | 23 | |
PasswordCount | Integer [0, ∞) | The number of incorrect password attempts before a user is locked out. | 3 | |
AllowPasswordResetRequest | Boolean | When true, users can request a password reset link be sent to the email associated with their account. | true | |
PasswordResetLinkDuration | Integer [1, ∞) | Controls how many minutes the password reset link is valid for. | 5 | |
NumSimulJobsAllowed | Integer [0, ∞) | Max number of jobs allowed to run simultaneously. Setting this number to zero will lead to a dynamically calculated limit based on the number of available CPU cores. | 7 | |
DefaultApiVersion | String | Used to set default API Version. | Blank | |
DataRetentionInterval | Integer [-1, ∞) | The length of time that specific historical data is retained. | 60 | |
DataRetentionMaxDirectorySize | Integer [-1, 100] | The percentage of disk space allowed for all logfiles located in specific directories. | 10 | |
LoginScreenMessage | String | To configure a security/branding banner message on the Engine. All users will see the banner message on the Login screen. | ||
UserSessionTimeoutMinutes | Integer [5, 1440] | This is used to configure the session’s idle timeout in minutes. The user will be logged out after the idle timeout. | 60 | |
MaximumMemoryForJobs | Integer [0, ∞) | When set to 0, the maximum memory for jobs is calculated automatically. Adjusting this default is only recommended for Containerization Masking if a limit in pod resources is not defined. | 0 |
NumSimulJobsAllowed setting should be set based on engine configuration. It is risky to run many jobs at once in an environment where you have scheduled more jobs than the system has memory for. When the system runs out of memory all jobs will fail.
Algorithm group settings
Setting Group | Setting Name | Type | Description |
|
---|---|---|---|---|
algorithm | DefaultNonConformantDataHandling | String {DONT_MASK, FAIL} | Default algorithm behavior for Handling of Non-conformant Data patterns. | DONT_MASK |
Database group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
database | DB2zDateFormat | String | Default Date String format to use for DB2 zOS if the database is not using one of the pre-defined IBM DB2 zOS Date String formats. Default is ISO Date String format. | yyyy-MM-dd |
LDAP group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
ldap | Enable | Boolean | Used to enable and disable LDAP authentication | false |
LdapHost | String | Host of LDAP server | 10.10.10.31 | |
LdapPort | Integer [0, ∞) | Port of LDAP server | 389 | |
LdapBasedn | String | Base DN of LDAP server | DC=tbspune,DC=com | |
LdapFilter | String | Filter for LDAP authentication | (&(objectClass=person)(sAMAccountName=?)) | |
MsadDomain | String | MSAD Domain for LDAP authentication | AD | |
LdapTlsEnable | Boolean | Enable and disable the use of TLS for LDAP connections. | false |
In the LDAP group, once the "Enable" setting is set to "true", all users logging in will be authenticated via the LDAP server. Local authentication will no longer work. Before setting this to true set all other LDAP settings correctly and create the necessary LDAP users on the Continuous Compliance Engine.
Mask group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
mask | DatabaseCommitSize | Integer [1, ∞) | Controls how many rows are updated (Batch Update) to the database before the transaction is committed. | 10000 |
DefaultStreams | Integer [1, ∞) | Default number of streams for a masking job. | 1 | |
DefaultUpdateThreads | Integer [1, ∞) | Default number of database update threads for a masking job. | 1 | |
DefaultMaxMemory | Integer [1024, ∞) | Default maximum memory for masking jobs (in megabytes). | 1024 | |
DefaultMinMemory | Integer [1024, ∞) | Default minimum memory for masking jobs (in megabytes). | 1024 | |
CharStreamingBufferLimitRate | Integer [1, 50] | Used for calculating maximum allowed buffer size for Character streaming parsers to buffer data. Only used in JSON file and Document store type masking. | 25 | |
MdsQueryDataTableBatchSize | Integer [1, 5000] | Controls the table query batch size for database masking job and report generation. Increasing the batch size results in faster processing time of the masking job but will use more heap memory. | 100 |
Profile group settings
These settings apply only to the legacy profiler which has reached End of Life. Only the DefaultMultipleProfilerExpressionAlgorithm is still used by the ASDD profiler.
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
profile | EnableDataLevelCount | Boolean | When enabled (true), the Continuous Compliance Engine counts the number of rows in the profiled table. If the number of rows are less than or equal to DataLevelRows, then it uses the number of rows as the sample size. Otherwise, it uses DataLevelRows. | false |
DataLevelRows | Integer [1, ∞) | The number of rows a data level profiling job samples when profiling a column.
| 100 | |
DataLevelPercentage | Double (0, ∞) | Percentage of rows that must match the data level regex to consider this column a match, and thus sensitive. | 80.0 | |
IgnoreDatatype | String | Datatypes that a profiling job should ignore. Columns of these types will not be assigned a domain/algorithm pair. | BIT,BOOLEAN,CHAR#1,VARCHAR#1,VARCHAR2#1,NCHAR#1, | |
DefaultStreams | Integer [1, ∞) | Default number of streams for a profiling job. | 1 | |
DefaultMaxMemory | Integer [1024, ∞) | Default maximum memory for profiling jobs (in megabytes). | 1024 | |
DefaultMinMemory | Integer [1024, ∞) | Default minimum memory for profiling jobs (in megabytes). | 1024 | |
OptimizationLevel | Integer [0, 9) | Optimization level for the profiling job which is defined as below, | -1 | |
DefaultMultipleProfilerExpressionAlgorithm | String | Default Multiple PHI masking algorithm which will be used when the Multiple Profiler Expression will be true for profile job. This value is used by the ASDD profiler. | NullValueLookup |
ASDD group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
ASDD | DefaultMaximumColumnSize | Integer [1, 65536] | The maximum length of the column value to profile. Any value beyond that is truncated to this length for ASDD profiling and the remaining part is ignored. | 1024 |
DefaultTableSampleRows | Integer [1, ∞) | The number of database rows for the ASDD profiler to sample for each table. | 1000 | |
DefaultAssignmentThreshold | Integer [1, 100] | The confidence threshold that must be met or exceeded for the ASDD profiler to make a domain and algorithm assignment. Assignment threshold can otherwise be set on a profile set level; see Configuring profile sets. | 1 | |
DefaultJobExecutionStreams | Integer [1, ∞) | The number of streams to use by default for new ASDD profiler jobs | 1 | |
DefaultNullFilterThreshold | Integer [0, 100] | The percentage of column values that must be null or empty to trigger an additional query to retrieve more column values. | 75 | |
DefaultProfileSetName | String | The name of the default ASDD profile set that should be selected automatically while creating a Profile Job. | ASDD Standard | |
DefaultTableSampleLimitSeconds | Integer [0, ∞) | A subsequent not-null query is triggered if enough data is not fetched for a column during the initial sample query. This setting sets the query timeout for such queries, which can run forever for large tables. | 180 |
Job group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
job | JobLoggingLevel | String {Basic, Detailed} | Controls the amount of information being logged from a job's output. Warning: the Detailed setting may log sensitive information when errors occur. Although this information can be very valuable when debugging a problem, it should be used with care. | Basic |
SMTP group settings
Setting Group | Setting Name | Type | Description | Default Value |
---|---|---|---|---|
smtp | EmailEnabled | Boolean | Used to enable and disable SMTP Email authentication | TRUE |
Host | String | The address of the SMTP server used for sending emails | ||
Password | String | The password required to authenticate with the SMTP server | ||
EmailAddressFrom | String | The email address that will appear in the "From" field of sent emails | ||
EmailSubject | String | The subject line of the email being sent | ||
UserName | String | The username required to authenticate with the SMTP server | ||
Port | Integer | The port number used by the SMTP server, typically 25, 465, or 587 | 25 | |
TlsEnabled | Boolean | Whether to use SSL/TLS encryption for secure email sending | FALSE | |
AuthEnabled | Boolean | Indicates if authentication is needed to connect to the SMTP server | FALSE |