Editing Extract Properties in MariaDB Data Point
To change the configuration for data extraction, click Extract Properties tab.
In the canvas, the Extract Type drop-down displays the default extraction type JDBC.
- The extract properties can be set one time in the Admin module based on the company standards. Refer the page Editing Extract Properties for more details.
- The changes made to the properties in the data point automatically reflect to all the objects connected to this data point.
- Any extract properties that have been overridden in the dependent objects will not be effected by this change.
- During the execution of the job, the load command properties are defined automatically based on the values provided for the extract properties.
Based on the requirements, you can do the following:
- Change the default Property values, then a button appears beside the changed value and to revert click Reset to Default button.
- To rest all the values and set to accommodate general industry standards and data formats, click Reset All to Default link.
Extraction type: JDBC
| Property | Description | Default value | Other possible values |
|---|---|---|---|
| Extraction Mode | The extraction mode can either be File or Pipe. Pipe option is used to stream the data directly from Source to Target. When File option is used then the data extracted from source is staged on the agent server before loading into the target. | File | Pipe |
| File Type | Defines the format of extracted data. (Only delimited text format is supported) | Delimiter | |
| LF in String | Specifies if the source data has new line characters or not, so that these new line characters can be identified separately during extraction and load from the row delimiter new line character. | ON | OFF |
| Column Delimiter | Specifies the delimiter on the extracted file data. | ASCII 01 (SOH) | Any ASCII character |
| Compression | Use this option to specify if the data needs to be compressed before transfer from source agent to Target load agent. | NO | YES |
| Compression Type (Displays only when Compression is "Yes") | On choosing a compression format, the user is required to give the corresponding compression codec and compression type (Only default Compression Type is supported). | gzip | |
| Encryption | Use this option to specify if the data needs to be encrypted before transfer from source agent to Target load agent. This option is also used when the target is file system and the generated file is required to be encrypted.This encryption is not dependent on source and target systems using same or different agent. For detailed information refer Setting up encryption keys at Agents. | NO | YES |
| Encryption Type (Displays only when Encryption is "Yes") | When encryption is enabled then, user needs to specify the encryption algorithm to be used. AES can be used for symmetric key encryption. Choose from 128, 192 and 256 bit key sizes. | AES-128 | AES-192 AES-256 |
| Cipher Operation Mode (Displays only when Encryption is "Yes" and Encryption Type is "AES") | Specify the mode of operation, you can either use CBC (default mode) or GCM. In CBC mode, each block of plaintext is dependent on previous ciphertext block before being encrypted i.e block processing and where as in GCM its parallel processing and independent. | CBC | GCM |
| Encrypted AES SecreteKey (Displays only when Encryption type is "AES") | Specify if the AES key is generated by using secret key or not. | NO | YES |
| Row Delimiter | Specifies the character to be used to indicate the end of the row in the extracted data. | \\n (New Line Character) | Any ASCII character |
| Null Value | Specifies what string literal should indicate the null value in the extracted data. During the data load the column value matching this string will be loaded as null in the target. | NULL | Any string literal |
| Text Qualifier | Specify if the text columns in the source data needs to be enclosed in quotes. | Empty | Single Double |
| Escape Character | The character immediately following the escape character is escaped. This needs to be specified if the text qualifier is provided and the text qualifier character can appear in the source data. | Empty | \\ (Recommended)Any ASCII character |
| Date Style | Specifies how to interpret the date format | YMD | MDY,DMY,DMONY,MONDY,Y2MD, MDY2, DMY2, DMONY2 and MONDY2 |
| Date Delimiter | Specifies the separator used in the date format. (Only default delimiter is supported) | - | |
| Time Style | Specifies the format of the time portion in the data. | 24 HOUR | 12HOUR |
| Time Delimiter | Specifies the character used as separate the time components. (Only default delimiter is supported) | : | |
| Fetch Size | Specify the number of rows to be fetched at a time and processed in memory. If the value specified is zero then, all rows are extracted at once. | 1000 | 10 100 100000 Any numeric value |
| Include Header | Specifies if the column list needs to be included as header when extracting data. Generally enabled only when the target is flat file. | OFF | ON |
| Decimal Delimiter | Specify the decimal delimiter for float/double/numeric data types. | EMPTY | . , |
| Runtime Properties | MariaDB system variables that can be set according to the requirement. | EMPTY | Ex: set sql_select_limit=1; For more properties refer, |