To change the runtime properties, click Runtime Properties tab.

In the Canvas from the Run Property Type drop-down list, you can select Hive Properties, Hive Optimization, Map Reduce Properties, or All. When you change the default Runtime Property Type, a reset icon appears beside the drop-down list, and to revert click the reset icon.

Based on the requirements, you can do the following:

  • Change the default Property values, then a button appears beside the changed value and to revert click Reset to Default button.
  • To rest all the values and set to accommodate general industry standards and data formats, click Reset All to Default link.

Note: To view specific property details, you can enter the keyword in the search bar, and the particular property details are displayed. 

Run Property type: Hive Properties

PropertyDescriptionDefault valueOther possible values
Field DelimiterSpecifies delimiter for structures fields, the default is '\001', which is the CTRL-A.1

'

~

External TableTo create target table as external tableNoYes
Managed Table WareHouse Location (Displays only when External Table is "No")To manage location when target is being created as internal table

External Table WareHouse Location (Displays only when External Table is "Yes")To manage location when target is being created as external table/tmp
Escape CharacterThe character immediately following the escape character is escaped.This needs to be specified if the text qualifier is provided and the text qualifier character can appear in the source data.Empty\\ (Recommended)Any ASCII character
Collections Items DelimiterSpecifies a delimiter for array items\002
Line DelimiterSpecifies a delimiter for map keys. You can include a newline character only ('\n'). Newlines must not exist within the data itself, and they cannot be escaped\n
File FormatSpecify hive built in and custom-developed file format of the file.ORC

NONE

PARQUET

SEQUENCEFILE

AVRO

CUSTOM

TEXTFILE

RCFILE

Orc Compression Type (Displays only when File Format is "ORC")Specify the compression method to be applied on the file placed in DFSZLIB

NONE

SNAPPY

Orc Stripe Size (Displays only when File Format is "ORC")Specify the number of bytes in each stripe268435456
Row FormatSpecify the row format of the tableDELIMITEDSERDE
Orc Row Index Stride (Displays only when File Format is "ORC")Specify the number of rows between index entries (must be>=1000)10000
Serde Jar PathDefine the Serde Jar location in the DFS

Table PropertiesTBL PROPERTIES clause allows you to tag the table definition with your own metadata key/value pairs

Orc Create Index (Displays only when File Format is "ORC")Specify whether to create row indexesTRUE
Serde ClassThe fully qualified class name that implements the Hive SerDe interface org.apache.hadoop.hive.serde2.SerDecom.bizo.hive.serde.csv.CSVSerde
Serde PropertiesComma separated list of SerDe property and its value pairs. These go as “WITH SERDEPROPERTIES” in DDL of the table"separatorChar"="\u0001","quoteChar"="'","escapeChar"="\u0004"
Input Format Class (Displays only when File Format is "CUSTOM")

In the file_format to specify the name of a corresponding Input Format

For example, 'org.apache.hadoop.hive.contrib.fileformat.base64.Base64TextInputFormat'.



Output Format Class (Displays only when File Format is "CUSTOM")In the file_format to specify the name of a corresponding output Format

Orc Compression Size (Displays only when File Format is "ORC")Specifies number of bytes in each compression chunk262144
CompressionUse this option to specify if the data needs to be compressed before transfer from source agent to Target load agent.FALSETRUE
Compression Type ( Displays only when Compression is "True")On choosing a compression format, the user is required to give the corresponding compression codec and compression type.NONE

RECORD

BLOCK

Intermediate Compression ( Displays only when Compression is "True")Should the outputs be compressed before being sent across the network. Uses Sequence File compression.FALSETRUE
Intermediate Compression Codec ( Displays only when Compression is "True")If the outputs are compressed, how should they be compressed? (i.e. Snappy).

Orc bloom filter columns (Displays only when File Format is "ORC")Specifies comma separated list of column names for which bloom filter should be created

Orc bloom filter fpp (Displays only when File Format is "ORC")Specifies false positive probability for bloom filter (must>0.0 and <1.0)0.05

Run Property type: Hive Optimization & Map Reduce Properties

  • From the drop-down select Hive Optimization or Map Reduce Properties to modify the properties corresponding to these.
  • To add a new property, click Add icon.

New Runtime properties window appears and allows you to select from the list.

  • Select one or more properties and click Ok.

If the list does not display the required property, then you can add the property in Diyotta Admin. For more details to add these property, refer Editing Runtime Properties.
  • New entry for the property is generated.

  • The "Property" field displays the assigned property.
  • The "Value" field displays the default value assigned to the property and if required you can change the value.
  • You can change the value by clicking on the value under the "Value" field. This could be a drop-down or text box. Select or enter the value to be used for the property during runtime.
  • To delete any added property, select the property from the list and click Delete icon.
Note: From the list of properties, multiple properties can be selected and we can perform/apply these operations.

Propagate Runtime Properties to dependent objects

When a runtime property is added, modified or deleted in the data point, then the changes need to be propagated to the dependent objects so that these are effective during execution. To propagate changes follow below steps.

Step IAfter making changes in the runtime properties, save the data point.

Step IISelect the property that was added or modified and click Propagate icon. If a property was deleted, then without selecting any property, click the Propagate icon.

Step III: The Propagate Runtime Properties window opens and shows the selected properties and the impacted objects.

Step IV: In the wizard, click OK and wait for the propagation to complete.