To change the runtime properties, click Runtime Properties tab.
In the Canvas from the Run Property Type drop-down list, you can select Hive Properties, Hive Optimization, Map Reduce Properties, or All. When you change the default Runtime Property Type, a reset icon appears beside the drop-down list, and to revert click the reset icon.
Based on the requirements, you can do the following:
- Change the default Property values, then a button appears beside the changed value and to revert click Reset to Default button.
- To rest all the values and set to accommodate general industry standards and data formats, click Reset All to Default link.
Note: To view specific property details, you can enter the keyword in the search bar, and the particular property details are displayed.
Run Property type: Hive Properties
|Property||Description||Default value||Other possible values|
|Field Delimiter||Specifies delimiter for structures fields, the default is '\001', which is the CTRL-A.||1|
|External Table||To create target table as external table||No||Yes|
|Managed Table WareHouse Location (Displays only when External Table is "No")||To manage location when target is being created as internal table|
|External Table WareHouse Location (Displays only when External Table is "Yes")||To manage location when target is being created as external table||/tmp|
|Escape Character||The character immediately following the escape character is escaped.This needs to be specified if the text qualifier is provided and the text qualifier character can appear in the source data.||Empty||\\ (Recommended)Any ASCII character|
|Collections Items Delimiter||Specifies a delimiter for array items||\002|
|Line Delimiter||Specifies a delimiter for map keys. You can include a newline character only ('\n'). Newlines must not exist within the data itself, and they cannot be escaped||\n|
|File Format||Specify hive built in and custom-developed file format of the file.||ORC|
|Orc Compression Type (Displays only when File Format is "ORC")||Specify the compression method to be applied on the file placed in DFS||ZLIB|
|Orc Stripe Size (Displays only when File Format is "ORC")||Specify the number of bytes in each stripe||268435456|
|Row Format||Specify the row format of the table||DELIMITED||SERDE|
|Orc Row Index Stride (Displays only when File Format is "ORC")||Specify the number of rows between index entries (must be>=1000)||10000|
|Serde Jar Path||Define the Serde Jar location in the DFS|
|Table Properties||TBL PROPERTIES clause allows you to tag the table definition with your own metadata key/value pairs|
|Orc Create Index (Displays only when File Format is "ORC")||Specify whether to create row indexes||TRUE|
|Serde Class||The fully qualified class name that implements the Hive SerDe interface org.apache.hadoop.hive.serde2.SerDe||com.bizo.hive.serde.csv.CSVSerde|
|Serde Properties||Comma separated list of SerDe property and its value pairs. These go as “WITH SERDEPROPERTIES” in DDL of the table||"separatorChar"="\u0001","quoteChar"="'","escapeChar"="\u0004"|
|Input Format Class (Displays only when File Format is "CUSTOM")|
In the file_format to specify the name of a corresponding Input Format
For example, 'org.apache.hadoop.hive.contrib.fileformat.base64.Base64TextInputFormat'.
|Output Format Class (Displays only when File Format is "CUSTOM")||In the file_format to specify the name of a corresponding output Format|
|Orc Compression Size (Displays only when File Format is "ORC")||Specifies number of bytes in each compression chunk||262144|
|Compression||Use this option to specify if the data needs to be compressed before transfer from source agent to Target load agent.||FALSE||TRUE|
|Compression Type ( Displays only when Compression is "True")||On choosing a compression format, the user is required to give the corresponding compression codec and compression type.||NONE|
|Intermediate Compression ( Displays only when Compression is "True")||Should the outputs be compressed before being sent across the network. Uses Sequence File compression.||FALSE||TRUE|
|Intermediate Compression Codec ( Displays only when Compression is "True")||If the outputs are compressed, how should they be compressed? (i.e. Snappy).|
|Orc bloom filter columns (Displays only when File Format is "ORC")||Specifies comma separated list of column names for which bloom filter should be created|
|Orc bloom filter fpp (Displays only when File Format is "ORC")||Specifies false positive probability for bloom filter (must>0.0 and <1.0)||0.05|
Run Property type: Hive Optimization & Map Reduce Properties
- From the drop-down select Hive Optimization or Map Reduce Properties to modify the properties corresponding to these.
- To add a new property, click Add icon.
New Runtime properties window appears and allows you to select from the list.
- Select one or more properties and click Ok.
- New entry for the property is generated.
- The "Property" field displays the assigned property.
- The "Value" field displays the default value assigned to the property and if required you can change the value.
- You can change the value by clicking on the value under the "Value" field. This could be a drop-down or text box. Select or enter the value to be used for the property during runtime.
- To delete any added property, select the property from the list and click Delete icon.
Propagate Runtime Properties to dependent objects
When a runtime property is added, modified or deleted in the data point, then the changes need to be propagated to the dependent objects so that these are effective during execution. To propagate changes follow below steps.
Step I: After making changes in the runtime properties, save the data point.
Step II: Select the property that was added or modified and click Propagate icon. If a property was deleted, then without selecting any property, click the Propagate icon.
Step III: The Propagate Runtime Properties window opens and shows the selected properties and the impacted objects.
Step IV: In the wizard, click OK and wait for the propagation to complete.