To change the settings for data load, click Load Properties tab.

In the canvas from the Load Type drop-down list, you can select Avro or File, and the default load type is Avro. When you change the default load type, a reset icon appears beside the drop-down list, and to revert click the reset icon.

Based on the requirements, you can do the following:

  • Change the default Property values, then a button appears beside the changed value and to revert click Reset to Default button.
  • To rest all the values and set to accommodate general industry standards and data formats, click Reset All to Default link.
Note: To view specific property details, you can enter the keyword in the search bar, and the particular property details are displayed.

Load type: AVRO

PropertyDescriptionDefault ValueOther possible values
Name SpaceCan generate desired ‘namespace’ into Avro schema file

File NameSpecifies the name of the Avro fileinstNm.dat
Schema NameCan generate desired ‘name’ into Avro schema file.

File LocationSpecifies the location of the Avro file$$SrcFileDir$$TgtFileDir
File SplitSelect Yes if you want Split the file else select NoNoYES
Split File On (Displays only when File Spilt is "yes")Select whether to split the file based on size or record countSizeRecord count
No of records per fileSpecifies the number of records per each file1000
Max Size Per File in KB (Displays only when File Spilt is "yes")Specify the size of each file when Split File ON property is enabled1024
Create Avro Schema FileSpecifies whether to create Avro schema file or notNoYES
Schema File (Displays only when Create Avro Schema File is "yes")Specifies the file location along with path

Create Trailer FileAsks whether to create trailer file or notNoYES
Trailer File Template (Displays only when Create Trailer File is "yes")Specifies the Trailer File templateDisplays the file template
Trailer FileSpecify to maintain the details of splitted avro files and schema file details like file path, count and total no. of records in each splitted file.

Compression CodecSelect the compression codex from XZ, Snappy, bzip2 or NoneNone

bzip2

snappy

deflate

xz

Compression Deflate Level (Displays only when Compression Codec is "deflate" or "xz" )Specifies the compression deflate level-1
Bad DirectorySpecifies the directory name where all Bad Records encountered while uploading data to Avro are stored$$BadFileDir
Bad Records ThresholdSpecifies number of bad records allowed in the bad directory for a successful Data Flow0
Feedback Batch SizeSet the batch size1000
File FormatSpecifies the type of fileavro
Generate Checksum?Used to provide some assurance that a downloaded file has not been altered.NoYES
Checksum Algorithm (Displays only when Generate Checksum is "yes")To generate checksums by using Algorithm (MD5,SHA1,CRC32..e.t.c).MD5

SHA1

SHA256

SHA512

CRC32

Writer EncoderSpecify the encoding format of the system.UTF-8

ISO-8859-1
UTF-16
UTF-16BE
WINDOWS-1252

Sync IntervalThe stream is flushed by default at the end of each synchronization interval, syncInterval - the approximate number of uncompressed bytes to write in each block64000

Load type: File

PropertyDescriptionDefault ValueOther possible values
File NameSpecify name of the fileinstNm.dat
File TypeSpecify type of the fileDelimiter

Fixed Width

Excel

Is Decryption Required?Specify this option if the source is encrypted file and want to decrypt the data before loadingNOYES
Enable Uncompress?Specify this option if the source is compressed file and want to uncompress the file before loadingYESNO
Include Header In FileSpecify if the header needs to be included in target file.NOYES
Create Layout FileSpecify the layout format to be createdNONE

CSV, AVSC

Layout File Name (Displays only when Create Layout File option is chosen as CSV or AVSC)Specifies the default layout file name for the chosen CSV or AVSC option.instanceNm_layout.avsc
Note: The changes are purely based on the architectural requirements.