Avro Data object corresponds to structure of Avro files. The Avro data object can be associated with file server data points, like, file data point, rest data point, GCS data point, AmazonS3 data point, Hadoop DFS data point, JMS data point or Kafka data point. For more details on these data points, refer Working with Data Point.

To work with Avro Data Object, follow below steps:

Step I: Edit, create or import a File data object

It is recommended to create data object by importing the structure from the data file or schema file. Here, you can either select a schema file (avsc) or a GCS file type while importing and it is used in creating a Avro Data object.

Step II: Configuring attributes

The Avro Data Object contains following fields. These fields can be edited by clicking on the field and inputting the required value.

  • Level: Level number is used to specify the level of data in a record. Level numbers describe the hierarchy of the data items/ variables declared. They are used to differentiate between elementary items and group items. The level numbers include levels from 01 to 49 and special purpose levels 66, 77 and 88.
  • Attribute: Displays the name of the field at a certain position in the file. This is a text field and name can be defined as needed. For Avro data object the name of the field is immaterial. These field names will be used when this data object is used to create data objects of other types.
  • Data Type: Displays the datatype associated with the field. This is a list of applicable data types. You can select appropriate type for the field.
  • Precision: Displays the precision for the varchar, number and decimal data type. This is a text field and name can be defined as per the data in the file.
  • Scale: Displays the scale for the number and decimal data type. This is a text field and name can be defined as per the data in the file.
  • Not Null: Displays if the field can have NULL values. This is a checkbox and can be defined as per the data in the file. For Avro data object this field is immaterial. This property will be used when this data object is used to create data objects of other types.
  • Key Type: Displays if the field is key field. This is a dropdown with values Primary Key and Foreign key. Select one of these if the field is one of these keys. For Avro data object this field is immaterial. This property will be used when this data object is used to create data objects of other types.
  • Description: Displays any details provided for the field. This is a text field and you can enter any detail as need.

Note:

  • Following operations are allowed on the database entries: Add, Cut, Copy, Paste, Up, Down, Delete, and Search.
  • From the list of attributes, multiple attributes can be selected and we can perform/apply these operations.
  • To add a new attribute, click Add. By default a new attribute is populated in the last row. If you want to add an attribute at a specific position then, select the attribute in the position prior to it and click Add.
  • To search for a specific attribute, enter the keyword in the search bar, and the page displays the related attributes.

Step III: Configuring properties

The properties tab displays the object level details relevant to extract the data from file.

The grid displays following fields,

PropertyDescriptionDefault valueOther possible values
File NameThe name of the schema is assigned as File name along with .avro extension and its editable at runtimeSSCM_DETAIL_SG.avro
Name Space

Describes the name of the namespace in which the object resides

SSCM_DETAIL_SG
File LocationSpecify the file location/tmp
Schema NameDisplays the schema file nameSSCM_DETAIL_SG

NoteTo view specific property details, enter the keyword in the search bar, and the page displays the related properties.

Step IV: Save the changes

To save the changes made to the data object, refer Saving Data Object.

Note: