Components Common
ETL jobs are defined in evs files and consist of components connected by flows.
In this chapter, all components are described.
There are two kinds of component commands:
- ‘Component’, i.e. component name begins with capital letter – these are intended to be used in evs file (EVL job structure files),
- ‘evl component’, i.e. command ‘evl’ with the first argument to be a name of the component – these invocations are to be used as standalone commands, which can be run from commandline.
Common Options
Mostly this invocation schema is applied on all components.
<Component> F_IN... F_OUT... EVD_IN... EVD_OUT... EVM
[ -v|--validate ]
[ -x|--text-input ]
[ -y|--text-output ]
where <Component> is the name of the component, like Read or Join.
F_IN... F_OUT...-
are names of the input flows or files and of the output flows or files.
- Flow names must be unique in each evs file. Convention is to have these names in CAPITAL LETTERS.
- Files – if
F_INorF_OUTcontains ‘/’ (foreslash), then it is treated as a file, not a flow. Remember that one can use special files, like /dev/stderr or /dev/null or in general /dev/fd/N.
EVD_IN... EVD_OUT...-
specifies the evd with the information about data structure, i.e. field names, data types, nullability, separator, etc. It can be:
- EVD file – a file with the data structure definition;
- Inline EVD – evd can be specified inline by options this way:
-D | --input-definition-
for input evd (when both – input and output – evd have to be specified);
-d | --output-definition-
for output evd (when both – input and output – evd have to be specified);
-d | --data-definition-
for input evd (whenonly one evd is needed by
<Component>).
EVM-
specifies the evm file which contains the mapping of the component.
All options are interpreted as in shell, so must be specified in one line or separated by ‘\’ at the end of the line.