EVL – ETL Tool


Products, services and company names referenced in this document may be either trademarks or registered trademarks of their respective owners.

Copyright © 2017–2022 EVL Tool, s.r.o.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.

Table of Contents

Aggreg

(since EVL 1.0)

Applies aggregation mapping on each group of records based on the <key>.

Aggreg

is to be used in EVS job structure definition file. <f_in> and <f_out> are either input and output file or flow name.

evl aggreg

is intended for standalone usage, i.e. to be invoked from command line and reading records from standard input and writing to standard output.

EVD, EVM and EVS are EVL definition files, for details see evl-evd(5), evl-evm(5) and evl-evs(5).

Synopsis

Aggreg
  <f_in> <f_out> (<evd_in>|-D <inline_evd>) (<evd_out>|-d <inline_evd>)
  <evm> --key=<key>   
  [-c|--check-sort] [-i|--ignore-case] [-x|--text-input] [-y|--text-output]    
  [-o <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]    
  [--reject <f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]   

evl aggreg
  (<evd_in> | -D <inline_evd>) (<evd_out>|-d <inline_evd>)
  <evm> --key=<key>
  [-c|--check-sort] [-i|--ignore-case] [-x|--text-input] [-y|--text-output]
  [-o|--output <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [-r|--reject <f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]
  [-v|--verbose]

evl aggreg
  ( --help | --usage | --version )

Options

-c, --check-sort

check if the input is really sorted according to specified key

-D, --input-definition=<inline_evd>

either this option or the file <evd_in> must be presented. Example: ‘-D 'id int, user_id string'

-d, --output-definition=<inline_evd>

either this option or the file <evd_out> must be presented. Example: ‘-d 'user_sum long'

-i, --ignore-case

be case insensitive for key fields

-k, --key=<key>

group by this key, where <key> is comma separated list of fields with type (either DESC or ASC, default type is ASC). Example: ‘--key='id,user_id DESC'

-o, --output=<f_out>

when output() function is used in the mapping, out structure is forwarded into <f_out>

--output<n>=<f_out>

when function ‘output(<n>)’ is used in mapping, where <n> is an integer from 4 to 16, out structure is forwarded into <f_out>

--outputs=<varname>

specifies an array ‘${<varname>[@]}’ which contains filenames to be used for output(N) functions in mapping. Example: for ‘--outputs=OUTFILE’, ‘${OUTFILE[120]}’ is the filename used for ‘output(120)

-r, --reject=<f_out>

when reject() function is used in the mapping, input record is rejected into <f_out>

--reject<n>=<f_out>

when function reject(<n>) is used in mapping, where <n> is an integer from 4 to 16, input record is rejected into <f_out>

--rejects=<varname>

specifies an array ‘${<varname>[@]}’ which contains filenames to be used for reject(N) functions in mapping. Example: for ‘--rejects=REJECTS’, ‘${REJECTS[1000]}’ is the filename used for ‘reject(1000)

-x, --text-input

suppose the input as text, not binary

-y, --text-output

write the output as text, not binary

Standard options:

--help

print this help and exit

--usage

print short usage information and exit

-v, --verbose

print to stderr info/debug messages of the component

--version

print version and exit

Examples

  1. To print to stdout average of amount values:
    evl aggreg -D 'amount int' -d 'avg int' average.evm -k '' -xy <in.txt
    

    File ‘average.evm’ might look like this:

    VARIABLES:
    int count;
    long sum;
    
    INITIALIZE:
    count=0;
    sum=0;
    
    COMPUTE:
    sum += *in->amount;
    count++;
    
    FINALIZE:
    *out->avg = sum/count;