EVL – ETL Tool


Products, services and company names referenced in this document may be either trademarks or registered trademarks of their respective owners.

Copyright © 2017–2022 EVL Tool, s.r.o.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.

Table of Contents

Filter

(since EVL 1.0)

Filter records by the <condition>. Records for which the <condition> is false, are forwarded to the reject file or flow if specified.

In most cases filtering records would be better to do in ‘Map’ component using ‘discard()’ function. Check ‘man evl-map’ for details.

Filter

is to be used in EVS job structure definition file. <f_in> and <f_out> are either input and output file or flow name.

evl filter

is intended for standalone usage, i.e. to be invoked from command line and read records from standard input and write to standard output.

EVD is EVL data definition file, for details see evl-evd(5).

Synopsis

Filter
  <f_in> <f_out> (<evd>|-d <inline_evd>) <condition> [--reject=<f_out>]
  [-x|--text-input] [-y|--text-output]

evl filter
  (<evd>|-d <inline_evd>) <condition> [--reject=<f_out>]
  [-x|--text-input] [-y|--text-output]
  [-v|--verbose]

evl filter
  ( --help | --usage | --version )

Options

-d, --data-definition=<inline_evd>

either this option or the file <evd> must be presented. Example: -d ’id int, user_id string(6) enc=iso-8859-1’

-r, –reject=<f_out> catch rejected records into file or flow.

-x, --text-input

suppose the input as text, not binary

-y, --text-output

write the output as text, not binary

Standard options:

--help

print this help and exit

--usage

print short usage information and exit

-v, --verbose

print to stderr info/debug messages of the component

--version

print version and exit

Examples

Command line invocation examples: 1. To print to stdout only records with value of id less than 100:

evl filter -d 'id int' -xy < in.txt '*id<100'

Field ‘id’ is a pointer, so to get the value, ‘*id’ must be used.

2. Print to stdout only records where ‘id1’ is different from ‘id2’, records with the same ids will be send into ‘same_ids.csv’:

evl filter -d 'id1 int,id2 int' -xy -r same_ids.csv < in.csv '*id1 != *id2'

EVL job examples: 3. In an ‘evs’ file:

Filter  SOURCE OUTPUT  evd/sample.evd  "*currency == \"EUR\" && price"

This example filter out records with NULL ‘price’ and with currency equals ‘EUR’. (‘price’ is a pointer, so simply specifying ‘price’ in the condition means ‘price != nullptr’.)