Table of Contents

Products, services and company names referenced in this document may be either trademarks or registered trademarks of their respective owners.

Copyright © 2017–2020 EVL Tool, s.r.o.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts.

Version 1.0

Released: 2017/07

Changes: First official version was released in the summer 2017 after more than a year of design and development and after first industry implementation in T-Mobile CZ.

New features:

  • Lookup tables’ – lookup loaded into memory and used in mappings.
  • Checksum functions’ – standard checksum function for strings: ‘md5’, ‘sha224’, ‘sha256’, ‘sha384’, ‘sha512’.
  • HDFS support
  • Spark code generation – Parquet and Impala integration
  • Job Manager

New components:

  • Aggreg’ – do aggregation for groups of records.
  • Cat’ – concatenate several input flows into single output one.
  • Comp’ – use custom component, which is actually another job.
  • Cut’ – omit fields from input by the output data definition.
  • Filter’ – for simple one- or two-way switch. For more complex use ‘Map’.
  • Join’ – join two input flows by the key. Catch left/right or even unmatched records.
  • Map’ – transform input fields and write into output fields.
  • Read’ – read file(s) into output flow, uncompress if needed.
  • Sort’ – sort, deduplicate, check sort; simply the output is always sorted by the key.
  • Tee’ – replicate one input flow to several output ones.
  • Trash’ – like /dev/null.
  • Write’ – write the flow into file, compress if needed.

New commands:

  • Mkdir
  • Mv