EVL – ETL Tool

Release Notes

Versions numbering: EVL x.y.z

x – major release, i.e. big changes must happen to advance this number

y – minor releases, i.e. introduce new features

z – bugfixes

Overview

Version 1.0 (2017/07)

Initial version of “classical” ETL tool with HDFS support.
New components: Aggreg, Cat, Comp, Cut, Filter, Join, Map, Read, Sort, Tee, Trash, Write.
New commands: Mv, Mkdir.

Version 1.1 (2017/11)

Read/Write-Avro (in flat format only), Read/Write-Kafka, Read/Write-Teradata. Validation functions within mapping: ‘is_valid_*’.
New components: Head, Readavro, Readkafka, ReadTD, Tail, Validate, Writeavro, Writekafka, WriteTD.

Version 1.2 (2018/01)

Nested fields, Partitioning/parallelism, Read/Write JSON/Avro with full nested fields support.
New utilities: evd_remove_comments, evd_to_avro_scheme.
New components: Assign, Depart, Gather, Merge, Partition, Readjson, Run, Watcher, Writejson.

Version 1.3 (2018/03)

Read/Write XML, Records generator.
New components: Generate, Readxml, Writexml.
New command: Cp.

Version 2.0 (2018/11)

EVL Job Manager renamed to EVL Worklow, man pages, Read/Write PostgreSQL, standalone components, ‘in_to_out()’ enhanced, Shared lookup tables, Read/Write Parquet, sort within a group.
New utilities: evl_increment_run_id – unique EVL run ID.
New components: Echo, Lookup, Readparq, ReadPG, RunPG, Sortgroup, Writeparq, WritePG.
New commands: Fr, Ls, Rm, Spark.

Version 2.1 (2019/04)

Microservices introduced, Read/Write Oracle, Write XLSX and QVX, Docker image available.
New components: ReadOra, RunOra, WriteOra, Writeqvx, Writexlsx.
New commands: Mail, Test.

Version 2.2 (2019/10)

EVL Manager – graphical web UI, output to Qlikview/Qliksense’s QVD file,
New utilities: csv2evd, csv2qvd.
New components: Readxls, Readxlsx, Tac, Wc, Writeqvd.

Version 2.3 (2020/04)

EVL Manager enhancements, Read and Write supports also sftp://, gs://, s3://, Clang compiler can be used, debug mode.
New utilities: qvd_header.
New components: Readqvd.
New commands: Chmod.

Version 2.4 (2020/10)

EVL Manager – Microsevices integration, crontab generation, anonymize IBAN, IP addresses manipulation, MySQL/MariaDB, password handling.
New components: Readmysql, Runmysql, Writemysql.
New commands: cancel, crontab, info, manager, Sleep.
New utilities: json2evd, qvd2evd.

Version 2.5 (2021/04)

Read and Write supports also Samba and database URIs, randomize IBAN, EVD manipulation components.
New components: Readevd, Writeevd.

Version 2.6 (2021/10)

New options for Readqvd to speed up reading subset of QVD file, secret password handling.
New components: Wc.
New commands: Chown, Rmdir.