Trevas

Trevas

Trevas is a Java engine for the Validation and Transformation Language (VTL), an SDMX standard that allows the formal definition of algorithms to validate statistical data and calculate derived data. VTL is user oriented and provides a technology-neutral and standard view of statistical processes at the business level.

What is Trevas?

Trevas (Transformation engine and validator for statistics) is a Java engine for the Validation and Transformation Language (VTL), an SDMX standard that allows the formal definition of algorithms to validate statistical data and calculate derived data. VTL is user oriented and provides a technology-neutral and standard view of statistical processes at the business level. Trevas supports the latest VTL version (v2.0, July 2020).

What does it do?

For actual execution, VTL expressions need to be translated to the target runtime environment. Trevas provides this step for the Java platform, by using the VTL formal grammar and the Antlr tool. For a given execution, Trevas receives the VTL expression and the data bindings that associate variable names in the expression to actual data sets. The execution results can then be retrieved from the bindings for further treatments. Trevas provides an abstract definition of a Java VTL engine, as well as two concrete implementations:

  • an in-memory engine for relatively small data, for example at design time when developing and testing VTL expressions on data samples
  • an Apache Spark engine for Big Data production environments

Trevas Modules

Trevas is made of several modules:

  • vtl-engine: Execution engine and its base Java implementation.
  • vtl-model: Model defining the objects used in the engine.
  • vtl-parser: Parser generated by Antlr from the VTL 2.0 formal grammar.
  • vtl-spark: Execution of VTL transformations by Spark.
  • vtl-sdmx: Tools for the use of SDMX metadata
  • vtl-jdbc: Tools for the use of SQL data sources.
  • vtl-csv: Tools for the use of CSV data sources.
  • vtl-jackson: JSON serialization / deserialization of data sets.

Focus on Trevas SDMX

Trevas SDMX provides tools to interact with SDMX 3.0 metadata. It provides, in particular, functions to build Trevas datastructures reading sdmx:mes (handling Dataflows, VtlMappingSchemes, DataStructures), to run TransformationSchemes and produce peristent datasets as ouput.

Installation

Trevas is published on Maven