000%

ScalaPipeline Framework

<!-- A data pipeline framework built with Scala. -->

&Role

  • * Developer

&Stack

  • * Scala 3
  • * sbt

&Year: * 2024

&Type: * Personal

ScalaPipeline is a data pipeline framework I want to build in Scala. The idea is to read data from a source, apply some transformations and checks, then write the result somewhere else, with types so that the compiler catches the mistakes.

For now it is only the start. The sbt project is set up with Scala 3 and the code is just a hello world. I wrote the list of what I want to do in the README (modular steps, error handling, sources and outputs you can add, logs, parallel processing) but none of it is built yet.