A modern and performant/robust dag processor for data pipelines allowing processing data in/out from storages like S3 or Iceberg/delta lakes without interruptions.

Whenever it is not feasible for an Apache Airflow / NIFI / Hadoop flying circus alike. Legacy software could remain operating on your storage/lake data in conjunction with Thund handling In/Out. For a complete modern stack combine Apache Arrows Balista/Datafusion in combination with Thund.

If you dont get it , no worries its an early experiment , perhaps "Grímnismál" (Year 1300-1325) in the Poetic Edda explains it goal better
"Thunda's waters hast'ning fleet,
Touch not Valgom! with thy feet."

• Apache Arrow centric
• Minimalistic / Performant / Robust
• Conform to Brokkr's Software critera https://github.com/Ignalina/brokkr

Goals below are to be sorted for V1,V2 or V never

• Fix eventhandler/step arguments from simple reader to functions for create reader and writer.
• Picture of the watcher->eventhandlers mechanics and tossing of parameters.

• Alloy component , Could Arrow references be used betwen Golang-Rust ?
• Support for Arrows filesystem HDFS,
• Incorporate RCLONE
• Graph support
• Add handlers to Arrow->Tantivy/Apache flight/kafka/delta-rs
• Handlers Deployable/Callable from minifi

• Steps spread out on multiple Processors
• Jaeger
• Metrics
• Static Deployment via ipmi
• Deployment via kubernetes, as static as possible.

Translations poeems describing Thund Germanic mythology
Learn pronounce in Icelandic ÓÐSMÁL