TIL about Bytewax
Bytewax is an open source Python framework for building highly scalable dataflows in a streaming or batch context.
Heard about the project via a Python Podcast.__Init__ episode. Being a Message Processing Nerd (TM), I was both excited and skeptical. There have been a fair number of Python stream processing libraries, none of which I’ve really cottoned to, primarily because they often seem like second class citizens in a Java world.
Bytewax takes a different angle, tightly coupling with Rust. And when Zander Matheson, CEO and co-founder of bytewax, started riffing on Frank McSherry’s “Scale at What COST?” and timely dataflow then I became really intrigued. I will definitely have to give this framework a testdrive.