👋 Welcome to Flyte Monthly! Stay up to date with Flyte news, features, releases, events, success stories, and more.
A year of Flyte wrapped
A lot can happen in a year. For Flyte, it’s been quite a ride.
After 642 commits, dozens of hours of discussions and multiple interactions, Flyte has grown as a robust platform — and especially as a healthy community.
In 2023, we proposed an update to the project’s governance model. The model defines a set of roles and responsibilities for a Technical Steering Committee that includes members from different organizations. The committee has been generous with its time to support other contributors, add insights to multiple discussions and shape major initiatives.
This year, Flyte has seen 11 releases thanks to the collective efforts of maintainers and more than 130 contributors from organizations in all timezones. They demonstrate why this community is a great place to be.
During the new year, we plan to build additional mechanisms to ensure that community feedback drives project roadmap decisions and creates opportunities to join working groups that implement major initiatives.
The data science teams at Warner Bros. Discovery juggled multiple environments with notebooks and faced compatibility issues with Airflow, leading to inefficiencies and duplication of efforts. Flyte’s Python support harmonizes development and deployment, offering workflow compositions and Kubernetes-based scalability. Read the case study.
👥 Contributor's corner
New committer
Han-Ru Chen was unanimously selected as a new committer to Flyte. In a little more than six months, Han-Ru has contributed to 60+ PRs, from improving the Databricks plugin to creating content that lowers barriers for other new contributors. The Flyte community is lucky to count on Han-Ru’s dedication, passion, excellence and overall teamwork attitude. Let’s keep building together!
🎟️ Attend upcoming events
The Essential Role of Vector Databases in LLMOps with Zilliz— Fireside Chat
We were proud to sponsor the most recent Seattle MLOps Community happy hour. Huge thanks to the Seattle chapter for organizing and to everyone who joined us. It was great meeting new and familiar faces and discussing all the cool projects everyone’s been working on. Be on the lookout for more events in the new year!
As seen on the Community Sync
We had the privilege to learn first-hand how QuantumBlack, AI by McKinsey is leveraging Pandera, the OSS data validation framework, to build an SSOT solution where they: - Added PySpark support to enable big data workflows - Built utilities like on/off switch and validation depth controls - Added column metadata to auto-generate data requests
By default, Flyte stores metadata and raw data produced by Task executions on a single bucket.
If you need to specify separate storage buckets for raw data and metadata, add the name of your raw data bucket to the corresponding section of your Helm values file:
flyte-core
configmap.core.propeller.rawoutput-prefix
flyte-binary
configuration.storage.userDataContainer
Learn more about how Flyte handles data in the docs.
