Announcing the Kubeflow Spark Operator: Building a Stronger Spark on Kubernetes Community

Apr 15, 2024 • Vara Bonthu , Chaoran Yu , Andrey Velichkevich , Marcin Wielgus • 4 min read operators We’re excited to announce the migration of Google’s Spark Operator to the Kubeflow Spark Operator , marking the launch of a significant addition to the Kubeflow ecosystem. The Kubeflow Spark Operator simplifies the deployment and management of Apache Spark applications on Kubernetes. This announcement isn’t just about a new piece of technology, it’s about building a stronger, open-governed, and more collaborative community around Spark on Kubernetes. The journey of the Kubeflow Spark Operator began with Google Cloud Platform’s Spark on Kubernetes Operator (https://cloud. google. com/blog/products/data-analytics/data-analytics-meet-containers-kubernetes-operator-for-apache-spark-now-in-beta). With over 2. 3k stars and 1. 3k forks on GitHub, this project laid the foundation for a robust Spark on Kubernetes experience, enabling users to deploy Spark workloads seamlessly across Kubernetes clusters. Growth and innovation require not just code but also community. Acknowledging the resource and time limitations faced by Google Cloud’s original maintainers, Kubeflow has taken up the mantle. This transition is not merely administrative but a strategic move towards fostering a vibrant, diverse, and more actively engaged community.