THREAD: Is it possible that Kubeflow pipeline is one of the best CI/CD tools for Kubernetes?
I spent some time playing with Kubernetes & @kubeflow pipelines, and they have one feature which is just great:
You can define the pipeline with real code!
I spent some time playing with Kubernetes & @kubeflow pipelines, and they have one feature which is just great:
You can define the pipeline with real code!
1/7
Kubeflow Pipelines is a CI/CD tool for machine learning.
Every step of the pipeline runs in a container — just like other CI/CD tools (Drone, Jenkins X, etc.)
Kubeflow Pipelines is a CI/CD tool for machine learning.
Every step of the pipeline runs in a container — just like other CI/CD tools (Drone, Jenkins X, etc.)
2/7
But instead of writing YAML to define the steps, you can use a real programming language!
You can use Python to write the code and then submit it to Kubeflow.
If you prefer, you can still use YAML to define the pipeline.
But instead of writing YAML to define the steps, you can use a real programming language!
You can use Python to write the code and then submit it to Kubeflow.
If you prefer, you can still use YAML to define the pipeline.
3/7
What's excellent about Kubeflow is that you can decorate your existing Python function and make them a step in the pipeline.
Kubeflow packs & runs the function into a container and takes care of passing arguments in stdin/stdout.
What's excellent about Kubeflow is that you can decorate your existing Python function and make them a step in the pipeline.
Kubeflow packs & runs the function into a container and takes care of passing arguments in stdin/stdout.
4/7
But there's more.
You can compile the pipeline into YAML and upload it or...
You can use the API to submit it.
Recap:
1. write code to define the steps
2. use code to submit the steps as a pipeline
But there's more.
You can compile the pipeline into YAML and upload it or...
You can use the API to submit it.
Recap:
1. write code to define the steps
2. use code to submit the steps as a pipeline
5/7
If you're using a Jupyter Notebook, the experience is seamless.
1. decorate your functions
2. call the pipeline API
3. enjoy your model trained at scale
If you're using a Jupyter Notebook, the experience is seamless.
1. decorate your functions
2. call the pipeline API
3. enjoy your model trained at scale
6/7
Since the pipeline is just code, you can express all your logic in Python.
No more clever YAML!
Why isn't writing code over YAML more popular?!
What do you think?
Also...
Since the pipeline is just code, you can express all your logic in Python.
No more clever YAML!
Why isn't writing code over YAML more popular?!
What do you think?
Also...
7/7
If you like CI/CD pipelines and machine learning, you should tune in for tomorrow's webinar on Kubeflow with @SoulmanIqbal
This thread is based on his work.
Register here: event.on24.com/wcc/r/2451691/…
If you like CI/CD pipelines and machine learning, you should tune in for tomorrow's webinar on Kubeflow with @SoulmanIqbal
This thread is based on his work.
Register here: event.on24.com/wcc/r/2451691/…
