← All projects
MLOps & Cloud Intermediate

PipelineSmith

Reusable ML CI/CD with a Jenkins shared library

One shared pipeline library that every ML repository can reuse: test, train, evaluate, containerise and deploy with a five-line Jenkinsfile.

SHIPSTACK · PROJECT PipelineSmith Reusable ML CI/CD with aJenkins shared library Intermediate MLOps & Cloud 1 Git push 2 Shared library 3 Build image 4 Kubernetes 5 milestones

The scenario

PipelineSmith solves the copy-pasted-Jenkinsfile problem for ML teams. You build a Jenkins shared library with stages for data checks, training, evaluation gates, image builds and Kubernetes deployment, then onboard two sample ML projects to it.

System architecture

01 Git push Any ML repo
02 Shared library Standard stages
03 Train + evaluate MLflow tracking
04 Build image Docker registry
05 Kubernetes Rolling deploy

High-level flow of PipelineSmith, from input to output.

What you will learn

  • Jenkins shared libraries
  • Evaluation gates in CI
  • Deploying models to Kubernetes

What you will build

  • Shared library
  • Two onboarded ML repos
  • Model deployment to a cluster

Build plan

  1. Jenkins setup
  2. Library stages
  3. Eval gate
  4. K8s deploy
  5. Onboard repos

Tech stack

JenkinsGroovyDockerKubernetesMLflow