MLOps Isn't Optional: Getting a Model From Notebook to Production Without Losing It

How to move an ML model from a notebook into reliable production: reproducible training, a model registry, CI for ML, safe deploys and drift alerts.

MLOps Isn't Optional: Getting a Model From Notebook to Production — Blurbee Solutions

A data scientist builds a model in a notebook. The results look great. Six months later it is still not in production, or it is in production and nobody is sure which version is running, what data trained it, or whether it still works. This is the most common story in applied machine learning, and it is almost never a modelling problem.

A notebook is not a system

Notebooks are good for exploration. They are poor foundations for production because:

  • Data was prepared by hand, in cells run in a particular order, sometimes more than once.
  • The environment is one person's laptop, with library versions nobody recorded.
  • Results cannot be reproduced, because the data, code and settings were never captured together.
  • There is no way to deploy, monitor or roll back.

Google's 2015 paper on hidden technical debt in machine learning systems made the point memorably: the model code is a small box in the middle of a much larger system of data collection, validation, serving and monitoring. MLOps is the practice of building that larger system properly.

Step 1: make training reproducible

To reproduce a model, you need four things versioned together:

  1. Code: the training and preprocessing code in version control.
  2. Data: a reference to the exact dataset or snapshot used.
  3. Configuration: hyperparameters and settings.
  4. Environment: library versions, captured in a container or lock file.

We turn the notebook into a pipeline that takes these as inputs and produces a model and its metrics as outputs. Anyone on the team can run it and get the same result.

Step 2: validate the data

Most production ML failures are data failures. Pipelines should check incoming data for missing values, unexpected ranges, schema changes and distribution shifts before training or prediction. Bad data should stop the pipeline loudly, not produce a quietly worse model.

Step 3: a model registry

Every trained model is stored with its metrics, data reference, code version and configuration. The registry tracks which version is in staging and which is in production. That makes rollback a one-step operation and answers the question every auditor and stakeholder eventually asks: which model made this prediction, and why?

Step 4: CI for machine learning

Changes to training code, features or configuration trigger automated checks: unit tests, data validation, a training run on a sample, and evaluation against a fixed test set. A model that performs worse than the current production version on the agreed metrics does not get promoted.

Step 5: deploy safely

We usually package the model as a containerised service behind an API, or as a batch job for scheduled predictions. New versions go out safely:

  • Shadow deployment: the new model runs alongside the current one on live traffic, and its predictions are recorded but not used.
  • Canary release: a small share of traffic moves to the new model, with close monitoring.
  • Rollback: the previous version is always one step away.

Step 6: monitor for drift

A model starts ageing the day it ships. Customer behaviour shifts, new products appear, upstream systems change. The model keeps producing predictions, just worse ones, and nothing breaks loudly.

Monitoring covers:

  • Input drift: are incoming features distributed like the training data?
  • Prediction drift: has the distribution of outputs changed?
  • Accuracy, where ground truth arrives later, such as whether a predicted churn actually happened.
  • Operational health: latency, errors and throughput.

Alerts on significant shifts tell you when to investigate or retrain.

Step 7: make retraining routine

With a reproducible pipeline and monitoring in place, retraining becomes routine: triggered on a schedule or by drift alerts, evaluated automatically, and promoted through the same safe deployment path. How often depends on how fast your data changes.

Start small

You do not need a large platform to begin. Version control, a simple experiment tracker and registry, a scheduled training pipeline, a containerised service and basic monitoring cover most first production models. Add more as the number of models grows.

LLMs follow the same rules

For applications built on large language models, the principles are the same: version everything, evaluate before release, monitor afterwards. The details differ. You version prompts and retrieval settings alongside models, and evaluation relies on curated test sets. See prompt engineering as a system and fine-tuning vs prompting vs RAG.

Signs your ML work needs MLOps now

  • A model is in production and nobody can say exactly which data trained it.
  • Retraining is a manual project that happens rarely, if ever.
  • Predictions have quietly got worse and it was noticed by a user, not a dashboard.
  • Only one person can deploy or update the model.
  • You cannot roll back to the previous model version in minutes.
  • Auditors or customers ask how a prediction was made, and the answer takes days.

Any one of these is a risk. Two or more means the model is running on luck.

Roles and ownership

MLOps is partly about tools and mostly about ownership. Someone must own each production model: its monitoring, its retraining schedule, and the decision to promote or roll back a version. Data scientists, engineers and the business owner of the prediction should agree the success metrics and alert thresholds before launch, not after the first incident.

Start with one model

Our first fixed-scope project takes one model from notebook to a reproducible pipeline, a registry entry and a deployed API with basic monitoring. Two to four weeks. We deliver MLOps for teams across India, the United States, the United Kingdom, the UAE and Canada; see AI and ML services.

MLOps, in numbers

2015Google's 'Hidden Technical Debt in Machine Learning Systems' showed model code is a small part of a real ML systemSculley et al., NeurIPS 2015
4 versionsWhat must be versioned together to reproduce a model: code, data, configuration and environmentBlurbee MLOps standard
DriftReal-world data changes after launch; unmonitored models degrade silentlyBlurbee delivery practice
Shadow firstNew models run alongside the current one on live traffic before they take overBlurbee delivery standard
Where effort goes in taking a model to production

Blurbee delivery practice, indicative split for a first production model. The modelling itself is rarely the largest part.

  • Data pipelines and validation ~30%
  • Serving, integration and APIs ~25%
  • Monitoring and retraining ~20%
  • Reproducible training and registry ~15%
  • Model changes ~10%
Share of effort

Why data teams take models to production with Blurbee

We turn notebooks into pipelines you can rerun, version and roll back, and we monitor the model after launch, because the day a model ships is the day it starts going out of date.

  • Fixed-scope first projectTwo to four weeks, priced as a number rather than an estimate, so you have evidence before a longer commitment.
  • One delivery lead, namedYou always know who to ask. No account manager relaying questions to a team you never meet.
  • Code and IP are yoursAssigned in writing, handed over with repositories, infrastructure definitions and documentation whether or not you continue with us.
  • Performance budgets in CICore Web Vitals and bundle size gate the build. Slow does not ship.
  • Accessibility to WCAG 2.2 AAReviewed with automated and manual passes, not claimed on a slide.
  • Written overlap disciplineStanding calls inside your working window, everything else asynchronous with a decision log.

Frequently asked questions

Frequently asked questions

What is MLOps?

Why do so many ML models never reach production?

What does a model registry do?
What is model drift?
How do you monitor a model in production?
How often should models be retrained?
What is shadow deployment?
Do we need a big MLOps platform?
How is MLOps different for LLM applications?
How long does it take to productionise a model?

Start with a fixed-scope first project

Two to four weeks, priced as a number. Code, accounts and IP stay in your name.