Building a machine-learning model is often the visible part of an AI project.

It is also frequently the easiest part.

The harder question is what happens after deployment.

What happens when the data changes?

When a new model version performs worse?

When nobody remembers which dataset produced the current model?

When a regulator asks how a prediction was made six months ago?

When a pipeline silently fails?

When accuracy starts declining but the true labels needed to confirm the problem will not arrive for weeks or months?

Those questions became the focus of a systematic literature review I completed for my MSc module Enterprise AI Solutions.

I reviewed thirteen peer-reviewed studies looking at one central problem:

What makes an AI system scalable and robust once it moves into a real enterprise environment?

The literature pointed to a surprisingly consistent answer.

The model itself is only one component.

What determines whether enterprise AI survives is the infrastructure, governance and operational discipline built around it.

Production Changes the Problem

A model in development usually lives in a relatively controlled environment.

The dataset is known.

The code is available.

Experiments can be repeated.

Failures are inconvenient, but rarely catastrophic.

Production is different.

Real systems deal with:

  • changing data
  • multiple teams
  • live customers
  • infrastructure failures
  • regulatory requirements
  • evolving business rules
  • competing model versions
  • delayed feedback
  • operational deadlines

A model can perform extremely well in a notebook and still fail as part of a real business process.

That is why production AI cannot be treated as "train the model, deploy it, done."

Deployment is closer to the beginning of the operational lifecycle than the end.

Five Capabilities Kept Appearing

Across the studies, five capabilities repeatedly emerged as critical.

They were:

  1. Data governance
  2. MLOps
  3. Versioning
  4. ML-specific CI/CD
  5. Continuous monitoring

What interested me most was how dependent they are on each other.

None of them works particularly well in isolation.

1. Data Governance Comes Before Reliable AI

Machine learning depends on data.

That sounds obvious, but in production it creates questions that are much harder than simply obtaining a dataset.

Who owns the data?

Where did it come from?

Who changed it?

Is the definition of a field still the same as it was six months ago?

Can a model prediction be traced back to the data that influenced it?

The literature repeatedly highlighted data ownership, lineage and quality controls as fundamental.

Without them, many later MLOps capabilities become weaker.

A model registry can tell you which model version was deployed.

But if you cannot identify exactly which data produced that model, reproducibility is incomplete.

Likewise, monitoring a model is difficult if upstream data quality is not monitored as well.

This is why data governance should not sit somewhere outside the AI lifecycle as an administrative concern.

It is part of the engineering foundation.

2. MLOps Is More Than Automation

MLOps is often described as the machine-learning equivalent of DevOps.

That is useful, but incomplete.

In enterprise environments, MLOps is not simply about automating training and deployment.

It coordinates:

  • data engineers
  • data scientists
  • software engineers
  • infrastructure teams
  • security teams
  • business owners
  • governance functions

A production model moves through all of these areas.

MLOps provides the processes and tooling needed to move models through development, testing, deployment and monitoring in a controlled way.

This becomes particularly important when an organisation is managing tens or hundreds of models rather than one experimental system.

Manual processes that work for a single data-science team quickly become fragile at enterprise scale.

3. Version Everything

Traditional software development already relies heavily on code versioning.

Machine learning adds several additional moving parts.

A production prediction may depend on:

code version + dataset version + feature definitions + model version + configuration

Changing any one of these can change the result.

That makes reproducibility much more complicated than simply storing Python code in Git.

The literature strongly supported versioning across the complete ML lifecycle.

The practical reason is simple.

Imagine deploying a new model and discovering that performance has deteriorated.

Can you restore the previous version?

Can you recreate it?

Do you know exactly which data was used to train it?

Can you identify which feature pipeline was active at the time?

If the answer is no, rollback becomes much more difficult.

Versioning is therefore not administrative overhead.

It is a recovery mechanism.

4. CI/CD for Machine Learning Is Different

Continuous integration and continuous deployment are already standard practices in software engineering.

But machine learning introduces failure modes that traditional software tests do not detect.

Code can execute perfectly while the model becomes useless.

That means an ML pipeline needs to test more than whether the application builds successfully.

Depending on the system, automated checks may need to examine:

  • data quality
  • schema changes
  • missing values
  • feature distributions
  • model performance
  • bias
  • drift
  • compatibility between pipeline components

This was one of the clearest lessons from the review.

A passing software test does not mean a machine-learning system is healthy.

The code may be correct while the assumptions behind the model have changed.

That is why enterprise ML requires CI/CD practices designed around both software and data.

5. Monitoring Starts After Deployment

A conventional software system often fails visibly.

An API returns an error.

A service crashes.

A database becomes unavailable.

Machine-learning systems can fail much more quietly.

The service continues responding.

Predictions continue appearing.

Nothing crashes.

But the predictions gradually become worse.

That makes monitoring especially important.

The literature highlighted several areas that may need continuous observation, including:

  • input data distributions
  • prediction distributions
  • model performance
  • feature behaviour
  • infrastructure health
  • drift indicators

The difficulty is deciding when a change is meaningful.

Not every shift in data means the model is broken.

Real populations change.

Seasonality changes behaviour.

Markets evolve.

The monitoring system therefore has to distinguish between normal variation and deterioration significant enough to require intervention.

That is not always straightforward.

The Automation Debate

One of the most interesting disagreements in the literature concerned automation.

Some architecture-focused research strongly favoured highly automated ML pipelines.

The appeal is understandable.

Automation improves speed.

It reduces manual effort.

It makes processes repeatable.

It helps organisations scale.

But studies involving highly regulated or high-risk environments were much more cautious.

In those settings, human approval gates remained important.

A bank, healthcare organisation or critical infrastructure provider may not want a newly trained model to automatically replace the current production model simply because it passed a technical threshold.

There may be questions about:

  • compliance
  • fairness
  • explainability
  • business impact
  • operational risk

This produced what I think was one of the most important conclusions from the review:

Automation should not be the objective by itself.

The objective should be reliable and proportionate control.

A Risk-Based Lifecycle Makes More Sense

Instead of asking:

"How much of our ML lifecycle can we automate?"

a better question may be:

"Which decisions are safe to automate, and which require human oversight?"

The answer depends on the consequences of failure.

A low-risk recommendation system may justify extensive automation.

A model influencing credit, healthcare or critical infrastructure may need additional approval stages.

That suggests a risk-based lifecycle.

Automation can still be used aggressively where the consequences are manageable.

Where decisions carry greater legal, financial or social consequences, additional controls can be introduced.

This is more practical than assuming every organisation should aim for either complete automation or complete manual control.

The Gap Between Data Governance and MLOps

One of the clearest gaps I found in the literature was that data governance and MLOps are often discussed separately.

Governance research tends to focus on:

  • ownership
  • accountability
  • quality
  • compliance
  • lineage

MLOps research tends to focus on:

  • pipelines
  • automation
  • deployment
  • model registries
  • monitoring

But production AI needs both.

A highly automated MLOps platform built on poorly governed data can simply automate unreliable decisions faster.

Likewise, excellent governance policies that are disconnected from engineering workflows may exist only as documentation.

The stronger architecture is one where governance becomes executable.

For example:

data-quality rules become automated pipeline checks.

lineage is captured automatically.

approved datasets are enforced technically.

model versions are connected to training data.

deployment policies reflect risk classification.

That connection between governance and tooling is where I think a lot of future enterprise AI work needs to go.

Drift Is Easy to Discuss and Hard to Prove

Another issue that stood out was model drift.

In theory, the concept is simple.

The world changes.

The relationship between inputs and outcomes changes.

The model becomes less accurate.

In practice, detecting that reliably can be difficult.

One major reason is label delay.

Suppose a model predicts whether a customer will default on a loan.

You cannot immediately know whether the prediction was correct.

The true outcome may not be available for months.

The same problem appears in:

  • customer churn
  • fraud
  • insurance
  • medical outcomes
  • equipment failure

You may observe that incoming data has changed.

You may see that prediction distributions have shifted.

But you may not yet have the labels needed to confirm that predictive performance has actually deteriorated.

That means organisations often need to monitor proxy signals before ground truth becomes available.

And proxies are imperfect.

This makes drift detection one of the areas where production ML remains much harder than diagrams sometimes suggest.

Robustness Is a System Property

The main conclusion I took from the review can be summarised fairly simply:

A robust model does not automatically create a robust AI system.

You can have an excellent algorithm surrounded by:

poor-quality data,

weak access controls,

no reproducibility,

manual deployment,

missing monitoring,

and unclear ownership.

That system is not robust.

Conversely, a slightly less sophisticated model operating inside a disciplined lifecycle may be far more reliable in production.

That changes how enterprise AI should be evaluated.

The question is not only:

How accurate is the model?

It is also:

Can the organisation operate it safely, repeatedly and transparently over time?

The Model Is Only One Layer

I now think of production AI as a stack.

At one layer sits the model.

Around it are:

Data governance

Version control

Testing

Deployment pipelines

Monitoring

Security

Human oversight

Accountability

Remove enough of those layers and even a technically strong model becomes fragile.

That is why successful enterprise AI is as much an organisational and software-engineering challenge as it is a machine-learning problem.

What I Took Away from the Review

Before doing this literature review, it was easy to think of deployment as the final stage of an AI project.

Train.

Test.

Deploy.

Finished.

The research changed that view.

Production is where a different kind of engineering starts.

Models have to be watched.

Data has to be governed.

Versions have to be traceable.

Failures need recovery paths.

Deployment needs controls.

And organisations need to know who is responsible when something goes wrong.

That is why the biggest lesson from the review was not really about model architecture.

It was this:

Shipping a model is relatively easy. Keeping it trustworthy after it enters the real world is the hard part.

And that is ultimately what separates an AI experiment from an enterprise AI system.