A new and exciting area is machine-learning operations or MLOps. Today, we’re going to get into how to start and evaluate your machine-learning operations environment.
Hi, I’m Peter Nichol, Data Science CIO.
What is ModelOps?
One of the fascinating aspects of data science is understanding models. Usually, we begin with model operations or ModelOps. ModelOps includes everything that’s required to orchestrate your AI pipeline. A traditional definition of ModelOps might be the lifecycle management of all AI and decision models (including models based on machine learning, knowledge graphs, rules, optimization, linguistics, and agents). ModelOps is the foundation of many data-science environments.
MLOps or machine-learning operations is a subset of ModelOps. MLOps involves models that are trained using data. These models don’t simply replicate or duplicate human actions. They can learn based on outcomes and data loaded or digested into the model. Mature machine-learning operations have to do with the encapsulation and monitoring of those different areas. Essentially, MLOps holds the technical requirements required to keep our models up and running.
Feeding and caring for MLOps
MLOps isn’t explicit. This creates a challenge when we’re trying to deal with models—especially machine-learning models—as they’re not a simple business rules engine. These models have implied inferences that evolve. The data learns on its own. This leads to one of the biggest challenges—model degradation. The models can become highly inaccurate quickly when not properly maintained, especially if there’s an unplanned pivot in the environmental situation. A good example was the Coronavirus. Imagine a ModelOps environment that was creating forecasts in February 2020, and imagine the models it forecasted for March 2020. These didn’t have the same level of quality in terms of accuracy.
Machine-learning models can lose their predictive efficacy quite rapidly.
Here’s one more example. Let’s say we built a machine-learning model and operationalized it using ModelOps and MLOps. Our model was created for the travel industry. The model took into account factors such as airplane reservations, room, and board and built a package based on target demographics, location, and some other geographic factors. All this information was used to develop custom-priced packages with optimal price points. As soon as the Coronavirus wave spanned the country, our model almost immediately became valueless. Consider for a minute how quickly this model became useless. Almost overnight, that model turned out to be grossly inaccurate and, as a result, it probably didn’t help promote the growth it was intended to promote.
Models must be continually tuned and adapted to new conditions. Where does this tuning, maintenance, and upkeep responsibility live? It’s possible ModelOps lives under the CIO. Assuming the CIO owns AI architects and other types of business architects, this might be a logical place of ownership.
Where does MLOps live within the organization?
ModelOps could also live under a CTO. Many CTOs already have accountability for compliance and internal targets that lend themselves well to owning ModelOps.
Lastly, ModelOps might be most at home being nested under the Chief Data Officer’s responsibilities. It’s logical that ModelOps and MLOps are part of larger and more complex organizational digital transformations. These digital transformations also cover cloud-first and move-to-the-cloud initiatives that help to orchestrate new technical pipelines.
How do you get started with MLOps?
As you start thinking about how to apply improved MLOps operational efficiencies in your organization, allow me to provide a few pieces of insight:
- Define a common vision – The team needs to understand where the flagpole is located.
- Establish a leader –You must find a leader who understands how to lead.
- Identify candidate models for experimentation – Choose models that your team has defined that exhibit the highest degree of future potential.
- Define the operational requirements – Have a shared definition of what MLOps is and where it will live within your organization.
- Create visualizations (dashboards) – Make benefits tangible by evangelizing the benefits with a dashboard or visualization that allows people to understand where you are in the lifecycle of delivery and realization of that value for your MLOps initiative.
- Create a learning process – None of this stuff works perfectly the first time. You must provide feedback loops for learning and for the team to evolve.
It’s vital throughout the process to collaborate and work cross-functionally to engage individuals and groups that might not be technical but can provide insights based on their domain expertise.
Hopefully, you found the insights on ModelOps and MLOps useful and the discussion helped to add some clarity to a typically grey area of operational excellence.
If you found this article helpful, that’s great! Check out my books, Think Lead Disrupt and Leading with Value. They were published in early in 2021 and are available on Amazon and at http://www.datsciencecio.com/shop for author-signed copies!
Hi, I’m Peter Nichol, Data Science CIO. Have a great day!