MLOps moves into production environments for pipeline automation

A new and exciting area is machine-learning operations or MLOps. Today, we’re going to get into how to start and evaluate your machine-learning operations environment.

Hi, I’m Peter Nichol, Data Science CIO.

What is ModelOps?

One of the fascinating aspects of data science is understanding models. Usually, we begin with model operations or ModelOps. ModelOps includes everything that’s required to orchestrate your AI pipeline. A traditional definition of ModelOps might be the lifecycle management of all AI and decision models (including models based on machine learning, knowledge graphs, rules, optimization, linguistics, and agents). ModelOps is the foundation of many data-science environments.

MLOps or machine-learning operations is a subset of ModelOps. MLOps involves models that are trained using data. These models don’t simply replicate or duplicate human actions. They can learn based on outcomes and data loaded or digested into the model. Mature machine-learning operations have to do with the encapsulation and monitoring of those different areas. Essentially, MLOps holds the technical requirements required to keep our models up and running.

Feeding and caring for MLOps

MLOps isn’t explicit. This creates a challenge when we’re trying to deal with models—especially machine-learning models—as they’re not a simple business rules engine. These models have implied inferences that evolve. The data learns on its own. This leads to one of the biggest challenges—model degradation. The models can become highly inaccurate quickly when not properly maintained, especially if there’s an unplanned pivot in the environmental situation. A good example was the Coronavirus. Imagine a ModelOps environment that was creating forecasts in February 2020, and imagine the models it forecasted for March 2020. These didn’t have the same level of quality in terms of accuracy.

Machine-learning models can lose their predictive efficacy quite rapidly.

Here’s one more example. Let’s say we built a machine-learning model and operationalized it using ModelOps and MLOps. Our model was created for the travel industry. The model took into account factors such as airplane reservations, room, and board and built a package based on target demographics, location, and some other geographic factors. All this information was used to develop custom-priced packages with optimal price points. As soon as the Coronavirus wave spanned the country, our model almost immediately became valueless. Consider for a minute how quickly this model became useless. Almost overnight, that model turned out to be grossly inaccurate and, as a result, it probably didn’t help promote the growth it was intended to promote.

Models must be continually tuned and adapted to new conditions. Where does this tuning, maintenance, and upkeep responsibility live? It’s possible ModelOps lives under the CIO. Assuming the CIO owns AI architects and other types of business architects, this might be a logical place of ownership.

Where does MLOps live within the organization?

ModelOps could also live under a CTO. Many CTOs already have accountability for compliance and internal targets that lend themselves well to owning ModelOps.

Lastly, ModelOps might be most at home being nested under the Chief Data Officer’s responsibilities. It’s logical that ModelOps and MLOps are part of larger and more complex organizational digital transformations. These digital transformations also cover cloud-first and move-to-the-cloud initiatives that help to orchestrate new technical pipelines.

How do you get started with MLOps?

As you start thinking about how to apply improved MLOps operational efficiencies in your organization, allow me to provide a few pieces of insight:

  1. Define a common vision – The team needs to understand where the flagpole is located.
  2. Establish a leader –You must find a leader who understands how to lead.
  3. Identify candidate models for experimentation – Choose models that your team has defined that exhibit the highest degree of future potential.
  4. Define the operational requirements – Have a shared definition of what MLOps is and where it will live within your organization.
  5. Create visualizations (dashboards) – Make benefits tangible by evangelizing the benefits with a dashboard or visualization that allows people to understand where you are in the lifecycle of delivery and realization of that value for your MLOps initiative.
  6. Create a learning process – None of this stuff works perfectly the first time. You must provide feedback loops for learning and for the team to evolve.

It’s vital throughout the process to collaborate and work cross-functionally to engage individuals and groups that might not be technical but can provide insights based on their domain expertise.

Hopefully, you found the insights on ModelOps and MLOps useful and the discussion helped to add some clarity to a typically grey area of operational excellence.

If you found this article helpful, that’s great! Check out my books, Think Lead Disrupt and Leading with Value. They were published in early in 2021 and are available on Amazon and at for author-signed copies!

Hi, I’m Peter Nichol, Data Science CIO. Have a great day!

Previous articleExplaining the buzz around non-fungible tokens
Next articleA dynamic collaboration model for introducing change that sticks
Peter is a technology executive with over 20 years of experience, dedicated to driving innovation, digital transformation, leadership, and data in business. He helps organizations connect strategy to execution to maximize company performance. He has been recognized for Digital Innovation by CIO 100, MIT Sloan, Computerworld, and the Project Management Institute. As Managing Director at OROCA Innovations, Peter leads the CXO advisory services practice, driving digital strategies. Peter was honored as an MIT Sloan CIO Leadership Award Finalist in 2015 and is a regular contributor to on innovation. Peter has led businesses through complex changes, including the adoption of data-first approaches for portfolio management, lean six sigma for operational excellence, departmental transformations, process improvements, maximizing team performance, designing new IT operating models, digitizing platforms, leading large-scale mission-critical technology deployments, product management, agile methodologies, and building high-performance teams. As Chief Information Officer, Peter was responsible for Connecticut’s Health Insurance Exchange’s (HIX) industry-leading digital platform transforming consumerism and retail-oriented services for the health insurance industry. Peter championed the Connecticut marketplace digital implementation with a transformational cloud-based SaaS platform and mobile application recognized as a 2014 PMI Project of the Year Award finalist, CIO 100, and awards for best digital services, API, and platform. He also received a lifetime achievement award for leadership and digital transformation, honored as a 2016 Computerworld Premier 100 IT Leader. Peter is the author of Learning Intelligence: Expand Thinking. Absorb Alternative. Unlock Possibilities (2017), which Marshall Goldsmith, author of the New York Times No. 1 bestseller Triggers, calls "a must-read for any leader wanting to compete in the innovation-powered landscape of today." Peter also authored The Power of Blockchain for Healthcare: How Blockchain Will Ignite The Future of Healthcare (2017), the first book to explore the vast opportunities for blockchain to transform the patient experience. Peter has a B.S. in C.I.S from Bentley University and an MBA from Quinnipiac University, where he graduated Summa Cum Laude. He earned his PMP® in 2001 and is a certified Six Sigma Master Black Belt, Masters in Business Relationship Management (MBRM) and Certified Scrum Master. As a Commercial Rated Aviation Pilot and Master Scuba Diver, Peter understands first hand, how to anticipate change and lead boldly.