Is DataOps the same as DevOps? It’s not, and I’m going to explain why.
Hi, I’m Peter Nichol, Data Science CIO.
One of the challenges we see with development operations (“DevOps”) is that folks don’t understand what it means. The term is becoming more and more popular, so I thought I’d take a minute to explain it. DevOps is a lot different than DataOps. DataOps focuses on automation. It’s a process-oriented methodology that data-science teams use to simplify or streamline a data-science workflow, typically generating analytics. Where did the concept of DataOps originate?
It came from Edward Deming, an American engineer, statistician, professor, author, lecturer, and management consultant. He’s also referred to as “the father of quality.” In general, he focused on quality and production control and, more specifically, on the area of statistical process control. If we look at DataOps—specifically the role of DevOps—we start to understand what DataOps is all about.
Let’s break down what DataOps means. The foundation of DataOps has three main principles.
First, DevOps is focused on the delivery and development of some entity. Second, it’s based on agile—commonly applied to software development—and on the Theory of Constraints, which centers around removing obstacles and trying to take the shortest path to achieve an outcome. Third, DataOps is made up of lean manufacturing principles. Lean manufacturing emphasizes quality and production efficiency and uses statistical process control to monitor and control process variance or deviations.
Okay, now that you understand the foundation, where’s this going? When we start to think about value and how it comes into the picture, we can add a data factory or data-orchestration concept. The benefit of data operations is that it’s not only the outcome that we’re looking for—i.e., a validated analytical model with visuals—but we’re also making sure the data goes into the process correctly. We focus on the throughput as well—what’s happening with that data and information as they translate through the model.
Lastly, who’s using data operations? This is the big difference between DataOps and DevOps—the users. DevOps typically has engineering—software engineers that know multiple languages all dedicated to streamlining the orchestration of that technical delivery. In DataOps, we have other roles. We now add data scientists and typically data analysts that don’t care about all the technical-orchestration details. They want to make sure that their models can be simulated, executed, and that the data or the outcomes—and, ultimately, the insights—are usable and can be transformed into some benefit for their business users.
As you think about DataOps versus DevOps, reflect for a minute. There might be an opportunity in your environment to emphasize DataOps to orchestrate and streamline the workflow of how your team generates quality analytics for your business partners.
Hi, I’m Peter Nichol, Data Science CIO. Have a great day!