Are you trying to deliver more functionality into production at a faster cadence? Are users asking your team to prioritize time-to-market over quality? Do you have release cycles longer than 3-4 days? Do you constantly review and prioritize issues that continue to resurface?
You might be in a great position to benefit from DevOps.
Hi, I’m Peter Nichol, Data Science CIO.
DevOps integrates the development and operations activity of the software development process to shorten development cycles and increase deployment frequency. DevOps centers around a handful of critical principles. If you’re ever unclear about how DevOps might apply, bring yourself back to the fundamental principles. This helps to drive the intent behind the outcomes achieved by adopting and implementing DevOps.
DevOps critical principles
- Continuous delivery: constantly delivering useable functionality
- Continuous improvement: introducing a steady pace of improvements to existing models
- Lean practices: improving efficiency and effectiveness by eliminating process waste
- System-oriented approaches: focusing on conditions required for maximum system effectiveness
- Focus on people and culture: optimizing people and inspiring culture to drive positive outcomes
- Collaboration (development and operations teams): establishing a shared vision and shared goals to enable inclusion
DevOps automates interactions that otherwise might result in unplanned delays or bottlenecks.
What is DevOps, and where did it originate?
Today, we’re going to talk about DevOps—development operations. DevOps embraces the idea of automated collaborations to develop and deploy software.
The term “DevOps” was coined in 2009 by Patrick Dubois, the father of DevOps. DevOps is a combination of development and operations. It’s a kind of classic example of yin and yang:
- Developers want change
- Operations want stability
Combining development and operations sounds simple enough, but what elements make up development and operations? Let’s take a minute to define these terms clearly.
Development includes:
- Software development
- Build, compile, integrate
- Quality assurance
- Software release
- Support of deployment and production tasks as needed
Operations includes:
- System administration
- Provisioning and configuration
- Health and performance monitoring of servers, critical software infrastructure, and applications
- Change and release management
- infrastructure orchestration
What problem does DevOps solve?
Developers typically want to release more code into production, enabling increased user functionality. This results in an environment of constant change and often questionable stability. Meanwhile, operations is focused on stability and less so on changes. Operations wants to establish a stable and durable base for operational effects; i.e., supporting production-grade and commercial applications that don’t fare well with constant changes.
The conflicting objectives between development and operations create polarization of the team. This polarization creates a natural conflict. DevOps attempts to bridge this gap through automation and collaboration.
DevOps connects your development operations teams to generate new functionality while maintaining existing or commercial functionality in production. DevOps primarily has three goals that are targeted either when it’s first introduced into an organization or after an internal DevOps optimization.
DevOps goals:
- Continuous value delivery
- Reduction in cycle time
- Improved speeds to market
Value delivery validates that what’s developed is useful and has business adoption. Cycle-time reductions go hand in hand with an increase in deployment frequency. Lastly, speed to market looks at the demand intake process and reduces the time from the initial request to when usable functionality is available in production. These combined benefits drive the majority of use cases in support of DevOps.
DevOps benefits
The more time you spend exploring the benefits of DevOps, the more obvious how expansive the benefits derived from implementing and adopting DevOps are. Below I’ve listed several benefits; however, let me first expand on a few of my favorites:
- Continuous value delivery: as we automate these different processes through the development and operations lifecycle, we remove waste and provide the ability to achieve continuous value.
- Faster orchestration through the pipeline: once we start to automate, we reduce the manual handoffs. As a result, that functionality moves through, gets tested, and gets put into production much faster.
- Reduced cycle times: as we put functionality into production and automate it, the time it takes from start to finish begins to decrease. This results in the perception of increased value or a faster cadence of the value we can provide.
Here are some additional and expected benefits of adopting DevOps principles:
Benefits of DevOps
- Continuous software delivery
- Faster detection and resolution of issues
- Reduced complexity
- Satisfied and more productive teams
- Higher employee engagement
- Greater development opportunities
- Faster delivery of innovation
- More stable operating environments
- Improved communication and collaboration
How do we measure success?
It sounds like DevOps has benefits. That’s great. But how do we measure the success of DevOps efforts? Entire books have been written on this topic. There’s no lack of knowledge available that’s been published and even posted on the Internet. However, I boil down measuring DevOps success into three buckets:
- Performance
- Quality
- Speed
By starting with these anchors for managing success, you’re sure to have a simple model to communicate to your leadership. Although there are hundreds of other metrics, I find these to be the most effective when presenting quantified outcomes of DevOps implementations and adoptions.
Performance
- Uptime
- Resource utilization
- Response time
Quality
- Success rate
- Crash rate
- Open/close rates
Speed
- Lead time
- Cycle time
- Frequency of release
Putting DevOps into action
We covered what DevOps is, where it originated, the problem it solves, the benefits, and how to measure success. Now we’re ready to discuss how to put DevOps into action. This begins by understanding the DevOps lifecycle:
- Build
- Test
- Deploy
- Run
- Monitor
- Manage
- Notify
I’ll expand on products that are used across each phase. However, to do that, I’m going to simplify the lifecycle steps into three phases:
- Develop and test: continuous testing
- Release and deploy: short release cycles
- Monitor and optimize: improved product efficiency
Here are best-in-class products that are commonly found in a DevOps pipeline:
Develop and test – continuous testing
- Confluence
- AccuRev
- JUnit
- IBM Rational
- Fortify
- Visual Studio
- VectorCast
Release and deploy – shorter release cycles
- ServiceNow
- Maven
- Artifactory
- NAnt
- Jenkins
- Bamboo
Monitor and optimize – improved product efficiency
- Zabbix
- Graphite
- Nagios
- QA Complete
- VersionOne
- New Relic
When we think about the aspects of DevOps, we quickly land on what type of tools we can introduce into these environments.
- Development: Visual Studio, Jira, GitLab, Azure DevOps, GitBucket
- Build: Jira, GRUNT (Bamboo plugins), Bamboo SBT, Gradle, and Jenkins
- Test: Visual Studio, TestLink, TestRail, Jenkins, LoadImpact, and JMeter
We also have DevOps tools like Jenkins, Chef, Puppet, and others that provide workflow and workflow automation.
A quick DevOps example
Many of my LinkedIn followers asked for a practical example of how DevOps is leveraged in the wild. Enjoy.
Situation: A new data-science and advanced-analytics team was recently launched that focuses on data literacy to allow access to data that must be validated and published. However, data stewards are reporting data publishing is taking too long and resulting in reporting on old data sets. The opportunity is to get data into the hands of power users (scientists, bioinformatics, and the data community) more quickly. Today, users are reporting slowness in data availability.
Complication: The data isn’t being ingested from a single source. In fact, because of the intelligence generated from the data sourcing, the timing of sources varies based on data timeliness. The team is leveraging a data lake to capture data relating to patient longitudinal, claims, EHR, EMR, and sales data such as IQVIA, Veeva CRM, SalesForce Marketing Cloud (SFMC) that includes promotional activities, general marketing, and financial data.
Resolution: The team developed an entire automation workflow in Jenkins. This resulted in a 42% improvement in data timeliness. Data was refreshed every four days, not every seven days.
Here’s the general approach that was taken to achieve the results:
- Develop + build + test
- Daily scrum – maximum of 15 minutes where each team member can share updates
- Sprint review
- Daily/weekly status updates to client
- Automated tasks to cover:
- Build management
- Automated unit testing
- QA environment configuration
- Code completion
- Code freeze
- Regression testing on preview/stage environment before going live
- Go live
DevOps offers opportunities to get to market faster, decrease cycle times, and ultimately get functionality to your business users more quickly. Decrease your dependence on manual processes and introduce DevOps throughout your delivery pipeline.
If you found this article helpful, that’s great! Also, check out my books, Think Lead Disrupt and Leading with Value. They were published in early 2021 and are available on Amazon and at http://www.datsciencecio.com/shop for author-signed copies!