Portfolio velocity: the new measure of business value realized

Welcome to the last time you report on the number of projects your team delivered.

Are you still talking about how many projects your team delivered last year? Please don’t. You’re misrepresenting what your team, department, and organization delivered. Stop talking about projects completed and start talking about portfolio velocity.

At the beginning of each year, CIOs need to commit to deliver work. This could refer to a group of critical projects, features, buckets of work, or even just finishing certain carryover projects. Each year, we make these commitments, and each year we tell ourselves there must be a better way.

The fallacy of using projects as a metric for capacity

How does your team estimate their capacity for the year? The most common measure is the simplest: the number of projects completed. On the surface, this sounds reasonable. We have critical projects. If those projects are completed, we declare victory. There’s just one minor problem: this method is flawed.

Organizational growth presents huge opportunities, and it carries equal challenges. Living and dying by the number of projects your department delivers doesn’t work. Here’s an example of why:

Year One went great. There were 100 projects completed. The team is perceived as really delivering. In Year Two, the team struggled to deliver 50 projects. And, by Year Three, the team delivered only 25 projects. What happened? The budgets for the projects were doubled each year. Staff was added. There’s no excuse. The peppering of reasons—ranging from too much work to not enough time—fall on deaf ears.

This isn’t the full story. We need to add context. When the team was delivering 100 projects, they weren’t complex, averaging less than two months with budgets of under $50K. Companies mature as they grow. Year Two required additional roles beyond the original technical lead and developer. An architect modeled the solution. The security lead addressed authentication and authorization. The quality lead tested for external interfaces, which didn’t exist in Year One. A project manager managed the vendor relationship and procurement process.

In Year Three, complexity, effort and the budget increased. Projects required more rigor to ensure success working across time zones, engaging third parties, and with more business and technology complexity. Projects involved multiple departments, which required shared services—e.g., legal, procurement, infrastructure, service desk, etc.—and demanded an average of four to nine months of effort to deliver. Projects also required more funding for delivery given the increased number of participants.

We know that going from 100 to 50 to 25 projects delivered per year isn’t a story we want to tell. Our team has worked harder each year. We’re building a team of more competent staff. The organization has provided greater funding each year. We need to rewrite this story.

Agile velocity

Velocity is a great tool to measure the performance of a team. A team’s velocity is the amount of work a team delivers over a fixed period. In Scrum, velocity is commonly measured in “points,” also known as story points. The points measure the effort for a given story, i.e. a specific piece of work.

Consider three factors when estimating points: complexity, effort, and doubt.

Using a scale of 1 to 5, 1 is the least complex and 5 is the most complex. What matters is the relative value of the points. We could have chosen 1,000 to 5,000 or 100,000 to 500,000. What matters is that a 2 is twice the value of a 1. Agile teams can have trouble with absolute values. The concept of triangulation helps ensure consistency. Periodically during the estimating process, compare a 1-point and a 2-point story. Discuss whether the 2-point story is twice as complicated.

The estimation fallacy

In his book, The Black Swan: The Impact of the Highly Improbable, Nassim Taleb describes several lessons that impact how we estimate:

  1. Epistemic arrogance: people claim they saw it coming, e.g., we estimate well in general except for ‘’
  2. Narrative fallacy: missing cause and effect, people bend stories to fit current understanding, e.g., this project should be like that one
  3. Asymmetry: an unequal upside and downside, e.g., missing by a factor of 2 is easy

Michael Bolton provides a great example of project estimation fallacies with Black Swan-like events. In his example, a project is estimated to take 100 hours with each task requiring 1 hour. He added in some basic assumptions:

  • 50 percent of tasks complete in 30 minutes, not 1 hour, i.e. the task is completed early
  • 35 percent of tasks complete on time
  • 15 percent of tasks experience bad luck.

Our 15 percent of “bad luck” tasks is broken out into the following assumptions:

  • Little slips: 8 tasks slip to 2 hours
  • Wasted mornings: 4 tasks slip to 4 hours
  • Lost days: 2 tasks (one in 50) slips to 8 hours
  • Black cygnets (baby black swans): 1 task slips to 16 hours, i.e. 1 in 100 tasks takes two days

Using this simple example, the probabilistically average project would come in 24 percent late. A project that was committed for Q4 would be delivered in Q1.

Introducing portfolio velocity

Our goal is to estimate the departmental capacity to deliver value. How do we measure this? Portfolio velocity.

Accuracy doesn’t matter, but consistency does. We can apply this new measure to estimate our departmental capacity. To calculate your portfolio velocity, perform the following steps:

  1. Select a project
  2. Estimate the complexity on a scale of 1 to 5, with 5 being the most complex
  3. Estimate the effort by estimating the number of months to complete the project ranging from 1 to 12
  4. Calculate the product by multiplying the complexity by the effort to determine the project velocity
  5. Sum the velocity for all projects to determine the portfolio velocity
  6. Calculate historical portfolio velocity for past years
  7. Determine reasonable growth rates on top of last year’s velocity
  8. Rank the projects in the portfolio 1 through n
  9. Draw an above and below the line based on your new forecasted velocity

By completing through step 5, you now have, in your hands, the forecasted demand. To determine if this velocity is reasonable, we need to conduct a brief exercise to discover the historical portfolio velocity by looking at prior years and conducting a similar exercise for projects from these years.

Let’s assume we determined that the velocity for Year One was 200 points, Year Two was 300 points, and Year Three is forecasted at 1200 points. Already our story is changing for the better. However, it’s clear our capacity isn’t going to increase four-fold.

Assuming moderate growth of 10 percent, we can forecast our Year Three velocity: 330 = 100 percent + (10 percent x 300). Using the ranked list, we can determine which projects will be delivered in Year Three and keep us at or slightly below our target portfolio velocity.

Estimation exploratory options

The 1-to-5 complexity scale works well. Alternatively, a base-2 sequence (0, 1, 2, 4, 8, 16, 32, 64, 128) or a Fibonacci-like sequence (0, 1, 2, 3, 5, 8, 13, 20, 40, 100) can help make the difference between projects more distinct.

However, humans are notoriously poor estimators, and I’ve found that teams more effectively estimate based on common orders of magnitude such as 1 to 5 or 1 to 10. It’s easier for a team to estimate the size of a mouse at 1 and an elephant at 5 versus a mouse at 1 and an elephant at 8 or a cargo ship at 64.

Measure what’s valuable

Too often, we get swept away by maintaining historical measures. Being accountable is good. Ensure you’re holding your team, department, and organization accountable for the right measures.

Make this year different—with hard commitments—by determining your portfolio velocity.

Previous articleDo you have a data strategy to achieve better organizational analytics?
Next articleFirst, define “good” to scale your organization
Peter is a technology executive with over 20 years of experience, dedicated to driving innovation, digital transformation, leadership, and data in business. He helps organizations connect strategy to execution to maximize company performance. He has been recognized for Digital Innovation by CIO 100, MIT Sloan, Computerworld, and the Project Management Institute. As Managing Director at OROCA Innovations, Peter leads the CXO advisory services practice, driving digital strategies. Peter was honored as an MIT Sloan CIO Leadership Award Finalist in 2015 and is a regular contributor to CIO.com on innovation. Peter has led businesses through complex changes, including the adoption of data-first approaches for portfolio management, lean six sigma for operational excellence, departmental transformations, process improvements, maximizing team performance, designing new IT operating models, digitizing platforms, leading large-scale mission-critical technology deployments, product management, agile methodologies, and building high-performance teams. As Chief Information Officer, Peter was responsible for Connecticut’s Health Insurance Exchange’s (HIX) industry-leading digital platform transforming consumerism and retail-oriented services for the health insurance industry. Peter championed the Connecticut marketplace digital implementation with a transformational cloud-based SaaS platform and mobile application recognized as a 2014 PMI Project of the Year Award finalist, CIO 100, and awards for best digital services, API, and platform. He also received a lifetime achievement award for leadership and digital transformation, honored as a 2016 Computerworld Premier 100 IT Leader. Peter is the author of Learning Intelligence: Expand Thinking. Absorb Alternative. Unlock Possibilities (2017), which Marshall Goldsmith, author of the New York Times No. 1 bestseller Triggers, calls "a must-read for any leader wanting to compete in the innovation-powered landscape of today." Peter also authored The Power of Blockchain for Healthcare: How Blockchain Will Ignite The Future of Healthcare (2017), the first book to explore the vast opportunities for blockchain to transform the patient experience. Peter has a B.S. in C.I.S from Bentley University and an MBA from Quinnipiac University, where he graduated Summa Cum Laude. He earned his PMP® in 2001 and is a certified Six Sigma Master Black Belt, Masters in Business Relationship Management (MBRM) and Certified Scrum Master. As a Commercial Rated Aviation Pilot and Master Scuba Diver, Peter understands first hand, how to anticipate change and lead boldly.