BigBatchTwilightZone

In the previous post in the Twilight Zone series we highlighted the misconception that Agile is for software development only.  In this post and subsequent posts we will highlight how Batch Sizes are effected by many areas across an organization beyond development.  Put your chin strap on…this may get a little heavy!

Don Reinertsen’s book, Principles of Product Development Flow(YOU should read it!) lists Reducing Batch Size as one of the eight principles of product development flow.  Before we jump to how you can reduce batch size, lets take a look at the cause and effect of large batch sizes.  By the way…much of this content can be found in Reinertsen’s book.  Did I mention YOU should read it?

The Cause

Today most project management is focused on wringing out efficiencies by institutionalizing large batch sizes.  This leads organizations to functionally align into big batch silos – Requirements, Architecture, Development, Test, and Operations.  The thought is that we can gain efficiencies through economies of scale and uninterrupted work.  And guess what…its true!  There are local efficiencies gained by pushing large batches of work through functional silos.  But how does this effect the agility of the whole organization?

The Effect…let the games begin!

  1. Increased cycle time(i.e. slower delivery speed.)
    • Its gonna take longer to build more stuff – duh!
  2. Increase variability of flow
    • What’s that mean?  It means there will actually be more frequent and larger interruptions in work.
    • Think about it, every time a big batch of work enters a silo a flurry of activity must happen to service it.  When a big batch leaves a silo, activity subsides.  Hot, cold, hot, cold, …etc.
  3. Slower feedback
  4. More risk
    • The longer it takes, the more could change(requirements, customer, technology, …etc)
    • Big planes are harder to land than small planes  (btw…in the English language we call that an analogy 😉
    • Slower feedback increases the consequence of errors.  i.e. the later we learn about a problem, the more it costs to fix it.
  5. More overhead
    • More uncertainty in process means more status meetings so we can get updates on our uncertainty.  Making sense yet folks?
  6. Reduces efficiency
    • Wait I thought it gained efficiency?  Well…yes…locally!  However its at the expense of destroying important feedback loops that cause defects to grow exponentially as batches are pushed through each functional silo.
      • I know this never happens…but what if a redundant defect is found by Test or Ops?  Perhaps an assumption was wrong, the customer changed their mind, …etc.
    • Think of how this effects the efficiency of the whole organization!
  7. Lowers motivation and urgency
    • Are you more likely to feel an urgency to deliver on day one of a two week sprint or on day one of a 100 day milestone?  Me thinks two weeks. 🙂
    • Which scenario would you feel more motivated by responsibility?
      • If you are responsible for shepherding a capability from concept to consumption…oh lets say…something like a USER STORY.
      • You were just capturing the requirements for a capability, but you knew several other groups would share in the responsibility to deliver it. (Arch, Dev, Test, Ops, …etc)
  8. Increased project slippage
    • More uncertainty in process…means more uncertainty 😉
  9. Leads to Larger Batches
    • Often called Death Spiraldevelopment.  Here’s what the customer is thinking
      • “If I’m not getting a capability for 6 months or a year I’m giving you everything I got!  Prioritization…what’s that?  Everything’s a P1!”
  10. The entire batch is limited by its weakest link
    • If 1 out of the 30 requirements in the batch has a security relevant change then we will treat the whole batch as a security relevant change.
    • For those of us who have dealt with security relevant changes, we know this can be a huge pain and often leads to large delays.

Whoo!  That was a mouth full.  In the next post we’ll describe some practical steps you can take to reduce batch size so that you don’t have to deal with the stuff above.  And I promise we will outline specific areas of your organization that will be involved to Reduce Batch Size!

 

TWILIGHT ZONE

One of the preconceived notions that we run into while consulting is that Agile is only for software development teams.  As if Agile is some Twilight Zone that only applies to developers.  If we lived in the Twilight Zone, that might be true.  But we live in the real world, and we have to deliver value to our customers.  Agile practices and methods usually start cropping up on software development teams first, but quickly expose the lack of agility in other areas of the organization.  If you want to deliver software at scale with agility, you must consider how you do business outside of development!  

This is what Lean|Agile is all about.  Lean|Agile goes beyond development and addresses the organizational impediments that stand in the way of agility.  Our Lean|Agile service teaches and trains the essential practices needed to discover and deliver increments of value in a sustainable, predictable way. This means we must consider all stakeholders involved with the creation of value – Customers, Requirements, Architecture, Management, Development, Operations, …etc.

We’ve found that this common misunderstanding, has required us to do a lot more upfront training.  Essentially we need to baseline the thinking of our clients so that we are able to help our clients achieve agility at scale.  In the next week’s post we will begin to describe how one specific principle of Lean|Agile development, Reducing Batch Size, effects organizations outside development.  Stay tuned!

It Depends

November 25, 2012 — Leave a comment

“It depends.”  The unfortunate answer of some consultants to questions that they frankly don’t know the answer to.  I would be lying if I said I have never used those two words in front of a client before.  However, over the past few years I have made a personal pivot away from this oft used response.

In my experience I’ve found that clients want answers…NOW!!!  I think this has led many of us in the consulting community to generate half baked answers when put on the spot.  Thus, the famous “it depends” response was born.

I’ll propose something different.  Just be honest and provide principled guidance.  Personally, I use the Scaled Agile Framework’s House of Lean and Agile Manifesto as the foundation of my thinking.  Both provide a mature set of principles that complement one another very well.  The key word there is principles.  When we get pressed for answers we don’t know, we can be simply honest and say, “I don’t have a great answer for you.  However, lets take a look at how that suggestion would align with our set of Lean|Agile principles.”

I would then suggest we guide our clients through some principles in the context of the their possible decisions.  For instance, a few weeks ago I was asked to help define an engineering plan for a continuous delivery architecture.  This is not a trivial question and it certainly does not have a trivial answer.  On top of that, deployment pipelines are context specific to your system.  Since I did not have a deep understanding of my client’s system, my response back was largely based around my knowledge of continuous delivery systems and Lean principles.

  • I asked questions like:
    • How do tooling decisions effect feedback speed.
    • What is the “u-curve” optimization for manual test feedback cadence within the pipeline?
    • How is management going to encourage and support the whole team to develop a “push” mentality and a collective ownership culture?

By answering some of these questions we were able to design a high level architecture that allowed for the delivery of small grained changes to my client’s system with incredibly fast feedback.  I found this a much more satisfying way of answering questions I didn’t have the answer for.  Not to mention my client and I were able to learn together!

How many times has your team spent time futzing with build scripts and managing test environments or monitoring systems? This is a reality for development shops today. While this activity is absolutely essential to maintain stable assets, it does not scale well. As large organizations continue to demand more software solutions, we must think of different ways to manage, build, and deliver our assets. 20 different teams tinkering with Jenkins could get ugly fast. Not to mention the backlash effect of instituting a Jenkins change control board(JCCB)! 😉

What if scaled application developers didn’t have to worry with this headache? What if they just plugged their code and tests in and voilà…it works? 😀

Well, drum roll…ladies and gentlemen, I introduce to you Delivery as a Service(DaaS). DaaS represents the ops interface that developers interact with. This allows developers to remove themselves from the mundane tasks associated with building and maintaining a delivery architecture. Modern tooling, Lean Agile thinking, and mature engineering practices have made this possible.

DaaS is made possible by the thinking behind Continuous Delivery. Jez Humble and David Farley wrote a fantastic book in 2010, called Continuous Delivery, that outlined how to build more reliable systems by potentially delivering every application change to production fast, really fast. The success and maturation of Continuous Delivery systems is changing the mindset of traditional development. Companies such as Etsy, Netflix, and Flickr are pioneering the engineering practices required to build and deploy at scale.

With the help of these innovators and the work of the Scaled Agile community, we’re making sense of developing and delivering at scale. The Scaled Agile Framework calls out a program level team that would provide DaaS called the System Team. We think that a lot of innovation will occur at this level within SAFe over the next two years. If anyone has any experience with Continuous Delivery/Deployment at scale, we’d love to collaborate.

It’s an exciting time to be pushing code!

With the election coming up on Tuesday, we thought we’d have some fun with it.  We often get asked how we use Lean to drive our Agile coaching.  In this video we have outlined a few high level principles that guide our thinking.  The full answer will be addressed in a future blog.

Enjoy this ALL-STAR cast as they show off their daily standup skills!