A Brief History of Time

In order to understand DevOps, we have to travel back in time to the old age when there was nothing but programmers.

As the Tao teaches us:

The programmers of old were mysterious and profound. We cannot fathom their thoughts, so all we do is describe their appearance:

  • Aware, like a fox crossing the water
  • Alert, like a general on the battlefield
  • Kind, like a hostess greeting her guests
  • Simple, like uncarved blocks of wood
  • Opaque, like black pools in darkened caves

The Programmer gave birth to machine language. Machine language gave birth to the assembler. The assembler gave birth to the compiler. Now there are thousands of languages. Each language has its purpose, however humble. Each language expresses the Yin and Yang of software. Each language has its place within the software.

At the time, software development offices were mostly called labs, and programmers were scientists. In order to create a good application, programmers had to fully understand the problem the application was solving. They had to know where the application will be used and what kind of system will be running it. In essence, programmers knew everything about their application from specification, through development, to deployment and support.

And then human nature kicked in, and we started to ask for more. More speed, more features, more users, more everything.

Being modest, humble, and peaceful creatures, programmers had very little chance to survive such burst of needy users always asking for more. The best chance to prevail was to start evolving in different directions, and create castes. Soon, programmers were extinct as forefathers of new breeds called developers, software engineers, network administrators, database developers, web developers, system architects, QA engineers, and many more. Evolving fast and adapting to challenges from the outside world became part of their DNA. New caste could evolve in a matter of weeks. Web developers became back-end developers, front-end developers, PHP developers, Ruby developers, Angular developers … it was all going to hell.

Soon they all forgot that they came from the same father, a Programmer. A simple and peaceful scientist that just wanted to make the world a better place. They started fighting each other, claiming that each of them is true descendant of “The Programmer” and that their blood is more pure than others.

As time passed by, they reduced their interaction to a minimum and spoke to each other only when they really had to. They stopped celebrating success of their distant family members, and eventually even stopped sending a postcard every once in awhile.

If they only searched their feelings, they would discover that the spark of the Programmer was in their hearts, waiting to shine and bring peace to the galaxy.

The Programmer

In their selfish and egocentric race to conquer the world, programmer’s descendants forgot the very purpose of their job - solving problems for their clients. Clients started to feel the pain of such behavior as projects got delayed, too expensive, or even failed.

Every once in awhile, a bright star would shine and someone would get an inspiration to try and make peace among The Descendants. They came up with Waterfall. This was a brilliant idea, as it utilized the fact that different groups of developers communicated only when necessary. When one group finished their part of the job, it communicated with the next group to send work down through the process and never looked back at it.

Waterfall

This worked for a while, but as usual, humans (Clients) wanted more again. They wanted to take more active part in the whole process of creating software, give their opinion more often, and ask for changes even when it was too late.

Software projects became so prone to failure that it was accepted as an industry standard. Statistics showed that over 50% of projects were failing, and it seems like there was nothing to be done about it (ZDNet, CNet)

Every generation had a few open-minded individuals who knew that all these groups of developers have to find a way to work together, communicate, and be flexible to assure their clients will receive the best possible solution. There are traces of these attempts even as early as 1957, by colleagues of the great John Von Neumann. However we had to wait until early 2001 to start the revolution, when a dirty dozen created an Agile Manifesto.

The Agile Manifesto is based on twelve principles:

  1. Customer satisfaction by early and continuous delivery of valuable software
  2. Welcome changing requirements, even in late development
  3. Working software is delivered frequently (weeks rather than months)
  4. Close, daily cooperation between business people and developers
  5. Projects are built around motivated individuals, who should be trusted
  6. Face-to-face conversation is the best form of communication (co-location)
  7. Working software is the principal measure of progress
  8. Sustainable development, able to maintain a constant pace
  9. Continuous attention to technical excellence and good design
  10. Simplicity—the art of maximizing the amount of work not done—is essential
  11. Self-organizing teams
  12. Regular adaptation to changing circumstance

Agile manifesto was the first big step in bringing peace to the Galaxy and restoring balance to the Force. For the first time in a very long time, connecting all stakeholders in software development process was based on the cultural and “human” way, as much as on procedural and mechanized manner. People started to talk to each other, meet on a regular basis, and exchange ideas and comments all the time. They realized that they have much more in common that they thought, and clients became part of the team, not just some external factor that was expected to send the money and ask no questions.

Agile

There were still a few hurdles to overcome, but the future seemed brighter than ever. Being agile means being open and willing to adapt to constant changes. However, with all too many changes, it is difficult to stay focused on the ultimate goal and delivery. And that is how Lean Software Development came to life.

Hooked on LSD (pun intended) and risking to be exiled and outcast, some of the descendants looked for solutions outside of their caste and the software industry. They found salvation in the works of a major car manufacturer. Toyota Production System was amazing in its simplicity and it was so obvious that its lean manufacturing can be easily applied to software development.

There are 7 principles of lean:

  1. Eliminate Waste
  2. Build Quality In
  3. Create Knowledge (Amplify learning)
  4. Defer Commitment (Decide as late as possible)
  5. Deliver as fast as possible
  6. Respect People (Empower the team)
  7. Optimize the Whole

Added on top of Agile, Lean principles supported the mentality and the focus on doing the right things, while being flexible during the whole process.

Once Agile and Lean were adopted by software development teams, it took just one more step to apply the whole set of principles to IT as a Whole - which finally brought us to DevOps!

Enter DevOps - Three lane highway

The old-school view of software development teams included business analysts, system architects, front-end developers, back-end developers, testers, and so on. Optimizing software development process, including agile and lean principles, was mostly focused on these. Once the application was “production ready”, it was sent over to “Operations” including systems engineers, release engineers, DBAs, network engineers, security professionals, etc. Removing the great wall between Dev and Ops is the main driving force of DevOps.

DevOps is the result of implementing lean principles to the whole IT value stream. IT value stream extends development into production, combining all ‘distant relatives’ that descended from the original Programmer.

The best explanation of DevOps is given by Gene Kim, and If you haven’t read The Phoenix Project I suggest you take some time and do it.

You should not hire a DevOps engineer and DevOps should not be a new department in your IT. DevOps is a culture, mindset, and it is part of IT as the Whole. There are no tools that will make your IT a DevOps organization, and no level of automation will empower your teams to deliver maximum value to your clients.

DevOps is usually referred as method of three ways, but I see them as three lanes of the same highway. You start in lane one, then you speed up and switch to lane two, and eventually you are speeding by in the third lane.

  • Lane one - Performance of the system as whole is main focal point and it is emphasized over performance of any individual element of the system

  • Lane two - Make sure that there is a constant feedback loop sent upstream, and not ignored

  • Lane three - Nurture experiments, constant improvement, and fail fast

Lane one - Getting up to speed

Understanding the importance of the whole system, and prioritizing work items properly is the first thing an IT organization has to learn when adopting DevOps principles. No one in the IT value stream is allowed to create a bottleneck and reduce the flow of work.

DevOps - Understanding the whole system

Assuring uninterrupted workflow is the ultimate goal for everyone included in the process. Regardless of the role a person or a team has, it is imperative that they seek to achieve profound understanding of the system. Adopting this way of thinking has a direct impact on quality, as defects are never sent down the stream because they would cause bottlenecks.

Making sure that work is not stalling is not enough. A productive organization should always seek to increase the flow. There are numerous methodologies to increase the flow. You can find a solution in Theory Of Constraints, Six Sigma, Lean, or Toyota Production System. Feel free to pick any of these, come up with your own, or combine them.

DevOps principles do not care what team you belong to, if you are a System Architect, DBA, QA, or network administrator. The same rules apply to everyone, and everyone is expected to follow two simple principles:

  • Keep the system flow uninterrupted
  • Increase and optimize workflow at all times

Lane two - Gear up

Uninterrupted system flow is one-directional, and it is expected to go from development to operations. In an ideal world this means that software is created quickly with high quality, deployed to production, and delivering value to clients.

However, DevOps is not Utopia, and if one-directional delivery was possible the waterfall method would be sufficient. Evaluating deliverables and communicating up the flow is very important to assure quality. First “upstream” communication channel that has to be implemented is Ops to Dev.

Feedback

Giving a high-five to yourself is easy, but asking for feedback and giving feedback is the way to see the real picture. It is imperative that each small step downstream is followed by an instant upstream confirmation.

It does not matter how you establish the feedback loop. You can invite developers to join support team meetings, or bring in the network admin to weekly sprint planning. As long as your feedback is in place, and comments are picked-up and handled, your organization is speeding down the DevOps highway.

Lane three - Warp 10

DevOps fast lane is not for weak minded. To get on the DevOps fast lane your organization must be mature enough. It is all about taking risks and learning from failure, continually experimenting, and accepting that repetition and practice is the prerequisite to mastery. Quite often you will hear the term Kata, and this is for a reason. Continuous training and repetition leads to mastery because it makes complex moves become a routine.

But before you start combining moves, it is imperative that you master each individual step first.

“What is appropriate for the Master is not appropriate for the novice. You must understand Tao before transcending structure.”

Kata

DevOps Third Way/Fast Lane include allocating time for continuous experimenting on a daily basis, constantly rewarding the team for taking risks, and introducing faults into the system to increase resilience.

In order to assure that your organization will not implode when implementing these kinds of measures, you must create frequent feedback loops between all teams, while making sure that all bottlenecks are clear and system flow is uninterrupted.

Implementing these measures makes your organization alert at all times and capable of responding to challenges quickly and effectively.

Summary - DevOps organization checklist

Here is a checklist that you can use to validate how DevOps enabled your IT organization is. Please, feel free to comment on the article and add your own points.

  • There is no wall between development team and operations team. Both are part of the same process
  • Work that is sent over from one team to another is always verified and top quality
  • There is no “piling” of work, and all bottlenecks are handled
  • Development team is not using Operations time for its activities, because deployment and maintenance are part of the same time box
  • Development team does not deliver the code for deployment at 5PM on Fridays, leaving operations to clean up their work over the weekend
  • Development environments are standardized, and operations can easily reproduce and scale them into production
  • Development team can deliver new versions as they find appropriate and operations can easily deploy them into production
  • There are clear communication lines between all teams
  • All team members have time to experiment and work on improvement of the system
  • Defects are introduced (or simulated) and handled in the system on a regular basis. Lessons learned from each experiment are documented and shared with all relevant people. Incident handling is a routine and mostly handled in a known way

Conclusion

Using modern DevOps Tools like Chef, Docker, Ansible, Packer, Troposphere, Consul, Jenkins, SonarQube, AWS, etc. does not mean that you are applying DevOps principles. DevOps is a way of thinking. We are all part of the same process, we share the same time and deliver value together. Every person that takes part in the software delivery process can speed up or slow down the entire system. A bug created during development can bring down the system, as well as the wrong firewall configuration.

All of the people are part of DevOps, and once your organization understands that, tools and technology stack that will help you manage it will magically show up.

About the author

Demir Selmanovic, Bosnia and Herzegovina
member since May 23, 2014
Demir is a developer and project manager with over 15 years of professional experience in a wide range of software development roles. He excels as a solo developer, team member, team leader, or manager of multiple distributed teams. He works closely with clients to define ideas and deliver products. [click to continue...]
Hiring? Meet the Top 10 Freelance DevOps Engineers for Hire in December 2016

Comments

Ricardo Momm
Excellent article, congrats!
Stan Banes
As the child of one of the original programmers, father was teaching computer science in early 50's, I really appreciate this article. It has always been my feeling that there needs to be very close interaction between developers and users. For one thing, the more the programmer understands the user, and their problems, the better, and more applicable, the code. While I have developed my specialities, I always feel that the programmer must walk the paths trod by the users, no matter how humble. It needs to be that we apply our arcane arts to make it possible for the ultimate user to have a better life.
Ben Harberts
White, all Cis-Male washing of your imagery. I wonder can I only hire white people through your worldwide service?
Alex Urevick-Ackelsberg
A friend sent me this because it reminded him the agile dev workflows we've opened up for ourselves (and increasingly others) of our new CI/Testing/approval tool Probo.ci . We're in a private beta, but if you're interested and ping me I'd be happy to send you a code so you can try it out.
Филидор Зелёный
Using modern DevOps Tools like Chef, Docker, Ansible, Packer, Troposphere, Consul, Jenkins, SonarQube, AWS, etc. does not mean that you are applying DevOps principles. Golden words
Rachel Guerin
Educational to say the least -
sisustusliikekuisti
Very good writeup especially for us who are on the outskirts of sw development. It makes it easier to understand today's process. Thanks.
purple
Very enlightening article especially for some of us where we are. If only they implemented such a things in our country
chollie
A nice high level overview for the layman
Vanessa
This article is great. Star Wars, the Tao and serious snark? You are my internet hero of the day.
Demir Selmanovic
Hero for a day is better than a Prole (1984) for a lifetime :)
Demir Selmanovic
I don't think that DevOps principles are limited by traditional borders and countries. It is a matter of making decision and starting from internal organization. You could be the first to change your own team, and start delivering more to your clients than your competitors.
Aamir
An awesome article, I never saw such good way of DevOps explaination till yet.
comments powered by Disqus
Subscribe
The #1 Blog for Engineers
Get the latest content first.
No spam. Just great engineering and design posts.
The #1 Blog for Engineers
Get the latest content first.
Thank you for subscribing!
You can edit your subscription preferences here.
Trending articles
Relevant technologies
About the author
Demir Selmanovic
JavaScript Developer
Demir is a developer and project manager with over 15 years of professional experience in a wide range of software development roles. He excels as a solo developer, team member, team leader, or manager of multiple distributed teams. He works closely with clients to define ideas and deliver products.