Agile: A History

Although the term “Agile” was only formally introduced in 2001, the underlying concepts powering the methodology have existed in various forms since the early 20th century. Beginning in the 1920s, pioneering statistician Walter Shewhart developed a set of techniques now known as the “plan-do-study-act” cycle, which he used to improve processes at the Bell Telephone Company. Shewhart’s techniques were championed in the 1950s by W. Edwards Deming, a mathematician and management consultant. In 1955, Deming gave a series of lectures in Japan in which he advocated the application of statistics to business processes, with the ultimate goal of improving product quality, eliminating waste, addressing bottlenecks and inconsistencies, as well as generally reducing the overall burden of work. Toyota and a range of other Japanese companies quickly adopted his techniques, which ultimately helped to fuel the explosive growth of the Japanese manufacturing industry in the postwar era.

In the 1980s, professors Hirotaka Takeuchi and Ikujiro Nonaka published a series of articles for the Harvard Business Review in which they analyzed the business practices of a number of successful companies, including Toyota, Honda, and Fuji-Xerox. Takeuchi and Nonaka attributed the success of these companies to a combination of speed and flexibility in product development. The business methodology at play found parallels in the game of rugby, which emphasizes teamwork and tactical flexibility in pursuit of victory. In 1995, software engineer Ken Schwaber and management theorist Jeff Sutherland created Scrum, a software development system named after an element of rugby play. Scrum presented an attempt to extract a workable methodology from Takeuchi and Nonaka’s analysis. Scrum has since become the most popular Agile methodology in the world, used to manage both technical and non-technical project work.

In 2001, 17 executives in the software development industry, including Schwaber and Sutherland, met at the Snowbird Ski Resort in Utah to discuss possible innovations in the software field. In particular, the group endeavored to address the limitations inherent in the highly linear, design-based “waterfall” methodology, which was the industry standard at the time. At the end of the retreat, the group released a document entitled Manifesto for Agile Software Development, marking the birth of the Agile methodology.

Dictionary: Waterfall Model. The waterfall model is one of the oldest approaches to software design. It is highly linear in nature, with progress flowing sequentially through a series of well-defined stages: conception, initiation, analysis, design, construction, testing, deployment, and maintenance.

The waterfall design is an example of a manufacturing-derived “traditional” PM strategy adapted to creative work. Early software designers, seeing the processes that their colleagues in hardware manufacturing used to develop their products, adopted these processes in their own field. The nature of manufacturing means that changes to project parameters can be debilitatingly expensive, necessitating a process model that attempts to anticipate all variables and is extremely resistant to change. Such a rigid framework, however, is not as well-suited to the fluid processes of creative enterprises like software development. Agile thinking embraces flexibility, allowing developers to adapt to changing design needs.

The Agile Manifesto outlines a series of four values essential to the Agile approach: (1) individuals and interactions over processes and tools; (2) working software over comprehensive documentation; (3) customer collaboration over contract negotiation; and (4) responding to change over following a plan.

The first value, “individuals and interactions over processes and tools,” outlines the ideal working conditions of an Agile team. In an Agile operation, people come first. The work environment is collaborative and supports creative thinking. Eschewing older working models based on top-down management and highly linear procedural workflows, Agile thinking favors face-to-face interaction and group problem-solving.

Expert Opinions: “Back in 2001, a group of people got together and they laid out a manifesto for better ways to build software. They called it the Manifesto for Agile Software Development. The first thing they wrote in their manifesto is that we value individuals and interactions more than we value processes and tools. Processes are good, and of course we need tools, but we value people and people interacting with each other more than anything else. It turns out that when people interact with each other, we get better products and our customers are happier and the build team is happier.”
— M&A Science Interview with Richard Kasperowski, Author, Consultant and Harvard Instructor

The second value, “working software over comprehensive documentation,” declares a preference for quickly fielding workable prototypes or models over completing a comprehensive design upfront. For many years, the standard working model in software design was known as “Big Design Up Front,” an approach mandating the completion of a comprehensively documented, fully complete design before coding can begin. This development process contributes to long project lead-times, making it difficult to modify code in response to a customer’s changing needs. By prioritizing working prototypes, the Agile approach puts software into the hands of customers quickly, allowing developers to incorporate end-user feedback into subsequent versions, ultimately ensuring that the product grows organically to meet unforeseen needs and achieve optimal functioning.

The third value, “customer collaboration over contract negotiation,” emphasizes direct and frequent contact with the customer, to ensure that the project adapts to their needs. In traditional negotiations, a software developer typically receives a detailed description of the project requirements from the customer and completes a software design accordingly. The Agile methodology values a more interactive approach, which assumes that designs will evolve over time to accommodate changing conditions.

The fourth value, “responding to change over following a plan,” encourages adaptivity and flexibility over adherence to a static plan. Traditionally, software developers pursued projects according to highly detailed, predetermined plans with limited flexibility. As a result, developers were inclined to perceive any changes as negative events to be avoided at any cost. By encouraging the development of a project through many iterations, the Agile approach embraces change more effectively.

Following the release of the Manifesto for Agile Software Development, the technology community quickly adopted the four tenets outlined above. The tremendous success of the Agile approach in the technology domain sparked prompt interest in other industries, including biotech, defense, financial services, and marketing. Leading companies in these industries began tailoring Agile processes to address their most pressing project management challenges.

How can one PM technique be so applicable and successful across such varied industries? It is actually pretty simple: Each of these industries face fluctuating priorities, rely heavily on knowledge management, and benefit greatly from effective collaboration. All of these conditions are incredibly well suited for Agile. Agile’s flexible nature also allows each industry to implement the specific values that best improve their greatest challenges, making Agile scalable and constructive.

‍Expert Opinions, on Transaction vs. Transformation:
“If you want to be a more future-oriented M&A professional, you need to take an Agile approach to things… So think transformation, not transaction… It's about leveraging the opportunities from a deal, but to do that you need to take a much more of a transformational mindset, as opposed to a transactional view…. When you're thinking about M&A, don't think about continuity, think about opportunity… Don't think about minimizing risk, think about living and breathing risks. Don't look at transaction value, look at transformation value.”
— Toby Tester, Senior Consultant at BTD

‍Agile also emphasizes constant improvement towards best practices. Any industry can truly benefit from this approach. An Agile team is never satisfied with a process just because it works, and there is no such thing as good enough. Every working cycle strives to incorporate lessons from completed tasks into future processes, resulting in continuous refinement. Even if a team achieves something resembling a “best practice” in one project, the goals and parameters of their next project will be different — so teams will execute, refine, and continue to improve in accordance with the principles of the Agile methodology.

The original AgileManifesto, penned at Snowbird, is a concise and effective plan for implementation of Agile across many diverse industries. The Manifesto presents a broad strategic vision. No highly detailed technical framework will be able to achieve the goal of continuous improvement in every situation. All of the proprietary, industry-specific Agile methodologies — SAFe, LeSS, DAD, Nexus, etc. — work best in the context for which they were designed. The high-level values of Agile thinking, however, are universal in their applicability, helping companies improve their process flow immensely.