A Behind the Scenes Look at the Writing of the Agile Manifesto

A Behind the Scenes Look at the Writing of the Agile Manifesto

It’s February 2021 and we are celebrating 20 years since the writing of the Agile Manifesto. Here’s a behind the scenes look of what happened back in Snowbird 2001 when 17 software development thought leaders authored the Agile Manifesto.

1990s – Before the Agile Manifesto

Let’s first go back another 10 years to explore what was happening in the 1990s. In the late 80s and early 90s, Kent Beck, the creator of XP or extreme programming, worked with Ward Cunningham, the creator of the first wiki, on better approaches to developing software. Kent got to refine and evolve a lot of these practices along with Ron Jeffries and later Martin Fowler while working on the C3 project at Chrysler and it’s at that time that the group raised awareness of these practice via the wiki as well as Kent’s first XP book. James Grenning also got exposed to XP in the late 90s and took a particular interest in applying the XP practices to embedded devices. We will refer to this group of Agile Manifesto authors as the XP contingent.

Around the same time, on the east coast in Boston, Jeff Sutherland was experimenting with different approaches to managing projects based on lean thinking and ideas from the New New Product Development Game by Takeuchi and Nonaka. Ken Schwaber was coming up with something similar based on complex process control theory and his work at DuPont Advanced Research facility. Ken and Jeff collaborated to create Scrum and introduced it at an OOPSLA conference in 1995. Mike Beedle was probably one of the first scrum practitioners that helped them validate and evolve the framework. We will refer to this group of Agile Manifesto authors as the Scrum contingent.

Jon Kern was working with Peter Coad on another approach called Feature Driven Development (FDD), Arie Van Benneken was in Europe working with the DSDM consortium on yet another approach called Dynamic System Development Method.

Jim Highsmith was working on another approach called Adaptive Software Development and Alistair Cockburn was doing research with IBM and developing another approach called Crystal.

All these approaches at that time before the writing of the Agile Manifesto were referred to as lightweight methods or lightweight processes.

2000s – Leading up to the Agile Manifesto

In 2000, Kent Beck was having an XP leadership retreat in rogue river valley in Oregon. Robert Martin (“Uncle Bob”), was also at that meeting. Uncle Bob was another Object Oriented Programming (OOP) expert and had met Kent and became an XP fan and advocate. By the way, most of these folks came from the OO community. In the late 90s there was a move going from procedural to Object Oriented languages, and these guys were the experts in the field and in particular the small talk programing language.

At that meeting, there was talk as to whether XP should be larger than the technical XP practices. XP was very specific on technical practices to ensure technical excellence like pairing, test driven development, continuous integration, automated acceptance tests. So the discussion was around whether it should be larger than that and more encompassing. There wasn’t much traction on this idea at that meeting, however, a couple of weeks, Martin Fowler, who was also at the meeting met up with uncle Bob to discuss the idea further and they agreed that it had merit and probably needed a separate event that included a broader group than just the XP folks.

Uncle Bob reached out to Alistair Cockburn. Alistair had organized events in the past at Snowbird and on cruise ships in Norway and was about to organize another one. However, Alistair liked Uncle Bob’s idea and they decided to merge mailing lists.

Invites went out to also include Andy Hunt and Dave Thomas who did not have a methodology per say but had written the Pragmatic Programmer article which later became a book and later a book series. It was more about culture, attitude, behavior and professionalism of developers and the ideas in it matched with most of the practices and approaches described in the lightweight methods.

Brian Marick was also invited to bring in a testing perspective. Brian was doing research on ways of improving testing and building quality in instead of leaving testing to the very end. Stephen Mellor was also invited. Stephen was working on UML modeling and auto-generating code from UML. He was known as the process guy and considered himself a “spy”.

Picking a location was next. It boiled down to Anguilla in the Caribbean or Snowbird. “Big” Dave Thomas was another OO expert who had a place in Anguilla. Alistair Cockburn and Jim Highsmith were living in Salt Lake City and offered to organize. In the end Salt Lake City won mainly because of easier access via direct flights through the SLC airport. Not all invitees ended up making it (like “Big” Dave Thomas), but the 17 mentioned above did.

2001 – The Writing of the Agile Manifesto

The meeting was called the lightweight methods conference and was held in the Aspen room in the Lodge at Snowbird. Participants arrived on Sunday, February 11, 2001 and departed on Wednesday February 14, 2001. The agenda included a morning session, lunch, afternoon skiing or hot tub, evening session, and dinner.

Going into the meeting, folks added some thoughts to the Wiki. Alistair had printed out a few pages from the wiki and shared those with the group. Here are Jon Kern’s notes:

What was the group looking to get out of the lightweight summit? Consolidation of principles, paper, articles and book collaboration, conference, newsletter, consortium.

Here’s Uncle Bob’s initial thoughts: “I want two things done.

1. Setup a website that people can go to learn about LWPs.

2. Create a manifesto that the group signs, describing the fundamental tenets of LWPs and recommending them to the industry at large.”

Alistair Cockburn’s thoughts going into the meeting: “I am interested in fending off the fat methodology armory, the vast quantity of RUP, Anderson, SEI, etc. salespeople putting ideas into CIOs’ minds that they should have lots of paperwork to be “safe”, countering the “I want turnkey methodology, one that work straight out of the box.”

Andy Hunt’s thoughts: “…Not only are CIOs that they need lots of paperwork to be “safe”, but they need lots of bodies as well…there needs to be a major effort put forth the ideas that “less is more” across the board in software development.”

Day 1 of the Writing of the Agile Manifesto

The first night was an informal meet and greet as not all participants knew each other.

The next Day, as the meeting was getting started, the XP folks pulled out some index cards and distributed them to the group. Participants wrote down what they wanted to get out of the conference, the cards were collected, and the group did some affinity mapping to come up with topics to discuss over the next 2 days.

The different contingents started out by describing their approaches: XP, Scrum, Crystal, DSDM, ASD, etc… Below are Jon Kern’s notes:

During the afternoon break, some went skiing, I believe that included at least Alistair, Jim, Mike, Jon, James and Martin.

On the evening agenda was coming up with a new name. The group did not want to be known as lightweights and did not like the name lightweight methods or lightweight processes, so they wanted to come up with a new name. They brainstormed many options including lean, responsive, adaptive, and agile. In the end, they went with Agile. Adaptive was probably the runner up but was eliminated because Jim Highsmith already had Adaptive Software Development.

Day 2 of the Writing of the Agile Manifesto

Lots of great discussions happened over the 2 day event however, late into day 2, nothing actionable had come out. The team was kind of in analysis paralysis until Dave Thomas and Martin Fowler tried to summarize the discussions so far based on the common ground between the different methods.

Throughout there was a desire to not be negative or against something but for something else, for something better.

Dave and Martin wrote down 3 values. The group then gathered and discussed further. The values grew to five and then they settled on four values.

They tweaked these a lot until all were on board with the final statements. Below are some of Andy Hunt’s notes where you can see how the order changed over and over.

Another tweak was changing pointless documentation to comprehensive documentation to remain positive and to change code to software to make it a bit broader.

The Agile Manifesto

It is important to highlight that the first part says “We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: “

Also note that the last sentence says “That is, while there is value in the items on the right, we value the items on the left more.”

The Principles Behind the Agile Manifesto

The meeting ended with just the value statement. The team did not have enough time to refine the principles. Here are Andy Hunt’s notes showing the early formation of the principles.

Over the next month or so, the team finished up the principles over email. The final version of the principles had a lot of wiggle room and were not as refined as the values. They contain some vague language and the one principle that likely had the most discussion was principles 3:

“Deliver working software frequently, from a couple of week to a couple of months, with a preference for the shorter timescale.”

Some felt that you had to delivery within a couple of weeks, others felt that there was no way one could do that. Interesting that 20 years late, we still have these same discussions with teams just starting with their Agile adoption.

Also note how the principles were targeted to customers, managers, and the team.

The group met a couple of months later at OOPSLA and discussed if they should own this movement and decided that they are just the instigators or the catalysts and they established a non profit, the Agile Alliance, to open it up and get the community to continue promoting their ideas.

Ward Cunningham created a website posted the the Manifesto and gave page visitors the ability to sign it. Things took off from there. And the rest is history as they say.

20 Years Later

So 20 years later, what would the authors of the Agile Manifesto change and when will it be updated?

The authors feel that the Manifesto is a snapshot of their thinking at that time so updating it is not on the table.

Looking back, here are things they wish they did differently.

Diversity

Every one that attended up at the writing of the Agile Manifesto was a middle aged white man. The group feels that they would have benefited from diversity. The invite list did include at least 1 women but none showed up. Some wish that Mary Poppendieck, an expert in Lean Software Development, was there but the group did not know her at the time. Mary did end up playing a big role right after the creation of the Agile Alliance.

The Name Agile

Some Agile Manifesto authors feel that in hindsight they should have come up with a different name. “Agile” is too desirable, and so everyone wants to be Agile and everyone claims they are Agile even though there is nothing “Agile” about what they are doing. Agile has been watered-down and the meaning behind “Agile” today is not what the authors intended when the coined the term. Maybe a less desirable word would have been better but at that time XP or “extreme” scared people.

Dogma

The Agile Manifesto is a statement of values and principles. It’s about a way of thinking, an approach or a mindset. It is something to build on and not a codification of principles. Many take as doctrine, gospel or rules you have to follow and forget the uncovering part, the learning part, and the adjusting part.

Process and Tools

The Agile Manifesto authors want to remind as that the 1st Agile value is about People and Interactions over Processes and Tools. Many immediately jump into tooling. We are moving to Agile. What tool should we use? Or we are using Jira, therefore we are Agile.

There is even more focus on the processes and some authors particularly call out SAFe as very heavy on process and looking more and more like RUP which is what started this movement to begin with (See Alistair’s comments above). So it’s kind of like we’ve come full circle.

Software

Some Agile Manifesto authors feel that Agile has moved beyond just Software and maybe they should have gone with Working Product or Working Solution instead of Working Software

Technical Practices

Other authors of the Agile Manifesto regret not including more technical practices as the focus on “working” has been forgotten. Many teams build and build without delivering working software frequently. This cannot happen with out focus on technical practices like pairing, test driven development, continuous integration, automated acceptance tests, etc. 20 years later, many teams still do not apply any of the original XP practices that started this movement.

Collaboration

The Agile movement was about brining the business and development closer together. It was about mending the business/IT relationship and making it less adversarial, breaking the silo and collaborating with the customer. It was about giving the development team autonomy, mastery and purpose, empowering them to think and be creative and innovative, and have them think of the business value they are delivering and not just the function or method they are coding. It was mostly created by developers to provide them with a better working environment that delivered better value.

This succeeded in the early 2000s, but Agile today has been taken over by management and is used to manage developers instead of empower them. As a result, developers are not interested in Agile and some dislike it due to how it’s being used. There needs to be a renewed focus on the first value of Individuals and Interactions.

References

Check out these recaps from the authors of the Agile Manifesto along with Jon Kern’s and Andy Hunt’s notes and the pictures posted by Alistair Cockburn.

Article by the Atlantic

Jim Highsmith’s recap

Dave Thoma’s Recap

Martin Fowler’s recap

Robert Martin (“Uncle Bob”)’s recap

Martin Fowler’s blog post

Agile Uprising Manifesto Review

Jon Kern’s Notes

Andy Hunt’s Notes

Alistair Cockburn’s Picture