I believe all Agile principles are equal in importance and they work together holistically: focus on customers, iterative development, self-organised teams, technical excellence, and continuous improvement. But this last principle is “more equal than the others“:
Continuous improvement is the heart of Agile. Without reflection, there is no learning, without learning there is no continuous improvement, and no agile transformation can happen if teams don’t improve on their work and processes.
Unfortunately, the retrospective is also the most underestimated event and the one to be dropped by the teams that feel “they spend too much time in meetings“.
One of my teams decided they don’t need retrospectives, as they communicate plenty throughout the day and improve things on the way. I stepped aside and allowed the team to experiment. A few weeks after, three different members of the team, plus their manager (who was the one to decide the retrospectives take up too much of their time) asked me to run regular retrospectives, as the team was dealing with issues on all fronts (communication, collaboration, lack of clarity about what is valuable, no understanding of backlog prioritisation, to name a few).
There is no doubt that retrospectives are the key of a successful agile transformation, but they need to be used effectively and be properly facilitated.
In this post I will look into why we need retrospectives (in agile environments but also any other organization and framework), what makes a retrospective successful and a few example of unsuccessful retrospectives, or “retrospective smells“.
A 2014 Harvard Business School study confirmed that we do not learn from experience, we learn from reflecting on experience. The finding came during a time when experience and practical application of skills were more valued than any other type of learning (on an individual and organisational level).
“We argue that once an individual has accumulated experience with a task, the benefit of accumulating additional experience is inferior to the benefit of deliberately articulating and codifying the previously accumulated experience.”From: Learning by Thinking: How Reflection Improves Performance, by Giada Di Stefano, Francesca Gino, Gary Pisano and Bradley Staats, HBS, 2014
The study confirms assumptions made by companies like Toyota, that made continuous improvement (reflect and improve) part of their company philosophy, while most agile frameworks have some form of continuous improvement events (e.g. Retrospectives in Scrum, kaizen events in Lean).
“Without retrospectives you will find that the team keeps making the same mistakes over and over again”.Henrick Kniberg, Scrum and XP from the Trenches, InfoQ, Toronto, 2007
Norm Kerth, also known as the father of retrospectives, created The Law of Wisdom Acquisition: “It is much easier to identify another’s foolishness than to recognize one’s own.“, he mentioned in his book Project Retrospectives: A Handbook for Team Reviews. Since it’s not natural to stop working and reflect on how you can improve, this process needs to be formalised and transformed into a ritual.
Here’s where retrospectives come in.
“Retrospective rituals are more than just a review of the past. They also provide a chance to look forward, to plot the next project, and to plan explicitly what will be approached differently next time.“Kerth, Norman. Project Retrospectives: A Handbook for Team Reviews
There is one primordial condition for a successful retrospective: it has to be safe. Safe implies that the participants can speak openly and honestly about work they (or others) did, proactively lookings for ways to do it better. That translates also into consciously looking for issues or problems to solve, which is not an easy process (I still cringe when I read a past blogpost and see errors or mistakes in, no matter if I wrote them yesterday or last year).
The responsibility for safety belongs to everyone in the room but the facilitator has to initiate, monitor and control safety. She needs to create a safe space, where people trust that speaking up will not bring them retribution. Norm Kerth came up with the Kerth’s Prime Directive for retrospectives:
Kerth’s Prime Directive
Regardless of what we discover, we must understand and truly believe that everyone did the best job her or she could, given what was known at the time, his or her skills and abilities, the resources available, and the situation at hand.Kerth, Norman. Project Retrospectives: A Handbook for Team Reviews
The Prime Directive is a great way to start the retrospective and have a discussion about how the retrospective should proceed.
I would add to that another rule: “what is discussed in the retrospective, stays in the retrospective“, to help create a space of safety and honesty. The experiments (actions) the teams decides to work on will probably be shared with other (as they might be added to the backlog), but otherwise the discussion is meant for the team and team only.
The anatomy of a retrospective
If Norm Kerth is the father of retrospectives, Esther Derby and Diana Larsen are the mothers. Their book, Agile Retrospectives. Making Good Teams Great, is considered a manual for retrospectives, teaching how to run a successful retrospective and giving plenty of ideas of retrospective activities.
Derby and Larsen present the structure of a regular retrospective (1 hour, most successful retrospectives I run are of around 1.5 hours). Here’s how an 1.h hours retrospective should be structured to achieve its goal of continuous improvement:
1. Set the stage
This is probably the most overlooked stage of the retrospective, as it’s considered too “fluffy” in a software development environment. The goal is to set the stage for the conversation and focus everyone in the room on the work at hand.
- As a facilitator, thank everyone for their being there, state the purpose of the session, and remind everyone how long the meeting will last.
- Have every single person speak. If a person says something (anything) in the first 5 minutes of the retrospective, they are more likely to speak up during the session.
- Next outline the approach for the session; I ask the team how they would like to proceed with the sessions, giving them some options or activities. When I am aware of existing issues within the team, I propose a specific activity to address those.
- If the team has team rules, team values or working agreements, bring them up (e.g. don’t speak over each other, no phones or laptops, no blame-session, etc.).
- If there are pending improvement actions from previous retrospectives, review those and their status (ideally they were completed).
- Duration: 5% (5 min).
2. Gather data
The purpose of the second part of the retrospective is to make sure that everyone has a shared understanding on how the sprint went.
I ran a retrospective where the entire team discussed how great the sprint went, how much work they had done, and how excited they are for what’s coming next. I noticed that one of our colleagues was quiet and din’t join the conversation, so I asked: “How were things for you this iteration, Jenny?”. She let out a heavy sigh and said: “This was probably the worst sprint I had since I joined this company“. This silenced the room and everyone turned to listen. Jenny had just joined the team, it was her first sprint, and wasn’t up to speed with the communication style, fast feedback loops, the stack, and everything else that the team had been building in over a year of working together. As an outsider, everything was very fast and the team didn’t notice her struggling, in their haste to deliver the sprint goal.
This is the session when you make sure that everyone has the same information on the sprint. No matter how tight teams are, each person has their own approach and struggles to deal with and it’s essential to bring them out.
Data can be a conversation, a timeline of the sprint, task boards, charts, etc. Don’t look only at hard facts, feelings and emotions are also indication of systemic issues, so consider them as well (e.g. a timeline that includes feeling charts, smiley faces, colored post-its that indicate feelings, etc.).
Duration: 30-50% (20-40 min).
3. Generate insights
The next step in a retrospective is to ask “Why?” and discuss ways about what to do differently. Considering the data (strengths and issues as well), the team generates insights to improve how they work.
The risk here is that the team focuses too low level, on symptoms rather that cause of issues, so as a facilitator you should be helping them determine patterns, conditions, interactions that contributed to their success or were in the way of their success.
Generating insights allows the team to take a step back, see the big picture, see the systemic issues and look for the root causes.
Duration: 20-30% (15-20 min).
4. Decide actions
By now, the team has a list of improvements and experiment ideas that would possibly improve their work. Now we need to take these items, prioritise them and decide on the top actions to proceed with during the following sprint(s).
Pick maximum 1-2 experiments or initiatives to work on in the following iteration. Too many things to do means nothing done. Make sure that these actions are being done, and are not just thrown on a board and forgotten about.
A suggestion here is to have an improvement backlog, where you track your experiments and their completion status. I prefer to add them to the backlog, so that during the planning session time is set aside for improvements.
The final step during this session is to have the team commit to the experiments proposed, individually. Otherwise, everyone will assume that the other person will work on them and in the end nobody does.
Duration: 15-20% (10-15 min).
5. Close the retrospective
At the end of the retro, make a summary of what was discussed and the actions to be taken, decide how to document them (add them to backlog, take photos of cards and boards from the session and add them to the team space, etc.) and how the team will follow up on them.
Remember the rule: “what is discussed in the retrospective, stays in the retrospective“. The learning belongs to the team, not the coach, the facilitator, the team manager, etc.
As a closing for the session, you can ask everyone express gratitude or appreciation, to close on a high positive note. You can also take a few minutes to do a retrospective of the retrospective, get feedback on how to run more effective retrospectives.
Duration: 10% (10 min).
When you plan the timing for your activities, consider also shuffle time (moving between activities, buffers, potential breaks, etc.), of about 10-15% of the entire time for the session.
There are plenty of ideas of activities set for each stage of the retrospective, some in Esther Derby and Diana Larson’s book, but also plenty of websites (my favorite is TastyCupcakes). I will present my favourite activities for each retrospective stage in a following post.
Retrospectives gone wrong
There are plenty of ways for the retrospectives to go wrong; you can identify them based on how the team runs the sessions (following the structure above) and the output of the sessions. Liz Sedley and Rachel Davies call them “retrospective smells” in Agile Coaching:
- Ideas fest: the team comes up with ideas without looking into insights from the last iteration. Actions that aren’t connected to real team-problems won’t push for the team improvement.
- History lesson: the team dives deep into issues and strengths, but no action is taken to proceed with real improvement. Thus change is not planned for the next iteration and the team doesn’t move forward (even though they understand each other better from their conversations).
- Change the world: the team enthusiastically commits to a lot of actions that they don’t have the time to complete. Each retrospective adds more actions to the list, but no real improvement is done.
- Wishful thinking: the team decides on vague improvements, that are not really actions and don’t guide the team members on what exactly to do (e.g. “refactor more”, “improve communication”, etc.)
- No time to improve: the team doesn’t really run retrospective, they have short discussions at the end of the iteration about how to improve and call them a retrospectives. Individual improvement ideas are lost in translation, as there is no space for them to get the team’s attention and commitment.
- Hot air: these retrospectives are complaint sessions and can become downright hostile for everyone present; the facilitator has help dissipate the complaint energy and help the group learn from the complaints, without playing blame-games. The discussion should be guided towards what the team can do, to generate improvement actions.
- Learning by Thinking: How Reflection Improves Performance, by Giada Di Stefano, Francesca Gino, Gary Pisano and Bradley Staats, 2014
- Ideas of retrospectives: TastyCupcakes.org
- Retrospective facilitation techniques – liberating structures.
- More retrospective ideas.