Case Study: How to Increase Self-Organization in a Complex Environment

The restructuring of an existing value stream represents the greatest possible tactical challenge that managers can face in the context of agile transformation. It directly impacts the organization’s ability to deliver and affects established social structures. It is also on the borderline between two domains: complicated and complex; some aspects of the target structure are known with a fair degree of certainty based on experience, but we cannot predict how the transformation process will proceed or be received by employees.

In this case study, we show how the right handling of constraints and practices allowed our client’s transformation team to handle complex and complicated questions to maneuver their organizations through this challenge.

Setting the Scene: The Challenge

The client introduced Agile working methods in 2022 and decided to restructure from component teams to cross-functional teams in the summer of 2023. The project involved over 100 developers across Europe.

This transformation was led by a core team consisting of the Chief Product Owner, a transition coordinator, and an internal and an external Agile coach. Additional support was provided by an extended core team including all Scrum Masters and Product Owners of the value stream, as well as the formal managers and leads.

The Four Phases of Transformation

The transformation process was divided into four phases, each presenting specific challenges for the transformation.

Phase 1: Kickoff and Preparation

The starting points for every successful change project, according to John Kotter’s Leading Change, are the ‘Sense of Urgency’ and the so-called ‘Guiding Coalition’. Of course, these also apply to the reorganization of a value stream. But beyond these important starting points, the core team also asked itself:

• How do we approach this?
• How do we want to position ourselves as a team?
• How do we deal with resistance?
  

The first question can be answered by the Cynefin framework. Among other things, it distinguishes between known and unknown contexts and specifies how complicated or complex challenges can be overcome.

Complicated or Complex?

The Cynefin framework provides a good overview of the distinction between complex and complicated domains.

A situation is part of the complicated domain when causal relationships and the effects of an intervention are known in advance. In this domain, those involved are able to analyze the environment and understand it, and they can identify good practices that help to achieve a clear goal. Groups or organizations can be aligned towards the common goal through governance. (cf. ‘Governing Constraints’).

In contrast, complex environments are not known to the extent that causal effects can be predicted in advance; at best, they can be explained retrospectively. Since not all influencing factors are known, an in-depth analysis is not possible. Instead, an experimental approach and probing can be used to try to recognize and determine the influencing factors and make the procedure ‘repeatable’. In complex environments ‘enabling constraints’ allow a better understanding of the environment, and develop and adjust practices emergently based on new experiences. This allows a group or organization better understand the context and move forward.

For obvious reasons, it’s helpful to start a transformation with things that are relatively well-known and understood. Since the change was driven by the market, the reorganization was first examined by the Product Owners. Analyzing their market helped them to determine seven target groups that are served by the value stream. Each Product Owner picked a topic, which was then outlined in more detail. Scrum Masters were subsequently brought on board. Together with the Product Owners and the managers of the teams, they were to make a significant contribution to better understanding and addressing initial concerns.

Conclusion of the Kickoff

In addition to the common goal and the shared sense of urgency, the setup of the transformation team and the extended transformation team was key to later success. By the end of the kickoff, everyone involved had a common understanding of why we as a leadership team wanted to take the next step towards cross-functional teams. Regarding the timing, everyone was willing to try to see if we could achieve this by the end of the next major iteration, which was scheduled for a month and a half later. After two days of the kickoff, such an ambitious goal is certainly possible, but it cannot be set in stone. Nevertheless, the next tasks and responsibilities were clarified during the kickoff.

“Self-organization plays an important role. However, self-organization does not mean ‘everyone can decide everything’ nor ‘everyone decides everything together’. Leadership needs to provide an environment and constraints that allow teams to improve their capabilities to self-organize. This also includes clear responsibilities and a shared understanding of what teams can decide by themselves, what is management’s responsibility, and where we both need to align. Either way, the most important aspect of self-organization is a team’s capability and willingness to take over responsibilities. However, between a fully centralized and a fully decentralized organization, there is plenty of room for joint development of the organization.”

Lothar Fischmann, Senior Agile Coach at agile42

Phase 2: Dry Run

A week before the actual restructuring event, we had a dry run. The purpose of the dry run was to ensure everyone involved understood the staffing process and refine constraints. It was clear to everybody that narrowly defined rules may accelerate the staffing process but would impair the self-organization and motivation of the teams. Besides that, there was also a very practical need for a self-organized approach: well-balanced teams relied on numerous individual factors, including preferences and competencies that couldn’t be fully managed by one person alone.

A complex challenge ahead was how to assign teams to different locations, mainly because not all skills were available everywhere. This meant that teams with varying expertise couldn’t always work together in the same place. Everyone agreed it’s best to have ‘as few locations as possible per team’ and for each colleague to have a local partner to collaborate with. However, there’s no fixed number of locations per team that fits every situation.

Trying to find an ‘ideal number’ or measure distances between locations isn’t a solution because individual factors come into play. These factors included the following:

• How comfortable would employees feel with a foreign language?
• Who likes to start their working day earlier and can therefore work more easily with colleagues from earlier time zones? 
• How willing are individual employees to travel?

To address these or similar factors, representatives simulated and resolved a series of questions during the dry run. The key questions that emerged for discussion were:

• What variables should employees consider when selecting a team/topic?
• How does our organization handle diverse interests or potential conflicts?
• Which ‘constraints’ can help us to shape the staffing process to everyone’s satisfaction?

Takeaways from the Dry Run

The dry run helped participants to explore complex questions by following a Cynefin approach of ‘Probing, Sensing and Responding’. This was applied to Product Owners’ presentations, where they received feedback, as well as the serious discussions that would prepare everybody for the staffing process. The dry run was characterized by open and honest discussions around different topics, such as the following:

• What would it mean to have a joint goal for the staffing process? The value stream won’t succeed if one team gains the most senior and elite developers, but another one is not able to deliver anything.
• What happens if developers choose the topic that interests them more vs. the topic they are more highly skilled in?

At the end of the dry run, there was a common understanding of constraints, ‘escalation levels’, and a positive ‘confidence vote’. All participants were certain that the staffing of cross-functional teams could be carried out satisfactorily in this way.

When introducing the ‘levels of escalation’ we emphasized that ‘escalating’ means lifting a discussion to another level. This has nothing to do with ‘drama’ or ‘what happened at a recent party’. 

Phase 3: The Staffing Event

It would be unrealistic to create a rigid schedule for the first self-organized large group event and expect everything to unfold as planned. However, it is still possible to adequately prepare for such an event. In addition to establishing a shared understanding of collective goals, constraints, and escalation levels, various other influencing factors play a crucial role in the success of the staffing event. We’ll unpack them below.

Target Corridor

Insights from both intuition and past experiences shared by external coaches helped the value stream outline an estimated timeframe for the staffing process. The initial half-day was designated for introductions, background information, and topic presentations. Following this, the staffing activities were scheduled for approximately one-half to one-and-a-half days. This window provided flexibility for the leadership team to address any challenges that might arise, ensuring alignment with the shared goals. If it became apparent by the end of the first day that numerous unresolved issues remained, the extended transformation team stood ready to implement additional measures to introduce more containing constraints.

Shared Ownership

In the dry run and the opening of the staffing event itself, it was communicated that it was not about a single person or a single topic ‘winning’, but that the common goal must be that all products in the value stream can be delivered. The event can only be a success if all teams are confident that they can deliver value independently and overcome the challenges ahead.

Iterative Approach

The staffing process took place in several rounds, which were initially 45 minutes long but were later reduced to 30 minutes. At the end of each round, there was a synchronization with all participants in which teams gave updates on their progress or impediments they were facing. This increased involvement and gave everybody an overview of how they could contribute.

Personal Interaction

Product Owners and Scrum Masters were asked to prepare a ‘market stall’ for their colleagues, with a display showcasing their ideas, topics, and priorities. Developers walked around these stalls, browsing the teams’ displays, exchanging ideas with potential future teams, and getting to know colleagues and topics better. In addition to the restructuring process, the event also included a joint dinner and team-building exercises.

Make Priorities Explicit

Once the developers had walked around the stalls and informed themselves about the details of each of the future teams, they were able to make their priorities transparent. Each developer was asked to report on up to three topics. Knowing the individual priorities improved the moderation of conflicts of interest.

Dealing with Contingencies

Once it was decided that the future setup would contain one hardware and eight software teams, it was clear that not all teams would make decisions and give the green light simultaneously, and some issues would be resolved more quickly than others. To make good use of the event and the time together on site, a side program was created that teams could start if they finished earlier. The focus here was on things that could be done better while everyone was onsite’.

We’re Only Done Once Everyone is Done

For their orientation, all teams were given a ‘Definition of Done’ with quality criteria that had to be fulfilled. Based on this checklist, the teams received a ‘preliminary okay’ from the CPO and were able to join the side program. However, since staffing decisions elsewhere can also affect teams that were thought to be finished, the rule was “We are only done once everyone is done”.

Outcomes of the Staffing Event

By the end of day one, two roles still had to be filled. These were resolved by midday on the second day, which was right in the middle of the targeted time box.

Phase 4: Post-Match Time

The remaining question was: How could everybody, including stakeholders, management or coaches recognize whether the new teams work out?

The extended transformation team was able to rely on an indicator that is inherent to all Scrum Teams: Sprint Goals. These helped to answer the key questions:

• Are the new teams able to independently set appealing Sprint Goals that add value to their stakeholders?

• Are they able to pursue this target as part of an iteration? and 
• Do they develop a sense of how much they can promise and deliver over the next iteration?

The importance of Sprint Goals is not only due to their significance for stakeholders and management: pursuing a common goal also has a motivating effect on the entire team and catalyzes the team development process.

The Final Outcome

“In the next couple of weeks around half of the teams managed to take responsibility for their Sprint Goals independently or with a little support and develop a good feeling for the team’s performance. The joint and cross-team discussion on Sprint Goals helped us to generate initial successes. On the one hand, these pioneers naturally motivate other teams, and on the other, it also allows us to recognize where further coaching support can be targeted.”

Lothar Fischmann, Senior Agile Coach at agile42