June 16, 2009

Scrum - Challenges, Risks & Anti-Patterns

Article originally published in the German IX Magazin, Heise 2009 providing an overview of the Scrum methodology

Finally! After so many years working on projects that simply didn’t work, today there is a methodology for delivering projects on time and budget: Scrum. Many companies have made the change and proven that it works. But is it easy? No, Scrum is not easy. To follow the principles of Scrum requires courage, complete transparency and the confidence to make mistakes. Only people who openly and confidently handle the issues that arise in order learn from them will be successful in using Scrum to develop software and systems in the long run.

What is Scrum?

Scrum is an iterative and incremental process for software development and the organization of teams: Tasks are delivered faster and with higher quality. This is possible because of the high level of self-organisation in the development team, which chooses which features to develop and how to break these features into tasks. The customer requirements are regularly prioritized and quickly realized.

This article starts with a short presentation of Scrum, followed by a description of the principles behind Scrum, the reasons some companies are skeptical of realising the benefits of Scrum, and some of the common myths surrounding Scrum.

Scrum in a Nutshell

”Scrum is a simple 'inspect and adapt' framework that has three roles, three ceremonies, and three artefacts designed to deliver working software in Sprints, usually 30-day iterations.” (1. Scotland, A. Scrum @ the BBC. in JAOO. 2005. Aarhus, Denmark: EOS.)

Scrum roles

In contrast to traditional project management methods, Scrum avoids a hierarchy of managers such as the product manager, project manager or team leader. There are just three roles defined in Scrum:

  • The Product Owner
  • The Scrum Master
  • And the team.

These three roles are peers, each with well-defined and distinct responsibilities. The Product Owner (PO) is responsible for the vision of the product, the gathering and prioritization of requirements and control over the budget: Fundamentally, the PO is responsible for delivering the expected return-on-investment (ROI). The Scrum Master (SM) resolves problems, takes responsibility for the Scrum process itself, and coaches the team. The Development Team is a self-organized unit, responsible for the development and technical quality of the product.

Scrum Ceremonies - From the Sprint Planning Meeting to the Retrospective

Scrum Projects are divided into iterations, often called Sprints, which are time-boxed (kept to a fixed length of time). A typical Sprint is time-boxed to between two and four weeks.

At the beginning of each Sprint there is a Sprint Planning Meeting during which the PO presents to the Development Team a prioritised Product Backlog consisting of a list of business requirements, described in a short form known as a User Story. The team ask questions of the PO in order to expand on the information written in the User Story to better understand the context of the User Story, and then roughly estimates the effort. In contrast to traditional project management approaches, features are not allocated to the Development Team, with the expectation of when the features should be delivered. Instead, the Development Team selects the User Stories it feels comfortable delivering within a fixed time-box, the Sprint.

Once a preliminary commitment has been made by the Development Team to the PO, the team and the SM begin breaking down the User Stories one by one into more detailed tasks. The team finally commits to a Sprint Backlog, consisting of a number of User Stories taken from the Product Backlog which have been broken down into tasks. These User Stories and associated tasks provide the basis for the daily tracking of progress through the Burndown Chart, which shows how many tasks or User Stories are completed on a daily basis throughout the Sprint.

Once the Sprint starts, the Sprint Backlog will not be changed. The team can focus on a delivering the User Stories, confident that the requirements will not change during the Sprint. In Scrum, the team and the PO define explicitly what it means to complete a feature, and the team will complete all related tasks such as technical or user documentation and testing before showing the User Story as complete.

During a Sprint the team works undisturbed on the tasks in the Sprint Backlog until they are done. It meets daily in a meeting, the Daily Stand-up, time-boxed to 15 minutes.

Here the each member of the team will briefly answer the following questions:

  • What did I accomplish yesterday?
  • What will I do and achieve today?
  • Are there any problems?

At the end of the Sprint, the team presents the completed User Stories to the PO during the Sprint Review Meeting. The PO sees only completed work - no more "it's 80% complete" - and is responsible for accepting or rejecting each User Story depending on the pre-defined acceptance criteria included in the User Story description.

Following the Sprint Review, the team holds a Retrospective, a meeting which serves as a reflection on the previous sprint; what things went well, what things went poorly, and what things can be improved. As a result, the team iteratively reviews the process and can make small adjustments and evaluate the impact of each change, leading to continual incremental improvement.

During the whole process, the SM moderates the entire process and prevents external influences from disturbing the roles working within the Scrum framework.

Changes and Challenges when Adopting Scrum

(Ken Schwaber, The Enterprise and Scrum, 2007)

  • Staff turnover will occur
  • Conflict will occur
  • Product management’s job will change and will be harder
  • Engineering is accountable for quality
  • Compensation policies need to change
  • Job will change
  • Management’s primary responsibility will shift from command to servant leadership
  • Management turnover will occur

Scrum makes employees leave the company

At first sight this sounds very dramatic, but it does not have to be. Scrum creates transparency and this quickly prevents anyone hiding behind or pointing at the (supposed) mistakes of others. Through constraining the work to a fixed Time Box, the tasks within the team and the transparency in respect to what is delivered removes the potential for common misunderstandings or excuses such as ”I’ve been waiting for this or for that”, or “I had to do so many other things for the customer that I simply didn’t get to it”. Yes, the successful introduction of Scrum may lead to the fact that some employees and manager don’t feel comfortable in their company. Perhaps some of these employees will leave because not everyone goes along with the changes that Scrum brings. But if we are honest it’s often best for both parties - Scrum brings a lot of transparency, and the impact of avoiding transparency has a detrimental impact on both employees and the company, with or without Scrum.

Scrum leads to conflicts

Scrum is a very simple framework, but the combination of transparency with short fixed-duration sprints often exposes pre-existing conflicts or inefficiencies between development teams, business owners and management.

The simple rules of Scrum mean that every cause of difficulties in the software and/or product development process in a company will be recognized. This is where the courageous companies win, and some companies that are uncomfortable with recognising and tackling the problems will often step back and fail to remove them. That Scrum cannot function in this environment should be clear. However this failure to recognise or tackle systemic problems often leads to the statements or excuses that ”Scrum is chaotic” or ”Scrum doesn’t work for us”.

The spectrum of possible conflicts that can appear through the introduction of Scrum is broad. It starts with managers who like transparency in development, but who don’t want to share their visions or who think that the rules are only for others. The traditional project leaders first need to understand how their role will change with Scrum, and ensure full cooperation with the customers of the development teams.

As more and more companies transition to Scrum, experience shows that professional support and advice is needed. Although the Scrum framework is simple, it is still a major change, and the advantage of an independent and experienced guide while making the transition has been proven over and over again.

(At this point I’d like to express my gratitude and thanks to all my colleagues and advisors that, through ”Scrumtisch,” ”Scrum Breakfasts” and other initiatives, continue to make Scrum successful, and answer questions and offer support at anytime.)

Scrum changes the role perception

Just yesterday I finished a presentation for project leaders and developers and recognised that the most important question from the audience was: How does Scrum fit with our established structures and roles? We already covered the three Scrum roles, the Product Owner, the Scrum Master and the Team. Unfortunately, the nearest equivalent to be found in traditional projects: Product Owner instead of Product Manager, Scrum Master instead of Team- or Project leader, or Development Team instead of developers and testers does not properly differentiate between the responsibilities of each Scrum role.

Product Owner

The nomenclature is definite. In Scrum, the PO is not only the manager of a product, but also the owner of the product, from vision to conception to delivery. Therefore he/she is the one responsible for the creation of the product. Being a Product Owner means:

  • You are responsible for the successful outcome of the product delivered by the team.
  • You make business decisions around importance and priorities.
  • You deliver the vision of the product to the team.
  • You prepare the User Stories for the team to develop.
  • You should possess extensive domain knowledge.
  • You validate the work delivered and test it for quality.
  • You communicate on a continual base with all stakeholders, financiers and the team.
  • You control the financial side of the project.
  • You are responsible for the return-on-investment of the product development.

Scrum Master and Team

Unlike many other development methods, a Scrum development team is not simply the executive organ that receives its tasks from the project leader. Rather, it decides itself which requirements or User Stories it can accomplish in one Sprint. The team constructs the task list themselves and is responsible for any permutation to those; in other words, the team becomes a manager. This new self-conception of the team and the alignment of tasks and responsibilities necessarily change the role of the team leader/project leader.

The Scrum Master does not need to delegate all the work or to plan the project. Rather he/she ensures that the team is able to reach the self-made goals, removing impediments and providing an ideal working environment for the team. This change in roles is one of the most important aspects to understanding Scrum with the intent of introducing it in your own company.

Scrum Anti-Patterns

Scrum works! Innumerable projects prove that. But, and of course there must be a ”but,” Scrum only works when one has the courage to really realize the rules and ideas behind Scrum. Scrum is more than just the summation of all the roles, ceremonies and artifacts.

We do not expect a perfectly smooth introduction of Scrum. Everyone involved needs to get used to the new framework and will be unsure what needs to be done if something unexpected happens. Problems often lead to ”known” mechanisms ”just for this one case” instead of solving the root cause of the problem, as required in the Scrum method.

Once the synergy between teams that is created by Scrum is broken, the impact of Scrum, the outstanding increase of productivity, gets lost rapidly.

Here are some of the worst, as in most destructive, Scrum Anti-Patterns we have seen: - The Scrum Master assigns tasks, thereby destroying the self navigating and self organized team. - The sprint gets disturbed externally. New tasks are included or changes to the committed User Stories are allowed. - There are no Review Meetings. - There are no Retrospectives. - There are no Daily Stand-ups. - The Product Owner doesn’t have adequate expertise or experience and can’t stand up for his role. - There is no ‚”1-to-1 Agreement‚” between the PO and the Team. - The participants aren’t trained well enough in Scrum. - All meetings aren’t time-boxed. - Instead of features, activities are monitored. - Organisations think that Scrum can be rolled out after a few Scrum Masters are sent to certification courses.

Scrum Hit list - 10 good reasons for Scrum

  1. There are many good reasons to work with Scrum. The 10 most important include: Competitiveness - In many markets, the competitive environment is changing faster and faster. Only companies that are flexible and agile have a fast enough time-to-market to stay competitive and create a competitive advantage for themselves.

  2. Develop what is needed - Scrum allows the incremental development of features. The customer receives the first working versions early and as well as seeing progress can, if necessary, add new ideas.

  3. Confidence through transparency - Transparency plays a critical role at various levels in Scrum and is one of the primary drivers which makes Scrum so efficient. Because of transparency all the stakeholders are informed where the project is at any given time. This helps in discovering weaknesses and it makes effective teamwork possible.

  4. Build quality in - Testing is an integral component of Scrum in every sprint. Only if the software is tested and documented is it ‚”ready‚”. Methods such as continuous integration are very applicable for increasing the quality of every object from the beginning.

  5. Recognize risks in time - systematic risk management - Regular releases establish the conditions in which to recognize problems in time and to react promptly. Long-term statements about time to completion and product delivery roadmaps are possible because of factors like the team velocity (a measure of how much a Team can do per Sprint).

  6. To have the costs under control - The duration of a project is usually fixed, but the effort and the complexity of the work is usually only roughly estimated. Scrum allows changes to feature specifications, in collaboration with the customer, creating transparency and allowing definitive cost accounting.

  7. Changes are welcome - The Scrum framework is not afraid of changes. On the contrary, changes can be shown to the Product Owner at any time and can be realized in the following sprint without generating conflict. That way the customer gets the product he or she desires.

  8. Efficient collaboration and customer satisfaction - Scrum involves regular communication between all the participants of the project. Communication, collaboration, respect and understanding what the customer requires or what the team develops is the basis for delivering successful projects.

  9. Scrum is perfect for system development as well - As opposed to the widely-held view, that Scrum is only good for small products, Scrum is very good for the development of complex systems and extended projects. The exact planning and practices such as the continuous integration of new functionalities provide visibility of progress, and ensure that there won’t be a big surprise at the end of the project. Many large systems are developed with the same agile principles behind Scrum.

  10. Last but not least - Scrum is fun - Not task selection necessarily, but collaborative working and decision-making as a team is an important part of Scrum. Software development is seen for what it is: A creative and multi-dimensional activity that only can work properly when everyone takes responsibility.

Conclusion

Finally! After so many years working on projects that simply didn’t work, today there is a methodology for delivering projects on time and budget: Scrum. Many companies have made the change and proven that it works. But is it easy? No, Scrum is not easy. To follow the principles of Scrum requires courage, complete transparency and the confidence to make mistakes. Only people who openly and confidently handle the issues that arise in order learn from them will be successful in using Scrum to develop software and systems in the long run.

Scrum works, although we have to accept that fixed price projects without detailed preplanning and expensive specifications are possible. Confidence not only increases the productivity in the team, but also the cooperation and trust between the customer and the service provider. Communication is just as important for their development as the development itself. Then Scrum will start to convince the skeptical people in your organisation.

Image of marion

Marion Eickmann

I am one of the founders of agile42. Even though I am not an engineer I consider myself almost a "Techi" as I have been working in the field of software development for 10 years now.
blog comments powered by Disqus
Image of marion

Marion Eickmann

I am one of the founders of agile42. Even though I am not an engineer I consider myself almost a "Techi" as I have been working in the field of software development for 10 years now.

Latest Posts

Mike Freislich on the inevitability of change

Video of Mike Freislich interview with Klaus Leopold on the Lean Business Agility podcast

Image of abragad

Alessio Bragadini

Web community manager of agile42, trying to post relevant, informational, fun bits of content on the blog and social networks

Agile Portfolio Management in it-daily.net

A new article in German magazine it-daily.net

Image of marion

Marion Eickmann

I am one of the founders of agile42. Even though I am not an engineer I consider myself almost a "Techi" as I have been working in the field of software development for 10 years now.

Why Agile Transformations Fail – Do You Fall Into The Same Pitfalls?

Insights from TAC2017: Why do agile transformations fail? Identify whether your team or organization falls into the same pitfalls.

Image of hwong

Hazel Wong

Marketing Assistant at agile42. Passionate about gaining insights from data in order to create content that resonates with the audience. Eager to help teams and companies open their mindset about the application of agile methods to address their challenges.

Sponsoring Let's Test South Africa

Let's Test South Africa 2017 sponsored by agile42

Image of joperold

Joanne Perold

Agile Coach in South Africa. Explorer, learner, experiencer, part time philosopher, working with teams and organisations to be more agile.

Niels Verdonk, new Certified Scrum Trainer in the Netherlands

Niels Verdonk has qualified as a Scrum Alliance Certified Scrum Trainer

Image of andreat

Andrea Tomasini

I am an Agile Coach and Trainer and I am helping customers all around the world to become more Agile. I am more and more keen on adopting adaptive emergent approaches to improve people's quality of life. Through an holistic and pragmatic approach - I consider Lean and Agile very powerful frameworks - it is possible to improve results, performance and also personal satisfaction.