Feb. 3, 2011

UAT on a Scrum Team (Part 2 of 2)

In the previous blog post on this topic we covered the principles relevant to incorporating User Acceptance testers into the Scrum team. This is a complex and potentially difficult impediment at the organisational level. Assuming you’ve managed to get the testers incorporated into the team, the next impediment is sure to raise its head.

Dealing with the impediment at the team level

This manifests during the sprint as follows. During the early part of the sprint, the testers in the team may be idle or prepping test cases and documentation. Perhaps they will test a few smaller stories that are completed part way through the sprint. Then, shortly before the end of the sprint the remainder of the stories simultaneously become “Ready for Test” and the poor tester goes from idle to over-worked overnight.

I usually find that the proximal cause of this is that team is not collaborating on a limited number of stories and completing them before moving on to the next story. There are a number of layers to this problem that we need to peel back to understand how to go about addressing this.

There are two insights that you will need to spark within the team. Firstly, you need to direct the team's attention to the idea that testing is an activity and not a vocation. Secondly, from an external quality perspective, testing only at the end of the story is inadequate. These insights are linked. There are likely to be a number of practises that the team will need to adopt which take place throughout the sprint. This will involve testers and users in more activities where they collaborate with teammates. Conceptually it moves the act of testing away from one of “verification of functionality” towards one of “delighting our users”.

On a practical level, the team needs to start incorporating technical practises that allow for testing (and quality) to be part of the entire stream of activity to deliver a completed story. This "Acceptance Test Driven Development" will require the tester(s) who have expertise in understanding how good software is built, to collaborate with those who are currently writing the tests at an acceptance level. This will make the understanding of what must be built much less ambiguous. For users this means helping to clarify the requirement early and upfront rather than communicating via failed test cases.

Testers can also assist at a unit test level, and with integration testing, participating during the "development" phase of the story. I advocate developers and testers pairing when writing code and this includes writing unit and other tests. It has significant benefits for the quality the team “bakes into” their code. It also begins the process of writing much more readable code and creating a common “business-based” language to express the functionality. This in turn is also reflected in the code. (For more on this look into Domain Driven Design and its practise of Ubiquitous Language)

Regression testing MUST be automated. It is impossible to sustain any form of incremental development without automating regression testing. The load on manual testing will escalate astronomically fast without this in place.

With the above in place, this should mean that the UA level testing should be possible to execute by anyone in the team, and not just a specialist testing person or a user. Testing as a result becomes an activity everyone in the team can (and should) be doing, at all levels of the development effort.

This should also result in software where the testing is much less "black box" and thrown over the fence to the testers. All types of testing are executed during the sprint and this implies a better level of "done" for the stories produced. This will require new skills from the team and will likely mean a slow down in the rate of delivery, at least initially. But if you take a hard look at the software you're producing now, it is likely the case that the quality is largely unknown which has a direct impact on your predictability.

So that's broadly the "WHAT" you need to do. The “HOW” you might go about doing it requires a key insight. As a coach/ScrumMaster you need to examine the level of team trust. What I'm referring to above, are the practises that the team will need to adopt to address this perceived "bottleneck". Implementing these practises is largely a question of trust. Getting users into the team is a potential hurdle at the organisational level that is about trust. Getting testers to relinquish their hold on the pass/fail test at the end of the story is a hurdle at the team level and is about the trust between teammates.

Let’s assume for the moment you have managed to get the users and testers as part of the team. A mechanism you can use to catalyse the discussion that is needed can be achieved by agreeing with the team that they will limit their work-in-progress. By limiting this to one or two concurrent stories, the team must quickly learn how to collaborate on completing a story before moving onto the next. This will start the conversation on what testing is needed, and how the tester(s) and user(s) can contribute earlier in the development of the story (writing acceptance test and the all the other practises described above).

It will also hopefully catalyse the insight that trust is needed between team mates of differing core skills to contribute by working outside their core skills (developers doing testing for example). My advice (again) is to help them start small and figure out what the testers (or users) need in order to feel that teammates have adequately tested the story. Larger contributions such as putting testing frameworks in place I would suggest are secondary to building the trust between skill silos.

Building trust is usually a product of starting small and showing frequent small gains. As testers within the team see how their contributions to the full development process show impact they will start to become more comfortable with the team’s delivery. By understanding what they need in terms of preparation and execution of test cases and showing how other teammates can contribute you should see trust building over time.

It is vital that these actions are agreed and tracked over time, examined in retrospectives and generally made visible. This will help the team build acceptance of the shift from testing as vocation (I am a tester) to testing as an activity (I am a Scrum team member who has expertise in testing).

Lastly I’d like to deal with two common patterns I observe. UAT or testing sprints and teams who may resist trying out the practises described above.

As you can probably tell I am not in favour of recommending separate sprints for UAT. Where UAT is executed outside the team there are a number of effects that are likely to undermine the team's effectiveness.

1) It creates an "us vs. them" dynamic that undermines good communication.

2) There is a problem incorporating the feedback from the UAT into the current sprint.

Either it is only addressed in the following sprint in which case you are increasing the gap between when the work is done and when the fix is done. This leads to a reduction in the context available to the team. This context is vital in increasing the effectiveness of the team. Getting feedback while the code is fresh in your mind is usually much better than waiting on average three weeks (or one and a half sprints) between the events.

The alternative is to have the team fix the issue immediately. This leads to interrupting the team (which breaks their context for the story they were working on). This interruption is cognitively expensive since the team loses their flow on the current stories to work on the error. In addition to this overhead incurred, the rate at which interruption arrive can be unpredictable which undermines the relationship between the team's commitment and their delivery. This can have long term and serious implications not only for predictability (see above for why this is bad) but also team morale.

3) The team is no longer delivering "done" software at the end of the sprint. This has implications for their empirical record of delivery. (See above for why undermining predictability is bad)

In my experience some teams which have been reasonably successful without this approach in place and have shown "good" velocity are typically the ones which have the hardest time accepting that they will need to reduce the amount they work on. This is usually because velocity is seen as a metric of the team’s performance, and anything that might reduce velocity will cause the team to be considered “bad”. Unwavering velocity is a “smell” as much as a wildly varying velocity. If the team is chasing velocity it is likely that they are being incentivised on velocity.

As coach/ScrumMaster you will need to address this impediment, usually by engaging Product Owners and managers in a discussion about the actual purpose of velocity.

For the team I suggest drawing their attention to the lack of "done" software that they are actually producing. In reality their "true" velocity is much less than what they think they've been producing. This is usually also a "lull" before the impact of their quality hits them, usually in the form of massive technical debt or tidal waves of defects.

This means that as coach/ScrumMaster you need to create a space of safety for the team to acknowledge what they probably know to be true with respect to the quality they're producing. It means negotiating with the Product Owner and management to let their velocity drop as they take on these new practises. This will take courage from you but the key thing for successful Scrum teams to learn is the value of learning from failure.

As with the approach to getting the user into the team, get the team to try this for a period and measure the impact. Use the arguments of improved predictability and quality to engage the support of stakeholders to allow the team to try a new way of working.

Conclusions and Insights

In conclusion I would venture that everything you need is already in the Scrum framework. In confronting any impediment it is vital that as a coach or ScrumMaster you bear in mind the principles and values that underlie the framework. In this case they are “Cross-Functional Teams” and “Potentially Shippable Code”.

I must reflect that Ken Schwaber and his relentless pursuit of having Scrum teams producing truly “done” software have critically challenged my thinking on this topic. The insight I hope I have added to the debate that has largely centred on the technical practises (automation, regression testing, acceptance test driven development) is that this must be supported by a truly cross-functional team. In many ways I think eXtreme Programming got this right by demanding that the Customer play a role on the team.

The “Customer” is I think a much better way of articulating and delineating the responsibility for acceptance of the software than the “Product Owner”. This may be mere semantics but it feels like “Customer” as a role is “nearer” to that of the user. It also reflects directly a principle from the Agile Manifesto (you know, on that second page that some people never click through to); “Business people and developers must work together daily throughout the project.” I think irrespective of the role, if we can serve this principle our chances of success are greatly enhanced.

And finally thanks to gmangarfield who asked the question that started me thinking about this, and Sam for suggesting it become a blog post.

Written by former Scrum Sense employee, Carlo Kruger.
Image of peterhundermark

Peter Hundermark

Peter has worked with iterative and incremental software development processes since 1999, focusing on Scrum and Agile practices since 2006. In 2007 he started Scrum Sense in South Africa. He has introduced Scrum into scores of development teams locally and in Brazil. He leads certified Scrum training classes in South Africa and elsewhere. He is a Certified Scrum Coach and Certified Scrum Trainer.
blog comments powered by Disqus
Image of peterhundermark

Peter Hundermark

Peter has worked with iterative and incremental software development processes since 1999, focusing on Scrum and Agile practices since 2006. In 2007 he started Scrum Sense in South Africa. He has introduced Scrum into scores of development teams locally and in Brazil. He leads certified Scrum training classes in South Africa and elsewhere. He is a Certified Scrum Coach and Certified Scrum Trainer.

Latest Posts

Leading and Lagging Indicators: What is the right way to measure performance?

How can you measure and track your performance using leading and lagging indicators? 

Image of hwong

Hazel Wong

Marketing Assistant at agile42. Passionate about gaining insights from data in order to create content that resonates with the audience. Eager to help teams and companies open their mindset about the application of agile methods to address their challenges.

Changing organizational culture at Siemens Digital Factory

Siemens DF extended the use of agile methods the right way, through a thorough adoption of the essence and practice of being Agile

Image of kpogorzala

Konrad Pogorzala

Presenting at the Regional Scrum Gathering South Africa 2018

agile42 present at the Regional Scrum Gathering to be held in Durban, South Africa, 8-9 November 2018

Image of joperold

Joanne Perold

Agile Coach in South Africa. Explorer, learner, experiencer, part time philosopher, working with teams and organisations to be more agile.

Scrumtisch December 2018

The Berlin Scrum User Group meets on December 13th at agile42, Gruenberger Str. 54, 10245 Berlin.

Image of aballer

Alexandra Baller

agile42 Team Assistant

Motivation to Get Better

When Turkish real estate site Zingat realized that their Scrum adoption suffered from anti-patterns, they decided it was time to improve

Image of ayse.turunc

Ayşe Turunç

As an Agile coach, I strongly believe in people talent, in collective intelligence and that happy teams are more efficient. I'm looking forward to put my talent to help teams and individuals to work better together and grow.
Image of ebru4984

Ebru Yalçınkaya

I act as a change agent where the teams, domains need to enhance agility to reach their goals, to create a shared vision if needed. I coach every kind of team , every domain, like management teams or like customer care, technology and sales groups.