WIN #008: Overcome Scatter-Gather Downsides

[click to enlarge]
How to deal with the Scatter-Gather downsides? Find out in this week’s ideas newsletter.



❓ What if team members leave the team?

❓ Wouldn’t it be great if the effects of their departure were minimized?

💡 Effective teamwork and essential collaboration serve as very effective risk mitigation tools here.

Teamwork is more about collective knowledge, experience, skills, and problem-solving ability than about producing flat-out results.

Of course, what is produced matters in the end, but it comes as a natural consequence of all the aforementioned.

🤖 Consider Tesla Dojo for example. It is an impressive tech feat nonetheless. It is a supercomputer that centralizes all the experience acquired by all the agents in real-time and makes it available back to them. Effectively, what is learned by one is immediately available to all the others. Sort of a hivemind.

Now, think about what changes if any of the agents in such a system are removed. The system’s learning capabilities are reduced by the missing agent, of course, but all the knowledge is still there, and readily available for the next agent to be included.

Switch back to human-based teams now. Humans are not Tesla’s agents, of course. People come with all the advantages, flaws, and varieties inherent in humans. And people cannot relay their thoughts and experience back to the collective in a uniform and ready-to-use manner.

Think about what would happen if you made all the learning and experience together. I am not talking about working alone, in isolation, and telegraphing results back to the group. I am talking about actually acquiring knowledge and experience together, by working together, making decisions together, and making mistakes together.

💡 This takes you as close as possible to the described knowledge-acquisition/sharing system.

❗️ People leave teams all the time, due to various circumstances. Having knowledge concentrated in any one individual increases the risk of incapacitating the team if that happens.

💡 Use the power of essential collaboration, and if possible pairing and/or mobbing to maximize the effect of collective knowledge, experience, and skill building.

💡 Use the power of collective knowledge, experience, and skill building to minimize the effects of possible fluctuations within the team.

Harness this power to mitigate team fluctuation risks.



❓ A SW dev team where everybody collaborates instead of working independently – does this sound like a myth?

💡 Here is a handy idea for achieving that without going to extremes ⤵️

The whole team collaborating and working together all the time sounds idealistic to many.

It raises more questions than providing answers just thinking about the concept.

The first things that come as comments when discussion arises are:

➡️ This is wasteful!
➡️ Everyone works on everything, so everyone must know everything for it to work!
➡️ The most creative ideas come from “zoning in” and working alone!
➡️ Everyone on the team is a professional, so assign tickets and get it done!
➡️ They are experts in their areas and won’t work on anything else!
➡️ Nobody is accountable!

Working together as a team often translates to images of people paring or working in a mob-programming fashion.

Although those might be the most potent forms of teamwork, they are equally hard to achieve.

Fortunately, for those lacking faith, there is a stepping stone to achieving such a meaningful collaboration. It is an organizational pattern called “Scatter-Gather,” coined by Tim Ottinger.

In such an approach, team members work collaboratively at least at two points:

1️⃣ at the beginning of the process, by creating individual atomic tasks from a user story, for individual team members to work on.
2️⃣ at the end of all tasks, by integrating results.

It is where the pattern got its name. The work is “scattered” across the team and then “gathered” and integrated together after individual tasks are done.

This approach works well for not-so-cohesive teams:

➡️ Groups with low-level of trust
➡️ Groups of narrowly skilled individuals, typically juniors
➡️ Geographically distributed groups
➡️ Groups distributed across non-overlapping timezones.

💡 Upon mastering the approach, the team can further be encouraged and coached to try more effective collaborative approaches.

💡 If you are working in groups of individual contributors, which are teams only by label, try this approach!



❓ That Scatter-Gather approach works like a charm! Why doesn’t everybody work like that?

💡 Although tempting as a bandaid for missing teamwork, the Scatter-Gather approach comes with notable issues. Read on ⤵️

📚 The idea behind the Scatter-Gather approach is to split the work and “scatter” it across the team, only to “gather” it once all the pieces are done. This is where the results of work on pieces are integrated into the whole.

Although there are benefits to working in isolation, one developer per user story, there are downsides as well:

➡️ Waiting for everything to be integrated to get any meaningful feedback from the system, or the users.
➡️ Leaving the feature in an unusable state (“components”) until everything is done.
➡️ Insisting on specialization and specialists, leads to work queueing and waiting for them to be available.
➡️ Knowledge silos are inevitable. Deep expertise in small parts of the system forms in individuals.
➡️ Testing is postponed until everything is finished, since only parts, and not a testable whole is finished.
➡️ Given the expertise, work is spread in time, and work from different unfinished stories piles up on experts, leading to frequent context-switching.
➡️ Progress of the whole story is unclear.
➡️ Collaboration is suffering due to individual assignments.

The list is not complete, of course. I am pretty sure that you can come up with some more entries.

💡 Even though it is an advancement compared to strictly individual work on stories, the Scatter-Gather approach is a suboptimal way of working.

💡 Employ it with care as a stepping stone towards more effective approaches (e.g., pairing and mobbing).



❓ Scatter-Gather is appealing, but it has downsides. Should I use it?

💡 There are tangible benefits of using the Scatter-Gather approach, and with these ideas, some of the downsides could be circumvented ⤵️

❗️ You can at least mitigate the effects of some of them, if not avoid them altogether:

1️⃣ Get rid of upfront design by an individual (usually an architect or a senior).

💡 Involve the whole team in detailed design discussions and decision-making, and JUST BEFORE YOU “SCATTER” THE WORK. I detailed the approach in my earlier posts.

2️⃣ Address knowledge siloing.

Team members working on “scattered” items will discover new things unknown to the rest of the team.

💡 Collaborate whenever there is a new case or failed attempt. Share newly acquired knowledge right away.

3️⃣ Prevent delays and piling up of items at specialists.

Work items tend to pile up at a certain point if there is a strict emphasis on specialized work (e.g., frontend, backend, testing) leading to overburdening and delays.

💡 Encourage team members to work outside their main expertise, at least in a supporting capacity.

❗️Keep in mind that team results are more important than individual ones.


Build highly effective software development teams

Subscribe to the weekly ideas newsletter and get tips and insights to help you with building highly effective software development teams. You’ll get tips on how to do so every week.