May 23, 2019 at 07:00 PMHow to conduct a Distributed Paperless quarterly planning and not screw it up?
Given: A company which uses the Scaled Agile Framework (SAFe) to scale Agile development across the organization; 10 development teams combined into one big team (Agile Release Train, according to SAFe terminology) to deliver a common product; the need for a two-day quarterly planning (PI Planning) to determine the work plan of IT teams for the next 3 months *; three development offices with the distance between the most remote ones exceeding 6 thousand kilometers and corresponding working time difference of 5 hours; previous planning experience which implied usage of analogue boards / whiteboards / highlighters / sticky notes and respective physical presence of all key employees in the same room.
* This heavyweight construct “The work plan of IT teams for the next 3 months” threatens to increase the size of the text significantly, so hereinafter I’m going to replace it with “the commitment”. Accordingly, to draw up and adopt a work plan will be “to commit”.
1) Fatigue with analog methods of work. While spaceships are plowing the Space, and Elon Musk is boring his tunnels, we, the IT guys, have been persistently writing with highlighters on sticky notes sticking them on the boards — there is really some kind of dissonance in this, isn’t there? That’s what our commitment looked like a while ago:
Yes, this is a sheet of paper about 2 by 5 meters in size, with each sticky note to be turned into a JIRA task after the planning …
2) Fatigue of our nomadic colleagues. Despite the fact that our head office is located in a rather pleasant and warm country, I was surprised to find out last year that they are far from being happy about all the trips back and forth. My arguments “Oh well, you can just have some fun relaxing in the sun at the seaside” were received not very warmly. As it turned out not everyone is ready for frequent business trips, which often are not welcome by their families, some colleagues just couldn’t bear all the time in the air considering the distances between the offices and the frequency of the meetings.
3) Engaging more IT colleagues in the planning process. As is clear from the above paragraph, we could hardly afford to have all IT staff from the three offices coming for planning, so only the Chosen Ones were selected, which kind of excluded the others from the process. This was hardly beneficial for the team spirit in general.
4) Costs optimization. Yes, this is a sensitive moment — there are quite a lot of key people and having them fly back and forth four times a year is by no means cheap.
Everything begins much earlier than the actual planning. In order to work productively on the Commitment during the planning, you have to have something to commit for. To ensure this, I have to “load” business owners with work on probable initiatives (or, in SAFe terminology, Epics, but hereinafter I’m going to use our usual term) as early as possible. Ideally, this process should be completely divorced from the iterative nature of quarterly planning and go its own Kanban way. We took SAFe Portfolio Kanban as a basis:
We have a separate JIRA project with three issue types: Epics, Business Initiatives and Architectural Initiatives; as well as the corresponding Kanban Board. The algorithm for the Initiative’s Business Owner is simple: he adds an issue to this project, and it goes to Backlog by default, which is the first status of our Kanban flow — Funnel:
Here the Initiatives not yet ready for Review are stored. The Business Owner works on the content of the Initiative until he feels ready for the next step. The minimum required at this stage is to have filled-in fields of Problem Statement, Desired Outcome and Measure of Success, as well as a slightly more detailed, but simple and clear presentation. At this stage it is important to leave aside technical solutions and focus on business parameters. When all data is collected, Business Owner moves the task to the next status — Reviewing.
We conduct Reviews on a weekly basis for both Business and Architectural Initiatives. Ideally this is a five-minute presentation with subsequent Q&As. As a result of the Review a decision is taken about what happens to the Initiative next. It can:
At this stage we — Yippee!!! — can finally engage IT guys: analysts, architects, leads, anyone. By “we” I mean myself as a Release Train Engineer. But ideally it should also be Business Owners, who have already gained some experience and autonomy necessary for engaging the right teams and independently launching analytics. Again, I am writing about the ideal case here. In fact the level of self-organization of our colleagues is floating, and in some cases they ask for help of a specially appointed facilitator, but I try to step back from this practice.
The purpose of this stage is preliminary analytics, or, as we call it, Pre-discovery. As a result we should get a minimum that would allow us to commit: proposed solutions, risks and dependencies, non-functional and infrastructure requirements, user maps, architectural schemes. In addition, we ask the teams to give a preliminary estimation in story points at the feature level — this will allow us to determine priorities at the end of the quarter.
A personal Kanban Board is created at this stage for each Initiative, in which features and stories can be seen, with a preliminary link to a specific iteration, which is provided by a separate JIRA workflow. So by changing the status we can assign the issue to some iteration. Swimlanes in Kanban Boards are configured by development teams. Since our JIRA ecosystem is rather complicated, during Pre-discovery and Discovery we create tasks in separate projects in order not to litter the Teams’ backlogs:
Also at the Pre-discovery stage, we involve Architects or, as it’s been often practiced lately, their trusted people — “EA Ambassadors”. Their task is to convey the vision of the architectural department to all the participants, to help find the best solution, and finally to approve this solution with the company's Head of Enterprise Architecture.
When all the people involved believe that the Initiative has been analyzed enough and is ready for the next step, it is moved to the next status — Portfolio Backlog.
At this stage it is important to conduct a WSJF prioritization (Weighted Shortest Job First). To do this, we have a big meeting 3 weeks before the planning, which, unfortunately, up to now has always lasted many hours. During this meeting members of the Management Board evaluate the Business Value, Time Criticality, Risk Reduction and Opportunity Enablement with the help of Fibonacci-aligned planning poker:
To be able to track the history of Initiatives, for each of them I add a label in JIRA with quarter data: 2018Q4, 2019Q1, etc.
Looking ahead, let me describe all the possible statuses. After the final commitment at PI Planning, Initiatives successfully taken into work are moved to the Implementing status, while those ‘not fitted’ receive Parked status and might be considered for the next quarters. Delivered initiatives are moved to the Done status.
As a result, we have the following picture on the Kanban Board:
Of course, we are not even in the middle of our way yet, but at the moment I can already note that thanks to the Portfolio Backlog implementation
A month before PI Planning is when the busiest time comes. Too many things need to be taken into account, and in order not to leave anything out, I have to create a multipage “logistic” Google Form. Let me describe the most important tabs from this Form and the activities around them.
Feedback. A few days after each PI Planning we conduct a Retrospective, which helps us not to forget what conclusions we came to and how we need to adjust the course. This is an important part in terms of continuous improvement.
Technical preparation. With transition to remote planning, specific requests have appeared, such as quality of Internet connection (prioritization and optimization of routes for JIRA and Confluence) and video- and audio presence (we use the Logitec Group kits, Jabbra microphones and personal headphones in various combinations, webcams, Zoom software for video conferencing).
Facilitators. It’s a list of employees responsible for facilitation in each of the working groups, indicating their location and a link to the permanent Zoom-channel of the working group.
Audience. The complete list of all invitees.
Checklist. The full list of important tasks with deadlines and responsible persons. To give you some insight below you can find several examples of the most typical and vital tasks relevant for each PI Planning: training of Facilitators (a training manual has been prepared with all the necessary steps such as organizing a working team, timing the meetings and decomposing the Initiative into a list of features); update of working groups’ location plans for each office; control of the Velocity update for all development teams; ordering meals; creating reports from previous quarters; printouts of lists of Initiatives and schedules.
After all the preparatory running around, this moment finally comes — the start of PI Planning. In two days we have to discover all Initiatives, make sure that the information is sufficient and commit. After a few pep talks the working groups go to their places and get to business. At the moment the number of groups is distributed among the three offices as 4x4x2, and remote users are connected to the required group via a dedicated Zoom channel.
Upon completion of the Discovery on each of these Initiatives, the facilitator makes sure that all the necessary data, such as Acceptance Criteria, technical solution, risks, dependencies, the necessity of a new infrastructure, is available and marks the Initiative with the ‘IT Session Finished’ checkbox for easy tracking of the progress. After that the working group can move on to the next Initiative.
After a dozen of PI Plannings the feeling of being under constant stress and time pressure, which was with us from the very beginning, has significantly faded, and now it’s all more like work as usual. In the afternoon on the first and second days of the Planning it’s the time for unhurried commitment according to priorities. To perform a commitment, I have several separate structures. The first one is a structure containing a list of features and stories scheduled for the commitment:
Unfortunately, this structure on the screenshot contains only tasks that did not fit into the commitment of the current quarter, so the sample is unrepresentative. In any case, in the search field I can select the Initiative needed in order of priority by number, which is put in a separate field for each feature and story, change the issues’ status from Planned to Committed and put them to the desired iteration, thereby committing for them:
As a result, the issue will disappear from this structure and appear in a new, which reflects the growing commitment:
As you can see on the screenshot, in this structure the stories fall into the folder of the development team to which they belong and are grouped by iteration. Here, I see the total Velocity of the team in the folder name, and in addition in the Commitment column, the overcommitment is automatically determined and highlighted for each team, by using a custom formula.
Of course, if the first and highest priority Initiatives are included into a commitment mostly easily, soon, as more and more teams fill their backlogs to the end, there could be respective problems with some of the initiatives, after certain arguments and discussions, being postponed (parked) as a result.
At the end of this rather straightforward process the teams swear to deliver their commitment on the company's flag (ok, not really) and hurry to their homes (well, again, not really, mostly they flee to some bar to celebrate).
While everyone is drinking, the RTE is working on creating commitments in the form in which it could be distributed to the development teams’ backlogs and monitored throughout the quarter. To do this, I use the 'Bulk Clone Professional for Jira' plugin — it adds an item doing collective cloning to the Bulk Change menu and has such necessary features as links cloning, updating links between newly created clones, updating Epic links, adding Summary prefix and automatic labeling:
I add status as a prefix, because at the planning stage the statuses reflect the planned iteration of completion. As a result I can see right away in the Summary when we are planning to finish the feature or story.
At first, I clone absolutely all issues into a separate Global Backlog project, since we keep epics in it, and I need to have actual epic-story connections in newly cloned tasks. And already as a second step, I make JQL-requests for each development team separately and move the stories into the working projects of the teams.
As a result, I create a structure that reflects the current Commitment and consists of final stories, which I then use for monitoring:
In general, the advantages of this approach are the following:
Dear Friends, of course I understand that the above overview is rather superficial, and a lot of things could be revealed in a more detailed way — monitoring distribution of capacity among Business and Architectural initiatives and operational tasks, calculating custom formulas in structures, managing risks and much more. But I still don’t know whether this topic is interesting to the audience, and if yes, than what exactly. If I see any interest, I will be glad to share my insight on the relevant topics.
And one more thing — it’s hard to believe it would be possible, but still I would really like to avoid butthurt related to Agile, frameworks, effective and “effective” managers in the comments. Please remember that I described the technical part of the whole process in the hope of getting interest of the people who are close to the topic, and I look forward to relevant discussions.
See you in the comments!
* This heavyweight construct “The work plan of IT teams for the next 3 months” threatens to increase the size of the text significantly, so hereinafter I’m going to replace it with “the commitment”. Accordingly, to draw up and adopt a work plan will be “to commit”.
Why do we need this?
1) Fatigue with analog methods of work. While spaceships are plowing the Space, and Elon Musk is boring his tunnels, we, the IT guys, have been persistently writing with highlighters on sticky notes sticking them on the boards — there is really some kind of dissonance in this, isn’t there? That’s what our commitment looked like a while ago:
Yes, this is a sheet of paper about 2 by 5 meters in size, with each sticky note to be turned into a JIRA task after the planning …
2) Fatigue of our nomadic colleagues. Despite the fact that our head office is located in a rather pleasant and warm country, I was surprised to find out last year that they are far from being happy about all the trips back and forth. My arguments “Oh well, you can just have some fun relaxing in the sun at the seaside” were received not very warmly. As it turned out not everyone is ready for frequent business trips, which often are not welcome by their families, some colleagues just couldn’t bear all the time in the air considering the distances between the offices and the frequency of the meetings.
3) Engaging more IT colleagues in the planning process. As is clear from the above paragraph, we could hardly afford to have all IT staff from the three offices coming for planning, so only the Chosen Ones were selected, which kind of excluded the others from the process. This was hardly beneficial for the team spirit in general.
4) Costs optimization. Yes, this is a sensitive moment — there are quite a lot of key people and having them fly back and forth four times a year is by no means cheap.
Part Zero: Working with Portfolio Backlog
Everything begins much earlier than the actual planning. In order to work productively on the Commitment during the planning, you have to have something to commit for. To ensure this, I have to “load” business owners with work on probable initiatives (or, in SAFe terminology, Epics, but hereinafter I’m going to use our usual term) as early as possible. Ideally, this process should be completely divorced from the iterative nature of quarterly planning and go its own Kanban way. We took SAFe Portfolio Kanban as a basis:
We have a separate JIRA project with three issue types: Epics, Business Initiatives and Architectural Initiatives; as well as the corresponding Kanban Board. The algorithm for the Initiative’s Business Owner is simple: he adds an issue to this project, and it goes to Backlog by default, which is the first status of our Kanban flow — Funnel:
Here the Initiatives not yet ready for Review are stored. The Business Owner works on the content of the Initiative until he feels ready for the next step. The minimum required at this stage is to have filled-in fields of Problem Statement, Desired Outcome and Measure of Success, as well as a slightly more detailed, but simple and clear presentation. At this stage it is important to leave aside technical solutions and focus on business parameters. When all data is collected, Business Owner moves the task to the next status — Reviewing.
We conduct Reviews on a weekly basis for both Business and Architectural Initiatives. Ideally this is a five-minute presentation with subsequent Q&As. As a result of the Review a decision is taken about what happens to the Initiative next. It can:
- be returned to Funnel for revision,
- be abolished without a chance for further life (in this case I use a special Out-of-Date status hidden on the Kanban Board),
- be approved and moved to the next stage — Analyzing.
At this stage we — Yippee!!! — can finally engage IT guys: analysts, architects, leads, anyone. By “we” I mean myself as a Release Train Engineer. But ideally it should also be Business Owners, who have already gained some experience and autonomy necessary for engaging the right teams and independently launching analytics. Again, I am writing about the ideal case here. In fact the level of self-organization of our colleagues is floating, and in some cases they ask for help of a specially appointed facilitator, but I try to step back from this practice.
The purpose of this stage is preliminary analytics, or, as we call it, Pre-discovery. As a result we should get a minimum that would allow us to commit: proposed solutions, risks and dependencies, non-functional and infrastructure requirements, user maps, architectural schemes. In addition, we ask the teams to give a preliminary estimation in story points at the feature level — this will allow us to determine priorities at the end of the quarter.
A personal Kanban Board is created at this stage for each Initiative, in which features and stories can be seen, with a preliminary link to a specific iteration, which is provided by a separate JIRA workflow. So by changing the status we can assign the issue to some iteration. Swimlanes in Kanban Boards are configured by development teams. Since our JIRA ecosystem is rather complicated, during Pre-discovery and Discovery we create tasks in separate projects in order not to litter the Teams’ backlogs:
Also at the Pre-discovery stage, we involve Architects or, as it’s been often practiced lately, their trusted people — “EA Ambassadors”. Their task is to convey the vision of the architectural department to all the participants, to help find the best solution, and finally to approve this solution with the company's Head of Enterprise Architecture.
When all the people involved believe that the Initiative has been analyzed enough and is ready for the next step, it is moved to the next status — Portfolio Backlog.
At this stage it is important to conduct a WSJF prioritization (Weighted Shortest Job First). To do this, we have a big meeting 3 weeks before the planning, which, unfortunately, up to now has always lasted many hours. During this meeting members of the Management Board evaluate the Business Value, Time Criticality, Risk Reduction and Opportunity Enablement with the help of Fibonacci-aligned planning poker:
To be able to track the history of Initiatives, for each of them I add a label in JIRA with quarter data: 2018Q4, 2019Q1, etc.
Looking ahead, let me describe all the possible statuses. After the final commitment at PI Planning, Initiatives successfully taken into work are moved to the Implementing status, while those ‘not fitted’ receive Parked status and might be considered for the next quarters. Delivered initiatives are moved to the Done status.
As a result, we have the following picture on the Kanban Board:
Of course, we are not even in the middle of our way yet, but at the moment I can already note that thanks to the Portfolio Backlog implementation
- Business owners have begun to work through their Initiatives in detail and thoroughly prepare for the Review.
- The Reviews have become more meticulous in a good way.
- Teams have more time for Pre-discovery.
- I do not lose old Initiatives — I can always go back to the Parked, not delivered or forgotten Initiatives and work on them further.
Tools used at this stage:
- Atlassian Jira Software Server
- Plugin Colors for Jira — for highlighting Business and Architectural Initiatives
- The ‘Structure — Project Management at Scale’ plugin — to visualize the structure made of Initiatives and features belonging to them, and their WSJF prioritization
- Atlassian Confluence — source of internal documentation
- Lucidchart and its plugins for Jira and Confluence — for diagrams drawing
Part I. Preparation for PI Planning
A month before PI Planning is when the busiest time comes. Too many things need to be taken into account, and in order not to leave anything out, I have to create a multipage “logistic” Google Form. Let me describe the most important tabs from this Form and the activities around them.
Feedback. A few days after each PI Planning we conduct a Retrospective, which helps us not to forget what conclusions we came to and how we need to adjust the course. This is an important part in terms of continuous improvement.
Technical preparation. With transition to remote planning, specific requests have appeared, such as quality of Internet connection (prioritization and optimization of routes for JIRA and Confluence) and video- and audio presence (we use the Logitec Group kits, Jabbra microphones and personal headphones in various combinations, webcams, Zoom software for video conferencing).
Facilitators. It’s a list of employees responsible for facilitation in each of the working groups, indicating their location and a link to the permanent Zoom-channel of the working group.
Audience. The complete list of all invitees.
Checklist. The full list of important tasks with deadlines and responsible persons. To give you some insight below you can find several examples of the most typical and vital tasks relevant for each PI Planning: training of Facilitators (a training manual has been prepared with all the necessary steps such as organizing a working team, timing the meetings and decomposing the Initiative into a list of features); update of working groups’ location plans for each office; control of the Velocity update for all development teams; ordering meals; creating reports from previous quarters; printouts of lists of Initiatives and schedules.
Part II. PI Planning and the process of Commitment
After all the preparatory running around, this moment finally comes — the start of PI Planning. In two days we have to discover all Initiatives, make sure that the information is sufficient and commit. After a few pep talks the working groups go to their places and get to business. At the moment the number of groups is distributed among the three offices as 4x4x2, and remote users are connected to the required group via a dedicated Zoom channel.
Upon completion of the Discovery on each of these Initiatives, the facilitator makes sure that all the necessary data, such as Acceptance Criteria, technical solution, risks, dependencies, the necessity of a new infrastructure, is available and marks the Initiative with the ‘IT Session Finished’ checkbox for easy tracking of the progress. After that the working group can move on to the next Initiative.
After a dozen of PI Plannings the feeling of being under constant stress and time pressure, which was with us from the very beginning, has significantly faded, and now it’s all more like work as usual. In the afternoon on the first and second days of the Planning it’s the time for unhurried commitment according to priorities. To perform a commitment, I have several separate structures. The first one is a structure containing a list of features and stories scheduled for the commitment:
Unfortunately, this structure on the screenshot contains only tasks that did not fit into the commitment of the current quarter, so the sample is unrepresentative. In any case, in the search field I can select the Initiative needed in order of priority by number, which is put in a separate field for each feature and story, change the issues’ status from Planned to Committed and put them to the desired iteration, thereby committing for them:
As a result, the issue will disappear from this structure and appear in a new, which reflects the growing commitment:
As you can see on the screenshot, in this structure the stories fall into the folder of the development team to which they belong and are grouped by iteration. Here, I see the total Velocity of the team in the folder name, and in addition in the Commitment column, the overcommitment is automatically determined and highlighted for each team, by using a custom formula.
Of course, if the first and highest priority Initiatives are included into a commitment mostly easily, soon, as more and more teams fill their backlogs to the end, there could be respective problems with some of the initiatives, after certain arguments and discussions, being postponed (parked) as a result.
At the end of this rather straightforward process the teams swear to deliver their commitment on the company's flag (ok, not really) and hurry to their homes (well, again, not really, mostly they flee to some bar to celebrate).
Tools used at this stage:
- Atlassian Jira Software Server
- The ‘Structure — Project Management at Scale’ plugin – to monitor the Discovery process and during the execution of the commitment.
Part III. Cloning issues into the working JIRA ecosystem of the Company
While everyone is drinking, the RTE is working on creating commitments in the form in which it could be distributed to the development teams’ backlogs and monitored throughout the quarter. To do this, I use the 'Bulk Clone Professional for Jira' plugin — it adds an item doing collective cloning to the Bulk Change menu and has such necessary features as links cloning, updating links between newly created clones, updating Epic links, adding Summary prefix and automatic labeling:
I add status as a prefix, because at the planning stage the statuses reflect the planned iteration of completion. As a result I can see right away in the Summary when we are planning to finish the feature or story.
At first, I clone absolutely all issues into a separate Global Backlog project, since we keep epics in it, and I need to have actual epic-story connections in newly cloned tasks. And already as a second step, I make JQL-requests for each development team separately and move the stories into the working projects of the teams.
As a result, I create a structure that reflects the current Commitment and consists of final stories, which I then use for monitoring:
In general, the advantages of this approach are the following:
- It is easier for an IT guy to type new features and stories rather than to write them down with a highlighter on a sticky note.
- Many things, such as capacity remains and WSJF update depending on the stories estimations, are calculated automatically using custom formulas.
- Thanks to cloning, the original Commitment is preserved for history and we can always return to it.
- It saves us both time and energy at the stage of planning preparation — paper handling takes energy.
- Of course, it's great that we no longer need to add issues to JIRA by typing in sticky notes.
Tools used at this stage:
- Atlassian Jira Software Server
- The ‘Bulk Clone Professional for Jira’ Plugin– for cloning features and stories into working JIRA projects.
- The ‘Structure — Project Management at Scale’ plugin – for the creation of the final Commitment structure.
Epilogue
Dear Friends, of course I understand that the above overview is rather superficial, and a lot of things could be revealed in a more detailed way — monitoring distribution of capacity among Business and Architectural initiatives and operational tasks, calculating custom formulas in structures, managing risks and much more. But I still don’t know whether this topic is interesting to the audience, and if yes, than what exactly. If I see any interest, I will be glad to share my insight on the relevant topics.
And one more thing — it’s hard to believe it would be possible, but still I would really like to avoid butthurt related to Agile, frameworks, effective and “effective” managers in the comments. Please remember that I described the technical part of the whole process in the hope of getting interest of the people who are close to the topic, and I look forward to relevant discussions.
See you in the comments!