Better software through software architecture and devops

@jamessnape

Posts

  • Close up shot of two kings being thrown onto a poker table

    Estimation is hard and the easy path sometimes a little too tempting.

    Introduction

    The ability to estimate something is a core engineering skill. In software estimation is mostly about how long something will take to complete. I know lots of developers say they are terrible at estimation and will do anything to avoid giving a number. I think a lot of this has to do with our desire to be correct all the time but refusing to estimate isn’t helpful; you just need to accept that sometimes your estimates may not be accurate.

    Relative vs Absolute Sizing

    Humans in general are not great at working out the absolute size of something whether it be the length of a stick in meters or a task in hours. A good example of this is - quickly tell me how many characters are in the quote below?

    #############################

    Its quite hard to tell quickly by not counting individual characters. I tried by halving the string and then again and again until I could easily count 4 then multiplied by the segments I divided. This was a quick estimate of 8x4=32. The actual string length is 29 so I’m out by 3, not too bad.

    What humans are very very good at is estimating the relative size to something else. For example, is the second string longer, shorter or the same size as the previous one?

    ##############################

    What about this one?

    ############################

    Even though they are not right next to each other I’m sure you found it easy enough to guess they are longer and shorter respectively, even if its only by one character.

    When it comes to story estimation, we find it much easier to estimate by comparing to another, similar story and this is where planning poker often fails.

    Planning Poker

    I’m sure everyone knows how planning poker is supposed to work. Briefly its a little like this:

    1. Product Owner gives an overview of the story to be estimated
    2. The team ask questions to clarify assumptions and risks
    3. Each team member picks a card to represent the estimate for the story (commonly fibonacci story points)
    4. Everyone turns over their card
    5. The highest and lowest card holders get an opportunity to justify their position
    6. Repeat the card picking and justification until a consensus is formed

    There is even a pretty good extension for Azure DevOps to do all this remotely.

    This should be a good way of estimating as it depends on the wisdom of crowds to come to a consensus. My issue is that by default this is an absolute estimation method. Even worse, its an absolute estimate of a non-specific measure - story points.

    The part that is missing from most planning poker sessions is a menu of stories for comparison. This is especially true at the start of a project and the shear number of stories that need estimation create more temptation to take shortcuts.

    An Alternative

    My preferred method is to estimate all stories in one go by arranging them relative to each other. This is known as Affinity Estimation.

    The first stage to to add a bunch of cards to a big board and then, one-by-one, align them based on their relative complexity. You have to be careful to take input from everyone and ensure decisions are not biased by others. The same level of justification takes place over positioning. At no point do you assign an absolute value.

    Initial sorting

    If a story isn’t ready or its not possible to estimate I colour the card red and move it away for later refinement and splitting.

    You can put all the stories in a long line and just play higher or lower until the complete list is in relative size order but I find its easier to roughly group items and only place them in new columns when they are obviously bigger or smaller.

    Once you have a rough grouping its time to make some decisions over the items not obviously in one column or another.

    Bucketing

    Finally, check that each column represents stories that are approximately twice as big as the previous column. Only then can you assign your story point values.

    Assigning values

    This leaves you with a final block of estimations to paste back into Azure DevOps.

    Final estimations

    Don’t throw this board away. When you come to estimate the next bunch of stories if you add them to this board you already have a ready made set of reference stories. As it grows bigger you might want to prune it and possible remove any that ended up not being representative of the effort assigned originally.

    These reference stories can also be the basis for relative planning poker sessions in the future.

    Summary

    Whilst I’m a fan of #NoEstimates and Lean methods which drastically reduce the important of estimating, we all have a responsibility for estimating our work. Planning Poker isn’t the only game in town and its often abused so why not consider alternate ways of sizing your stories. Maybe try White Elephant Sizing once in a while too?

    Photo by Michał Parzuchowski on Unsplash

    This entry was posted in agile  and tagged #estimation #planning poker #opinion  on .
    Discuss this on Twitter or LinkedIn
  • Scrum Master with kanban board.

    A little bit of styling can really help with your process. Using card colours can remove motion waste and save you time searching for things. Highlight, don't hide and run stylish stand-ups.

    Introduction

    Following on from my previous post there is much more styling you can do in Azure Boards. My favourite thing to do is to really lean in to the process with styles. Take daily stand-ups, they happen at the same time every working day and typically ask the same three questions:

    1. What did I complete since the last meeting?
    2. What will I do before the next meeting?
    3. What blockers and other issues are standing in the way of the sprint goal?

    All of these questions should be answered in the context of the sprint goal so lets make that really clear using a marketplace extension: Azure DevOps - Sprint Goal. This creates an extra tab at the top where you can keep your goal in mind.

    Next, remember that the reason for the stand-up is to discuss the sprint work items and their progress not focus on the team members doing the work. To that end I prefer to Group By: Parents so that work items are visible on the left.

    Group By Parents

    This does mean that some team members might speak multiple times as each item comes up but the order of discussion is in backlog priority. If you want to quickly see all work for a person use the person filter at the top.

    This is what a typical board might look like (note the sprint goal in the tabs at the top):

    Typical task board

    Even without seeing the detail I can tell this board has:

    • one task recently completed [green]
    • a task that is blocked (and its corresponding work item also blocked) [red]
    • a task where someone is requesting help [blue]
    • a task that is too large and needs to be split into something smaller [purple]
    • a task that has not moved recently and could be blocked [yellow]

    There is also an unrealistic amount of work scheduled for today since there is 12 hours of task in progress.

    How does this help with the standup? Lets go through the questions above, one by one. But first, these depend on having your stand-up at the same time every day. Styling based on date fields is a bit of a blunt instrument so if you vary your standup time they won’t work.

    What did I complete since the last meeting?

    For this I want to highlight tasks that were moved to the done state since the last meeting.

    Tasks done since last meeting

    • State = Done
    • Closed Date > @Today-1 (this is the magic, since yesterday selector)

    During the standup you can concentrate on your green tasks for this question.

    What will I do before the next meeting?

    Most people move tasks to in progress when they start working on them. I prefer to use it as a today view and place all the work I intend to do today in this bucket. That way I can see that:

    • I’m not taking on too much work because the sum of hours should be less than a working day. Typically 6 or 8 hours depending if you factor in some slack time to cover non-project work.
    • If at the end of the day there are tasks left I can make a call to work a little longer or accept the impact.
    • Similarly, if I finish early then I can also make a call to pull more work into the in progress bucket or see if anyone needs help on their tasks.

    Before I start moving tasks I want to make sure they are all small enough to fit into a single day’s work. This style rule shows them as [purple]. Its purple because I associate the colour with royalty and they are too posh to actually do any work.

    Tasks that are too large

    • Remaining Work > 8 (or use 6 if you factor in slack time)

    What blockers and other issues are standing in the way of the sprint goal?

    This last question uses the remaining style rules. We already saw how to highlight blocked tasks:

    Blocked Task Rule

    • Blocked = Yes
    • State = In Progress

    and blocked backlog items:

    Blocked Backlog Item Rule

    • Work Item Type <> Task
    • Tags Contains blocked

    But there are other styles that can highlight blockers. Firstly, if a task has not moved from the in progress state in the last day then something must be wrong since they are all supposed to be less than a day in size. Note, if you decide to stop working on a task and defer it to a later date I encourage you to update the remaining work value and move it back to the to do pile to keep the in progress = today list correct.

    To highlight these stuck tasks the following rule works.

    Task In Progress To Long

    • State = In Progress
    • State Change Date < @Today

    Finally, the [blue] task is simply a team member adding a help-needed tag. Blue represents ‘emergency services’ in my head. Since the goal of a sprint is to complete work and not start work, I prefer that the team works as a team instead of a bunch of individuals working in their own areas. To that extent, its always better to help someone else complete a backlog item than it is fo you to start something that potentially won’t get done in time. This is especially true in the second week of a two-week sprint.

    Task with Help Needed

    • Tags Contains help-needed

    Summary

    Just a quick note on colour blindness. I know many folks might have trouble differentiating the colours I’ve used here so work with your team to identify combinations of hue and brightness that allows everyone to visually distinguish each style.

    I really like using styles to highlight areas for action. If you have a style rule not listed here then I would love to learn about it.

    Photo by Parabol | The Agile Meeting Toolbox on Unsplash This is clever marketing by offering great images for free so when attribution is made then your company name shows up in relevant web sites. However I am not making any recommendations here; only attributing the photo.

    This entry was posted in agile  and tagged #lean #azure devops #azure boards #card styles #blocked #opinion  on .
    Discuss this on Twitter or LinkedIn
  • Brightly coloured graffiti decorating the High Street Subway in Belfast City Centre.

    Tags are the simplest of concepts in Azure DevOps and other work item tracking systems. When you combine them with visual styles they become a powerful user experience feature.

    Whilst I was researching content for Task, state, column or work item? I came across Promoting “Blocked” to State from Status on Agile Board for better visibility which, IMHO, is absolutely the worst way to solve the problem of blocked work items. One of the comments even mentions creating a swimlane to put blocked items in which is especially bad since you can collapse the lane and forget about all your problems.

    There are a number of reasons why a blocked column is a poor choice most of which discussed here:

    Using my rules for states also implies that the only thing you can do to a blocked work item is to unblock it which isn’t the case.

    A far better way to indicated blockage is with tags. Leave the work item where it is and add the blocked tag. Don’t forget to add a comment as to why its blocked and, if possible, some tasks for getting it unblocked. The plain visual doesn’t stand out much but you can apply some colour either as a tag colour: Blocked tag colour dialog or card styling: Blocked card style dialog

    which looks like Blocked card … no missing that.

    The same principle also works for tasks on on the taskboard but instead a field called blocked is used instead. Blocked task

    Tags are so flexible you can use them in all sorts of ways. Some ideas for you:

    • Forgo the initiative backlog level, its always got only one or two items in and never any useful description or acceptance critera. Instead just tag your work items.
    • Use tags to differentiate different types of work item such as story, question, decision etc. This is more often used in the basic process.
    • Use tags to call out things - needs-discussion, ux-required, out-of-date, expired, etc

    Unfortunately, since tags have very little governance they can get a little out of hand. I use Tags Manager to merge misspellings and other issues.

    Photo by K. Mitch Hodge on Unsplash

    This entry was posted in agile  and tagged #lean #azure devops #azure boards #card styles #blocked #opinion  on .
    Discuss this on Twitter or LinkedIn
  • The look of someone who is not where he is supposed to be. My dog Thula looking rather sheepish as a puppy.

    It can be confusing knowing how to represent something in Azure DevOps. Should it be a work item? What about a task or new work item state? Here is my guide for you to classify with confidence.

    Introduction

    I see a lot of different interpretations on how to classify content in Azure DevOps. What makes something a work item? What is the difference between a state and a board column? I mean why am I even discussing states and columns in the same post as tasks and work items? Let’s dive in and look at each individually.

    Work Item

    The primary use for a work item is to capture something of value that needs to be delivered. If you were to describe the item to your customer and they respond with disinterest in you working on it then maybe you should consider one of the other three. I choose the work item level depending on the type of value:

    • Epic: it is a waste to just call these ‘large stories’. Instead consider them as goals or capabilities you want to deliver.
    • Feature: a complete chunk of value your end customer will understand. Imagine writing a user guide, these are the sections that users will search for.
    • PBI: might not all be user stories but most are and since stories are small and negotiable they tend to be considered a bullet point on a feature for end users. This is especially true once you have split stories to fit in sprints.

    Work Item State

    I don’t think folks pay much attention to states and generally treat them as a set of steps that work goes though until its done. This gives rise to my pet hate - the testing state, the idea that you don’t do any testing until the work has cleared development.

    I prefer to think of work item states in the same way as state machines:

    • A work item can only exist in one state at a time (obviously).
    • A state defines the set of events that can be handled and actions that can be performed.
    • Events determine transitions to other states.

    This is why, for cross functional scrum teams, the four states should be all that are needed:

    • New: candidate ideas, not yet on the backlog.
    • Approved: work items that are on the product backlog.
    • Committed: work items that match the sprint goal and are on the sprint backlog.
    • Done: work items that meet the done criteria and are potentially shippable.

    A cross functional team will perform all sorts of tasks on a work item in a committed state - writing code, updating documentation, writing test automation, performing exploratory testing, bug fixing, merging pull requests, etc. All of these happen iteratively and often concurrently.

    If your states represent the steps in a waterfall process then you are doing mini-waterfalls.

    More states equals longer cycle times - I once had a customer who had 20 states from new to done and was releasing work quarterly. By the time we finished transforming their process we had 4 states and releasing weekly.

    So where does it make sense to create new states? The only times I’ve needed to are for integrating into a wider, non-agile, organization at a feature or epic level. For example, if you have mandatory a hand-off of features to other teams - for example quality assurance then adding a new state at the feature level is possible.

    Board Column

    Board columns are a little like states but are only visible to the team that owns the board. For this reason the rules governing their use can be a little more relaxed but less is still better. An example of board columns I use regularly at feature or epic level is to rename the in progress column to now then add next and later columns. This gives a simple roadmap view of the future but its no different than using iteration paths.

    Task

    You have probably figured out by now that I think nearly everything that isn’t of identifiable value to the customer should be a task. Personally, I use tasks for everything. If its not written as a task I invariably forget about it. Plus the little dopamine hit you get from ticking a task off feels great throughout the day.

    Testing…task; writing documentation…task; reviewing a PR…task; updating a build script…task; if in doubt…make it a task.

    The one time this sometimes isn’t so simple is for bugs. If you find a bug in a PBI before its done is it really a bug? I wish Azure DevOps would allow you to create bug-tasks like JIRA that work like tasks but visually appear as bugs. Until then I prefer to create another task but style the card on the task board. Bug work items are reserved for defects found after the PBI has been completed and the sprint ended. This differentiation also helps calculate the escaped defect ratio for sprints.

    To aid with all these tasks there is a fabulous extension called 1-Click Child-Links but sadly I don’t think its been maintained and there are now plenty of issues. Alternatively using the az boards work-item command line can alleviate some of the grunt work.

    Finally

    Hopefully the above provides a little bit of guidance on the use of various Azure Board features. What I haven’t talked about are tags which are incredibly useful when it comes to removing columns and states. More on this in a later post.

    Photo by James Snape on Flickr

    This entry was posted in agile  and tagged #azure boards #azure devops #opinion #work items  on .
    Discuss this on Twitter or LinkedIn
  • Grey spiral staircase looping upwards to a small patch of blue sky at the top.

    An early task for your new project it so configure an iteration path. Going a little further than the standard iteration model can help with planning later.

    Introduction

    The online documentation for iteration paths has a graphic which could cost you some time in the future. Care to guess?

    Example release and sprints

    In case its not obvious, the sprint numbering starts from 1 again for each release. In my experience releases can move around but you really want sprints to be a consistent drum beat for the project.

    Sprints

    This first question you need to answer is how long will a sprint be? You shouldn’t be changing the length at all so picking a length can be tricky. The scrum guide says this:

    Sprints are the heartbeat of Scrum, where ideas are turned into value.

    They are fixed length events of one month or less to create consistency. A new Sprint starts immediately after the conclusion of the previous Sprint.

    A single week is less common but useful where you are not delivering software and have very short deadlines; though at this point maybe a simple kanban would be more efficient. The most common length is 2 weeks. Microsoft DevDiv adopted a 3 week duration. 4-weeks seems a little too long for me, long enough to be doing mini-waterfalls which is to be avoided.

    I once heard somewhere is the length of a sprint should be as long as you can go and risk going in the wrong direction. Since feedback from customers only comes at the end of each sprint the opportunity for course correction is similarly timed.

    Personally, if you are delivering software I would pick two weeks unless you can make a great case for three. This is especially true if you are new to agile - you really want that feedback regularly.

    I would also recommend that you don’t start/end sprints on Mondays or Fridays - these days are often when folks are not around for various reasons. Plus, who wants to spend all day in sprint meetings on a Friday.

    You can now go and create a bunch of sprints in Azure DevOps. Create a nice long, but flat, list of sprints at the project level with start and end dates.

    Releases and other Milestones

    Only after you have your sprint list should you apply any hierarchy to it. A sprint should deliver potentially shippable code but often not released. A release is a more formal affair. Scheduling with milestones can help with planning at epic or feature level for larger projects.

    For large, long running product deliveries your planning will reach further out than your PBI backlog. At range, things get fuzzy so trying to plan in detail is a futile waste of effort. Remember that any work items specified but not delivered are considered inventory and incomplete work depreciates rapidly with knowledge and feedback gained along the way.

    To help, I’m a fan of multiple levels as shown below (also from Microsoft). At the top you have a strategic outlook for the next 12 months. These consist of epics only. Below that a roadmap of features you plan to deliver over the next six months. PBIs don’t start to appear until the quarterly plan which is your main backlog. Finally tasks are created during sprint planning.

    Sprints and Plans

    However you break this up you will have a hierarchy of iterations similar to the following:

    • Vision-2024
      • Roadmap-2024-H1
        • Plan-2024-Q2
          • Sprint-1
          • Sprint-2
          • Sprint-3
          • Sprint-4
          • Sprint-5
          • Sprint-6
      • Roadmap-2024-H2
        • Plan-2024-Q3
          • Sprint-7
          • Sprint-8
          • Sprint-9
          • Sprint-10
          • Sprint-11
          • Sprint-12
        • Plan-2024-Q4
          • Sprint-13
          • Sprint-14
          • Sprint-15
          • Sprint-16
          • Sprint-17
          • Sprint-18
    • Vision-2025
      • Roadmap-2025-H1
        • Plan-2025-Q1
          • Sprint-19
          • Sprint-20
          • Sprint-21
          • Sprint-22
          • Sprint-23
          • Sprint-24
        • Plan-2025-Q2
      • Roadmap-2025-H2
    • Vision-2026

    With a structure like this you can throw epics and features into a future planning bucket without worrying which iteration. This is a bit generic though; far better if you have catchy names for the vision, roadmap and plan levels since its easier to remember which one is current.

    Sprints won’t neatly fit into calendar boundaries like quarters, halves and years but its not a big deal. Just put an extra sprint somewhere or one less to keep them from drifting. Make sure the numbering is sequential still. Teams just work with sprints and its confusing if the sequence jumps.

    Also, there are no releases on this list, just planning windows. Releases are orthogonal to iterations and move around a lot so the release a work item is completed in may not be the one originally intended. You can use build and release numbers for tracking this instead.

    Since teams can use different iterations then unfortunately you need to select iterations at a team level too which just needs a lot of clicking. Go click one more time on Team Configuration Option: Auto Populate Team Iterations with Project Iterations if you would rather have them auto populated instead.

    If you are going to be doing a lot of this then scripting the creation of iterations is a way to save some time.

    Summary

    Usually iterations don’t get much thought at the start of a project and end up being rearranged as you need but they can be a useful way of planning deliveries at just the right level.

    Photo by Henry & Co. on Unsplash

    This entry was posted in agile  and tagged #agile #Azure DevOps #opinion  on .
    Discuss this on Twitter or LinkedIn