Deadlines and Estimates in Agile

Time is a key resource in software development and we need to consider it in two fundamental ways:

(1) Total amount of effort measured by time goes into delivering software.

(2) Deadlines by which the software needs to be completed.

The project management triangle provides the insight that these two elements of time are intertwined. If attempt to complete a project quicker (reduce TIME), we need to exert more effort to do so (increase COST).

Project-triangle-en

But chances are that if you are developing software, you are not too bothered with project management theory but instead are using an agile methodology to organise your work.

However, to some degree deadlines and estimates cannot be avoided. If the costs resulting from this effort outweigh the benefits derived from the software, it is probably a good idea not to develop in the first place. Moreover, often external factors dictate a specific date by which a piece of work needs to be completed.

This article is a discussion of how deadlines and estimates fit within the world of agile. First I will briefly discuss in how far these are integral parts of agile methodologies. Finally I will include with a few recommendations of how best to accommodate the requirements of deadlines and estimate in a post project management world.

While there are many flavours in agile, I will focus my discussions here on Scrum and Kanban, given that these are often considered the two most popular implementation of the agile manifesto.

Deadlines

No one likes deadlines in software development since they are often either missed or lead to inferior software being produced as a deadline approaches.

Scrum is organised around sprints, with each having a start and an end date. The end date effectively functions as a deadline by which a key deliverable (increment) should have been completed. For Kanban, in contrast, deadlines are not part of core framework of how work is organised. It is possible in Kanban to assign due dates to individual tasks but this is not aligned with the primary objective of Kanban – which is to increase to continuous flow of tasks worked on by the team.

Thus Scrum cannot work without deadlines while Kanban might not work very well with them.

Estimates

No one likes estimates in software development since they are often inaccurate.

In agile, it is generally emphasised that estimates are not such a big deal. One way this is done is by assigning funny dimensions to estimates instead of duration in hours or costs in dollars (such as story points or t-shirt sizes). However, estimates are still important and a key part of agile methodologies. In Scrum, for instance, estimates are important for determining the product backlog (which features to implement and which order) and for sprint backlog planning (can all selected tasks be completed in the sprint). In Kanban, estimates are not as essentials as in Scrum. That is, it is possible to practice Kanban without estimates but not Scrum. However, it is still often sensible to provide estimates in Kanban in order to determine the order of work and to measure velocity.

How To Do It Right

We have seen that deadlines and estimates are still important factors to consider when doing agile software development. Granted, if one would like to place less emphasis on these, choosing Kanban over Scrum provides some reprieve. However, often external factors make it impossible to go without deadlines and estimates; and, indeed, there might be some benefits to these in certain circumstances.

Given the many bad experiences we have had with deadlines and estimates, one might be inclined to avoid these whenever possible. I disagree with this. Software is not developed in isolation; it is developed in the context of delivering value to someone and without estimates it is often difficult to assess whether value will actually be delivered. Moreover, often external deadlines cannot be avoided in software development projects – since these are mandated by business needs or other factors.

Thus I think we should overcome our antipathy to estimates and deadlines and, rather than trying to avoid them, we should ask ourselves how we can build them into our processes in the best way.

In this spirit, I provide a number of ideas of how we can implement deadlines and estimates within an agile environment in a way which is most beneficial for all stakeholders involved in the following:

Don’t Reward and Punish

The single most risk I see in deadlines and estimates is that they have the potential to negatively affect the culture of a team or organisation. Deadlines and estimates can be used as a way to exert power from the top to the bottom and reward and punish based on these. This, I think, is very dangerous. If teams must fear that they will be punished if a deadline is not met or an estimate is inaccurate, they will loose confidence and satisfaction in their work. This is simply because they would be rewarded and punished based on a measure which is not completely under their control. For instance, estimating effort for software development tasks is fiendishly difficult if not impossible to do, thus perfectly possible to get wrong. Furthermore, work might often prevented from being completed on time by parties external to the team.

Few and Big

Generally there is a cost involved with every estimate and deadline. The cost of deadlines is additional stress on the team as a whole and team members. The cost of estimates lies in creating and maintaining them but also, just as for deadlines, in the additional stress placed on individuals and teams.

Thus I think it is best to work with big and few estimates rather than with many small ones. For instance, rather than breaking a project up into many small steps, leave it as one deadline and one aggregate estimate (that is visible) – then let the team and individuals break it up into as many smaller tasks (with or without deadlines) as they choose.

Enshrine Team Ownership

In agile, it is usually left to the team to determine their own estimates. However this is an area where management is prone to interfere in the teams turf. For instance by questioning estimates or using them for determining the performance of teams. I think it is critical that teams don’t just get to set their estimates but are constantly reminded that these are their estimates and will not be used against them or without their approval.

Deadlines should likewise be determined in collaboration with the teams rather than being imposed on them. There should be a consensus when a deadline should be set rather than it being delivered top-down.

Track Proactively

Given we work with few and big deadlines and estimates, it is imperative that we are actively keeping track of weather deadlines and estimates are stay within acceptable boundaries. This is to assure that if things do go off the rails, we can react early in a way least disruptive to the work of the team. This most be done with great care though; specifically in a way that asserts the team that enquiries are solely to assist the team rather than to ‘control’ or to dote out punishments.

Use for Feedback Loops

While I believe, as said, that deadlines and estimates should not be used for performance tracking, I do think they can be very useful for building meaningful feedback loops. For instance, a team may track over time how accurate their estimates are or how well the team is tracking to achieve a big deadline from small, internal deadlines. These feedback loops should be kept free from criticism (or praise) – they are just means to keep track and improve ourselves.

Standardise

While team ownership of deadlines and estimates is very important, it is also important to try to standardise things were it doesn’t hurt. For instance, using one measure for estimates across the organisation, shouldn’t interfere with team autonomy too much (such as choosing T shirt sizes or estimates in hours).

Make It Easy

Lastly, I think it should be made easy for teams to work with deadlines and estimates. Ideally there should be easy to use tools which can be used across the organisation.  This aids communication across teams and makes it easier to move from one team to another. Moreover, data for feedback looks might be gathered across teams; always with the caveat that this should never be linked to performance management.

 

 

Task Management, Planning and Time Tracking

I’ve long been looking for a tool that allows me to plan tasks, manage them and track time for myself and for a small team. Unfortunately, I have not been very successful thus far to find a effective, robust and affordable solution for this.

The latest setup I am using is to use Checkvist for planning and managing tasks and timetrack for tracking time. However, unfortunately I found it becomes a bit cumbersome to plan and manage larger projects with Checkvist and there is no integration between Checkvist and timetrack.

Given these limitations with my current setup, I decided to once again look around for a new tool. Some of my key requirements are:

  • Tracking time per project and task
  • Being able to plan a large number of tasks. Specifically I often have to deal with a lot of ‘optional’ tasks, so tasks which might or might not have to be done.
  • Being able to track the completion status of tasks
  • Nice visualizations and dashboards to see how much time is spent on projects and tasks and how overall project completion is coming along.
  • Cross-platform support, in particular an option to access the tool through a web interface.
  • Ideally they offer a free plan or basic plan that allows creating unlimited projects and tasks
  • Exporting tracked time and projects

With that in mind, I explored the options in the following. To make a long story short, the only solution I could find which satisfied my requirements was MeisterTask.

MeisterTask

meistertask

MeisterTask is a task management solution which integrates with the mind mapping solution MindMeister. I have always been fond of mind maps, so I was excited to investigate this solution further.

I was delighted to find out that meistertask comes with Time Tracking. This works very well.

For planning tasks, I like the feature that it is possible to define relationships between tasks, such as ‘is blocked by’. If tasks, which block another task, are not completed, the blocked task is visually marked as being blocked:

blocked

Unfortunately, the free account of MindMeister is quite limited; it is only possible to create three mind maps or so. Also I found the feature to create tasks from mind map nodes not all that useful. However, I don’t think MindMeister integration is necessary to work productively with MindMeister.

The biggest disadvantages of MindMeister as far as I can see is that the Developer API is quite limited. For instance, I can’t find ways to retrieve and write time tracking data.

Asana

It seemed to me that Asana does not support time tracking. I tried setting up time tracking with Hubstaff, which seemed quite promising to me. Unfortunately, after creating a new task on a project in Asana, there was a significant delay until that task was available in Hubstaff. I think it is quite critical to being able to create new tasks and report time into them straight away.

Wrike

It looks like time tracking in Wrike is only available for Business accounts. These currently start at $24.80 pm which I think is a bit pricey.

Avanza

Avanza does not support unlimited number of projects for their basic accounts.

Trigger

Too many limitations on the free account for my liking!

 

 

Why the Ideal Task is Necessary, Rewarding and Fun

When it comes to organizing our lives, structured and logical approaches are in no short supply. Divide large projects into small tasks. Prioritize what is more important over what is less so. Identify the critical path of tasks leading most surely to success. All these are activities of the conscious and deliberate mind.

Some psychologist like to call this dimension of our mind System 2. System 2 takes care of conscious reasoning and helps us to think through things logically. As might be implied from the very designation of this system, there is a corresponding System 1. This system is generally responsible for subconscious and automatic activities of our brain; which helps us to drive cars and climb stairs without us having to ‘think’ about it explicitly. Some people also like to call these two aspects of our mind the conscious and the subconscious in a dramatic simplification of scientific terminology.

However, notwithstanding the terminology employed, System 1 and System 2 don’t get very far without each other’s help. If we would be consciously thinking about every action we have to take, we would move as fast as a computer from the 1960s rendering msn.com. If we wouldn’t be consciously thinking about anything … well, no need to think too much about that.

It is concerning then that modern methodologies to organize our work rest firmly on the principles of system 2. Anybody who has employed these systems on any significant scale certainly can relate to the picture of being stuck moving at a speed which feels like it could be so much faster. System 1 is undoubtedly the system with more raw power; even if this power is often not directly available to us. However, there are various ways how we can harness the abilities of system 1 – especially if we speak its language of values and emotions.

Thus, I have a proposal here how we could bring a bit of System 1 into the way we organize our tasks. Specifically, I propose that for every task we need to undertake we assess the following three dimensions:

  • Necessity: Does this task need to be done.
  • Reward: Do I expect that completing this task will feel personally rewarding.
  • Fun: Do I think it will be fun to complete this task.

Depressingly, I have little doubt that many of our To Do items will score heavy on the first dimension, necessity – while being less accentuated for the latter two. All the more reason to think carefully about these dimensions and asses their impact for our well-being and productivity.

Necessity

If we do not do what needs to be done, we won’t progress in life. Increasing the level of necessity of our tasks is an exercise in reduction. Cutting out the tasks which are really not that important to have more time for what really matters. There are many systems available to accomplish this, for instance the very popular Getting Things Done.

Reward

We all had these days were we are running around like crazy all day working on one important matter after the other. Only to be rewarded with the stale feeling at the end of the day that really we haven’t accomplished anything at all. On the other hand, sometimes we do something small and seemingly unimportant – such as sending a short message to our loved one or finally sending off this message to Aunt Mary -, which leave us with a feeling of grand satisfaction with ourselves and our life. A lot of the research in the field of Positive Psychology has looked into the various aspects which moderate what leaves us feeling good and what doesn’t.

To some degree it is our individual preferences which affect how rewarding a task feels. Is family the most important for you in your life? Then picking up your daughter from school will certainly feel rewarding. Do want to become the most successfully mergers and acquisitions lawyer in history. Then putting in another hour for the next big case will feel rewarding without a doubt. However, while we are usually quite good at figuring out which tasks will be necessary and which not when we put our mind to it (that is system 2, of course), we are often complete failures at figuring out what tasks will be truly rewarding. Also, just as necessity is not fixture and can change based on our goals, how rewarding something feels is not unalterable. It depends to a large degree on our attitude towards the task.

Keeping this in mind, I have assembled a few rules of thumb to make the tasks we need to do more rewarding and to figure out ways to identify tasks which are rewarding:

  • Do things for others.
  • Attempt difficult things.
  • Think about how a task relates to your life goals (Surely, there’ll be some connection).

Fun

We are usually quite good at indulging in things which are fun to do. Watching our favourite drama show with a good cold beer, go out drinking and bet on horse races, checking the latest gossip about Aunt Mary on Facebook – the opportunities are endless. Unfortunately, many activities linked to issues of greater import in our life, such as family chores and – behold – work, do not overlap with what we consider fun (the exceptions proving the rule).

Unfortunately, we are not great experts in fun either (that is after passing into the second decade of existence). Since, often what we consider fun in the beginning quickly turns into a habit – until we forgot why we started doing it to begin with (think watching television, drinking and betting on horse races and checking Facebook). Thankfully, how entertaining an activity is not only determined by what we do but also by how we do it. Clearing out the rain gutters just becomes this much more fun if you do it with your best buddy while wearing fish masks.

Thus, here a preliminary short list of ideas how any tasks can become more fun.

  • Do things with others.
  • Do things differently or with different tools.
  • Do new things.
  • Do little things in a weird and stupid way.

You might have guessed the punchline of this article from its subtle title: Key to a successful organization of your tasks is to maximize how necessary, rewarding and fun your tasks are. This would assure you are getting more done while being more productive and altogether more happy.

However, this is easier said than done. Thus, if we cannot find tasks that fall within each of these categories, at least we should try to have a healthy mix of tasks falling within the different categories. For instance, to assure that in a day or week, we have some tasks which are necessary but also some tasks which are fun and rewarding.