How to update project when reality differs from plan
Carl Worth
When I started looking for project management software I wanted
something that operates on plain-text files, supports resource
leveling, and generates reports through a command-line interface.
I was delighted to find TaskJuggler which meets all of my criteria, so
thanks!
My team and I have been experimenting with it, and we're quite happy
with what we've been able to do as far as planning tasks in the future.
But when it comes to shifting to actual tracking of tasks as development
continues, I haven't yet found a way to make things work for us, and I'm
wondering if I'm missing something.
As an example scenario, I've got a situation where in spite of all of
the cleverness of the TaskJuggler scheduling, one of our developers
completes work on a task which TaskJuggler doesn't schedule to start for
some time.
If I run a timesheet showing this work through TaskJuggler I get
warnings such as:
timesheet.tji:2: Warning: r1 worked more on tasks.t1
5.0d instead of 0.0d
timesheet.tji:2: Warning: r1 requests less remaining effort for task tasks.t1
0.0d instead of 5.0d
What am I expected to do at this point? The User Manual says:
Project managers should use the printed output of this command
to update the project plan accordingly.
But I'm not sure how to actually do that.
What I would like to do is to be able to increment the "now" value and
generate a report that shows this task that has just been completed in
the past. But everything I've tried still has TaskJuggler scheduling it
out into the future.
Any suggestions for me?
I will be happy to generate a minimal file if that would be useful, but
I feel like I'm missing something basic here.
Thanks in advance,
-Carl
Chris Schlaeger
> If I run a timesheet showing this work through TaskJuggler I get
> warnings such as:
>
> timesheet.tji:2: Warning: r1 worked more on tasks.t1
> 5.0d instead of 0.0d
> timesheet.tji:2: Warning: r1 requests less remaining effort for task tasks.t1
> 0.0d instead of 5.0d
>
> What am I expected to do at this point? The User Manual says:
>
> Project managers should use the printed output of this command
> to update the project plan accordingly.
>
> But I'm not sure how to actually do that.
Are you regularly freezing your project? If so, then all you need to
do is edit the booking statements in the generated file to reflect the
reported data. If the task has enough effort reported as bookings, it
will no longer show up in the future. If your plan contains a larger
effort number than was actually needed, you need to update the plan to
match the bookings and hence the reality.
Chris
Carl Worth
> > Project managers should use the printed output of this command
> > to update the project plan accordingly.
> >
> > But I'm not sure how to actually do that.
>
> Are you regularly freezing your project?
Thanks for the reply, Chris.
I'm not regularly doing anything yet, since I'm still investigating
taskjuggler and trying to find a way to make it work for us. :-)
Since I first asked my question, I think I've learned enough that I
actually have taskjuggler working now in a way that will fit the
lightweight workflow that my team wants. But I think the answer does
point to a place or two where perhaps the documentation could be
improved.
Here's where things stand for us:
* The timesheet and bookings mechanisms that taskjuggler provides are
a non-starter for us. These are all perceived as too heavyweight
for a couple of reasons:
1. Updating a task requires multiple steps
2. The timesheet/bookings workflow results in log-like
(history) information being appended to our source file.
My team is much more interested in a workflow where something like
"mark this task as complete" involved a single step of updating that
task itself in the input file. And any log-like information is
expected to be captured in our git history, and not the input file.
* Adding "trackingscenario plan" to our input made taskjuggler useful
to us, where previously we were just confused by its behavior.
We overlooked trackingscenario several times in the documentation
since it's documented as a mechanism to select "which scenario is
used to capture what actually has happened". This seemed irrelevant
to us since we are currently using only a single scenario.
But the side-effect of having "trackingscenario" present, (that the
scheduler will only schedule tasks in the future), was critical for
us. We were all very confused whenever now was advanced beyond the
project start date since taskjuggler would schedule things "in the
past" in a way which had no bearing on how things happened in the
real world.
I'm not sure what new users would expect, but everyone on our team
would have been less confused if taskjuggler had been in "projection
mode" by default. What's the rationale for having it otherwise? When
is that mode useful?
* With taskjuggler now in projection mode, we get the benefit of its
scheduling for future work. And we can log complete work in the past
by simply setting the "end" date for each task. That's nice and
lightweight and just what we want, (whenever you complete a task,
set the "end" to the date the task was completed and ensure the
"effort" number matches the time actually spent).
* The only thing we're now missing in our workflow is a convenient way
to get an in-progress task to span across the "now" boundary. It
would be nice if just setting "start" on the task, (to a date in the
past), did this, but apparently that's not compatible with the way
projection mode schedules only in the future. So there's a feature
request.
In summary, we're quite happy with our lightweight workflow now, (no
tasksheets, no bookings, but we still get an accurate record of tasks in
the past by setting "end").
Perhaps Chapter 7 of the TaskJuggler User Manual ("Day to Day Juggling")
could be augments to give more specifics about how to do a lightweight
workflow? It does already have encouraging text about lightweight
workflows:
Of course, there are projects that are done using strict project
management techniques that require detailed status
tracking. Both extremes probably have their fans and TaskJuggler
offers good support for both extremes as well as various
techniques in between.
But then the chapter goes on to describe how to do detailed status
tracking with timesheets, etc. and doesn't provide much on how to get by
with less.
All in all, though, we're really happy with TaskJuggler. It's does so
much more than other planning packages we've looked at, and is the only
one that gives us what we really want, (a task file managed with git,
yes!).
Thanks again,
-Carl
Chris Schlaeger
> Here's where things stand for us:
>
> * The timesheet and bookings mechanisms that taskjuggler provides are
> a non-starter for us. These are all perceived as too heavyweight
> for a couple of reasons:
>
> 1. Updating a task requires multiple steps
Unless each project participant is changing the TJ files, this step
will always involve at least two people and hence 2 steps.
> 2. The timesheet/bookings workflow results in log-like
> (history) information being appended to our source file.
>
> My team is much more interested in a workflow where something like
> "mark this task as complete" involved a single step of updating that
> task itself in the input file. And any log-like information is
> expected to be captured in our git history, and not the input file.
The bookings are vital for the scheduling. If you don't provide
bookings for your tracking scenario, TJ will assume that no work has
been done prior to now.
The status messages may or may not be relevant to you. But when using
them with journals in reports they can be quite powerful. It will tell
you who reported what and when. You basically get a log for each task
or person.
> * Adding "trackingscenario plan" to our input made taskjuggler useful
> to us, where previously we were just confused by its behavior.
>
> We overlooked trackingscenario several times in the documentation
> since it's documented as a mechanism to select "which scenario is
> used to capture what actually has happened". This seemed irrelevant
> to us since we are currently using only a single scenario.
>
> But the side-effect of having "trackingscenario" present, (that the
> scheduler will only schedule tasks in the future), was critical for
> us. We were all very confused whenever now was advanced beyond the
> project start date since taskjuggler would schedule things "in the
> past" in a way which had no bearing on how things happened in the
> real world.
>
> I'm not sure what new users would expect, but everyone on our team
> would have been less confused if taskjuggler had been in "projection
> mode" by default. What's the rationale for having it otherwise? When
> is that mode useful?
When you specify a tracking scenario, all work prior to 'now' must be
reported as bookings. This would get very confusing if your now date
passes the project start date and tasks start to shift as time
progresses due to missing bookings. If you use 'trackingscenario' you
must freeze your project regularly.
> * With taskjuggler now in projection mode, we get the benefit of its
> scheduling for future work. And we can log complete work in the past
> by simply setting the "end" date for each task. That's nice and
> lightweight and just what we want, (whenever you complete a task,
> set the "end" to the date the task was completed and ensure the
> "effort" number matches the time actually spent).
You will not have any resource allocations for those past tasks. The
reported effort will be 0 without bookings.
> * The only thing we're now missing in our workflow is a convenient way
> to get an in-progress task to span across the "now" boundary. It
> would be nice if just setting "start" on the task, (to a date in the
> past), did this, but apparently that's not compatible with the way
> projection mode schedules only in the future. So there's a feature
> request.
That's probably due to no freezing your project properly. Just a
guess, but there shouldn't be any problems with tasks crossing the
'now' date.
> Perhaps Chapter 7 of the TaskJuggler User Manual ("Day to Day Juggling")
> could be augments to give more specifics about how to do a lightweight
> workflow? It does already have encouraging text about lightweight
> workflows:
>
> Of course, there are projects that are done using strict project
> management techniques that require detailed status
> tracking. Both extremes probably have their fans and TaskJuggler
> offers good support for both extremes as well as various
> techniques in between.
>
> But then the chapter goes on to describe how to do detailed status
> tracking with timesheets, etc. and doesn't provide much on how to get by
> with less.
>
> All in all, though, we're really happy with TaskJuggler. It's does so
> much more than other planning packages we've looked at, and is the only
> one that gives us what we really want, (a task file managed with git,
> yes!).
The lightweight case mentioned in the manual is to use 'complete'
attributes. Your described workflow may work, but I'm not convinced
yet that it doesn't have some undesired side effects.
But that chapter is clearly not very comprehensive yet. I just don't
have time to work on it.
Chris
Carl Worth
> On Mon, Mar 12, 2012 at 11:51 PM, Carl Worth <cwo...@cworth.org> wrote:
> >
> > 1. Updating a task requires multiple steps
>
> Unless each project participant is changing the TJ files, this step
> will always involve at least two people and hence 2 steps.
Yes. The participants are changing the TJ files directly, and we want
only one step here. [*]
> The bookings are vital for the scheduling. If you don't provide
> bookings for your tracking scenario, TJ will assume that no work has
> been done prior to now.
Do I need to have actual "booking" entries for this? I seem to see
TaskJuggler handling these the way I want by setting an "end"
date. Maybe I just don't know what you mean by "TJ will assume that no
work as been done". (When I set "end", tasks do get scheduled prior to
now, and are charted as if 100% complete.)
> The status messages may or may not be relevant to you. But when using
> them with journals in reports they can be quite powerful. It will tell
> you who reported what and when. You basically get a log for each task
> or person.
We are now using journals for milestones, (to capture events that cause
slips, etc.). But for day-to-day changes to tasks, we're all much more
comfortable with the log data we can get with "git log". That's what
most of the team will use to examine history (beyond what's obvious from
looking at a report that covers some time in the past).
> When you specify a tracking scenario, all work prior to 'now' must be
> reported as bookings. This would get very confusing if your now date
> passes the project start date and tasks start to shift as time
> progresses due to missing bookings. If you use 'trackingscenario' you
> must freeze your project regularly.
As I mentioned before, we are using "end" to make completed tasks appear
in the past.
The bookings stuff doesn't look appealing to us as it's inherently
heavyweight, (lots of steps, file sizes growing as project progresses,
etc.).
The project freezing stuff doesn't look appealing to us as we really
don't stick 100% to the schedule produced by TaskJuggler. So it's
actually what we want for any tasks that haven't been manually marked as
completed to shift into the future. One of the most important things we
want out of TaskJuggler is "tell us what our schedule looks like for the
future work that isn't done yet" and that's exactly what we get if we
can manually move completed tasks to the past and let uncompleted tasks
get rescheduled.
> You will not have any resource allocations for those past tasks. The
> reported effort will be 0 without bookings.
It looks to me like I'm seeing resource allocations. For example, I've
got a resourcereport covering completed tasks, (all with "end" times
before ${now}), and that's got tasks listed for each user that has
allocated themself to a task and marked it with an end date. I'm seeing
effort values as explicitly entered by our users.
And it looks like we could get TaskJugglers view of the past to
accurately reflect reality if we also adjusted task effort values to
match how long a task actually took, (which, for our programming tasks,
we never know all that accurately in advance). So it seems we could get
a lighter-weight equivalence to bookings by simply addding "allocate",
and "end" and adjusting "effort".
As it turns out, our team isn't that interested in modelling the
past. Once a task is complete we don't want to think about it, (nor
account for exactly how many hours/days/weeks/whatever it took). So our
current plan is to just mark an "end" date accurately, leave the
"effort" as we originally estimated it, and let TJ's ALAP scheduling
create whatever fiction it wants to for the past.
That's not really ideal, but we'll probably mitigate by simply adjusting
our reports to not show much detail for the past. We could probably get
closer to the "true past" without much more work if TJ could be
convinced to over-allocate resources in the past. (For things like,
"Yes, I expected all 6 of these tasks to take 1 day of effort, but I
found that one bug-fix took care of all of them, so they all got fixed
on the same day". We don't want to bother with trying to parcel up that
day among those 6 tasks.)
> > * The only thing we're now missing in our workflow is a convenient way
> > to get an in-progress task to span across the "now" boundary. It
> > would be nice if just setting "start" on the task, (to a date in the
> > past), did this, but apparently that's not compatible with the way
> > projection mode schedules only in the future. So there's a feature
> > request.
>
> That's probably due to no freezing your project properly. Just a
> guess, but there shouldn't be any problems with tasks crossing the
> 'now' date.
What I mean is that setting an "end" date (before 'now') on a task will
make TJ schedule it at that end date (or earlier). But when setting a
"start" date (before 'now'), TJ won't schedule that task start before
'now', (as far as I can see).
> The lightweight case mentioned in the manual is to use 'complete'
> attributes.
Ah, OK. We did try that at first, and then found that it didn't do what
we want, (since it doesn't affect scheduling at all). For now, we're
completely ignoring the complete attribute.
> Your described workflow may work, but I'm not convinced
> yet that it doesn't have some undesired side effects.
Right. It does sound like we've found a way to use TaskJuggler that's
quite distinct from how it was intended to be used. It does appear to be
working for us, but it would be a shame if it suddenly stopped working
in some future version only because the current behavior isn't actually
intentional.
I hope I've successfully described the way we want our scheduling
workflow to work. TaskJuggler happens to be fitting the bill quite well,
and if that could become a documented rather than accidental situation,
that would be great! :-)
> But that chapter is clearly not very comprehensive yet. I just don't
> have time to work on it.
I understand that situation all too well. Thanks for your code and for
taking your time to talk to me about it.
-Carl
[*] Here's some background on what our team was doing prior to using
TaskJuggler:
Previously, the team was using a flat list of tasks in a
version-controlled wiki page. Common operations were "add a new task",
"sign up for a task", and "mark a task as complete".
We liked how lightweight that system was, but there wasn't any
task-dependency information nor prediction of how long things would
take. So my goal has been to find/create a system to add those things,
but with an interface that is just as easy as we had before.
I had started with inventing a custom syntax for maintinaing tasks,
dependency, and status in a text file. Then I was working on tools to
convert that to something like Planner's XML format, and had been
planning on writing command-line based frontends to Planner's
code. Fortunately, before I got very far down that path, I found that
Planner didn't support resource-leveling, and in the discussion of that
I found a reference to TaskJuggler. I was delighted to find it had a
task-description syntax as minimal as what I had come up with, (and
almost the exact same syntax other than the addition of '!' for task
dependencies).
Chris Schlaeger
> Do I need to have actual "booking" entries for this? I seem to see
> TaskJuggler handling these the way I want by setting an "end"
> date. Maybe I just don't know what you mean by "TJ will assume that no
> work as been done". (When I set "end", tasks do get scheduled prior to
> now, and are charted as if 100% complete.)
This took me some time to hunt down, but it turns out to be a serious
bug. TJ should never make any resource allocations in a tracking
scenario prior to the now date. Due to a bug in the scheduler, ALAP
with start and end dates were not handled this way. I've fixed that
now.
> As I mentioned before, we are using "end" to make completed tasks appear
> in the past.
>
> The bookings stuff doesn't look appealing to us as it's inherently
> heavyweight, (lots of steps, file sizes growing as project progresses,
> etc.).
>
> The project freezing stuff doesn't look appealing to us as we really
> don't stick 100% to the schedule produced by TaskJuggler. So it's
> actually what we want for any tasks that haven't been manually marked as
> completed to shift into the future. One of the most important things we
> want out of TaskJuggler is "tell us what our schedule looks like for the
> future work that isn't done yet" and that's exactly what we get if we
> can manually move completed tasks to the past and let uncompleted tasks
> get rescheduled.
Ok, I understand your use case now. But even before the bug fix, TJ
was not supporting this mode of operation completely. Tasks that
extend across the now date will not be handled as you would like.
Since they don't have a fixed end date yet, TJ assumes no work has
been done on them and all effort will be scheduled starting at the now
date.
>> You will not have any resource allocations for those past tasks. The
>> reported effort will be 0 without bookings.
>
> It looks to me like I'm seeing resource allocations. For example, I've
> got a resourcereport covering completed tasks, (all with "end" times
> before ${now}), and that's got tasks listed for each user that has
> allocated themself to a task and marked it with an end date. I'm seeing
> effort values as explicitly entered by our users.
As said, the before 'now' allocations are the result of a bug. It's
not consistent with the definition of a tracking scenario.
> And it looks like we could get TaskJugglers view of the past to
> accurately reflect reality if we also adjusted task effort values to
> match how long a task actually took, (which, for our programming tasks,
> we never know all that accurately in advance). So it seems we could get
> a lighter-weight equivalence to bookings by simply addding "allocate",
> and "end" and adjusting "effort".
>
> As it turns out, our team isn't that interested in modelling the
> past. Once a task is complete we don't want to think about it, (nor
> account for exactly how many hours/days/weeks/whatever it took). So our
> current plan is to just mark an "end" date accurately, leave the
> "effort" as we originally estimated it, and let TJ's ALAP scheduling
> create whatever fiction it wants to for the past.
>
> That's not really ideal, but we'll probably mitigate by simply adjusting
> our reports to not show much detail for the past. We could probably get
> closer to the "true past" without much more work if TJ could be
> convinced to over-allocate resources in the past. (For things like,
> "Yes, I expected all 6 of these tasks to take 1 day of effort, but I
> found that one bug-fix took care of all of them, so they all got fixed
> on the same day". We don't want to bother with trying to parcel up that
> day among those 6 tasks.)
You may not care about a high level of accuracy and detail for the
past of your project, but that's hard for TJ to deal with. The reports
allow for a very high level of accuracy and detail to be reported. So,
internally, TJ must always provide enough data to be able to generate
such reports. I've tried to make it as painless as possible to provide
such detail, but your mileage clearly seems to vary.
You can probably still use TJ your way after the bug fix, but any
effort reporting will only work if you provide exact data for when
which resource woked on what task.
> What I mean is that setting an "end" date (before 'now') on a task will
> make TJ schedule it at that end date (or earlier). But when setting a
> "start" date (before 'now'), TJ won't schedule that task start before
> 'now', (as far as I can see).
You mean it won't allocate any resources. Yes, and that's expected
since all resource activity before now must be provided via bookings.
> Right. It does sound like we've found a way to use TaskJuggler that's
> quite distinct from how it was intended to be used. It does appear to be
> working for us, but it would be a shame if it suddenly stopped working
> in some future version only because the current behavior isn't actually
> intentional.
>
> I hope I've successfully described the way we want our scheduling
> workflow to work. TaskJuggler happens to be fitting the bill quite well,
> and if that could become a documented rather than accidental situation,
> that would be great! :-)
I hope the bug fix does not negatively affect your workflow, but I
needed to fix that. I'll think about a way to specify the already
completed effort before the now date. The scheduler can easily deal
with this, but the reporting of any effort related values will be not
possible. Maybe I can find a way to contain this somehow.
Chris