background job execution
Mariusz Jakubowski
I was talking with Maxence about background job execution. I'm sending
this email to discussion list so we all can share ideas about it.
My ideas of fixing ProgressDialog:
1) extract the run() method from ProgressDialog to a new class (JobInfo)
2) this new class will make computations for the job progress
3) these computations should be accessible (create fields and getters instead of local variables)
It will make GUI update possible not only on ProgressDialog, but also on background job execution monitor (I'm thinking about JTable here)
4) synchronize calls when they should be synchronized
I'm thinking about three possible solutions for threading
1)
a) job thread
b) job monitor thread (making job progress computations)
c) EDT (Event Dispatch Thread)
There should be synchronization between (a) and (b) - job monitor
thread should update job progress info (JobInfo)
and between (b) and (c) - GUI should be updated in EDT with new values.
This situation is similar to the one that currently is implemented but changes in the GUI are now made in (b) and should be made in (c).
All computations should be separated from all GUI updates, and GUI updates should be scheduled using SwingUtilities.invokeLater.
If we use JTable and TableModel it can get tricky.
2)
a) job thread
b) EDT with swing.Timer for job monitor
we need only one synchronization between (a) and (b)
Job monitor will not be in a separate thread but will be scheduled using the swing timer. One loop of the current run() method will be executed for one
timer event.
The task from swing timer runs in EDT thread so no sync will be needed for updating the GUI - after calculating new values we could update GUI elements.
The computations performed in the job monitor are not expensive so they can be run in EDT thread without a user noticing it.
3)
use SwingWorker (it requires Java 6), I have not investigated it further.
I've created quick prototype for (1) but now I think that solution (2)
is better.
What do you think about these solutions?
This is only "job progress" part, a first step to introduce background
jobs. There are also other issues (e.g. FileJob <-> FileTable sync). I
want first to solve this "progress" task and then look into other
issues.
Best Regards
Mariusz
Maxence Bernard
Thanks for taking the time to detail the changes you have in mind. I
meant to reply earlier but got hit by a nasty flu, which rendered my
attempts to work on late hours useless :)
I agree that it's a good idea to move the job progress computation
code out of ProgressDialog#run() and into a new class. This class
should be agnostic of the component that will render this information:
right now we only have one such component (ProgressDialog), but we'll
eventually have more, for instance one that displays the progress of
all jobs on a single window (akin to Firefox's 'Downloads' window).
Regarding the threading model, using Swing timers (2/) makes perfect
sense for periodic painting and should be easier for us than having to
manage EDT scheduling ourselves (1/).
SwingWorker (3/) does not seem like a viable option yet, because of
the dependency on Java 1.6.
I also have some ideas for reworking the Job API and the FileJob class
in particular. Here are some of the issues I have with the current API :
- It is hard to factor and reuse code from one job to another, leading
to code duplication
- FileJob is both the abstract class to be extended to implement a new
job, and also the class that schedules and runs the job, which is
confusing and tricky
- Overall file progress is inaccurate (guess work at best), due to the
fact that the file structure is not known in advance / discovered on
the fly.
- FileJob implementations are a mix of file operations and user
interaction, i.e. there is no clear separation between UI and actual
file work, leading to lots of nested statements and complicated code
paths
- Job dialogs need to perform some I/O-bound file operations to
instantiate a FileJob, and that currently happens in the EDT, which in
certain (rare) cases can cause the application to freeze
Here's what I have in mind to overcome those limitations:
1) move the Job running/scheduling code to a new dedicated class
JobRunner
2) break down the actual job's code into smaller pieces: Joblets. A
Joblet would be a very simple interface, that would basically boil
down to a execute() method that performs a very simple and atomic file
operation, such as copying a single file, creating a directory,
setting a file's permissions, etc... A FileJob as we know it would
become a set of Joblets, which would need to be built in advance,
during a new 'preparation' phase that would be scheduled and executed
by JobRunner. Additional interfaces would be provided for some of the
more-advanced job features and implemented by Joblet implementations
on a case-by-case basis : Pausable (to allow the Joblet to be
paused), Monitorable (to allow the Joblet's progress to be
monitored) ...
JobRunner's mission would be to :
- execute the job preparation phase during which Joblet instances are
created
- execute Joblets one after the other
- interact with the end-user when an interaction is needed
- notify listeners when the job's state has changed (JobListener
interface).
All of this would happen in a dedicated thread as it is now. JobRunner
would be controlled (start / stop / pause / ...) by a UI component
like the current ProgressDialog.
Does that sound like a reasonable plan ? I can draft a proposal for
this, in which I would detail the signature of all these classes.
Note that concurrent job execution will also require an overhaul of
file API, which in certain places is not thread-safe. I can take care
of that part, since I know the ins and outs of the code and it's
probably not the most exciting thing to do anyway :)
I expect those changes to the job API to take a fair amount of time to
mature and stabilize. For that reason, I suggest we create a separate
0.9 branch where we experiment and break things for a while ;) , while
keeping the main 0.8 branch stable and yielding one or two additional
0.8.x releases. This 0.9 branch would then be moved back to the trunk
when it's stable.
What do you think ?
Cheers,
Maxence