GSoC Student- James

[James Legg]

Hello,
I am James Legg, a 2nd year undergraduate at the University of York
(UK), studying Computer Science and Mathematics. I would like to do a
Google Summer of Code placement on Hugin. I would like to create a
OpenGL preview window, as suggested at:
http://wiki.panotools.org/SoC_2008_ideas#OpenGL_accelerated_hugin_preview
I am thinking this should be an optional feature, that acts as a near
drop in for the existing preview, but hopefully faster and therefore
more interactive. I would also like the new preview to indicate what
part of the panorama an input image covers, so selecting which images
to display is easier.

I'm concerned because in the 3rd week of coding phase 1 I will be
sitting exams, so I will not be able to do half of coding phase 1!
However, after that point I will be able to work full time. Is this
acceptable?

I have done plenty of programming in c++ before, although mainly on
smaller projects written be me. I have been using Hugin and panorama
tools for 7 months now, for stitching panoramas, for creating HDR
photographs, and also I have used enblend to make seemless textures
from photographs. I haven't got any examples of these on the internet
yet, but I will attempt to upload some before the application
deadline.

I haven't submitted patches to hugin or other open source projects
before, but I see I'll have to contribute at least one patch before
being accepted: Here's a small patch for Hugin that sets the default
button and focusing on the HFOV dialogue (the one that appears when
loading images without the right EXIF lens data from the assistant
tab). With this, when the dialogue appears the user can type a
horizontal field of view without using the mouse, and also press enter
instead of clicking 'OK'. I don't think it was a reported bug, but I
find it an improvement in usability. This isn't much of a patch (it is
only two statements), so I may write another if I see something else I
can fix.

James

[Yuval Levy]

Hi James,

welcome to the community.

James Legg wrote:
> http://wiki.panotools.org/SoC_2008_ideas#OpenGL_accelerated_hugin_preview

I assume you have read the thread linked below and are aware of the
redundancies with FreePV.
<http://groups.google.com/group/hugin-ptx/browse_thread/thread/2c0c7530294b7057>

We welcome as many applications as students want to apply, but I suspect
there won't be place for more than one in the viewer's area.

> I'm concerned because in the 3rd week of coding phase 1 I will be
> sitting exams, so I will not be able to do half of coding phase 1!
> However, after that point I will be able to work full time. Is this
> acceptable?

last year we had a good experience and a bad one. It all depends on the
student. If you know you can't code during three weeks of coding phase
1, you should try to fit in those three weeks earlier in the schedule.
Just working three weeks less is not acceptable, and postponing the work
to after GSoC will reflect bad on both you and us, as the code delivered
on pencil-down date is what is being evaluated.

> photographs. I haven't got any examples of these on the internet
> yet, but I will attempt to upload some before the application
> deadline.

thanks. coding skills are more important at this stage than user skill.
If your time is short, spending it on demonstrating proficiency with the
code is a better use of it.

> Here's a small patch for Hugin [...] I don't think it was a reported bug, but I

> find it an improvement in usability. This isn't much of a patch (it is
> only two statements), so I may write another if I see something else I
> can fix.

Thank for the patch, it's a good start. I have a suggestion for you. The
dialog you worked on has another usability issue: if I load ten images
without EXIF data, it will pop up ten times. It would be nice if it had
an option to apply the data entered on the first picture to all pictures
being loaded. Can you do that?

thanks
Yuv

[James Legg]

Hi,

On 19/03/2008, Yuval Levy <goo...@levy.ch> wrote:
> welcome to the community.

Thanks!

> I assume you have read the thread linked below and are aware of the
> redundancies with FreePV.

I thought (I could be wrong) that the FreePV and Hugin's preview were
quite unrelated, and FreePV wasn't meant to support the requirements
that Hugin's preview would have, but I accept there is some overlap.

> We welcome as many applications as students want to apply, but I suspect
> there won't be place for more than one in the viewer's area.

I guess I'll have to find something else to work on, at least as a backup then.

> > I'm concerned because in the 3rd week of coding phase 1 I will be
> > sitting exams, so I will not be able to do half of coding phase 1!
> > However, after that point I will be able to work full time. Is this
> > acceptable?
>
>
> last year we had a good experience and a bad one. It all depends on the
> student. If you know you can't code during three weeks of coding phase
> 1, you should try to fit in those three weeks earlier in the schedule.
> Just working three weeks less is not acceptable, and postponing the work
> to after GSoC will reflect bad on both you and us, as the code delivered
> on pencil-down date is what is being evaluated.

I don't have a particularly busy term, so I'm sure I can fit some work
in before I have to divert my attention to exams.

> thanks. coding skills are more important at this stage than user skill.
> If your time is short, spending it on demonstrating proficiency with the
> code is a better use of it.

I'll upload the code to something I've created then, and continue with patches.

> Thank for the patch, it's a good start. I have a suggestion for you. The
> dialog you worked on has another usability issue: if I load ten images
> without EXIF data, it will pop up ten times. It would be nice if it had
> an option to apply the data entered on the first picture to all pictures
> being loaded. Can you do that?

There's another issue here. I tried this and realised that if you
enter different data in each time, only the first lens settings were
used for all pictures of the same size:
wxAddImagesCmd::execute() in wxPanoCommand.cpp asks for lens
parameters for anything without these on EXIF, and then overwrites if
it thinks the image uses an existing lens (which it guesses by make,
model, and focal length if it can get those, or image size otherwise).

I've written a patch that has the requested option implemented, and
also only asks when it will not overwrite it. This makes the option
slightly redundant (since most panoramas will be stitched using images
wxAddImagesCmd::execute() thinks have the same lens anyway).

I take it this should solve bug 1908785:
https://sourceforge.net/tracker/?func=detail&atid=550441&aid=1908785&group_id=77506

I think it may be preferable to either:
(i) remove the new option, but leave it only asking for images which
are likely to have different parameters.
or (ii) If the option isn't selected, create a new lens (or unlink the
HFOV and crop factor) for the next image without HFOV and crop factor
in the EXIF data, then ask again. This probably gives the intended
existing functionality, but with an extra option.

The new option will appear on the dialogue in some circumstances that
do not use it, for example if there are no more images, or if an image
with a rectilinear projection and a FOV >180 degrees is found. I could
try to fix these, but wondered if solution (i) above was better.
Opinions please?

James

[J. Schneider]

> I think it may be preferable to either:
> (i) remove the new option, but leave it only asking for images which
> are likely to have different parameters.
> or (ii) If the option isn't selected, create a new lens (or unlink the
> HFOV and crop factor) for the next image without HFOV and crop factor
> in the EXIF data, then ask again. This probably gives the intended
> existing functionality, but with an extra option.
>
> The new option will appear on the dialogue in some circumstances that
> do not use it, for example if there are no more images, or if an image
> with a rectilinear projection and a FOV >180 degrees is found. I could
> try to fix these, but wondered if solution (i) above was better.
> Opinions please?

What about the case when I use scans that have the same size but must
have different lenses because of different d, e parameters?

regards
Joachim

[James Legg]

On 20/03/2008, J. Schneider <j-sc...@gmx.de> wrote:

> What about the case when I use scans that have the same size but must
> have different lenses because of different d, e parameters?

I don't think the assistant tab caters well for this use of Hugin, but
that could be changed. I guess your scans don't have EXIF data, and
therefore when loading them you are asked for irrelevant lens
properties multiple times, and they are all assigned the same lens? In
that case, this patch will only allow you to get away with entering
one made up field of view instead of as many as you have images, so
the request for lens properties is still unwanted.

An option to indicate that the input images are scans might be useful.
Perhaps this should be added to the assistant tab, and the behaviour
of the assistant tab changed when it is selected? This would automate
the process described at
http://hugin.sourceforge.net/tutorials/scans/en.shtml, and of course
not ask for lens properties.

Do you want a feature like this? I could attempt to create this,
however I don't know how many of Hugin's users stitch flat scanned
images and would appreciate the extra option on the first screen they
see when they start the program!

James

[Pablo d'Angelo]

Hi James,

Thanks for your interest in participating in GSoC 2008!

James Legg wrote:
> Hi,
>
> On 19/03/2008, Yuval Levy <goo...@levy.ch> wrote:
>> welcome to the community.
>
> Thanks!
>
>> I assume you have read the thread linked below and are aware of the
>> redundancies with FreePV.
>
> I thought (I could be wrong) that the FreePV and Hugin's preview were
> quite unrelated, and FreePV wasn't meant to support the requirements
> that Hugin's preview would have, but I accept there is some overlap.

Actually, in freepv, the renderer needs to be optimized, too, but it has
other, more serious problems that need to be fixed first.

> I've written a patch that has the requested option implemented, and
> also only asks when it will not overwrite it. This makes the option
> slightly redundant (since most panoramas will be stitched using images
> wxAddImagesCmd::execute() thinks have the same lens anyway).
>
> I take it this should solve bug 1908785:
> https://sourceforge.net/tracker/?func=detail&atid=550441&aid=1908785&group_id=77506
>
> I think it may be preferable to either:
> (i) remove the new option, but leave it only asking for images which
> are likely to have different parameters.
> or (ii) If the option isn't selected, create a new lens (or unlink the
> HFOV and crop factor) for the next image without HFOV and crop factor
> in the EXIF data, then ask again. This probably gives the intended
> existing functionality, but with an extra option.

My intention for the dialog was exactly what you describe in (i), and
creates a new lens if the images are likely to have different parameters. I
think this cares for 90% of the use cases. I don't think the extra option
would care for most of the other users either, since manually deciding which
images should use different parameters is hard to do at import time, because
usually one can't tell by looking at the filename anyway.

ciao
Pablo

[Yuval Levy]

@James: thanks for the patch and the analysis! I have not had time to
look at the patch yet.

while you are at it, why don't you start filling out a proposal on
http://wiki.panotools.org/SoC_2008_student_proposals ?

Pablo d'Angelo wrote:

> James Legg wrote:
>> I think it may be preferable to either:
>> (i) remove the new option, but leave it only asking for images which
>> are likely to have different parameters.
>> or (ii) If the option isn't selected, create a new lens (or unlink the
>> HFOV and crop factor) for the next image without HFOV and crop factor
>> in the EXIF data, then ask again. This probably gives the intended
>> existing functionality, but with an extra option.
>
> My intention for the dialog was exactly what you describe in (i), and
> creates a new lens if the images are likely to have different parameters. I
> think this cares for 90% of the use cases. I don't think the extra option
> would care for most of the other users either, since manually deciding which
> images should use different parameters is hard to do at import time, because
> usually one can't tell by looking at the filename anyway.

How will the app know which images are likely to have different parameters?

I think a simpler solution is to ask the user if he is importing images
that share the same parameters or not (i.e. *with* the option).

If the option is yes, then all images should be assigned the same lens
and the information is entered once in the pop up dialog.

If the option is no, then the important thing is to create the different
unlinked lenses in the model. It is indeed hard to enter the lens
information at import time for the reasons mentioned by Pablo.

as far as I am concerened the logic could stop here. Just generate empty
lenses and let the user fill the data in the Image Parameter tab.

What could be helpful still (but we are talking a marginal set of use
cases) is if in that case the dialog pops up for each image (as it does
before the patch) *with an added thumbnail* and with the fields
pre-filled from the previous dialog, so that the user can:
- click ok to accept the prefilled data and move on to the next image
- edit the lens data for the current image

But this is spinning *very* far.

IMO just popping the dialog *once* with an option to use the same lens
or multiple lenses is the best trade-off between usability and complexity.

Yuv

[James Legg]

On 21/03/2008, Yuval Levy <goo...@levy.ch> wrote:

> Pablo d'Angelo wrote:
> > My intention for the dialog was exactly what you describe in (i), and
> > creates a new lens if the images are likely to have different parameters. I
> > think this cares for 90% of the use cases. I don't think the extra option
> > would care for most of the other users either, since manually deciding which
> > images should use different parameters is hard to do at import time, because
> > usually one can't tell by looking at the filename anyway.
>
>
> How will the app know which images are likely to have different parameters?

This currently appears to be guessed by camera make and model, if in
the EXIF data; and image size otherwise. Obviously it doesn't always
get it right, but I can't imagine it will create too many lenses in
the vast majority of cases, so it is perhaps a good start.

> I think a simpler solution is to ask the user if he is importing images
> that share the same parameters or not (i.e. *with* the option).
>
> If the option is yes, then all images should be assigned the same lens
> and the information is entered once in the pop up dialog.
>
> If the option is no, then the important thing is to create the different
> unlinked lenses in the model. It is indeed hard to enter the lens
> information at import time for the reasons mentioned by Pablo.
>
> as far as I am concerened the logic could stop here. Just generate empty
> lenses and let the user fill the data in the Image Parameter tab.

If the images don't all use the same lens, it isn't necessarily the
case that each image has it's own lens.

I agree no user input should be done 'per image' during import.

I suggest a small number of preset options are presented if no EXIF
data is found in multiple images, with at least the options:
* All images are from the same lens, it has projection X, HFOV Y /
Crop Factor Z.
* Something different, show me the Camera and Lens tab.

I think any options that create multiple lenses should be accessible
from the Camera and Lens tab.

> What could be helpful still (but we are talking a marginal set of use
> cases) is if in that case the dialog pops up for each image (as it does
> before the patch) *with an added thumbnail* and with the fields
> pre-filled from the previous dialog, so that the user can:
> - click ok to accept the prefilled data and move on to the next image
> - edit the lens data for the current image
>
> But this is spinning *very* far.
>

Adding this much functionality in here will make the user to learn how
to set up lenses on import, and how to do the same thing with the
Camera and Lens tab, which is difficult since the two things act in
different ways.

James

[Robert Krawitz]

Date: Sat, 22 Mar 2008 00:57:56 +0000
From: "James Legg" <lanky...@gmail.com>

On 21/03/2008, Yuval Levy <goo...@levy.ch> wrote:

> I think a simpler solution is to ask the user if he is importing images
> that share the same parameters or not (i.e. *with* the option).
>
> If the option is yes, then all images should be assigned the same lens
> and the information is entered once in the pop up dialog.
>
> If the option is no, then the important thing is to create the different
> unlinked lenses in the model. It is indeed hard to enter the lens
> information at import time for the reasons mentioned by Pablo.
>
> as far as I am concerened the logic could stop here. Just generate empty
> lenses and let the user fill the data in the Image Parameter tab.

If the images don't all use the same lens, it isn't necessarily the
case that each image has it's own lens.

FYI, I created one panorama using two lenses -- 6 shots with a 28 mm
lens covering the entire frame, and 12 more with a 50 mm covering the
center of the frame. This greatly reduced the number of shots I
needed, while still achieving a very high resolution for the most
important part of the image. Having to enter lens data for each image
separately would be unnecessarily tedious in this case.

--
Robert Krawitz <r...@alum.mit.edu>

Tall Clubs International -- http://www.tall.org/ or 1-888-IM-TALL-2
Member of the League for Programming Freedom -- mail l...@uunet.uu.net
Project lead for Gutenprint -- http://gimp-print.sourceforge.net

"Linux doesn't dictate how I work, I dictate how Linux works."
--Eric Crampton

[J. Schneider]

Yuval Levy schrieb:

> IMO just popping the dialog *once* with an option to use the same lens
> or multiple lenses is the best trade-off between usability and complexity.

I think so , too. I would add just one thing: Prefill the other lenses
with the value of the first lens (perhaps optional). This would cater
for the case with scans: The value would be at least close to the
correct value as a start for optimization. (I guess you have to fill in
something anyway?)

regards
Joachim

[Klaus]

Regarding parameters d,e, please correct me
if I am wrong, but if one unchecks that "inherit" box
then images with the same lense number do have
their individual d and e values.

Cheers Klaus

[J. Schneider]

Robert Krawitz schrieb:

> FYI, I created one panorama using two lenses -- 6 shots with a 28 mm
> lens covering the entire frame, and 12 more with a 50 mm covering the
> center of the frame.

What about this:
When images are dropped onto hugin, hugin first inspects the EXIF values
of all images. Depending on what it finds, options are presented to the
user.

all with complete exif -> as now

all with incomplete exif (usually crop factor missing) -> ask for
missing value

all without exif -> ask for parameters and ask if they apply to all
images; if no -> create multiple lenses
add a checkbox for scans: if checked, optimizsation for d, e is already
selected, also vignetting centre.

regards
Joachim

[prokoudine]

On Mar 24, 8:15 pm, "J. Schneider" <j-schn...@gmx.de> wrote:
> When images are dropped onto hugin, hugin first inspects the EXIF values
> of all images. Depending on what it finds, options are presented to the
> user.

Or just remember lenses across sessions and upon importing build a
list of all images and let user pick a lens

Alexandre

[James Legg]

@Klaus
You are basically correct, though I would create one more lens, where
no parameters are optimised, to use as an anchor.

@everyone
I've tried to use these suggestions and made a new patch.

With this patch, when an image without full field of view information
is found, the box appears with the following differences:
-The OK button is gone.
-There is a button that creates one single lens, which will then be
used by all of the remaining images without field of view information.
-There is a button to create uniquely numbered copies of that lens for
each image without FOV info. I personally think this is unnecessary,
but some of the comments above show that people use Hugin this way.
-There is a new button for scans. It doesn't work yet as I haven't
worked out how the variables work yet. How would I set only d, e and
roll as not inherited and needing optimisation?
-The Cancel button now aborts image adding instead of making some
specific lens.
- The message at the top has changed to reflect the new options.

There's a couple more things I think should be added which I haven't done yet:
Firstly, we should be able to select a lens already in the project if
there is one there already.
Secondly, the current method of making new lenses by other EXIF data
has been lost, perhaps another option should be added if there are
more than two camera makes/models used in the import?

As I am fairly new to Hugin's source code, I would appreciate it if
someone were to review my code, and tell me how to use the lens
variables. I don't have a wide range of images with different EXIF
data to test with, so I might have missed something.

The return values of the getLensInfoFromUser and HFOVDialog::ShowModal
functions has been changed, and some macro constants to describe the
new return values are given in HFOVDialog.h.

James

[Pablo d'Angelo]

Hi James,

Sorry for the long delay before answering. Too much stuff going on everywhere.

James Legg wrote:
> @Klaus
> You are basically correct, though I would create one more lens, where
> no parameters are optimised, to use as an anchor.
>
> @everyone
> I've tried to use these suggestions and made a new patch.
>
> With this patch, when an image without full field of view information
> is found, the box appears with the following differences:
> -The OK button is gone.
> -There is a button that creates one single lens, which will then be
> used by all of the remaining images without field of view information.

What happens if images with different size or orientation (for example
images autorotated by the raw converter) are used? Almost always one does
not want these to share a lens, even if the user thinks its alright to do
so, because he shot the images with the same settings, but just rotated the
camera.
Especially the principal point (shift of image center) depends on the
orientation (portrait vs landscape vs upside down). A 90° turn between left
and right can't be discovered by this, but this is probably not so
important, since most users will probably either turn the camera 90° left or
90° right and thus won't have mixed images.

> -There is a button to create uniquely numbered copies of that lens for
> each image without FOV info. I personally think this is unnecessary,
> but some of the comments above show that people use Hugin this way.
> -There is a new button for scans. It doesn't work yet as I haven't
> worked out how the variables work yet. How would I set only d, e and
> roll as not inherited and needing optimisation?
> -The Cancel button now aborts image adding instead of making some
> specific lens.
> - The message at the top has changed to reflect the new options.

I don't like to turn the HFOV Dialog into a general "choose type of
panorama" dialog. It is also used when reloading the exif data of a single
image, for example, where the options above won't make sense (to me at
least). Also, what happens when a single images is added to an existing and
a user selects "Scans". Will the existing images be changed, too?

I think such a general functionality should be placed in a separate project
wizard or maybe replacement/enhancement of the Assistant tab, instead of
being part of the "Add images" function.
This is outside the current development scope, as the current priority is on
fixing bugs for the next release.

Until this happens, I'd actually rather like to see a fixed version of the
(intended) current behaviour, namely:

1. When a new image is added, check if it fits any already existing image in
the project. Check all metadata required (image size, EXIF make, camera
name, focal length, crop factor). If it matches, just reuse that lens.

2. If no crop factor is in the EXIF data, check if the project contains an
image from the same camera and reuse its crop factor, then do the check at
1.). Otherwise show the HFOV Dialog again (with the camera make and model
display).

Alexandre Jenny from Autopano also maintains a crop factor database which we
could probably use (I haven't checked lately, but last year it was available
under an CC license). In the future, we could probably use that, too.

3. If no crop or focal length is in the EXIF data, try to find a matching
lens by looking at the already added images (image size, possible EXIF make
and model.).

This is a weaker check and could lead to assigning an image to an existing
lens by mistake.

In this case, the user could be asked if all images in the project are shot
with the same camera settings. The current code just assumes that (at least
that was the intention). Given that most project are shot with the same lens
settings, and most of the time, the images contain an exif focal length
which will be used in 1.), there are only very few cases where this will
result in bad lens settings for ordinary panoramas:
Same image size + no exif data + changed zoom or camera.

For this special case, I have expected that the user is smart enough to fix
this in the Lens tab.

Alternatively it would be possible to ask the user (with a simple pop up
dialog), if he uses images with different camera or zoom setting.

User answers yes: reuse first lens with matching image size.
User answers no or scan: give every image that can't be assigned with 1.) or
2.) a different lens, and leave the HFOV to be filled in the Camera and Lens
tab.

The different lenses option could be then used for the scan usecase as well
(scans won't have EXIF data, I think and would always end up with the dialog
asking for different). For the scan case, it would be nicer to assign one
lens to the "reference" image, and another lens to the other images, though.
As mentioned above, a dedicated "align scans" mode/wizard would be the best
solution.

Summary:
The solution above will do the right thing for typical panoramas, when the
exif data is complete. It will ask for HFOV/crop factor once if the EXIF
info lacks the crop factor. If different zooms or orientations are used,
appropriate lenses are created automatically, and the Align button should
work very nicely without touching the Camera and Lens tab, even if the user
zooms or uses different lenses.

If the users adds images without EXIF data, he is asked if the camera
settings were the same or not. If not, generate separate lenses, and let the
user fill out the details in the Camera and Lens tab.

> There's a couple more things I think should be added which I haven't done yet:
> Firstly, we should be able to select a lens already in the project if
> there is one there already.
> Secondly, the current method of making new lenses by other EXIF data
> has been lost, perhaps another option should be added if there are
> more than two camera makes/models used in the import?
>
> As I am fairly new to Hugin's source code, I would appreciate it if
> someone were to review my code, and tell me how to use the lens
> variables. I don't have a wide range of images with different EXIF
> data to test with, so I might have missed something.

The lens variables actually turned out to be a weak spot in hugin's design.
The idea is that lens variables hold the parameters that are common to a
single lens (or images of a zoom lens with same focal length).
Then they can be set to inherit, forcing the same values for all images.
This results taking priority of the lens variables.

Code to link "v" of lens 0:

Lens l = pano.getLens(0);

LensVariable lv = map_get(l.variables, "v");

lv.setLinked(false);

pano.updateLensVariable(0, lv);

Hmm, to late already, will look at the patch in detail tomorrow....

ciao
Pablo