Hirise image alignment

[Magda Pilarska]

Hi all,

I am currently working on aligning series of HIRISE images into mosaic. I want to align the images using cam2map, however the images after using cam2map are still shifted.

As far as I understand, using stereo I can align a pair of images, but using cam2map I should be able to get images properly fitting with each other. Am I right? 

Is there probably other possibility of aligning a series of overlapping images into one reference using the pipeline?

Here is a part of my code:

cam2map from=adjusted_scene.cub map=map.cub to=new_map.cub

Thanks for any help!

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

Greetings,

ASP is primarily concerned with aligning of stereo DEM data. 

What can work for you is to run bundle adjustment on the images that you want to mosaic, and those should be the raw .cub images, before cam2map. After that, they would be consistent with each other. But they would need to be projected onto a DEM, which can be done with our tool, named mapproject, invoked with --bundle-adjust-prefix, where the prefix is the output bundle adjustment prefix from the previous step.

Regretfully this will make things better but won't fix them fully unless your DEM is consistent (well-aligned) with the images (note that cam2map uses a DEM too, but it is implicit). 

The best you could do would be to create a n ASP stereo DEM from some of your HiRiSE images after bundle adjustment, and maproject onto that, using as before the --bundle-adjust-prefix option.

If you want to project onto a different DEM, the currently bundle-adjusted cameras would need to be aligned to it. That can be done by using pc_align tool, with your desired DEM as the reference, the ASP stereo DEM from earlier as the source, and then that alignment would need to be applied to the cameras too, by invoking yet again bundle adjustment, with 0 iterations, with the --initial-transform option specifying the output transform of pc_align (for example, output_prefix/run-transform.txt) and the bundle adjust option --input-adjustments-prefix referring to the previous bundle adjustment prefix. This would move the cameras from being aligned to the ASP DEM to being aligned to your own DEM. 

This is a bit technical, though note that I described all steps. The basic idea is that if you don't want shifts or seams after mapprojection, all images and the DEM to project to must be in perfect agreement among themselves, and what I provided above is a recipe for achieving that, if perhaps a complex one.

Oleg

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Magda Pilarska <pil...@gmail.com>
Sent: Tuesday, October 27, 2020 8:27 AM
To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>
Subject: [EXTERNAL] Hirise image alignment

 

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/08fecff8-fa50-4789-bfb0-05f43f557c94n%40googlegroups.com.

[Magda Pilarska]

Hello again,

I have gone through your idea of image bundle adjustment and then creating DTMs, but unfortunately I have stopped on bundle adjustment process. I will try to describe my workflow in detail to finally ask you for some advice how to make the adjustment more accurate.

I have 15 HISRISE images that I want to adjust using bundle_adjust function. The input images are before cam2map, as you suggested (they are right after hiedr2mosaic). To perform the adjustment, I have measured 9 GCP on the images, prepeared the *.gcp file as it is described in the ASP documentation. 

Then I ran the bundle_adjust function. And analysing the results I noticed some problems. Namely, in initial_residuals_no_loss_function_pointmap_point_log final_residuals_no_loss_function_pointmap_point_log, the resudual values seems to be too high for some points (above 2-3 pixels). Residuals for GCP in thta files are approximately 200-300, which is a lot.

Additionally, I have these errors in the log file and terminal: could not triangulate point. If too many such errors, perhaps your baseline is too small, or consider decreasing --min-triangulation-angle or using --forced triangulation-distance and also a warning: GCPS are over 100 km from the other points. Are your lat/lon coorinates swapped? I think they are not swapped because in the *csv files the lat/lon are the same as for the automatically generated points.

Finally, I would obviously like to avoid the errors and warnings mentioned above and reduce the residual values for GCPs. I was thinking about changing the parameters of cost function and robust threshold, as well as the ip-detect-method seems to be important. 

Do you have some advices about that?

Thank you in advance!

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

I will suggest you start with two images rather than 15, to get comfortable.

The initial residuals could be big. You should look at the final residuals to see if they became small. They should, I hope, or at least you should see some changes. You can also examine the match file for two images in stereo_gui, such as stereo_gui left.cub right.cub run_ba/run-left__right.match and see if they look ok. 

If you examine how the bundle adjustment process works, hopefully the cost function will go down, and the residuals in those files too. 

I suggest you ignore this warning "could not triangulate point" unless in the final "no_loss" residual files in the bundle adjustment output directory you see no entries corresponding to your GCP (look towards the end of that file, and you should see a comment saying "# GCP" for entries corresponding to your GCP). 

I think you don't need to change any parameters for cost function or robust threshold, for now, these are rather advanced options.

If 

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Magda Pilarska <pil...@gmail.com>

Sent: Monday, March 1, 2021 9:43 AM

To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>

Subject: Re: [EXTERNAL] Hirise image alignment

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/e2826760-7f9d-4022-bdfa-ad0d8c35d6bcn%40googlegroups.com.

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

(Sent this too early.)

If things still don't work, you can share the final residual bundle adjustment file, or at least a portion of it towards the bottom, having the residuals for GCP and some before that. 

---

From: Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC] <oleg.al...@nasa.gov>
Sent: Monday, March 1, 2021 10:09 AM
To: Magda Pilarska <pil...@gmail.com>; Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>

[Magda Pilarska]

Hi all,

I have tested some approaches of the image alignment using bundle adjustment and GCPs. You have recommended that I can send some results and even the file with the residuals. I have attached the best results that I have achieved as well as I did a printscreen of the last rows. 

I am quite satisfied with the results.The mean residual value for automated points was 0,22450 (standard deviation 1,05259), the mean residual value for GCPs was 7,73667 (standard deviation 3,22842). I would be grateful if you could write a few words of comments.

Most important parameters I used: cost function L1, uniqueness threshold, ip detect method 2, max disp-error 4

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

Dear Magda,

Thank you for sharing the results of bundle adjustment with GCP.  I looked at the residuals in the attachment, and the ones early on are even smaller. That the GCP have a few pixels of error may be acceptable, it may reflect uncertainty in picking them. 

I am a bit surprised you had to use the L1 cost function. Both the L1 and L2 cost functions can do poorly if there is an outlier as it will try to minimize its huge residual at the expense of everything else. The default cost function, which I think is Cauchy, attenuates large residuals.  In your case likely the Cauchy cost function would have worked just as fine since you have no big outliers. 

The ip detect method should also not have been so crucial. The value of 2, which I think is ORB, is a binary feature detector. It is normally less robust than ip detect method 1 which is SURF and than method 0, but if the images are similar enough all these should work equally well.

The key test for you is to mapproject the images you align this way and see if they project properly on the ground. Ideally that "ground" should be a DEM created with ASP using exactly the same cameras you just bundle adjusted, or else a preexisting DEM consistent with your GCP. After you mapproject, hopefully you see no discrepancy as to how the images appear either on top of each other or on top of the DEM.

Oleg

---

From: Magda Pilarska <pil...@gmail.com>
Sent: Friday, March 5, 2021 9:55 AM
To: Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC] <oleg.al...@nasa.gov>
Cc: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>

[Magda Pilarska]

Thank you for your comments.

Well, the ip detect method seems to have a big influence. For SURF method (0) the errors for GCPs were much higher than fot method 2(approx. 200-300 px).

I looked at some match points and I think they look fine, however for some images there are less match points, probably because of the poor texture on the images. 

Nevertheless I compared the DTMs generated from the same stereopair with and without bundle adjustment and they look worse after hillshading them. I also used the same stereo.default file and parameters. Maybe this is the influence of the noise. Thus, I have a question about the further step, after bundle adjustment. Should I change something in stereo,default file, e.g. the pre alignment options? I have of course defined the --bundle-adjust-prefix and I use the oriented images just after hiedr2mosaic, i.e. id_RED.mos.hijitreged.norm.cub.

Regards,

Magda

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

Magda,

If method 2 worked better for you, that's good. That's why we have multiple methods for interest point detection. 

I don't think any of this should affect stereo preprocessing. 

I will suggest you examine the run where the DEM looks worse. Particularly, you can open its interest points match file in stereo_gui and see how it looks. A good trick sometimes is to copy the cleaned interest points from bundle adjustment to the stereo directory while renaming it to remove the "-clean" suffix and then ASP should pick that one up. That in case bundle adjustment creates better interest points that stereo. 

When the data texture is not so good, things can indeed go wrong sometimes.

Anyhow, I hope you playing around for a while with things will improve the results.

Oleg

---

From: Magda Pilarska <pil...@gmail.com>
Sent: Monday, March 8, 2021 1:16 PM