unit test concerns and questions

[Stephen Bialkowski]

Hi,

Thank you for making this publicly available!  Unfortunately, this isn't just a statement of praise...

I had run into 2 unresolved dependencies.  I have provided a work around for each (1, 2).  I think there are likely better work arounds.  I also question the output of the image test (3, 4).  

I don't dare muck with python too much.  I've read reinstalling can render the OS useless.  I had to re-install pip.  That's about as far as I dare take it.  

$ sudo ./unit_test.sh -e xmas

...

real 0m1.593s

user 0m1.160s

sys 0m0.416s

./unit_test.sh: line 69: gprof2dot: command not found

./unit_test.sh: line 69: dot: command not found

$ which gprof2dot

/home/parallels/.local/bin/gprof2dot

$ echo $PATH | grep parallels

[...]:/home/parallels/.local/bin:[...]

1) unit_test.sh can't find gprof2dot even though it's path is in $PATH.  

When I add echo $PATH to the top of the script, I see:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

When the same command is run from terminal, I see the path I would expect.  I don't know how $PATH is populated in the bash script.  But it certainly isn't pulled in from .bashrc.  So, for now, I'll force the path by putting the following at the top of the script.  It seems like such a hack!  But I'm over it.  

export PATH=/home/parallels/.local/bin:$PATH

2) pip refuses to install dot.

$ pip install dot

Collecting dot

  Could not find a version that satisfies the requirement dot (from versions: )

No matching distribution found for dot

You are using pip version 8.1.1, however version 18.0 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

DON'T DO IT!  It was following this advise that caused me to have to reinstall pip.  pip could no longer execute anything as it couldn't resolve symbol "main".  If you trashed your pip install, this may work for you:

sudo python -m pip uninstall pip && sudo apt install python-pip --reinstall

Solution: this command resolve the dot dependency.  

$ sudo apt install python-pydot python-pydot-ng graphviz

3) Is this to be expected after processing image9.bmp twice?

It seems startracker.py throws an exception.  

$ sudo ./unit_test.sh -i xmas

rgb.solve_image('xmas/samples/image9.bmp')

Time1: 0.0

Time2: 0.0209100246429

Time3: 0.0264129638672

DEC=325.917697

RA=226.479015

ORIENTATION=-67.095071

stars 3

ang_rate: 

Time4: 0.0270230770111

Time5: 0.0272269248962

Time6: 0.0272490978241

...

rgb.solve_image('xmas/samples/image9.bmp')

Time1: 1.90734863281e-06

...

0.989962220192,-0.137079447508,-0.0344090275466,-0.52653747797,-0.52269923687,-0.670480251312,0.861267456085 0.987475812435,-0.153092607856,-0.0381328649819,-0.519310355186,-0.512702524662,-0.683705210686,0.856955949765 0.992016375065,-0.0864524319768,-0.0918122828007,-0.484214663506,-0.584132134914,-0.651402950287,0.868609285023 

quit()

Traceback (most recent call last):

  File "startracker.py", line 522, in <module>

    exec(data)

  File "<string>", line 1, in <module>

  File "/usr/lib/python2.7/site.py", line 375, in __call__

    raise SystemExit(code)

SystemExit: None

Traceback (most recent call last):

  File "startracker.py", line 522, in <module>

    exec(data)

  File "<string>", line 1, in <module>

  File "/usr/lib/python2.7/site.py", line 375, in __call__

    raise SystemExit(code)

SystemExit: None

4) Images: median, 0, 3, 4, and 7 have much less output than the others.  

I take that to mean there is no solution.  Is this expected?  I think I saw an explanation for why 7 and median don't have solutions.  Not sure about the rest...

Regards,

Stephen 

[Andrew Tennenbaum]

On Mon, Aug 27, 2018 at 5:21 PM Andrew Tennenbaum <aztenn...@gmail.com> wrote:

The best place to start is without a doubt to read through unit_test.sh, and make a calibration folder with your own images. Once that works, I’d recommend you read through startracker.py, and customize it to suite your needs.

Take a look at the changes oresat has made:

https://github.com/oresat/oresat-star-tracker

On Mon, Aug 27, 2018 at 5:13 PM Stephen Bialkowski <stephen.bi...@gmail.com> wrote:

More specifically, I render a 3D simulation of the earth and stars. I pulled in data from the hip2 catalog, solve for RA/DEC using NOVAS, render them, and pull the frame buffers back in to test for accuracy.  I'm really not sure how to best get started with OpenStarTracker.  Any quick tips on how to dive in?  

On Mon, Aug 27, 2018 at 2:54 PM Andrew Tennenbaum <aztenn...@gmail.com> wrote:

Hi, thanks for sharing this! I assume you are on mac os? If so, I will add a section to the install guide for people using the software on mac

The exceptions happen when after exiting, so they are not really an issue, but I should probably clean them up anyway

Some of the xmas images don't solve. They are intentionally low quality, to test the robustness of the star identification algorithm. You can also try the new-api branch, which has a number of other test cases

Do you mind if I ask what you were intending to use openstartracker for?

--
You received this message because you are subscribed to the Google Groups "OpenStartracker" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openstartrack...@googlegroups.com.
To post to this group, send email to opensta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openstartracker/0eaa6a54-7d58-4cd3-9fe5-da200962def6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Umair Khan]

Full disclosure - I'm not "officially" involved with the OpenStarTracker project, I've just been working with it for a while.

I'm afraid I can't really comment with regards to the dependencies. For my use, I didn't have a need for the profiling functions build in, so I just got rid of the dependencies and removed that part of the code.

The exceptions after processing the last image are to be expected. The "vanilla" version of startracker.py listens to a socket for things to execute, and unit_test.sh sends things to that socket (rgb.solve_image( __ )). At the end, unit_test.sh sends a quit() command, which is what raises the exception to exit the program. You can find a detailed explanation of quitting in Python in this StackOverflow answer.

If a given solve_image() command produces no DEC/RA/ORI output, then you are correct, it means that image has not solved. In the xmas image set, images median, 7, and 9 should not solve. The median image is a black background to test false positives and I believe 7 and 9 are superimposed combinations that should not solve. I'm not quite sure why images 0, 3, and 4 are not solving. Maybe try recalibrating with -r? I find that results sometimes change with a recalibration.

In any case, the xmas image set is pretty outdated now, I think it was a first-pass type thing that had some problems with it. I believe the feb16 image set should be the go to for testing now, though the README obviously has not been updated.

[Stephen Bialkowski]

Umair,

Thanks for addressing the image solutions discrepancy.  

sudo ./unit_test.sh -crei xmas provides consistent results.  

...

camera coverage simulation percent:

0

...

100.0 78.5285064688

...

If I'm reading the output correctly, the score is 78.5% for the real results, and 100% for the other results.  

This satisfies the >70% criteria.  So, I don't think things are horribly wrong.  I don't like starting from something that raises eyebrows (no solutions for images 0, 3, 4).  However, there is a solution for all images except median in the feb16 folder.  Perhaps it's just a matter of the code changing over time, and not realizing the original expectation no longer holds.  

Regards,

Stephen