Celestial navigation AI project

[Jan Bilek]

Howdy team,

I am running with my son a hobby project to build an airship UAV (see bbblimp.com if interested), while the goal it is not getting there, but "share the road". Long story short, as a part of our project, my son expressed an interest in the Machine Learning so I formulated for him a "Celestial navigation" project. Current rough plan I have comes like this:

1. Get a Jetson Nano and familiarize with it (PyTorch)
2. Prepare a training set with 20 brightest stars - 100 pictures each
3. Do multiple training runs through multiple layer settings to see which provide best result (known star identification success rate)
4. Meanwhile write code which will translate star's inclination back to the Earth coordinates (date + time)

It may look trivial and I understand there are multiple holes/caveats, but I really don't want to make too complex for him to be able to achieve some progress there. (Hope you understand.)

Back to you, I would like as you for your comments:

1. Checking through the API, I currently consider Stellarium as being a perfect platform to prepare initial training set (API gets all parameters which I plan to iterate over). Plan is to get RemoteControl plugin + Python and drive local Stellarium to provide images needed. Any comments, ideas?
2. Would you have any recommendation(s) on C/C++ library which would be good for those inclination and time operations -> calculation back to GPS coordinates on a geoid?
3. Anything you see around we can reuse from Stellarium or feed back to Stellarium community? You guys are doing amazing job for decades and we are ready to contribute back (both more than fluent in C/C++/Python/Linux - not too much in Orbital mechanics).

Thanks in advance for any feedback & Cheers,
Jan

[Georg Zotti]

It may be possible to use Stellarium screenshots as training set. You should most likely use Perspective projection with a fixed field of view like that of your real camera, and play with star sizes. Are 20 stars enough? Probably use 100 stars or more, but I am not in AI.

I think it would be very nice to see an open-source and well-documented Python-based user interface that connects via RemoteControl. Maybe also as training exercise how Qt/PySide connects with a Qt C++ program.

About GPS/UTM coordinates, I expect the QGIS or OSGeo communities should have transformation libraries in Python.

I am not sure if the whole thing is more than a learning experience. Commonly UAVs have GPS navigation.

Good luck!

Georg

[Greg Miller]

There are quite a few good libraries out there.  For Python, astropy or pyephem would be good to start with.  NOVAS is a good C library.  You might also look at IAU's SOFA libraries which have been converted into many different languages.

--
You received this message because you are subscribed to the Google Groups "Stellarium" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stellarium+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/stellarium/d8f20e11-4b9e-4aed-96e3-78c8f043a6a9n%40googlegroups.com.

[Jan Bilek]

Hi Georg,

Thank you for your comments and hints on the perspective projection settings. I honestly think that 20 stars is a good start, of course I'll be delighted to see that with 100 stars (5-times larger training set, training time, output layer .... ). ;) But hey, let's start with baby steps.

Marking down your idea on "Open-source and well-documented Python-based user interface that connects via RemoteControl" - I have something in my pocket here, I'll share a draft once I have it working so you can provide your feedback. Well, Qt is not my strongest subject (I have grown on wxWidgets) and I had a serious headbang with it about 6 months ago. Still, let me give it a go.

Thanks for those hints on QGIS or OSGeo, we'll check those out.

Finally in regards to your "Commonly UAVs have GPS navigation" comment - agreed. GPS is an obvious pick, on the other hand it is not bringing any opportunity for a project on a right scale and which would combine technologies & ideas in this unique way like this celestial navigation project. I am imagining 500yrs technology redone in a new way for our 100yrs old project. :D

Long story short, I am just trying to have some good time with my 14yrs old son here. I'll be of course happy if this (airship) project & its spinoffs will inspire the others. ;)

Thanks again & Cheers,
Jan

--
You received this message because you are subscribed to a topic in the Google Groups "Stellarium" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stellarium/G62d68r0WKE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stellarium+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/stellarium/910f1b23-9826-4e3d-8c99-5afe3505a258n%40googlegroups.com.

[Jan Bilek]

Hi Greg,

Thanks for the list of those libraries - having them short-listed like you did for us will save us plenty of time going through that common trial & error exercise.

Cheers,
Jan

You received this message because you are subscribed to a topic in the Google Groups "Stellarium" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/stellarium/G62d68r0WKE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to stellarium+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/stellarium/CAP_5TC%2B15UAkxk3toULdYZeb7oeZq%3DaLKZJwRYrDb5%3DEEwy7hg%40mail.gmail.com.

[Georg Zotti]

I just assume 20 stars is not enough, as you likely don't identify 20 stars by their actual image characteristics (they just are white pinpoints in the images), but patterns of stars. So even 100 is likely a too low number. Anyway, I never played with those tools, and you will know better what you are doing.

From the images and attitude sensors, you will have to identify the stars and derive star altitudes. BTW, apart from star altitudes you need a good time signal. Again, GPS helps for that ;-)

Good luck and have fun with this project!

Georg

[Jan Bilek]

Hi Georg, all,

Thanks for all your support, I think I have it! Honestly - going through all that C++ code you wrote - hats off! For a 20+ year project this is a very clean job! I was surprised how quickly I was able to get my head around it - so much better than the others (got stuck in a Marlin code for 3D printers lately and that's a maze!).

Anyway, I've tried to propose/post a patch through github, but hit some problems with authentication etc. - well, I don't mind much - still got working what I needed - being able to generate all the pics here through your scripting engine.

As per the report on github - I needed a tiny patch to be able to generate a directory sub-tree. I am adding it here so you can consider merging it on your own (here I am hereby giving away all IP for this code now and forever).

I am also adding the script I developed to generate my training set (it is still generating) - feel free to add it as well, but it is +- useless without that patch #1.

Hey guys you are the best - seriously! Thank you for all your hard work you keep investing in this project!

Thank you again & Best Regards from Brisbane Australia,

Jan

To view this discussion on the web visit https://groups.google.com/d/msgid/stellarium/3f7942be-de24-46a4-a98c-1f815403cd89n%40googlegroups.com.

[Jan Bilek]

Hi team,

Just FYI that it all worked out and we have all the training data we desired.

I've been able to document it on our blog https://bbblimp.com/2022/08/17/training-set-with-stellarium/ including big credits to you all!

Thanks again and cheers,
Jan

[Georg Zotti]

Ah, that looks like a serious training set... :-) Yes, I misunderstood previously. 20 bright stars *and their surroundings*.

Now we will be happy to see your own images analyzed and stars identified!

Keep the fun going!

Georg

[Jan Bilek]

Hi Georg, team,

Reporting back with some awesome progress - we've finished the first run with 10% training data from Stellarium on grayscale Resnet 18 neural network and got 67% success rate with a star identification! Unfortunately I can say how many epochs it took to get there as the training script died (leaving trained network weights settings in a file), but still I think this is super-awesome - demonstrating that this concept works! (One training epoch on 10% data took ~28 minutes.)

Next stages are:

1/ Rerun this with Resnet30 + on a whole training set (pictures needed to be scaled down to 224 x 224 grayscale). Hoping to get a success rate above 90%!

2/ Start working on recalculation back to celestial coordinates

3/ Prepare Stellarium Proof-Of-Concept

This is where I've been thinking if you can share your thoughts. My current rough POC plan is to:

1/ Ask Stellarium to pick random datetime + location

2/ Identify 4 brightest stars above horizon

3/ Return their screenshots + inclinations + datetime

4/ Log everything in a file - including data for validation (star names, location)

5/ Identify stars with pre-trained network

6/ Use inclinations + datetime to identify location

7/ Compare with validation data from Stellarium

I think this should be all ok-ish, the only thing I am not sure about would be point 2/. Would you be able to point me in a direction if there are any analytics available in Stellarium to be able to identify 4 brightest stars above the horizon, or suggest any clever trick to get around this? 

Thanks again & Cheers,
Jan

To view this discussion on the web visit https://groups.google.com/d/msgid/stellarium/12cce22b-165a-4fe0-a784-1829ddbc5501n%40googlegroups.com.

[Georg Zotti]

1) look into the training script for the celestial navigation package. (scripts/random_location.ssc)

2) There is no such functionality. You could just have a list of the ~30 brightest stars and check their altitude and apparent magnitude (including extinction) for each random datetime/location, sort and select.

You can extend the mentioned script to pick the 4 brightest stars, center on them (How sensitive is the AI to off-center brightest stars?), store data retrieved via core.getObjectInfo().

I just see you could be fooled by planets, or even worse, bright satellites, of course. ISS is as bright as Jupiter, and others can still be annoyingly bright. What about airplane landing lights?

For real-world conditions, another obstacle certainly is cloud cover. You favorite star may be blanked out by just that cloud, or you may just have 2 holes in the clouds that provide a few medium-bright stars.

I guess "hardening" the system will be a future extension!

HTH, G.