Astrometry.net docker build - web: image uploaded, but can't produce results

[bthoven]

I'm trying to build its docker myself to get the latest updates. My build is based on the official github
Official docker build instruction
The image build was successful for both the solver and the webservice (webui) parts.
I'm able to spin both containers up. Tried solve-field command inside the container terminal and both the solver and webservice containers can solve my images. Actually, I don't have to spin up the solver container because the webservice container includes the solver part in the built image.

The problem is with the webservice container. Though it can solve my image with solve-field command line, I can't via its web interface. I loaded up the web UI, uploaded the same photo, then it went nowhere from then. I don't know how to solve the problem. Has anyone successfully done it with these GitHub dockerfiles? Thanks

I have attached the log entries for reference.

[bthoven]

My docker image at docker hub is bthoven/astrometrynetweb:latest, if you wish to try and help...thanks.

[bthoven]

Sorry, this is the config I use. As mentioned, with this config, it can solve-field successfully.

[Dustin Lang]

Hi,

See the instructions here about how to use that docker container -- you need to mount index files into the container, and you should create a "docker.cfg" config file (like astrometry.cfg, but with container paths) to control it.

https://github.com/dstndstn/astrometry.net/tree/main/docker#webservice

cheers,

dustin

On Sun, Sep 8, 2024 at 11:58 PM bthoven <bth...@gmail.com> wrote:

Another log file showing when log entries when starting up the container. I saw this log entry at the first line. Would that be the cause of the issue?:
./astrometry/docker/webservice/run.sh: line 9: /index/docker.cfg: No such file or directory

--
You received this message because you are subscribed to the Google Groups "astrometry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to astrometry+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/66c52699-98b2-4f77-bca0-2079fc8ae606n%40googlegroups.com.

[bthoven]

Thanks. astrometry.cfg is there and mounted; and as mentioned I can go inside the webservice container terminal to run solve-field with my photos successfully using the index folder/files I downloaded and mounted to the container.
The problem is about submitting photo, either by uploading a photo file or specifying image url, one pressing upload, it did nothing.

Btw, what is docker.cfg?

[Dustin Lang]

Like I said, the file needs to be called docker.cfg, not astrometry.cfg

Specified here

https://github.com/dstndstn/astrometry.net/blob/main/docker/webservice/solvescript-docker.sh#L12

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/ad404cc6-b744-4aeb-aab4-da9b5314ef6fn%40googlegroups.com.

[bthoven]

astrometry.cfg is the file that tells where the index files are, inparallel option, depth, etc. Are we talking about the same config file? If so, it must be named astrometry.cfg, on both inside the container and on host. I've tried renamed it on my host file to docker.cfg, the solve-plate was not successful reporting it can't find astrometry.cfg!

[Dustin Lang]

Yes, that's the config file I'm talking about.

The link I sent above is the script that the web service runs inside the container.  It looks for the config file at /index/docker.cfg .

The reason to change the name is that you probably need the paths to be different inside and outside the container.  So having astrometry.cfg (for use outside the container) and docker.cfg (for use inside the container) lets you use the same directory for both.

--dstn

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/8809056c-9312-4166-b974-1d6e957bfc2an%40googlegroups.com.

[bthoven]

I also can't find docker.cfg file anywhere inside the container

[Dustin Lang]

You need to create a suitable docker.cfg file and mount it into the container.

Please read https://github.com/dstndstn/astrometry.net/tree/main/docker#webservice

again

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/1f0c3d6d-8a9a-4e19-bfb6-a008644b7bd6n%40googlegroups.com.

[bthoven]

As the docker.cfg file you mentioned doesn't exist in the container, I have to create it myself by going into the webservice container and copying the astrometry.cfg (which was automatically created by docker build) from /user/local/etc folder to /index/docker.cfg. I also have to amend the docker.cfg file to point to folder /index for index files location. It still doesn't work.

I have no idea how else to proceed. I've been working on docker a lot, but for this case, I may have to give it up for now.

The other way to prove your point is you could run a container from my image built at dokerhub? It is named "bthoven/astrometrynetweb:latest. I built it from the dockerfile without modification. When you do it and if it works, could you show me how you do it if it's different from what was stated in the github documents.

Thanks.

[bthoven]

Just in case you spin up a container by using my docker image. So far I've tried these two objects via both solve-field command line and Webui upload.
solve-field --downsample 2 https://stellarscenes.net/object/m35.jpg --no-plots --overwrite --scale-units arcsecperpix --scale-low 12 --scale-high 13
solve-field --downsample 8 --no-plots --overwrite https://live.staticflickr.com/65535/50313416261_140ba06405_o.png --scale-units arcsecperpix --scale-low 1 --scale-high 2

[bthoven]

Below is the log entries just after spinning up webservice container, anything to do with "TypeError....">

Traceback (most recent call last):
  File "process_submissions.py", line 1093, in <module>
    main(opt.jobthreads, opt.subthreads, opt.refreshrate, opt.maxsubretries,
  File "process_submissions.py", line 884, in main
    print('New UserImages:', maxui-first_maxui, '; Jobs completed:', n_jobs_done)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'
Watching for file changes with StatReloader
Initial PYTHONPATH: /src:/usr/local/lib/python
Initial sys.path:
/src/astrometry/net
/src
/usr/local/lib/python
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/usr/local/lib/python3.8/dist-packages
/usr/lib/python3/dist-packages
Initial PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Final PYTHONPATH: /src:/usr/local/lib/python
Final sys.path:
/src/astrometry/net
/src
/usr/local/lib/python
/usr/lib/python38.zip
/usr/lib/python3.8
/usr/lib/python3.8/lib-dynload
/usr/local/lib/python3.8/dist-packages
/usr/lib/python3/dist-packages
/src/astrometry
Final PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/src/astrometry/solver:/src/astrometry/util:/src/astrometry/plot
get_temp_file(): /tmp/proc-subs/tmp1qgdkdgt
Tempdir: /tmp/proc-subs
Processing jobs with 3 threads
Processing submissions with 2 threads
Refresh rate: 5.0 seconds
Submission processing retry limit: 20
Maximum UserImage id on startup: None

[Dustin Lang]

Have you checked whether the official docker images work?  I just confirmed that it works for me.

Note that the container's startup script will create a docker.cfg file if one does not exist, and that file will work, assuming you have mounted the directory containing your index files into /index in the container.

cheers,

dustin

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/97784f62-149b-4b53-ab86-37f567bea189n%40googlegroups.com.

[bthoven]

Where is the official docker image?
I look at the script, it's true that it will create the config file if it doesn't exist. That's why I wonder why I can't find even the /index folder, let alone the config file, inside the container. Off course, I map my host folder to /index in the container (which I had to create manually because I can't find it).

[Dustin Lang]

Look, it's right here in the documentation.

docker run -p 8000:8000 -v ~/astrometry_indexes:/index astrometrynet/webservice

astrometrynet/webservice

 

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/6f1de8a3-8343-4fb6-b887-e5cf191a1a76n%40googlegroups.com.

[Dustin Lang]

I just pushed a fix for that process_submissions.py error (only happens when the database is empty, ie, in docker containers), and pushed new docker images tagged "dev": astrometrynet/webservice:dev

cheers,

dustin

[bthoven]

Thanks. I tried both the "latest" and "dev", but "illegal instruction"
root@87a1f8346148:/src# solve-field --downsample 2 https://stellarscenes.net/object/m35.jpg --no-plots --overwrite --scale-units arcsecperpix --scale-low 12 --scale-high 13
Illegal instruction

[Dustin Lang]

Okay, weird, what CPU do you have?  "cat /proc/cpuinfo" ?  (Or "docker run ubuntu:22.04 cat /proc/cpuinfo")

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/d7e4c143-c9c4-4994-87ac-75743a0db52cn%40googlegroups.com.

[bthoven]

I'm running it on Unraid. Hardware CPU is 

root@bthoven-unraid:~# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 62
model name      : Intel(R) Xeon(R) CPU E5-2650L v2 @ 1.70GHz
stepping        : 4
microcode       : 0x42e
cpu MHz         : 1696.214
cache size      : 25600 KB
physical id     : 0
siblings        : 20
core id         : 0
cpu cores       : 10
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear flush_l1d
vmx flags       : vnmi preemption_timer posted_intr invvpid ept_x_only ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips        : 3392.54
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

[Dustin Lang]

Hi,

I just pushed a new astrometrynet/solver:dev that includes "valgrind", could you please run "valgrind solve-field -v   [plus your usual command-line argos]"

?

thanks,

dustin

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/a9369b6b-20ed-47c5-98cc-20cd8ed868c6n%40googlegroups.com.

[bthoven]

here it is:

root@bthoven-unraid:~# docker run -v /mnt/user/unraid_shared_disk/Astrometry_net/index:/usr/local/data -it astrometrynet/solver:dev /bin/bash
root@2f57b477da48:/src# valgrind solve-field -v --downsample 2 https://stellarscenes.net/object/m35.jpg --no-plots --overwrite --scale-units arcsecperpix --scale-low 12 --scale-high 13
==9== Memcheck, a memory error detector
==9== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==9== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==9== Command: solve-field -v --downsample 2 https://stellarscenes.net/object/m35.jpg --no-plots --overwrite --scale-units arcsecperpix --scale-low 12 --scale-high 13
==9==
Reading input file 1 of 1: "https://stellarscenes.net/object/m35.jpg"...
Base: "./m35", basefile "m35.jpg", basedir ".", suffix "jpg"
Downloading...
Running: curl --output ./m35.jpg https://stellarscenes.net/object/m35.jpg
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  160k  100  160k    0     0   137k      0  0:00:01  0:00:01 --:--:--  137k
Checking if file "./m35.jpg" ext 0 is xylist or image: image
  (not xyls because: Failed to open FITS table ./m35.jpg: Failed to open FITS file "./m35.jpg")
Running command: /usr/local/bin/image2pnm --infile ./m35.jpg --uncompressed-outfile /tmp/tmp.uncompressed.ZKVhl9 --outfile /tmp/tmp.ppm.COqvUT --ppm --mydir /usr/local/bin/solve-field
Running: /usr/local/bin/image2pnm --infile ./m35.jpg --uncompressed-outfile /tmp/tmp.uncompressed.ZKVhl9 --outfile /tmp/tmp.ppm.COqvUT --ppm --mydir /usr/local/bin/solve-field
jpegtopnm: WRITING PPM FILE
  jpg
Running: pnmfile /tmp/tmp.ppm.COqvUT
Converting PPM image to FITS...
Running: ppmtopgm /tmp/tmp.ppm.COqvUT | /usr/local/bin/an-pnmtofits > /tmp/tmp.fits.wxiMqp
Illegal instruction
augment-xylist.c:605:run Failed to run command: ppmtopgm /tmp/tmp.ppm.COqvUT | /usr/local/bin/an-pnmtofits > /tmp/tmp.fits.wxiMqp
 ioutils.c:568:run_command_get_outputs Command failed: return value 132
==9==
==9== HEAP SUMMARY:
==9==     in use at exit: 1,702 bytes in 42 blocks
==9==   total heap usage: 1,502 allocs, 1,460 frees, 154,430 bytes allocated
==9==
==9== LEAK SUMMARY:
==9==    definitely lost: 167 bytes in 1 blocks
==9==    indirectly lost: 0 bytes in 0 blocks
==9==      possibly lost: 0 bytes in 0 blocks
==9==    still reachable: 1,535 bytes in 41 blocks
==9==         suppressed: 0 bytes in 0 blocks
==9== Rerun with --leak-check=full to see details of leaked memory
==9==
==9== For lists of detected and suppressed errors, rerun with: -s
==9== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[Dustin Lang]

Oh, it looks like the illegal instruction is coming from the "ppmtopgm" program.  That's part of the "netpbm" package, not astrometry.net.

cheers,

dustin

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/99cf8ead-7de2-474d-a8e4-22700f1cd385n%40googlegroups.com.

[bthoven]

How could we solve this?

[Dustin Lang]

The docker container is just installing the standard Ubuntu 22.04 netpbm package, so this is really not a problem I have any control over.

I would assume that if you build your own container, you will encounter the same problem.

Perhaps you could build netpbm from source.

cheers,

dustin

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/494997b3-edb4-4dc0-834e-ff6c57911bbcn%40googlegroups.com.

[bthoven]

As my docker build for the solver part works perfectly, but have problem with the webservice build part.

Would it be technically possible that you use my solver image to build the webservice image part on your end and let me test it?

My solver image is at docker hub named bthoven/astrometrynetsolver:latest.

Thanks

[bthoven]

My solver Docker build, capable of 'solve-field', was created on a Debian 11.1.0 host. I attempted to build the webservice image on top of the functioning solver image on a host running either Debian 11.1.0 or CentOS 9 Stream. Although both builds completed without errors, submitting photos through the web UI resulted in the photos being uploaded but not processed beyond being written into the data folder.
That's why I need you to help me try building webservice image on top of my working solver image.

[bthoven]

During the testing of your webservice container, the uploaded photo was processed correctly until the solving phase, where it was terminated at netpbm command. However, since working netpbm was already included in my solver image, it's highly likely that your webservice, build upon my functioning solver build, could be working.

[Dustin Lang]

Why don't you just build your own webservice container on top of your solver build --- after doing a "git pull" to get the recent fix to process_submissions.py??

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/a14d7516-198f-4553-b216-370097963025n%40googlegroups.com.

[bthoven]

That was what I did today. Still has the same result.

[Dustin Lang]

Okay, please send logs.

[bthoven]

I send here two log files. One when starting webservice container. You may notice there are "type error" messages in the log, not sure it is relevant. The other file contains log entries after I submit one photo via webui. It goes on like this forever without producing any results.

[Dustin Lang]

Like I said, you need to git pull

to get the new fix for that type error.

I fixed it yesterday.  https://github.com/dstndstn/astrometry.net/commit/3c19ff78ec3a73fd54cb89613062b48ba6124a2a

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/6d3f4003-0821-49e9-82b9-2dd007950367n%40googlegroups.com.

[bthoven]

I freshly git pull it to my Centos host today (after you fixed it yesterday, my local time gmt+7) and built from that.

[Dustin Lang]

You need to make sure the new code actually got pulled into the container.  Docker caches aggressively.

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/f6b95fa8-81fd-45b0-9c87-49176366d111n%40googlegroups.com.

[bthoven]

Thanks. It's working now. 

I rebuild both the solver and webserver. Formerly, I rebuilt just to webserver part. The fix is actually in the solver part.
As you suggest, the webservice need to have docker.cfg file inside folder /index. So now my index files and docker.cfg are in the same /index folder; and the webservice is working fine.

However, if I go into the webservice terminal and run solve-field command manually, solve-field can't find the config file; thus can't find index files. It reports this at the solving stage:You must list at least one index in the config file (/usr/local/bin/../etc/astrometry.cfg). My question is why the solve-field command running inside the container terminal can't find the config/index files?

[Dustin Lang]

You can do "solve-field -c /index/docker.cfg" to use that same config file.

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/94943a59-bc11-473d-aed8-f8c928db6d67n%40googlegroups.com.

[bthoven]

I did try -c /index/docker.cfg option, but it still can't find it. These are my command and list of files in /index folder which has docker.cfg file.

root@0e06d04eddda:/src# solve-field -c /index/docker.cfg --downsample 2 https://stellarscenes.net/object/m35.jpg --no-plots --overwrite --scale-units arcsecperpix --scale-low 12 --scale-high 13

Reading input file 1 of 1: "https://stellarscenes.net/object/m35.jpg"...

Downloading...
Running command: /usr/local/bin/image2pnm --infile ./m35.jpg --uncompressed-outfile /tmp/tmp.uncompressed.Ko4F0C --outfile /tmp/tmp.ppm.5PadvI --ppm --mydir /usr/local/bin/solve-field
jpegtopnm: WRITING PPM FILE
Read file stdin: 679 x 453 pixels x 1 color(s); maxval 255
Using 8-bit output
Extracting sources...
Downsampling by 2...
simplexy: found 147 sources.
Solving...


---------------------------------------------------------------------

You must list at least one index in the config file (/usr/local/bin/../etc/astrometry.cfg)

See http://astrometry.net/use.html about how to get some index files.
---------------------------------------------------------------------

solve-field.c:545:run_engine engine failed.  Command that failed was:
  /usr/local/bin/astrometry-engine ./m35.axy
 ioutils.c:568:run_command_get_outputs Command failed: return value 255
root@0e06d04eddda:/src# ls -l /index
total 35642336
-rw-r--r-- 1 root root      9915 Sep  9 23:12 docker.cfg
-rw-r--r-- 1 root root        37 Sep 10 20:43 docker.cfg_
-rw-rw-rw- 1 root root 164995200 Mar  6  2013 index-4107.fits
-rw-rw-rw- 1 root root  94550400 Mar  6  2013 index-4108.fits
-rw-rw-rw- 1 root root  49772160 Mar  6  2013 index-4109.fits

[bthoven]

ok. It has to be --config /index/docker.cfg

root@0e06d04eddda:/src# solve-field --config /index/docker.cfg --downsample 2 https://stellarscenes.net/object/m35.jpg --no-plots --overwrite --sca

le-units arcsecperpix --scale-low 12 --scale-high 13
Reading input file 1 of 1: "https://stellarscenes.net/object/m35.jpg"...
Downloading...

Running command: /usr/local/bin/image2pnm --infile ./m35.jpg --uncompressed-outfile /tmp/tmp.uncompressed.zPuy8E --outfile /tmp/tmp.ppm.kDHhxZ --ppm --mydir /usr/local/bin/solve-field

jpegtopnm: WRITING PPM FILE
Read file stdin: 679 x 453 pixels x 1 color(s); maxval 255
Using 8-bit output
Extracting sources...
Downsampling by 2...
simplexy: found 147 sources.
Solving...

Reading file "./m35.axy"...
  log-odds ratio 80.5621 (9.71994e+34), 12 match, 0 conflict, 12 distractors, 17 index.
  RA,Dec = (92.1873,24.2823), pixel scale 12.7044 arcsec/pix.
  Hit/miss:   Hit/miss: +-++++--++--+-++----+--+(best)----------------------------------------------------------------------------
Field 1: solved with index index-4111.fits.
Field 1 solved: writing to file ./m35.solved to indicate this.
Field: ./m35.jpg
Field center: (RA,Dec) = (92.185682, 24.283031) deg.
Field center: (RA H:M:S, Dec D:M:S) = (06:08:44.564, +24:16:58.913).
Field size: 2.3952 x 1.59957 degrees
Field rotation angle: up is -177.598 degrees E of N
Field parity: neg
Creating new FITS file "./m35.new"...

[bthoven]

Thanks a lot for your patience with me on solving the issue. I really appreciate your support.

As you always build the latest images and host them on docker hub, in the future version I may not need to build it by myself. My question is have you been able to build the working images, by avoiding the 'netbpm' problem? If not solved yet, in the mean time, you may use my current builds to be pushed to your docker hub. They are now on my docker hub repository, named:

bthoven/astrometrynetsolver:latest

bthoven/astrometrynetweb:latest

[Dustin Lang]

Oh right, oops, sorry! :)

To view this discussion on the web visit https://groups.google.com/d/msgid/astrometry/2aec90ca-ee43-43fa-9114-fcd5ed717833n%40googlegroups.com.

[bthoven]

Fyi, I built my images on Debian 11.1.0 VM (debian-11.1.0-amd64-netinst.iso) on my Unraid server.