Query/Retrieve remote Conquest server - error ‘Sorry, there was a problem starting the job!’

Bahman

unread,

Oct 2, 2017, 5:01:36 AM10/2/17

to OpenREM

I get ‘Sorry, there was a problem starting the job!’ when I query a remote server. The log file in media folder does not give a clue as to the nature of the problem. When I check the remote server status it shows only a successful C-Echo request from OPENREM and not a Query/ Retrieve request. See the last two lines of the openrem log file:

2017 09:26:08] INFO [remapp.netdicom.tools:34] Association response received

[02/Oct/2017 09:26:08] INFO [remapp.netdicom.tools:114] Returning Success response from echo to 10.145.92.63 4006 BIOIMAGING

Ed McDonagh

unread,

Oct 2, 2017, 5:20:35 AM10/2/17

to Bahman, OpenREM

Hi Bahman

Have you started Celery successfully? Is there anything in the Celery log file? See http://docs.openrem.org/en/0.7.4/startservices.html#celery-task-queue for details.

Kind regards

Ed

--
You received this message because you are subscribed to the Google Groups "OpenREM" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrem+unsubscribe@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
Visit this group at https://groups.google.com/group/openrem.
For more options, visit https://groups.google.com/d/optout.

Bahman

unread,

Oct 2, 2017, 6:25:41 AM10/2/17

to OpenREM

Hi Ed

I tried to start celery using command below but I get access denied error. I am running celery from the server logged in as system admin and have given every user read/write access to media folder.

C:\Python27\Lib\site-packages\openrem>celery worker -n default -A openremproject

-c 4 -Q default --pidfile=d:\openrem\media\celery\default.pid --logfile=d:\open

rem\media\celery\default.log

-------------- celery@default v4.0.2 (latentcall)

---- **** -----

--- * *** * -- Windows-2008ServerR2-6.1.7601-SP1 2017-10-02 11:13:34

-- * - **** ---

- ** ---------- [config]

- ** ---------- .> app: openremproject:0x3b18cc0

- ** ---------- .> transport: amqp://guest:**@localhost:5672//

- ** ---------- .> results: disabled://

- *** --- * --- .> concurrency: 4 (prefork)

-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)

--- ***** -----

-------------- [queues]

.> default exchange=default(direct) key=default

C:\Python27\Lib\site-packages\openrem>Traceback (most recent call last):

File "<string>", line 1, in <module>

File "c:\python27\lib\site-packages\billiard\spawn.py", line 159, in spawn_mai

n

new_handle = steal_handle(parent_pid, pipe_handle)

File "c:\python27\lib\site-packages\billiard\reduction.py", line 126, in steal

_handle

_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)

WindowsError: [Error 5] Access is denied

On Monday, October 2, 2017 at 10:20:35 AM UTC+1, Ed McDonagh wrote:

Hi Bahman

Have you started Celery successfully? Is there anything in the Celery log file? See http://docs.openrem.org/en/0.7.4/startservices.html#celery-task-queue for details.

Kind regards

Ed

On 2 October 2017 at 10:01, Bahman <bka...@gmail.com> wrote:

I get ‘Sorry, there was a problem starting the job!’ when I query a remote server. The log file in media folder does not give a clue as to the nature of the problem. When I check the remote server status it shows only a successful C-Echo request from OPENREM and not a Query/ Retrieve request. See the last two lines of the openrem log file:

2017 09:26:08] INFO [remapp.netdicom.tools:34] Association response received

[02/Oct/2017 09:26:08] INFO [remapp.netdicom.tools:114] Returning Success response from echo to 10.145.92.63 4006 BIOIMAGING

--
You received this message because you are subscribed to the Google Groups "OpenREM" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openrem+u...@googlegroups.com.

Ed McDonagh

unread,

Oct 2, 2017, 6:43:46 AM10/2/17

to Bahman, OpenREM

That might be a problem with Celery 4.0 on Windows. Try this command instead:

celery worker -n default -P solo -Ofair -A openremproject -c 1 -Q default --pidfile=d:\openrem\media\celery\default.pid --logfile=d:\openrem\media\celery\default.log

This is the command David Platten has been using successfully on Windows with Celery 4.0

To unsubscribe from this group and stop receiving emails from it, send an email to openrem+unsubscribe@googlegroups.com.

Bahman

unread,

Oct 2, 2017, 8:38:38 AM10/2/17

to OpenREM

Thanks Ed. There was no error when running the modified command but error: ‘Sorry, there was a problem starting the job!’ persists when the query is submitted with loads of errors in .\media\celery\default.txt ( See below error listed in celery default log file).

Could it it be that I am missing an important software component?

[2017-10-02 11:13:34,244: CRITICAL/MainProcess] Unrecoverable error: TypeError('argument 1 must be an integer, not _subprocess_handle',)

Traceback (most recent call last):

File "c:\python27\lib\site-packages\celery\worker\worker.py", line 203, in start

self.blueprint.start(self)

File "c:\python27\lib\site-packages\celery\bootsteps.py", line 119, in start

step.start(parent)

File "c:\python27\lib\site-packages\celery\bootsteps.py", line 370, in start

return self.obj.start()

File "c:\python27\lib\site-packages\celery\concurrency\base.py", line 131, in start

self.on_start()

File "c:\python27\lib\site-packages\celery\concurrency\prefork.py", line 112, in on_start

**self.options)

File "c:\python27\lib\site-packages\billiard\pool.py", line 1008, in __init__

self._create_worker_process(i)

File "c:\python27\lib\site-packages\billiard\pool.py", line 1117, in _create_worker_process

w.start()

File "c:\python27\lib\site-packages\billiard\process.py", line 122, in start

self._popen = self._Popen(self)

File "c:\python27\lib\site-packages\billiard\context.py", line 383, in _Popen

return Popen(process_obj)

File "c:\python27\lib\site-packages\billiard\popen_spawn_win32.py", line 64, in __init__

_winapi.CloseHandle(ht)

TypeError: argument 1 must be an integer, not _subprocess_handle

[2017-10-02 11:19:11,930: CRITICAL/MainProcess] Unrecoverable error: TypeError('argument 1 must be an integer, not _subprocess_handle',)

Traceback (most recent call last):

File "c:\python27\lib\site-packages\celery\worker\worker.py", line 203, in start

self.blueprint.start(self)

File "c:\python27\lib\site-packages\celery\bootsteps.py", line 119, in start

step.start(parent)

File "c:\python27\lib\site-packages\celery\bootsteps.py", line 370, in start

return self.obj.start()

File "c:\python27\lib\site-packages\celery\concurrency\base.py", line 131, in start

self.on_start()

File "c:\python27\lib\site-packages\celery\concurrency\prefork.py", line 112, in on_start

**self.options)

File "c:\python27\lib\site-packages\billiard\pool.py", line 1008, in __init__

self._create_worker_process(i)

File "c:\python27\lib\site-packages\billiard\pool.py", line 1117, in _create_worker_process

w.start()

File "c:\python27\lib\site-packages\billiard\process.py", line 122, in start

self._popen = self._Popen(self)

File "c:\python27\lib\site-packages\billiard\context.py", line 383, in _Popen

return Popen(process_obj)

File "c:\python27\lib\site-packages\billiard\popen_spawn_win32.py", line 64, in __init__

_winapi.CloseHandle(ht)

TypeError: argument 1 must be an integer, not _subprocess_handle

[2017-10-02 11:20:27,957: CRITICAL/MainProcess] Unrecoverable error: TypeError('argument 1 must be an integer, not _subprocess_handle',)

Traceback (most recent call last):

File "c:\python27\lib\site-packages\celery\worker\worker.py", line 203, in start

self.blueprint.start(self)

File "c:\python27\lib\site-packages\celery\bootsteps.py", line 119, in start

step.start(parent)

File "c:\python27\lib\site-packages\celery\bootsteps.py", line 370, in start

return self.obj.start()

File "c:\python27\lib\site-packages\celery\concurrency\base.py", line 131, in start

self.on_start()

File "c:\python27\lib\site-packages\celery\concurrency\prefork.py", line 112, in on_start

**self.options)

File "c:\python27\lib\site-packages\billiard\pool.py", line 1008, in __init__

self._create_worker_process(i)

File "c:\python27\lib\site-packages\billiard\pool.py", line 1117, in _create_worker_process

w.start()

File "c:\python27\lib\site-packages\billiard\process.py", line 122, in start

self._popen = self._Popen(self)

File "c:\python27\lib\site-packages\billiard\context.py", line 383, in _Popen

return Popen(process_obj)

File "c:\python27\lib\site-packages\billiard\popen_spawn_win32.py", line 64, in __init__

_winapi.CloseHandle(ht)

TypeError: argument 1 must be an integer, not _subprocess_handle

[2017-10-02 13:22:13,944: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 2.00 seconds...

[2017-10-02 13:22:15,956: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 4.00 seconds...

[2017-10-02 13:22:19,971: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 6.00 seconds...

[2017-10-02 13:22:25,993: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 8.00 seconds...

[2017-10-02 13:22:34,115: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 10.00 seconds...

[2017-10-02 13:22:44,155: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 12.00 seconds...

[2017-10-02 13:22:56,191: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 14.00 seconds...

[2017-10-02 13:23:10,207: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 16.00 seconds...

[2017-10-02 13:23:26,247: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 18.00 seconds...

[2017-10-02 13:23:44,513: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 20.00 seconds...

[2017-10-02 13:24:04,571: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 22.00 seconds...

[2017-10-02 13:24:26,611: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 24.00 seconds...

[2017-10-02 13:24:50,815: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 10042] An unknown, invalid, or unsupported option or level was specified in a getsockopt or setsockopt call.

Trying again in 26.00 seconds...

Ed McDonagh

unread,

Oct 2, 2017, 8:44:26 AM10/2/17

to Bahman, OpenREM

Did you install RabbitMQ? http://www.rabbitmq.com/install-windows.html

To unsubscribe from this group and stop receiving emails from it, send an email to openrem+unsubscribe@googlegroups.com.

Bahman

unread,

Oct 2, 2017, 9:31:06 AM10/2/17

to OpenREM

RabbitMQ is running as a NT service. See below output from rabbitmqctl status

C:\Program Files\RabbitMQ Server\rabbitmq_server-3.6.12\sbin>rabbitmqctl status

Status of node rabbit@NNVMPACSMAIL01

[{pid,4504},

{running_applications,

[{rabbit,"RabbitMQ","3.6.12"},

{mnesia,"MNESIA CXC 138 12","4.15"},

{os_mon,"CPO CXC 138 46","2.4.2"},

{ranch,"Socket acceptor pool for TCP protocols.","1.3.0"},

{ssl,"Erlang/OTP SSL application","8.2"},

{public_key,"Public key infrastructure","1.4.1"},

{asn1,"The Erlang ASN1 compiler version 5.0","5.0"},

{rabbit_common,

"Modules shared by rabbitmq-server and rabbitmq-erlang-client",

"3.6.12"},

{syntax_tools,"Syntax tools","2.1.2"},

{xmerl,"XML parser","1.3.15"},

{compiler,"ERTS CXC 138 10","7.1"},

{crypto,"CRYPTO","4.0"},

{sasl,"SASL CXC 138 11","3.0.4"},

{stdlib,"ERTS CXC 138 10","3.4"},

{kernel,"ERTS CXC 138 10","5.3"}]},

{os,{win32,nt}},

{erlang_version,

"Erlang/OTP 20 [erts-9.0] [64-bit] [smp:2:2] [ds:2:2:10] [async-threads:64]

\n"},

{memory,

[{connection_readers,0},

{connection_writers,0},

{connection_channels,0},

{connection_other,0},

{queue_procs,2744},

{queue_slave_procs,0},

{plugins,0},

{other_proc,20932272},

{metrics,184872},

{mgmt_db,0},

{mnesia,63664},

{other_ets,1630112},

{binary,127552},

{msg_index,43912},

{code,21422798},

{atom,891849},

{other_system,21841857},

{total,67141632}]},

{alarms,[]},

{listeners,[{clustering,25672,"::"},{amqp,5672,"::"},{amqp,5672,"0.0.0.0"}]},

{vm_memory_calculation_strategy,rss},

{vm_memory_high_watermark,0.4},

{vm_memory_limit,1717800140},

{disk_free_limit,50000000},

{disk_free,52220010496},

{file_descriptors,

[{total_limit,8092},

{total_used,2},

{sockets_limit,7280},

{sockets_used,0}]},

{processes,[{limit,1048576},{used,174}]},

{run_queue,0},

{uptime,596717},

{kernel,{net_ticktime,60}}]

Ed McDonagh

unread,

Oct 2, 2017, 10:50:33 AM10/2/17

to Bahman, OpenREM

It appears you may have hit new problems with Celery not being developed for Windows anymore. Although there does seem to be some action now to make Windows a targeted platform once more, if not a first class citizen.

However, you seem to be running 4.0.2, which isn't the latest version on PyPI. David Platten, which version do you run?

Which version of amqp are you running? (pip list)

To unsubscribe from this group and stop receiving emails from it, send an email to openrem+unsubscribe@googlegroups.com.

David Platten

unread,

Oct 2, 2017, 2:16:03 PM10/2/17

to OpenREM

OK. This is related to the version of celery. Version 4.1.0 that was installed by default on my recent Windows test system doesn't work.

For windows we need to use celery version 3.1.25. Use the following command to install this specific version:

pip install celery==3.1.25

Bahman, hopefully your errors will then go away.

v3.1.25 was the last "3" version of celery. The "4" versions don't work with Windows as far as I can tell.

Regards,

David

Bahman

unread,

Oct 3, 2017, 4:16:29 AM10/3/17

to OpenREM

Hi David & Ed

Working behind proxy has complicated celery 3.1.25 installation. If my description of the problem does not make sense to you it is because I know nothing about Python and its package dependencies .

I tried to install celery 3.1.25 and besides usual Proxy Errors I get the following error:

No matching distribution found for celery==3.1.25

I then downloaded celery-3.1.25-py2.py3-none-any.whl and attempted:

pip install celery-3.1.25-py2.py3-none-any.whl

which returned error: No matching distribution found for kombu<3.1,>=3.0.37 (from celery==3.1.25)

cannot find kombu3.1 on the net to download. I cannot understand this error as I understand celery==3.1.25 needs kombu4.

The full error messages are listed below:

C:\Users\Administrator>pip install celery==3.1.25

Collecting celery==3.1.25

Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connec

tion broken by 'ProxyError('Cannot connect to proxy.', error('Tunnel connection

failed: 407 Proxy Authentication Required',))': /simple/celery/

Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connec

tion broken by 'ProxyError('Cannot connect to proxy.', error('Tunnel connection

failed: 407 Proxy Authentication Required',))': /simple/celery/

Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connec

tion broken by 'ProxyError('Cannot connect to proxy.', error('Tunnel connection

failed: 407 Proxy Authentication Required',))': /simple/celery/

Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connec

tion broken by 'ProxyError('Cannot connect to proxy.', error('Tunnel connection

failed: 407 Proxy Authentication Required',))': /simple/celery/

Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connec

tion broken by 'ProxyError('Cannot connect to proxy.', error('Tunnel connection

failed: 407 Proxy Authentication Required',))': /simple/celery/

Could not find a version that satisfies the requirement celery==3.1.25 (from v

ersions: )

No matching distribution found for celery==3.1.25

C:\openremfiles>pip install celery-3.1.25-py2.py3-none-any.whl

Processing c:\openremfiles\celery-3.1.25-py2.py3-none-any.whl

Collecting kombu<3.1,>=3.0.37 (from celery==3.1.25)

Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connec

tion broken by 'ProxyError('Cannot connect to proxy.', error('Tunnel connection

failed: 407 Proxy Authentication Required',))': /simple/kombu/