Container service stuck.... XNAT 1.8

[Daniel Beasley]

Hi,

We're running dcm2niix on a large number of sessions, xnat 1.8. After running, it is throwing an error, this fromt he container.log:

ERROR org.nrg.xnat.configuration.SchedulerConfig - Something happened while hajava.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans vat org.springfram

So after running successfully on many, now there are sessions stuck on 'finalizing' and '_waiting'.....

Any idea how to fix?

Many thanks,

Daniel

[Daniel Beasley]

Ah, that was from configuration.log...

[Daniel Beasley]

I did see these errors as well:

org.nrg.xnat.eventservice.services.impl.EventSubscriptionEntityServiceImpl - Could not load Action Provider for key:org.nrg.containers.services.CommandActionProvider:1

ERROR org.nrg.xnat.eventservice.services.impl.EventSubscriptionEntityServiceImpl - Could not validate Action: org.nrg.containers.services.CommandActionProvider:1.

I then cleared all stopped docker containers ( a lot of dcme2niix), and it seems to be 'completing' a lot of them. I forgot to say, I'm running XNAT as a docker stack, using swarm. I may need to play with my settings?

Dan

[Herrick, Rick]

Can you post more of the stacktrace from each of those errors?

 

-- 

Rick Herrick

Sr. Programmer/Analyst

Neuroinformatics Research Group

Washington University School of Medicine

Phone: +1 (314) 273-1645

 

From: xnat_di...@googlegroups.com <xnat_di...@googlegroups.com> on behalf of Daniel Beasley <dgd...@gmail.com>
Date: Monday, April 26, 2021 at 3:13 AM
To: xnat_discussion <xnat_di...@googlegroups.com>
Subject: [XNAT Discussion] Re: Container service stuck.... XNAT 1.8

* External Email - Caution *

--
You received this message because you are subscribed to the Google Groups "xnat_discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to xnat_discussi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/xnat_discussion/c41dd0d6-3b29-4721-999e-bc2630688e21n%40googlegroups.com.

 

---

The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.

[kel...@wustl.edu]

Hi Dan,

The error 

        org.nrg.xnat.eventservice.services.impl.EventSubscriptionEntityServiceImpl - Could not load Action Provider for key:org.nrg.containers.services.CommandActionProvider:1

        ERROR org.nrg.xnat.eventservice.services.impl.EventSubscriptionEntityServiceImpl - Could not validate Action: org.nrg.containers.services.CommandActionProvider:1.

is from Event Service, indicating that the service can no longer find a Container Service command (id:1) linked to a triggered subscription. This usually means that the command associated with an event subscription was deleted. Note that deleting and adding a command with the same name will still invalidate the Event Service subscription because subscriptions and commands are associated via command id, which is different for every new command.

You can revalidate a subscription by opening the Event Service page, editing the subscription in question, and choosing the new target command action.

I'm not sure if this is related to the error you're seeing in configuration.log.  It would be good to see the full stacktrace, as Rick suggests.

Best,

Matt

[Daniel Beasley]

Hi Rick, Matt,

Ah, yep, I think the subscriptions were changed a bit, I think there was duplications (Command Automation on container service as well) so I deleted an event subscription I think. Can I remove things from the JMS Queue? The container service still works, but I still have that _waiting and _finalising ones.

2021-04-23 08:45:40,414 [taskScheduler-4] ERROR org.nrg.xnat.configuration.SchedulerConfig - Something happened while handling a scheduled task

java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext
    at org.springframework.context.support.AbstractRefreshableApplicationContext.getBeanFactory(AbstractRefreshableApplicationContext.java:164)
    at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1088)
    at org.nrg.framework.services.ContextService.getBean(ContextService.java:143)
    at org.nrg.xdat.security.helpers.Users.getAdminUser(Users.java:211)
    at org.nrg.containers.jms.tasks.QueueManager.run(QueueManager.java:41)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Many thanks,

Dan