Error when using submitBatchJob

[Linoy Tsaban]

Hello, 

I've been using mutalyzer programatically for research for a while now. In the last few days I have not been able to run these two lines of code - 

job_id = mutalyzer.submitBatchJob(encoded_variants,process='NameChecker')
while (mutalyzer.monitorBatchJob(job_id)) > 0:
                          continue

I keep getting the error below, and I'm not sure why (since it is the same code I used before multiple times and it worked), I thought maybe you have seen it before and might be able to give me advice. 
 
urllib.error.URLError: <urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>

I've seen multiple possible explanations online, but haven't been able to resolve it yet.

Thanks!

Best
Linoy

[J.F.J. Laros]

Dear Linoy,

Thank you for using Mutalyzer.

Everything seems to work on our end, but I did notice that a large number of
batch jobs are being processed at the moment (around 480,000 variants in 50
jobs), so that could be the reason things are getting a bit slow.

Which e-mail address do you use for the services? I could check whether any of
your jobs are running at the moment.


With kind regards,
Jeroen.

> --
> You received this message because you are subscribed to the Google Groups "Mutalyzer" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mutalyzer+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mutalyzer/b5dbbca6-47c0-4d61-b2a6-1135d2221b20n%40googlegroups.com.


--
Jeroen F.J. Laros - J.F.J...@lumc.nl

[Linoy Tsaban]

Thank you for your response! 

I wasn't able to submit a job programmatically with my code, so I submitted it manually without an email.
It does work when I submit manually, I can tell there is progress, I was just wondering why my code crashes since I rather do it automatically.

Thanks again, 

Linoy 

[J.F.J. Laros]

Dear Linoy,

I can not reproduce this behaviour. Could it be that network policies have
changed at your institute (e.g., a mandatory proxy)?

Perhaps you can try the following command to test this hypothesis?

curl "https://mutalyzer.nl/json/checkSyntax?variant=AB026906.1:c.274del"


With kind regards,
Jeroen.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mutalyzer/a86af9bb-d730-419e-a8e7-c23cec4d929bn%40googlegroups.com.

[Winslow Johnson]

Greetings,

Please allow my to drop into your thread to add another example. My institution regularly uses submitBatchJob for your NameChecker service as part of our variant calling pipelines. We too have been having difficulty with this process since approximately 5:00AM UTC on August 5th.

According to our log files, our client is able to connect to mutalyzer and submit batch jobs. But then it waits for many hours, periodically calling monitorBatchJob, until we terminate our process.

Hopefully another case example is helpful here (and yes, I'm able to receive a response from the above curl command, so it's unlikely to have been a networking policy change).

Cheers,

Winslow Johnson

University of Washington - Department of Laboratory Medicine and Pathology

[Winslow Johnson]

To further clarify, in some instances we've let our processes run until the batch completes. In the past, a NameChecker submission with ~1000 entries took on the order of a minute to complete. In the past few days, it's taken ~8-24 hours.

[J.F.J. Laros]

Dear Winslow,

Thank you for this additional information.

It looks like your problems are not directly related to the connection issues
Linoy experiences.

I had a look at the database to see which jobs are in the queue and what type
of variants are being checked. It looks like large amounts of chromosomal
variants are being checked at the moment. Since the reference sequences of
these variants are large, it takes Mutalyzer a lot of time to process these
variants.

Currently, there are around 340,000 variants in the queue, it should take an
other 78 hours to process them all, after which I hope everything returns to
normal.


With kind regards,
Jeroen.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mutalyzer/dffc0cdb-ef7c-4dd1-8944-60f633a91fc1n%40googlegroups.com.