KEGG API error, abortin

166 views
Skip to first unread message

Marta Acin

unread,
Feb 27, 2020, 9:11:18 AM2/27/20
to DuctApe
Hi,

I am trying to map genomic and phenomic data for a single strain using dape map. However i get the following error:

14:55:28 - show_pathway failed!
14:55:31 - show_pathway failed!
14:56:06 - KEGG API error, aborting
14:56:06 - Failure! KEGG API error

In the log file there is an error on communication with the http server for KEGG API:

2020-02-27 14:55:10 - ductape.kegg - [DEBUG] - show_pathway failed! Attempt 8
2020-02-27 14:55:10 - ductape.kegg - [DEBUG] - HTTP Error 403: Forbidden
2020-02-27 14:55:11 - ductape.kegg - [DEBUG] - show_pathway failed! Attempt 8
2020-02-27 14:55:11 - ductape.kegg - [DEBUG] - HTTP Error 403: Forbidden
2020-02-27 14:55:28 - ductape.kegg - [WARNING] - show_pathway failed!
2020-02-27 14:55:31 - ductape.kegg - [WARNING] - show_pathway failed!
2020-02-27 14:56:06 - ductape.kegg - [ERROR] - KEGG API error, aborting
2020-02-27 14:56:06 - ductape.terminal - [ERROR] - Failure! KEGG API error
2020-02-27 14:56:06 - ductape.actions - [DEBUG] - test - this is a test - single - Thu Feb 27 12:27:07 2020 - Thu Feb 27 14:56:06 2020

Is there anyway to fix this error?

Thanks!




ductape.log

Marco Galardini

unread,
Feb 27, 2020, 12:10:57 PM2/27/20
to Marta Acin, DuctApe
Hi,

judging by the error message I believe that it could be caused by an internet connection error on your end. I tried to connect to one of the URL that failed (you can find those in the last lines of the log, the ones that start with https://www.kegg.jp/kegg-bin/show_pathway) and it did work for me. Could you try again or from a different location?

Best,
Marco

--
You received this message because you are subscribed to the Google Groups "DuctApe" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/eb335c1a-2c92-4ab5-8d69-04c0ea1eee73%40googlegroups.com.


--
Marco Galardini

Miha Tome

unread,
Apr 17, 2020, 5:55:37 AM4/17/20
to DuctApe
Hey Marco,

I've got the exact same problem (one of my analyses' log included). 

from the terminal (for dape map):
    show_pathway failed!
    KEGG API error, aborting
    Failure! KEGG API error

The link you provided tries to redirects me and then goes "5 0 does not exist"
I've tried it on multiple platforms and different locations, its just not working.


Hope this helps you anyhow.

Best,
Miha


Dne četrtek, 27. februar 2020 15.11.18 UTC+1 je oseba Marta Acin napisala:
ductape.log

Marco Galardini

unread,
Apr 17, 2020, 8:32:29 AM4/17/20
to Miha Tome, DuctApe
Thanks,

I'll give it a try again and let you know. I hope kegg didn't change their API.

Best,
Marco

--
You received this message because you are subscribed to the Google Groups "DuctApe" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/b0649f7f-1cdd-42fc-82b5-019ef3f29ba7%40googlegroups.com.

Marco Galardini

unread,
Apr 17, 2020, 9:56:33 AM4/17/20
to DuctApe
Hi all,

I think the problem is caused by kegg adding a new "meta-pathway" to their list (map01230); I have added it to the pathways to ignore and have uploaded a new DuctApe version to pypi. You can install it by "python2 -m pip install DuctApe".

Hope this helps,
Marco
To unsubscribe from this group and stop receiving emails from it, send an email to ductape-users+unsubscribe@googlegroups.com.

Miha Tome

unread,
Apr 17, 2020, 1:54:00 PM4/17/20
to DuctApe
Hey Marco,

I've updated the softwere and it seems to solve the problem! Thank you.

Best,
Miha


Dne petek, 17. april 2020 15.56.33 UTC+2 je oseba Marco Galardini napisala:
Hi all,

I think the problem is caused by kegg adding a new "meta-pathway" to their list (map01230); I have added it to the pathways to ignore and have uploaded a new DuctApe version to pypi. You can install it by "python2 -m pip install DuctApe".

Hope this helps,
Marco

On Friday, April 17, 2020 at 8:32:29 AM UTC-4, Marco Galardini wrote:
Thanks,

I'll give it a try again and let you know. I hope kegg didn't change their API.

Best,
Marco

To unsubscribe from this group and stop receiving emails from it, send an email to ductap...@googlegroups.com.

Marco Galardini

unread,
Apr 17, 2020, 2:14:26 PM4/17/20
to Miha Tome, DuctApe
Great, luckily it was an easy fix.

Best,
M

To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/23a3cbea-9e98-4f8a-a3a0-0b98f6499fe3%40googlegroups.com.

Miha Tome

unread,
Apr 17, 2020, 5:35:43 PM4/17/20
to DuctApe
I've gor the same error again, but later in the process. Another new map or something similar?

23:17:08 - Plotting maps for biolog category carbon
23:17:08 - 1 - Checking connectivity
23:17:11 - 2 - Making room
23:17:11 - 3 - Fetching maps (pictures)
                                       Fetching maps (pictures)                                      
 64% [==========================================================---------------------------------]
                                        Item 280 on 434 total                                        23:29:27 - show_pathway failed!
23:29:40 - KEGG API error, aborting
23:29:40 - Failure! KEGG API error



Dne petek, 17. april 2020 20.14.26 UTC+2 je oseba Marco Galardini napisala:
Great, luckily it was an easy fix.

Best,
M

ductape.log

Marco Galardini

unread,
Apr 22, 2020, 11:40:08 AM4/22/20
to DuctApe
Hi,

thanks for spotting this other error; I have made yet another update that should catch any remaining issue of this kind. Please let me know if that is not the case.

Best,
Marco

Miha Tome

unread,
Apr 22, 2020, 11:58:40 AM4/22/20
to DuctApe
Hey Marco,

I'm getting an error when installing the newest version. Copy from the terminal:

python2 -m pip install DuctApe
Collecting DuctApe
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-bwMa4f/DuctApe/setup.py", line 17, in <module>
        long_description = open('README.md').read(),
    IOError: [Errno 2] No such file or directory: 'README.md'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-bwMa4f/DuctApe/



Is it something to solve on my side or is there a flaw on your end?


Dne sreda, 22. april 2020 17.40.08 UTC+2 je oseba Marco Galardini napisala:

Marco Galardini

unread,
Apr 22, 2020, 2:08:15 PM4/22/20
to DuctApe
Sorry, my bad! Should be fixed now

Miha Tome

unread,
Apr 22, 2020, 4:28:17 PM4/22/20
to DuctApe
The update is working now, thanks (:

However, I ran another mapping and it got the same error but a lot later:

22:11:09 - Plotting maps for biolog category nutrient stimulation
22:11:09 - 1 - Checking connectivity
22:11:12 - 2 - Making room
22:11:12 - 3 - Fetching maps (pictures)
                                       Fetching maps (pictures)                                      
 55% [==================================================-----------------------------------------]
                                        Item 240 on 430 total                                        22:21:21 - show_pathway failed!
22:22:12 - KEGG API error, aborting
22:22:12 - Failure! KEGG API error


Log included.

Best,
Miha

Dne sreda, 22. april 2020 20.08.15 UTC+2 je oseba Marco Galardini napisala:
ductape.log

Marco Galardini

unread,
Apr 22, 2020, 4:47:29 PM4/22/20
to Miha Tome, DuctApe
Ah I see, that's due to your URL being more than 2000 characters long. I will look into a fix later in the week.
Sorry about that.

Best,
Marco

To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/e1d21d13-d51f-4977-8ec8-cc18e7efb607%40googlegroups.com.


--
Marco Galardini

Marco Galardini

unread,
Apr 23, 2020, 10:21:40 AM4/23/20
to DuctApe
Should be "fixed" now; I have added a mechanism to reduce the length of the URL being sent to KEGG. If it's still too long it will throw a warning message and move on.
M
To unsubscribe from this group and stop receiving emails from it, send an email to ductape-users+unsubscribe@googlegroups.com.


--
Marco Galardini

Miha Tome

unread,
May 18, 2020, 4:00:53 AM5/18/20
to DuctApe
Hey Marco,

some form of this error is still persistant:

07:37:13 - Plotting maps for biolog category nutrient stimulation
07:37:13 - Showing differences in the activity >= 3
07:37:13 - 1 - Checking connectivity
07:37:17 - 2 - Making room
07:37:17 - 3 - Fetching maps (pictures)
                                       Fetching maps (pictures)                                      
 55% [==================================================-----------------------------------------]
                                        Item 240 on 430 total                                        07:45:54 - URL too long for pathway map00230, will skip
07:45:58 - URL too long for pathway map00230, will skip
07:46:06 - URL too long for pathway map00230, will skip
07:46:15 - URL too long for pathway map00230, will skip
07:46:26 - URL too long for pathway map00230, will skip
07:46:42 - URL too long for pathway map00230, will skip
07:46:58 - URL too long for pathway map00230, will skip
07:47:14 - URL too long for pathway map00230, will skip
07:47:34 - show_pathway failed!
07:48:24 - KEGG API error, aborting
07:48:24 - Failure! KEGG API error

added .log in the attachment as well.


Best,
Miha

Dne četrtek, 23. april 2020 16.21.40 UTC+2 je oseba Marco Galardini napisala:
Should be "fixed" now; I have added a mechanism to reduce the length of the URL being sent to KEGG. If it's still too long it will throw a warning message and move on.
M

On Wednesday, April 22, 2020 at 4:47:29 PM UTC-4, Marco Galardini wrote:
Ah I see, that's due to your URL being more than 2000 characters long. I will look into a fix later in the week.
Sorry about that.

Best,
Marco

ductape.log

Marco Galardini

unread,
May 18, 2020, 8:22:31 AM5/18/20
to Miha Tome, DuctApe
Hi,

sorry, I had forgotten to properly test the original fix. It should work now; could you download the latest master (https://github.com/combogenomics/DuctApe/archive/master.zip) and try yourself?

Best,
Marco

To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/d64461ca-3804-47e0-8cce-334bd4d54bed%40googlegroups.com.


--
Marco Galardini

Miha Tome

unread,
May 18, 2020, 9:04:25 AM5/18/20
to DuctApe
Sorry Marco, I'm a bit dumb when it comes to softwere in the unix enviromnet.

How can I test this, should I just copy the new files over the old ones?
(tried the "python2 -m pip install DuctApe"  but this doesn't seem to update, the version stays 0.18.0)

Best,
Miha

Dne ponedeljek, 18. maj 2020 14.22.31 UTC+2 je oseba Marco Galardini napisala:
Hi,

sorry, I had forgotten to properly test the original fix. It should work now; could you download the latest master (https://github.com/combogenomics/DuctApe/archive/master.zip) and try yourself?

Best,
Marco

Marco Galardini

unread,
May 18, 2020, 9:20:05 AM5/18/20
to Miha Tome, DuctApe
No worries!

You can download the zip archive I linked in my previous email, extract it somewhere, "cd" into it and then run your analysis normally by adding "python2" in front of each command (i.e. "python2 dape start").

Hope this helps,
Marco

To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/38be7e6d-5f85-440c-8917-dd3245bda63e%40googlegroups.com.


--
Marco Galardini

Marco Galardini

unread,
May 19, 2020, 5:31:55 PM5/19/20
to DuctApe
Hi,

I just uploaded a new version of DuctApe, which can be installed through pip (e.g. "python -m pip install DuctApe --upgrade"). Please let me know if the problem still persists.

Best,
Marco
To unsubscribe from this group and stop receiving emails from it, send an email to ductape-users+unsubscribe@googlegroups.com.


--
Marco Galardini

Miha Tome

unread,
May 21, 2020, 10:18:20 AM5/21/20
to DuctApe
Hey Marco,

I've tested the previous build and now this one.

The single organism went through without any problems, but when I ran mutant comparison:

15:57:22 - 4 - Generating interactive web pages
15:57:23 - 430 maps are available in /home/tmzl/Downloads/DuctApe-latest/Lasko/Lab_18_31_v5/tmp/keggmaps/add_carbon
15:57:23 - Plotting maps for biolog category nitrogen
15:57:23 - Showing differences in the activity >= 3
15:57:23 - 1 - Checking connectivity
15:57:26 - 2 - Making room
15:57:26 - 3 - Fetching maps (pictures)
                                       Fetching maps (pictures)                                      
 65% [===========================================================--------------------------------]
                                        Item 280 on 430 total                                        
16:10:10 - Failure! ('The read operation timed out',)

I've included the log in the attachment. 

I'll do a rerun, if there was just some glitch in the connection.

Best,
Miha

Dne torek, 19. maj 2020 23.31.55 UTC+2 je oseba Marco Galardini napisala:
Hi,

I just uploaded a new version of DuctApe, which can be installed through pip (e.g. "python -m pip install DuctApe --upgrade"). Please let me know if the problem still persists.

Best,
Marco

On Monday, May 18, 2020 at 9:20:05 AM UTC-4, Marco Galardini wrote:
No worries!

You can download the zip archive I linked in my previous email, extract it somewhere, "cd" into it and then run your analysis normally by adding "python2" in front of each command (i.e. "python2 dape start").

Hope this helps,
Marco

ductape.log

Miha Tome

unread,
May 21, 2020, 10:27:23 AM5/21/20
to DuctApe
ok, it happened again, but a lot earlier this time:

16:18:28 - 2 mutants are present
16:18:28 - Going to generate mutants metabolic map
16:18:28 - Going to generate the metabolic map for reference Gen1A_LAB
16:18:28 - Generating the reactions-only map
16:18:28 - 1 - Checking connectivity
16:18:33 - 2 - Making room
16:18:33 - 3 - Fetching maps (pictures)
                                       Fetching maps (pictures)                                      
  9% [========-----------------------------------------------------------------------------------]
                                         Item 40 on 430 total                                        
16:24:42 - Failure! ('The read operation timed out',)




Dne četrtek, 21. maj 2020 16.18.20 UTC+2 je oseba Miha Tome napisala:
ductape.log

Marco Galardini

unread,
May 21, 2020, 10:43:56 AM5/21/20
to Miha Tome, DuctApe
Hi,

it does seem a glitch due to a poor connection; how's you internet speed? Also, which python version are you using? It should do 8 attempts before stopping, so it might be that you are using python3 and the retry mechanism is not working as expected. Maybe try: "python2 dape map ..."?

M

To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/a2d8092e-4bd2-4474-8303-97375334427f%40googlegroups.com.


--
Marco Galardini

Miha Tome

unread,
May 22, 2020, 6:42:04 PM5/22/20
to DuctApe
Hey Marco,

generally the connection I'm using is stable, however I can't guarantee that there wasn't anything fishy happening at that moment.

I've ran the analysis a couple of times now, with python2 as a prefix, and it finishes everytime, which is great! Thanks for all the updates and everything.

(my python versions are Python 2.7.17; Python 3.6.9 - would it even run, if the default was python3?)

Cheers,
Miha



Dne četrtek, 21. maj 2020 16.43.56 UTC+2 je oseba Marco Galardini napisala:
Hi,

it does seem a glitch due to a poor connection; how's you internet speed? Also, which python version are you using? It should do 8 attempts before stopping, so it might be that you are using python3 and the retry mechanism is not working as expected. Maybe try: "python2 dape map ..."?

M

Marco Galardini

unread,
May 22, 2020, 7:51:37 PM5/22/20
to Miha Tome, DuctApe
Hi Miha,

glad to hear it worked; I recently made the code compatible with python3, but there seems to be problems with the way exceptions happen inside threads (i.e. they propagate upstream and kill the parent process). I'll eventually fix the issue, but in the meantime please use python2.

Best,
Marco

To unsubscribe from this group and stop receiving emails from it, send an email to ductape-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ductape-users/5dd0975f-8bea-4351-b002-5968c876ec7b%40googlegroups.com.


--
Marco Galardini
Reply all
Reply to author
Forward
0 new messages