stop query

[Jasper Koehorst]

When running some heavy queries via either python or the webinterface I encountered no way of stopping that query as the web=interface becomes non-responsive and you only see a white screen with a log at the bottom.

I also tried stardog-admin with the kill query command but this just sits and waits as it is somehow not prioritised. The only option left is then kill 9 command but i'd rather not use this command as it kills the entire application.

Are there ways to circumvent this problem?

Thanks,

[Mike Grove]

On Tue, Jul 29, 2014 at 11:44 AM, Jasper Koehorst <jasperk...@gmail.com> wrote:

When running some heavy queries via either python or the webinterface I encountered no way of stopping that query as the web=interface becomes non-responsive and you only see a white screen with a log at the bottom.

I also tried stardog-admin with the kill query command but this just sits and waits as it is somehow not prioritised. The only option left is then kill 9 command but i'd rather not use this command as it kills the entire application.

Can you provide the data and the problematic query?  We'd like to look into this more.

Cheers,

Mike

 

Are there ways to circumvent this problem?

Thanks,

--
-- --
You received this message because you are subscribed to the C&P "Stardog" group.
To post to this group, send email to sta...@clarkparsia.com
To unsubscribe from this group, send email to
stardog+u...@clarkparsia.com
For more options, visit this group at
http://groups.google.com/a/clarkparsia.com/group/stardog?hl=en

[Jasper Koehorst]

It is not really specific for a type of query. Basically when performing any kind of heavy select statement where thousands / millions of triples are generated 12 of the 16 cores of the server are in use. I have a strong feeling that then all threads are in use and a new thread with either the website or the stardog-admin command are placed in the queue and do not get priority. I'll try to see if uniprot (which is still loading) gives the same issue as this was on my own data that is currently unavailable (see other thread). 

To unsubscribe from this group and stop receiving emails from it, send an email to stardog+u...@clarkparsia.com.

[Mike Grove]

Sure, if you have a difficult query that's consuming the majority of the resources on the system things can get sluggish, but it should not fail to respond.  If 12 of 16 cores are in use, are there multiple queries running at once?  How much memory is used/unused?  If you attach VisualVM to the process, do you see it GCing constantly?

The query runtime should move intermediate results off-heap as it detects pressure on the garbage collector, so memory/GC should not be an issue.  But I'm curious about how most of the cores are getting used up.  I'm also curious if we've come up with a sub-optimal plan making Stardog work harder than it needs to.

Cheers,

Mike

[Jasper Koehorst]

When performing a for me straight forward select statement CPU% jumps to 1270% and 22827/120866MB is in use. Web interface then takes minutes to go to a new page if not longer...

For example when the query runs and I go to the overview page I get silly information:

System Uptime: 45 years

System Information: undefined

Stardog Version: undefined

Stardog Home: undefined

SELECT  ?genome

FROM <http://mydata>

WHERE { 

  ?genome ssb:class ?class .

  ?class ssb:scaffold ?scaffold .

  ?scaffold ssb:contig ?contig .

  ?contig ssb:orf ?orf .

  ?orf ssb:protein ?protein .

  ?protein ssb:tag ?tag .

  ?tag ssb:accession "2048413" .

}

Op donderdag 31 juli 2014 13:41:38 UTC+2 schreef Michael Grove:

[Mike Grove]

On Fri, Aug 1, 2014 at 9:50 AM, Jasper Koehorst <jasperk...@gmail.com> wrote:

When performing a for me straight forward select statement CPU% jumps to 1270% and 22827/120866MB is in use. Web interface then takes minutes to go to a new page if not longer...

For example when the query runs and I go to the overview page I get silly information:

You're using the community edition, which is not able to access JMX to get that information, hence the kind of weird output.  2.2.1 has a modified overview which works better for both editions.

Cheers,

Mike

[Jasper Koehorst]

When the stardog start overview says:

This is a Developer license

Is that the same as community?

[Mike Grove]

On Fri, Aug 1, 2014 at 11:04 AM, Jasper Koehorst <jasperk...@gmail.com> wrote:

When the stardog start overview says:

This is a Developer license

Is that the same as community?

No, that's a developer license, I must have mis-read which version you were using.

Cheers,

Mike

[Jasper Koehorst]

To comment on this post,

The query is as follow:

SELECT ?protein ?sequence ?iacc ?idesc
FROM <http://ssb.wur.nl/ncbi>
WHERE { <http://csb.wur.nl/genome/Akkermansia_muciniphila_ATCC_BAA_835> ssb:class ?class .
       ?class ssb:scaffold ?scaffold . #1 to 1 relation
       ?scaffold ssb:contig ?contig .  #1 to 1 relation
       ?contig ssb:orf ?orf . #1 to 1000+ relation
       ?orf ssb:protein ?protein . #1 to 1 relation
       ?protein ssb:sequence ?sequence . #Side link to string
       ?protein ssb:interpro ?interpro . #1 to 10+ relation
       ?interpro ssb:interpro_accession ?iacc . #Side link to string
       ?interpro ssb:interpro_description ?idesc . #Side link to string
 }

If 12 of 16 cores are in use, are there multiple queries running at once? This is just 1 query

  How much memory is used/unused? 54596/120866MB and stable after 12 hours of running... 

If you attach VisualVM to the process, do you see it GCing constantly? Not sure what that means....

I'm also curious if we've come up with a sub-optimal plan making Stardog work harder than it needs to. To be honest, I think so as it is really a simple straight forward connection result collection (to my opinion at least, no expert in graph quering / solving ;) )

And of course this causes the entire stardog instance to clog up, freeze, become irresponsive and unable to stop the query or browse through the web interface.

Not sure if this is of any relevance...

java version "1.7.0_55"
OpenJDK Runtime Environment (IcedTea 2.4.7) (7u55-2.4.7-1ubuntu1~0.12.04.2)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)