Harvesting clients stuck

81 views
Skip to first unread message

Thomas Jouneau

unread,
Feb 11, 2021, 11:34:59 AM2/11/21
to Dataverse Users Community

Dear all

I'm having two harvesting clients stuck on "In Progress" for more than two weeks now. I can't stop or remove these clients, as buttons are greyed out.

Is there a command to manually stop them? Is it even a good idea? I want to do it properly and don't want to damage the database, as the first one has already harvested some records.

Thanks,

Thomas

Felipe Castro

unread,
Feb 12, 2021, 6:54:50 AM2/12/21
to Dataverse Users Community
Hi All!


I have the same problem. Does anyone know how to solve this?

Thanks!!

danny...@g.harvard.edu

unread,
Feb 12, 2021, 8:57:56 AM2/12/21
to Dataverse Users Community
Thomas, Felipe, sorry to hear about the trouble. I don't have an answer but I'll check around and try to get one today. I hope that others from the community will provide some details as well. As I mentioned in a recent Github issue, we'd like to get some attention on Harvesting in the near future and it sounds like this will be a good candidate.

- Danny

leo...@g.harvard.edu

unread,
Feb 12, 2021, 2:29:24 PM2/12/21
to Dataverse Users Community
Hello, 
The easiest way to reset a stuck harvest is by direct database queries: 

UPDATE clientharvestrun SET harvestresult=0 WHERE harvestingclient_id={ID} AND harvestresult = 2;

UPDATE harvestingclient SET harvestingnow = FALSE WHERE id={ID};

where {ID} is the database id of the harvesting client. 

If it's been stuck in this state for days, it's almost certain that it's not actively running anymore (i.e., something must have killed that job). But to be safe, I would make sure that Payara has been restarted since the start time of the last stuck harvest. 

I would suggest to check if there are any exception/error messages in the harvest log files (.../domain1/logs/harvest_{CLIENT}_{DATE}.log) that would help us identify what caused this. If there are no clear error messages in the harvesting logs, I would also check the general server logs (.../domain1/logs/server.log*) from around the time of the failed harvests for any entries from HarvesterServiceBean. 


Hope this helps,

-Leonid 


Thomas Jouneau

unread,
Feb 16, 2021, 2:37:23 PM2/16/21
to dataverse...@googlegroups.com, leo...@g.harvard.edu

Dear Leonid & all

I confirm the problem is now solved. Great help!

It probably originated in a server reboot - bad coordination with my admin. I'm going to perform another test run with this client.

Best

Thomas

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/65d15f50-122c-4289-8e1c-3af7005c2988n%40googlegroups.com.

Philip Durbin

unread,
Jun 11, 2021, 10:40:41 AM6/11/21
to dataverse...@googlegroups.com
I just wanted to mention that stuck harvests came up in chat today. The SQL statements above worked (thanks, Leonid!) and now there's an issue to create an API for those statements: https://github.com/IQSS/dataverse/issues/7940

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/77466f90-c621-febc-1306-9f8d3ba727bb%40gmail.com.


--
Reply all
Reply to author
Forward
0 new messages