Unable to disambiguate publications completely due to dropped web connections

62 views
Skip to first unread message

Meeks, Eric

unread,
Nov 16, 2022, 3:02:05 PM11/16/22
to profi...@googlegroups.com

Hey there. I at least *think* this is the problem. We run disambiguation once a week and we’re noticing that a bunch of the calls to get PMID’s from the http://profiles.catalyst.harvard.edu/services/GetPMIDs/default.asp endpoint are failing because the server is dropping the connection (details below). Is there anything we can do to prevent this? We haven’t had a clean run in quite a while now. Ideally (for us) this could be resolved on the server end but if it requires us to edit the script in the ProfilesRNS_CallPRNSWebservice to slow it down or make it robust, we can do that.

 

Anyone else seeing this?

 

Thanks,

Eric

 

Date                      11/13/2022 7:26:58 AM

Log                         Job History (Profiles PROD uc_PubMedDisambiguation_GetNewPubs)

 

Step ID                  1

Server                   SFPRF-PSIS02

Job Name                            Profiles PROD uc_PubMedDisambiguation_GetNewPubs

Step Name                          PubMedDisambiguation_GetPubs

Duration                              1.04:02:00

Sql Severity        0

Sql Message ID  0

Operator Emailed            

Operator Net sent          

Operator Paged

Retries Attempted          0

 

Message

Executed as user: CAMPUS\svc-ist-sql-sfis-prd. ...nexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host     at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)     --- End of inner exception stack trace ---     at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)     at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)     at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)     --- End of inner exception stack trace ---     at ST_5bdd6f6ad0c5405f9e307439c256c199.ScriptMain.Main()  Microsoft (R) SQL Server Execute Package Utility  Version 13.0.6300.2 for 64-bit  Copyright (C) 2016 Microsoft. All rights reserved.    Started:  7:26:58 AM  Error: 2022-11-13 08:44:24.19     Code: 0x00000004     Source: Store Error      Description: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script button and make sure it builds successfully.  End Error  Error: 2022-11-13 08:44:24.21     Code: 0xC0024107     Source: Store Error      Description: There were errors during task validation.  End Error  Error: 2022-11-13 08:44:24.21     Code: 0x00000006     Source: Script Task      Description: The script returned a failure result.  End Error  Error: 2022-11-13 10:35:47.76     Code: 0x00000004     Source: Store Error      Description: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script button and make sure it builds successfully.  End Error  Error: 2022-11-13 10:35:47.76     Code: 0xC0024107     Source: Store Error      Description: There were errors during task validation.  End Error  Error: 2022-11-13 10:35:47.76     Code: 0x00000006     Source: Script Task      Description: The script returned a failure result.  End Error  Error: 2022-11-13 13:18:14.17     Code: 0x00000004     Source: Store Error      Description: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script button and make sure it builds successfully.  End Error  Error: 2022-11-13 13:18:14.17     Code: 0xC0024107     Source: Store Error      Description: There were errors during task validation.  End Error  Error: 2022-11-13 13:18:14.17     Code: 0x00000006     Source: Script Task      Description: The script returned a failure result.  End Error  Error: 2022-11-13 13:58:24.53     Code: 0x00000004     Source: Store Error      Description: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script button and make sure it builds successfully.  End Error  Error: 2022-11-13 13:58:24.53     Code: 0xC0024107     Source: Store Error      Description: There were errors during task validation.  End Error  Error: 2022-11-13 13:58:24.53     Code: 0x00000006     Source: Script Task      Description: The script returned a failure result.  End Error  Error: 2022-11-13 14:03:37.69     Code: 0x00000004     Source: Store Error      Description: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script button and make sure it builds successfully.  End Error  Error: 2022-11-13 14:03:37.69     Code: 0xC0024107     Source: Store Error      Description: There were errors during task validation.  End Error  Error: 2022-11-13 14:03:37.69     Code: 0x00000006     Source: Script Task      Description: The script returned a failure result.  End Error  Error: 2022-11-13 15:59:36.08     Code: 0x00000004     Source: Store Error      Description: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script ...  The package execution fa...  The step failed.

 

 

Eric Meeks
My pronouns: he/him/his
CTO, Clinical & Translational Science Institute

University of California, San Francisco
490 Illinois St. 6th Floor

San Francisco, CA 94158
https://profiles.ucsf.edu/eric.meeks

 

Tim VanWagoner

unread,
Nov 16, 2022, 11:14:42 PM11/16/22
to ProfilesRNS
same thing happening here. Thought it might be something with our servers or setup. It happens about once per job and its a different profile each time.

it doesn't happen with a test database of just 250 names.

Brown, Nicholas William

unread,
Nov 17, 2022, 9:08:20 AM11/17/22
to profi...@googlegroups.com

We have had some problems on our disambiguation database server over the last few weeks. We are in the process of moving the disambiguation database onto new hardware, which we expect to complete in the next week. Hopefully this will fix these problems.

 

Nick

--
You received this message because you are subscribed to the Google Groups "ProfilesRNS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to profilesrns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/profilesrns/426fd5f7-1d7f-414e-9845-92facee86169n%40googlegroups.com.

Meeks, Eric

unread,
Nov 17, 2022, 1:34:45 PM11/17/22
to profi...@googlegroups.com

Got it.

We’ll let you know how things are in a few weeks. Because we have 6 institutions in one DB we hit the server pretty hard when disambiguation runs, so we’re probably a good stress test.

Thanks for the quick response!

Eric

 

From: profi...@googlegroups.com <profi...@googlegroups.com> On Behalf Of Brown, Nicholas William
Sent: Thursday, November 17, 2022 6:08 AM
To: profi...@googlegroups.com
Subject: RE: [ProfilesRNS] Re: Unable to disambiguate publications completely due to dropped web connections

 

We have had some problems on our disambiguation database server over the last few weeks. We are in the process of moving the disambiguation database onto new hardware, which we expect to complete in the next week. Hopefully this will fix these

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Meeks, Eric

unread,
Dec 7, 2022, 2:17:47 PM12/7/22
to profi...@googlegroups.com

Hey there. Just wondering, is the new server up? We are still seeing timeouts for a large percent of our calls so I wanted to check. Should we pick a different schedule or time when things are more quiet?

Thanks,

Eric

 

From: profi...@googlegroups.com <profi...@googlegroups.com> On Behalf Of Brown, Nicholas William


Sent: Thursday, November 17, 2022 6:08 AM
To: profi...@googlegroups.com

Subject: RE: [ProfilesRNS] Re: Unable to disambiguate publications completely due to dropped web connections

 

We have had some problems on our disambiguation database server over the last few weeks. We are in the process of moving the disambiguation database onto new hardware, which we expect to complete in the next week. Hopefully this will fix these

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

We have had some problems on our disambiguation database server over the last few weeks. We are in the process of moving the disambiguation database onto new hardware, which we expect to complete in the next week. Hopefully this will fix these problems.

Brown, Nicholas William

unread,
Dec 7, 2022, 2:48:15 PM12/7/22
to profi...@googlegroups.com

Eric

 

Unfortunately we are still working on transitioning to the new server. Hopefully it will be complete in a few days.

 

Nick

Meeks, Eric

unread,
Dec 7, 2022, 2:49:47 PM12/7/22
to profi...@googlegroups.com

That’s actually good news 😊

Thanks for the update!

 

From: profi...@googlegroups.com <profi...@googlegroups.com> On Behalf Of Brown, Nicholas William
Sent: Wednesday, December 7, 2022 11:48 AM
To: profi...@googlegroups.com
Subject: RE: [ProfilesRNS] Re: Unable to disambiguate publications completely due to dropped web connections

 

Eric Unfortunately we are still working on transitioning to the new server. Hopefully it will be complete in a few days. Nick From: 'Meeks, Eric' via ProfilesRNS <profilesrns@ googlegroups. com> Sent: Wednesday, December 7, 2022 2: 18 PM

Meeks, Eric

unread,
Dec 21, 2022, 2:20:08 PM12/21/22
to profi...@googlegroups.com

Hello again. Seems like the new server might be in place based on some luck we’ve had but I wanted to check. Any news on this?

Happy Holidays!

Thanks,

Eric

 

From: profi...@googlegroups.com <profi...@googlegroups.com> On Behalf Of Brown, Nicholas William
Sent: Wednesday, December 7, 2022 11:48 AM
To: profi...@googlegroups.com
Subject: RE: [ProfilesRNS] Re: Unable to disambiguate publications completely due to dropped web connections

 

Eric Unfortunately we are still working on transitioning to the new server. Hopefully it will be complete in a few days. Nick From: 'Meeks, Eric' via ProfilesRNS <profilesrns@ googlegroups. com> Sent: Wednesday, December 7, 2022 2: 18 PM

Meeks, Eric

unread,
Jan 12, 2023, 1:34:57 PM1/12/23
to profi...@googlegroups.com

We are still seeing issues with the disambiguation service. Any update on this?

Thanks and happy new year to everyone!

Eric

 

From: 'Meeks, Eric' via ProfilesRNS <profi...@googlegroups.com>
Sent: Wednesday, December 21, 2022 11:20 AM
To: profi...@googlegroups.com
Subject: RE: [ProfilesRNS] Re: Unable to disambiguate publications completely due to dropped web connections

 

Hello again. Seems like the new server might be in place based on some luck we’ve had but I wanted to check. Any news on this? Happy Holidays! Thanks, Eric From: profilesrns@ googlegroups. com <profilesrns@ googlegroups. com> On Behalf Of

Brown, Nicholas William

unread,
Jan 12, 2023, 1:45:30 PM1/12/23
to profi...@googlegroups.com

Eric, we are still working to transition to the new server.

Meeks, Eric

unread,
Jan 31, 2023, 2:08:10 PM1/31/23
to profi...@googlegroups.com

Just want to check on this again. We were able to disambiguate without error last weekend so I assume the new server is in place? In any case whatever happened seems to have worked!

Thanks,

Eric

 

From: profi...@googlegroups.com <profi...@googlegroups.com> On Behalf Of Brown, Nicholas William
Sent: Thursday, January 12, 2023 10:45 AM
To: profi...@googlegroups.com
Subject: RE: [ProfilesRNS] Re: Unable to disambiguate publications completely due to dropped web connections

 

Eric, we are still working to transition to the new server. Nick From: 'Meeks, Eric' via ProfilesRNS <profilesrns@ googlegroups. com> Sent: Thursday, January 12, 2023 1: 35 PM To: profilesrns@ googlegroups. com Subject: RE: [ProfilesRNS] Re: 

Brown, Nicholas William

unread,
Feb 1, 2023, 9:25:37 AM2/1/23
to profi...@googlegroups.com

Eric

 

The new server is not in place yet. I suspect the old server is just responding quicker as we have been transitioning other tasks off it.

 

I currently have everything working on the new server and am finishing testing right now. I expect to transition to the new server tomorrow or Friday.

Tim VanWagoner

unread,
Mar 7, 2023, 10:58:40 AM3/7/23
to ProfilesRNS
Just wanted to ask if this is now working as I'm still having failures on the disambiguation runs.

Brown, Nicholas William

unread,
Mar 10, 2023, 2:04:16 PM3/10/23
to profi...@googlegroups.com

We are hitting the new servers for disambiguation, in general these are running faster and with better stability than the old servers, and are being updated with new publications weekly. However we seem to have a timeout somewhere that is causing some failures, I am trying to track this down and hope to get it fixed in the next few days.

Reply all
Reply to author
Forward
0 new messages