PHINMS 3.0 Error

66 views
Skip to first unread message

Bruce Riddle

unread,
Mar 12, 2018, 3:13:19 PM3/12/18
to PHINMS User Community
Since January 26th, we have been trying to get PHINMS 3.0 up and running on Windows Server 2012 with 
SQL 2012 SP4.   PHINMS is one server and SQL on a second server behind a firewall. 

Many, many configurations changes have been tried and we keep 
getting the same message error highlighted in red below. It is very consistent. 

 We are a receive only using Route Not Read.  Everything
worked well under PHINMS 2.8 for what seems like years.  

SENDER LOG

Thread-87|03/12|08:50:43|C:\PHINMS30/shared/sendermultiblockDir// cleaner stopped!|

http-nio-5088-exec-6|03/12|08:50:44|Warning: DatabaseInfo:useridentifier unspecified|

http-nio-5088-exec-6|03/12|08:50:44|Warning: DatabaseInfo:passwordidentifier unspecified|

http-nio-5088-exec-6|03/12|08:50:44|Processing folderMap: C:\PHINMS30/config/sender/foldermap.xml|

http-nio-5088-exec-6|03/12|08:50:44|Error, decryption keystore: C:\PHINMS30/security/sender/cert.pfx not found|

http-nio-5088-exec-6|03/12|08:50:44|Error opening decryption keystore: C:\PHINMS30/security/sender/cert.pfx|

http-nio-5088-exec-6|03/12|08:50:44|Warning: keystore C:\PHINMS30/security/sender/cert.pfx does not exist|

http-nio-5088-exec-6|03/12|08:50:44|=== Spawning queue 0 ============|

Thread-90|03/12|08:50:44|Initializing requeue cachepath from C:\PHINMS30/\shared\requeueCache|

Thread-90|03/12|08:50:44|Spawning multi database poller 1...|

Thread-91|03/12|08:50:44|Connection established|

Thread-91|03/12|08:50:44|Waiting for records ...|

http-nio-5088-exec-6|03/12|08:50:44|Error creating connection pool for dbid: RNR|

http-nio-5088-exec-6|03/12|08:50:44||


I also errors in other logs that I am told to ignore (running 32 bit library in 64 bit environment, etc.) .


I still  see the message in the Wrapper.log about a conflict:

INFO   | jvm 1    | 2018/03/09 15:38:12 | WARNING - Unable to load the Wrapper's native library 'wrapper.dll'.
INFO   | jvm 1    | 2018/03/09 15:38:12 |           The file is located on the path at the following location but
INFO   | jvm 1    | 2018/03/09 15:38:12 |           could not be loaded:
INFO   | jvm 1    | 2018/03/09 15:38:12 |             C:\PHINMS30\bin\dbtools\hsqldb\.\wrapper.dll

The Tomcat log continues to bark about:

2018-03-12 08:11:30 Commons Daemon procrun stderr initialized
12-Mar-2018 08:11:35.116 SEVERE [main] org.apache.catalina.core.AprLifecycleListener.init An incompatible version 1.1.33 of the APR based Apache Tomcat Native library is installed, while Tomcat requires version 1.2.6
12-Mar-2018 08:11:35.182 SEVERE [main] org.apache.catalina.core.AprLifecycleListener.init An incompatible version 1.1.33 of the APR based Apache Tomcat Native library is installed, while Tomcat requires version 1.2.6
12-Mar-2018 08:11:35.187 SEVERE [main] org.apache.catalina.core.AprLifecycleListener.init An incompatible version 1.1.33 of the APR based Apache Tomcat Native library is installed, while Tomcat requires version 1.2.6
12-Mar-2018 08:11:35.187 SEVERE [main] org.apache.catalina.core.AprLifecycleListener.init An incompatible version 1.1.33 of the APR based Apache Tomcat Native library is installed, while Tomcat requires version 1.2.6
12-Mar-2018 08:11:35.200 SEVERE [main] org.apache.catalina.core.AprLifecycleListener.init An incompatible version 1.1.33 of the APR based Apache Tomcat Native library is installed, while Tomcat requires version 1.2.6
12-Mar-2018 08:11:35.204 SEVERE [main] org.apache.catalina.core.AprLifecycleListener.init An incompatible version 1.1.33 of the APR based Apache Tomcat Native library is installed, while Tomcat requires version 1.2.6
12-Mar-2018 08:11:35.314 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/8.5.11
12-Mar-2018 08:11:35.314 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Jan 10 2017 21:02:52 UTC
12-Mar-2018 08:11:35.314 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         8.5.11.0
12-Mar-2018 08:11:35.314 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Windows Server 2012
12-Mar-2018 08:11:35.314 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            6.2
12-Mar-2018 08:11:35.314 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
12-Mar-2018 08:11:35.315 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             C:\PHINMS30\jdk\jre
12-Mar-2018 08:11:35.315 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.7.0_79-b15
12-Mar-2018 08:11:35.315 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
12-Mar-2018 08:11:35.315 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         C:\PHINMS30\appserver
12-Mar-2018 08:11:35.315 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         C:\PHINMS30\appserver

Any ideas on what 'Error creating connection pool for dbid' actually refers to in PHINMS?  Any way to trace the error.  Connections to
the SQL database appear to work because we can see the queues and the message logs.   We have tried many numbers in the Pool Size
under Sender Configuration, Database, Database Item.  

The conversations in this group imply that some have PHINMS 3.0 up and running.  Any suggestions?

Thank you.

Bruce 

Schneider, Edward (MNIT)

unread,
Mar 12, 2018, 4:24:45 PM3/12/18
to PHINMS User Community

I am willing to venture a possibly less-than-informed opinion on part of this.  First, within SQL Server, check the configured size for the maximum number of connections, then the actual number of connections being made when the error you're experiencing occurs.  For a generally analogous set of procedures, see https://dba.stackexchange.com/questions/51219/max-connection-pool-capped-at-100.  You may have to set the number of connections you need in the connection string (within the PHINMS sender.xml and receiver.xml files tagged values for <databaseUrl>, or via the PHINMS console settings for those tags), as discussed in the link.  (And, by the way, is the related database in SQL Server actually called RNR?  Would there have been some change in that, as compared with your PHINMS 2.8 SQL Server backend setup?  Are your SQL Server login credentials up to date?)


As to the Tomcat-Native Library, the Tomcat 8.5 installed with PHINMS v. 3.0 has several alternative capabilities, depending on the configuration values in play in <PHINMS_installation_folder>\appserver\conf\server.xml, for how SSL/TLS works in the absence of a front-end proxying application (like IIS).  On the one hand, the default PHINMS 3.0 configuration relies on the Java Secure Sockets extension (JSSE).  Recent versions of JSSE can work with the latest Tomcat Native Libraries (version 1.2.6 minimum, based on your error message).  These permit fast, high-volume processing with the Java NIO connector, and can implicitly invoke OpenSSL techniques.  If your Tomcat native library version is earlier, NIO processing is considerably slower--although if your typical throughput is modest, that might not be significant; and there would also be no invoking of OpenSSL techniques.  The "traditional" (?) alternative was to use an APR [Apache Portable Runtime] configuration with OpenSSL (OpenSSL is not provided with PHINMS itself).  If your APR configurations are commented out (<!-- to start with and --> to end the comment) in the server.xml, then they aren't related to your problem with the library.  --The other side of this issue is that making a choice between the JSSE method and the APR method can affect the cipher suites available for encryption.  As PHINMS v. 3.0 relies on Java 7, the more sophisticated ciphers available lag behind current best choices--in particular they were limited by relying--at least in part--on CBC [chain -blocking] components, which have some potentially exploitable vulnerabilities.  To the extent that the recent Tomcat Native Library permits implicit JSSE use of OpenSSL capabilities, the available ciphers MIGHT be expanded--but I have yet to test that proposition.  (Note that if your traffic is supplementally encrypted at the message, as well as the SSL/TLS, levels, with different certificates at the respective levels, there may be less concern about any SSL-level encryption vulnerabilities.)

I'm running SQL Server 2008 R2 SP1, on a Windows Server 2008 box. I have a .NET script running from Visual Studio 2010 that does the following: Reaches into the ...


Edward A. Schneider
Information Technology Specialist/Middleware Administration | Application Data Services Unit

Minnesota IT Services | Partnering with Minnesota Department of Health
625 North Robert St.
P.O. Box 64975
St. Paul, MN 55164-0975
O: 651-201-4047
Information Technology for Minnesota Government | mn.gov/mnit


From: phi...@googlegroups.com <phi...@googlegroups.com> on behalf of Bruce Riddle <brucer...@gmail.com>
Sent: Monday, March 12, 2018 2:13:18 PM
To: PHINMS User Community
Subject: PHINMS 3.0 Error
 
--

---
You received this message because you are subscribed to the Google Groups "PHINMS User Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phinms+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Schneider, Edward (MNIT)

unread,
Mar 12, 2018, 5:12:49 PM3/12/18
to PHINMS User Community

One other thing.  If your SQL Server and your PHINMS aren't behind the same firewall, have you checked your firewall access rules for the traffic between them, both inbound and outbound?  (Also, if your SQL Server host--or for that matter, your PHINMS host--is running the Windows internal firewall, earlier experience with PHINMS 2.9 on Windows suggests it's quite difficult to get the internal firewall rules adjusted to permit necessary inbound and outbound traffic to PHINMS.  Probably better the internal Windows firewall(s) should be turned off entirely, and rely on a properly-configured external firewall or firewalls.)


Edward A. Schneider
Information Technology Specialist/Middleware Administration | Application Data Services Unit

Minnesota IT Services | Partnering with Minnesota Department of Health
625 North Robert St.
P.O. Box 64975
St. Paul, MN 55164-0975
O: 651-201-4047
Information Technology for Minnesota Government | mn.gov/mnit


From: phi...@googlegroups.com <phi...@googlegroups.com> on behalf of Bruce Riddle <brucer...@gmail.com>
Sent: Monday, March 12, 2018 2:13:18 PM
To: PHINMS User Community
Subject: PHINMS 3.0 Error
 

Bruce Riddle

unread,
Mar 13, 2018, 8:38:15 AM3/13/18
to phi...@googlegroups.com
Mr. Schneider,
   All good suggestions but so far no hits.  What is so weird is that 2.8.03 worked just fine.  
So, far no one has been able to identify what is different in 3.0 from 2.8.03 that is not
making it work.  
  Thank you.

Bruce 

To unsubscribe from this group and stop receiving emails from it, send an email to phinms+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "PHINMS User Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phinms+unsubscribe@googlegroups.com.

Bruce Riddle

unread,
Mar 14, 2018, 2:25:14 PM3/14/18
to PHINMS User Community
Fellow IT travelers, this just in from PHIN MS Land:

tested PHINMS 3.0 connection to SQL server database on my machine and recreated the issue that you are having: no connection pool. My team has recognized this as a bug and we are working on a fix for it. Thank you for your patience.
 
It has taken 47 days to get to this point.  

Thanks for all the helpful ideas and support. 

Bruce 

Preacher Man

unread,
Mar 15, 2018, 10:34:36 AM3/15/18
to phi...@googlegroups.com

Well said Edward!  If I may (ahem) editorialize a bit myself...


For those using "enterprise" backends for PHINMS (e.g. Oracle, MSSQL, etc) good communications with your dB admin is a must.  There are just too many variables that can mess up PHINMS dB connections. Besides firewall issues and allowed open connections, there may also be re-connection restrictions, timeouts, and of course permissions that dB admins may "tweak" for security reasons and cause issue.


One PHINMS client I managed used a backend that was on a lossy/slow network. PHINMS  sender requests could wait up to 10 minutes for a receiver reply. Re-connections would fail and after 5 tries the backend would lock out the account. Then one had to put in an issue with the dB admin and wait (sometimes days) for the account to be re-enabled.  The connection policies for the dB were tailored for interactive users.


As for Tomcat, I always relied on the APR implementations.  JSSE has a history of bugs and missing features.  Amazingly, I could code an OpenSSL connection in 'C' with fewer lines of code than an equivalent JSSE connection in Java.  Things may be better now as I have been out of that game for a few years.  Even so, updates to JSSE ciphers are not going to be much help until the PHINMS xml (payload) encryption classes are upgraded to use them, and I wouldn't hold my breath on that (if you peak inside an xml encrypted payload, you will probably see it is still using RSA-1 over Triple DES for certificate based encryption).


In any event, if you are using a proxy with PHINMS on a private network, the (big bad internet) SSL negotiations will be handled by the proxy and not a real issue - at least for a receiver. A "naked" PHINMS sender could use help there though, which is where 3.0 should be a real improvement. Nice to know CDC hasn't completely given up on PHINMS yet.


Ya'll keep the faith.


Tom




From: phi...@googlegroups.com <phi...@googlegroups.com> on behalf of Schneider, Edward (MNIT) <edward.s...@state.mn.us>
Sent: Monday, March 12, 2018 3:24 PM
To: PHINMS User Community
Subject: Re: PHINMS 3.0 Error
 
Reply all
Reply to author
Forward
0 new messages