BagIt Bags Zip file
Sherry Lake
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
Thanks!
Sherry Lake
James Myers
Yes, the Bag should be something you can unzip and inspect. If you haven’t set up the ArchiverClassName and related settings, it’s possible that the problem is misconfiguration.
If not, a problem with the Bag might indicate that some metadata and/or file problem exists – hopefully documented in the server log.
W.r.t. configuration, there are currently 3 options for where Bags should be sent when created
- to DSpace as the front end of Duracloud/Chronopolis (and DPN before it folded)
- to the Google cloud
- to a local file directory (which, as planned by Odum, could be synced with iRoDS for archival purposes).
The BagIt Export section (https://guides.dataverse.org/en/5.4/installation/config.html?highlight=archiverclassname#id104) has examples of the setup for all three.
All three share the same code for generating the Bag and it is streamed to the final destination wherever that may be. Network interruptions during the Bag transfer could result in an incomplete Bag. Since the Bag code is retrieving an zipping all of the files in the dataset as well, if any of those are missing/there are network access problems reading them, the Bag could also be incomplete. There is some functionality to retry connections in the code so truly intermittent problems shouldn’t be an issue but if a file just isn’t accessible, Bag creation would probably fail.
-- Jim
--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
dataverse-commu...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/dataverse-community/9e99b9ec-f30b-46c1-8efc-aff8320fa7f5n%40googlegroups.com.
Sherry Lake
James Myers
Ah – sorry. The short version is a typo. The long edu.harvard.iq.dataverse.engine.command.impl.LocalSubmitToArchiveCommand is the full name and package hierarchy for the class that’s being used. (All of the Dataverse code is in the edu.harvard.iq.dataverse package or some sub-package from there.).
If you can make an issue about the typo, that would be helpful (PR as well if you’re up for that).
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/e7b0fea0-b892-4168-969e-e5cce4e76e30n%40googlegroups.com.
Sherry Lake
":ArchiverClassName":"edu.harvard.iq.dataverse.engine.command.impl.LocalSubmitToArchiveCommand"
":BagItLocalPath":"/home/xw5d/BagItStorage/"
Attempt# 1 : Unable to retrieve file: https://dvdev.lib.virginia.edu/api/access/datafile/236171
javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:642)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:461)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:361)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:178)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at edu.harvard.iq.dataverse.util.bagit.BagGenerator$3.get(BagGenerator.java:989)
at org.apache.commons.compress.archivers.zip.ZipArchiveEntryRequest.getPayloadStream(ZipArchiveEntryRequest.java:62)
at org.apache.commons.compress.archivers.zip.ScatterZipOutputStream.addArchiveEntry(ScatterZipOutputStream.java:99)
at org.apache.commons.compress.archivers.zip.ParallelScatterZipCreator$3.call(ParallelScatterZipCreator.java:210)
at org.apache.commons.compress.archivers.zip.ParallelScatterZipCreator$3.call(ParallelScatterZipCreator.java:206)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
at java.base/sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:350)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:248)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:321)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:110)
at org.apache.http.ssl.SSLContextBuilder$TrustManagerDelegate.checkServerTrusted(SSLContextBuilder.java:416)
at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1509)
at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:626)
... 31 more
Caused by: java.security.cert.CertPathValidatorException: validity check failed
at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:233)
at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:141)
at java.base/sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:80)
at java.base/java.security.cert.CertPathValidator.validate(CertPathValidator.java:309)
at java.base/sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:345)
... 39 more
Caused by: java.security.cert.CertificateExpiredException: NotAfter: Sat May 30 06:48:38 EDT 2020
at java.base/sun.security.x509.CertificateValidity.valid(CertificateValidity.java:274)
at java.base/sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:687)
at java.base/sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190)
at java.base/sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144)
at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
... 44 more
]]
[2021-08-04T16:18:37.409-0400] [Payara 5.2021.4] [SEVERE] [] [] [tid: _ThreadID=28674 _ThreadName=pool-37-thread-1] [timeMillis: 1628108317409] [levelValue: 1000] [[
javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/MN2PR07MB73431F69F32D5ACC1A730978BFF19%40MN2PR07MB7343.namprd07.prod.outlook.com.
James Myers
This is pretty clearly something about the certificate – hard to know without being able to see it. In general, the Bag generator code is being pretty lenient about checking the certificate (since they are all for files that should be on your server anyway), but there must be something about it that Java doesn’t like. If this is a locally signed certificate just used for dev, it may be that the certificate isn’t in the Java keystore. It’s also possible that there’s something like the certificate not listing the dvdev.lib.viriginia.edu name as the server it is for, etc.
In any case, I think this is probably something that you can resolve locally, i.e. by adding your cert to the local java keystore on your machine (lots of web resources on how to do that). If not, or if that’s not clear, let me know and we might be able to do a quick screen share so I can see the cert/help debug.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-Xs3bcfss-K2-NV%3De6ze5a86fq6c8xjrZ_d2tNR5q%3D-DA%40mail.gmail.com.