Picapport Linux vs Windows

[Slarti Bartfass]

Hallo zusammen

Ich kämpfe ja seit mehr als einem Jahr damit, dass Picapport auf meine X86 Ubuntu 16.04 Linux es nicht schafft, das PAI Plugin über all 300k Fotos zu prozessieren. Das System crasht IMMER auf irgendeine Art und Weise. z.B: gerade jetzt (Details unten).

Die These ist ja, dass das 6GB RAM System zu klein wäre und 4 GB Ram fpr JAVA auch nicht reichen. Neulich bin ich auf die Idee gekommen, Picapport temporär auf meinem Windows Laptop mit 16 GB und 8 Core über eine Samba-Freigabe alle 300k Fotos mit PAI Tags versehen zu lassen. Die Standard-Installation (Installer, .Exe) stoppte oft einfach und ich bekam keine Debug-Logs, mit dem Jar-File ging dann alles, ich habe einfach die nahezu komplett identische Konfig von der Linux-Installation übernommen. Und siehe da, auf Windows über Samba lief das ganze viel stabiler und hat es geschafft, nahezu alle Fotos mit PAI zu taggen, ohne die Datenbank zu crashen etc.

Linux ist noch v10.3, Window 10.4.

VER  @ 07:36:28.381 PicApport Version 10.3 from 15.12.2021 loaded.
VER  @ 07:36:28.382 de.contecon.picapport.PicApport.Version=V10.3.1 15.12.2021
VER  @ 07:36:28.382 java.runtime.version: 1.8.0_261-b12
VER  @ 07:36:28.382 java.version: 1.8.0_261
VER  @ 07:36:28.383 java.vm.version: 25.261-b12
VER  @ 07:36:28.383 java.vm.vendor: Oracle Corporation
VER  @ 07:36:28.383 java.vm.name: Java HotSpot(TM) 64-Bit Server VM
VER  @ 07:36:28.383 java.specification.version: 1.8
VER  @ 07:36:28.384 java.vm.specification.version: 1.8
VER  @ 07:36:28.384 java.class.path: /opt/picapport/picapport-headless.jar
VER  @ 07:36:28.384 os.name: Linux
VER  @ 07:36:28.385 os.version: 3.13.0-147-generic
VER  @ 07:36:28.385 os.arch: amd64

...

EXCEP@ Exception at: 2022-08-23 07:31:26
EXCEP@          Msg:
EXCEP@ No more resources available in pool (max=100). Requested resource: null
EXCEP@ ------------------------------------------------------------
EXCEP@ com.orientechnologies.common.concur.lock.OLockException: No more resources available in pool (max=100). Requested resource: null
EXCEP@     at com.orientechnologies.common.concur.resource.OResourcePool.getResource(OResourcePool.java:74)
EXCEP@     at com.orientechnologies.orient.core.db.ODatabasePoolImpl.acquire(ODatabasePoolImpl.java:69)
EXCEP@     at com.orientechnologies.orient.core.db.ODatabasePool.acquire(ODatabasePool.java:132)
EXCEP@     at de.contecon.picapport.db.PicApportDBService.getPooledDbInstance(Unknown Source)
EXCEP@     at de.contecon.picapport.db.PicApportDBService.getDbWrapper(Unknown Source)
EXCEP@     at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.consume(Unknown Source)
EXCEP@     at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.run(Unknown Source)
EXCEP@     at java.lang.Thread.run(Thread.java:748)

Danach ist meistens die OrientDB kaputt:

EXCEP@ ============================================================
EXCEP@ Exception at: 2022-08-23 07:37:09
EXCEP@          Msg:
EXCEP@ Following files and pages are detected to be broken ['photo.cpm' :0;], storage is switched to 'read only' mode. Any modification operations are prohibi
ted. To restore database and make it fully operational you may export and import database to and from JSON.
        DB name="db_3_0_39"
EXCEP@ ------------------------------------------------------------
EXCEP@ com.orientechnologies.orient.core.exception.OPageIsBrokenException: Following files and pages are detected to be broken ['photo.cpm' :0;], storage is s
witched to 'read only' mode. Any modification operations are prohibited. To restore database and make it fully operational you may export and import database
to and from JSON.
EXCEP@     DB name="db_3_0_39"
EXCEP@     at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkReadOnlyConditions(OAbstractPaginatedStorage.java:6307)
EXCEP@     at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.checkLowDiskSpaceRequestsAndReadOnlyConditions(OAbstractPaginated
Storage.java:6268)
EXCEP@     at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2541)
EXCEP@     at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:2500)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.internalCommit(ODatabaseDocumentAbstract.java:2773)
EXCEP@     at com.orientechnologies.orient.core.storage.impl.local.OMicroTransaction.doCommit(OMicroTransaction.java:363)
EXCEP@     at com.orientechnologies.orient.core.storage.impl.local.OMicroTransaction.commit(OMicroTransaction.java:166)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.endMicroTransaction(ODatabaseDocumentEmbedded.java:919)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentEmbedded.executeSaveRecord(ODatabaseDocumentEmbedded.java:909)
EXCEP@     at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveNew(OTransactionNoTx.java:244)
EXCEP@     at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:174)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.saveInternal(ODatabaseDocumentAbstract.java:2148)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:2110)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:2006)
EXCEP@     at com.orientechnologies.orient.core.db.document.ODatabaseDocumentAbstract.save(ODatabaseDocumentAbstract.java:136)
EXCEP@     at de.contecon.picapport.db.Photo.saveToDocument(Unknown Source)
EXCEP@     at de.contecon.picapport.db.Photo.insert(Unknown Source)
EXCEP@     at de.contecon.picapport.db.DbWrapper.insertPhoto(Unknown Source)
EXCEP@     at de.contecon.picapport.db.DbWrapper.updatePhoto(Unknown Source)
EXCEP@     at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.handlePhotoUpdate(Unknown Source)
EXCEP@     at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.consume(Unknown Source)
EXCEP@     at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.run(Unknown Source)
EXCEP@     at java.lang.Thread.run(Thread.java:748)

[Christopher Stack]

I'm running on Centos and after a recent power outage I started running into the same issue. It's just a ton of the following:

ERROR@ 01:19:14.360 Exception Error PhotoCrawler.DirectoryConsumer.handleDirectoryUpdate:/srv/photos/path: com.orientechnologies.orient.core.exception.OPageIsBrokenException: Following files and pages are detected to be broken ['idxfuzzySearch.mbt' :1199;], storage is switched to 'read only' mode. Any modification operations are prohibited. To restore database and make it fully operational you may export and import database to and from JSON.   DB name="db_3_0_39"
ERROR@ 01:19:14.360 Exception Error PhotoCrawler.DirectoryConsumer.run: com.orientechnologies.orient.core.exception.OStorageExistsException: Atomic operation already started   DB name="db_3_0_39"
ERROR@ 01:20:14.387 Exception Error PhotoCrawler.DirectoryConsumer.run: com.orientechnologies.common.concur.lock.OLockException: No more resources available in pool (max=100). Requested resource: null

Is there anything I can do to recover? I can post separately if you all think this is not the same issue.

[PicApport]

...just stop the server, remove the .picapport/db directory and restart the server

[Christopher Stack]

Thanks, it appears to be working for me!