Sonatype Nexus DB switched to read-only

[dsen...@gmail.com]

Hi,

Or nexus server had been crashed, so we have to restart the application. 

However, while bringing the application up, the following error appeared: 

2023-08-29 15:06:54,041-0400 ERROR [qtp2028174808-490 <command>sql.select from asset where (attributes.nuget.is_prerelease=false) and (bucket=#18:4 or bucket=#20:4) GROUP BY name ORDER BY attributes.nuget.downl
oad_count DESC, component.ci_name ASC, component.version ASC LIMIT 40</command>]  *UNKNOWN org.sonatype.nexus.repository.storage.LastDownloadedHandler - Failed to update last downloaded time for request /@cryst
al/icons/-/icons-2.0.3.tgz
com.orientechnologies.orient.core.exception.OPageIsBrokenException: Following files and pages are detected to be broken ['component_bucket_name_version_idx.sbt' :283;'browse_node_asset_id_idx.sbt' :112;'asset_n
ame_ci_idx.sbt' :1552;'component_bucket_group_name_version_idx.sbt' :282;'browse_node_3.pcl' :306;'browse_node_component_id_idx.sbt' :74;'asset_bucket_rid_idx.sbt' :279;'browse_node_repository_name_parent_path_
name_idx.sbt' :1015;'asset_component_idx.sbt' :179;], storage is switched to 'read only' mode. Any modification operations are prohibited. To restore database and make it fully operational you may export and import database to and from JSON.

Is there any steps I can follow to recover DB as read-write?

Thank you in advance.

[dsen...@gmail.com]

Also, tried to access and checked the DB - 

2023-08-29 15:04:35:014 SEVER {db=component} Magic number verification failed for page '306' of 'browse_node_3.pcl'. [OWOWCache]
2023-08-29 15:04:35:015 SEVER {db=component} Magic number verification failed for page '307' of 'browse_node_3.pcl'. [OWOWCache]
2023-08-29 15:04:35:025 SEVER {db=component} Magic number verification failed for page '282' of 'component_bucket_group_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:026 SEVER {db=component} Magic number verification failed for page '283' of 'component_bucket_group_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:026 SEVER {db=component} Magic number verification failed for page '284' of 'component_bucket_group_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:028 SEVER {db=component} Magic number verification failed for page '285' of 'component_bucket_group_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:028 SEVER {db=component} Magic number verification failed for page '286' of 'component_bucket_group_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:095 SEVER {db=component} Magic number verification failed for page '179' of 'asset_component_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:096 SEVER {db=component} Magic number verification failed for page '180' of 'asset_component_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:096 SEVER {db=component} Magic number verification failed for page '181' of 'asset_component_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:186 SEVER {db=component} Magic number verification failed for page '1552' of 'asset_name_ci_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:187 SEVER {db=component} Magic number verification failed for page '1553' of 'asset_name_ci_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:187 SEVER {db=component} Magic number verification failed for page '1554' of 'asset_name_ci_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:187 SEVER {db=component} Magic number verification failed for page '1555' of 'asset_name_ci_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:196 SEVER {db=component} Magic number verification failed for page '283' of 'component_bucket_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:196 SEVER {db=component} Magic number verification failed for page '284' of 'component_bucket_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:197 SEVER {db=component} Magic number verification failed for page '285' of 'component_bucket_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:197 SEVER {db=component} Magic number verification failed for page '286' of 'component_bucket_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:197 SEVER {db=component} Magic number verification failed for page '287' of 'component_bucket_name_version_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:234 SEVER {db=component} Magic number verification failed for page '1015' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:234 SEVER {db=component} Magic number verification failed for page '1016' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:235 SEVER {db=component} Magic number verification failed for page '1017' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:235 SEVER {db=component} Magic number verification failed for page '1018' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:235 SEVER {db=component} Magic number verification failed for page '1019' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:236 SEVER {db=component} Magic number verification failed for page '1020' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:236 SEVER {db=component} Magic number verification failed for page '1021' of 'browse_node_repository_name_parent_path_name_idx.sbt'. [OWOWCach                                                e]
2023-08-29 15:04:35:238 SEVER {db=component} Magic number verification failed for page '74' of 'browse_node_component_id_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:238 SEVER {db=component} Magic number verification failed for page '75' of 'browse_node_component_id_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:243 SEVER {db=component} Magic number verification failed for page '112' of 'browse_node_asset_id_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:243 SEVER {db=component} Magic number verification failed for page '113' of 'browse_node_asset_id_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:255 SEVER {db=component} Magic number verification failed for page '279' of 'asset_bucket_rid_idx.sbt'. [OWOWCache]
2023-08-29 15:04:35:255 SEVER {db=component} Magic number verification failed for page '280' of 'asset_bucket_rid_idx.sbt'. [OWOWCache]OK
orientdb {db=component}> check database

Checking storage.Check of storage is started...$ANSI{green {db=component}} Error during data read for file 'component_bucket_name_version_idx.sbt' 1-th attempt
java.io.EOFException: End of file is reached
        at com.orientechnologies.common.io.OIOUtils.readByteBuffer(OIOUtils.java:364)
        at com.orientechnologies.orient.core.storage.fs.OFileClassic.read(OFileClassic.java:1

[dsen...@gmail.com]

While attempting to rebuild DB index - 

Rebuilding index(es)...$ANSI{green {db=component}} Error during index 'component_bucket_group_name_version_idx' delete
com.orientechnologies.orient.core.exception.OPageIsBrokenException: Following files and pages are detected to be broken ['component_bucket_name_version_idx.sbt' :283;'browse_node_asset_id_idx.sbt' :112;'asset_name_ci_idx.sbt' :1553;'component_bucket_group_name_version_idx.sbt' :282;'browse_node_3.pcl' :306;'browse_node_component_id_idx.sbt' :74;'asset_bucket_rid_idx.sbt' :279;'browse_node_repository_name_parent_path_name_idx.sbt' :1015;'asset_component_idx.sbt' :179;], storage is switched to 'read only' mode. Any modification operations are prohibited. To restore database and make it fully operational you may export and import database to and from JSON.
        DB name="component"

[dsen...@gmail.com]

Attempt to export the DB, the following error is thrown - 

- Cluster 'asset_2' (id=27)....Exception `5348D83C` in storage `plocal:/nexus/data3/data3/db/component`: 2.2.37 (build a7541e7ceeabf592dd9a7b2928b6c023cbc73193, branch 2.2.x)
java.lang.NullPointerException
        at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readFullEntry(OPaginatedCluster.java:1899)
        at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.readRecordBuffer(OPaginatedCluster.java:779)