Checksum error while restoring the data files

[Rahul Sharma]

Hi All,

I had taken a backup for entire mongo directory and now trying to retore the mongo (same version;3.4.10) with the backup copy but getting checksum error.

Can any one please help on how to handle this or is there a way to restore the backup which I have. 

[1517511192:590697][87337:0x7ffb74c1bd00], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 45056: block header checksum of 2440285694 doesn't match expected checksum of 3270845372

[Rahul Sharma]

Log details--

2018-02-01T19:22:04.412+0000 W -        [initandlisten] Detected unclean shutdown - /var/lib/mongodb/mongod.lock is not empty.

2018-02-01T19:22:04.436+0000 I -        [initandlisten] Detected data files in /var/lib/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.

2018-02-01T19:22:04.437+0000 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.

2018-02-01T19:22:04.437+0000 I STORAGE  [initandlisten]

2018-02-01T19:22:04.437+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine

2018-02-01T19:22:04.437+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem

2018-02-01T19:22:04.437+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1202M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),

2018-02-01T19:22:04.447+0000 E STORAGE  [initandlisten] WiredTiger error (0) [1517512924:447416][93120:0x7f94b1650d00], file:WiredTiger.wt, connection: read checksum error for 4096B block at offset 45056: block header checksum of 2440285694 doesn't match expected checksum of 3270845372

2018-02-01T19:22:04.447+0000 E STORAGE  [initandlisten] WiredTiger error (0) [1517512924:447496][93120:0x7f94b1650d00], file:WiredTiger.wt, connection: WiredTiger.wt: encountered an illegal file format or internal value

2018-02-01T19:22:04.447+0000 E STORAGE  [initandlisten] WiredTiger error (-31804) [1517512924:447535][93120:0x7f94b1650d00], file:WiredTiger.wt, connection: the process must exit and restart: WT_PANIC: WiredTiger library panic

2018-02-01T19:22:04.447+0000 I -        [initandlisten] Fatal Assertion 28558 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 361

2018-02-01T19:22:04.447+0000 I -        [initandlisten]

***aborting after fassert() failure

2018-02-01T19:22:04.466+0000 F -        [initandlisten] Got signal: 6 (Aborted).

 0xde9e36c651 0xde9e36b869 0xde9e36bd4d 0x7f94b0265390 0x7f94afebf428 0x7f94afec102a 0xde9d6167b3 0xde9e07e016 0xde9d620a4c 0xde9d620c71 0xde9d620ed3 0xde9ec7208f 0xde9ec725db 0xde9ec6ebcd 0xde9ec737a7 0xde9ec8d3d3 0xde9ecc6a0b 0xde9ed5181b 0xde9ed51d5d 0xde9ed5201c 0xde9ecd5131 0xde9ed47b48 0xde9ed11b3e 0xde9ed11c1b 0xde9ecc2c4d 0xde9e062d5f 0xde9e05b5cc 0xde9df4e8c0 0xde9d60290e 0xde9d622286 0x7f94afeaa830 0xde9d6814e9

----- BEGIN BACKTRACE -----

{"backtrace":[{"b":"DE9CDFB000","o":"1571651","s":"_ZN5mongo15printStackTraceERSo"},{"b":"DE9CDFB000","o":"1570869"},{"b":"DE9CDFB000","o":"1570D4D"},{"b":"7F94B0254000","o":"11390"},{"b":"7F94AFE8A000","o":"35428","s":"gsignal"},{"b":"7F94AFE8A000","o":"3702A","s":"abort"},{"b":"DE9CDFB000","o":"81B7B3","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},{"b":"DE9CDFB000","o":"1283016"},{"b":"DE9CDFB000","o":"825A4C","s":"__wt_eventv"},{"b":"DE9CDFB000","o":"825C71","s":"__wt_err"},{"b":"DE9CDFB000","o":"825ED3","s":"__wt_panic"},{"b":"DE9CDFB000","o":"1E7708F","s":"__wt_block_extlist_read"},{"b":"DE9CDFB000","o":"1E775DB","s":"__wt_block_extlist_read_avail"},{"b":"DE9CDFB000","o":"1E73BCD","s":"__wt_block_checkpoint_load"},{"b":"DE9CDFB000","o":"1E787A7"},{"b":"DE9CDFB000","o":"1E923D3","s":"__wt_btree_open"},{"b":"DE9CDFB000","o":"1ECBA0B","s":"__wt_conn_btree_open"},{"b":"DE9CDFB000","o":"1F5681B","s":"__wt_session_get_btree"},{"b":"DE9CDFB000","o":"1F56D5D","s":"__wt_session_get_btree"},{"b":"DE9CDFB000","o":"1F5701C","s":"__wt_session_get_btree_ckpt"},{"b":"DE9CDFB000","o":"1EDA131","s":"__wt_curfile_open"},{"b":"DE9CDFB000","o":"1F4CB48"},{"b":"DE9CDFB000","o":"1F16B3E","s":"__wt_metadata_cursor_open"},{"b":"DE9CDFB000","o":"1F16C1B","s":"__wt_metadata_cursor"},{"b":"DE9CDFB000","o":"1EC7C4D","s":"wiredtiger_open"},{"b":"DE9CDFB000","o":"1267D5F","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb"},{"b":"DE9CDFB000","o":"12605CC"},{"b":"DE9CDFB000","o":"11538C0","s":"_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv"},{"b":"DE9CDFB000","o":"80790E"},{"b":"DE9CDFB000","o":"827286","s":"main"},{"b":"7F94AFE8A000","o":"20830","s":"__libc_start_main"},{"b":"DE9CDFB000","o":"8864E9","s":"_start"}],"processInfo":{ "mongodbVersion" : "3.4.10", "gitVersion" : "078f28920cb24de0dd479b5ea6c66c644f6326e9", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.11.0-1016-azure", "version" : "#16-Ubuntu SMP Tue Dec 5 22:52:08 UTC 2017", "machine" : "x86_64" }, "somap" : [ { "b" : "DE9CDFB000", "elfType" : 3, "buildId" : "953D98B21F3F66FCA98875FCB02884203F3CEDC6" }, { "b" : "7FFC3CED1000", "elfType" : 3, "buildId" : "37B26AC94BA016FF834387DAB6E0D23F5B0176C6" }, { "b" : "7F94B11E0000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "DCF10134B91ED2139E3E8C72564668F5CDBA8522" }, { "b" : "7F94B0D9C000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "1649272BE0CA9FA22F082DC86372B6C9959779B0" }, { "b" : "7F94B0B94000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "89C34D7A182387D76D5CDA1F7718F5D58824DFB3" }, { "b" : "7F94B0990000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "8CC8D0D119B142D839800BFF71FB71E73AEA7BD4" }, { "b" : "7F94B0687000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "DFB85DE42DAFFD09640C8FE377D572DE3E168920" }, { "b" : "7F94B0471000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F94B0254000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "CE17E023542265FC11D9BC8F534BB4F070493D30" }, { "b" : "7F94AFE8A000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B5381A457906D279073822A5CEB24C4BFEF94DDB" }, { "b" : "7F94B1449000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "5D7B6259552275A3C17BD4C3FD05F5A6BF40CAA5" } ] }}

 mongod(_ZN5mongo15printStackTraceERSo+0x41) [0xde9e36c651]

 mongod(+0x1570869) [0xde9e36b869]

 mongod(+0x1570D4D) [0xde9e36bd4d]

 libpthread.so.0(+0x11390) [0x7f94b0265390]

 libc.so.6(gsignal+0x38) [0x7f94afebf428]

 libc.so.6(abort+0x16A) [0x7f94afec102a]

 mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0xde9d6167b3]

 mongod(+0x1283016) [0xde9e07e016]

 mongod(__wt_eventv+0x3D7) [0xde9d620a4c]

 mongod(__wt_err+0x9D) [0xde9d620c71]

 mongod(__wt_panic+0x2E) [0xde9d620ed3]

 mongod(__wt_block_extlist_read+0x8F) [0xde9ec7208f]

 mongod(__wt_block_extlist_read_avail+0x2B) [0xde9ec725db]

 mongod(__wt_block_checkpoint_load+0x26D) [0xde9ec6ebcd]

 mongod(+0x1E787A7) [0xde9ec737a7]

 mongod(__wt_btree_open+0xB43) [0xde9ec8d3d3]

 mongod(__wt_conn_btree_open+0x19B) [0xde9ecc6a0b]

 mongod(__wt_session_get_btree+0xFB) [0xde9ed5181b]

 mongod(__wt_session_get_btree+0x63D) [0xde9ed51d5d]

 mongod(__wt_session_get_btree_ckpt+0x14C) [0xde9ed5201c]

 mongod(__wt_curfile_open+0x161) [0xde9ecd5131]

 mongod(+0x1F4CB48) [0xde9ed47b48]

 mongod(__wt_metadata_cursor_open+0x6E) [0xde9ed11b3e]

 mongod(__wt_metadata_cursor+0x4B) [0xde9ed11c1b]

 mongod(wiredtiger_open+0x164D) [0xde9ecc2c4d]

 mongod(_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mbbbb+0x70F) [0xde9e062d5f]

 mongod(+0x12605CC) [0xde9e05b5cc]

 mongod(_ZN5mongo20ServiceContextMongoD29initializeGlobalStorageEngineEv+0x6B0) [0xde9df4e8c0]

 mongod(+0x80790E) [0xde9d60290e]

 mongod(main+0x966) [0xde9d622286]

 libc.so.6(__libc_start_main+0xF0) [0x7f94afeaa830]

 mongod(_start+0x29) [0xde9d6814e9]

-----  END BACKTRACE  -----