recovery process

41 views
Skip to first unread message

Jialing Pei

unread,
Jul 9, 2020, 2:23:18 PM7/9/20
to wiredtiger-users
Hi,

I'm playing around with wiredtiger source code using gdb and want to get a sense of how transaction recovery works. Several questions when running gdb:
1. It seems that the function __wt_log_scan in log.c is taking care of scanning logs and apply a call back function to the log record. One thing I'm quite confused is the following code in __wt_log_scan:

         if (reclen == 0) {
2248              WT_ERR(__log_has_hole(session, log_fh, log_size, rd_lsn.l.offset, &bad_offset, &eol));
2249              if (bad_offset != 0) {
2250                  need_salvage = true;
2251                  WT_ERR(__log_salvage_message(session, log_fh->name, "", bad_offset));
2252              }
2253              if (eol)
2254                  /* Found a hole. This LSN is the end. */
2255                  break;
2256              /* Last record in log.  Look for more. */
2257              goto advance;
2258          }
When encountering zeros, why wt is trying to find a hole? What's the point of walking through the entire file?  Based on my observation, logs(slots) are appended serially and a slot always waits for its previous slots to finish. How does a hole come out then? Why a hole simply indicates the very end? Why a no-hole indicates looking for more?

2.Upon recovery, if there's no hole it will go to advance as line 2257 suggested and it will call the __log_has_hole function again(walk through the file again) in advance part, what's the point of this duplicate calls to this function?

Appreciated!

Best,
Jialing

sue.l...@mongodb.com

unread,
Jul 14, 2020, 12:31:39 PM7/14/20
to wiredtiger-users
On Thursday, July 9, 2020 at 2:23:18 PM UTC-4 Jialing Pei wrote:
Hi,

I'm playing around with wiredtiger source code using gdb and want to get a sense of how transaction recovery works. Several questions when running gdb:
1. It seems that the function __wt_log_scan in log.c is taking care of scanning logs and apply a call back function to the log record. One thing I'm quite confused is the following code in __wt_log_scan:

         if (reclen == 0) {
2248              WT_ERR(__log_has_hole(session, log_fh, log_size, rd_lsn.l.offset, &bad_offset, &eol));
2249              if (bad_offset != 0) {
2250                  need_salvage = true;
2251                  WT_ERR(__log_salvage_message(session, log_fh->name, "", bad_offset));
2252              }
2253              if (eol)
2254                  /* Found a hole. This LSN is the end. */
2255                  break;
2256              /* Last record in log.  Look for more. */
2257              goto advance;
2258          }
When encountering zeros, why wt is trying to find a hole? What's the point of walking through the entire file?  Based on my observation, logs(slots) are appended serially and a slot always waits for its previous slots to finish. How does a hole come out then? Why a hole simply indicates the very end? Why a no-hole indicates looking for more?

This is not correct. In the usual codepath, writing a slot does not wait for earlier slots to be written. In __log_fs_write, the general code path just calls __wt_write. Therefore, slots can be written out of order and a hole can be the result.

The existence of a hole means we cannot recover anything after the hole because some data is missing. That is the end of the log on recovery. Detecting the end of a file and writing the next record into the next file means we have more to process. Because WiredTiger zeroes out the log files, the appearance of a hole looks very similar to the last record in the log file, then some zeroed space and the next record in the next log file. We have to be able to distinguish those cases. 

2.Upon recovery, if there's no hole it will go to advance as line 2257 suggested and it will call the __log_has_hole function again(walk through the file again) in advance part, what's the point of this duplicate calls to this function?

We can get to the code at the "advance" label without having called __log_has_hole and we need that information in some cases. 
Reply all
Reply to author
Forward
0 new messages