Does wiredtiger will be stuck? If I do the full table scan over a huge table which size is beyond page cache size.

22 views
Skip to first unread message

baoshang young

unread,
Mar 16, 2021, 11:23:14 PM3/16/21
to wiredtiger-users
I have read the page eviction code and I found that the page can't be evicted unless no sessions access it using hazard pointers. When I do the full table scan over a huge table, the hazard pointers of all page have visited can't be released. It will make page cache full and wiredtiger will be stuck. Am I right?

Haribabu Kommi

unread,
Mar 17, 2021, 12:47:53 AM3/17/21
to wiredtig...@googlegroups.com
Hi,

WT scan releases the hazard pointer on a page whenever it moves to the next page in the scan, so performing a full table scan doesn't lead to cache full.

Regards,
Hari

On Wed, Mar 17, 2021 at 2:23 PM baoshang young <zaor...@gmail.com> wrote:
I have read the page eviction code and I found that the page can't be evicted unless no sessions access it using hazard pointers. When I do the full table scan over a huge table, the hazard pointers of all page have visited can't be released. It will make page cache full and wiredtiger will be stuck. Am I right?

--
You received this message because you are subscribed to the Google Groups "wiredtiger-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wiredtiger-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wiredtiger-users/815876a2-6098-48ad-9123-e8cde26a3d02n%40googlegroups.com.

baoshang young

unread,
Mar 17, 2021, 1:15:06 AM3/17/21
to wiredtig...@googlegroups.com
Hi, Hari. Thanks a lot for your reply. If WT releases the prev page hazard pointers, how does WT implement the snapshot isolation?  The released pages could be modified by other threads and evicted to disk. If these pages were re-read in a same transaction, the data will be different from the first read.

'Haribabu Kommi' via wiredtiger-users <wiredtig...@googlegroups.com>于2021年3月17日 周三下午12:47写道:

Haribabu Kommi

unread,
Mar 17, 2021, 2:05:25 AM3/17/21
to wiredtig...@googlegroups.com
Hi,

WT stores multiple versions of data to support different readers in the system. According to the reader snapshot, it returns only the data that is visible.
As you said when the page gets evicted to disk, the latest version of data is written to the page and all the historical versions are saved in a different table called the history store.
Whenever the old reader requests to re-read the same data, WT internally fetches the data from the history store and returned.  Applications don't see different data compared to the first read.

Regards,
Hari

baoshang young

unread,
Mar 17, 2021, 2:24:20 AM3/17/21
to wiredtiger-users
Hi, Hari. 
Thank you very much! 
Regards,
ZaorangYang
Reply all
Reply to author
Forward
0 new messages