"ddsnap create" doesn't suspend the original device
Stephane Chazelas
while trying to improve the performances on my setup, I tried to
create the ddsnap snapshot devices as read-only.
I used to mount them read-only already, but now, I also use
dmsetup create -r
Now, the problem is that ext3 won't mount the snapshots anymore.
ddsnap_create: Created snapshot device snapstore=253:2 origin=253:0 socket=/dev/ddsnap/control snapshot=1
incoming: Client thread started, pid=10056 for snapshot 1
worker: Worker thread started, pid=10057 for snapshot 1
worker: worker recovering for snapshot 1
control: Control thread started, pid=10058 for snapshot 1
requeue_queries:
worker: worker resuming for snapshot 1
incoming: identify succeeded. chunksize 12
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access unavailable, cannot proceed.
Looking at the logs for when I wasn't using -r, I can see things
like:
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: dm-21: orphan cleanup on readonly fs
ext3_orphan_cleanup: deleting unreferenced inode 224436
ext3_orphan_cleanup: deleting unreferenced inode 224428
ext3_orphan_cleanup: deleting unreferenced inode 224422
ext3_orphan_cleanup: deleting unreferenced inode 224414
So even though the FS was read-only, ext3 was writing to it.
And that also means that ddsnap had done a snapshot at a time
when the FS was not synced to disk.
That kind of problem doesn't occur when one uses LVM snapshots
instead of ddsnap snapshots.
This is because I beleive lvm does suspend (and request the
overlying FS to sync) the original device before taking the
snashot.
I guess I could do that using dmsetup with ddsnap, but that
means I would have to put dmsetup, a shell, ddsnap onto a tmpfs
to avoid deadlocks since I'm ddsnapping the rootfs.
Couldn't ddsnap be modified so that it does something similar to
what lvm does (suspend, snap, resume) under mlock?
Another question: is there any performance impact of allowing
writes to the ddsnap snapshots. In particular, would that cause
"ddsnap delete" to take longer times?
regards,
Stéphane
Dan Kegel
> ...
> So even though the FS was read-only, ext3 was writing to it.
>
> And that also means that ddsnap had done a snapshot at a time
> when the FS was not synced to disk.
>
> That kind of problem doesn't occur when one uses LVM snapshots
> instead of ddsnap snapshots.
>
> This is because I beleive lvm does suspend (and request the
> overlying FS to sync) the original device before taking the
> snashot.
>
> I guess I could do that using dmsetup with ddsnap, but that
> means I would have to put dmsetup, a shell, ddsnap onto a tmpfs
> to avoid deadlocks since I'm ddsnapping the rootfs.
>
> Couldn't ddsnap be modified so that it does something similar to
> what lvm does (suspend, snap, resume) under mlock?
Sounds good to me. I've felt for some time that we should freeze
the filesystem during the snapshot, but Daniel felt that it was overkill.
Looks like it's needed after all.
Would you like to file a bug report with the info you gave?
> Another question: is there any performance impact of allowing
> writes to the ddsnap snapshots. In particular, would that cause
> "ddsnap delete" to take longer times?
I'll leave that question to the experts :-)
Jiaying Zhang
We do FS synchronization in zumastor that is a bash program running
on top of ddsnap. Every time after a ddsnap snapshot is created but
before calling 'dmset create', we sync the origin device with
'blockdev --flushbufs /dev/mapper/$vol'.
That kind of problem doesn't occur when one uses LVM snapshots
instead of ddsnap snapshots.
This is because I beleive lvm does suspend (and request the
overlying FS to sync) the original device before taking the
snashot.
I guess I could do that using dmsetup with ddsnap, but that
means I would have to put dmsetup, a shell, ddsnap onto a tmpfs
to avoid deadlocks since I'm ddsnapping the rootfs.
Couldn't ddsnap be modified so that it does something similar to
what lvm does (suspend, snap, resume) under mlock?
finishes. There is an ioctl to do that.
Another question: is there any performance impact of allowing
writes to the ddsnap snapshots. In particular, would that cause
"ddsnap delete" to take longer times?
When we write to a snapshot, the old data needs to be copied to
another place. So there will be some overhead on the performance.
It may have some impact on the time that 'ddsnap delete' takes
because more snapshot chunks are used to store the old data.
But I guess the impact is small.
Jiaying
Stephane Chazelas
[...]
> We do FS synchronization in zumastor that is a bash program running
> on top of ddsnap. Every time after a ddsnap snapshot is created but
> before calling 'dmset create', we sync the origin device with
> 'blockdev --flushbufs /dev/mapper/$vol'.
OK.
I did call sync(1), but that didn't help. I'll try with blockdev
--flushbufs. But I think you need to suspend at the same time,
I think that's what LVM does.
[...]
> When we write to a snapshot, the old data needs to be copied to
> another place. So there will be some overhead on the performance.
> It may have some impact on the time that 'ddsnap delete' takes
> because more snapshot chunks are used to store the old data.
> But I guess the impact is small.
[...]
OK, so if I understand correctly, the few writes that occur when
EXT3 fixes the FS upon mounting shouldn't make a difference.
Thanks
Stéphane
Jiaying Zhang
On Mon, May 12, 2008 at 11:54:02AM -0700, Jiaying Zhang wrote:
[...]
> We do FS synchronization in zumastor that is a bash program running
> on top of ddsnap. Every time after a ddsnap snapshot is created but
> before calling 'dmset create', we sync the origin device with
> 'blockdev --flushbufs /dev/mapper/$vol'.
[...]
OK.
I did call sync(1), but that didn't help. I'll try with blockdev
--flushbufs. But I think you need to suspend at the same time,
I think that's what LVM does.
sync should have the same effect as blockdev --flushbufs. Could you post
the steps to reproduce the problem?
[...]
> When we write to a snapshot, the old data needs to be copied to
> another place. So there will be some overhead on the performance.
> It may have some impact on the time that 'ddsnap delete' takes
> because more snapshot chunks are used to store the old data.
> But I guess the impact is small.
[...]
OK, so if I understand correctly, the few writes that occur when
EXT3 fixes the FS upon mounting shouldn't make a difference.
That is what I meant, :).
Jiaying
Stephane Chazelas
> On 5/12/08, Stephane Chazelas <Stephane...@yahoo.fr> wrote:
> > On Mon, May 12, 2008 at 11:54:02AM -0700, Jiaying Zhang wrote:
> > [...]
> > > We do FS synchronization in zumastor that is a bash program running
> > > on top of ddsnap. Every time after a ddsnap snapshot is created but
> > > before calling 'dmset create', we sync the origin device with
> > > 'blockdev --flushbufs /dev/mapper/$vol'.
[...]
> > --flushbufs. But I think you need to suspend at the same time,
> > I think that's what LVM does.
>
> sync should have the same effect as blockdev --flushbufs. Could you post
> the steps to reproduce the problem?
Well, it's just
sync; ddsnap create "$socket" "$id"
on an ext3 FS really (iostat shows an average of 233 blocks
written per second on that FS). Attached is the script that
takes the snapshots.
ddsnap reads /etc/ld.so.cache, /etc/localtime... and itself and
the FS is mounted without noatime, so that could explain.
I had the same problem with the 21:00 snapshot that was doing
blockdev --flushbufs instead of sync. Is the sync synchronous? I
mean, is the FS synced when sync returns?
regards,
Stéphane
Jiaying Zhang
E.g., mounting and umounting FS will cause writes at the block level. So it is expected
to see ext updates even though the FS is mounted readonly.
I am not sure why the FS was not in the sync state when the snapshot was taken.
I wonder if that is because ddsnap runs as the rootfs. When sync returns, all of IOs
that are submitted before the sync are guaranteed to finish. But there is no guarantee
about the IOs that are submitted after the sync. Could you try
"sync; ddsnap create; blockdev --flushbufs"? I.e., adding the 'blockdev --flushbufs'
after the 'ddsnap create'.
Jiaying
Stephane Chazelas
> Mounting a FS readonly does not mean that writes won't happen at the block
> level.
Hi Jiaying,
yes I saw that. But that doesn't mean that one shouldn't be able
to mount a read-only block device.
> E.g., mounting and umounting FS will cause writes at the block level. So it
> is expected
> to see ext updates even though the FS is mounted readonly.
I'd rather say "*may* cause writes". For instance, I don't think
the mount-count is incremented if the FS is mounted read-only.
And one should be able to mount an ext3 FS on a CDROM or on a
read-only file via a loop device.
> I am not sure why the FS was not in the sync state when the snapshot was
> taken.
> I wonder if that is because ddsnap runs as the rootfs. When sync returns,
> all of IOs
> that are submitted before the sync are guaranteed to finish. But there is no
> guarantee
> about the IOs that are submitted after the sync. Could you try
> "sync; ddsnap create; blockdev --flushbufs"? I.e., adding the 'blockdev
> --flushbufs'
> after the 'ddsnap create'.
I did that. It didn't help.
I was also suggesting that, as ddsnap is run after the sync, as
as running ddsnap involves reading a number of files, that would
cause the FS to be modified after the sync, and those
modifications not commited to disk.
But then, I tried to remount / with noatime, and it didn't help.
Now, maybe it's just that we're asking EXT3 to mount a FS that
has not been previously unmounted, and that it thinks it must
take some recovery action in any case.
But LVM snapshots don't suffer from the problem even when I use
-p r (creates read-only block devices).
The LVM and DM code is very convoluted, to say the least, but,
looking at the code and at ltrace/strace, it does do something
like "suspend" the device, take the snapshot, "resume" the
device. And syncing the device is done as part of the suspend,
and the fact that the device is "suspended" guarantees that the FS
is not alterered while the "snapshot" is taken.
lvcreate seems to be calling dm_tree_suspend_children(). The
libdevmapper.h file has:
H> /*
H> * Suspend a device plus all dependencies.
H> * Ignores devices that don't have a uuid starting with uuid_prefix.
H> */
H> int dm_tree_suspend_children(struct dm_tree_node *dnode,
H> const char *uuid_prefix,
H> size_t uuid_prefix_len);
H>
H> /*
H> * Skip the filesystem sync when suspending.
H> * Does nothing with other functions.
H> * Use this when no snapshots are involved.
H> */
H> void dm_tree_skip_lockfs(struct dm_tree_node *dnode);
H>
H> /*
H> * Set the 'noflush' flag when suspending devices.
H> * If the kernel supports it, instead of erroring outstanding I/O that
H> * cannot be completed, the I/O is queued and resubmitted when the
H> * device is resumed. This affects multipath devices when all paths
H> * have failed and queue_if_no_path is set, and mirror devices when
H> * block_on_error is set and the mirror log has failed.
H> */
H> void dm_tree_use_no_flush_suspend(struct dm_tree_node *dnode);
According to strace, it seems to perform (at least) a DM_DEV_SUSPEND
As I said, it could be done with "dmsetup suspend", "dmsetup
resume", but that means I would have to setup a temporary
filesystem on tmpfs with dmsetup, ddsnap, busybox, their libs,
/dev and /proc, which is a bit overkill, so it would be nice if
ddsnap could do the suspend and resume under mlock.
Cheers,
Stéphane
Stephane Chazelas
[...]
> As I said, it could be done with "dmsetup suspend", "dmsetup
> resume", but that means I would have to setup a temporary
> filesystem on tmpfs with dmsetup, ddsnap, busybox, their libs,
> /dev and /proc, which is a bit overkill, so it would be nice if
> ddsnap could do the suspend and resume under mlock.
I just did that, and it worked. However, I got a:
May 13 14:00:14 ant kernel: WARNING: at kernel/mutex-debug.c:82 debug_mutex_unlock()
May 13 14:00:14 ant kernel: Pid: 27050, comm: dmsetup Not tainted 2.6.24.2 #2
May 13 14:00:14 ant kernel: [<c0131563>] debug_mutex_unlock+0x114/0x171
May 13 14:00:14 ant kernel: [<c02bb728>] __mutex_unlock_slowpath+0x2b/0xeb
May 13 14:00:14 ant kernel: [<c0175bbc>] sync_dirty_buffer+0x7b/0xbf
May 13 14:00:14 ant kernel: [<fd9aaa77>] ext3_commit_super+0x3a/0x4e [ext3]
May 13 14:00:14 ant kernel: [<fd96e7b2>] journal_unlock_updates+0x15/0x3b [jbd]
May 13 14:00:14 ant kernel: [<fd9abde9>] ext3_unlockfs+0x39/0x4b [ext3]
May 13 14:00:14 ant kernel: [<c0174c8c>] thaw_bdev+0x23/0x67
May 13 14:00:14 ant kernel: [<f887049f>] unlock_fs+0x1d/0x2e [dm_mod]
May 13 14:00:14 ant kernel: [<f8870fa8>] dm_resume+0x9a/0xcb [dm_mod]
May 13 14:00:14 ant kernel: [<f8873ea3>] dev_suspend+0x115/0x180 [dm_mod]
May 13 14:00:14 ant kernel: [<f8874710>] ctl_ioctl+0x1fb/0x262 [dm_mod]
May 13 14:00:14 ant kernel: [<c0167556>] dput+0x17/0xcd
May 13 14:00:14 ant kernel: [<c0160fff>] __link_path_walk+0x9da/0xb23
May 13 14:00:14 ant kernel: [<f8873d8e>] dev_suspend+0x0/0x180 [dm_mod]
May 13 14:00:14 ant kernel: [<f8874515>] ctl_ioctl+0x0/0x262 [dm_mod]
May 13 14:00:14 ant kernel: [<c0162f21>] do_ioctl+0x51/0x55
May 13 14:00:14 ant kernel: [<c01630ea>] vfs_ioctl+0x1c5/0x249
May 13 14:00:14 ant kernel: [<c011e13d>] tasklet_action+0x2a/0x5a
May 13 14:00:14 ant kernel: [<c01631a2>] sys_ioctl+0x34/0x52
May 13 14:00:14 ant kernel: [<c0103dca>] syscall_call+0x7/0xb
May 13 14:00:14 ant kernel: [<c02b0000>] tcp_v6_connect+0x4ca/0x64c
May 13 14:00:14 ant kernel: =======================
The warning is on
DEBUG_LOCKS_WARN_ON(lock->owner != current_thread_info());
Not sure if it's because the resume was done by a different
process as for the suspend.
I'm not very confident with that solution as even though I tried
to take the maximum of precautions, I sense it could fail in
many ways for instance in the future if any of the commands
involved in the process are upgraded, so it would be nice if
ddsnap had builtin support for that.
Please find the updated do-snapshot script attached. There's a
run_in_fortress() function that sets up a "fortress" FS in tmpfs
and runs a shell command line. That is the shell equivalent of a
mlockall(2).
Best regards,
Stéphane
Jiaying Zhang
We did try to wrap 'ddsnap create' with 'dmsetup suspend' and 'dmsetup resume'
in zumastor before. We decided to switch to the 'blockdev --flushbufs' approach
mostly because the former would cause a period of denial-of-service during the
snapshot creation. This period can take several seconds because 'dmsetup suspend'
needs to wait for all of the IO requests to be synchronized and no further IOs can
make progress until the device is resumed. 'blockdev --flushbufs' on the other hand
still allows further IOs.
Your experiments are ahead of us. We haven't tried ddsnap as rootfs ourselves.
I think we should set up a testing machine here and run your script. It's possible
that we do need to suspend the device during a snapshot creation to guarantee
that FS is in a consistent state. If that is the case, we can add this option in
ddsnap and zumastor.
Jiaying
Jiaying Zhang
It may be a false warning.
Jiaying
Stephane Chazelas
[...]
> 'dmsetup resume' in zumastor before. We decided to switch to
> the 'blockdev --flushbufs' approach mostly because the former
> would cause a period of denial-of-service during the snapshot
> creation. This period can take several seconds because
> 'dmsetup suspend' needs to wait for all of the IO requests to
> be synchronized and no further IOs can make progress until the
> device is resumed. 'blockdev --flushbufs' on the other hand
> still allows further IOs.
However the suspend-resume is being used by LVM and I'm not
aware of anyone reporting issues with that. Taking LVM snapshots
of the rootfs looks pretty seemless to me (a lot more than
deleting ddsnap snapshots for instance ;)). Is there anything
different in ddsnap that makes that approach inappropriate?
It could be made an option in ddsnap.
--
Stéphane
Stephane Chazelas
> Do you have DEBUG_LOCKING configurations on in your kernel .config?
> It may be a false warning.
I do have DEBUG_MUTEXES as well as a number of other
CONFIG_DEBUG's which I didn't explicitely select. I'll try and
find out how they got selected.
Anyway, I'm not too worried about that warning.
--
Stéphane
Jiaying Zhang
The differences I can think of are that 'ddsnap server' runs in user-space
and there is extra overhead on writes. I don't know how LVM avoids this
performance issue. We need to spend more time looking at this.
Jiaying
Stephane Chazelas
[...]
> > aware of anyone reporting issues with that. Taking LVM snapshots
> > of the rootfs looks pretty seemless to me (a lot more than
> > deleting ddsnap snapshots for instance ;)). Is there anything
> > different in ddsnap that makes that approach inappropriate?
>
>
> The differences I can think of are that 'ddsnap server' runs in user-space
> and there is extra overhead on writes.
Well, there is extra overhead on writes only when comparing to
snapshot_origin that have to service few snapshots. From 4, 5
snapshots on, ddsnap should be more efficient, shouldn't it as
that's the whole point of it.
> I don't know how LVM avoids this
> performance issue. We need to spend more time looking at this.
[...]
I don't think LVM avoids it. I suspect that if you do an LVM
snapshot on a FS that already has a big number of snapshots
while there is extensive (I)O ongoing, the snapshot creation
will take some time, but I can't see how you would avoid that.
And that time should be bound as the processes write()s will be
suspended as the FS is locked.
One could do a flushbuf followed by a suspend I guess to reduce
the time for the "suspend".
--
Stéphane
Stephane Chazelas
[...]
> > > We did try to wrap 'ddsnap create' with 'dmsetup suspend' and
> > > 'dmsetup resume' in zumastor before. We decided to switch to
> > > the 'blockdev --flushbufs' approach mostly because the former
> > > would cause a period of denial-of-service during the snapshot
> > > creation. This period can take several seconds because
> > > 'dmsetup suspend' needs to wait for all of the IO requests to
> > > be synchronized and no further IOs can make progress until the
> > > device is resumed. 'blockdev --flushbufs' on the other hand
> > > still allows further IOs.
[...]
> and there is extra overhead on writes. I don't know how LVM avoids this
> performance issue. We need to spend more time looking at this.
I had a look at the zumastor code:
[[ $filesystem = "xfs" ]] && xfs_freeze -f $(mount_name $vol -1)
ddsnap create $server $id >/dev/null 2>/dev/null
status=$?
[[ $filesystem = "xfs" ]] && xfs_freeze -u $(mount_name $vol -1)
# flush origin device to prevent the race between origin writes and snapshot create
blockdev --flushbufs /dev/mapper/$vol
It looks like that xfs_freeze does the same thing as dmsetup
suspend, doesn't it?
Also, why flushing the bufs *after* the snapshot has been
created?
--
Stéphane
Stephane Chazelas
[...]
> rootfs ourselves. I think we should set up a testing machine
> here and run your script. It's possible that we do need to
> suspend the device during a snapshot creation to guarantee
> that FS is in a consistent state. If that is the case, we can
> add this option in ddsnap and zumastor.
Hi Jiaying,
in case you're interested, here is a step by step procedure to
do that. Hopefully I didn't forget some of the steps.
First, you probably need a debian unstable system, you need
the root FS to be on an LVM device and to be using an initrd to
boot the system. You need ddsnap support in the kernel (as a
module), and the dmsetup, udev, lvm2, ddsnap, busybox packages
to be installed.
as root:
cd /
tar xvf /tmp/ddsnap-root.tar.gz
(where ddsnap-root.tar.gz is attached)
vi /etc/cron.d/snapshots
and update to reflect the name of the LV you want to snapshot.
update-initramfs -u -k 2.6.24.2
(adapt the kernel version)
You need to create an LVM LV for the snapstore.
That VG must have the same name as the origin LV put with the
"-ddsnap" suffix.
So, if your rootfs is on /dev/vg/lv, the snapstore must be
/dev/vg/lv-ddsnap or /dev/other-vg/lv-ddsnap (or
/dev/mapper/vg-lv--ddsnap)
In theory, you don't need to initialise the snapstore as the
initrd script will try to do that if it fails to use it, but I'm
not sure if I ever tried that.
Then reboot, making sure you use the new initrd.
The initrd script will start the ddsnap agent and server and
move your lv to /dev/mapper/vg-lv-ddreal and make your
/dev/mapper/vg-lv the -1th ddsnap. The sockets are in
/dev/ddsnap
Then the cron job will start taking snapshots every hour.
Note that you can't do anything anymore LVM-wise to your rootfs
LV. In particular, don't try to take LVM snapshots of it with
lvcreate!
--
Stéphane
Dan Kegel
<Stephane...@yahoo.fr> wrote:
> [[ $filesystem = "xfs" ]] && xfs_freeze -f $(mount_name $vol -1)
> ddsnap create $server $id >/dev/null 2>/dev/null
> status=$?
> [[ $filesystem = "xfs" ]] && xfs_freeze -u $(mount_name $vol -1)
> # flush origin device to prevent the race between origin writes and snapshot create
> blockdev --flushbufs /dev/mapper/$vol
>
> It looks like that xfs_freeze does the same thing as dmsetup
> suspend, doesn't it?
Maybe. Does dmsetup suspend invoke xfs_freeze? If so, then we could try
uniformly doing dmsetup suspend. As Jiaying noted, there were
problems with this in the past, but I agree, it's something we should
try if we can.
> Also, why flushing the bufs *after* the snapshot has been
> created?
Jiaying or Daniel may remember...
- Dan
Jiaying Zhang
We added xfs_freeze hook in snapshot creation mostly because 'sync' does
not work as usual with xfs. But xfs_freeze does lock FS, which sync does not do.
'dmsetup suspend' also locks FS. Looking at the code, xfs_freeze and 'dmsetup suspend'
basically do the same thing.
Also, why flushing the bufs *after* the snapshot has been
created?
When ddsnap server creates a snapshot, there may still be some inflight IOs to the
origin device. If we read the created snapshot device immediately before those IOs
finish, we may read stale data. We run blockdev --flushbuf after creating a snapshot
but before a snapshot device is created to prevent invalid access to the stale data.
Jiaying
Jiaying Zhang
Jiaying
Christopher Smith
>
>
> On 5/14/08, *Stephane Chazelas* <Stephane...@yahoo.fr
invokes xfs_freeze, so one is basically the block layer abstraction of
the other.
--Chris
Stephane Chazelas
[...]
> > ddsnap create $server $id >/dev/null 2>/dev/null
> > status=$?
> > [[ $filesystem = "xfs" ]] && xfs_freeze -u $(mount_name $vol -1)
> > # flush origin device to prevent the race between origin writes and
> > snapshot create
> >
> > blockdev --flushbufs /dev/mapper/$vol
> > Also, why flushing the bufs *after* the snapshot has been
> > created?
>
> When ddsnap server creates a snapshot, there may still be some
> inflight IOs to the origin device. If we read the created
> snapshot device immediately before those IOs finish, we may
> read stale data. We run blockdev --flushbuf after creating a
> snapshot but before a snapshot device is created to prevent
> invalid access to the stale data.
Hi Jiaying,
I don't follow you here.
I wouldn't expect the creation of the snapshot device (we're
talking of echo ... ddsnap ... id!=-1 | dmsetup create, here
right?) to have any incidence on the content of the snapshot,
and I'd expect the "ddsnap create" to freeze the FS in time into
the new snapshot.
So if one does *not* do the flushbufs *before* the "ddsnap
create", then I would expect the snapshot to contain stale data.
--
Stéphane
Dan Kegel
That solves the ugly API issue, maybe...
Jiaying Zhang
On Wed, May 14, 2008 at 10:38:55AM -0700, Jiaying Zhang wrote:
[...]
> > [[ $filesystem = "xfs" ]] && xfs_freeze -f $(mount_name $vol -1)
> > ddsnap create $server $id >/dev/null 2>/dev/null
> > status=$?
> > [[ $filesystem = "xfs" ]] && xfs_freeze -u $(mount_name $vol -1)
> > # flush origin device to prevent the race between origin writes and
> > snapshot create
> >
> > blockdev --flushbufs /dev/mapper/$vol
[...]
> > Also, why flushing the bufs *after* the snapshot has been
> > created?
>
> When ddsnap server creates a snapshot, there may still be some
> inflight IOs to the origin device. If we read the created
> snapshot device immediately before those IOs finish, we may
> read stale data. We run blockdev --flushbuf after creating a
> snapshot but before a snapshot device is created to prevent
> invalid access to the stale data.
[...]
Hi Jiaying,
I don't follow you here.
Hmm. I guess I need to give more details on ddsnap to explain this.
Basically, when the kernel ddsnap thread receives an origin write request,
it sends a query to the user-level 'ddsnap server'. 'ddsnap server' does
the proper handling such as copying the old data to the snapshot store
and then acks the request. After receiving the reply from the 'ddsnap server',
the kernel ddsnap thread then submits the write request to the physical
device. So there is a window between the time that 'ddsnap server' acks
the write request and the time that the physical write really happens.
If during this window, a snapshot is created and an application reads the
same block that the write is targeted to, it would read the stale data.
So the purpose of having a 'blockdev --flushbuf' after 'ddsnap create' and
before 'dmsetup create' is to prevent users from reading invalid snapshot
data, not to guarantee the consistent state of the origin volume.
Is it a little clearer? Hopefully I haven't made you more confused :).
I wouldn't expect the creation of the snapshot device (we're
talking of echo ... ddsnap ... id!=-1 | dmsetup create, here
right?) to have any incidence on the content of the snapshot,
No, it doesn't.
and I'd expect the "ddsnap create" to freeze the FS in time into
the new snapshot.
We will look at how to guarantee this.
Jiaying