Patch for adding zumastor get/set properties
Jiaying Zhang
properties after a volume/target/source is defined. We talked about adding
a 'zumastor set/get property' command before and here is a proposal patch
I implemented this week.
The syntax is:
zumastor get <vol> <property>[,...] [source] | [target <host>]
zumastor set <vol> <property>=<value>[,...] [source] | [target <host>]
The commands take two variables for volume and schedule properties, e.g.
zumastor get <vol> cachesize,mountpoint,hourly,daily
zumastor set <vol> cachesize=256M,mountpoint=/volmnt,hourly=24,daily=7
For replication related variables, the commands take four variables on a master
and three variables on a slave, e.g.
zumastor get <vol> period,compress,ratelimit target <host>
zumastor set <vol> period=3600,compress=0,ratelimit=10M target <host>
zumastor get <vol> period source
zumastor set <vol> period=3600 source
DanP proposed to use 'zumastor redefine' commands instead, i.e., every
'zumastor define' would have a corresponding 'zumastor redefine' command
that changes certain properties. This will allow us to reuse the current option
parsing code in the 'zumastor define' commands. It also allows users to
follow the same syntax as the existing 'zumastor define' commands.
After rethinking about it, I feel it is indeed an attractive alternative approach.
One concern I have though is that we may need to add a lot of 'redefine'
handling in the existing 'zumastor define' command code because
many properties require special cleanup when we change them. So it will
add more code complexity. Another concern is that users are forced to
follow the 'zumastor define' command syntax to change any property.
E.g., they need to use
'zumastor redefine volume <vol> <origin> <snapstore> --cachesize xxx'
to change ddsnapd cachesize even though the <origin> and <snapstore>
parameters are not required.
So any user's opinions here?
The patch of implementing 'zumastor get/set property' is attached, as well
as a cbtb test that includes more examples.
Jiaying
Dan Kegel
> The syntax is:
> zumastor get <vol> <property>[,...] [source] | [target <host>]
> zumastor set <vol> <property>=<value>[,...] [source] | [target <host>]
I'm not a user, but I play one on TV :-)
I think it'd be worth writing up a short comparison of how to do
a few common tasks in various systems
(old zumastor, proposed new zumastor, zfs, netapp).
Here's a start, with just two tasks.
Probably belongs on a wiki or in a text file in our doc directory...
1. Disabling atime update on access:
Netapp:
$ vol options volname no_atime_update on
see http://ecserv1.uwaterloo.ca/netapp/man/man1/na_vol.1.html
ZFS:
$ zfs set atime=off volname
Old Zumastor:
?
Jiaying's proposal:
?
2. Scheduling snapshots:
Netapp:
snap sched [ vol_name [ weeks [ days [ hours[@list] ] ] ] ]
see http://ecserv1.uwaterloo.ca/netapp/man/man1/na_snap.1.html
ZFS:
no provision for it -- you have to put it in crontab yourself.
Rolling over old snapshots is also manual, but is built on snapshot renaming.
See e.g. http://blogs.sun.com/mmusante/entry/rolling_snapshots_made_easy
Old Zumastor:
?
Jiaying's proposal:
?
Daniel Phillips
Six new commands (three kinds of get and three kinds of set) versus one
new command "redefine", which otherwise uses syntax that already exists.
Daniel
Daniel Phillips
Let me point out that your proposal has one big advantage: it exists
and has a patch. I'm actually inclined to say, ok put it in, it solves
the problem, not the way I like, but then this is not a perfect world.
Still... if you suddenly were to realize that "redefine" is a better
way to go, I would like that even more.
Daniel
Dan Kegel
> On Friday 09 May 2008 19:49, Jiaying Zhang wrote:
>> So any user's opinions here?
>
> Let me point out that your proposal has one big advantage: it exists
> and has a patch. I'm actually inclined to say, ok put it in, it solves
> the problem, not the way I like, but then this is not a perfect world.
IMHO this is an area where we don't need to rush... let's
not put it in until we're sure we have a coherent user story.
I'd actually be happy putting this off until 0.10, and focusing
0.9 on just that one optimization.
Daniel Phillips
The best approach, indeed.
Daniel
Shapor Naghibzadeh
I agree. We don't want to be changing the user interface constantly.
One thing I don't like about it is how much code it adds:
bin/zumastor | 321 ++++++++++++++++++++++++++++++++++++++++++++++------
lib/zumastor/common | 135 ++++++++++++++++-----
2 files changed, 389 insertions(+), 67 deletions(-)
Shapor
Jiaying Zhang
Six new commands (three kinds of get and three kinds of set) versus one
new command "redefine", which otherwise uses syntax that already exists.
Only two new commands, 'zumastor get' and 'zumastor set'. I gave six examples
on how to use the commands in my last email. With redefine approach, we need
to add 'zumastor redefine volume', 'zumastor redefine master', 'zumastor redefine target',
'zumastor redefine source', and 'zumastor redefine schedule'. 'zumastor redefine'
actually serves the same purpose as 'zumastor set' with different syntax:
zumastor redefine <type> --property <value>
Jiaying
Jiaying Zhang
On Fri, May 9, 2008 at 7:49 PM, Jiaying Zhang <jiay...@google.com> wrote:
> The syntax is:
> zumastor get <vol> <property>[,...] [source] | [target <host>]
> zumastor set <vol> <property>=<value>[,...] [source] | [target <host>]
I'm not a user, but I play one on TV :-)
I think it'd be worth writing up a short comparison of how to do
a few common tasks in various systems
(old zumastor, proposed new zumastor, zfs, netapp).
Here's a start, with just two tasks.
Probably belongs on a wiki or in a text file in our doc directory...
1. Disabling atime update on access:
Netapp:
$ vol options volname no_atime_update on
see http://ecserv1.uwaterloo.ca/netapp/man/man1/na_vol.1.html
ZFS:
$ zfs set atime=off volname
Old Zumastor:
?
or via the 'zumastor remount' command that Frank added several months ago:
'zumastor remount <mountpoit> <mount options>
Jiaying's proposal:
?
In addition to the old ways, a user can also use
'zumastor set <vol> mountops="noatime"
2. Scheduling snapshots:
Netapp:
snap sched [ vol_name [ weeks [ days [ hours[@list] ] ] ] ]
see http://ecserv1.uwaterloo.ca/netapp/man/man1/na_snap.1.html
ZFS:
no provision for it -- you have to put it in crontab yourself.
Rolling over old snapshots is also manual, but is built on snapshot renaming.
See e.g. http://blogs.sun.com/mmusante/entry/rolling_snapshots_made_easy
Old Zumastor:
?
Users can change snapshot schedules later with the same command, but
our current implementations does not remove old snapshots if the number
of schedules snapshots is decreased.
Jiaying's proposal:
?
zumastor set <vol> hourly=xxx.
The old snapshots are removed if the number of hourly snapshots is decreased.
Jiaying
Daniel Phillips
> > Six new commands (three kinds of get and three kinds of set) versus one
> > new command "redefine", which otherwise uses syntax that already exists.
>
> Only two new commands, 'zumastor get' and 'zumastor set'. I gave six examples
> on how to use the commands in my last email. With redefine approach, we need
Only two new command words, but six different command forms, each needing
documentation and specific code to implement.
The redefine strategy adds one new command word and no new command forms,
and can be documented generically (i.e., define|redefine ... the define form
provides default settings for any options not given explicitly in the
command while the redefine form changes only the options specified for
the command.
> to add 'zumastor redefine volume', 'zumastor redefine master', 'zumastor
> redefine target',
> 'zumastor redefine source', and 'zumastor redefine schedule'. 'zumastor
> redefine'
> actually serves the same purpose as 'zumastor set' with different syntax:
> zumastor redefine <type> --property <value>
I expect that "redefine" can be implemented with considerably less code,
and certainly requires less documentation. The easiest way for me to prove
the documentation point is to provide a patch for the man page.
Regards,
Daniel
Jiaying Zhang
On Saturday 10 May 2008 11:28, Jiaying Zhang wrote:
> > Six new commands (three kinds of get and three kinds of set) versus one
> > new command "redefine", which otherwise uses syntax that already exists.
>
> Only two new commands, 'zumastor get' and 'zumastor set'. I gave six examples
> on how to use the commands in my last email. With redefine approach, we need
Only two new command words, but six different command forms, each needing
documentation and specific code to implement.
The redefine strategy adds one new command word and no new command forms,
and can be documented generically (i.e., define|redefine ... the define form
provides default settings for any options not given explicitly in the
command while the redefine form changes only the options specified for
the command.
As I mentioned, 'zumastor redefine' serves the same purpose as 'zumastor set'.
We may still want to provide 'zumastor get' command so that users have an
easy way to obtain the current value of individual properties. I think 'zumastor redefine'
and 'zumastor set' both have their advantages. 'zumastor redefine' can follow
the same syntax as 'zumastor define'. But one problem is that users don't really
need to provide all of the parameters to change a property. Another problem
is that some options don't take any values, like --export and --nocompress. We need
to add a corresponding opposite option for each such option so that users can
use 'zumastor redefine' to turn an option off.
> to add 'zumastor redefine volume', 'zumastor redefine master', 'zumastor
> redefine target',
> 'zumastor redefine source', and 'zumastor redefine schedule'. 'zumastor
> redefine'
> actually serves the same purpose as 'zumastor set' with different syntax:
> zumastor redefine <type> --property <value>
I expect that "redefine" can be implemented with considerably less code,
and certainly requires less documentation. The easiest way for me to prove
the documentation point is to provide a patch for the man page.
I am not sure how much code 'redefine' can save. As I said, many properties
need special handling when their values are changed. E.g., when we change
the volume mountpoint, we need to check if the '--samba' option is set and
if so, we need to remount the snapshot samba mountpoints. zumastor 'redefine'
and 'define' can share the option parsing code, but we do need to add
additional 'redefine' handling code for each command.
As I said, I think both approaches have their advantages. We may want
to ask uses' opinions to decide which way to use.
Jiaying
Pau Garcia i Quiles
I know this sounds a bit egoistic but I'd like to have both approaches :-)
The advantage of 'redefine' over 'set' is it's the same syntax 'define' uses, which is quite intuitive, especially when using the command line.
The good thing about 'set' and 'get' is they make it easy to develop GUIs, at least until we have some 'libzumastor'. Take, for instance, this use case in a hypothetical Zumastor GUI:
- When you load the GUI, you parse the output of 'zumastor status' to get all the information about all the volumes. Using 'get' would also be OK.
- User wants to change an option. If the only way to do that were 'zumastor redefine', the GUI would need to collect all the options related to that volume. Using 'zumastor set', the GUI business logic is easier.
- Now imagine you are showing a window containing only the information about a volume, or only partial information: 'get' comes of hand here because that way you don't need to parse the whole 'zumastor status' output. That's, of course, assuming the information goes directly from Zumastor to the GUI. If the workflow is zumastor -> database -> GUI maybe 'zumastor get' does not make much of a sense because what you will usually want is to have all the info (i.e. parsing 'zumastor status') in the database (for trending, for example).
Pau Garcia i Quiles
http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)
Daniel Phillips
> On 5/10/08, Daniel Phillips <phil...@phunq.net> wrote:
> As I mentioned, 'zumastor redefine' serves the same purpose as 'zumastor
> set'.
And I completely overlooked that, thanks for mentioning it again.
> We may still want to provide 'zumastor get' command so that users have an
> easy way to obtain the current value of individual properties. I think
> 'zumastor redefine'
OK, I see where you are going, let me digest that for a bit.
> 'zumastor redefine' can follow the same syntax as 'zumastor define'. But
> one problem is that users don't reallyneed to provide all of the parameters
> to change a property.
I am not sure what you are saying. Example?
> Another problem is that some options don't take any values, like --export
> and --nocompress. We need to add a corresponding opposite option for each
> such option so that users can use 'zumastor redefine' to turn an option off.
For example, --export=no and --compress=no or --compression=none
> I am not sure how much code 'redefine' can save. As I said, many properties
> need special handling when their values are changed. E.g., when we change
> the volume mountpoint, we need to check if the '--samba' option is set and
> if so, we need to remount the snapshot samba mountpoints. zumastor
> 'redefine'and 'define' can share the option parsing code, but we do need to add
> additional 'redefine' handling code for each command.
Indeed. The alternative needs to be coded. I certainly can't take
the time to do it before I get back from Linuxtag, June 4th.
> As I said, I think both approaches have their advantages. We may want
> to ask uses' opinions to decide which way to use.
And, guess what, a user just showed up (pgquiles) and says he wants
both.
Well, let's email about it a little more, the discussion is going
somewhere. As usual, when you stick to your point, chances are there
is a very good reason for it :-)
Once again, if we go with your formulation just as you have it I am not
going to jump off a bridge, but still let's take our time. Writing the
man page changes for both approaches seems like a good thing to do.
The redefine method might need to be augmented with some kind of "get",
indeed. What are some examples where a user would need the get, as
opposed to just printing out status?
If I did augment redefine with a get-like command, I would favor doing
it by adding new features to status, rather than adding a new command.
Daniel
Dan Kegel
> I know this sounds a bit egoistic but I'd like to have both approaches :-)
>
> The advantage of 'redefine' over 'set' is it's the same syntax 'define'
> uses, which is quite intuitive, especially when using the command line.
I'm not sure keeping both (except transitionally?) is a good idea.
Having two ways to do things makes it harder to tell our story properly
to new users.
Once we have 'set', 'define' could lose all of its options, and just
create a new node in the namespace to hang set's upon, perhaps.
Daniel Phillips
> The good thing about 'set' and 'get' is they make it easy to develop GUIs,
> at least until we have some 'libzumastor'. Take, for instance, this use case
> in a hypothetical Zumastor GUI:
> - When you load the GUI, you parse the output of 'zumastor status' to get
> all the information about all the volumes. Using 'get' would also be OK.
> - User wants to change an option. If the only way to do that were 'zumastor
> redefine', the GUI would need to collect all the options related to that
> volume. Using 'zumastor set', the GUI business logic is easier.
> - Now imagine you are showing a window containing only the information about
> a volume, or only partial information: 'get' comes of hand here because that
> way you don't need to parse the whole 'zumastor status' output. That's, of
> course, assuming the information goes directly from Zumastor to the GUI. If
> the workflow is zumastor -> database -> GUI maybe 'zumastor get' does not
> make much of a sense because what you will usually want is to have all the
> info (i.e. parsing 'zumastor status') in the database (for trending, for
> example).
OK, it seems to me that the primary driver for a "get" command is
adding a gui. Would it not be a good idea for the developer of the
gui to develop whatever interfaces are needed at the same time?
In this case it is particularly simple to develop such interfaces,
based on a well-defined filesystem layout as it is. For example,
if the gui is to be developed in python, then develop the access
methods you need in Python as well using your knowledge of the
structure of the filesystem database, which has proved to be quite
stable.
Daniel
Daniel Phillips
> Once we have 'set', 'define' could lose all of its options, and just
> create a new node in the namespace to hang set's upon, perhaps.
Which would regularize the command set, but would force the user to
enter multiple commands where today a single command is sufficient.
Daniel
Dan Kegel
> > Once we have 'set', 'define' could lose all of its options, and just
> > create a new node in the namespace to hang set's upon
>
> user to enter multiple commands where today a single
> command is sufficient.
This is why I wanted to compare command sets for
the various popular products in the space. Could be
that other products are more regular and verbose because
people love a little bit more verbosity if it makes
things easier to learn. Especially if they often tend to
use scripts on top of the raw interface, which might well be
the case.
Jiaying Zhang
I am not sure what you are saying. Example?
E.g., the syntax of 'zumastor define volume' is
"zumastor define volume <vol> <origin> <snapstore> [options]"
A user needs to run the following command to change the cachesize property
with the 'zumastor redefine' command:
"zumastor redefine volume <origin> <snapstore> --cachesize xxx".
But the <origin> and <snapstore> parameters are not needed here.
> Another problem is that some options don't take any values, like --export
> and --nocompress. We need to add a corresponding opposite option for each
> such option so that users can use 'zumastor redefine' to turn an option off.
For example, --export=no and --compress=no or --compression=none
Yes. We can also go this way by changing the current option handling code.
Jiaying
Jiaying Zhang
our discussion: http://code.google.com/p/zumastor/issues/detail?id=137
There are two existing issues that are related to the problem:
issue 49: --samba only shows snapshots taken after --samba
and issue 105: missing zumastor forget schedule.
Issue 105 was target to 0.9. We may want to defer it to 0.10 as well.
Jiaying