How do I avoid a full disk shadow copy when I shutdown/boot a node back into the cluster?

[Len Whitwer]

Hi All:

How do I avoid a full disk shadow copy when I shutdown/boot a node
back into the cluster?

Have two rx3600's with a SAN on each running OVMS I64 ver 8.4 with
clustering and Volume Shadowning.

When one node (Clustered) goes down and then comes back into the
cluster a
COMPLETE shodow copy action is initiated on there-join. This takes
about 9 hours
which is unacceptable. There has to be a way around this where only
the files
that have CHANGED get updated!!!!!

Anyone have any ideas???

Len Whitwer

-Len Whitwer
Senior Sales Rep.
Puget Sound Data Systems
19501 144th Ave NE , Suite D-100
Woodinville, WA 98072
Ph: 425-488-0710
Fax: 425-488-6414
Email: l...@psds.com
Web Page: www.psds.com

[Ken Fairfield]

On Wednesday, April 4, 2012 12:03:32 PM UTC-7, Len Whitwer wrote:
> Hi All:
>
> How do I avoid a full disk shadow copy when I shutdown/boot a node
> back into the cluster?
>
> Have two rx3600's with a SAN on each running OVMS I64 ver 8.4 with
> clustering and Volume Shadowning.
>
> When one node (Clustered) goes down and then comes back into the
> cluster a
> COMPLETE shodow copy action is initiated on there-join. This takes
> about 9 hours
> which is unacceptable. There has to be a way around this where only
> the files
> that have CHANGED get updated!!!!!

1) Be sure you've configured and enabled HBMM (Host-Based Mini-Merge)
on all your shadow volumes.

2) Be sure you've called the appropriate shutdown procedures for
your various applications in SYSHUTDWN.COM.

Regarding (2), VMS will do its best to kill all processes on the
system before dismounting the shadow volumes, but there are a
variety of reasons it might no succeed. It's up to the system
manager to invoke any and all application shutdown procedures
(think databases for example) during system shutdown.

Regarding (1), the system disk is always subject to merge on
shutdown, even a normal/controlled shutdown. With the move to
FibreChannel (from CI), volume merges are no longer done in the
storage controller. Rather, as of v7.3-2 (or late v7.3-1 with
appropriate ECOs), HBMM uses the functionality initially developed
for mini-copy to keep track of modified blocks (groups of 127
blocks usually) that will need to be merged should the shadow
volume be dismounted abnormally. Of course for this to work,
you need to be in a cluster and at least one cluster member
must survive the reboot or crash of some other system. The
surviving member will merge the volume(s).

-Ken

[Jose Baars]

And to add to that: HBMM works like a charm.
Before 2003 or so, we used to have shadow copies
of hours at node shutdown and startup. After that,
it was a matter of minutes.

It is a reason to not have a shadowed system disk,
but that decision is yours.

[abrsvc]

We need a few more details. For example: is mini-merge enabled? How are the 2 SANs available to the cluster? Can each system see each SAN? etc...

Dan

[VAXman-]

In article <6e2ac757-dacb-4e35...@pg6g2000pbb.googlegroups.com>, Len Whitwer <l...@psds.com> writes:
>Hi All:
>
>How do I avoid a full disk shadow copy when I shutdown/boot a node
>back into the cluster?
>
>Have two rx3600's with a SAN on each running OVMS I64 ver 8.4 with
>clustering and Volume Shadowning.
>
>When one node (Clustered) goes down and then comes back into the
>cluster a
>COMPLETE shodow copy action is initiated on there-join. This takes
>about 9 hours
>which is unacceptable. There has to be a way around this where only
>the files
>that have CHANGED get updated!!!!!
>
>Anyone have any ideas???

Shadow device is? DSAxxx?

What devices are part of that shadow set?


I crash one of my systems regularly (development) and the shadow sets
all come up with mini-merge. Very quickly, all of my shadow sets are
available.

--
VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG

Well I speak to machines with the voice of humanity.

[Robert A. Brooks]

On 4/4/2012 3:23 PM, Ken Fairfield wrote:
> Rather, as of v7.3-2 (or late v7.3-1 with
> appropriate ECOs), HBMM uses the functionality initially developed
> for mini-copy to keep track of modified blocks (groups of 127
> blocks usually) that will need to be merged should the shadow
> volume be dismounted abnormally.

HBMM on V7.3-1 was only sent to Intel under a special arrangement.

V7.3-2 UPDATE 6 (I think) was the first generally-available release of HBMM.

-- Rob

[Phillip Helbig---undress to reply]

In article
<6e2ac757-dacb-4e35...@pg6g2000pbb.googlegroups.com>, Len
Whitwer <l...@psds.com> writes:

> How do I avoid a full disk shadow copy when I shutdown/boot a node
> back into the cluster?

(DIS)MOUNT/POLICY=MINICOPY.

> When one node (Clustered) goes down and then comes back into the
> cluster a
> COMPLETE shodow copy action is initiated on there-join. This takes
> about 9 hours
> which is unacceptable. There has to be a way around this where only
> the files
> that have CHANGED get updated!!!!!

That is what MINICOPY is for. (It is groups of 127 blocks, not files,
which have changed which is the basis.) I went from 9 hours to a few
seconds in such cases (slower hardware and smaller disks than you have,
though, but the improvement should be about the same).

Also consider MINIMERGE for unexpected problems.

[Phillip Helbig---undress to reply]

In article
<1281120.105.1333567428620.JavaMail.geo-discussion-forums@pbnt10>, Ken

Fairfield <ken.fa...@gmail.com> writes:

> 1) Be sure you've configured and enabled HBMM (Host-Based Mini-Merge)
> on all your shadow volumes.

In his case (one node and its shadowset members go away and come back),
isn't MINICOPY more relevant?

Another idea: I have this set up:

SHADOW_MBR_TMO=300 ! allow one node to reboot

The idea is that if the reboot takes less than 5 minutes, no copy or
merge will be needed, but of course I/O will hang during that time.
Whether or not this is the better solution than MINICOPY/MINIMERGE
depends on your applications. It's a DYNAMIC parameter, so you could
have a default value for unplanned reboots and perhaps change it
temporarily for planned reboots.

[Phillip Helbig---undress to reply]

In article
<97577968-73f9-447a...@l18g2000vbx.googlegroups.com>,

What you can't do is use HBMM on a shadowed system disk. But you can
set up HBMM on other disks in the cluster. Actually, you can have HBMM
on a shadowed system disk if you set DOSD.

[Ken Fairfield]

On Wednesday, April 4, 2012 1:29:05 PM UTC-7, Robert A. Brooks wrote:
> On 4/4/2012 3:23 PM, Ken Fairfield wrote:
> > Rather, as of v7.3-2 (or late v7.3-1 with
> > appropriate ECOs), HBMM uses the functionality initially developed
> > for mini-copy to keep track of modified blocks (groups of 127
> > blocks usually) that will need to be merged should the shadow
> > volume be dismounted abnormally.
>
> HBMM on V7.3-1 was only sent to Intel under a special arrangement.

That woulda' been me. :-} ;-) Intel D1C in Hillsboro, OR, plus
others in our "virtual factory".

> V7.3-2 UPDATE 6 (I think) was the first generally-available release of HBMM.

Right. Unfortunately, we were stuck on 7.3-1 and weren't able
to upgrade to 7.3-2. Very strict company-wide change control,
plus moving off VMS for the new and subsequent fabs... :-(

-Ken

[Len Whitwer]

>     -Ken- Hide quoted text -
>
> - Show quoted text -

Thanks Ken. Will check out HBMM.

Len

[Len Whitwer]

On Apr 4, 2:41 pm, hel...@astro.multiCLOTHESvax.de (Phillip Helbig---
undress to reply) wrote:
> In article
> <6e2ac757-dacb-4e35-97eb-fea92c6d9...@pg6g2000pbb.googlegroups.com>, Len

Thanks for this info. Will continue and get back to you if more
questions.

Len

[Keith Parris]

On 4/4/2012 1:03 PM, Len Whitwer wrote:
> How do I avoid a full disk shadow copy when I shutdown/boot a node
> back into the cluster?

I'm wondering if it is a Full-Copy or a Full-Merge that you are
experiencing?

> Have two rx3600's with a SAN on each running OVMS I64 ver 8.4 with
> clustering and Volume Shadowning.

You mention there's a SAN on each node. Is that two different SANs or
did you mean that there is one SAN with all the disks accessible from
both nodes?

If the SANs are separate, that would imply MSCP Serving is required for
one node to access the disk on the other node's SAN. In this case,
rebooting of the node would remove access to the shadowset members on
that node's SAN for more than SHADOW_MBR_TMO seconds, causing the other
node to remove that shadowset member from the shadowset, requiring a
Full-Copy to resynchronize when the node reboots and access is restored
to the disks on that SAN. If this is the case, then what you want is a
Mini-Copy operation to resynchronize, and $DISMOUNT/POLICY=MINICOPY done
from the node which will remain up will allow a Mini-Copy instead of a
Full-Copy on the node's return. For this to occur automatically even in
the event of an unplanned node outage, Google AMCVP, Automatic Mini-Copy
on Volume Processing, to see how to set this up.

If there is one SAN to which both nodes have direct access, then it's
most likely that the shadowset isn't getting properly dismounted and a
Full-Merge is the result. If this is the case, then setting up a
Mini-Merge policy should do the trick.

To clarify:

A Merge operation is required when a node leaves the cluster while it
has a shadowset mounted.

A Copy operation is initiated when a $MOUNT command is invoked to add a
shadowset member into a shadowset.