Ditto report about compacted tapes

[Kris Buelens]

We are estimating how much time it would take to transfer a backup that would be taken on virtual tapes to a DRP center.

(we'd probably backup to a virtual tape on a local Wintel server, then we'd send that backup Wintel-to-Wintel to the DRP site)

I performed a DITTO tape map to see how much data are stored on the 3592/E06 backup cassette (the backup was taken by FAVOR)

Ditto tells me
  Total data mapped: 107630013914 bytes, 1642339 blocks, 16.33814% of  media capacity
  Note: Compressed data format present, compression ratio is  1:5.1  

So,  +/- 107GB.

The question is: what does Ditto report?  Does it report how much data is stored after compaction? With other words: would it without compaction be +/- 500GB?

Kris Buelens,
     --- VM/VSE consultant, Belgium ---
-----------------------------------------------------------------------

[Duerbusch, Tom]

Ditto doesn't know.

Ditto is doing reads from the controller.

The controller is reading the compressed data from the drive and unpacking it.

And the data comes to Ditto, uncompacted.

So, there is 107,630,013,914 bytes  (107 GB of data).

If you are sending the data via the mainframe, then you are transmitting 107 GB of data.

However, assuming you have a hardware based virtual tape system, if you go thru the back end, and send the compressed file, then you would be sending a fifth of it (1:5 compression).

I think this is also the same if you are using VSE Virtual Tape (remote vs VSAM), as you can compress remote files.

Tom Duerbusch

THD Consulting

_______________________________________________
VSE-L mailing list
VS...@lists.lehigh.edu
https://lists.lehigh.edu/mailman/listinfo/vse-l

--

 

[Kris Buelens]

Yes indeed, the controller is unpacking the data, and Ditto can count the bytes it gets, after unpacking thus.

But, how does Ditto know the data are compressed?  The controller must tell it. 
Similar: how can Ditto know the compression ratio? Again: the controller must tell it, I see no other way.
So, as the controller seems to report 'things' about compression, it could also tell how much space is occupied by the compressed data.

Consequently: Ditto could report the compressed size (if the the controller reports it), or Ditto can report the decompressed size.  And I'd like to know what is reported.

We're not thinking of using a VTS, but the Java based virtual tapes in VSE.  There ZIP format is supported, so we'd gain space too, for simplicity, I guess the ZIP compression is the same as the compression on 3592 1:5.1

Step 1

  DASD -> vTape -> Wintel -> zipped_on_local_disk

Step2 (using FTP for example)
  zipped_on_local_disk -> network -> zipped_on_DRP_site_disk

Step3: restore if DRP-Test or real disaster

  zipped_on_DRP_site_disk -> vTape -> DASD

The question is: do we need to transfer +/- 107GB or 107/5=+/-20GB?

Kris Buelens,

     --- VM/VSE consultant, Belgium ---
-----------------------------------------------------------------------

[Kris Buelens]

The way to find the answer was so simple, should have thought immediately to execute

   PIPE TAPE|COUNT Bytes|Cons

and the answer was 107630013514   same as what Ditto reports

Conclusion: Ditto's tape map reports the uncompressed size.

Kris Buelens,
     --- VM/VSE consultant, Belgium ---
-----------------------------------------------------------------------

[Kris Buelens]

Thanks a lot for your information.

I already told teh customer that I used these virtual tape support in VSE, but just to install fixes or a new release, not for backups.  It is one of the 3 ways we see now: sending cassettes by airplane, Vtape, PPRC on DS8K.  The last is the best, but has its price...

So, but your experience shows VTape is not the best thing for production.  We've a confcall with the customer next Monday..

Kris Buelens,
     --- VM/VSE consultant, Belgium ---
-----------------------------------------------------------------------

2017-01-21 16:29 GMT+01:00 Kevin Corkery <kcor...@live.com>:

Kris …

 

You can let the Java Based VSE VTape server compress the file om the fly.  This is done by giving the .ZAWS suffix to your target LAN-based file.  This will be reversed going into VSE.  I’ve been seeing about a 1:5 compression using this method.  Also, my experience is that the reliance on TCP/IP for virtual tape is a real limiting factor; it’s really slow going to/from a WinTel platform.  It’s good enough for lightweight use but can limit its usefulness for production.  We use a Universal Software Virtual Tape Appliance which runs at ESCON speeds to produce virtual tapes using the same compression facility as the VSE VTape server.  There are some products available that use FICON.  Most vendors claim VSE VTape compression compatibility but I’ve only verified the VTA personally.  The site-to-site transfer is most dependent on the network infrastructure that is used.  At one time we had a Riverbed device installed that tripled the throughput, for instance.  Some of the virtual tape vendors have imbedded  capabilities that, much like the Riverbed device, “De-Dup” data on the fly.  In our case our remote is just a mount point on a server 1000 miles away and we use Microsoft’s Robocopy to sync the directories containing our tapes.  This automatically handles new objects whenever they’re created ; no need to code another script  on a file-by-file basis.

 

 

Kevin P Corkery

Independent Consultant

Voorhees, New Jersey

 

 

 

From: VSE-L [mailto:vse-l-bounces+kcorkery=live...@lists.lehigh.edu] On Behalf Of Kris Buelens
Sent: Saturday, January 21, 2017 2:26 AM
To: VSE Discussion List
Subject: Re: Ditto report about compacted tapes

 

Yes indeed, the controller is unpacking the data, and Ditto can count the bytes it gets, after unpacking thus.

But, how does Ditto know the data are compressed?  The controller must tell it. 
Similar: how can Ditto know the compression ratio? Again: the controller must tell it, I see no other way.
So, as the controller seems to report 'things' about compression, it could also tell how much space is occupied by the compressed data.

Consequently: Ditto could report the compressed size (if the the controller reports it), or Ditto can report the decompressed size.  And I'd like to know what is reported.

We're not thinking of using a VTS, but the Java based virtual tapes in VSE.  There ZIP format is supported, so we'd gain space too, for simplicity, I guess the ZIP compression is the same as the compression on 3592 1:5.1

Step 1

  DASD -> vTape -> Wintel -> zipped_on_local_disk

Step2 (using FTP for example)
  zipped_on_local_disk -> network -> zipped_on_DRP_site_disk

Step3: restore if DRP-Test or real disaster

  zipped_on_DRP_site_disk -> vTape -> DASD

The question is: do we need to transfer +/- 107GB or 107/5=+/-20GB?

 

Kris Buelens,

     --- VM/VSE consultant, Belgium ---

 

[Kris Buelens]

Hi Kevin,

Do you know if VTape has size restrictions when backup to Wintel (I though about this when you mention the 4GB limit of VTape to VSAM)

I read about VTA and see that they emulate 3490's.  Does that mean the virtual cassettes are "small" as well. That is: if the current backup is 107GB (20 20GB after compression) it would need many virtual cassettes and some mechanism to mount a "next cassette".

Kris Buelens,
     --- VM/VSE consultant, Belgium ---
-----------------------------------------------------------------------

2017-01-21 18:11 GMT+01:00 Kevin Corkery <kcor...@live.com>:

Kris …

 

There’s many ways to “skin the cat” on virtual tape.  At one time we used to take all our production backups to VSAM VTape and then back those files up to a SDLT tape (this was on a Flex-ES system).  When the Flex went away we did the same but to 3590 carts.  This left an onsite backup and a offsite backup on 1-2 physical tapes.  The advantage was the VSAM VTape is really fast but is limited to 4GB per tape.  In the meantime we also started experimenting with asynchronously copying the VSAM Vtapes to LAN-Based using the VSE native support.  Although slow it did work and we had lots of capacity available in the late evening/early morning timeframes.  Ultimately, we went with the VTA and completely abandoned real tapes.

 

Another approach may be to use a Linux guest and use Linux Fast Path to process the tapes.  This would be much faster than TCP/IP on the real network.  Off course, there would need to be some mapping of the drives where the eventual data is to be stored.  Others on the list may be more familiar with this facility.

 

,,, Kevin

 

 

From: VSE-L [mailto:vse-l-bounces+kcorkery=live...@lists.lehigh.edu] On Behalf Of Kris Buelens
Sent: Saturday, January 21, 2017 11:42 AM
To: VSE Discussion List
Subject: Re: Ditto report about compacted tapes

 

Thanks a lot for your information.

I already told teh customer that I used these virtual tape support in VSE, but just to install fixes or a new release, not for backups.  It is one of the 3 ways we see now: sending cassettes by airplane, Vtape, PPRC on DS8K.  The last is the best, but has its price...

So, but your experience shows VTape is not the best thing for production.  We've a confcall with the customer next Monday..

Kris Buelens,
     --- VM/VSE consultant, Belgium ---

 

[Duerbusch, Tom]

The restriction on size of a remote VSE VTAPE file, is the size restriction of the destination file system.

Tom Duerbusch

THD Consulting

--