Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Interesting NFS bug...

5 views
Skip to first unread message

JF Mezei

unread,
Dec 20, 2009, 1:58:54 AM12/20/09
to
OS-X mounted disk via NFS on VMS (VMS acts as NFS server).

ditto used to copy files tro the OS-X disk.

the last bit of index.html from VMS:

> 22534D56 20796220 64657265 776F5022 "Powered by VMS" 0000C0
> 72742F3C 0005003E 64742F3C 0005003E >...</td>...</tr 0000D0
> 3E64743C 3E72743C 00150000 0000003E >.......<tr><td> 0000E0
> 0000003E 72742F3C 3E64742F 3C3E703C <p></td></tr>... 0000F0
> 2F3C0007 00003E65 6C626174 2F3C0008 ..</table>....</ 000100
> 003E6C6D 74682F3C 0007003E 79646F62 body>...</html>. 000110
> 00000000 00000000 00000000 0000FFFF ................ 000120
> 00000000 00000000 00000000 00000000 ................ 000130
> 00000000 00000000 00000000 00000000 ................ 000140
> 00000000 00000000 00000000 00000000 ................ 000150
> 00000000 00000000 00000000 00000000 ................ 000160
> 00000000 00000000 00000000 00000000 ................ 000170
> 00000000 00000000 00000000 00000000 ................ 000180
> 00000000 00000000 00000000 00000000 ................ 000190
> 00000000 00000000 00000000 00000000 ................ 0001A0
> 00000000 00000000 00000000 00000000 ................ 0001B0
> 00000000 00000000 00000000 00000000 ................ 0001C0
> 00000000 00000000 00000000 00000000 ................ 0001D0
> 00000000 00000000 00000000 00000000 ................ 0001E0
> 00000000 00000000 00000000 00000000 ................ 0001F0


After copy to OS-X:


> 000011c0 74 3d 22 50 6f 77 65 72 65 64 20 62 79 20 56 4d |t="Powered by VM|
> 000011d0 53 22 3e 0a 3c 2f 74 64 3e 0a 3c 2f 74 72 3e 0a |S">.</td>.</tr>.|
> 000011e0 0a 0a 3c 74 72 3e 3c 74 64 3e 3c 70 3e 3c 2f 74 |..<tr><td><p></t|
> 000011f0 64 3e 3c 2f 74 72 3e 0a 0a 3c 2f 74 61 62 6c 65 |d></tr>..</table|
> 00001200 3e 0a 0a 3c 2f 62 6f 64 79 3e 0a 3c 2f 68 74 6d |>..</body>.</htm|
> 00001210 6c 3e 0a 00 3f 00 a0 00 39 00 0b 01 05 00 20 20 |l>..?...9..... |
> 00001220 20 31 35 00 38 00 04 00 3f 00 a0 00 00 40 00 00 | 15.8...?....@..|
> 00001230 08 00 a0 00 38 00 04 00 3f 00 a0 00 00 40 00 00 |....8...?....@..|
> 00001240 3b 00 a0 00 4a 00 08 00 82 00 a0 00 3f 00 a0 00 |;...J.......?...|
> 00001250 6a 00 0b 01 0b 00 32 31 2d 4e 4f 56 2d 32 30 30 |j.....21-NOV-200|
> 00001260 39 00 00 00 38 00 04 00 3f 00 a0 00 00 40 00 00 |9...8...?....@..|
> 00001270 08 00 a0 00 38 00 04 00 3f 00 a0 00 00 40 00 00 |....8...?....@..|
> 00001280 3b 00 a0 00 4a 00 0a 00 82 00 a0 00 3f 00 a0 00 |;...J.......?...|
> 00001290 c5 00 0b 01 14 00 53 4d 54 50 25 22 64 6a 66 30 |......SMTP%"xxxx|
> 000012a0 31 40 62 69 6b 65 6f 64 79 73 00 00 38 00 04 00 |x@xxxxxxxx..8...|
> 000012b0 3f 00 a0 00 00 40 00 00 08 00 a0 00 38 00 04 00 |?....@......8...|
> 000012c0 3f 00 a0 00 00 40 00 00 3b 00 a0 00 4a 00 0c 00 |?....@..;...J...|
> 000012d0 82 00 a0 00 3f 00 a0 00 5f 01 0b 01 1e 00 52 65 |....?..._.....Re|
> 000012e0 3a 20 4c 61 73 74 20 57 65 65 6b 73 20 54 72 61 |: Last Weeks Tra|
> 000012f0 69 6e 20 41 64 76 65 6e 74 75 72 65 38 00 04 00 |in Adventure8...|
> 00001300 3f 00 a0 00 00 40 00 00 08 00 a0 00 82 00 c0 00 |?....@..........|
> 00001310 82 00 c0 00 40 00 c0 00 00 00 00 01 00 00 00 00 |....@...........|
> 00001320


In other words, the NFS server on VMS filled data beyond the end of file
with text from another file. So much for VMS file system security.


Performing the copy from VMS ($COPY on VMS with Unix acting as NFS
server) did not add content to the end of file.

However, one cannot copy a whole directory tree this way. Backup creates
files on the other system with file extensions as part of file name. so
instead of "myimage.jpg" you have "myimage.jpg;3"

Looks like FTP is needed.

Bob Koehler

unread,
Dec 20, 2009, 10:29:33 AM12/20/09
to
In article <00a6591e$0$1587$c3e...@news.astraweb.com>, JF Mezei <jfmezei...@vaxination.ca> writes:
> OS-X mounted disk via NFS on VMS (VMS acts as NFS server).
>
> ditto used to copy files tro the OS-X disk.
>
> the last bit of index.html from VMS:
>
>
> In other words, the NFS server on VMS filled data beyond the end of file
> with text from another file. So much for VMS file system security.

Do you have file hioghwater marking turned on? The NFS server should
honor this, if it does not, then maybe you're running the wrong IP
stack.



>
> However, one cannot copy a whole directory tree this way. Backup creates
> files on the other system with file extensions as part of file name. so
> instead of "myimage.jpg" you have "myimage.jpg;3"

Have you tried copy [source...]*.*;* dest: ?

JF Mezei

unread,
Dec 20, 2009, 1:11:54 PM12/20/09
to
Redid another test.

While the first one had tagged on contents of an email, this time
around, it provided totally different content at the end of the file.

This would indicate to me that it is not reading "beyond the end of
file", but rather not zeroing a 512 byte buffer before writing fewer
than 512 bytes to it and then sending the full 512 bytes to the client.

> 000011a0 74 22 20 20 73 72 63 3d 22 2f 76 6d 73 2f 70 6f |t" src="/vms/po|
> 000011b0 77 65 72 65 64 62 79 32 2e 67 69 66 22 20 61 6c |weredby2.gif" al|


> 000011c0 74 3d 22 50 6f 77 65 72 65 64 20 62 79 20 56 4d |t="Powered by VM|
> 000011d0 53 22 3e 0a 3c 2f 74 64 3e 0a 3c 2f 74 72 3e 0a |S">.</td>.</tr>.|
> 000011e0 0a 0a 3c 74 72 3e 3c 74 64 3e 3c 70 3e 3c 2f 74 |..<tr><td><p></t|
> 000011f0 64 3e 3c 2f 74 72 3e 0a 0a 3c 2f 74 61 62 6c 65 |d></tr>..</table|
> 00001200 3e 0a 0a 3c 2f 62 6f 64 79 3e 0a 3c 2f 68 74 6d |>..</body>.</htm|

> 00001210 6c 3e 0a 6d 4d 69 4d 6e 6d 69 6e 6d 6d 6e 6d 6d |l>.mMiMnminmmnmm|
> 00001220 6d 69 6d 6d 6d 6d 6d 6d 6d 6e 6d 6d 6d 6d 6e 69 |mimmmmmmmnmmmmni|
> 00001230 6d 6d 49 69 4d 8d 6d 91 91 91 8d 6c 6d 6c 6c 6d |mmIiM.m....lmllm|
> 00001240 71 6d 6c 6d 6d 6c 4d 4c 68 6d 91 91 8d 4c 70 71 |qmlmmlMLhm...Lpq|
> 00001250 6d 6d 49 4d 69 49 4d 69 6d 69 6d 4d 69 6d 69 4d |mmIMiIMimimMimiM|
> 00001260 6d 6d 6d 6a 6d 6d 69 6d 6d 6d 69 6d 6d 69 6d 4d |mmmjmmimmmimmimM|
> 00001270 6e 6d 6d 6d 6e 69 6d 69 6d 6d 6e 6d 6d 6d 6d 6d |nmmmnimimmnmmmmm|
> 00001280 6e 6d 69 6d 69 49 6d 8d 8d 71 6c 91 91 91 6d 6d |nmimiIm..ql...mm|
> 00001290 6c 6c 71 6d 6d 4c 6c 6d 6d 6d 6c 6d 6d 6d 4c 6d |llqmmLlmmmlmmmLm|
> 000012a0 6c 49 49 6d 49 6d 69 6d 6d 49 6d 6d 6e 6d 6d 6e |lIImImimmImmnmmn|
> 000012b0 69 4d 69 6d 6d 4d 69 4d 69 6d 4d 69 4d 6d 69 6d |iMimmMiMimMiMmim|
> 000012c0 6d 69 4d 69 6d 6d 6d 6e 6d 6d 6d 6d 6e 6d 69 6d |miMimmmnmmmmnmim|
> 000012d0 69 6d 6d 49 6d 6d 49 69 71 91 8d 8d 91 91 8d 6c |immImmIiq......l|
> 000012e0 6d 4d 6c 6c 6d 6c 4d 4c 6d 6c 48 48 6d 4c 70 6d |mMllmlMLmlHHmLpm|
> 000012f0 4d 69 69 6d 49 6d 69 4d 69 4d 69 6d 6d 6d 49 6d |MiimImiMiMimmmIm|
> 00001300 69 6d 6d 4d 69 6d 69 6d 4d 69 6d 6d 69 4d 69 6e |immMimimMimmiMin|
> 00001310 6d 6d 69 6d 6d 6e 6d 6d 69 6d 6e 6d 6d 6d 6e 6d |mmimmnmmimnmmmnm|
> 00001320

And the plot thickens:

A couple minutes later:

> 00001160 22 20 0a 09 61 6c 74 3d 22 41 6c 70 68 61 20 50 |" ..alt="Alpha P|
> 00001170 6f 77 65 72 65 64 22 3e 0a 3c 2f 74 64 3e 0a 3c |owered">.</td>.<|
> 00001180 74 64 20 77 69 64 74 68 3d 22 32 30 25 22 3e 0a |td width="20%">.|
> 00001190 3c 69 6d 67 20 61 6c 69 67 6e 3d 22 72 69 67 68 |<img align="righ|
> 000011a0 74 22 20 20 73 72 63 3d 22 2f 76 6d 73 2f 70 6f |t" src="/vms/po|
> 000011b0 77 65 72 65 64 62 79 32 2e 67 69 66 22 20 61 6c |weredby2.gif" al|


> 000011c0 74 3d 22 50 6f 77 65 72 65 64 20 62 79 20 56 4d |t="Powered by VM|
> 000011d0 53 22 3e 0a 3c 2f 74 64 3e 0a 3c 2f 74 72 3e 0a |S">.</td>.</tr>.|
> 000011e0 0a 0a 3c 74 72 3e 3c 74 64 3e 3c 70 3e 3c 2f 74 |..<tr><td><p></t|
> 000011f0 64 3e 3c 2f 74 72 3e 0a 0a 3c 2f 74 61 62 6c 65 |d></tr>..</table|
> 00001200 3e 0a 0a 3c 2f 62 6f 64 79 3e 0a 3c 2f 68 74 6d |>..</body>.</htm|

> 00001210 6c 3e 0a |l>.|
> 00001213


Which means that it finally terminated the file properly. Interesting.


And the plot thickens even more...

Redoing the same test now brings back the same pattern as the first one
in this message beyond the end of file.

Michael Moroney

unread,
Dec 20, 2009, 11:23:30 PM12/20/09
to
JF Mezei <jfmezei...@vaxination.ca> writes:

>OS-X mounted disk via NFS on VMS (VMS acts as NFS server).

>ditto used to copy files tro the OS-X disk.

>the last bit of index.html from VMS:

>> 22534D56 20796220 64657265 776F5022 "Powered by VMS" 0000C0

...

I'm going to guess that the VMS disk has erase-on-delete and highwater
marking both turned off.

It was always possible to get old data that happened to reside in the
disk blocks once allocated to another file but not yet written, if these
bits are turned off (or didn't exist for old versions of VMS).

John Gemignani, Jr.

unread,
Dec 25, 2009, 11:52:57 PM12/25/09
to

"JF Mezei" <jfmezei...@vaxination.ca> wrote in message
news:00a6591e$0$1587$c3e...@news.astraweb.com...

Interesting.

A few facts about the VMS NFS server:

1) I believe that we (they, I don't work there anymore) recommend that high
water marking is disabled. This is an XQP feature that causes awful
performance issues for NFS.

2) When a non-stream file is accessed, the file is scanned to:
a) determine the actual length of the file in its stream equivalent form
(as it will be returned on read), and
b) create an offset translation map for on-the-fly conversion to stream
format.

If I were to be debugging this problem, I would be looking for a length
mismatch and perhaps even a confusion between file-handles (they have
embedded options). But then, while I do work on NFS today, I no longer do so
for HP.

John


John Santos

unread,
Jan 2, 2010, 9:08:08 PM1/2/10
to
In article <udCdnVnI6cY2C6jW...@dls.net>, john@nfw-
invalid.cibtrikker.com says...>

Interesting... I know of a bug in *TCPWare's* NFS server where the
workaround is to enable high-water marking. Don't know if this is
fixed in the latest version

>
> 2) When a non-stream file is accessed, the file is scanned to:
> a) determine the actual length of the file in its stream equivalent form
> (as it will be returned on read), and
> b) create an offset translation map for on-the-fly conversion to stream
> format.
>
> If I were to be debugging this problem, I would be looking for a length
> mismatch and perhaps even a confusion between file-handles (they have
> embedded options). But then, while I do work on NFS today, I no longer do so
> for HP.
>
> John

--
John Santos
Evans Griffiths & Hart, Inc.

John Gemignani, Jr.

unread,
Jan 5, 2010, 12:53:50 AM1/5/10
to

"John Santos" <jo...@egh.com> wrote in message
news:MPG.25a9c6173...@news.giganews.com...

That probably addresses Unix sparse files (which VMS did not support).

If you leave it to the XQP, it is quite expensive as it can tie up a thread
to
fill all of that space with nulls using smaller IO. I chose to have the NFS
server zero fill the area itself with massive asynchronous IO.


JF Mezei

unread,
Jan 5, 2010, 4:21:25 AM1/5/10
to
John Gemignani, Jr. wrote:

> If you leave it to the XQP, it is quite expensive as it can tie up a thread
> to
> fill all of that space with nulls using smaller IO. I chose to have the NFS
> server zero fill the area itself with massive asynchronous IO.


Since the redundant data at the end of the block varied depending on
previous NFS transfers, I would tend to think that this is not related
to the VMS file system itself, but rather about buffer management in the
NFS drivers.

0 new messages