Fun with Files-11 ODS-1

[Peter Long]

I've been having some fun with Files-11 ODS-1 

Basically I wanted to use the graphical tools on the Mac / Ubuntu to better explore the various RSX-11 disks I have running and to this end I used the  Filesystem in Userspace (FUSE) library and some C code to instantiate the RSX ODS-1 file system on my Mac and likewise on my Ubuntu machine.

It's still a work in progress - next is to interpret the FCS structure to better read the files, but for now, the files & directories are all there and some minor cleanups with dates, ownership and some performance tweaks.

I now way more about Files-11 ODS-1 that I even though I would !!

Cheers

Peter

[Johnny Billquist]

Very cool!

However, I would suggest you deal with directories without the .DIR
extension. Basically, making it look and behave more like under RSX and VMS.

A directory like [FOO] is FOO.DIR in the MFD. And of course, this can
continue with something like [FOO.BAR], which would then be BAR.DIR in
[FOO], and so on.

But maybe it's tricky to do this nicely under other OSes. Not sure about
that bit.

Johnny

> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pidp-11+u...@googlegroups.com
> <mailto:pidp-11+u...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pidp-11/628bff37.75e.8e353700.45e3%40insys.com.au
> <https://groups.google.com/d/msgid/pidp-11/628bff37.75e.8e353700.45e3%40insys.com.au?utm_medium=email&utm_source=footer>.

--
Johnny Billquist || "I'm on a bus
|| on a psychedelic trip
email: b...@softjar.se || Reading murder books
pdp is alive! || tryin' to stay hip" - B. Idol

[Peter Long]

Johnny

Thanks !!

Nice idea - easy enough to do as the file is separately flagged as to
whether it's a directory or regular file, so it would be just a case
of dropping the .DIR from the name, so then "cd IPLOG" would work


fwiw my understanding is that ODS-1 didn't support hierarchical
directory structures - that came in ODS-2

Work in progress ;-)

Cheers

Peter

> an email to pidp-11+u...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pidp-11/2087da60-51e9-aa65-17b7-fa336f793d94%40softjar.se.

[Johnny Billquist]

If you want to be really proper about it, it's supposed to be .DIR;1 even.

And actually, ODS-1 supposed hierarchical directory structures just
fine. It's just that the tools and libraries in RSX didn't. But have
have more or less added that on my development systems, and it works fine:

.set /host
Host=JOCKE RSX-11M-PLUS V5.0 BL88
.set /def
DU0:[BQT] Named TT15:
Protection UIC: [7,14]
.dir [.foo]


Directory DU0:[BQT.FOO]
24-MAY-22 01:54

TEST.TXT;1 1. 01-NOV-12 03:17
BAR.DIR;1 1. C 11-NOV-12 01:38

Total of 2./2. blocks in 2. files

.dir [.foo.bar]


Directory DU0:[BQT.FOO.BAR]
24-MAY-22 01:54

KAKA.DIR;1 1. C 11-NOV-12 01:38

Total of 1./1. blocks in 1. file

.set /def=[.foo.bar]
.dir [-]


Directory DU0:[BQT.FOO]
24-MAY-22 01:55

TEST.TXT;1 1. 01-NOV-12 03:17
BAR.DIR;1 1. C 11-NOV-12 01:38

Total of 2./2. blocks in 2. files

.


By the way, have you implemented support for anything else than 3+1
retrieval pointers? I haven't come around to testing this yet, but I
think 4+2 should also work in the latest version of M+.

Also, I guess you will have to dive into the file metadata a bit more
eventually, as well as figure out how you are going to deal with file
attributes. This might become tricky.

Johnny

[Peter Long]

Johnny

My reference was a DEC document "Files-11 On-Disk Structure
Specification" originally dated June 1975 and revised and edited
through to 1986

So I just supported the 3+1 retrieval pointers as other formats of 2+2
and 4+2 are described in the document with the comment "they will
never be supported" - lol

Similarly, the spec allows for hierarchical directory structures,
however the comment is "current implementations only implement a two
level directory hierarchy" - good to know that you've added support
for more that two levels to your system and it works well.

But yes, file attributes are next before anything else ;-)

> https://groups.google.com/d/msgid/pidp-11/31dacfd1-4687-0ed5-2121-30b32d6f4b67%40softjar.se.

[Johnny Billquist]

From the ODS-1 point of view, there is nothing changed if you add
deeper directory structures. It's already two levels in the system
normally. Adding more just means you need to loop through the directory
searching for each level down, instead of just first searching the MFD
and then the UFD a single time.

A directory file just contains the mapping between names and file ids
(as you probably know). How would you ever tell how deep you might be in
a directory structure? Basically, ODS itself puts no limits in here.
It's all user level code in RSX, actually.

If you have it revised until 1986, then I think you should already have
the multiheader files and structure level 3, which allows for up to 8G
disks with 65500 files with 3+1 retrieval pointers, right?

Let me know otherwise. I can't remember if there was any revision to the
ODS-1 written later, but if there was I should have it somewhere. But
those things mentioned were probably the last things added.

2+2 have never been implemented that I know of, and I see no reason
anyone would ever be interested in it.


The file attributes part is where I always gets headaches. This is so
different from unix like systems. Also a reason why I haven't even tried
working on NFS for RSX. Always the question on what size is a file in
bytes, how do you seek to some specific byte, how do you tell where you
actually are, how do you deal with implied CRLF records, and all the
other fun stuff...

Johnny

[Peter Long]

Johnny

A little more progress - command line options to lowercase the file/directory names and likewise drop the ".DIR" from directories - [I'm ignoring versions for the moment] 

The fun part of parsing the FCS record formats half done...

Onwards !!

Peter

 

 

To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-11/3cf0697d-d509-aed3-258b-27467cd1f3ff%40softjar.se.

[Johnny Billquist]

Cool.

My really curious question comes to how will you deal with things like
fseek on a variable length file with implied CRLF?

Johnny

On 2022-05-27 08:58, 'Peter Long' via [PiDP-11] wrote:
> Johnny
>

> A little more progress - command line options to lowercase the
> file/directory names and likewise drop the ".DIR" from directories -
> [I'm ignoring versions for the moment]
>
> The fun part of parsing the FCS record formats half done...
>
> Onwards !!
>
>
> Peter
>

>> --- Original message ---
>> *Subject:* Re: [PiDP-11] Fun with Files-11 ODS-1
>> *From:* Johnny Billquist <b...@softjar.se>
>> *To:* <pid...@googlegroups.com>
>> *Date:* Tuesday, 24/05/2022 10:36 AM

>>>>>>> <https://groups.google.com/d/msgid/pidp-11/628bff37.75e.8e353700.45e3%40insys.com.au?utm_medium=email&utm_source=footer

>>>>>>> <https://groups.google.com/d/msgid/pidp-11/628bff37.75e.8e353700.45e3%40insys.com.au?utm_medium=email&utm_source=footer>>.
>>>>>>>
>>>>>>
>>>>>> --  Johnny Billquist                  || "I'm on a bus
>>>>>> ||  on a psychedelic trip
>>>>>> email: b...@softjar.se             ||  Reading murder books
>>>>>> pdp is alive!                     ||  tryin' to stay hip" - B. Idol
>>>>>>
>>>>>> --  You received this message because you are subscribed to the
>>>>>> Google  Groups "[PiDP-11]" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send  an email to pidp-11+u...@googlegroups.com.
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/pidp-11/2087da60-51e9-aa65-17b7-fa336f793d94%40softjar.se

>>>>>> <https://groups.google.com/d/msgid/pidp-11/2087da60-51e9-aa65-17b7-fa336f793d94%40softjar.se>.

>>>>>>
>>>>>
>>>> -- Johnny Billquist                  || "I'm on a bus
>>>> ||  on a psychedelic trip
>>>> email: b...@softjar.se             ||  Reading murder books
>>>> pdp is alive!                     ||  tryin' to stay hip" - B. Idol
>>>>
>>>> -- You received this message because you are subscribed to the
>>>> Google Groups "[PiDP-11]" group.
>>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to pidp-11+u...@googlegroups.com.
>>>> To view this discussion on the web visit

>>>> https://groups.google.com/d/msgid/pidp-11/31dacfd1-4687-0ed5-2121-30b32d6f4b67%40softjar.se
>>>> <https://groups.google.com/d/msgid/pidp-11/31dacfd1-4687-0ed5-2121-30b32d6f4b67%40softjar.se>.

>>>>
>>
>> --
>> Johnny Billquist || "I'm on a bus
>>                                     || on a psychedelic trip
>> email: b...@softjar.se || Reading murder books
>> pdp is alive! || tryin' to stay hip" - B. Idol
>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "[PiDP-11]" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to pidp-11+u...@googlegroups.com.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/pidp-11/3cf0697d-d509-aed3-258b-27467cd1f3ff%40softjar.se
>> <https://groups.google.com/d/msgid/pidp-11/3cf0697d-d509-aed3-258b-27467cd1f3ff%40softjar.se>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "[PiDP-11]" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pidp-11+u...@googlegroups.com
> <mailto:pidp-11+u...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pidp-11/629076ae.75e.8e8de700.7cb%40insys.com.au
> <https://groups.google.com/d/msgid/pidp-11/629076ae.75e.8e8de700.7cb%40insys.com.au?utm_medium=email&utm_source=footer>.

[Peter Long]

As you’re hinting at, it’s probably not feasible but for what l’m looking to do that shouldn’t be a problem

If need be, I can selectively copy whole files to a native file system that actually supports fseek 😉

The FUSE abstraction is very linux/Unix like and even presenting the variable length implied CR structured files to it is a bit ugly 😳

But upside is I’m finding all kinds of cool stuff on disks which was my goal anyway

Peter

Sent from my iPhone

> On 27 May 2022, at 7:03 pm, Johnny Billquist <b...@softjar.se> wrote:
>
> Cool.

> To view this discussion on the web visit https://groups.google.com/d/msgid/pidp-11/e83f1416-dd6a-2dc8-ae5e-e65c0f7d5d24%40softjar.se.

[Johnny Billquist]

On 2022-05-27 12:25, 'Peter Long' via [PiDP-11] wrote:
> As you’re hinting at, it’s probably not feasible but for what l’m looking to do that shouldn’t be a problem

I'm sortof amazed you are even getting it somewhat working this far.
But I bet there are a number of internal inconsistencies happening. But
you probably aren't too concerned about them at this time. :)

> If need be, I can selectively copy whole files to a native file system that actually supports fseek 😉

That might be the trick, yes.

> The FUSE abstraction is very linux/Unix like and even presenting the variable length implied CR structured files to it is a bit ugly 😳

To say the least. You need to hide the record length, the potential
padding, and you need to add the LF (and maybe CR, depending on how you
want to do things). Meaning there is no easy mapping between the data
actually on the disk, and the data presented to FUSE. So reading or
writing a block, might end up with multiple blocks on the underlying
disk, and all kind of weird things like what offset are you actually on
now, where is the file eof, how large is the file, and god knows what
else...

> But upside is I’m finding all kinds of cool stuff on disks which was my goal anyway

Hey. That you could do directly under RSX... ;-)

Johnny