Is there a Prodos v2.4.2?

[I am Rob]

Came across a bug in v2.4.1 and want to test in a newer version if there is one.

Rob

[John Brooks]

On Thursday, January 11, 2018 at 8:41:07 AM UTC-8, I am Rob wrote:
> Came across a bug in v2.4.1 and want to test in a newer version if there is one.
>
> Rob

What is the bug?

I have Bitsy Bye fixes for the Appletalk workstation card and Mac LC Apple II card and was planning on releasing them with new features in 2.5 rather than doing a 2.4.2 release.

-JB
@JBrooksBSI

[I am Rob]

I will try to isolate it some more so can give a better account of the bug.

Here is what I have so far.

I converted some images to bitmap and imported to a disk image using CiderPress.

When booted into Prodos v2.4.1, if I load one of these image files, and then immediately try to save a file (can be either BIN or BAS), I get an I/O ERROR.

All other commands work ok. PREFIX, CATALOG, LIST, ETC.


What I have tried so far.

Booted Prodos v2.4.1 with 3 different Basic.systems - same result
Booted Prodos v2.0.3 with any Basic.system - everything ok, can SAVE files.
I believe this rules out Basic.system


If I take the same image file and save it to a Binary file I am able to save files after loading it.


Nothing looks out of the ordinary when doing a GETINFO on this file and info looks exactly the same as the binary file saved except for the name and time.

Filename : /MOVIE/FRAMES/FRAME.0001
Access : $E3
Destroy -1
Rename -1
Backup -1
Compressed -0
Encrypted -0
Invisible -0
Write -1
Read -1
File_Type : $06
Aux_Type : $5000
Stor_Type : $02 Sapling
Blks_used : $001D,29
Filelength: $00363E
Modified : 1-11-18 12:21
Created : 1-10-18 0:56


After further tests, it only happens when I use the directory name in the BLOAD.

1) BLOAD FRAMES/FRAME.0001 -
SAVE TEST - where TEST is an applesoft file gives an error

2) PREFIX FRAMES
BLOAD FRAME.0001
SAVE TEST - works ok and can save a TEST file

3) BLOAD FRAMES/FRAME.0001
BSAVE FRAMES/FRAME.00001,A$5000,L$363E
SAVE TEST - works ok
BLOAD FRAMES/FRAME.0001
DELETE TEST
SAVE TEST - gives error

This last test suggests the error is with the file itself or the use of the directory name with the file, but cannot find anything out of place with the GETINFO of the file.

[I am Rob]

More info.

tested with Applewin and Sweet16, with FRAMES volume in slot 7, drive 2
- same results

Also booted from original Prodos2.4.1.dsk disk - same error

No error when booted from Prodos v2.0.3

[frank_...@hotmail.com]

Hey Rob, do a BSAVE FILENAME,A$5000,L$400 from the prompt before you run your render program to create a preexisting file. Your render program should run fine after that.

I ran into the same thing with 2.4.1. If you create the file beforehand it won't give that error. Not sure if it's because it's using B offset, or because it's switching volumes, but can confirm it works under 2.0.3 and not 2.4.1.

John, here are 2 disk images, and some code to paste into an emulator if you want to recreate it.
http://frankjames.net/downloads/IO%20Error.zip

[I am Rob]

I did do that test as well, that was my 4th test. It is the same as test#3, but with the DELETE TEST line removed and any modification that I made to the TEST file was saved when the TEST file already existed.

[John Brooks]

I was unable to repro the bug using those steps.

Can you try to repro using /RAM or GS /RAM5 and see if it still fails? I'd like to remove the dependency on an existing disk image if possible.

-JB
@JBrooksBSI

[frank_...@hotmail.com]

Try this instead: boot HD to command line, paste this code

100 PRINT CHR$(4)"BLOAD /RAW/FRAMES/FRAME.00001.TIF,T$00,A$2000"
110 PRINT CHR$(4)"BSAVE /HD/TEST,b0,a$400,l$400"




PRODOS BASIC 1.6
COPYRIGHT APPLE 1983-92

]100 PRINT CHR$(4)"BLOAD /RAW/FRAMES/FRA
ME.00001.TIF,T$00,A$2000"
]110 PRINT CHR$(4)"BSAVE /HD/TEST,b0,a$4
00,l$400"
]RUN


I/O ERROR

BREAK IN 110
]

[John Brooks]

Can you reproduce the bug without using an existing HD?

I booted ProDOS 2.4.1 from 5.25 on a KEGS emulator and run this with no error:

CREATE /RAM5/FRAMES
CREATE /RAM5/FRAMES/FRAME.0001,T0
BSAVE /RAM5/FRAMES/FRAME.0001,T0,A$5000,L$363E
100 PRINT CHR$(4)"BLOAD /RAM5/FRAMES/FRAME.0001,T$00,A$2000" 
110 PRINT CHR$(4)"BSAVE /RAM/TEST,b0,a$400,l$400" 
RUN

PRODOS BASIC 1.6
COPYRIGHT APPLE 1983-92

]CREATE /RAM5/FRAMES
]BSAVE /RAM5/FRAMES/FRAME.0001,A$5000,L$
363E
]100 PRINT CHR$(4)"BLOAD /RAM5/FRAMES/FR
AME.0001,A$2000"
]110 PRINT CHR$(4)"BSAVE /RAM/TEST,b0,a$
400,l$400"
]RUN


]CAT /RAM

/RAM

NAME TYPE BLOCKS MODIFIED

TEST BIN 3 11-JAN-18

BLOCKS FREE: 116 BLOCKS USED: 11

]


-JB
@JBrooksBSI

[frank_...@hotmail.com]

No, I have to use the /RAW drive I supplied. I can't reproduce using the 2.4.1 DSK image or copying the "0001" file to /RAM. Tried converting the 2MG with CiderPress, and starting from new image in CiderPress and copying one file. Same issue. Same error in Virtual II, as well as AppleWin. Overwrite PRODOS and BASIC.SYSTEM with 2.0.3 on my /HD and it runs fine.

Again, easy workaround with CREATE. I like to use a special file type so I have to run CREATE anyway. I much prefer using 2.4.1 too, so would rather use the workaround. Just seems weird that BSAVE doesn't work as it 'should'.

As a side note though, plenty of things that 'should' have worked, did not actually work when writing my movie frame rendering code/process (Rob is probably currently running into this). "In theory there is no difference between theory and practice - in practice there is" --Yogi Berra

[John Brooks]

I can reproduce the error in Virtual ][ using your HD image.

ProDOS is returning error $51, bad directory structure. I will have to dig into the dir traversal code to see what's wrong.

While the HD image looks legal at first glance, there are a lot of files in one directory which is slow with ProDOS as directory blocks are walked in a linked-list.

-JB

[John Brooks]

I have found the bug. It is a corner case, but simple to repro and workaround, and shouldn't be hard to fix.

To trigger:
1) Access a file in a subdirectory with 256 or more directory entries (ie files & subdirs)
2) Next access a directory on a different drive
3) Access the root directory of the new drive

The bug is that the dir entry count high byte inside ProDOS is not zeroed when changing from a subdir on one drive to the root dir of a new drive. The root dir of the new drive is then thought to have >255 entries which flags an error $51 corrupt directory.

Until I release a fix, I recommend working around the problem by doing any of the following:

1) Access a subdir on the new drive instead of root

100 PRINT CHR$(4)"BLOAD /RAW/FRAMES/FRAME.00001.TIF,T$00,A$2000"

110 PRINT CHR$(4)"BSAVE /HD/DATA/TEST,b0,a$400,l$400"

2) Have 255 or fewer files per subdirectory

3) Use one 32MB drive per app (not possible for your use-case given the data size)

4) Access a subdir with <255 files on the current drive before changing to the root of the new drive

-JB
@JBrooksBSI

[I am Rob]

I created an empty directory on the same drive (even tried with 1 file in new directory), and I access it after loading a file from the directory with >255 files, but still get I/O error on save.

I realize that the high byte directory count is in the LC ($D000) area of Prodos. You wouldn't happen to have the exact location so I can Poke it to zero?

[John Brooks]

FE8D is the high byte of the dir count.

Try this patch to fix the bug:

E65C:0B BD 1B DC 9D 3E FE CA D0

-JB
@JBrooksBSI

[I am Rob]

Yes, that works. But curious as to how the high byte was stored in the first place as your code is a one-off where the high byte never gets stored? So one would think that, that byte always stays zero, so the error should be when the directory with >255 files is accessed, not the other way around.

Thanks for your version of Prodos 8.

Do you have a list of changes made to Basic.system (v1.6). I have also made quite a bit of improvements, mostly for speed. After I compare what you did, I could send you some more modifications if interested in doing a v.1.6.1.

Rob

[John Brooks]

The bug was that the routine needed to exit with X=0, but if the routine was accessing a root directory, it would bypass the subdir copy loop (which exited with X=0), and instead return X=$FF.

One of the caller routines expected X=0 so it could INX and then copy the word dir-entry-count to a ProDOS internal var. When the returned X was $FF, it would INX to $00 and copy only the low byte, leaving the high byte unchanged and inherited from the previously-accessed drive.

> Do you have a list of changes made to Basic.system (v1.6). I have also made quite a bit of improvements, mostly for speed. After I compare what you did, I could send you some more modifications if interested in doing a v.1.6.1.

Basic 1.6 has one code change made to quit back to ProDOS on Integer ROM machines instead of hanging, and the error message text was also changed. These changes did not alter code size or location for the bulk of basic.system, so you can just do a bin diff between 1.5 & 1.6.

I can merge in your improvements for ProDOS 2.5 which has more significant Basic.system changes. However, as that is still a ways out, you might want to release an optimized basic 1.7 earlier.

The two areas I've been working on regarding basic.system are:

1) Supporting new ProDOS features
2) Supporting Beagle Basic

-JB
@JBrooksBSI

[Michael J. Mahon]

<frank_...@hotmail.com> wrote:
> Try this instead: boot HD to command line, paste this code
>
> 100 PRINT CHR$(4)"BLOAD /RAW/FRAMES/FRAME.00001.TIF,T$00,A$2000"
> 110 PRINT CHR$(4)"BSAVE /HD/TEST,b0,a$400,l$400"
>

In my experience, BSAVE to an existing file using a B parameter results in
the non-overwritten bytes of the existing file being preserved. Omitting
the B parameter causes the existing file to be replaced by the new file.

I would expect an error if a BSAVE with a B parameter specified a
non-existent file. Without a B parameter, of course a new file is created
in any case.



--
-michael - NadaNet 3.1 and AppleCrate II: http://michaeljmahon.com

[I am Rob]

On Friday, January 12, 2018 at 2:25:02 PM UTC-6, Michael J. Mahon wrote:

> <frank> wrote:
> > Try this instead: boot HD to command line, paste this code
> >
> > 100 PRINT CHR$(4)"BLOAD /RAW/FRAMES/FRAME.00001.TIF,T$00,A$2000"
> > 110 PRINT CHR$(4)"BSAVE /HD/TEST,b0,a$400,l$400"
> >
>
> In my experience, BSAVE to an existing file using a B parameter results in
> the non-overwritten bytes of the existing file being preserved. Omitting
> the B parameter causes the existing file to be replaced by the new file.
>
> I would expect an error if a BSAVE with a B parameter specified a
> non-existent file. Without a B parameter, of course a new file is created
> in any case.
>

If that was the original case, it is no longer the case with my Basic.system.

I removed the use of having to use the CREATE file type command.

I now just type BSAVE filename,T$(hex#),A$(hex#),L$(hex#) and the saved file type is automatically implemented

I can also create empty files with the B parameter with no problem.

BSAVE filename,A$(hex#),L0,B0

[Michael J. Mahon]

That behavior would create problems for my application. I would probably
need to check for the file's existence, then DELETE it prior to the first
BSAVE. (The alternative, BSAVE, DELETE...BSAVE seems pretty gross, but
probably runs about as fast (slow?).

[Michael J. Mahon]

Actually, perhaps not.

What is the behavior when the file preexists and the B parameter is used,
but with non-zero length? (That's the only case I use.)

[James Davis]

JB,

When and where will you be posting an updated/fixed ProDOS-8 2.4.1 disk image?

James Davis

[John Brooks]

If testing goes well, in about a week. I will post it to callapple.org.

-JB
@JBrooksBSI

[frank_...@hotmail.com]

Thank you for all your hard work on ProDOS!

[frank_...@hotmail.com]

I'm still unable to BSAVE to root with the patch installed, unless creating the file beforehand. Should I be able to? Using the IO_Error.zip files I posted.

Frank

[I am Rob]

On Friday, January 12, 2018 at 7:39:03 PM UTC-6, Michael J. Mahon wrote:
> Michael J. Mahon wrote:

If a file exists, then using the B0 parameter will overwrite the file starting at the beginning of the file (same as if the B parameter was not present), or append to the existing file starting at the address specified by the B parameter.

[I am Rob]

When adding the patch directly to memory, also make sure the byte at $FE8D has also been zeroed, otherwise you will still get the I/O error.

[Michael J. Mahon]

But the general behavior should be that it replaces/adds data to an
existing file, but only L bytes. This may extend the file, but should never
truncate it. I agree that L equals 0 is a special case which I have never
found useful, and could, somewhat surprisingly, create a new empty file.

Without the B parameter, a new file is created, only as long as the L
parameter, so the rest of the preexisting file is effectively truncated.

[I am Rob]

> This may extend the file, but should never
> truncate it. I agree that L equals 0 is a special case which I have never
> found useful, and could, somewhat surprisingly, create a new empty file.
>
> Without the B parameter, a new file is created, only as long as the L
> parameter, so the rest of the preexisting file is effectively truncated.

I agree. Using the L and B parameter both set to zero is the same as using the CREATE command and really has no other use, and I even find the CREATE command un-useful, so got rid of it.

After all, if you are going to load/save something, you might as well set the length of data to load or save.

[I am Rob]

Instructions to fix Prodos v2.4.1

At the prompt type these in, or copy then use shift-insert, to paste in Applewin:


BLOAD PRODOS,TSYS,A$2000
CALL -151
385C:0B BD 1B DC 9D 3E FE CA D0
3D0G
BSAVE PRODOS,TSYS,A$2000
- PRODOS

[Oliver Schmidt]

Hi John,

>I have Bitsy Bye fixes for the Appletalk workstation card and Mac LC Apple II card and was planning on releasing them with new features in 2.5 rather than doing a 2.4.2 release.

It's a pity that you haven't open sourced Bitsy Bye by now for at
least two reasons:

1. The fixes you mention above would visible as explicit commits which
I'm pretty sure would be interesting to others (aka me) too.

2. It would allow others (aka me) to (try to) create variants of the
Bitsy Bye screen layout making that probably compatible with the taste
of a larger number of users.

Regards,
Oliver

[John Brooks]

Hi Oliver.

I intend to open source Bitsy Bye along with several new drivers as part of ProDOS 2.5.

The issue is that Bitsy Bye 2.4 is parsing directories in a way that is incompatible with ProDOS 2.5, so I want to hold off sharing that version.

Once the new 2.5 system is in place, it will be ready for sharing, customizing, and migrating into other apps.

-JB

[cb meeks]

Sorry if I'm late to the party here....but being a Commodore man for so many years, many of Apple software packages are foreign to me.

Anyway...my assumption (and I think I read it somewhere) is that Apple ProDOS is now open source and that some people (John Brooks?) took that code and released a new version this past year or so.

Please, correct me if I'm wrong on that...

Now, would someone mind explaining what Bitsy Bye is and how it plays into ProDOS? I've seen it mentioned several times but I'm jut not getting it.

Thanks for any clarification...

[John Brooks]

ProDOS has not yet been open sourced by Apple, but there are several disassemblies available (v1.7 & v2.0.3).

I have put together a build pipeline that allows me to modify ProDOS and for the last couple years I have been making improvements.

A major new feature in the 2.4 release was replacing the OS ram-resident program launcher with a new version called Bitsy Bye. Bitsy Bye is an interactive, menu-based system for choosing which program to run next after the previous program exits with the quit or bye command. See screenshots and description here:

http://www.callapple.org/uncategorized/announcing-prodos-2-4-for-all-apple-ii-computers/

For a demo of Bitsy Bye in use, see this Assembly Lines #41 video:

https://youtu.be/-_6P_a5C4Z0?t=9m53s

-JB
@JBrooksBSI

[Oliver Schmidt]

Hi John,

>I intend to open source Bitsy Bye along with several new drivers as part of ProDOS 2.5.

Cool :-)

>The issue is that Bitsy Bye 2.4 is parsing directories in a way that is incompatible with ProDOS 2.5, so I want to hold off sharing that version.

I see, although - as mentioned in PM - I personally would prefer to
see the 2.4 version released and the changes you made/make for 2.5
made visible as changes so that the history becomes available too.

That could even become a positive thing for you personally. I imagine
people having written software that's incompatible with 2.5 asking you
what they need to change. Then you could - beside giving a concpetual
answer - point them to your BB change as an example what needs to be
done. Coding more as a social process than a purely result driven
me-myself-and-I thing ;-)

>Once the new 2.5 system is in place, it will be ready for sharing, customizing, and migrating into other apps.

Looking forward to it, although I'd have loved to be able to
contribute to 2.5 instead to post-2.5 ...

Regards,
Oliver

[John Brooks]

On Friday, January 12, 2018 at 7:00:02 PM UTC-8, James Davis wrote:

The ProDOS 2.4.2 announcement and disk image are here:

http://www.callapple.org/vintage-apple-computers/apple-ii/announcing-prodos-2-4-2-for-all-apple-ii-computers/

-JB
@JBrooksBSI

[James Davis]

Thanks John !!!