Correcting LBA and --format

[Adam Billyard]

Happy New Year all.

Recently returned to trying to get a 40MB Tektronix image working - with the help of David I got a 20MB disk image that has given no problems but it is only 20MB and space is getting tight.

So after *much* trial and error I have a 40MB disk image that boots - but does still occasionally have read/write errors and then tears itself to bit wanting to delete corrupt files etc.

mfm_util --analyze throws up some LBA errors.  Looking at mfm_decoder.c I'm unsure whether those messages mean "its wrong", or "its wrong and I've fixed it".  David?

Also, it shows the format as Adaptec but Adaptec is an unsupported argument to --format..  Bit confused.

Adam

--------------------------  output below --------------------

./mfm_util --analyze --ext filename --emu ../emufile_40mb
Original decode arguments: --heads 6 --cylinders 830  --rate 10000000
./mfm_util: unrecognized option '--rate'
Matches count 36 for controller Adaptec
Header CRC: Polynomial 0x41044185 length 32 initial value 0x0
Sector length 512
Data CRC: Polynomial 0x41044185 length 32 initial value 0x0
Number of heads 6 number of sectors 18 first sector 0
Interleave (not checked): 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Command line to read disk:
--format Adaptec --sectors 18,0 --heads 6 --cylinders 830 --header_crc 0x0,0x41044185,32,6 --data_crc  0x0,0x41044185,32,6 --sector_length 512 --retries 50,4 --drive 0
Unknown header flag byte 24 on cyl 0 head 0 physical sector 1                  
Unknown header flag byte 3c on cyl 0 head 0 physical sector 2
Unknown header flag byte 26 on cyl 0 head 0 physical sector 3
Unknown header flag byte 3e on cyl 0 head 0 physical sector 4
Unknown header flag byte 26 on cyl 0 head 0 physical sector 5
Unknown header flag byte 3e on cyl 0 head 0 physical sector 6
Unknown header flag byte 06 on cyl 0 head 0 physical sector 7
Bad sectors on cylinder 336 head 1 LBA: 36339                                  
Bad sectors on cylinder 337 head 4 LBA: 36455
Bad sectors on cylinder 339 head 2 LBA: 36692
Bad sectors on cylinder 516 head 1 LBA: 55779                                  
Bad sectors on cylinder 517 head 4 LBA: 55895
Bad sectors on cylinder 697 head 1 LBA: 75327                                  
Bad sectors on cylinder 698 head 4 LBA: 75443
Bad sectors on cylinder 700 head 2 LBA: 75680  

[Al Kossow]

On 1/6/24 1:51 AM, 'Adam Billyard' via MFM Discuss wrote:
> Happy New Year all.
>

> Recently returned to trying to get a *40MB* Tektronix image working - with the help of David I got a 20MB disk image that has given no
> problems but it is /only/ 20MB and space is getting tight.

>
> So after *much* trial and error I have a 40MB disk image that boots - but does still occasionally have read/write errors and then tears
> itself to bit wanting to delete corrupt files etc.

It would be nice to get pictures of the boards in the mass storage box.
As far as I know, the MFM bridge is a standard Adaptec 4000A

[Al Kossow]

This may be the problem. I don't remember how you tell the 4000 the geometry of the drive.
If you have to do this at SCSI initalization or if it is stored on the drive itself.

[Adam Billyard]

Will have to disassemble and take photos - the HD controller is under the floppy controller

Adam

[David Gesswein]

With Al's question I looked at the manual. At format time the information
is stored in the headers of cylinder 0. See 2-4 of
http://www.bitsavers.org/pdf/adaptec/ACB-4000/400003-00A_ACB-4000A_Users_Manual_Oct85.pdf

That's likely the Unknown header flag byte messages being printed. Their is a
06 which could be heads. 830 = 0x33e. Their is 0x3e but not 0x3 unless its
combined with some other value.

The other messages Bad sectors on cylinder is the sector CRC doesn't match
the data. If it was fixable with ECC it will print a message saying that so
these were not corrected. Assuming that was all created on the emulator those
aren't good.
Reading from real disks bad sectors are expected but if image created
with emulator you shouldn't get them.

Do the number of these grow as you use the disk? Not sure why larger disk
would cause the emulator to act different but also not sure why the
controller would corrupt sectors with a larger disk. It will take some thought
on how to troubleshoot. One way may be to create a large file and leave it
writing a known pattern and reading it back to check. If that fails since we
know what it was trying to write seeing what did get written may help determine
what is going wrong.

Not sure what you mean by Adaptec is an unsupported argument to --format.

Always good to give the command your trying and the output from the command
when reporting problems.

On Sat, Jan 06, 2024 at 01:51:40AM -0800, 'Adam Billyard' via MFM Discuss wrote:
> Happy New Year all.
>

> Recently returned to trying to get a *40MB* Tektronix image working - with

> the help of David I got a 20MB disk image that has given no problems but it

> is *only* 20MB and space is getting tight.

>
> So after *much* trial and error I have a 40MB disk image that boots - but
> does still occasionally have read/write errors and then tears itself to bit
> wanting to delete corrupt files etc.
>

> *mfm_util --analyze* throws up some LBA errors. Looking at *mfm_decoder*.c

> I'm unsure whether those messages mean "its wrong", or "its wrong and I've
> fixed it". David?
>

> Also, it shows the format as *Adaptec* but Adaptec is an unsupported

> --
> You received this message because you are subscribed to the Google Groups "MFM Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mfm-discuss...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/af26dbc5-9bca-45b7-8181-f08f22d97c3bn%40googlegroups.com.

[Adam Billyard]

On Saturday 6 January 2024 at 17:57:37 UTC d...@pdp8online.com wrote:

With Al's question I looked at the manual. At format time the information
is stored in the headers of cylinder 0. See 2-4 of
http://www.bitsavers.org/pdf/adaptec/ACB-4000/400003-00A_ACB-4000A_Users_Manual_Oct85.pdf

That's likely the Unknown header flag byte messages being printed. Their is a
06 which could be heads. 830 = 0x33e. Their is 0x3e but not 0x3 unless its
combined with some other value.

The other messages Bad sectors on cylinder is the sector CRC doesn't match
the data. If it was fixable with ECC it will print a message saying that so
these were not corrected. Assuming that was all created on the emulator those
aren't good.
Reading from real disks bad sectors are expected but if image created
with emulator you shouldn't get them.

But on tek4404 you have to use the formatw command to low level format before starting.  Perhaps its screws up some data?

(I notice the sysreformat floppy come with a unique badblocks file for the Micropolis HD it is associated with - slightly insane and begging for catastrophe back in the day..)

Trying to find a format that the 4404 accepts has proved frustrating.  Endless "Physical format: Controller error" when formatw starts.

In the end I took the known good 20MB image using 3 heads, and modified your copy_emu.c prog to sling data from heads 0-2 to heads 3-5 to at least get a file of the right size and that formatw didn't reject with a Controller error.    As I say, messy..

 

Do the number of these grow as you use the disk? Not sure why larger disk
would cause the emulator to act different but also not sure why the
controller would corrupt sectors with a larger disk. It will take some thought
on how to troubleshoot. One way may be to create a large file and leave it
writing a known pattern and reading it back to check. If that fails since we
know what it was trying to write seeing what did get written may help determine
what is going wrong.

I've repeatedly run "devcheck /dev/disk +V"  which read/writes every single block and gives a clean bill of health.  Zero errors.

 

Not sure what you mean by Adaptec is an unsupported argument to --format.

 I mean if you use:
./ext2emu --ext /dev/zero --emu ../emufile_NEW --cyl 830 --heads 6 --format Adaptec

It responds with: "Unknown controller Adaptec. Choices are: ..."

[David Gesswein]

On Sat, Jan 06, 2024 at 10:28:33AM -0800, 'Adam Billyard' via MFM Discuss wrote:
>
>
> On Saturday 6 January 2024 at 17:57:37 UTC d...@pdp8online.com wrote:
>
>> Reading from real disks bad sectors are expected but if image created
>> with emulator you shouldn't get them.
>
>

> But on tek4404 you have to use the *formatw* command to low level format

> before starting. Perhaps its screws up some data?
> (I notice the sysreformat floppy come with a unique badblocks file for the
> Micropolis HD it is associated with - slightly insane and begging for
> catastrophe back in the day..)
>
> Trying to find a format that the 4404 accepts has proved frustrating.
> Endless "Physical format: Controller error" when formatw starts.
>

Sounds like Tek only intended for it to be used with disks that they
preformatted and it can't properly format a random disk since it needs
to read something from it first.

> In the end I took the known good 20MB image using 3 heads, and modified

> your *copy_emu.c *prog to sling data from heads 0-2 to heads 3-5 to at

> least get a file of the right size and that formatw didn't reject with a
> Controller error. As I say, messy..
>

Interesting that worked since the sector headers would have been wrong.

> I've repeatedly run "devcheck /dev/disk +V" which read/writes every single
> block and gives a clean bill of health. Zero errors.
>

I'm still confused. You said

but
does still occasionally have read/write errors and then tears itself to bit
wanting to delete corrupt files etc.

Your saying devcheck reports no errors but the operating system does report
errors?

> I mean if you use:

> *./ext2emu --ext /dev/zero --emu ../emufile_NEW --cyl 830 --heads 6
> --format Adaptec*

>
> It responds with: "Unknown controller Adaptec. Choices are: ..."
>

ext2emu is a separate program that knows about less formats. We can add support
but would need to figure out how the disk geometry is encoded in the
unknown header flag bytes. I think the only way to to reformat with
different parameters and see how they changes.

For your Tek4404_Uniflex_2.0_st80_wmgr_uemacs_telnet.emu
--sectors 18,0 --heads 3 --cylinders 830
We have

Unknown header flag byte 24 on cyl 0 head 0 physical sector 1
Unknown header flag byte 3c on cyl 0 head 0 physical sector 2

Unknown header flag byte 46 on cyl 0 head 0 physical sector 3

Unknown header flag byte 3e on cyl 0 head 0 physical sector 4

Unknown header flag byte 06 on cyl 0 head 0 physical sector 5

Unknown header flag byte 3e on cyl 0 head 0 physical sector 6
Unknown header flag byte 06 on cyl 0 head 0 physical sector 7

For your latest image with 6 heads we have

Unknown header flag byte 24 on cyl 0 head 0 physical sector 1
Unknown header flag byte 3c on cyl 0 head 0 physical sector 2
Unknown header flag byte 26 on cyl 0 head 0 physical sector 3
Unknown header flag byte 3e on cyl 0 head 0 physical sector 4
Unknown header flag byte 26 on cyl 0 head 0 physical sector 5
Unknown header flag byte 3e on cyl 0 head 0 physical sector 6
Unknown header flag byte 06 on cyl 0 head 0 physical sector 7

They are different but not in a way that makes sense to me.

[Adam Billyard]

> On 6 Jan 2024, at 19:14, David Gesswein <d...@pdp8online.com> wrote:
>
> On Sat, Jan 06, 2024 at 10:28:33AM -0800, 'Adam Billyard' via MFM Discuss wrote:
>>
>>
>> On Saturday 6 January 2024 at 17:57:37 UTC d...@pdp8online.com wrote:
>>
>>> Reading from real disks bad sectors are expected but if image created
>>> with emulator you shouldn't get them.
>>
>>
>> But on tek4404 you have to use the *formatw* command to low level format
>> before starting. Perhaps its screws up some data?
>> (I notice the sysreformat floppy come with a unique badblocks file for the
>> Micropolis HD it is associated with - slightly insane and begging for
>> catastrophe back in the day..)
>>
>> Trying to find a format that the 4404 accepts has proved frustrating.
>> Endless "Physical format: Controller error" when formatw starts.
>>
> Sounds like Tek only intended for it to be used with disks that they
> preformatted and it can't properly format a random disk since it needs
> to read something from it first.
>
>> In the end I took the known good 20MB image using 3 heads, and modified
>> your *copy_emu.c *prog to sling data from heads 0-2 to heads 3-5 to at
>> least get a file of the right size and that formatw didn't reject with a
>> Controller error. As I say, messy..
>>
> Interesting that worked since the sector headers would have been wrong.

Indeedy. But presumably the Tek formatw command writes in the LBA as it goes so fixes it?

>
>> I've repeatedly run "devcheck /dev/disk +V" which read/writes every single
>> block and gives a clean bill of health. Zero errors.
>>
> I'm still confused. You said
> but
> does still occasionally have read/write errors and then tears itself to bit
> wanting to delete corrupt files etc.
>
> Your saying devcheck reports no errors but the operating system does report
> errors?

So. What happens is I get what looks like a good 40MB disk image. I start installing the OS / tools etc and then after installing some larger files it starts throwing write errors and it all goes south. On the next boot with the diskrepair (fsck) deleting lots of corrupt files including the kernel..

I have done this loop *many* times.

But with the duplicating heads0-2 to 3-5 I got an image that also had some write errors but I was able to run a block check and note down every bad block and get them added to .badblocks.

Perhaps I’ve got them all? I added the ones thrown up by —analyze command I posted - and they ‘re all already in .badblocks.

As you say, having *any* bad blocks would seem impossible.

Block 0 is a boot sector
Block 1 is SIR
Block 2-3000 are File Descriptors

Different contents so different headers?

>
> --
> You received this message because you are subscribed to the Google Groups "MFM Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mfm-discuss...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/ZZmmpMBw05yvdsdv%40hugin3.

[Adam Billyard]

Reading the comment about perhaps Tek expecting data on the disk before formatting made me try just initializing a plain vanilla disk:

./mfm_emu --drive 1 --file TEST --initialize --cyl 830 --heads 6 

Mount it and try and format it, it all runs to completion and then prints:

6 Heads
829 Cylinders <==  ??
  18 Sectors/Track
015DBC Total blocks (hex)
No bad blocks detected

Disk write write error - Code: 01 - sector: 0x004270

829 Cylinders??   Any guesses?

[David Gesswein]

Did you specify 830 to the format program or did it try to figure
it out itself? Its possibly it reserves one cylinder for system purposes.

Not sure about the disk write error. What does mfm_util show if you have
it create an extracted data file from TEST?

There are a couple people now who are seeing some strange behaviors recently
so its possible that we are seeing a bug. So far nothing is clear on where it
may be.

On Sun, Jan 07, 2024 at 07:59:15AM -0800, 'Adam Billyard' via MFM Discuss wrote:
> Reading the comment about perhaps Tek expecting data on the disk before
> formatting made me try just initializing a plain vanilla disk:
>
>

> *./mfm_emu --drive 1 --file TEST --initialize --cyl 830 --heads 6 *

> Mount it and try and format it, it all runs to completion and then prints:
>
>
>
>
>
>
>

> * 6 Heads829 Cylinders <== ?? 18 Sectors/Track015DBC Total blocks (hex)No
> bad blocks detectedDisk write write error - Code: 01 - sector: 0x004270*

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/59a390f1-ce43-4158-ada3-b725d2c054ffn%40googlegroups.com.

[Adam Billyard]

On 7 Jan 2024, at 16:19, David Gesswein <d...@pdp8online.com> wrote:

Did you specify 830 to the format program or did it try to figure
it out itself? Its possibly it reserves one cylinder for system purposes.

For this test I simply used below (ie it initialized fresh data)

./mfm_emu --drive 1 --file TEST --initialize --cyl 830 --heads 6 

When I run:  ./mfm_tul —analyze —ext filename —emu ../emu/TEST 

I get bad LBAs that remind me of the badblocks I’ve had in the past  (always grouped around 16800s,  36300s, 55800s and 75300s)

Here is the output: 

Original decode arguments: --heads 6 --cylinders 830  --rate 10000000

./mfm_util: unrecognized option '--rate'

Matches count 36 for controller Adaptec

Header CRC: Polynomial 0x41044185 length 32 initial value 0x0

Sector length 512

Data CRC: Polynomial 0x41044185 length 32 initial value 0x0

Number of heads 6 number of sectors 18 first sector 0

Interleave (not checked): 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Command line to read disk:

--format Adaptec --sectors 18,0 --heads 6 --cylinders 830 --header_crc 0x0,0x41044185,32,6 --data_crc  0x0,0x41044185,32,6 --sector_length 512 --retries 50,4 --drive 0 

Unknown header flag byte 24 on cyl 0 head 0 physical sector 1                  

Unknown header flag byte 3c on cyl 0 head 0 physical sector 2

Unknown header flag byte 26 on cyl 0 head 0 physical sector 3

Unknown header flag byte 3e on cyl 0 head 0 physical sector 4

Unknown header flag byte 26 on cyl 0 head 0 physical sector 5

Unknown header flag byte 3e on cyl 0 head 0 physical sector 6

Unknown header flag byte 06 on cyl 0 head 0 physical sector 7

Missing LBA address 16848 to 16890                                             

Missing LBA address 16893 to 16899

Missing LBA address 16956 to 17015

Missing LBA address 17019 to 17023

Missing LBA address 17064 to 17135

Missing LBA address 17172 to 17260

Missing LBA address 36288 to 36330                                             

Missing LBA address 36333 to 36339

Missing LBA address 36396 to 36455

Missing LBA address 36459 to 36463

Missing LBA address 36504 to 36575

Missing LBA address 36612 to 36700

Missing LBA address 55728 to 55770                                             

Missing LBA address 55773 to 55779

Missing LBA address 55836 to 55895

Missing LBA address 55899 to 55903

Missing LBA address 55944 to 56015

Missing LBA address 56052 to 56140

Missing LBA address 56160 to 56249                                             

Missing LBA address 75168 to 75210                                             

Missing LBA address 75213 to 75219

Missing LBA address 75276 to 75335

Missing LBA address 75339 to 75343

Missing LBA address 75384 to 75455

Missing LBA address 75492 to 75580

Missing LBA address 75600 to 75689                                             

Found cyl 0 to 828, head 0 to 5, sector 0 to 17                                

Expected cyls 830 doesn't match cyls found 829

Expected 89640 sectors got 88248 good sectors, 1392 bad header, 0 bad data

0 sectors marked bad or spare

0 sectors corrected with ECC. Max bits in burst corrected 0

To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/ZZrO%2BfjSJsmubXkL%40hugin3.

[David Gesswein]

I'm going to retest on all my machines and see if I I see issues. A couple
people have been seeing issues recently. Likely will take a couple days.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/A8801908-83B3-4E64-BC80-BEC7BBF73B7F%40googlemail.com.

[Adam Billyard]

Thank you. Do you need the TEST emu file I created and tried to format?

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/ZZyaePJVIstJ6Mb4%40hugin3.

[David Gesswein]

Not right now. Assume its pretty similar to the one you sent a while ago with
similar issues.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/81D071D7-07B5-40A2-BC94-F4DAF156B037%40googlemail.com.

[Adam Billyard]

Yes - but possibly with less “transformations” as I tried things to fix it. The TEST file is just a fresh "—initialize —cyl 830 —heads 6” file that has been scribbled on by Tek4404 only.

Shout if/when you need.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/ZZ1sBq8%2BhnUkHjDu%40hugin3.

[Adam Billyard]

Got around to disassembling the MSU so I could take pictures of the HD controller. 

Yes, its an Adaptec at highest resolution I can do in <5MB

Still getting annoying write errors when theer is a lot of disk traffic.  
Does this help?

[David Gesswein]

Doesn't really help. Is the error still only happening on larger disks?
I assume you have no repeatable way of causing an error other than run for
a while? Repeatable makes it much easier to troubleshoot.

I assume you don't have a working larger disk to see if the problems happens
with a real disk also?

> --
> You received this message because you are subscribed to the Google Groups "MFM Discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to mfm-discuss...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/b1e91af0-50c8-425a-87c9-34b244c7d6f8n%40googlegroups.com.

[Adam Billyard]

I have a 20MB disk image that seems much more reliable - but not 100% 

I have a 40MB disk image that is unreliable.

But this photo confirms the Tek4404 uses the exact controller board shown in the Adaptec 4000 doc, and the track layout in the docs is different from any that mfm_emu offers - so that would be a potential cause of write errors, no?

doc link: http://www.bitsavers.org/pdf/adaptec/ACB-4000/400003-00A_ACB-4000A_Users_Manual_Oct85.pdf 

Adam

To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/Za7hL5Y7Tt8ZApLw%40hugin3.

[David Gesswein]

mfm_emu doesn't know anything about track layout. For most systems the
low level format will rewrite format which your systems does but especially
on the bigger disk something goes wrong. Some systems the manufacturer
preformatted disks sold so the system couldn't reformat new drives in the
field. The question on if you had a useable real disk of the size your
having problems with is to determine if its a Tektronix problem or emulator
problem.

The new ext2emu changes will get you closer but track 0 header won't
have the correct header information describing the disk format so the
controller still won't be able to accesss the disk until it is reformatted.

Could you write a test program and run it against the drive from a floppy?
Does your computer support two disk drives so you could boot off one and
then access a second for testing?

> To view this discussion on the web visit https://groups.google.com/d/msgid/mfm-discuss/0FF8CE58-1BAA-409E-8A22-8B8D72CBBC04%40googlemail.com.