Finding duplicates of files on your CP/M disks.

[Peter Onion]

I'm in the process of moving my machine from RomWBW-3.5.1 to RomWBW-3.6 . At the same time I'm moving from using a 2½" IDE disk to SD-CARD and USB storage (USB for backing up).  I've produced some scripts to help doing this.

First off I have a USB-IDE interface that allows me to read the IDE drive on my Pi5, so using the cpmtools on there I can list the contents of all the RomWBW slices on the disk.  This requires some extra entries in the diskdefs file (RomWBW/Tools/cpmtools/diskdefs) which normally only has entries for the first four slices:

# First 12 slices of wbw_hd1k
# Assumes standard 1MB prefix
# Offset of any slice (in tracks) = 128 + (1024 * <slice_num)

diskdef wbw_hd1k_0
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 128T
  os 2.2
end

diskdef wbw_hd1k_1
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 1152T
  os 2.2
end

diskdef wbw_hd1k_2
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 2176T
  os 2.2
end

diskdef wbw_hd1k_3
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 3200T
  os 2.2
end


diskdef wbw_hd1k_4
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 4224T
  os 2.2
end

diskdef wbw_hd1k_5
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 5248T
  os 2.2
end

diskdef wbw_hd1k_6
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 6272T
  os 2.2
end

diskdef wbw_hd1k_7
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 7296T
  os 2.2
end


diskdef wbw_hd1k_8
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 8320T
  os 2.2
end

diskdef wbw_hd1k_9
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 9344T
  os 2.2
end

diskdef wbw_hd1k_10
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 10368T
  os 2.2
end

diskdef wbw_hd1k_11
  seclen 512
  tracks 1024
  sectrk 16
  blocksize 4096
  maxdir 1024
  skew 0
  boottrk 2
  offset 11392T
  os 2.2
end

You will need to alter the path in this bit of shell sript which creates files slice_0 to slice_10 listing all the files for all the users in each slice.

for a in 0 1 2 3 4 5 6 7 8 9 10 ; do sudo /home/petero/RC2014/RomWBW-3.6/Tools/Linux/cpmls -f wbw_hd1k_$a  /dev/sda  > slice_$a ;  done

Then I used this bit of python to look for duplicated files across all the slices and users.

import re
files = {}

for slice in range(8):
    print("Reading slice_",str(slice))

    with open("slice_" + str(slice)) as file:
        for line in file:
            line = line.strip()
            if line:
                x = re.search("^[0-9]+:$",line)
                if x:
                     user = line
                else:
                    name = line
                    if name in files:
                        files[name].append([slice,user])
                    else:
                        files[name] = [[slice,user]]

print("DUPLICATES:")
for name,users in sorted(files.items()):
    if len(users) > 1:
        print(name,users)

This prints a list of files that are duplicated with the slice number and user number like these examples:

c.com [[2, '0:'], [6, '0:']]

cpp.com [[2, '0:'], [4, '0:'], [6, '0:']]

m80.com [[3, '0:'], [7, '0:']]
mac.com [[0, '15:'], [1, '0:']]

PeterO

[Ed Silky]

Nice Peter, thanks for sharing this.

--
You received this message because you are subscribed to the Google Groups "RC2014-Z80" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rc2014-z80+...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/rc2014-z80/09244ca4-7493-456c-8fa3-d470bfe77776n%40googlegroups.com.