Back Up verification, phantom music files and how long will your music library play for?
AndyC
I’ve developed a Windows program that other b2 users may find useful. It analyses your Back Up in comparison with your b2’s database and reports any missing music and unwanted extra music. It also sorts your database into a more thorough A-Z order & reports on any editing & renaming that may be beneficial. It’s freeware.
Sorting your b2 Music in A-Z order
This is the main core of the program and is there because I’m afraid the ‘A-Z’ button in the UI isn’t as thorough as I’d like: It doesn’t play all the albums by a given artist one after the other but instead plays all the albums in a global A-Z order irrespective of the artist. Also it doesn’t order numbers correctly – it goes 1, 10, 11, 12, 2, 3, 4 etc.
My program’s A-Z sorting fixes these problems & does more: It automatically ignores leading ‘The’ & ‘A’ (‘The Beatles’ is sorted under ‘B’ not ‘T’). If you add a full stop after a leading word, this word is ignored: Thus ‘Al. Stewart’ is sorted under ‘S’ not ‘A’, without having to call him ‘Stewart, Al’.
You also get 2 reference text files – the 1st lists all the albums (just like Settings>Maintenance>List Albums) plus a comprehensive header listing lots of the ‘Info’ data plus the no. of hidden mirror MP3s. The 2nd text file is the same but it also lists every track by name. I find these useful (when uploaded to a mobile device) when I’m shopping for new music away from my b2.
(Incidentally, whilst my program’s count of the number of different Tracks & Albums agrees with b2’s count, its Artists count (366) is one greater than b2 makes it (365). There’s a list of the different artists in one of the text files.)
Whilst doing the A-Z sorting, it also analyses your database and tells you things you might want to edit using the UI, such as any unknown artists, albums & tracks, any uncompressed WAV tracks, any ‘unusual’ characters in artist, album & track names (such as foreign language and other difficult-to-display characters) & very long artist, album & track names – the ones that take forever to scroll across the display.
(Incidentally, a link to this original part of the program was available via the Brennan website for a while, until the A-Z Sorting option was introduced in an S/W Upgrade.)
OPTION - Analysing your Back Up
Optionally, you can plug your external Back Up into your PC, wait for Windows to recognise it then run the program to have it compare the Back Up with the b2db database. The program accesses the Back Up in ‘READ ONLY’ mode so it can’t corrupt it in any way. Discrepancies are listed in ‘Back Up Problems.txt’.
To detect whether Back Up tracks exist, are corrupted or are zero length, the time duration of each track is read from its header. I couldn’t find any way to analyse MP3s in this way, so an estimate of their duration is made. (Interestingly I played a one hour MP3 track on my b2 and it kept updating the track duration on-screen readout for over 20 seconds until it was happy with its final guess. Methinks I’m not alone in having this trouble with MP3s!)
Having recorded time durations for every track, now the 2 text files listing the albums and tracks have track time durations added, just like you get on the back of better CDs. There are also grand totals, which admittedly are more fun than useful: However, I can now proudly tell you the fairly irrelevant but interesting fact that the 1,724 albums in my music library can play 24 hours a day for 68 days, 23 hrs, 54 mins & 48 secs before repeating any music!
Listening to longer tracks, especially live ones, is a guilty pleasure of mine. Therefore, one of the text files also lists all the tracks > 10 minutes, in descending time order of course.
Of course, this is all a little beside the point; the real reason for reading the Back Up is to find any problems in its accuracy compared to what’s in the b2. These problems fall into 2 categories:
b2 music missing from the Back Up – ‘Export to C’ now does its job well & all the music is successfully exported. Reported discrepancies here tend to be encoder related ‘false positives’.
When artist, album & track names have unusual characters – usually foreign accented letters or odd punctuation, b2 has trouble displaying them and generally renders them as double spaces on its scrolling screen (likely because they are encoded as 2 bytes rather than just 1). When such characters pass through encoders in computers, they often are ‘translated’ into a suitable local character and thus the whole artist, album or track name is very subtly altered.
When b2 exports to and imports from a Back Up, these subtle changes aren’t important because the names are never being compared with their original names, they’re just being read and the process works fine.
Unfortunately, this is no longer true when comparing the artist, album & track names in Back Up with the originals in b2db via Windows and its encoders. For the few names that contain such unusual characters, the names in b2’s database & the Back Up no longer exactly match and an error is flagged up. Windows has numerous encoders (my version of Word lists 135 of them!) and the 'translation' process isn't predictable as far as I'm aware. Therefore I haven't found a way of automatically reversing the encoding process so that the names in Back Up & b2db exactly match again.
Actually, I have found that the most frequent problem comes from the humble apostrophe, which can be represented by at least 3 different codes, depending on the native keyboard encoder of the person who originally entered the information in the Musicbrainz or Freedb databases. When you edit such apostrophes in the UI, you can often see them very subtly change shape on screen as the pixel patterns of the different codes isn't exactly similar.
In general, these ‘false positive’ missing artists, albums & tracks will be fairly rare in your music library and there are 2 solutions to this problem –
- either ignore them after manually checking that they exist OK in the Back Up. (However time data for these tracks won’t be logged),
- or edit these characters out of each name in the UI, substituting the nearest equivalent from your keyboard. (This is already common practice: eg. the German umlaut (2 dots over an ‘o’) is often replaced by ‘oe’.) Then freshen up the Back Up, delete the now unwanted older artists, albums & tracks from it and rerun this program to check that all is now OK.
Unwanted Extra Tracks in the Back Up. Obviously encoder problem ‘false positive’ artists, albums & tracks that are falsely reported as missing from the Back Up will also be listed here under their ‘other’ name as unwanted extras.
Discounting these, the most common reason for unwanted extra tracks is renaming or ‘delete & reload’ of newly remastered albums. (The old music stays in the Back Up alongside the new.)
These unwanted extras are listed in a text file so you can easily find & manually delete them from the Back Up without having to trash the entire Back Up.
Once these few ‘encoder’ & ‘rename’ items were accounted for in my 22,000+ track Back Up, I found there remained 24 corrupted tracks listed as not in b2. These ‘Phantom Tracks’ were corrupted in that their names had a double file extension – ‘.flac.flac .’ Heaven only knows where they originated from – certainly not from me. (File extensions are never shown in the UI.)
However, when I deleted them from the Back Up & did another ‘Export to C’, they all reappeared again! The conclusion is they exist on b2’s HDD, but ‘Scan Disk’ doesn’t recognise them, ‘Clean Up’ doesn’t delete them, yet ‘Export to C’ sees them as valid music and exports them. So, like the Hydra’s heads, when you delete them from the Back Up, they come back again! And, because Scan Disk doesn’t put them in b2db, you can’t see or delete them in the UI!
There are not many of these ‘cuckoos in the nest’ in my b2’s HDD so not much disk space is wasted. But they shouldn’t be there! Until there’s a b2 command to delete them, the only way I can think to do it is via NAS and Windows Explorer. (Getting NAS working seems far easier in Windows 10 than earlier version of Windows.)
Interesting. Has anyone else’s b2 got any of these ‘.flac.flac.’ tracks too, I wonder?
Incidentally I found no evidence of zero length tracks which could have been caused by the premature termination of the ‘Export to C’ process. (This was a bug in earlier S/W Upgrades.)
Program Run Time. Reading information from the Back Up is a slowish process, at about 1,000 tracks per minute from an HDD & 2x or 3x times faster from an SSD. Therefore Back Up Analysis is an option in the program so, if you just want to reset b2db after a Scan Disk, you can do it quickly.
Conclusion. I’ve had a lot of fun developing & testing this program. And not a little exasperation too at times! However, having run my Back Ups through it a few times and done some editing, old music deleting and general tidying, I'm now confident that they are ‘fit for purpose’.
Maybe a future Software Upgrade will include ‘Check Back Up’ (with no encoder problems! J), an enhanced ‘Clean Up’ (to get rid of those .flac.flac files) and ‘A-Z Sorting’ on steroids! Until then, I hope my program proves useful to others. Finally, thanks, Martin and the team at Brennan, for a really great music player.
I tried to attach the ZIP file to this post but it won't upload - probably doesn't like the .exe in it! So I've just attached the Read Me file (which is also in the ZIP file.)
The Dropbox Link is: https://www.dropbox.com/s/rhwaki7l9c5qe7v/b2%2BBackUpSort7.3.zip?dl=0