B2 Export not copying one album.

[Nick B-T]

I have a B2 480G with the following loaded:

Software Version B2B Oct 22 2021 12:29:15

9152 tracks in 586 albums 473 artists

0 WAV 8995 FLAC 157 MP3 0 AAC

Capacity 480.00G Used 200.46Gb

USB 0 tracks in 0 albums 0 artists

0 Youtubes

All FLAC tracks were compressed on my Mac, and all track names / ID tags were taken from various databases and "corrected" on the Mac with Squeed. No problems so far.

I loaded the Brennan from an external drive through USB A, and all worked fine. I have since purchased downloads, set the track names / tags to my preferred format and loaded those via the web interface.

When I do an export to USB A, all albums are exported correctly (validated against the primary source).

With the exception of one, which never exports. It appears in the web interface, with the details for artist, album and tracks correctly displayed, and it plays fine. However, the export only copies the artist name, the album and tracks are missing.

Other software handles it fine - VLC on iPhone has no problems loading or playing.

Details of the folder structure, and a screenshot of the web interface attached.

For the time being, I'm keeping it in a folder on the Mac ('Brennan Backup Failures') so it can be reloaded separately if needed.

Nick 

[JFBUK]

Hi Nick,

what size USB drive are you exporting to ?

Have you looked at the storage usage stats of the drive when its connected to your Mac ?

Does this happen to be the last album you uploaded to your B2 ?

Have you run a scan disk recently ?

John

[Mark Fishman]

Also, although the semicolon is a "legal" character in a filename, it does have special meaning for some *application* code, so I'd be very suspicious of that semicolon in the artists and album names. If you have other artist and album names with semicolons that export correctly, then this should not be a problem, but personally I try to avoid things that act as command separators or that truncate names on some filesystems, and that includes semicolons.

[Peter Lowham]

Hi Nick,

Another possibility is that the artist name is quite long, it is 78 characters long I'm looking through another user's 'b2db' file which is the B2's Internal database.  Do you have other long artist names?

Regards,

Peter.

[Nick B-T]

Dear All,

To answer your questions in turn:

John - the B2 has a 480GB SSD. I'm exporting to a 500GB SSD (not a thumb drive) which was formatted as FAT32 on the Brennan, and has 295GB free space. The album was one of the first loaded from USB (I did them in batches of 50, and it was in the first batch). Since then, I've loaded several hundred more via USB and a couple of dozen through the web interface.I have run several scans, all finish with the correct track count. Via NAS, I can see the file structure, and it appears to be OK.

Mark and Peter - I use semicolons in many of my filenames. Almost all my classical albums are formatted as "Artist1[; artist2; ... artistN] as the artist level folder, and [Composer1 - work1[; work2; ... workN;] ... ComposerN - work1[; work2; ... workN]. I don't have any artist, album or track names over 100 characters, and none of the others are problematic. I did test before I made the final data load, and didn't encounter any problems (one of my early difficulties was a track name >170 characters, which made for some interesting entertainment, and allowed me to use the B2 display panel as disco lights.

The only real problems I've had with track names, apart from the >170 length are with the already known ' " ' and '/' appearing in track/ID tag data downloaded from external databases (Musicbrainz, Discogs and FreeDB included). I've removed all contentious characters except ' " ', and use only ' ' '  ';'  '.' and '-' as punctuation in track/ID tag details ( except where something has escaped ...). At some point, I shall have a cleanup by editing a copy of an export on the Mac to clean any ' " ' that I've missed, and reloading (proof of concept already tested on a small set).

I'll delete the widowed artist name, reload via the web interface and do another backup.  Will report back in due course.

Many thanks for your help,

Nick

[Nick B-T]

Dear All,

The problem appears to have been the comma in the artist name. I deleted the artist/album and reloaded via the web interface and did another export. It failed, but the log flashed past so quickly that I couldn't capture it.

Then I changed from:

'Academia Strumentale Italiana, Verona; Choir of Radio Svizzera; Diego Fasolis'

to:

'Academia Strumentale Italiana - Verona; Choir of Radio Svizzera; Diego Fasolis'

and the export worked.

Logs of the load and successful export attached.

The matter of "rogue characters" is well known. I would suggest that it would be good practice to build in a set of routines that are called by every load method to identify where there is a non-compliant track/tag name, and to make it compliant. Then these errors wouldn't happen, and the universally hated ▯ (U+25AF ?) wouldn't blight our display panels.

It's > 30 years since I was a programmer, but part of IBM Assembly Language basic training (c.1980) was the identification and elimination or correction of dangerous data before any kind of processing. Never assume that the user will do what you intend them to do - the side of the user interface that walks and talks is dangerous and untrustworthy!

Thanks again,

Nick

[Nick B-T]

Acksherly, I spoke too soon, so I now repent at leisure.

The log showed that the export had been successful. However, I now have a second widowed artist level folder in the export. One with a hyphen instead of a semicolon, and with no album folder or tracks. So the log lies ...

I tried a second change, as several of the track names had commas in. Those were changed to semicolons and another export done.  Again the log indicated success, and again the export disc does not have the files on it. 

I'm at a loss, so shall continue with my 'Brennan Backup Failures' separate folder UFN.

BTW, I'm quite happy to be a beta tester, so long as you don't mind a contentious aspie.

Regards,

Nick 

[Mark Fishman]

Commas *should* be OK, so there's definitely something else going on with that album. If I understand correctly, you can import it from USB but you can't export it to USB again.

Oh, yes: validity checking for inputs is essential for reliability -- I am no longer astonished, but I remain appalled, at how much software doesn't do that as a matter of course. See https://m.xkcd.com/327/ for a wonderful example.

[Nick B-T]

Problem solved.  And it is a problem, especially if your first language is one which uses diacritial marks. (That includes English, unless I'm being naïve.)

If either the artist folder or the album folder have a character which uses a diacritical mark (e.g. é, ü, ô) then that folder is exported, and its contents are not exported.  I fixed the original problem when I had an encounter with another album. This time, Communiqué, by Dire Straits. The artist folder exported, and the other DS albums also exported, but the flac files and cover for Communiqué did not export.  I deleted the album and reimported as 'Communique', which, TBH looks dire, and the tracks exported. Deleted and reverted to prevoious, and the export failed. Repeated the rename without the acute accent, and ran again, and the export worked.

So I tried the same with the original album, by changing 'Buxtehude - Membra Jesu nostri; Rosenmüller - Sinfonia XI' to: 'Buxtehude - Membra Jesu nostri; Rosenmuller - Sinfonia XI' (replacement of the umlauted 'u' in 'Rosenmüller'. This time the export worked fine. Reverted to original name, delete and reload, and it failed.

This appears to be a folder-level problem. If a folder contains a character with a diacritical mark, then the folder itself is exported, but the contents are not.

It doesn't apply to tracks, so: 'DakhaBrakha/Yahduky/01 - Шо З-Под Дуба (Shaw From Under The Oak)'

and: 'Academia Strumentale Italiana - Verona; Choir of Radio Svizzera; Diego Fasolis/Buxtehude - Membra Jesu nostri; Rosenmuller - Sinfonia XI/01 - Johann Rosenmüller - Sinfonia XI' (Rosenmüller at track level) 

do not cause any problems.

A fix would be welcome.  It really grates to watch 'Dire Straits/Communique/04 - Communiqué' on the front panel, and anywhere else.  We might have left the EU, but we're still supposed to be providing world-class software to the EU. Are you asking them as part of Brexit to forgo some of their national characteristics?

BTW, I also had to add English translations to the Dakhabrakha tracks so they would display.  Apparently, Cyrillic doesn't work on the B2 display, though it's fine in the web interface.  These are downloads from a reputable (non-Apple) site. Their track/tag naming is good, with few errors. 

Any system should be able to handle them as provided. Why doesn't the B2?

Nick

[Mark Fishman]

Please keep (firmly) in mind that at least two operating systems are involved -- Linux (fairly stripped down, on the B2), and whatever other system you use to look at it using either the web interface or the network-share interface. Also note that the B2 -- for reasons which escape me (intentionally, on my part: I refuse to accept the offered explanation) -- stores files on a filesystem which is not "native" to Linux. (FAT32 is a Microsoft filesystem, and does not support all of the attributes or capabilities of any of the filesystems developed for Unix-like operating systems. ext2/3/4 would be better choices, IMHO.)

I suggest that you do several things:

(a) look at the contents of the b2db file in which the B2 stores the names that you have provided for artist, album, and tracks. Look at it in a text editor, and probably you should do so on both a Linux system and on whatever else you have available. Pay particular attention to those names with the diacritical markings.

(b) look at the folder and file names in the B2's music folder, and definitely do so both from a terminal window (ssh session) on the B2 and from the file-manager view on your other computer, looking at the B2's shared music folder.

Somewhere in there they will look different. The b2db file is what controls what gets exported, or displayed in the web interface, but the actual exporting is done by a Linux process, so the names had better match or something won't be found -- and my initial suspicion is that something doesn't match.

Yes, I know, Unicode, 16-bit characters, yadda, yadda: unfortunately not used universally and what actually gets written to disk or in files gets displayed differently and interpreted differently depending on the application, the OS, the manner of access, etc..

To the best of my knowledge, English -- except for words that have been subsumed in their original "foreign" form, like communique-with-an-accent so you know to prononce the final e -- does not use any diacritics. It does have diphthongs -- the conjoined ae in encyclopaedia, for example -- and those don't display well, either, which is probably why Noah Webster (curse that man!) spelled it "encyclopedia". If you pretend to be American, or can Anglicise everything, you can use ASCII.

Nice bit of detective work, by the way -- thanks -- m.

[Daniel Taylor]

Hi Nick,

Thanks for the qualitative investigation.  That uncovers what, in my opinion, is a bug.  Unfortunately, in light of what Mark says, it's a bug that's very unlikely to be fixed.  Nonetheless, it's really good to know about it.  Many of us probably have incomplete backups of our precious music.  And now we know what to look for, and in some cases, have the knowledge we need to deal with it.

I just checked my backup of Communiqué, and thanks to my awareness of the B2's difficulty dealing with diacritics, I had already renamed it so that it could be handled correctly.

[Mark Fishman]

If you're worried about incomplete backups, AndyC (as known here on the forum) has written a program for Windows) whose initial purpose was to sort the b2db file alphanumerically so the artists are in sequence, and their albums are in sequence and collected appropriately. BUT -- and this is the really good news -- if you connect your B2 export disk to your computer when you run the program,  it will compare the contents of the backup disk to the listing in the b2db file, telling you if anything is missing or extra in either direction.

[PMB]

Hi All,

I have added this to the bug list in case anything can be done.

Paul

Brennan Support.