Bulk Upload still uploads albums in reverse order

[Mark Fishman]

I'm bumping this issue back up as apparently nothing has been done about it.

Anyone who has albums that are intended to play in a specific sequence -- e.g., operas in Wagner's Ring cycle, tone poems that form a sequence similar to a symphony, rock albums that came out in a specific order, and so on -- needs them to be listed in the b2db file in the right sequence. That usually means they have to be written to the disk in the right sequence.

I have previously reported that albums ripped and/or sorted into the right sequence before uploading -- either by copying them into a new directory one at a time in the right sequence or by using a program called FAT32sort  that actually sorts the directory entries on a FAT32 disk -- upload in REVERSE album sequence using Bulk Upload in the web UI. It's still happening, even after several web updates of the B2 software.

Software Version B2 Mar 10 2022 10:28:42

I uploaded this directory of albums (this is the artist directory as listed in a command window):

<DIR>          (Dvorak) Overtures (Kertesz, LSO)
<DIR>          (Dvorak) Requiem Op89 B165 (Kertesz, LSO)
<DIR>          (Dvorak) Scherzo capriccioso Op66 B131, Symphonic Variations Op78(28) B70 (Kertesz, LSO)
<DIR>          (Dvorak) Serenade in D minor Op44 B77 (Kertesz, LSO)
<DIR>          (Dvorak) Symphonic Poems (after K.J. Erben) (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 1 in C minor Op3 'The Bells of Zlonice' B9 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 2 in B-flat major Op4 B12 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 3 in E-flat major Op10 B34 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 4 in D minor Op13 B41 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 5 in F major Op76 B54 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 6 in D major Op60 B112 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 7 in D minor Op70 B141 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 8 in G major Op88 B163 (Kertesz, LSO)
<DIR>          (Dvorak) Symphony 9 in E minor Op95 'From the New World' B178 (Kertesz, LSO)

These albums uploaded using Bulk Upload in reverse order -- not in semi-random order, not last directory first and then the others in correct order, just BACKWARDS, up from the bottom. Here's a screenshot of the unsorted listing in the web UI after uploading:

(It's a screenshot, so it's not a complete list, but trust me, the rest are also in reverse order).

So that's the order they are now on the B2's disc, and also listed in the b2db.

WHY? Why does Bulk Upload work BACKWARDS through a directory instead of starting at the beginning and working down the list?

And more importantly, when (not if) will it be corrected?

-- m.

[Tim]

I am not able to answer Mark's specific query but really only commenting on my experience because I too have albums that need to play in the right sequence, and entirely understand Mark's concern. I have just under 3000 albums on my B2, they are all classical and a large percentage are operas comprised of two or more albums.  Playing in the right order is really important.  Many years ago when I bought my B2 and transferred the albums from the JB7  to the B2 I discovered that albums were out of sequence.  Also, even worse, tracks were out of order within the album.  I also realised that scan disc juggled the sequence of albums.

I liaised with Martin and he introduced the numbering system.  I now number all of my albums so that even when you run scan disc the order stays the same.  I know that my be laborious but it works for me and ensures that albums play in the correct sequence for me.

Tim

[Mark Fishman]

Tim,

yes, but...

Numbering affects the playing order of TRACKS within a single album. ALBUMS play in the order they are listed in the b2db, numbers or not, and b2db is built by Scan Disk -- which for some bizarre reason goes to the extra effort of crawling through the disk directories in physical order instead of asking the operating system to return the list of filenames. So to get the album names into a specific sequence requires uploading or importing (or ripping) them in exactly that sequence, and then never renaming them.

Here's an example from a FAT32 disk that I have on my computer. The DIR command in a command window still returns the physical order of files on a FAT32 disk, so although these aren't music files, you can see the difference between the physical order and the operating system order by comparing DIR to ls, which gives the default sorted sequence as seen from a Linux shell prompt looking at the same disk:

 Volume in drive G is FAT32
 Volume Serial Number is 2A0A-EA8E

 Directory of G:\TMcopy

03/01/2019  08:11 AM    <DIR>          .
03/01/2019  08:11 AM    <DIR>          ..
11/08/2012  08:51 PM               699 7za-AES.txt
07/22/2013  08:29 PM             3,584 audibleplist.tar
12/18/2015  05:23 PM           124,885 Ghost of Christmas Future Conditional Imperfect.jpg
12/12/2014  01:59 PM         1,129,472 TWD75schedule.tar
03/02/2019  01:39 PM    <DIR>          Desktop
07/06/2015  10:33 AM            15,364 .DS_Store
07/01/2015  08:26 PM    <DIR>          Audible-aax_20111114
03/01/2019  01:58 PM    <DIR>          Audible-reconvert
08/16/2012  01:49 PM             1,000 com.audible.data.plist
               6 File(s)      1,275,004 bytes
               5 Dir(s)  43,562,336,256 bytes free

$ ls -l TMcopy/
total 1344
-rw-r--r-- 1 mfishman None     699 Nov  8  2012  7za-AES.txt
drwxr-xr-x 1 mfishman None       0 Jul  1  2015  Audible-aax_20111114
-rw-r--r-- 1 mfishman None    3584 Jul 22  2013  audibleplist.tar
drwxr-xr-x 1 mfishman None       0 Mar  1  2019  Audible-reconvert
-rw-r--r-- 1 mfishman None    1000 Aug 16  2012  com.audible.data.plist
drwxr-xr-x 1 mfishman None       0 Mar  2  2019  Desktop
-rw-r--r-- 1 mfishman None  124885 Dec 18  2015 'Ghost of Christmas Future Conditional Imperfect.jpg'
-rw-r--r-- 1 mfishman None 1129472 Dec 12  2014  TWD75schedule.tar

If I had used ls -lR it would have given me listings of all the subdirectories recursively, ALSO IN SORTED ORDER. Why one would want to go through the directories as if they were files (they are, yes, fine, but...) instead of just calling the OS functions to get the names handed to you already sorted is beyond me. There might be a reason for building b2db with such extra effort, but I can't think of it.

As a result, upload order is crucial.

Cheers -- m.

[Tim]

Mark, I know that my albums play in the order which I have numbered them not the order in which they are downloaded.

Tim

[Daniel Taylor]

While I sympathize and agree that the problem should be fixed - and should not be terribly difficult to fix - I wonder if arranging the albums in reverse order for upload would reverse the reversed order, thereby putting the albums in the corret order.

[Richard Stamper]

Not that I'm going to try this, but the b2db file looks like it's a simple text format so you could edit it to put albums in the order you want?  

Or less invasively, create playlists of albums in the right order?

Richard

[Mark Fishman]

That's a very interesting idea, Daniel. How would you accomplish it? Bulk upload works by starting at the artist level, and takes everything below that, so in the example I mentioned earlier, I point the Bulk Upload bat the "Kertesz" folder and it takes all the albums underneath, each with its tracks. Are you suggesting that I shoud deliberately misname the albums so they sort the other way? or just that I rip them in reverse sequence so they're on my computer in the other order (even though Windows won't care and will still show them to me in the same order)?

Until we know what, exactly, the web UI is doing in a Bulk Upload, we don't really know why it's getting the folders in reverse sequence.

[Dennis]

You could consolidate multi CD sets into a single  folder, and use three digit track numbers to preserve the sequence. 

[Daniel Taylor]

Hi Mark,

Yeah, I was thinking of ripping them in reverse order.  But maybe there's more to it than that.

I thought of Dennis's idea too (which I like).  But I figured you'd already thought of it and had ruled it out for your own reasons.

[PMB]

Hi Mark,

I've made a note to get this checked.

Paul

Brennan Support.

[Mark Fishman]

Let's be really clear about this:

 - an "album", for hard-disk-based players, is not necessarily the same as the physical CD it came on. An "album" can be an entire 5-CD opera, or it can be one three-track concerto even though the physical CD contained two such works.

 - the B2 plays TRACKS in their numbered (or alphanumeric) sequence regardless of the order in which they were uploaded or imported

 - thus there is no problem playing the CONTENTS of an album in the right order, just number them properly (which I do)

 - the sequence of playing two or more ALBUMS is defined either by the b2db, which uses the "order-on-HD" and NOT the alphanumeric sort, or by creating a playlist and keeping that in some desired order

 - you can't have as many playlists as you can have albums, so that's awkward

The proper solution would be to have Scan Disk build b2db in SORTED order (which should be relatively easy as every operating system has a function to provide sorted directory listings) instead of crawling the physical directory structure and building the list that way. If b2db remains in physical order, though, a workaround would be to have Bulk Upload work as one would expect, i.e., not backwards.

[Philip Davies]

if you are prepared to edit the files on disk then there is a simple workaround that works for me. You can do this using the NAS function using a windows PC or if like me you spend your life with linux, then just log in and use the low level command line. 

I have a number of albums that are 2 disk sets ( eg war of the worlds ) even though I don't have that much classical. What I did was after ripping them ( and letting the B2 compress to flac automatically ) 

 I  renumbered the tracks to add a hundreds number for the album disk   then I put all of the files into one folder and delete the other folders - after doing this it is most important that you kick off a scan to rescan the disk before playing anything.   so I ended up with this folder name and set of tracks

In the folder     "/media/hdd1/music/Jeff Wayne/The War of the Worlds"     I have these files 
#
101 The Eve Of The War.flac
102 Horsell Common And The Heat Ray.flac
103 The Artilleryman And The Fighting Machine.flac
104 Forever Autumn.flac
105 Thunder Child.flac
201 The Red Weed (Part 1).flac
202 The Spirit Of The Man.flac
203 The Red Weed (Part 2).flac
204 Brave New World.flac
205 Dead London.flac
206 Epilogue (Part 1).flac
207 Epilogue (Part 2).flac

they then play in the correct order as they appear like one album. As I say this is only something as a workaround but for me with only a dozen or so double albums it works well. 

If you dont know linux, then do this with the NAS interface in windows, but for any linux hacks this is the commands you need  remembering the "

in disk 1 

    for i in 0* 1*  2*   ; do mv "$i" "1$i" ; done

in disk 2 

    for i in 0* 1*  2*   ; do mv "$i" "2$i" ; done

and so on 

then move the renamed files  from disk 2 etc into disk 1 folder  (eg mv 2* ../xxxxx  to avoid album art and toc files etc ),
rename the disk 1 folder to remove the "disk1" from the name and delete the other folders (carefully) 
obviously you need to take care with the album art  and I never touch the toc files 

then scan the disk from the GUI  <<<  if you forget this you will generally be disappointed. 

Also as I say you either need to stop auto compression or wait for it to finish for this to work. if you move whilst compressing the system gets confused. 

[Mark Fishman]

Philip,

Thank you, and I think you've missed my point.

I rip my CDs on my computer, not on the B2, and all the tracks from one album (NOT "one CD") are already in one directory, numbered in the proper sequence. Getting the tracks of an album to play in correct sequence is  not a problem.

The problem is getting two or more ALBUMS (not "two or more CDs that are part of one album") to play in a specified sequence, without putting them in a playlist.

b2db, which is built by running Scan Disk, lists artists and albums in the physical order of entries in the disk directory, NOT in any kind of sorted order. The physical order of entries in a FAT32 directory , which is what the B2 uses, is the order in which they are created (not "creation date", but the actual sequence in which the entries are written) until you delte something, and then the next time you create a directory entry it might or might not go there depending on length. Deleting things really scrambles the discussion. Let's not do that.

b2db controls the order in which ALBUMS are played -- not the order in which the TRACKS inside the album are played, but the order in which the "next" album will be picked. So to get albums to be played in a specific order, you have to put them onto the B2 in that order. You can import them if you build a USB drive just for those few albums in the correct structure, or you can use Bulk Upload from the web UI -- which, it seems, uploads them in reverse order of how the computer lists them. Not a scrambled order, mind you : actually BACKWARDS.

The tracks inside those albums will still play in the correct order, because they are numbered, and the B2 plays tracks in numbered order (even if they are out of order as listed in b2db -- I've looked). But the albums will NOT play in numbered sequence (Album 1, then Album 2, and so on), unless they are in that sequence in b2db, which means they have to be uploaded in that sequence.

Editing b2db in a text editor is dangerous, and tedious. Aside from the risk of corrupting the information, it would have to be done again after every Scan Disk, because Scan Disk completely rebuilds the file according to the unsorted physical directory sequences. AndyC on this forum has provided a program that sorts the b2db outside of the B2 -- it requires a Windows computer -- and that is a helpful workaround, but is not really a substitute for having the B2 keep the file in order by itself, or play albums and artists in a sorted order unless they are in playlists or a random mode.

I hope this is clear. -- m.

[JFBUK]

just to add.

This  "feature" is not just confined to Mark's specific user experience.

This thread explores how the sequence of the b2db can effect playback of all music stored under a particular artist (no alpha sort switched on)

https://groups.google.com/g/brennanb2/c/NUoOElJEUQY/m/X0lAIBYTBgAJ

Mark also made some very enlightening contributions (as always) to this thread

John

[Dennis]

On Friday, April 22, 2022 at 5:36:49 AM UTC-4 Mark Fishman wrote:

 - the sequence of playing two or more ALBUMS i

The obnoxiously named Classical Mode eliminates this problem by stopping at the end of each disc folder. What is a circumstance in which you would want the B2 to play the "next" folder?

 

[Mark Fishman]

The most obvious would be Wagner's Ring cycle, of course. Another might be to play the three "Times of Day" symphonies by Haydn (#6, 7, 8), which are each independent works but form a nice team. Possibly several albums by The Beatles, or any number of other groups whose recordings show both continuity and development.

Just as examples, of course.

[Dennis]

On Friday, April 22, 2022 at 11:03:10 AM UTC-4 Mark Fishman wrote:

...."Times of Day" symphonies by Haydn (#6, 7, 8), which are each independent works but form a nice team. 

And after #8 the B2 will automatically play "something" else. The only way I know to prevent this would be to enable Classical Mode -and- put the three symphonies in a playlist, in which case the physical sequence doesn't matter?

 

[Mark Fishman]

My experience is that Classical Mode will halt the playback after one album, even if you're playing a playlist.

Apart from that, you seem to be implying that the current upload behavior -- uploading an artist's albums in reverse order -- is fine by you, since you could always construct what would no doubt be a temporary playlist with the albums in a sequence that you like. As we've seen in other discussions, if you select an artist from the artist listing in the center panel of the web UI, the albums play in the b2db order that shows up in the right hand top panel, and you can't modify that order by naming the albums with leading numbers -- so if you want a specific sequence you have to make a playlist that explicitly contains album names, not just the artist name.

The number of playlists is rather severely limited, compared to the number of albums, tracks, or artists -- so you like making and deleting temporary playlists, yes?

[Ray Dion]

Deleted first replay because I wanted to add something.

First I want to say that I would not expect any device like the B2 to play 'albums' in a specific order. Like a juke box you load it up and hit play it will play the albums in the order the jukebox was loaded. The only difference here is a person is loading the jukebox and they don't go back and reorder stuff based on customer preference. The customer must open the jukebox and reorder the stuff. The difficulty is the B2 keeps replacing the jukebox every time you run scandisk and reloads it in some internal order. I don't call it a problem because it is not a feature of the B2 to play albums in an order except for the use of playlists.

A solution was put forth to use playlists to define these albums to be played in order but the B2 has a limited number. What if you used one playlist and put each of the sequenced album sets in order? Can you start a playlist in the middle and have it continue to play albums in the order defined by the play list. Then you only need one list. You will have to keep a list nearby that indicates where each sequence of albums starts. This might even allow one album to be in multiple sequences.

I can't test this for a few days because I am out of pocket.

Thinking about this some more, if this works (multiple chains in a single playlist) then you could have a bit of fun with it.

Place an introduction before each chain (an album with a  30 second track for each intro) that could be anything from an orchestra tuning up to a recorded voice introducing the set.

A single piece of intermission music that ends a chain so you know the next one is starting.

I would probably put this album on your blacklist so it does not affect random play.

[Mark Fishman]

Ray,

"Like a juke box you load it up and hit play it will play the albums in the order the jukebox was loaded."

Yup, exactly how I expect it to work. The problem is that it doesn't load in the order I am trying to load it. That's the whole point of Bulk Upload, isn't it? You have a bunch of albums in folders under their artist folders, you point the Bulk Upload at the top level, and it uploads everything under that. I expected it to load things in one of two orders: either the order that Windows shows them to me (sorted alphanumerically), or in the order I put them on my computer's disk. (In my case those are often the same order.)

It doesn't do either of those. It loads them in reverse of the sorted order.

So, it seems I have two choices to control "the order the jukebox was loaded." Either I have to rip everything on the B2 directly, and never edit the artist or album names (which risks changing the order of entries in the disk directory), or I have to use the File Upload function in the web UI instead of the Bulk Upload. File Upload works because it only lets me select items from one directory per upload, and is a PITA.

It does look as if I'm stuck with playlists at the moment.

You also write,  "I would not expect any device like the B2 to play 'albums' in a specific order." I have another similar device, at a similar price point, unfortunately now no longer supported by its manufacturer. That one also allows ripping, importing, playing Internet Radio, and so on, but it differs from the B2 because it uses metadata tags instead of filenames and foldernames to keep track of what's loaded, so there are very flexible ways to search and sort based on things like genre, and it relieves the pressure to put all the info into a filename. It also allows an "on the fly" playlist, so you can start playing something and queue up something else while that plays, to play next. The on-the-fly list goes away after it plays, or when you switch to something else; no need to manually create and delete a "temporary" playlist.

I'm not suggesting that the B2 should do all of that. I just want to know what to expect when I use a feature, and Bulk Upload violates what I think are reasonable expectations. In the absence of a manual or document saying that behavior is intentional, I consider it a bug.