Transferring files

[Vinegarjoe]

Hi there

Trying to transfer my iTunes DBase from USB HDD to Brennan B2

Used Import from C but seems to be taking forever, around ten minutes per track. 

It's a 143GB database. (could be a while ....lol)

There must be a better way???

Any help appreciated.

Rgds. Phil

[Peter Lowham]

Hi Phil,

I've been doing a lot of work on 'Export' over the last two or three weeks, mainly because I had an issue with Export crashing after about 5 hours run time.

From that, I can tell you that the B2 will Export at around 34GB/hour (the later model) or 24GB/hour (earlier model).

I haven't had the time or kit to test the 'Import' yet, but I would think that it should run, at the very least, at half of that rate, if not much much better.

What I have found is that there are two important factors to understand to allow Export to work correctly, and I am assuming that these factors will apply to import.

Those are:

1. Each 'Artist' folder must ONLY contain 'Album' FOLDERS.  If there are any files in the 'Artist' folder, (.mp3, .txt, .jpeg, etc.) these will cause serious problems.

2. Each 'Album' folder must ONLY contain FILES (any files) with NO folders (i.e. sub folders) in them.

I've just spent 3 days doing a forensic analysis and clean up of my collection which had quite a few of these problems.

When I get a chance, I'll have a go at Import and we'll see what happens. 

My database is 200GB and the Export runs in 8 hrs 45 minutes

Regards,

Peter.

[Vinegarjoe]

Cheers for that, Peter.

Have got it going a bit better now. Up to about 2500 tracks so far but will be a while yet.

Will see then what the data looks like.

Again, thanks for the help.

Rgds. Phil

[Daniel Taylor]

Another thought:

With the Brennan units, it's best to not ask them to do more than one thing at a time. If you were starting fresh, it would be a good idea to remove your WiFi and/or Bluetooth dongles before starting the Import function.  But since you're already some ways into it, it maybe okay to just continue.  Do close any browser windows that you might have open to the WebUI.

The Export function will search for where it left off if stopped early and continue from there.  But the Import funtion will start over as if it had never started before.

Also, during the import, don't play any music on the B2 and don't try to rip any CDs.  Basically, just let it sit there and do the Import with no other tasks as distractions.

[Mark Fishman]

> 1. Each 'Artist' folder must ONLY contain 'Album' FOLDERS.  If there are any files in the 'Artist' folder, (.mp3, .txt, .jpeg, etc.) these will cause serious problems.

Yes, that's important -- not only for Export but also for the bulk upload function. I had a few "artist" folders on my computer that had extra jpg covers in them, and trying to use bulk upload on those folders ALWAYS produced the browser security message that the Brennan B2 gives when the web UI doesn't like something. Then it put the extra files under "unknown".

I think the Brennan sees those files, tries to get album and artist info for them by going up a directory level, and the browser balks at that, blocking the attempt to search "upward".

[Mark Fishman]

On Thu, Jun 18, 2020 at 7:34 AM Vinegarjoe wrote:

HI Mark

Did the transfer, however Brennan shows no albums and no artists. So got summit wrong.Trying again now. About two thirds through downloading from iTunes to USB then try again getting it to the Brennan.

Thx

Phil

A few things to look at:

(easiest to try, so do this first) Have you run the Scan Disk function on the B2? Sometimes that  fixes the album count.

(a little subtler) Does the scrolling display on the B2 say that it's using the Hard Disk? There have been occasional instances of the B2 switching to the SD card, which has no music on it.

(on your iTunes copy) The folder structure on the USB device for import MUST be:

b2export > music > [artist folders] > [album folders] > [tracks]

In theory only the [artist] > [album] > [tracks] part matters, but some people have found that the two higher-level ones I mention can make a difference.

(on your iTunes copy) The B2 can use AAC m4a files, but only if they are not the old "protected" (DRM) kind. And I don't think it can use ALAC files, which iTunes can give the same m4a extension. Do you know what format your iTunes files are in?

Good luck this time around -- m.

[PMB]

Hi Mark,

The B2 and BB1 will play ALAC files but not AIFF ones.

ALAC files are shown as AAC on the Brennan but are listed as M4A when viewed on Windows.

Paul

Brennan Support.

[Peter Lowham]

Hi Guys,

I have just started my first ever 'Import' of my music collection into my B2.  I have spent the last few days 'spring cleaning' the folders and files, having found a lot of 'dust and dirt' to clean out. Most of the cleaning involved removing FILES from the <Artist> folders and removing FOLDERS from the <Album> folders.

Having done all of that I was able to exactly reconcile folder and file counts between my NAS backup and the 'Export' backup.

I have 21904 files in 2075 folders.  The files break down into 14129 mp3, 5664 flac and 2111 'non music' (.jpg, .txt, toc, etc.) files. There are 613 artists and 1462 albums.

The Import rate after 1.5 hours of running is 27GB per hour (early model B2).  My collection is 203 GB, so I am estimating a completion time of 18:30 this evening if all goes well. So the 'Import rate seems to be slightly faster than the 'Export' rate (24 GB per hour)

I'm also monitoring the memory usage (per the 'Export' issue) and it does look like the 'Import' has a similar memory leakage issue.  So I might have to apply the '4 hour run rule' but I'll see how I get on.

@Phil, your data volume is 143 GB so your Import should have completed in about .5.3 hours (using early model B2 estimates).  It should be faster than that if you have the later model B2.

Regards,

Peter.

[Mark Fishman]

If Import is not incremental, but starts over each time, the "4 hour run" rule would have to be implemented by breaking up the folders to be imported into smaller groups.

A size estimate for each group would be useful.

[Mark Fishman]

I have had a private reply from Vinegarjoe. His second try at importing his iTunes files was successful:

On Fri, Jun 19, 2020 at 7:53 AM Vinegarjoe wrote:

Hi Guys.

Successful transfer of all files. Took a while but now there. How do I know if I have an early or late box?

Sorry for all the questions, and thanks for all the help

Rgds

Phil

 

[Peter Lowham]

Hi Mark,

Hmmm, I hadn't thought about that.  Thanks very much for the input. I'm 4 hours into the run and 109 GB of data has been restored so far (out of 203 GB total).  However, the buffers/cache memory figure has been dropping over the 4 hours from 256M to 87M (this is very similar to the effect that I'm seeing in 'Export'). If this figure drops to near 0 then the B2 crashes or the run stops (usually crashes).

I'm going to let this run continue without the '4 hour run' rule and monitor what happens.  The 'Export' will normally run for about 5 hrs 50 mins until the crash.

Then I can investigate whether or not 'Import' is working incrementally.

I can break to collection into 4 hour chunks quite easily, so that is not a problem to do.  I'll let you know how I get on.

Regards,

Peter.

[Peter Lowham]

Hi Phil,

The early model B2s have the SD card protruding out of the side of the case.  The later B2s the micoSD card is contained completely inside the case.

How long did the Import take approximately?

Regards,

Peter.

[Vinegarjoe]

Sorry Mark, sent to wrong place.

Next question (Doh!!!) Upload did not complete all of the files in database. Trying to work out what, and how many, has been missed. Found one to start with and tried to manually upload using Web UI and bulk upload.

When I try it it says, "Folders hidden by browser for security reasons - music uploaded to Artist/Album Unknown." and obviously does not appear in Brennan database.

Looks like maybe not a Brennan problem but any help would still be appreciated.

Any ideas what causes this.

Rgds

Phil

[Peter Lowham]

Hi Phil,

It looks like some of your <Artists> folders (e.g. .jpg, .txt. .pdf, .etc.) you have some FILES .  Each <Artist> folder must ONLY contain FOLDERS.

Also, each <Album> folder should ONLY contain FILES, with no folders (i.e. sub folders).

Those are the most likely problems.

Regards,

Peter.

[Vinegarjoe]

Hi Peter, thx.

Checked out the one in question. Artist is 1 folder, Album is 1 folder containing 12 files. Nothing untoward, but still will not upload. Album had rather a long title, so reduced that to see if it would help but nothing, still the same.

Rgds

Phil

[Vinegarjoe]

Hi again

Found the album in a file called "Unknown" at the bottom of the page. Edited it with the artist name, then did a sort and it appears in the right place.

Still no idea of what, though.

Rgds

Phil

[Vinegarjoe]

The problem now is, I have absolutely no idea what has been uploaded and what hasn't, short of going through it all line by line.

Would be good to get a solution that will achieve a full upload.

Rgds

Phil

[Vinegarjoe]

Started another import, to see if it cures itself. Checked a few go the "offending" files and can't find anything wrong. Doesn't seem to like individual imports via Web UI.

Anyhows be a while now before I see the results.

Rgds

Phil

[Peter Lowham]

Hi Guys,

@Mark; the 'Import' is NOT incremental (just as you suggested), it works in 'Overwrite' mode.  My Import fell over 5 hrs 15 mins into the run (as I had expected).  Upon rebooting the B2 and restarting the Import, it started at the beginning of the Export file and just began overwriting the already restored files from the first run.  Threfore, the '4 hour maximum run time rule' won't work for Import.

This means that a corollary of that rule is adopted, which is to break any Import into a series of volumes, each volume being taking no more than 4 hours to complete.

That in turn means that each volume of a collection should be no more than 100 GB in size.  Then each volume is imported one at a time.

I am basing the calculation on a transfer rate of 27 GB/ hr x 4 hours = 108 GB rounded down to 100 GB (transfer rate on my earler model B2).

The later model B2 might transfer at a rate of 38 GB/hr but I am only estimating this on data that other users supplied to me for 'Export' timings.

I'm about to begin testing this now.

You should power down and reboot your B2 before starting the first Import, and also for each time before you start each of the  the next Import 'set'.

Regards,

Peter.

[Mark Fishman]

Is it really necessary to *power down*? There's a "reboot" buried in one of the maintenance menus, and as far as I can tell that actually does a standard Linux reboot, i.e., it stops all running processes, and then hands control to whatever runs the startup routine. (There's actually a reboot script in the appropriate place where I would normally also expect a shutdown script, only there's no shutdown script...)

Maybe just running reboot from the maintenance menu is sufficient, and avoids all the usual worries about "Saving" and removing power.

[Peter Lowham]

Hi Mark,

I looked at just doing the 'Maintenance', 'reboot' option and tried it out for one run, Like you, I want a cleaner process.  However, what happened in that run was that, after the first reboot, I checked the memory buffers leakage situation which showed as released (good), BUT when I resumed the Export, it crashed shortly afterwards (bad!).  So it looked like there was something else causing an additional problem within the Linux environment. So for now I'm doing the full powerdown and restart cycle.

I am planning to check this out again, but I am backlogged with test tasks.  I can only run one situation at a time and, with each run taking 5 to 9 hours, it is slow progress!

I have now started the 'managed Import volume' test about 30 minutes ago, so my B2 is tied up with this for now.

Regards,

Peter.

[Mark Fishman]

Peter,

only a comment, as you seem to have the processes well defined:

 -- "Export" is incremental, so each "4-hour run" of large exports (i.e., more data than can be copied in under ~5 hours) must be done to the same hard disk and an existing folder structure in order to capture all the files.

 -- "Import" is NOT incremental, so to restore an "Export" set the existing exported folder structure has to be broken up into sections that can each be copied back in under ~5 hours.

That is inconvenient, to say the least: you have to create one giant exported folder tree, and then you can't import that without breaking it up!

As Nero Wolfe used to say, Pfui.

-- m.

[Mark Fishman]

Peter, I am still puzzled by this:

> Early model B2s transfer data at 24 GB per hour.

> Later model B2s transfer data at 34 GB per hour, so the '4 hour run rule' could probably be extended to 5 hours,

> but I have no way of testing for this. I have obtained the 'data transfer rate' figure from other forum users who kindly sent me their 'Export' stats recently.

Your earlier discussion seemed to indicate that the "crashes" (abnormal termination) of the export and import processes were due to running out of working memory, owing to memory "leaks" during the process.

If the newer B2 runs faster -- copying more data per hour -- would it not also run out of memory faster? If it *has* more memory, that would help, but otherwise I fail to see how you can estimate that the newer B2 should take longer to crash than the old ones. I would assume that the "safe" amount of data per run would remain the same.

(In a private message I sent you about 9 days ago I included some lines from dmesg, but the key bit here is that my B2 (with the internal microSD card) seems to have 512MiB of RAM. What does yours have?)

-- m.

[Peter Lowham]

Hi Mark,

First off, I cannot locate a private mail from you around that date, so could you send it to me again please?  I've also checked my Spam folder, but there is no sign of it.

Now to the main point.  I made the statement on an assumption that the later B2s had 1GiB of main memory.  However, now that you have informed me that your B2 has 512MiB, then your point is very valid and my assumption is wrong, so thank you for that correction.  My early B2 also has 512MiB, so I concur with you that the later model B2s are likely to hang or crash earlier in the run.

If my further assumption that the leakage is exacerbated by volume, then I estimate that the later B2 would get into difficulties after about 4.5 hours of running, but this is very much a 'finger in the air' estimate.

I'm in the process of documenting the memory usage over a run that crashed.  It will show a gradual increase of 'Used' buffers to almost the 'Total' figure along with a gradual decline of 'Free' and 'Buff/Cache' buffers to almost 0, when the crash or abort occurs.

I will modify the post above and repost, and will delete this one above.

Many thanks for your help with this,

Regards,

Peter.

[Peter Lowham]

Hi Mark,  (this post replaces the deleted post above as it contains a correction; thank you Mark Fishman for this)

Excellent summary; you have hit the nail on the head dead centre!

Here is what I have learned in addition.

To be clear for others who might just have picked up on these posts, I am using an early model B2 (2016 with SD card protuding out of the case). The USB backup device is a 1 TB Seagate HDD unit.

The music collection is 217 GB volume, 21904 files, 2075 folders broken down into 14129 mp3, 5664 flac, 0 wav, 0 aac, 2111 'non music' (.jpg, discid, toc, pdf, etc.)

For the 'Export' function

Maximum 'safe' volume size (for a 4 hour run) is 100 GB; equivalent track count (.flac) approximately 5000 tracks (but refer to Note 1 below).

My 'Export' will fail consistently after 5 hours 50 minutes if I ignore this rule (based on approximately 15 Export test runs).

The dependency on success seems to be 'volume' related rather than 'number of tracks'.

Early model B2s transfer data at 24 GB per hour.

Later model B2s transfer data at 34 GB per hour (based on data received from other forum users who are running 'Export').

For the 'Import' function

Maximum 'safe' volume size (for a 4 hour run) is 100 GB; equivalent track count (.flac) approximately 5000 tracks (but refer to note 1 below).

My 'Import' will fail after 5 hours and 15 minutes if I ignore this rule (but based on two Import runs so far).

The dependency on success seems to be 'volume' related rather than 'number of tracks'.

Early model B2 transfer data at 25 GB per hour.

Later model B2 probably transfer data at a higher rate, but I have no data to go on.

General

My 'Import' test runs last night ran to successful completion this morning. I broke my full collection into three volumes, Import[section 1] @ 101 GB, Import [section 2] @ 62 GB and Import [section 3] @ 54 GB.

During the tests I have reconciled the Export files and Import files counts down to file type level and they are a perfect match.  

From this, I have no doubt that the Export and Import functions work absolutely correctly as long as the two 'run rules' are adopted.

Once the technical problem is addressed, we can drop the rules and go back to Exports and Imports that 'just work'.

Note 1. The 'tracks equivalent' figure is based on 'popular music' tracks say of up to 5 minutes play time.  Classical and jazz collections could be different, due to potentially longer play times per track so the 'volume' rule should be applied.

Regards,

Peter.

[Mark Fishman]

Peter, I've resent the info that I sent privately on the 11th. I've sent it directly to you because my question is a technical one and would cause eyes to glaze over [OK] or confuse people [not OK]. :^)

I hope it is helpful. -- m.

[Peter Lowham]

Hi Mark,

Yes, I've just received your mail, thank you.

I'm just digesting your information now; you're right, this is very interesting to me. I agree with your reasoning on this, and am going to do some comparisons with the information that I have gathered over the last two weeks.

As they say 'I'll be in touch'.

Regards,

Peter.

[keithG]

Hi Peter,

I've been following this topic with interest.

My current collection of 315 Albums - 6120 tracks - 113 Gb may be the borderline to this problem.

(So far ) I've not experienced any problem with 'Export to C' or 'Backup to C' as I use both (belt & braces approach).

B2 Software - 13.Nov.2018
B2 dates from 2016 with side card slot - SD card only unit
SD Card - 256 Gb class 10

Using the 'Restore from C' process I did not encountered any problem with a full recovery of 113G to a 240G SSD connected to USB-B (which equates to the internal HDD). This took under 2h 50m, without problems, but indicates a transfer speed of 40 Gb/hour.

Using 'Export to C' for music stored on the SD card to a 500G Seagate drive took about 2h 43m, indicating a transfer speed of 41 Gb/hour.

I've not used the 'Import USB C' for some time as I prefer the convenience of the incremental update capability of 'Restore from C' as I can download mp3's and then transfer then to the B2 as a custom album from the backup.

The transfer speed is most likely to be limited by the performance of the attached source and destination media.

 

From your investigations so far, do you think this problem will also apply to 'Backup to C' and hence I need to be watchful. At present I rely on matching the B2 info page with an 'mp3 + flac' file count from the backup drive.

Regards

Keith

[Peter Lowham]

Hi Keith,

Thank you for your information; very interesting!

Your (very) improved Export times are almost certainly down to the SSD card.  The early B2 (like yours and mine) transfer data to a USB HDD backup device at about 24 GB per hour.  Your transfer rate of 41GB per hour is an impressive perforance boost.

I have been having some discussions with Mark Fishman (who posts often on this forum) on this topic over the last week or so and he raised an interesting point.  That is, as the performance of the B2 goes up, is the Export 'failure point'  likely to occur earlier in the cycle.  My view is that this is likely to be the case, but I can only theorise about that because I can only test on my early model.

My view is that the Export problem does seem to be data volume based.  On my B2, the Export failure occurs at about 5 hours 50 minutes, which is after about 125 GB of data. The reason that I chose a maximum Export run time of 4 hours was that I hope that it would make all Exports reliable, independent of the B2 model.

So, in terms of risk of failure, my view is that for my early B2, up to 100 GB of data is 'safe' (Green), 100 - 120 GB is 'at risk' (Amber) and above 120 GB is 'danger' (Red)

That would translate to your early B2 SSD model to up to 2.5 hours is 'safe' (Green), 2.5 - 3 hours is 'at risk' (Amber) and over 3 hours is danger (Red).  This would put you in the 'Amber' zone, but these figures are very arbitrary.  Is your collection expanding at this time?

The other item I have in mind is that if you look at your Export backup, and do 'Properties' on the 'b2Export\music' folder, does the folder count come to (# Artists + 315)?  That is one indication of a complete and healthy Export.

I have to emphasise that these data are my estimates, based only on one model, but supplemented by other forum users who have kindly supplied me with their data and experiences.

I haven't done any testing 'Backup' or 'Restore' but I can set up a test to see if these functions suffer from the same problem.  I'll let you know when i have some information, but I have a backlog of tests already so it will be a few days before I can get back to you.

Regards,

Peter.

[keithG]

Peter,

I'm going to try exceeding the 130Gb boundary by creating dummy albums (renamed copies) and adding them to the b2 backup. Then do a full backup using 'Backup to C'.

Please confirm whether you are using GB based on 1024 or  GB based on bytes (base 1000 as used on the B2 display).

Regards

Keith

[Daniel Taylor]

Keith,

Unless you have a JB7, for which you need backward compatibility, you should use the Export (and Import) command.  Those commands maintain the folder structure that the B2 uses.  Don't complicate your situation - unless you have to move files between a JB7 and B2.

[Mark Fishman]

Do we have any idea if the Backup and Restore processes also leak memory and crash a B2, similar to how Export and Import do? If the code is basically what was used in a JB7, it might not have the same issues.

[Peter Lowham]

Hi Mark,

I don't yet know if Backup and Restore have a similar problem.  I'm planning to set up a test for these, but it probably won't happen for a day or two as I have a backlog of other tests to run first.

Regards,

Peter.

[Peter Lowham]

Hi Keith,

My figures and calulations use GB (base 1000) and not GiB (base 1024).

Regarding the use of 'Backup to C' do you also have a JB7? 

Regards,

Peter.

[Daniel Taylor]

I cannot remember seeing if anyone has done a Backup from a JB7 and then a successful Restore onto a B2.  If anyone has done that, either successfully or not, would you please report your experience here?  It could be of assistance as we try to get this sorted.

[Nigel Burdett]

I had a JB7 some time ago and have now replaced it with two 500GB B2s. When I bought the first B2 around 5yrs ago I simply used the old JB7 Backup/Restore functions and it worked perfectly. In fact it is only in the last month or so when this Forum said that I should use Export/Import that I have changed.

 It is only a small collection--about 74GB and 4633 tracks so takes about 2 hours to backup

[Margaret Hall]

On acquiring a B2 in early 2018 I simply did a restore from my JB7 backup & all went smoothly. There were then c.1200 CDs in the collection. But when I discovered how much better FLAC files sounded (to me anyway) than mp3, I wished I hadn't done a restore, & re-ripped the entire CD collection. 

[keithG]

Hi Peter,

Perhaps I've missed something on this thread... but ..

I carried out both 'Export to C' and 'Backup to C' processes today to see how close to the 'red' zone I was after adding a number of test albums. However both processes were successful, and the transfer rate showed no sign of slowing towards the end. All files and folders were copied/created without the B2 complaining. Perhaps I didn't increase far enough.

B2 Gb (info display) 145.82Gb  (135.8 GB to base 1024)
Tracks - 8052  (7004 flac, 1048 mp3)
Albums 387  (351 flac, 36 mp3) - note: all mp3 tracks in 'albums' limited to 35 tracks/album
Artists - 84

Note : My music is stored on the SD card (256Gb class 10). Could this make a difference?

The main result is that I appear to have a safety margin of 70 albums, 1900 tracks or 30Gb which will satisfy a few years.

Regards
Keith

[Peter Lowham]

Hi Keith,

Thanks again for the further data.  According to my estimates you should be well into the 'Red' zone at 148 GB of data.  I did some head scratching and keyed your data into my spreadsheet.  That was when I noticed something else.  Recently I had added in to my spreadsheet a new column entitled 'Software Version'.  I only have about four entries because I had not been asking for that information, but you did provide that.

So, I have noticed that your B2 is running version 'Nov 13 2018' and that got me to wondering if perhaps you do not have the 'Export' problem at all because you are on a different release.

Therefore I dug out an old B2 image, 'Jan 04 2018' and flashed that on to a SD card.  I kicked off a new Export at 21:50 tonight using this version, so it will be tomorrow morning before I can make any further observations.  It is interesting to note that the GNU/Linux version is v '3.12.28' on that Jan 2018 image, whereas all of my tests are running on June 2020 image which uses GNU/Linux  v '4.19.'

I'll post an update tomorrow.

Regards,

Peter.

[Peter Lowham]

Hi Paul,

I think that I can report on some progress with the Export/ Import issue.  Last night I flashed an older image onto an SD card (Jan 04 2018) and plugged this into my long suffering B2.  I did not change anything else, just the SD card.  I kicked off an Export at 21:50. 

This morning I checked on progress and have found that the Export completed at 04:08 am.  So this Export took 6 hrs 20 mins to run.  I have done a high level reconcilation (number of files, number of folders and Export volume size) and it is complete. 

B2 Version	GNU/ Linux	Export	Import	Notes
	O/S	Time	Time
Jun 04 2020	4.19.23	8 hrs 45 mins	8 hrs 30 mins	Needs workarounds to complete
Jan 04 2018	3.12.28	6 hrs 20 mins	TBA	No workarounds used, ran to completion

Above is the table of results.  As you can see, the earlier version also ran much faster.  Not only that, but the B2 was in much better condition in the sense of memory available after the Export had completed.  The B2 did not require a power down and reboot to recover the memory buffers.

I have only done one run, and I have yet to run an Import, but already the situation looks much improved when using this release.  I'm thinking now that there is a problem lurking in the O/S.

I am planning to run a full Import later today if I have time.

Do you know if any other GNU/ Linux versions were used between the two that I have listed above?

Regards,

Peter.

[PMB]

Hi Peter,

Thanks for the update. I will pass this to Martin Brennan.

Paul

Brennan Support.

[Mark Fishman]

Peter:

Just an additional bit of data:

I bought my B2 in August 2018. I updated the software via wifi a couple of times since -- not every time there was an updated release, and I am currently running Software Version B2 Feb 12 2020 11:00:53 according to the web UI. My Linux version (uname -a) says:

Linux brennanb2 3.12.28-quick #1 PREEMPT Sun Feb 26 14:36:16 GMT 2017 armv6l GNU/Linux

I assume you have a B2B version, whereas I do not. Wasn't it the B2B update that changed the Linux kernel? Perhaps you could try applying a recent B2 update to your Jan2018 SD card to see what recent software does on the older kernel.

I'd run some Export tests but at the moment I have only 7691 tracks in 520 albums 192 artists (0 WAV 7681 FLAC 10 MP3 0 AAC) in 112.70Gb so I'm not convinced it would tell us anything. Since I rip on my computer (after discovering early on that the B2 didn't populate most tag fields) I have a backup of my music automatically, so I haven't done an Export since late 2018, when I had much less on the B2.

-- m.

[Peter Lowham]

Mark,

Thank you very much for that data; that's exactly what I needed to know.  Now that you have confirmed that your B2 is running 3.12.28, that means that I should be able to upgrade my card to the latest B2 release.  Until you told me that, I did not know whether or not there had been any other versions of GNU/Linux adopted by Brennan between Jan 2018 and now.

To answer your question, yes, I am running the B2B version of software.  So I can take your advice and upgrade my 'B2 Jan 2018' version and repeat my tests.

I am currently running a full Import (without workarounds) on the 'B2 Jan 2018' version, this won't finish until 15:00 today (if it does complete!), but I am becoming more confident that we have 'cornered' the base problem.

The memory management on the 3.12.28 looks and feels much better as the Export and Import processes run.  In the meantime, I'm off to Argos to collect a couple of SD cards, but I probably won't need them now!

Anyway, thanks again.  Your information could be the key to cracking open these issues.

Regards,

Peter.

[keithG]

Peter,

I was planning to carry out a backup (export) this evening after reconfiguring the B2 to the internal SSD. This currently has 113Gb (310 Albums, 6100 tracks)

Let me know if additional data would be of any assistance, as I can add 70 test albums to increase the backup to 146Gb (390 Albums, 8100 tracks). I can delete these afterwards.

I've just carried out a web upgrade on a copy of the B2 (13.Nov.2018) software to 2.Jun.2020 version on a spare SD card. This is NOT the B2B version.

I don't know the Linux version (or how to find it)

Regards

Keith

Regards
Keith

[Peter Lowham]

Hi Keith,

Apologies for not replying sooner.  Thank you for the offer, but I think that we are now at a point where we have a much better understanding of what is happpening, so expanding your music collection won't be necessary at this time.

However, I am still interested in your data so please post here as the information is very helpful.

 If you have access to the Linux command line, you can get the GNU/Linux version by typing this command at the # prompt.

uname -r

I'm reasonably sure that this will show:

3.12.28-quick

which is GNU/Linux version 3.12.28.  The B2B version will show '4.19.23'.

Regards,

Peter.

[Peter Lowham]

Hi Paul,

The 'Import' that I ran last night completed successfully without workarounds at 06:20am (Tueday).  This was using my standard data set but using the 'B2 Jan 04 2018' and it completed in 6 hrs and 15 mins.

Export/ Import Runs
B2 Version	GNU/ Linux	Export (Full)	Import (Full)	Notes
	O/S	Time	Time
B2B Jun 04 2020

4.19.23

8 hrs 45 mins

8 hrs 30 mins

Will not complete without workarounds

B2 Jan 04 2018

3.12.28

6 hrs 20 mins

6 hrs 15 mins

No workarounds used, ran to completion

B2 Jun 04 2020	3.12.28	Running now	To be run
All of the above runs to date were performed using the data set described below.
Data volume	218	GB
# Folders	2075
# Files	21904
# Artists	613
# Albums	1462
# .flac	5664
# .mp3	14129
# .wav	0
# .aac	0
# Other files	2111
Total files	21904

To conclude, 'Export' and 'Import' are working completely well, so far, on the 'B2' versions of software (on GNU/Linux 3.12.28) but will fail to complete in some situations (with a data set larger than approximately 120 GB) when using the 'B2B' versions (on GNU/Linux 4.19.23).

I upgraded my 'B2 Jan 04 2018' version tonight to 'B2 Jun 04 2020' via 'Web Upgrade' and am now running the full 'Export' again.  This should complete at about 02:05 am this morning (Wednesday), if my estimate is correct. 

I'll update you again when this 'Export' run completes.

Regards,

Peter.

[Daniel Taylor]

Hi Peter,

How can you tell at what time the Export or Import reaches completion? 

[Peter Lowham]

Hi Daniel,

 

The  timings are quite straight-forward to see even if the Export or Import are unattended.  I use 'WinSCP' to connect into the B2. 

 

For Export, I go into the 'Remote File' window , select '/media/usba/b2export/music' and sort the the window by 'Date Modified' column. This give a listing of the folders and  files by load time. Then just scroll down to the last artist and album at the bottom of the window.

For Import, I select '/media/hdd1/music' and do the same as above.  The last folder/ file in the window gives the final completion time.

Regards,

Peter.

 

[Daniel Taylor]

Hmmm, interesting.  When looking at the date/timestamps on the files in Linux, it must access that file information in a different way than Windows does.  When I look at the files on the Export disk in Windows, they all have the same timestamp of 01/01/1980.  I've noticed that for other users, that date is 12/31/1979.

[Peter Lowham]

Hi Daniel,

Yes, I've seen that issue myself during this testing.  I've put this on my list for investigation, but it will be a while before I get to it because the list is getting longer by the day.

In the Imports that I am doing, the files are correctly dated when loaded.  However, I can also tell from the 'b2db' file, the order in which the albums are loaded and that also helps.

Regards,

Peter.

[Peter Lowham]

Hi Daniel,

Attached is a screenshot of the 'Remote files' window for my current Export.

Regards,

Peter.

[Leslie Brownlee]

Hi Peter, you've been keeping yourself busy. You say you updated the B2 software yesterday. Should that not read 'B2 Jun 02 2020' rather than 'B2 Jun 04 2020'.  

Always like to see a list of what some other people listen to, Gives a few suggestions of new artists to check out. Good to see the local boy "Van The Man" there. Always meant to check out Michael Chapman after hearing a track I liked called" Kodak Ghosts". Seeing his name on your list has reminded me to to do that.

You're doing some great investigative work. Great to have your experience in the forum and I'm sure Martin appreciates it.

Regards,

Leslie.

[Peter Lowham]

Hi Leslie,

You are correct, 'Jun 02 2020' should be the date.  I have corrected that in my spreadsheet; thank you for that.  I'll delete and repost the item also.

Hopefully, e are getting near the end of a 'saga'! 

Regarding 'Van the Man', back in about 1972 -74, I had friends who lived in Cyprus Park, Belfast. Van's family lived (possibly still do) just around the corner in Cyprus Avenue, and we used to see him walking around from time to time. As you will know, one of his songs in 'Astral Weeks' is entiltled 'Cyprus Avenue'.

Regards and thanks,

Peter.

[Peter Lowham]

Hi Paul,    (with date correction in the table below to deleted post above)

An update regarding last night's and the morning's 'Export' and 'Import' runs. I did a standard 'Web Upgrade' on the 'B2 Jan 04 2018' software yesterday which brought the software up to  'B2 Jun 04 2020'.  Both runs were done without any workarounds.

The 'Export' was started at 19:33 (Tuesday) and completed successfully at 02:03 this morning, giving a run time of 6 hrs 30 mins.

The 'Import was started at 02:11 this morning and completed successfully at 08:29, giving a run time of 6 hrs18 mins.

Export/ Import Runs
B2 Version	GNU/ Linux	Export (Full)	Import (Full)	Notes
	O/S	Time	Time

B2B Jun 02 2020

4.19.23

8 hrs 45 mins

8 hrs 30 mins

Will not complete without workarounds

B2 Jan 04 2018

3.12.28

6 hrs 20 mins

6 hrs 15 mins

No workarounds used, ran to completion

B2 Jun 02 2020

3.12.28

6 hrs 30 mins

6 hrs 18 mins

No workarounds used, ran to completion

All of the above runs were performed using the data set as described below.

Data volume	218	GB
# Folders	2075
# Files	21904
# Artists	613
# Albums	1462
# .flac	5664
# .mp3	14129
# .wav	0
# .aac	0
# Other files	2111
Total files	21904

To conclude, 'Export' and 'Import' are working completely well on the 'B2' versions of software (on GNU/Linux 3.12.28) but will fail to complete in some situations (with a data set larger than approximately 120 GB) when using the 'B2B' versions (on GNU/Linux 4.19.23).

It would appear that the Export/ Import issues are restricted to those who are using the 'B2B' versions of software and this is most likely caused by some sort of issue in GNU/Linux 4.19.23.

I also think that the issue is not restricted to Export/ Import processes.  I have noticed while running these tests using 'B2B'  that other processes appear to cause memory leakage over a period of time.  An example of one is 'Scan Disk' (although the leakage appears to be minor).  Also 'Scan Disk' will run in my B2 in something like 30 seconds to 1 minute on the 'B2' software but takes almost 5 minutes on the 'B2B' software.

Also, as can be seen from the table above, the data transfers rate while using the 'B2' versions is noticeably higher using the 'B2' versions.

I believe that the root cause of a number of issues reported to the forum is a problem with memory management within GNU/Linux 4.19.23.

I'm planning to have a look at what happens witin the B2 while ripping and compressing.

Regards,

Peter.

[David Drury]

I’ve been following this thread with interest. I appreciate all the work Peter is undertaking. Unless I’m miss reading it, I’m not sure about the conclusion that B2 versions of the software are working completely well for Export. I have an early 2TB B2 which is kept up to date with software changes (B2 version) and is generally well behaved. I haven’t done a full Export for a about three months so I don’t have full data available but I can say that my B2 has never managed a full Export in one go. It usually takes about three goes over about eight hours. As it’s incremental and seems to capture everything I hadn’t really thought of this as a problem! (I’m not an expert but I do go back as far as the Sinclair Spectrum so I’m always pleasantly surprised if anything I do with computers works perfectly!).
Just for information my B2 data are:
9943 tracks (7516 FLAC, 305 MP3, 2122 AAC)
766 albums
240 artists
213.89 Gb used
Backup I-O DATA 1TB HDD

[keithG]

Hi Peter,

I previously reported that 'Restore' with software B2  2.Jun.2020 ran ok. This is no longer the case. While recovering the B2 music to it's normal 'play' configuration, I've had two successive Restore failures after 3 hours, which represented about 110 Gb transfer point.  Reversion to software B2 13.Nov.2018 (17:10) resolves the problem. The difference could be use of Format HDD (earlier) against Delete Music (now), but this was not recorded. At some point I  noticed that Format HDD, does not clear the 'GB used' on the info display, so I used Delete Music instead.

The only anomaly between the two versions that may be of interest. At the point of failure (crash) with software B2  2.Jun.2020, and switching off/on and selecting info showed 0 albums/0 tracks and 113.9 Gb used. When the SD card with software B2 13.Nov.2018 was fitted, info showed 0 albums/0 tracks but only 108.7 Gb used. Different counting methods?  The attempted file transfer was 115.36 Gb.

This is an infrequent operation as Restore is incremental, but I need to retain a serviceable SD with this software version.

Please do NOT raise the criticism that I should be using Export. In short, using the Restore process allows me to control the default album play sequence, which is my main requirement. Playlists deselect and reset to the beginning when you power off.

Regards
Keith

[Peter Lowham]

Hi David,

Thank you for your information.  These data do provide me with even more food for thought and analysis.  I have keyed your data into my estimator spreadsheet and am looking at this to see if I can see any new possibilities that would account your situation.

Please could you post the exact version of software that you are using, as that is one of the newer factors that  I am taking into account?

With regard to the incremental nature of Export, that is true, the Export will keep incrementing using repeated runs.  The one issue, I have is that I have encountered, is potential Export file corruption when the Export process 'crashed'.  I found some 'Artist' folders that appeared to be OK, but in fact were empty.  That is one of the reasons that when I talk about file reconciliation, I am counting folders and files right down to file type level. An Export that is missing files is of no use to anyone and, even worse, it leads to a false sense of security.

Regards,

Peter.

[Peter Lowham]

Hi Keith,

Thank you for your update, again this information is most useful.  First of all, I'm not going to criticize anyone on the use of Backup/ Restore versus Export/ Import.  There is one basic fact which is both sets of functions which are provided and supported by Brennan and therefore should work correctly.  It doesn't matter which type you choose to use.

Anyway, back to your data.

You are now the second person (Keith also reported this earlier this morning) to report the problem while not on 'B2B' software.  I'm reprocessing this information now to see if I can further refine where the problem is tripping in.

I'm just starting to flash another SD card with the base 'B2B' code to test that version.  I'll report back when I have done a run or two.

Regards,

Peter.

[Brennan Support]

Hello Peter

Thanks for your tests and posting your observations - this is just to let you know that we are currently looking at this problem.

We have managed to replicate the problem and are currently trying to find out where the problem is  (basically is it the operating system or the app) and also testing potential workarounds (later kernel, different algorithms)

As you know the problem doesnt manifest itself untl around 120G has been transferred (several hours) so its rather slow to debug.

FYI the old B2 uses uClib and B2B uses glib as well as there being a more recent OS - so its not just a linux version change.

I'll let you know as soon as I have anything definite

Martin

[Peter Lowham]

Hi Martin,

Many thanks for your update.  I understand your difficulties with debugging the problem.  I've had a few 3-4am sessions just to watch the runs over the last week or so!

I've has two reports this morning which suggest that your point is valid; this is that it is not just the Linux version change after all. So I have just flashed another SD card with 'B2B Nov 06 2019' in order to run the Export/ Import tests again.

I'll keep you and Paul posted on the results of these tests.

Regards,

Peter.

[David Drury]

Peter
Currently my B2 is on software version B2 2 June 2020 but I haven’t done an Export with that version as I haven’t added much lately. My last full Export was on 29 February 2020 and it would probably have been on the B2 24 February 2020 version.
Regards, David

[Peter Lowham]

Hi Keith,

Here are a couple of observations that might help you.

Your first point regarding the use of 'Format HDD' vs 'Delete Music.  When you use 'Format HDD', the B2 database will not be aware of this action.  Therefore the reported 'Info' will be icorrect.  If you run 'Scan Disk' after the 'Format HDD' you should get a correct report from the 'Info' window.  When using 'Delete Music', this process will update the 'Info' table, and therefore will always be correct.

On your second point, I think that running 'Scan Disk' would have corrected the 'Info' window.  Regarding the difference in the 'Total's it looks like the 'Restore' did not complete fully in either run.  After the restore, do you do a file level count reconciliation between the 'Backup' files and the 'Restore' files? That is the only way to be sure that all files have been restored.

On point 3, you state the 'Restore' is incremental.  I have not yet had time to test 'Backup' and 'Restore' but the equivalent to 'Restore is 'Import'. 'Import' it is NOT incremental but operates in 'overwrite' mode. I think that 'Restore would also operate in 'overwrite' mode but I could be wrong on that assumption.  'Export' and 'Backup are incremental.

Regards,

Peter.  

[keithG]

Hi. Peter

I often compile Albums from downloaded mp3's and use 'Restore' to add them to the B2,I assure you that Restore is incremental in software version 13..Nov,2018 (17:10)..

Yes, I do a file count between the B2 display and a count on the pc (excluding toc)

Thanks for the format vs delete music info. I'm still learning the details...

I'm having real problems compiling this message as the message window has collapsed to half a line in height

Regards

Keith

[Brennan Support]

Hello Peter

I have prepared an offline software release - you can find a link to it here - with a description of some other stuff I'd been working on.

There appears to be a problem with disk caching and I was able to provoke an out of memory - albeit after six hours - from the command line - without the main B2 program running. So its an OS issue as you suspected.

The fix basically tells the OS not to cache during a backup and I was able to backup well beyond the 120G limit.

I wanted you to have an early opportunity to try it - I will roll it out as a web upgrade when I have completed some other work.

Martin

[Peter Lowham]

Hi Martin,

Thanks for your update.  I have downloaded and 'SW Upgrade'd your revised version (showing as 'B2B Jun26 2020 11:32:04'.

I have set up a fresh full Export (start time 13:55) and I will update you later on today as to progress.

It is good to know that you have reproduced the problem without the B2 application running, as I have been there many times over the years in my working life!  I did think that it was not an 'Export/ Import' issue because I was noticing other functions that were showing similar behaviour.

Thanks and regards,

Peter.

[Robert]

I have a large collection that I'd like to import to my B2. If I format the drive in the B2 and then remove the drive and connect it to my computer as an external USB drive can I just transfer the 16,423 files in one pass. It seems very time consuming to break up the import into 100MB groups, watching the time, etc due to the restrictions of the B2 at this time? Kudos to Peter to figuring out what was occurring. 

[Peter Lowham]

Hi Robert,

The following is only applicable if your software version is 'B2B <any date>'.  If you are on a software version of 'B2 <any date>.' you should be able to Import in one run.

I am testing a 'B2B' fix release right now that Martin sent me today. If the release is good, then Martin will incorporate that in the next 'Web Update'.

So, you can either hold off until the next Web Update is released or you can use the '100 GB rule' if you can't wait until then.

Currently your file 'Import' will fail at about 5000 tracks if using '.flac' format, or at 15000 files if using '.mp3' format, to give you some idea of estimates.

I hope this helps.

Regards,

Peter.

[Robert]

Hi Peter, My software version is B2B with latest update in June. At this point the collection is in WAV format and that is the format I'm importing to the Brennan. I'm not sure if WAV has more, less or the same restrictions as FLAC or MP3? 

Is my "other suggestion" an option, remove the B2 SSD and connect it as an external USB drive and dump everything at once? I'm not sure if the "import" function on the B2 allows the B2 to read everything property as opposed to just reinstalling the fully loaded SSD after moving everything onto it externally? Thank you again for your help with this issue.   

[Daniel Taylor]

Although I'm not 100% certain, I'm pretty sure that if you take out the SSD and copy to it from you computer, you can then put it back in the B2 and have success.  A few very important points though:

You'll need an enclosuer for the disk.  I bought one for around $12.

You need to arrange the folder structure exactly as the B2 expects to see it.  In the root, you need a folder named music, within which are all the Artist folders (and nothing else), within which are all the Album folders for each artist (and nothing else), within which are all the tracks for that album and the coverart.jpg file (and nothing else.  music\Artist\Album\tracks

If you have a Mac, I understand that it creates other files that correspond to the music track files.  Those will take up space and are not needed by the B2.  You can either delete them ahead of time, or, once the disk is back in the B2, run the Cleanup command (Maintenance menu).

Also, once back in the B2, run the Scan Disk command (Settings menu).

That ought to work.  Others please add anything I missed.

[Peter Lowham]

Hi Robert,

First off, the 'break down' volume size is 100 GB, not 100 MB.  Perhaps this was just a typo in your post.

Anyway, in answer to your queries, I have updated my 'Estimator' spreadsheet with the details that you have provided.  Here are my estimates (subject to variation, but should be reasonably good).

Based on '.wav' files which are much larger than '.flac' or '.mp3', each 'sub-volume' of 100 GB should contain approximately 2000 '.wav' files.  This would mean that you would need to break your collection down into 8 'sub-volumes', each one would take about 3 hours to complete.

Your second  query as to whether you could remove the SSD and use it as an external drive?  Yes, that is entirely feasible to do.  One of the beauties of the Brennans it that they are so 'open'.  All you need to do on the SSD is to create a directory named 'music' in the top directory (\) and copy your folders to there in the format of 

<Artist>\<Album>

Then when the copy is complete, plug the SSD back in to the B2, boot up the B2 and run 'Scan Disk' and the B2 will find the new music files.

Then you are ready for hours/days of listening pleasure.

Regards,

Peter.

[Peter Lowham]

Hi Robert,

On re-reading my own post, the SSD will already have the 'music' diectory on it when you use the SSD as an external device, so that is where you place your 'Artists\Albums folders.

So that makes it even simpler!

Regards,

Peter.

[Peter Lowham]

Hi Martin,

I have been running Export and Import tests over this weekend and can report back as follows.

Using 'B2B Jun 26 2020, and using my standard music data file of 218 GB, Export ran to completion successfully in 6 hours 55 minutes (with no work-arounds used).

In addition, I was watching the memory usage over this period (free -m) and the memory usage behaviour looked much more like it was on the 'B2' software, that is 'more like as expected'.  The B2 did NOT require a reboot after Export completion, which it always did on the previous releases of 'B2B' due to memory loss (used buffers sitting at 384/ 396 on the earlier B2B versions).

Next step was to run a full Import into the B2.  This is where the report is not good.  The Import ran for about 4 hours 38 minutes before it started to act strangely and after about 10 minutes of erratic behaviour, the B2 rebooted itself and sat at the main menu.  This behaviour is identical to the previous versions of B2B software.  The memory usage is also the same pattern as previous B2B releases (used buffers climbs steadily to almost max memory and the free+buffers/cache drops to near zero)

I ran Import twice with the same result each time.

So, in conclusion, Export will function correctly on this release, but Import will still fail after approximately at around 130 - 140 GB data volume.

Regards, 

  

[Mark Fishman]

Peter,

Martin wrote:

"The fix basically tells the OS not to cache during a backup"

I guess now he has to tell the OS not to cache during a restore...

:)

-- Mark F.

[Peter Lowham]

Hi Mark,

Yes, looks like that could be the case. All being well, we are nearing the end of these problems!  Judging from the number of posts on these issues, I think that the web posting will quieten down somewhat.

Regards,

Peter.

[Peter Lowham]

Hi All,

You have probably seen the updates with regard to Export and Import but I'll post the update here in any case.

The 'B2B Jun 29 2020' release has fixed the Export/ Import problems that were found over the last few months.  I have completed two full Export and Import cycles with my music collection of 218 GB volume and they ran perfectly.  The transfer rate has also improved; my early model B2 'Exported' at about 30 GB per hour (was 24 GB per hour) and the 'Import' ran at 31 GB per hour (was 26 GB per hour), so my Exports and Imports run for about 7 hours each.

In addition, the memory usage looks much improved, which I think will probably result in fewer lock-ups and hangs in general.

@Keith; I have completed my first full 'Backup'' using the same music volume as above, and it also ran perfectly.  It took 6 hours 20 minutes.  I'm just setting a full 'Restore' at the moment and I'll let you know how that goes, but I'm not expecting any problems.

Regards,

Peter.

[Peter Lowham]

Hi Keith,

The 'Restore' that I was running on my B2 has completed successfully; 218 GB restored in 7 hours and 34 minutes.  I believe that the 'Export/ Import' and 'Backup/ Restore' issues have now been resolved in the 'B2B Jun 29 2020' release of the software.

Regards,

Peter.

[Mark Fishman]

Peter, not to be a pest, but I assume that the b2.tar.gz file that contains the fix for the B2B versions also runs on the B2 versions of Brennan's software? I haven't bothered upgrading to the B2B kernel because I have no need for the Bluetooth stuff, and my motto has always been "if it ain't broke, don't fix it".

Thoughts?

-- m.

[PMB]

Hi Mark,

The release notes on the latest Software page - link - for 29th June 2020 does say it's for B2 and B2B versions.

Paul

Brennan Support.

[Peter Lowham]

Hi Mark,

WRT the 'B2' and 'B2B' versions of Jun 29 2020, I would think (but I don't know for certain) that each version of the release has the relevant fixes and updates applied. So the B2B version probably has all of the fixesand updates including the 'Export/ Import' fix applied whereas the B2 version will have the fixes and updates for all except the 'Export/Import' fixes, since the 'Export/Import' fixes are only applicable to the GNU/Linux version 4.19.

Regards,

Peter.

[Mark Fishman]

Peter, thank you for pointing me to the UK page for the software. While the US page links to the same file, the notes are missing from there, so it was good to read them at the UK page.

I imagine that it would be possible for a web upgrade to pull different files by identifying the existing version of Linux -- but there's only the one file available for download for anyone who wants to do an offline upgrade using a USB stick, or who might want to archive the software in case someday they'd like to go back to an earlier version. As you have shown, and as a few others here have expressed as a preference, there can be reasons for wanting a version of the Brennan software that isn't current.

I believe, therefore, that regardless of whether one is running a B2 or B2B version of the underlying kernel, the B2 application is the same for both.

Cheers -- m.

[Peter Lowham]

Hi Mark,

Yes, the B2 application is the same for both the 'B2' and 'B2B' versions. I do know that all of the latest enhancements that are in 'B2B' are also in the 'B2' versions, except for the Bluetooth update.  During the search for the 'Export/ Import' problem, I flashed six SD cards with different versions and dates of B2 and B2B releases.  The only high-level difference between any two parallel releases is the provision of the later version of Bluetooth.

I believe that the fix for the 'Export/ Import' issue was to disable disk caching in GNU/Linux 4.19.  The other fixes and updates would be the same for both versions of GNU/Linux (B2 and B2B).  Possibly, the B2 application makes a decision at boot time as to whether disk caching should be enabled or disabled.  This would make the 'Web Upgrade' file compatible with both versions.

Just postulating!

Regards,

Peter.

[Daniel Taylor]

Peter,

I think Martin said (on the Latest Software page) that 26 June 2020 software requires an "offline" update.  I interpret that to mean using the Etcher program.  And that implies a further update to the R.Pi OS.  Or am I confused?  Well, actually, I AM confused.  Just trying to get this straight.

[Peter Lowham]

Hi Daniel,

The 'Jun 26 2020' version did not require flashing via Etcher.  It was just a 'b2.tar.gz' file like any other 'Web Upgrade' file.  The reason that Brennan team put it there in 'offline' mode was to enable people like me to download and test the fix, while at the same time not allowing the fix to escape into the wild until the fix was confirmed as good. If the file was presented on the Web site in the normal way, then possibly hundreds of users could have downloaded the file via 'Web Upgrade' and not realise that they were on a test version of software. So the 'offline' presentation is a good way to present a test release.

So I just copied the offline file to a USB stick and then ran 'SW Upgrade'. 

Regards,

Peter.