Code 2 - "Memory nearly full" warning

[Iain Bell]

Hi,

I was editing some album names this afternoon via my PC (the B2 connected as an NAS), when the Web UI stopped responding. I checked the B2 and "Code 2" was showing on the display. I restarted the B2 and then as it scrolled through  after the details of no. of albums, tracks etc I got a message saying "Memory nearly full".

I searched this forum, but didn't find anything that could help in this instance given that I think I still have plenty of space on the HD.

According to the WebUI I have the following on the B2:

Software Version B2B Mar 22 2021 09:07:23

66941 tracks in 10076 albums 1799 artists

0 WAV 66941 FLAC 0 MP3 0 AAC

Capacity 2000.00G Used 1358.23Gb

Earlier in the year I had a nervous afternoon when I read on here that there was a limit of 10,000 albums, but then found out that this has been increased to 15,000 with the proviso that the no. of flac tracks doesn't exceed 100,000. I read also that long track titles can cause issues, but just randomly picking some of the ones which have been limited in length (by B2 or dBpoweramp?) it looks like the longest ones are about 80 characters long, which is well short of the 170 limit (if I've understood it correctly).

Would be grateful for any help in what I should do next to clear (if possible) the warning message. 

Best regards,

Iain

[JFBUK]

Hi Iain,

what software were you using to edit the filenames  and was  the "memory nearly full" message displayed by that software?

I remember that I ran into a memory issue when trying to edit a file on the B2 using Winscp .

The B2 can struggle if you are doing multiple things simultaneously.

I have never seen a "code 2" message on the B2  and there are more experienced brains on here that  will pick that up. 

What might help is if you can capture any debug information from the Web UI.

click the "cog wheel" then the "spanner/wrench" bottom middle on the settings pop up window.

Copy and paste the contents into a document you can post here

John

[Iain Bell]

Hi John,

Thanks for your suggestions.

I was actually editing the album names via the web UI (using the "Rename" option) and not having anything else going on (that I know about) in the background, so it looks to be a different problem....

I'm attaching the debug info you mentioned - I'm afraid it doesn't mean much to me and i'm not sure how far the log goes back, but hopefully they'll be someone who can decipher it and advise if there is something there ;-)

Best regards,

Iain

[Edwyn Corteen]

Hi Iain

You have lots of Playlist Bad entry errors in the log, first thing to do is go to Settings/Scan Disk and see if this fixes things for you.

[Iain Bell]

Hi Edwyn,

 Thanks for your suggestion, but have now run Scan Disk and still get the ”Memory nearly full” message.

 Best regards,

 Iain

[Daniel Taylor]

Hi Iain,

As I read your first message, my thought was that when they increased the album limit from 10,000 to 15,000, if there was code that checked for the previous limit, it may not have been changed.  So I don't think you need to worry.

[PMB]

Hi Iain,

Try closing the web UI and then clear browsing data ('cookies and other site data' and 'cached images and files' - for Chrome). Open a new browser page and enter the IP Address manually.

There is a 'Free Space' utility in the Maintenance menu which will give a more accurate figure for the 'used' amount - check the Stats after running it.

Paul

Brennan Support.

[Iain Bell]

Hi Paul,

I tried what you suggested, but no luck. Late last night I played around a bit more and was hoping that as Daniel suggested the warning might have been left over from a previous limit, but when I tried adding ca. 2.5 GB of new albums via a USB stick I got the same warning, about a third of the way through before the import came to an abrupt end - checking via Windows Explorer about 30% of the USB stick got transferred. (Interesetingly, though, just prior to attempting the import I edited some existing album titles via the NAS route with Windows Explorer and it seemed to work - my attempts at doing the same thing earlier yesterday with the Web UI were what caused the original Code 2 warning to appear.) 

After running Scan Disk I then get the following:

Software Version B2B Mar 22 2021 09:07:23

0 tracks in 0 albums 0 artists

0 WAV 0 FLAC 0 MP3 0 AAC

Capacity 1953.04G Used 1359.31Gb

USB 0 tracks in 0 albums 0 artists

I ran Free Space, but apart from the red light flickering for some moments the display just reverted to the intro with the scrolling details of Network etc. The Web UI now shows no music even though 1354.31GB are used.

Best regards,

Iain

[Peter Lowham]

Hi Iain,

The following is supposition mainly, but here goes.

The 'Memory nearly used' message might be a reference to the management of the B2's internal 'b2db' database system which is held in the B2's main memory (and is not connected to the amount of HDD space).

There is an indication of this in that your B2 is showing '0 wav, 0 flac, 0 mp3, 0 aac'.  This information is built from the 'b2db' file.  If this file becomes corrupt, then the B2 will not 'know' what is in its music collection.  

I would run 'Scan Disk' first to see if the figures above have changed.  

To have a look at 'b2bd'  could you procure a USB stick (a small stick will do) and run 'Settings' --> 'Maintenance' --> 'b2db to USB C' 

This will send the 'b2db' file to the USB stick.  If you then take the USB stick and plug it into your PC/Mac/Chromebook.  First of all, look at the size of the file and post that here.  Then you can open the file with a text editor (I use 'Notepad++) and scroll down to the last line in the file and post the line number here, then we can see if that might be part of the problem.

Alternatively, if you want, you can post the 'b2bd' file here as an attachment, either option is fine.

Regards,

Peter.

[Iain Bell]

Hi Peter,

Thanks for your suggestion.

After running Scan Disk again I now get the following:

Software Version B2B Mar 22 2021 09:07:23

0 tracks in 0 albums 0 artists

0 WAV 0 FLAC 0 MP3 0 AAC

Capacity 2000.00G Used 0.00Gb

USB 0 tracks in 0 albums 0 artists

(I checked via the NAS route and the files are still visible)

I'm attaching the b2db.txt file - it doesn't seem to have much info :-(, but maybe will give an indication of what has gone wrong.

Thanks again for your help.

Best regards,

Iain

[JFBUK]

Iain,

there is a newer release of the B2 software, May 12th.

I would run a web upgrade and install that to see if it has any impact on your problem.

John

[Iain Bell]

Hi John,

Thanks. Just upgraded, unplugged and restarted the B2, but still showing contents (or lack of!) as above.

Best regards,

Iain

[JFBUK]

Hi Iain,

I am a mine of ideas, unfortunately not many successful :(

1. Have you tried backing up your HDD to a USB drive ?

2. In your original post you said you were connected to the B2 as a NAS ?

Is this still working ?

If these work it might give some indication to the health of the HDD

John

[Peter Lowham]

Hi Iain,

I have just looked at your 'b2db' file and I believe that it confirms my thought that you have hit either an unknown limit or a bug in the 'b2db' database management within the B2.

My estimate for your collection is that 'b2db' file should contain just over 87,000 lines of album and track names, and would be about 2,400,000 bytes in size.

Your 'b2db' only contains 2 lines, both of which are the header lines, with no music tracks listed.

So, because of this limit/bug, your B2 is no longer able to rebuild the 'b2db' file and this is why your music collection is not showing in the WebUI.

The way to get around this would be to delete something like 20 albums (temporarily), run 'Scan Disk' again and then wait for a fix to be produced.  Do you have NAS access set up to your B2?

I also reckon that your music collection is sitting on the HDD, probably is OK, but the B2 cannot see it until the limit/bug is fixed.

Do you have a recent 'Export' near to hand?

Regards,

Peter.

[Peter Lowham]

Hi Iain,

I've just realised that you have a NAS connect to your B2, my apology for missing that.

So if you can copy say 20 or so CDs from the B2 to your PC/Mac for safe keeping, then delete those from the B2, then run 'Scan Disk' your B2 should be back in operation.

The 'b2db' file is stored on the B2's HDD but when the B2 is booted up, this file is read into the B2's main memory and kept there to enable the B2 fast access and reference.  This is probably where these error/ warning messages are coming from.

@Paul; I think that this issue is one for you guys now!

Regards,

Peter.

[Iain Bell]

John, 

any ideas are gratefully received ;-) - I had forgotten about updating the software when I read about it a few weeks ago, so it was good to do be reminded! I have a full backup from 18th May and copies of what I've transferred since then so hopefully I won't have to re-rip anything 

Peter,

 
I cut out about 4GB of CDs from the B2 (pasted it on an external HD), ran Scan Disk on the B2 and now have the following:

Software Version B2B May 12 2021 11:36:30

66741 tracks in 10068 albums 1801 artists

0 WAV 66741 FLAC 0 MP3 0 AAC

Capacity 2000.00G Used 1351.08Gb

USB 0 tracks in 0 albums 0 artists

So, definite progress! :-)  - I still get the Memory nearly full message scrolling up so I guess I'm not out of the woods yet.....

I exported a copy of b2db and that has a size of 6,233 kb - on my version of Notepad there are no line nos. as such, but if I open the status bar once I've highlighted the music files then it mentions a figure of 86880, which would seem to line up with your prediction (allowing for the moved files).

It's great to be back in business again, but I'm assuming I'll need to take break from transferring music across to the B2 until the Brennan team can advise what I should do next so that I don't end up with the same limit issue again.

Thanks again to you and everyone who made suggestions!

Best regards,

Iain

[Peter Lowham]

Hi Iain,

Well, that is good news!!  I did the lines calculation on my 'Estimator Spreadsheet' and the answer is 86,879 lines, so a differnce of 1 there, so that looks good.  The overall size of 6,233KB is over 2 times my expectation, but this is a new estimate for me as I have never seen this problem before.  I only added this new estimation in this morning.

If you are OK with this could you post 'b2db' here as an attachment?  If not, don't worry about it.  I would run through it to understand the size difference between the actual and the estimate for a better understanding.

Regarding your current situation, I would not rip any or CDs or upload any more music.  There is obviously a limit or bug that you have hit, and until it is fixed, you will just hit the same problem again.  You collection is the largest that I have data for, with only one other forum user following close behind, so that would explain why we haven't seen this problem until now.

The other query is regarding your 'bad playlists' messages in the debug log.  It could be that these messages are related to the 'out of memory' message.  So let us know about that sitation.

Keep us posted!

Regards,

Peter.

[Jeff. M.]

If these were ripped to flac and MP3 would not quite a bit of the disc be taken up with mirror MP3 files. I don't think this shows on the info panel, does it? Sure someone will tell me otherwise.

[Iain Bell]

Hi Peter,

No problem re the b2db file - please see attachment. Thanks once again for your help with this!

Paul,

I guess I should wait to hear back from you and the Brennan team to see what can be done?

Best regards,

Iain

PS Peter, when I try to send the message with theb2db attachment I get an error message, so I'll mail it to you separately and see if that works.

[PMB]

Hi Iain,

I have asked Martin about this issue. Please don't add or delete any music or try to fix it before we get back to you.

Paul

Brennan Support.

[Iain Bell]

Hi Paul,

Thanks - no problem to leave the transferring of music for a bit - it gives me a chance to have a break and listen to some of what I've already done ;-) If you need the b2db file let me know and I can mail it.

Best regards,

Iain

[PMB]

Hi Iain,

Please send the b2db file to -   thebrennanb2    @    gmail    .   com   - no spaces.

Paul

Brennan Support.

[Peter Lowham]

Hi Iain,

I have downloaded your 'b2db' file and have looked at it in some detail.  I now understand why your file is so large.

The bottom line is that your artist/ album, and track name lines are each much longer than 'normal'.  On average each line in your file is 80 - 85 characters long, whereas my b2db file averages about 35 chararacters per line.

So your file is currently 6,381,897 bytes in size with 86,879 lines of data, and this looks OK in terms of content and structure.  As an example, my file is 20267 lines and 645,000 bytes.  

@Paul; Iain's current 'b2db' file still functions correctly albeit with warning messages, but it is almost 6.5MB in size.  At some point, when the record collection increases by say 10 or 20 CDs, 'Scan Disk' will fail to run properly and the 'b2db' file will only contain the first two header lines and no music data.

Regards,

Peter.

[PMB]

Hi Iain,

Martin suggests having a look at the debug monitor when you see the 'code 2' message or try anything, like Scan Disk, that fails - web UI >> COG tab >> spanner tab - there will be a lot of stuff in the debug as it starts running at boot up. Only the later sections will be relevant here.

Paul

Brennan Support.

[Iain Bell]

Peter,

Yes, I noticed when I saw that b2db file on screen for the first time that some of the tracks actually went up to 120 or so characters - these come from the various databases which dBpoweramp offer when I rip via that - sometimes they are even longer on the databases and I've noticed that they get cropped when saved on my usb stick - I had presumed this was a limit that dBpoweramp or Windows set. Album titles tend to be longer as well as I've tried to come up with a method of being able to classify my collection along the lines that the old Penguin Guide to CDs used i.e. composer - music type (e.g. orchestral/chamber/instrumental/vocal/opera) - performer. (The music type was an afterthought, so the b2db file has a mixture of album title style - I'm classifying all new rips that way and will at some stage have to go back and add that to the first 3 or 4 thousand albums I created :-( - something for the long dark winter nights!)

Paul,

I'll send the b2db file in a minute or two - re Martin's suggestion, do I take it that I should transfer back some of the files I moved yesterday to force the B2 to display "Code 2" and then send across the debug monitor? At the moment I guess I'm under whatever limit is causing the problem as the B2 is behaving well ;-)

Best regards,

Iain

[Mark Fishman]

If the Raspberry Pi has 512MB of RAM, why is it running out of memory because of a 6.5MB file?

(a) Linux is normally very good at managing a swap file

(b) is there really a need to keep the b2db file in RAM?

(c) if there's a HDD, maybe swap space should be on the HDD instead of the SD card, to reduce wear

(d) because of the COMPLETE LACK of tag support, classical music on a B2 almost demands the creation of long album and file names.

Really, this is a significant, if "only" potential, problem for many of us.

On Friday, May 28, 2021 at 5:30:37 AM UTC-4 peter....@gmail.com wrote:

Hi Iain,

[PMB]

Hi Iain,

Currently if you run Scan Disk do you get 0 Tracks, etc? If so the last section of the debug will be useful.

I suppose you will need to get the B2 to fail in order to glean useful info from the debug.

Paul

Brennan Support.

[Iain Bell]

Hi Paul,

Following Peter's suggestion last week I removed a large folder from the HD and must have brought myself under whatever the problem limit is, so that things were back to normal (Pls see details of what the B2 showed after ScanDisk in my post of 27th May 1601). I've now tried adding that folder back and got a "Code 2" message as expected - I'm attaching the debug log as an attachment. I've also run Scan Disk following the attempted transfer and now it shows 0 tracks etc again.

Best regards,

Iain

[Mark Fishman]

Right at the bottom of the debug log are these lines:

Track name space 5428476/6500000 

 Album name space 649984/650000 

 Artist name space 47938/200000 

 memoryStatusMessage () = Memory full

The "Album name space" is the one that seems closest to its limit (I'm interpreting the number before the slash as the actual space used, and the number after the slash as the allocated space).

It would be interesting to see what happens if you were to assign a bunch of existing tracks to an existing album and then remove the (now-empty) album folders, thus reducing the number of albums without reducing the number of tracks (or the length of tracknames). Just an experiment, you understand.

-- m.

[Iain Bell]

Hi Mark,

I tried your suggestion and instead of showing 0 tracks etc the display now seems to show once again the expected no. of tracks etc. Progress! ;-) It would be interesting to hear if the track and album name space limits are absolute or if the Brennan team are able to increase them,  if these are what are actually causing (or could cause) problems with large collections.

Thanks for your suggestion!

Best regards,

Iain

[PMB]

Hi Iain,

So it does appear that you are hitting the limit for Album name memory. I'll ask Martin B if it can be increased or if there is another work around.

Paul

Brennan Support.

[Iain Bell]

Hi Paul,

Thanks - can you also check if the Track name space limit could be tweaked as well - I've still got a few hundred CDs to rip before I'm done  - I think I should be OK, but a bit bigger safety margin would help ;-)

Best regards,

Iain

[PMB]

Hi Iain,

Martin has released a software update increasing the Album title memory space. Please try the new software and let us know ow it goes.

I'll have to ask about the Track space.

Paul

Brennan Support.

[Mark Fishman]

One of Linux's virtues is its memory management. Surely it should be possible to allocate these arrays dynamically? at least until one really does run out of (all) memory.

I understand that the RPi allocates some amount of RAM for graphics used. Given the Really Small Display that a B2 has to drive, I'd imagine that the smallest possible graphics allocation (64MB?) ought to be sufficient -- the web UI isn't part of local display memory. That leaves 448MB RAM for the OS, the B2 application, and the RAM copy of the b2db. It's none of my business, really, how much RAM is needed for each bit, but a paging file on the HDD would allow for virtual memory also, if 448 isn't enough.

Ah, well. I'm rambling again. Thanks for the update --  m,

[Iain Bell]

Hi Paul,

This was great news! I have this evening transferred another 20GB, run Scan Disk and all seems to be well!

Out of curiosity how much has the Album name memory space been increased by?

Best regards,

Iain

[PMB]

Hi Iain,

Glad to hear the update has worked. Martin said he increased it from 650k to 900k.

Let us know if you get near the Track limit.

Paul

Brennan Support.