"Looking for New Songs" ... for 12+ Hours

[Daniel Faigin]

Whenever I open Rocket Player (or resync via iSyncr), it starts up the "Looking for New Songs" process ... which takes forever (over 12 hours). It eventually completes overnight, but why does it take so long, especially when there usually are either no new songs (I only sync monthly due to the time it takes with 49K+ songs in the library), or under 100 new songs (I typically get 3-4 albums at a time, and it is new music + podcasts that get downloaded)?

Daniel

[Justin (JRT)]

Hi Daniel,

I am working on some scanning issues for the next release of Rocket Player. Please email Music....@jrstudio.com and let's get a support ticket going and get you in the beta group to gather more information on this.

[Daniel Faigin]

Just letting you know that the latest release DID NOT fix this problem. Yesterday, I did a periodic iSyncr, which did it's media stuff for 2 hrs afterwards (why that takes so long is another issue). I then started up Rocket Player, and it started "Looking for new songs". Over 20hrs later, it was still looking, so I just rebooted the phone. Even though I have a LOT of songs (over 49K), it shouldn't take that long. So there is still some bug there. Note that it seems to find the new songs farely quickly (within an hour the live lists seem to work right); it just never ends the searching. And, so you know, I have both iSyncr and Rocket on the list of apps that are always awake.

[And, before you ask, given I'm in cybersecurity, I don't load apps that aren't in the store, so loading the beta to test this is a non-starter.]

Daniel

[Justin (JRT)]

My guess here is that the Android Media Database is getting overwhelmed by a large library. What we should try now is putting a ".NOMEDIA" file in the "syncr" directory where all the music files are. This will tell the Android OS to stop scanning in all this music, and basically get out of our way. iSyncr and Rocket Player will be slow and have issues if the android media database is hanging or super slow.

[Daniel Faigin]

How do you create that file? What else would that impact (the response seems to imply that "Looking for New Songs" is coming from Android, and not Rocket)

[Justin (JRT)]

Great question. You create a ".NOMEDIA" file using a file explorer app. It can be any file of any size, it just needs to be named ".NOMEDIA". Also, before you create it, you'll want to go into Rocket Player's library settings and turn on "Ignore .NOMEDIA files". This will cause Rocket Player to scan your music files and the Android media database to leave us alone.

As Rocket Player's scanner scans both your files and the Android Media Database, I'm absolutely implying that the Android Media Database could be what is slowing us down, especially after the initial scan.

[Daniel Faigin]

I did create the file ( https://solutionexist.com/create-nomedia-file-android/ ) is useful for this. I'm not sure it solved the problem. After doing all the steps, it still has been "searching for new songs" for over 2 hours... It shouldn't take that long.

[Justin (JRT)]

Unless all the music disappeared, had to be re-imported, and you have a large library, it shouldn't take that long.

To make sure we've created the ".NOMEDIA" file properly, and the system scanner has had time to run and removed all music, try using another music player that doesn't have a configurable scanner like Rocket Player. Make sure that music player doesn't show any music in its songs tab.

[Daniel Faigin]

> Unless all the music disappeared, had to be re-imported, and you have a large library, it shouldn't take that long.

No music was added or deleted; just the .nomedia file was created.

> To make sure we've created the ".NOMEDIA" file properly, and the system scanner has had time to run 

> and removed all music, try using another music player that doesn't have a configurable scanner 

> like Rocket Player. Make sure that music player doesn't show any music in its songs tab.

(1) I confirmed the .nomedia file was created by copying it again, and the file manager informed me the file was already there.

(2) Amazon Music had previously shown the music in the offline library -- it still shows. Note that it never showed all the music on the SD card.

(3) YouTube Music shows nothing on the SD card, then but again, it never found anything on the SD card before. 

Given that the "Looking for New Music" message is initiated by Rocket Player and associated with Rocket Player in the notifications, I think it could be that the search algorithm used may have trouble with that many songs. I had that happen once with a programming contest problem in the 1980s: did great on small data, but with a large data file it just took forever because we had forgotten some tests.

Hopefully we can get this resolved before December; if we can't, I'll be glad to show you when we're out in Austin for a Computer Security conference then.

Daniel

[Justin (JRT)]

From your viewpoint, no music was added or deleted. Unfortunately, there could be an issue where the OS files command wrongly reports no files, we remove them all then, then on the next scan correctly reports the file and all the files come back. A weird bug like that would cause Rocket Player to have to re-import everything, reading every tag again, which is the only legitimate reason a rescan should ever take two hours.  (Okay okay, weird things like being configured to scan 1 million empty folders would cause us to be that slow too. :-D  )

Perhaps the database is getting deleted and we have to re-import? Does Rocket Player appear to be re-finding songs when it rescans?

Do you have "Rescan Art (Slow)" turned on?

Can you verify no music shows up in this music player? https://play.google.com/store/apps/details?id=com.jrtstudio.music

If any music shows up, try rebooting the phone and waiting a few minutes for the Android media scanner to remove all the music.

[Daniel Faigin]

On Wednesday, March 31, 2021 at 8:36:41 AM UTC-7 Justin (JRT) wrote:

From your viewpoint, no music was added or deleted. Unfortunately, there could be an issue where the OS files command wrongly reports no files, we remove them all then, then on the next scan correctly reports the file and all the files come back. A weird bug like that would cause Rocket Player to have to re-import everything, reading every tag again, which is the only legitimate reason a rescan should ever take two hours.  (Okay okay, weird things like being configured to scan 1 million empty folders would cause us to be that slow too. :-D  )

Perhaps the database is getting deleted and we have to re-import? Does Rocket Player appear to be re-finding songs when it rescans?

Not that I've noticed, but I haven't looked too closely (my primary music player is my iPod Classic; Rocket Player is the backup)

 

Do you have "Rescan Art (Slow)" turned on?

No. I'll note that this time, when I started Rocket Player, it didn't start the "scanning" (unlike every other time). So we'll see what happens the next time I iSyncr / Rocket.

Can you verify no music shows up in this music player? https://play.google.com/store/apps/details?id=com.jrtstudio.music

If any music shows up, try rebooting the phone and waiting a few minutes for the Android media scanner to remove all the music.

I'm going to wait on that -- got a lot to do today, and I'm always hesitant on adding more apps. 

[Daniel Faigin]

I happened to go back into the Rocket app: ... and it started the "Looking for new songs" again. As a point of information: It does the cybor-back and forth for a few minutes, and then the progress bar starts. It goes to 100% fairly quickly. It then sits at 100% for hours (sucking battery).

Hope that helps you figure out what I'm seeing.

[Justin (JRT)]

Cybor-back and forth means scanning all directories looking for songs.

Progress bar means importing all songs into the library

100% but not done means... importing playlists from the Android media database and backing up information to the cloud.

If you are a cloud subscriber. Try going into airplane mode and re-scanning. If that goes fast, the bug going to be in cloud backup. If you aren't a cloud subscriber, then I'm 90% sure the Android media database is hung/slow and ".NOMEDIA" should fix this once it is working properly.

[Daniel Faigin]

I'm not a cloud subscriber, so I'll reboot the phone, give it a day (as the .nomedia file is in the appropriate SD card directory), and see if it is faster to start up Rocket Player.

[Daniel Faigin]

I thought I would update this. As you recall, we had it only looking in the SYNCR directory, and had .NOMEDIA appropriately placed. It had gotten down to a reasonable time period.

Yesterday, I did my monthly update of my phone's music. Added over 200+ tracks to the phone, bringing it to 49,400 something. iSyncr didn't take too long, nor did its media rebuild afterwards. However, Rocket Player. ... progress bar to 100% in about 40 minutes. It's then been that way for over 9 hours.

This shouldn't happen every time I add new music and resync. So, in addition to the above, I think there's something in the algorithm that isn't recognizing "done". Perhaps adding in a background timer, so that if it runs from longer than expected, it just declares it to be done and closes things done cleanly.

I'm going to reboot the phone and see if that solves the problem.

[caro...@jrtstudio.com]

Oh no! Was the .NOMEDIA file still in the same folder?

If it is getting stuck still, we need to get a dev log when it is stuck.

Here's a tutorial on how to take a developer log: https://www.youtube.com/watch?v=YxQ3NTloD2c

If you don't mind, bring up developer options on your device, "Settings". To do this, go to "Settings - About (Phone) - Software Info" and select "Build Number" a total of 5 times.

Once you do this, please open the new setting "Developer Options", select, "USB Debugging," then "Take bug report (Power menu)". If this option isn't available then a "Take a bug report" setting should be. This "Take a bug report" button will also take a bug report, but requires navigating back to this screen while Rocket Player is frozen.

Now, when we press and hold the power button on your device, you should get an option to take a bug report. This will take a snapshot of your device, package them up and prepares them for sending.

It takes a minute or two after you tap it, and you'll see a notification once it's ready. Tap that notification to launch an email program where you can send this report to Music...@jrtstudio.com. Just details in the Subject line that this is associated with the forum post.

*Note the report may take as long as four (4) minutes to create.

[Daniel Faigin]

No change was made in the .NOMEDIA file from before. It was there before (which it was), I didn't make any changes to it.

As for the bug report: After rebooting the phone, it is no longer doing the "searching for new media" when I open Rocket Player. So I'm not sure what it would report. The issue happened after an iSyncr run,  then the first open of Rocket Player with a lot of new media files. I suspect that there's some end condition for the search loop that isn't being met for some reason -- that's why I suggested at watchdog timer that just kills the search after it runs for a configurable amount of time. One could always restart the scan if some file hadn't been found. That may be the easier way to solve the problem.

Otherwise, I can save these steps, and try again at the beginning of June, as I typically do the iSyncr updates monthly due to the time it takes (I update the iPod Classics daily; Rocket Player is the backup).

Daniel