transcoding and Apple Lossless (Pi, service)

[Martin Pergler]

Hi there. I’m a new user. Thanks to Mark and the other developers.

 

Am setting up to run sonspy on my Pi, serving up multiple customized indexes to parts of my music library, stored on a NAS.

So far: able to scan and run in WMP server mode, no problems. Thanks to Bart for the installation .docx (I’m happy to write up what I needed to change to run on the pi once I’m done).

 

But: having transcoding(?) problems trying to run as SMAPI service.

.mp3 and .flac files work fine, but I have lots .m4a files with Apple lossless inside. Those choke with Sonos telling me “this song is not encoded properly”. Those files do play fine when Sonos goes straight to the NAS (without sonospy at all) or with sonospy in WMP server mode, and they’re normal 44k/2 channel CD rips.

 

I have installed a version of ffmpeg compiled with codecs included and can manually convert the .m4a files to .flac, or even to .mp3 with the precise command line that transcode.py seems to use (line 152). Sonos/sonospy –s is happy to play those manually converted .mp3 or .flac files. So it doesn’t seem to be purely a ffmpeg problem.

 

Any ideas?

 

I used to be an inveterate computer tinkerer, but that was 20 years ago in grad school. So this is a crash course in Pi, Debian, Python all at once –please be gentle!

 

Thanks

 

---

Martin Pergler

m...@pergler.org

 

[Mark Henkelis]

Hi,

My first thought is that maybe the Pi isn't transcoding quickly enough, though I would expect a different error for that. Is the folder containing the ffmpeg executable on the PATH when Sonospy is running (are you running against the version with codecs included)? I also thought that .m4a was lossy, not lossless (though I don't really know much about it).

I need to do some tests to see what formats the streaming interface currently supports (in theory it supports less formats than native Sonos and WMP) - it does support FLAC, but it doesn't claim to in the documentation.

If you can send me an example file I can do the following tests (you could do some of them yourself):
1) running Sonospy on a Linux PC, with ffmpeg installed appropriately
2) transcoding to FLAC instead of mp3 (changing the output file type in transcode.py)
3) removing the m4a transcode (from smapitranscodetable_extension) to see whether it can be played natively
4) check the actual content of the m4a file

I can also turn debug on and check that transcoding is working properly.

Mark.

--
You received this message because you are subscribed to the Google Groups "Sonospy Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonospy-deve...@googlegroups.com.
To post to this group, send email to sonosp...@googlegroups.com.
Visit this group at https://groups.google.com/group/sonospy-devel.
For more options, visit https://groups.google.com/d/optout.

[per...@gmail.com]

Thanks Mark. I will send you an example .m4a file privately.

For info, .m4a is a container which may contain lossy Apple AAC data, or can contain Apple Lossless ALAC data. ALAC is similar to FLAC, except FLAC is open source but not supported by Apple devices or itunes. Both are supported, even encouraged, by Sonos natively.

I've tried the following, keyed to your questions

1. Don't have a Linux PC unfortunately, but ffmpeg on my pi should be fine -- see output below. 

pi@chickadee ~/LucyStorage/chickadee/sspy $ ls -l `which ffmpeg`

-rwxr-xr-x 1 pi pi 15009696 Dec 22 13:57 /usr/local/bin/ffmpeg

pi@chickadee ~/LucyStorage/chickadee/sspy $ ffmpeg -formats | grep -e "m4a\|mp3"

ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers

  built with gcc 4.8.3 (crosstool-NG linaro-1.13.1-4.8-2014.01 - Linaro GCC 2013.11) 20140106 (prerelease)

  configuration: --enable-cross-compile --cross-prefix=/home/pi/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf- --arch=armhf --target-os=linux --prefix=/home/pi/ffmpeg/build --enable-gpl --enable-libx264 --enable-libmp3lame --enable-libfdk-aac --enable-nonfree --enable-libaacplus --extra-cflags=-I/home/pi/ffmpeg/build/include --extra-ldflags=-L/home/pi/ffmpeg/build/lib --extra-libs=-ldl

  libavutil      54. 31.100 / 54. 31.100

  libavcodec     56. 60.100 / 56. 60.100

  libavformat    56. 40.101 / 56. 40.101

  libavdevice    56.  4.100 / 56.  4.100

  libavfilter     5. 40.101 /  5. 40.101

  libswscale      3.  1.101 /  3.  1.101

  libswresample   1.  2.101 /  1.  2.101

  libpostproc    53.  3.100 / 53.  3.100

 D  mov,mp4,m4a,3gp,3g2,mj2 QuickTime / MOV

 DE mp3             MP3 (MPEG audio layer 3)

2. Tried that (adding a .m4a.flac mapping and copying over and modifying the block around l.150 in transcode.py) -- no change (from debug it is correctly going down the .m4a.flac tree but Sonos doesn't like the output).

3. Also no luck. Removed it so sonospy tried to serve it natively; the Sonos controller greyed out the track as being an invalid format. (Though Sonos can read it natively from the NAS without sonospy, or with sonospy -w)

4. Sending to you, but here's the (edited) output of a manual ffmpeg run on one file:

pi@chickadee ~/LucyStorage/chickadee/sspy $ time ffmpeg -i /home/pi/LucyStorage/music/Adele/19/01\ Adele\ -\ Daydreamer.m4a -q:a 0 -map a -f mp3 test.mp3

ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/pi/LucyStorage/music/Adele/19/01 Adele - Daydreamer.m4a':

  Metadata:

    Source          : CD (Lossless)

    Encoder         : Apple Lossless

  Duration: 00:03:40.51, start: 0.000000, bitrate: 739 kb/s

    Stream #0:0(eng): Audio: alac (alac / 0x63616C61), 44100 Hz, stereo, s16p, 731 kb/s (default)

    Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 750x750 [SAR 96:96 DAR 1:1], 90k tbr, 90k tbn, 90k tbc

Output #0, mp3, to 'test.mp3':

Stream mapping:

  Stream #0:0 -> #0:0 (alac (native) -> mp3 (libmp3lame))

size=    6423kB time=00:03:40.52 bitrate= 238.6kbits/s

video:0kB audio:6422kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.013108%

real    1m1.594s

user    1m9.320s

sys     0m0.760s

*** Not sure if that's a decent processing speed for streaming to Sonos?

*** Finally, here's debugging output From pycpoint.log (running with -mtranscode -mproxy -mmediaserver)

DEBUG    transcode   : 111:  transcode() /home/pi/LucyStorage/music/Adele/19/01 Adele - Daydreamer.m4a

DEBUG    transcode   : 112:  transcode() m4a.mp3

DEBUG    mediaserver :2815:  processQueryTrack() row: (u'834607b567d453744b3d3db91683011d', u'a47f4cdce3efe9501d407b3b26700fa9', 0, u'Rolling in the Deep', u'Adele', u'Adele', u'21', u'Singer', 1, 734161, u'Adele', u'Adele', u'', u'', u'MPEG-4 audio', 229, 27695058, 1393695533, u'/home/pi/LucyStorage/music/Adele/21', u'01 Adele - Rolling in the Deep.m4a', 1, u'', u'/home/pi/LucyStorage/music/Adele/21/Folder.jpg', None, 0, 0, 0, 0, u'audio/mp4', 1409969930, 53, 0, 1456187551, u'', u'', 1456187551, u'', u'')

DEBUG    transcode   :  59:  checksmapitranscode() m4a

DEBUG    transcode   :  60:  checksmapitranscode() 0

DEBUG    transcode   :  61:  checksmapitranscode() 0

DEBUG    transcode   :  62:  checksmapitranscode() 0

DEBUG    transcode   :  63:  checksmapitranscode() 0

DEBUG    transcode   :  64:  checksmapitranscode() MPEG-4 audio

DEBUG    mediaserver :4373:  querymetadata() end: 1456339094.171

DEBUG    proxy       : 865:  processMediaMetadata() 1

DEBUG    proxy       : 866:  processMediaMetadata() [('Tsingletrack__0_0_0__834607b567d453744b3d3db91683011d', u'Rolling in the Deep', 'audio/mpeg', u'http://10.0.0.17:10243/WMPNSSv3/Nonclassical.db.834607b567d453744b3d3db91683011d.m4a.mp3', 'track', 'track', ('', u'Adele', '', '', '', u'21', u'http://10.0.0.17:10243/wmp/Nonclassical.db.53.jpg', '', u'Adele', '', '', 229))]

DEBUG    proxy       : 874:  processMediaMetadata() MEDIAMETADATA ret: http://10.0.0.17:10243/WMPNSSv3/Nonclassical.db.834607b567d453744b3d3db91683011d.m4a.mp3

DEBUG    proxy       : 660:  soap_getMediaURI() SMAPI_GETMEDIAURI ret: {'{http://www.sonos.com/Services/1.1}getMediaURIResult': u'http://10.0.0.17:10243/WMPNSSv3/Nonclassical.db.834607b567d453744b3d3db91683011d.m4a.mp3'}

After this gives an error message on Sonos, I noticed ffmpeg does seem to (have been) running:

pi@chickadee ~/LucyStorage/chickadee/sspy $ ps -f

UID        PID  PPID  C STIME TTY          TIME CMD

pi       22962 22961  0 13:08 pts/0    00:00:01 -bash

pi       23514     1  1 13:37 pts/0    00:00:08 python pycpoint.py -p -sSonospy=

pi       23610 23514  0 13:38 pts/0    00:00:00 [ffmpeg] <defunct>

pi       23746 22962  0 13:51 pts/0    00:00:00 ps -f

[Mark Henkelis]

Thanks.

I've done some investigating, but no luck so far. I get the same error on a Linux PC, so that should rule the Pi out. I get the same error when I create an m4a from an mp3 file, so that should rule out your rips.

I'll probably try a different decoder next - I guess it's possible that some tag data is not generated until the end of the transcode process, that Sonos needs up front.

Mark.

[per...@gmail.com]

Much appreciated, and overall would be great to get it to work. However, if it proves too much of a hassle, I could run a big batch job and convert all the ALACs to FLACs when I otherwise get set up properly. When I started ripping my CD collection, there was no clear winner between ALAC and FLAC as lossless formats. I chose ALAC since could also be read by itunes. It seems these days FLAC is more standard; if I were starting again that's what I would use, just up to now has not been a compelling reason to switch.

Found the following just now, which would seem to indicate that in another context lame can be used to transcode ALAC to mp3 on the fly. Might be another option? Can explore later this week - my current version of lame only takes wav input, so would need to explore how to get it to read .m4a.

http://blog.kollerie.com/2010/02/25/logitech_squeezebox_on-the-fly_transcoding_apple_lossless_to_mp3/

[per...@gmail.com]

Interesting:

"ffmpeg -i infile.m4a -q:a 0 -map a -f mp3 outfile1.mp3" (as one would type on the command line) and "ffmpeg -i infile.m4a -q:a 0 -map a -f mp3 - > outfile2.mp3" (piped, as transcode.py does) generate different output. Slightly different length files, though both play through my pc speakers fine (not near my Sonos today so couldn't check if Sonos likes both of them too) and their header is the same. 

[Mark Henkelis]

It looks like the extra data is a Xing VBR header. I'll look at whether I can get ffmpeg to write one for the piped version, or maybe force a CBR (-b:a 320k). On the outfile version that header appears to be updated later in the process (if I stop a transcode early the values differ).

[Mark Henkelis]

I've tried a few alternatives, but so far had no luck getting Sonos to accept the streamed file (though it accepts all of them if they are transcoded manually as you observed).

I'll do some more testing - I'm fairly sure it used to work....

On 25/02/16 15:13, per...@gmail.com wrote:

[per...@gmail.com]

Mark - I've now offline transcoded all my ALAC into FLAC and things work fine, without sonospy needing to transcode any longer. So I'm a happy camper. (It's in this process I discovered the database corruption that solved my problem in the other thread)

As you've mentioned, the transcoding-on-the-fly when running as a service seems to have somehow gotten broken.

In addition, what pushed me over the edge to do the switch was that I discovered that scanning was not properly reading "Composer" tags on .M4A ALAC files (they were coming up "unknown composer" even though tags were present in the file). Composer tags were being read fine from MP3 and FLAC files. Since no-one else seems to have been actively using ALAC, and it does seem to be a dead-end format, it may not be worth fixing this. But it may be worth flagging to potential new users that ALAC is not fully supported (I will do so in the Raspberry Pi doc file I wrote up).

Thanks for a great tool. My wife and I are happily browsing classical and nonclassical music with differently structured indexes now, which is exactly what we were looking for!

Martin

[Mark Henkelis]

Great to hear that Sonospy works well for you. Do you use Works? They may make your classical experience even nicer :o)

I'll have another look at ALAC when I get some time, thanks for the Pi doc (which I must remember to upload).

Mark.

[Martin Pergler]

Thanks Mark, will explore works when I have time. Attached is an improved version of the Pi doc.

--
You received this message because you are subscribed to a topic in the Google Groups "Sonospy Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sonospy-devel/7WK-XlNosgk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sonospy-deve...@googlegroups.com.

To post to this group, send email to sonosp...@googlegroups.com.
Visit this group at https://groups.google.com/group/sonospy-devel.
For more options, visit https://groups.google.com/d/optout.

--

Martin Pergler
m...@pergler.org

[Mark Henkelis]

I've uploaded this to GitHub now, thanks again. Mark.