Advice on indexing problem - excluding playlists

[Adam Bentley]

Hi all,

             on my first attempt with sonospy. Getting there slowly. Still trying to index my collection. Sadly although the main indexing is completed finally (after some processing hiccups caused by strange characters on some of the MP3 tags) the scan script now appears to be trying to process all the playlist files in my music folder. It appears many of these are corrupt or broken. Is there any way to stop the scan script bothering with all the M3U files? I guess I could just remove them all but that would be a last resort.

Any advice appreciated. Hopefully if I can get the scan to complete sucessfully I will actually be able to browse my library fully for the first time in years... thanks

Adam..

....7-bangles-walk_like_an_egyptian_(accapella_version).mp3

playlist track inserted: \\192.168.0.14\music\Bangles-Different_Light_(Special_Edition)-2CD-2010-WRE\000-bangles-different_light_(special_edition)-2cd-2010.m3u : \\192.168.0.14\music\Bangles-Different_Light_(Special_Edition)-2CD-2010-WRE\208-bangles-bangles_hit_medley_mix.mp3

C:\Python26\lib\cgitb.py:245: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6

  value = pydoc.text.repr(getattr(evalue, name))

** EXITING on unhandled exception - See errors/ErrorDump-20170101-195521.txt

errors/ErrorDump-20170101-195521.txt

<type 'exceptions.UnicodeDecodeError'>

Python 2.6.6: C:\Python26\python.exe

Sun Jan 01 19:55:21 2017

A problem occurred in a Python script.  Here is the sequence of

function calls leading up to the error, in the order they occurred.

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in <module>()

 2937 

 2938 if __name__ == "__main__":

 2939     status = main()

 2940     sys.exit(status)

 2941 

status undefined

main = <function main>

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in main(argv=None)

 2932             for path in args:

 2933                 if path.endswith(os.sep): path = path[:-1]

 2934                 process_dir(path.decode(enc), options, database)

 2935     filelog.close_log_files()

 2936     return 0

global process_dir = <function process_dir>

path = r'\\192.168.0.14\music'

path.decode = <built-in method decode of str object>

global enc = 'mbcs'

options = <Values at 0x265d188: {'verbose': False, 'databa...quiet': False, 'exclude': None, 'extract': None}>

database = r'C:\Users\adam\Desktop\Sonospy\sonospy\test.db'

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in process_dir(scanpath=u'\\\\192.168.0.14\\music', options=<Values at 0x265d188: {'verbose': False, 'databa...quiet': False, 'exclude': None, 'extract': None}>, database=r'C:\Users\adam\Desktop\Sonospy\sonospy\test.db')

 1670 

 1671                 # read playlist date and track details

 1672                 playlisttracks = read_playlistfile(ffn, filepath)

 1673 

 1674                 # check what has changed

playlisttracks = [(u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...tion)-2CD-2010-WRE\\101-bangles-manic_monday.mp3', u'1392228707.8', u'1291743766.0', 1, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...D-2010-WRE\\102-bangles-in_a_different_light.mp3', u'1392228708.58', u'1291743886.0', 2, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...10-WRE\\103-bangles-walking_down_your_street.mp3', u'1392228712.71', u'1291743750.0', 3, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...-2010-WRE\\104-bangles-walk_like_an_egyptian.mp3', u'1392228704.8', u'1291743762.0', 4, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...010-WRE\\105-bangles-standing_in_the_hallway.mp3', u'1392228710.69', u'1291743768.0', 5, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...ition)-2CD-2010-WRE\\106-bangles-return_post.mp3', u'1392228707.01', u'1291743762.0', 6, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...-WRE\\107-bangles-if_she_knew_what_she_wants.mp3', u'1392228708.27', u'1291743758.0', 7, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...Edition)-2CD-2010-WRE\\108-bangles-let_it_go.mp3', u'1392228709.95', u'1291743812.0', 8, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...n)-2CD-2010-WRE\\109-bangles-september_gurls.mp3', u'1392228705.67', u'1291743751.0', 9, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...10-WRE\\110-bangles-angels_dont_fall_in_love.mp3', u'1392228707.39', u'1291743745.0', 10, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...Edition)-2CD-2010-WRE\\111-bangles-following.mp3', u'1392228708.01', u'1291743742.0', 11, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...tion)-2CD-2010-WRE\\112-bangles-not_like_you.mp3', u'1392228711.9', u'1291743867.0', 12, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...010-WRE\\201-bangles-manic_monday_(extended).mp3', u'1392228706.01', u'1291744113.0', 13, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...0-WRE\\202-bangles-hero_takes_a_fall_(remix).mp3', u'1392228712.93', u'1291744115.0', 14, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...lking_down_your_street_-_james_(live_medley).mp3', u'1392228705.19', u'1291744078.0', 15, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...-bangles-walk_like_an_egyptian_(dub_version).mp3', u'1392228706.39', u'1291744070.0', 16, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...205-bangles-walking_down_your_street_(remix).mp3', u'1392228710.42', u'1291744119.0', 17, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...-if_she_knew_what_she_wants_(extended_remix).mp3', u'1392228712.23', u'1291744111.0', 18, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...es-walk_like_an_egyptian_(accapella_version).mp3', u'1392228710.21', u'1291744092.0', 19, 0), (u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...s-different_light_(special_edition)-2cd-2010.m3u', u'1392228706.34', u'1291743375.0', u'\\\\192.168.0.14\\music\\Bangles-Different_Lig...2010-WRE\\208-bangles-bangles_hit_medley_mix.mp3', u'1392228710.96', u'1291744122.0', 20, 0)]

global read_playlistfile = <function read_playlistfile>

ffn = u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u"

filepath = u"\\\\192.168.0.14\\music\\Beyonce - Discography...- B'Day (Deluxe) 2CDRip 2007 [Cov+2CD][MJN]\\CD1"

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in read_playlistfile(filespec=u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u", filepath=u"\\\\192.168.0.14\\music\\Beyonce - Discography...- B'Day (Deluxe) 2CDRip 2007 [Cov+2CD][MJN]\\CD1")

 2418     trackcounts = defaultdict(int)

 2419     pltrack = 1

 2420     for trackcountdata, trackdata in process_workvirtualfile_file(filespec, filepath, 'playlist'):

 2421 #        print "============="

 2422 #        print trackdata

trackcountdata undefined

trackdata undefined

global process_workvirtualfile_file = <function process_workvirtualfile_file>

filespec = u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u"

filepath = u"\\\\192.168.0.14\\music\\Beyonce - Discography...- B'Day (Deluxe) 2CDRip 2007 [Cov+2CD][MJN]\\CD1"

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in process_workvirtualfile_file(filespec=u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u", wvfilepath=u"\\\\192.168.0.14\\music\\Beyonce - Discography...- B'Day (Deluxe) 2CDRip 2007 [Cov+2CD][MJN]\\CD1", wvtype='playlist')

 2484             yield None, None

 2485             return

 2486         pltracks = read_playlist(filespec, ex)

 2487         for pltrack in pltracks:

 2488             pltrack = checkpath(pltrack, wvfilepath)

pltracks undefined

global read_playlist = <function read_playlist>

filespec = u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u"

ex = u'.m3u'

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in read_playlist(filespec=u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u", extension=u'.m3u')

 2520 def read_playlist(filespec, extension):

 2521     if extension in m3u_playlist_extensions:

 2522         return read_m3u_playlist(filespec)

 2523     if extension in pls_playlist_extensions:

 2524         return read_pls_playlist(filespec)

global read_m3u_playlist = <function read_m3u_playlist>

filespec = u"\\\\192.168.0.14\\music\\Beyonce - Discography...ov+2CD][MJN]\\CD1\\00. Beyonce - B'Day (CD1).m3u"

 C:\Users\adam\Desktop\Sonospy\sonospy\gettags.py in read_m3u_playlist(filespec=u'09 - Get Me Bodied (Extended Mix).mp3')

 2528 def read_m3u_playlist(filespec):

 2529     tracks = []

 2530     for line in codecs.open(filespec,'r','utf-8'):

 2531         if line == '': continue

 2532         if line.startswith('#'): continue

line = u'09 - Get Me Bodied (Extended Mix).mp3\r\n'

global codecs = <module 'codecs' from 'C:\Python26\lib\codecs.pyc'>

codecs.open = <function open>

filespec = u'09 - Get Me Bodied (Extended Mix).mp3'

 C:\Python26\lib\codecs.py in next(self=<open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>)

  682 

  683         """ Return the next decoded line from the input stream."""

  684         return self.reader.next()

  685 

  686     def __iter__(self):

self = <open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>

self.reader = <open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>

self.reader.next = <bound method StreamReader.next of <open file u'...\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>>

 C:\Python26\lib\codecs.py in next(self=<open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>)

  613 

  614         """ Return the next decoded line from the input stream."""

  615         line = self.readline()

  616         if line:

  617             return line

line undefined

self = <open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>

self.readline = <bound method StreamReader.readline of <open fil...\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>>

 C:\Python26\lib\codecs.py in readline(self=<open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>, size=None, keepends=True)

  528         # If size is given, we call read() only once

  529         while True:

  530             data = self.read(readsize, firstline=True)

  531             if data:

  532                 # If we're at a "\r" read one extra character (which might

data undefined

self = <open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>

self.read = <bound method StreamReader.read of <open file u'...\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>>

readsize = 72

firstline undefined

builtinTrue = True

 C:\Python26\lib\codecs.py in read(self=<open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>, size=72, chars=-1, firstline=True)

  475             data = self.bytebuffer + newdata

  476             try:

  477                 newchars, decodedbytes = self.decode(data, self.errors)

  478             except UnicodeDecodeError, exc:

  479                 if firstline:

newchars = u''

decodedbytes = 0

self = <open file u'\\\\192.168.0.14\\music\\Beyonce - ...\\CD1\\00. Beyonce - B'Day (CD1).m3u', mode 'rb'>

self.decode = <built-in function utf_8_decode>

data = '\x82j\x85 Vu (feat. Jay-Z).mp3\r\n13 - Ring The Alarm.mp...ment.mp3\r\n15 - Listen (from the motion picture Dr'

self.errors = 'strict'

<type 'exceptions.UnicodeDecodeError'>: 'utf8' codec can't decode byte 0x82 in position 0: invalid start byte

    __class__ = <type 'exceptions.UnicodeDecodeError'>

    __delattr__ = <method-wrapper '__delattr__' of exceptions.UnicodeDecodeError object>

    __dict__ = {}

    __doc__ = 'Unicode decoding error.'

    __format__ = <built-in method __format__ of exceptions.UnicodeDecodeError object>

    __getattribute__ = <method-wrapper '__getattribute__' of exceptions.UnicodeDecodeError object>

    __getitem__ = <method-wrapper '__getitem__' of exceptions.UnicodeDecodeError object>

    __getslice__ = <method-wrapper '__getslice__' of exceptions.UnicodeDecodeError object>

    __hash__ = <method-wrapper '__hash__' of exceptions.UnicodeDecodeError object>

    __init__ = <method-wrapper '__init__' of exceptions.UnicodeDecodeError object>

    __new__ = <built-in method __new__ of type object>

    __reduce__ = <built-in method __reduce__ of exceptions.UnicodeDecodeError object>

    __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.UnicodeDecodeError object>

    __repr__ = <method-wrapper '__repr__' of exceptions.UnicodeDecodeError object>

    __setattr__ = <method-wrapper '__setattr__' of exceptions.UnicodeDecodeError object>

    __setstate__ = <built-in method __setstate__ of exceptions.UnicodeDecodeError object>

    __sizeof__ = <built-in method __sizeof__ of exceptions.UnicodeDecodeError object>

    __str__ = <method-wrapper '__str__' of exceptions.UnicodeDecodeError object>

    __subclasshook__ = <built-in method __subclasshook__ of type object>

    __unicode__ = <built-in method __unicode__ of exceptions.UnicodeDecodeError object>

    args = ('utf8', '\x82j\x85 Vu (feat. Jay-Z).mp3\r\n13 - Ring The Alarm.mp...ment.mp3\r\n15 - Listen (from the motion picture Dr', 0, 1, 'invalid start byte')

    encoding = 'utf8'

    end = 1

    message = ''

    object = '\x82j\x85 Vu (feat. Jay-Z).mp3\r\n13 - Ring The Alarm.mp...ment.mp3\r\n15 - Listen (from the motion picture Dr'

    reason = 'invalid start byte'

    start = 0

The above is a description of an error in a Python program.  Here is

the original traceback:

Traceback (most recent call last):

  File "./gettags.py", line 2939, in <module>

    status = main()

  File "./gettags.py", line 2934, in main

    process_dir(path.decode(enc), options, database)

  File "./gettags.py", line 1672, in process_dir

    playlisttracks = read_playlistfile(ffn, filepath)

  File "./gettags.py", line 2420, in read_playlistfile

    for trackcountdata, trackdata in process_workvirtualfile_file(filespec, filepath, 'playlist'):

  File "./gettags.py", line 2486, in process_workvirtualfile_file

    pltracks = read_playlist(filespec, ex)

  File "./gettags.py", line 2522, in read_playlist

    return read_m3u_playlist(filespec)

  File "./gettags.py", line 2530, in read_m3u_playlist

    for line in codecs.open(filespec,'r','utf-8'):

  File "C:\Python26\lib\codecs.py", line 684, in next

    return self.reader.next()

  File "C:\Python26\lib\codecs.py", line 615, in next

    line = self.readline()

  File "C:\Python26\lib\codecs.py", line 530, in readline

    data = self.read(readsize, firstline=True)

  File "C:\Python26\lib\codecs.py", line 477, in read

    newchars, decodedbytes = self.decode(data, self.errors)

UnicodeDecodeError: 'utf8' codec can't decode byte 0x82 in position 0: invalid start byte

[Mark Henkelis]

Hi,

It looks like an encoding error in the playlist file(s), this particular file is using characters from the extended ASCII set to get French characters. I've seen that before when the files were created on Windows and the file system encoding is not set correctly. You could write a script to find those files and fix them.

If your playlist files are separated from your music files, then you can use the -e flag to exclude the root folder they are in from being scanned - I'm guessing that isn't the case or you could have just moved them yourself.

There is a flag to exclude files with particular extensions from being processed, but that flag only affects music files and not playlists - I guess that's a bug, nobody has asked for that before :o)

A quick look at the code suggests it's quite an easy fix, but obviously requires testing. If you want to test it for me, can you add the following line to gettags.py (in the sonospy folder), after line 1635:

            if ex.lower() in file_extn_exclusions: continue       

so 1635-37 should now look like this:

            ff, ex = os.path.splitext(fn)
            if ex.lower() in file_extn_exclusions: continue       
            if not ex.lower() in playlist_extensions: continue

Then if you add .m3u to file_extension_exclusions in scan.ini (in the sonospy folder)

    file_extension_exclusions=.part,.txt,.csv,.jpg,.png,.gif,.ffp,.md5,.wmv,.mpg,.mp4,.sfv,.m3u

and re-scan it should ignore the playlists (the full set of playlist extensions is below, if you use any of the others you'll have to exclude those too):

    m3u_playlist_extensions = ['.m3u', '.m3u8']
    pls_playlist_extensions = ['.pls']
    wpl_playlist_extensions = ['.wpl']

Please let me know how you get on.

Mark.

--
You received this message because you are subscribed to the Google Groups "Sonospy Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sonospy-deve...@googlegroups.com.
To post to this group, send email to sonosp...@googlegroups.com.
Visit this group at https://groups.google.com/group/sonospy-devel.
For more options, visit https://groups.google.com/d/optout.

[Adam Bentley]

Thanks Mark,

                    I finally have my first successful scan, albeit with a few errors here and there. 

Thanks so much for the quick response.

I have one more question if I may be so bold...

Is there a way to represent the file structure on my share (via some .ini  in Sonospy) or are just the views by tag type available? A good chunk of my stuff has poor tagging sadly...

I will keep playing anyhow.

thanks again

Adam.

[Mark Henkelis]

Sorry for the delay. There isn't currently a way in Sonospy to walk the
filesystem. I wanted to check how hard it would be to add that before
replying, and whether it would be workable (for instance if there are no
tags, then artist, duration, mime type etc will be missing).

I've knocked together an experimental solution, that allows basic
filesystem walking (starting from the root, if it's useable I can change
that), and playing of music files found (based on their extension -
again I can change that to use tags if they are available). It makes
assumptions about the artist and album name from the folder names (again
that can be changed).

If you want to trial it and send feedback, reply to this and I'll post
the code and instructions.

Mark.

[Adam Bentley]

Hi Mark,

           yes I'd be more than happy to try it. I have to say I'm blown away by the responsiveness and passion you have shown with regard to my questions. Just amazing. You put many commercial vendors to shame by comparison. :-)

many thanks once again.

Adam.

[Mark Henkelis]

OK, I've attached the code. You need to do the following:

1) Copy files.py into a folder and run it against a copy of your database (I'd keep a backup just in case - obviously you could create a new db from a scan of a portion of your collection and use that instead). This will create a new table called files in the database, that will contain a recursive folder hierarchy based on whatever was scanned previously (so it will only show what you scanned, if you excluded files with no tags etc then you won't see them):

    python files.py -d <database>

You'll see some "UNIQUE constraint failed" errors, you can ignore them (probably best to pipe them to a file and check there are no other errors).

2) Update your index ini file to include the following in the [indexes] section (and only that, it doesn't support sort orders etc), which will enable the filesystem index (you'll need to at least turn on alternative_indexing in pycpoint.ini) :

tree=file
    title=Files
        index=file

3) Overwrite the mediaserver.py file in the sonospy folder with the provided one, and restart Sonospy (pointing at the updated database from step 1 and the ini file from step 2).

If everything works you should see a Files index, and be able to walk down it and play the files it finds (as long as they are FLAC/mp3/wma/wav for now). It will use the folder names as artist and album names, so if your music is artist/album/track you should see them as captured (that track will be the filename).

I have only briefly tested it, on Linux, and only with the SMAPI interface (not WMP, though that should work).

Let me know how you get on :o)

Mark.

[Adam Bentley]

Completed all steps

1. Copied new modules into sonospy primary folder

2. ran files.py against my database (python files.py -d test.db)

3. amended pycpoint.py and set alternative_indexing to Y

4. edited userindex.ini and added the new lines for type 'file' at the bottom of the [index] section

5. restarted my server  (python pycpoint.py -p -wSonospy=SonosPY,test.db)

Didnt see a files entry in the Sonos controller.

Tried instead to add the new lines to defaultindex.ini.

I can then a see a 'Files' option in the menu but browsing it shows no content.

I am guessing there's something I've done wrong somewhere or perhaps I need to specify something additional when starting up. Or i need to set something else in pycpoint.ini

Any advice appreciated.

[MARK HENKELIS]

Sorry, re the ini I should have been more specific.

If you set alternative_indexing it looks in defaultindex.ini. If you also set user_indexes it looks in userindex.ini, or a name that you specify on the command line (and you can also manipulate other index stuff).

If you can see the Files option in the menu, then the indexing part is working. Did the controller say it couldn't find any data, or couldn't access the service? 

I see that you're using -w rather than -s, I didn't test with -w, only -s, so if you can try with that it will help.

I'll write a script tonight to check whether files.py processed your database correctly.

---

From: "Adam Bentley" <adam.d....@gmail.com>
To: "Sonospy Development" <sonosp...@googlegroups.com>
Sent: Thursday, 12 January, 2017 12:07:29 PM
Subject: Re: [sonospy-devel] Advice on indexing problem - excluding playlists

[Adam Bentley]

Ok. when i run with only python pycpoint.py -sSonospy=SonosPY,test.db 

the SonosPY service does not appear in the Sonos service list. 

When it is run with -w then it does but the files section shows ' No Selection Available' within Sonos.

let me know if you need more information.

thanks.

[MARK HENKELIS]

When you run with -s you have to add it to the controller with "add music services".

---

From: "Adam Bentley" <adam.d....@gmail.com>
To: "Sonospy Development" <sonosp...@googlegroups.com>

Sent: Thursday, 12 January, 2017 2:13:38 PM

[Adam Bentley]

Yes, I thought that was the case but sadly it does not appear in the list of available services. I've tried it with just -sSonospy=SonosPY,test.db  and also using -p for proxy mode as well. Sadly only -w (acting as WMP) seems to show up.

If there's a way to capture more debug output let me know.

BTW, if there's a way to thank you in terms of donations for the project let me know.

 

thanks.
Ad

[Mark Henkelis]

You're missing the -r flag from the end of the command line, to register the service. There's a "Sonospy install and quick config guide" that gives more detail.

[Adam Bentley]

Yep that now offers me the chance to register the service. However the files browse selection still shows no content at the moment  - text is 'No items found' inside Sonos App.

At least it's a step in the right direction.

thanks.

Adam.

[Mark Henkelis]

I've just realised that you are running on Windows (I think, from an earlier post) - if that's correct, can you save the attached updated version of files.py and run it please.

Then let me know what you get when you restart Sonospy.

[Adam Bentley]

ran it against my test.db file... lots of errors like below... but then...

columns parent, child, node are not unique

columns parent, child, node are not unique

columns parent, child, node are not unique

columns parent, child, node are not unique

Traceback (most recent call last):

  File "files.py", line 80, in <module>

    status = main()

  File "files.py", line 73, in main

    insert(cs2, specs[len(specs) -1], filename, 2, id, path)

  File "files.py", line 29, in insert

    cs2.execute('insert into files values (?, ?, ?, ?, ?)', (parent, child, node, id, path))

sqlite3.OperationalError: disk I/O error

so I rebuilt the db file from scratch (test2.db) and ran it again. Same result.

Plenty of free space on the volume....

Any thoughts?

[Mark Henkelis]

It's probably a locking issue, I'm guessing you have a lot of tracks. Can you try the attached updated files.py please.

[Adam Bentley]

Boom! Success! I can finally see all the files. Just Amazing.

I don't know quite what to say other than thank you. Hopefully this will benefit other users if it's rolled into the primary code base.

Outstanding work Mark. Once again if there's way I can support your efforts let me know.

kind regards.

Adam.

[Adam Bentley]

I should qualify that slightly. I am seeing some issues where not all the folders appear to be playable (no content even though there's stuff in there) however this could well be an indexing issue. I will do further investigation. Even so this is a massive step forward for me.

thanks.

[Mark Henkelis]

Check your ini settings - the scan.ini defaults (below) ignore empty tags and duplicates by default. Some file types are also excluded by default, plus remember that currently I've only added support for playing FLAC/mp3/wma/wav for the test, let me now if you have other file types.

You can install a tool to view the database (I use the SQLite manager add-on for Firefox), or I can send you a program to export your track data (or adjust files.py to tell you what it's doing).


# By default tags are gathered from all tracks encountered, including
# duplicates. If you want to ignore duplicate tracks, set
# ignore_duplicate_tracks to Y
# Note that after changing this option you will have to perform a full
# rescan

ignore_duplicate_tracks=Y

# If you are ignoring duplicate tracks and want to influence which file
# type takes precedence, set duplicate_tracks_precedence to the precedence
# you want, with higher precedences first. If you want the first track
# encountered to be selected, set duplicate_tracks_precedence to nothing.
# (note file types are case insensitive)

duplicate_tracks_precedence=FLAC,ogg,wma,mp3

# By default tags will be gathered from all tracks encountered. If you want
# to ignore music tracks that do not contain tags, set ignore_blank_tags
# to Y
# Note that after changing this option you will have to perform a full
# rescan

ignore_blank_tags=Y

# By default the scanner will try to read tags from all files it encounters,
# rejecting those that are not music files (but note that it will attempt to
# read incomplete music files). If you want to force the scanner to ignore
# specific filenames or extensions, specify them below:
# (note file names and extensions are case insensitive)

file_name_exclusions=.ds_store,desktop.ini,thumbs.db
file_extension_exclusions=.part,.txt,.csv,.jpg,.png,.gif,.ffp,.md5,.wmv,.mpg,.mp4,.sfv

--