1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

announcing jukebox - an mp3 player for pyhme

Discussion in 'Developers Corner' started by jbernardis, Dec 26, 2011.

  1. Jan 15, 2012 #21 of 153
    windracer

    windracer joined the 10k club

    11,580
    3
    Jan 3, 2003
    St. Pete, FL
    My Linux box is also an Atom processor (although a dual-core one) but the music library is on a different NAS, not local, so maybe that's why.
     
  2. Jan 15, 2012 #22 of 153
    jcthorne

    jcthorne Active Member

    2,722
    3
    Jan 28, 2002
    Houston
    It appears that jukebox in not having trouble with it, just that the tivo does not wait for the cache to load. Could the cache be preloaded when jukebox starts rather than waiting for tivo to ask for the app?

    The cache file is 121MB.

    Thanks.

    Code:
    HME Server for Python 0.19
    Sun Jan 15 17:43:44 2012 Thornolis Video Archive version 2.0c module initializin
    g
    Loading thumbnail cache
    100 thumbnails loaded from cache
    Sun Jan 15 17:43:44 2012 JukeBox version 1.0c module initializing
    Sun Jan 15 17:43:44 2012 Server Starts
    Registering: jukebox
    Registering: vidmgr
    192.168.1.109:40386 - - [15/Jan/2012 17:43:51] "GET /jukebox/ HTTP/1.1" 200 -
    192.168.1.109:40386 - - [15/Jan/2012 17:43:51] Starting HME: jukebox
    Jukebox thread entering startup
    Sun Jan 15 17:43:52 2012 Starting cache file load
    Sun Jan 15 17:45:06 2012 Starting sort
    Sun Jan 15 17:45:11 2012 Finished - 141767 music files, 9328 albums, 665 album a
    rtists, 3559 track artists
    Jukebox thread activating
    Server address = 192.168.1.128:9032
    192.168.1.109:40386 - - [15/Jan/2012 17:45:11] Ending HME: jukebox
    ----------------------------------------
    Exception happened during processing of request from ('192.168.1.109', 40386)
    Traceback (most recent call last):
      File "C:\Python27\lib\SocketServer.py", line 582, in process_request_thread
        self.finish_request(request, client_address)
      File "C:\Python27\lib\SocketServer.py", line 323, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "C:\python HME\start.py", line 139, in __init__
        client_address, server)
      File "C:\Python27\lib\SocketServer.py", line 641, in __init__
        self.finish()
      File "C:\Python27\lib\SocketServer.py", line 694, in finish
        self.wfile.flush()
      File "C:\Python27\lib\socket.py", line 303, in flush
        self._sock.sendall(view[write_offset:write_offset+buffer_size])
    error: [Errno 10053] An established connection was aborted by the software in yo
    ur host machine
    ----------------------------------------
    
     
  3. Jan 15, 2012 #23 of 153
    jcthorne

    jcthorne Active Member

    2,722
    3
    Jan 28, 2002
    Houston
    lrhorer,

    I admired your video collection and your organization of it. Never made fun of someone wanting to store more video than they will ever watch.

    I have been buying and collecting music for many years and yes, the collection is large and growing. I use it every day.

    This forum, of all places I did not feel I would have to justify the desire to catalog and present a large music collection. It occupies less than 2TB of storage so not that much by today's storage capacities.
     
  4. Jan 15, 2012 #24 of 153
    lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    I wasn't making fun of it or you. I'm just astounded at the size. I'm not sure how you took any of what I said as derision, but it certainly wasn't my intent.

    'To say the least. Of course I, too, have been collecting music for the better part of 4 decades, but obviously not nearly the volume you have, even if I were to rip every song on every album I own. As you can see from my stats, I only have an average of about 2 songs from each album in the library, since I rarely like more than 1 or 2 songs on any given album. I don't bother to rip any songs I don't like. Even applying a factor of 10 (most albums have somewhere around 15 - 20 songs on them), it still would only be about 10,000 files or so.

    I should certainly hope so.

    I did not ask or expect you to do so. It's entirely your business how much music you collect, or why. I never suggested otherwise.

    Well, 2T is quite a bit, especially for audio, but it's certainly not the unimaginable amount of space it was when I first got into computing.
     
  5. Jan 15, 2012 #25 of 153
    lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    Both sound a little slow, to me. Admittedly, I have much fewer files, by a factor of almost 3 or 5, but still it only takes 9 seconds on my server, and it's no Ferrari.
     
  6. Jan 15, 2012 #26 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    jcthorne - I sent you a private message with my email address. I want to send you a version of jukebox for you to try before general consumption.

    This version pre-loads the cache as you requested. Just so you know, the issue goes beyond when the cache is loaded. By preloading the cache, I've raised the problem of what happens when the cache changes. With the original design, all you had to do was exit the app and then re-enter it. By preloading, however, exiting the app has no effect - you'd instead need to exit the process (pyhme).

    I have solved this by starting a thread that looks for a changed timestamp on the cache file every 10 minutes (actually a config parameter) and reloading when it detects this. The app still needs to be exited to "bring in" the new cache, but that is the same as today.

    Overall I think it's a better design, but I'd like you to put it through its paces before I post it to github.
     
  7. Jan 15, 2012 #27 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    The time for building the cache depends largely on the artwork. This is when resizing is done. While I was developing the app, and before I put in the logic to resize the artwork, it took my rig about 25 seconds to build the cache. Adding the artwork logic took it up by a factor of 6 or 7.
     
  8. Jan 15, 2012 #28 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    lrhorer - I'm thinking of attacking this in two steps.

    First, the lyrics would be retrieved during cache building (or perhaps through a separate utility). I'm thinking that the lyrics themselves would be stored in a "parallel" file to the mp3 file (much like vidmgr uses parallel thumbnail and metadata files).

    Step 2, jukebox would allow you to display the contents of this file by pressing, say, the info button.

    This solution would allow you to create/edit your own file for those songs that prove troublesome for web downloading algorithms. It also allows you to put in any arbitrary information that you might want to have displayed.
     
  9. Jan 16, 2012 #29 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    A new version of jukebox - 1.0d - is up on github.

    jcthorne and I have been working together today to get jukebox working with his rather large collection. Basically what I did was add the ability to pre-load the cache at process start time rather that at app activation time.

    To control this, I have added 2 configuration options:

    preloadcache = True or False
    default is false and results in today's behavior.

    If preloading is done, there is a problem with how I know when to reload the cache if it happens to change. I didn't want to force people to restart the entire pyhme process, but that is what it would have taken if I didn't do anything else.

    What I did to solve this to to create a thread (only if preloadcache is True) that sleeps in an infinite loop, and when it wake up, it checks the timestamp on the cache file to see if it has changed. If it has, it reloads it. Note that if the GUI is active, reloading will have no effect on it - you MUST exit the app and re-enter to get the new cache. The frequency with which the thread wakes up is controlled by the second config.ini parameter:

    cachewatchinterval = 600

    This is the number of seconds between checking the file for changes. 10 minutes (600 seconds) is the default.

    Thanks again to jcthorne for helping with this effort.
     
  10. Jan 17, 2012 #30 of 153
    windracer

    windracer joined the 10k club

    11,580
    3
    Jan 3, 2003
    St. Pete, FL
    I guess that explains my 20 minute cache time, then ... all of my MP3 files have artwork and they're not local to jukebox.

    Code:
    Tue Jan 17 02:08:43 2012 starting cache build from MP3 files
    Beginning container: Music Library (/mnt/media/Music)
    . . . . | . . . . + . . . . | . . . . + . . . . | . . . . + . . . . | . . . . + . . . . | . . . . + . . . . | . .
    5763 music files in container
    5763 total files in ALL containers
    
    Tue Jan 17 02:27:10 2012 Starting sort
    Tue Jan 17 02:27:11 2012 Saving cache to disk
    Tue Jan 17 02:27:11 2012 Finished - 5763 music files, 2029 albums, 1247 album artists, 1244 track artists
     
  11. Jan 19, 2012 #31 of 153
    jcthorne

    jcthorne Active Member

    2,722
    3
    Jan 28, 2002
    Houston
    Still should not take that long.

    My 141k files took about 2 hrs to build the cache. They are all stored on the NAS. jukebox is running on a windows pc at the moment with pytivo.

    Great that it works though and only needs to build the cache when things change.
     
  12. Feb 17, 2012 #32 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    An update on Jukebox. I guess inactivity here means that the app has been working for everyone. I've been makeing a few changes however, and should shortly have an update.

    lrhorer has requested a lyrics retrieval feature, and although I was a bit reluctant, I have been working on it. I actually was just about completed with it when it stopped working altogether. I looked into why it was failing and discovered that the site I was using - azlyrics.com - was not responding. Thinking the site had crashed or was having other problems, I just stopped for the night. Over the next day or two, I noticed that I could access the site from work, but not from home. When I queried them about it, they stated that my IP address had been blocked because of what their bot determined was malicious behavior. I think something along the line of 10 requests in a minute or 100 in 15 minutes (which is probably what I exceeded) is considered malicious and the ip address is blocked for a week. If repeated, the block could be permanent.

    Well I was finally unblocked just today and was able to work through the last couple of issues. I just want to run with it myself for a few days to make sure it holds together. And of course since I don't want to be blocked again, I am pacing my testing :)
     
  13. Feb 25, 2012 #33 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    I just put version 1.0e of Jukebox up on git. This has the lyrics retrieval feature requested by lrhorer. I also fixed a little bug. In the past, I have been constantly concerned about jukebox being a memory hog. So much so that when the app deactivated, I went so far as to actually destroy my cache, thinking that I could somehow hasten garbage collection. Well that was all well and good until I was requested to pre-load the cache. When the cache is pre-loaded, I obviously don't want to destroy it, so I made this fix.

    A Note about lyrics retrieval - be aware of the restrictions of azlyrics.com that I discuss in the previous post. When I first saw it in operation, I thought it was cool, so I went crazy requesting lyrics for song after song. If you do this too, you will likely end up with your IP address blocked.
     
  14. Feb 28, 2012 #34 of 153
    lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    Hey Jeff, I have run across an issue.

    The player won't play a file by .38 Special. I suspect it is because it takes the leading period in the file name to mean the file is hidden. Of course, there aren't a lot of files with leading periods in the name, but renaming the file breaks the convention of <artist> - <Song>.mp3.
     
  15. Feb 28, 2012 #35 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    I assume the filename is something like ".38 Special - Hold on Loosely.mp3"?

    I'll look at how the logic breaks down on this one - sound's like it might be pretty simple.

    BTW in my collection, .38 Special is listed as 38 Special - without the leading period. I did this a while ago - not sure why, but that's why I didn't hit this issue.
     
  16. Feb 28, 2012 #36 of 153
    jbernardis

    jbernardis New Member

    1,072
    0
    Oct 21, 2003
    Princeton NJ
    Fixed.

    Version 1.0f is now up on git - see my signature.

    This was a simple change - one line had to be removed. The issue wasn't that files wouldn't play - they never even made it into the cache. Therefore, you will still be unable to play these songs until you rebuild the cache.
     
  17. Feb 28, 2012 #37 of 153
    windracer

    windracer joined the 10k club

    11,580
    3
    Jan 3, 2003
    St. Pete, FL
    Heh, same here. At least we all have good taste in 80s music. :D

    The new lyrics function is neat, BTW.
     
  18. Feb 28, 2012 #38 of 153
    lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    You assume correctly.
     
  19. Feb 28, 2012 #39 of 153
    lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    Of course we do! We're smart enough to be using vidmgr, aren't we? :cool:

    That's why I requested it. :)
     
  20. Mar 17, 2012 #40 of 153
    cassiusdrow

    cassiusdrow Member

    130
    0
    May 21, 2003
    NJ
    I love this! Between this and Pytivo, I can finally get rid of Galleon!

    Is there any way to prevent the sudden return to Live TV that sometimes happens when the music is done playing?
     

Share This Page