1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

pyTivo auto pushes

Discussion in 'TiVo Home Media Features & TiVoToGo' started by moyekj, Oct 18, 2009.

  1. Oct 18, 2009 #1 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I was looking for a way to setup automated "watch" folders with pyTivo similar to how one can setup auto watch folders with Tivo Desktop.
    The idea being that you simply drop files into a folder and those files are then automatically pushed to one of your Series 3 TiVos using pyTivo.

    EDIT: Since this 1st posting there this program has now evolved from a python script into a Java program with a GUI and ability to run as a service for Windows platforms or as background job for Mac/Linux platforms. The program download, documentation and source code are available from Google project:
    http://code.google.com/p/pytivo-auto-push/

    ----LEGACY PYTHON SCRIPT----
    Attached zip file contains current incantation of a python script to do this. Since this is to be used by pyTivo users who already have python installed I figured the best scripting language to use would be python even though I have little to no experience with it.

    To summarize what current script does:
    * Can be configured to watch multiple folders with ability for each to push to different Tivos (or all same Tivo if you wish)
    * Checks configured shares for video files to process and then sleeps for a period of time (10 seconds) before checking again. This happens in infinite loop.
    * If it is determined a video file is not changing (modification time & file size not changing) then and a pyTivo push of that file is initiated.
    * Processed video files are then "tagged" as processed by having an empty file of same name but with ".processed" suffix. This prevents pushing same files over and over.
    NOTE: ffmpeg is used to determine which files are video files, so you need to edit script to specify your ffmpeg binary location.

    There are comments at top of script with some instructions on how to set it up properly with pyTivo. Specifically you must ensure that the watchList settings in the script have corresponding pyTivo shares of same name and path in your pyTivo setup.
    Included in zip file is a run.bat file for windows users. You may have to edit run.bat to correct path to python executable in your installation.

    Obviously still a lot of room for improvement but I figured I'd get the ball rolling if this kind of capability is not already available.
    Down the road if there is interest in this then one can worry about details such as running as windows service for windows platforms, etc. For now though just run this script in a console/xterm.

    EDIT 1: Change to not bother to test files ending in .txt with ffmpeg
    EDIT 2: Script now searches folder hierarchy for video files instead of just top level files in each watch folder.
    EDIT 3: Wrap ffmpeg and file in quotes in isVideo function. Don't mark with .processed unsuccessful pushes.
     

    Attached Files:

  2. Oct 18, 2009 #2 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    NOTE: Just added support for full folder hierarchies under watch folders instead of just processing top level watch folder video files.
     
  3. Oct 19, 2009 #3 of 216
    reneg

    reneg Member

    699
    0
    Jun 19, 2002
    The script is working pretty good for me. On one occasion it tried to send pytivo a file with a .processed extension which crashed both pytivo and the script. I'll update to the latest version of the script and see if I can make it happen again.
     
  4. Oct 19, 2009 #4 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I think in EDIT 1 I updated to specifically ignore files ending in .txt or .processed so that should no longer be an issue.
     
  5. Oct 19, 2009 #5 of 216
    orangeboy

    orangeboy yes, I AM orangeboy!

    4,083
    0
    Apr 19, 2004
    East Moline, IL
    Bleh. I can't get the latest release to work:

    Code:
    Traceback (most recent call last):
      File "auto_push.py", line 151, in <module>
        if not os.path.exists(entry_processed) and isVideo(entry):
      File "auto_push.py", line 65, in isVideo
        proc = subprocess.Popen(command, bufsize=0, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
      File "c:\Python26\lib\subprocess.py", line 621, in __init__
        errread, errwrite)
      File "c:\Python26\lib\subprocess.py", line 830, in _execute_child
        startupinfo)
    WindowsError: [Error 2] The system cannot find the file specified
    My edits:
    Code:
    # *******Adjust these as needed ******
    watchList = [
       {"tivo": "TivoS3", "path": "F:\\ToTivo", "share": "DL380-Server"}
    ]
    ffmpeg       = "C:\\Program Files\\kmttg\\ffmpeg"
    processedExt = ".processed"
    host         = "media-center:9032"
    
    I've tried specifying the paths with both \ and \\ each having the same result. The paths were copied directly from Windows Explorer's address bar because I'm lazy and don't want to mistype.
     
  6. Oct 19, 2009 #6 of 216
    reneg

    reneg Member

    699
    0
    Jun 19, 2002
    Try adding the .exe to ffmpeg
     
  7. Oct 19, 2009 #7 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    As reneg pointed out the problem is the call to ffmpeg looks like is failing. I think your ffmpeg setting should be:
    ffmpeg = "C:\\Program Files\\kmttg\\ffmpeg\\ffmpeg.exe"
    I probably should add some exception handling to catch that problem so that it is clearer what is wrong.
     
  8. Oct 19, 2009 #8 of 216
    orangeboy

    orangeboy yes, I AM orangeboy!

    4,083
    0
    Apr 19, 2004
    East Moline, IL
    Thanks guys - that was exactly it. I think I had that same problem before, and didn't learn from it. :(

    Upon first run, I got the following:
    Code:
    watching file: F:\ToTivo\The Outer Limits - Expanding Human-1036_cut.mpg
    watching file: F:\ToTivo\processed\702 - Vaya Con Cornholio.avi
    watching file: F:\ToTivo\processed\Fringe - Dream Logic-205_cut.mpg
    watching file: F:\ToTivo\processed\Jeopardy-5769_cut.mpg
    watching file: F:\ToTivo\processed\Jeopardy-5770_cut.mpg
    watching file: F:\ToTivo\processed\Jeopardy-_cut.mpg
    watching file: F:\ToTivo\processed\Law and Order - Reality Bites-_cut.mpg
    watching file: F:\ToTivo\processed\Parks and Recreation - Sister City-_cut.mpg
    watching file: F:\ToTivo\processed\Smallville - Echo-_cut.mpg
    watching file: F:\ToTivo\processed\Star Trek The Next Generation - Brothers-177_
    cut.mpg
    watching file: F:\ToTivo\processed\Star Trek The Next Generation - Future Imperf
    ect-182_cut.mpg
    watching file: F:\ToTivo\processed\TMZ-3035_cut.mpg
    checking file: F:\ToTivo\The Outer Limits - Expanding Human-1036_cut.mpg
    checking file: F:\ToTivo\processed\702 - Vaya Con Cornholio.avi
    pushing file: F:\ToTivo\processed\702 - Vaya Con Cornholio.avi
    http://media-center:9032/TiVoConnect?Command=Push&Container=DL380-Server&File=/p
    rocessed/702%20-%20Vaya%20Con%20Cornholio.avi&tsn=TivoS3
    Traceback (most recent call last):
      File "auto_push.py", line 152, in <module>
        process(tivoName, share, path, entry)
      File "auto_push.py", line 117, in process
        push(tivoName, share, path, file)
      File "auto_push.py", line 91, in push
        result = urllib2.urlopen(url)
      File "c:\Python26\lib\urllib2.py", line 124, in urlopen
        return _opener.open(url, data, timeout)
      File "c:\Python26\lib\urllib2.py", line 389, in open
        response = self._open(req, data)
      File "c:\Python26\lib\urllib2.py", line 407, in _open
        '_open', req)
      File "c:\Python26\lib\urllib2.py", line 367, in _call_chain
        result = func(*args)
      File "c:\Python26\lib\urllib2.py", line 1140, in http_open
        return self.do_open(httplib.HTTPConnection, req)
      File "c:\Python26\lib\urllib2.py", line 1113, in do_open
        r = h.getresponse()
      File "c:\Python26\lib\httplib.py", line 974, in getresponse
        response.begin()
      File "c:\Python26\lib\httplib.py", line 391, in begin
        version, status, reason = self._read_status()
      File "c:\Python26\lib\httplib.py", line 355, in _read_status
        raise BadStatusLine(line)
    httplib.BadStatusLine
    With this from pyTivo:
    Code:
    2009-10-19 14:15:06,750 ERROR pyTivo: Exception during request from ('192.168.0.60', 1041)
    Traceback (most recent call last):
      File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
        self.finish_request(request, client_address)
      File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
        self.RequestHandlerClass(request, client_address, self)
      File "C:\Program Files\pyTivo\httpserver.py", line 56, in __init__
        client_address, server)
      File "C:\Python26\lib\SocketServer.py", line 615, in __init__
        self.handle()
      File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
        self.handle_one_request()
      File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
        method()
      File "C:\Program Files\pyTivo\httpserver.py", line 82, in do_GET
        self.handle_query(query, tsn)
      File "C:\Program Files\pyTivo\httpserver.py", line 132, in handle_query
        method(self, query)
      File "C:\Program Files\pyTivo\plugins\video\video.py", line 331, in Push
        file_info['valid'] = transcode.supported_format(file_path)
      File "C:\Program Files\pyTivo\plugins\video\transcode.py", line 840, in supported_format
        if video_info(inFile)['Supported']:
      File "C:\Program Files\pyTivo\plugins\video\transcode.py", line 633, in video_info
        mtime = os.stat(inFile).st_mtime
    WindowsError: [Error 3] The system cannot find the path specified: '\\\\dl380-server\\Playground\\ToTivo\\processed\\processed\\702 -
    Vaya Con Cornholio.avi'
    I take it with the new release, I don't need the "processed" directory (the files moved to root watched directory), and I need to manually add the .processed extension to those files already transferred?
     
  9. Oct 19, 2009 #9 of 216
    orangeboy

    orangeboy yes, I AM orangeboy!

    4,083
    0
    Apr 19, 2004
    East Moline, IL
    OK, I did all this, and auto_push is working as expected. Adding the .processed extension to the video files seemed easier than creating 0 byte files in a "quick and dirty" kind of way.
     
  10. Oct 19, 2009 #10 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    The idea is to not interfere at all with original video files since they could be used for other purposes and you don't want to start changing file extensions. The auto_push script automatically creates the empty .processed files anyway so there is nothing manual to do (other than being annoyed by these empty .processed files I guess :D ).
     
  11. Oct 19, 2009 #11 of 216
    orangeboy

    orangeboy yes, I AM orangeboy!

    4,083
    0
    Apr 19, 2004
    East Moline, IL
    In actuality, I could have deleted the files that were in the /processed directory, and not messed with .processed files at all! I don't archive or transcode for other devices
     
  12. Oct 20, 2009 #12 of 216
    mathwhiz

    mathwhiz Member

    388
    0
    May 28, 2000
    Okemos, MI, USA
    A while ago I had posted (not sure if it was here or in the pyTiVo forums) that I had developed code to do auto-transfers. My current incarnation works only with RSS feeds. It checks the RSS feed, downloads the file, and then pushes it to the TiVo, then deletes the file. If anyone is interested I can cleanup my code and post it here. I've been using this for many months and works reliably most of the time. Once in a while it hangs and I have to restart the service.
     
  13. Oct 20, 2009 #13 of 216
    reneg

    reneg Member

    699
    0
    Jun 19, 2002
    It's posted on the pyTivo forums. From what I can tell, it seems a little bit out of date with current builds. I'd give it a try if it were in sync with current pytivo builds.

    My end goal is to have a single program which reads the www.hd-trailers.net RSS feed, downloads new movie trailers in a configured resolution to my PC, clean up trailers older than x days on my PC, and automatically push the downloaded movie trailers to my Tivo. Right now, I'm using one program I found on the net to download/clean up trailers on my PC and this script to push those trailers to my Tivo. I haven't had a lot of time to play around with python lately, so my progress on a single program has been limited.
     
  14. Oct 21, 2009 #14 of 216
    mathwhiz

    mathwhiz Member

    388
    0
    May 28, 2000
    Okemos, MI, USA
    I'll post my latest code sometime soon. I've been keeping my code updated with the latest build, just haven't posted it anywhere because I didn't see much interest in it. The way it works is you give it a list of RSS feeds, it downloads the file in the feed, pushes it to the TiVo, then deletes the file. It keeps track of while files were processed from the feed as well.
     
  15. Oct 23, 2009 #15 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I've now got a Java version with a GUI to aid in setup if anyone is interested. You now point it at the pyTivo.conf file and it gets most information needed from there (pyTivo port, ffmpeg location and the video shares available, and also checks if tivo username & password are set). It also uses mdns to poll for available Tivos to push to on your local network. Once setup you can also run as a service in Windows or background process on other OSs. I don't want to put too much more effort into it if I'm going to be the only one using it. If there is some interest I can setup a Google project home for it.
     
  16. Oct 23, 2009 #16 of 216
    lrhorer

    lrhorer Active Member

    6,924
    0
    Aug 31, 2003
    San...
    I'm interested. Two things which would be really nice, however, are the ability to confirm the transfer succeeded and the ability to run a script of the user's choice against content which transfers successfully. Given the nature of the push, however, I'm not sure this is possible.
     
  17. Oct 23, 2009 #17 of 216
    orangeboy

    orangeboy yes, I AM orangeboy!

    4,083
    0
    Apr 19, 2004
    East Moline, IL
    I'm using it :)
     
  18. Oct 23, 2009 #18 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    Checking if the pyTivo push call was successful is already in place in the new code - however that just checks that pyTivo was able to initiate the push, not that the full push itself to the Tivo completed successfully. And of course there is no way for a process outside of pyTivo web server itself to know when the push has actually completed.
     
  19. Oct 23, 2009 #19 of 216
    reneg

    reneg Member

    699
    0
    Jun 19, 2002
    Isn't it possible to grab the NPL from the Tivo to verify a transfer is in progress or has completed? Tivo could make this easier by providing web access to the todo list and recording history list.

    I'm using your script, but don't need any fancy config tools.
     
  20. Oct 23, 2009 #20 of 216
    moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    Checking NPL may provide some clues especially if there is an accompanying metadata as part of the push where for example the title & description pulled from TiVo should match that of the metadata file. Note sure if there is a clue in NPL xml for still-transferring pushes vs completed pushes.

    Aside from the graphical config and pulling of data from pyTivo.conf the Java version also enables to run as a service for Windows machines which I am familiar with doing based on how I did it with kmttg. I'm not familiar enough with python to figure out how to run python script as a service in Windows. Perhaps using the messy "srvany" helper script but there is likely a more elegant way.

    The ultimate place for auto push capability is as part of pyTivo itself since then it is possible to check if a push actually goes through and potentially execute a post-processing step, etc.
     

Share This Page