Tivo Proxy Server

[Allanon]

Hi, I'm writing a simple proxy server for my TiVo that will stream and decode .tivo files directly from the TiVo. This will allow media players like VLC and media servers like TVersity to play and server the files. I am also going to server a RSS feed so media servers can find the Tivo files.

Right now I have a simple prototype that works with VLC:

import cgi, os, SocketServer, sys, time, subprocess
import urllib2
from SimpleHTTPServer import SimpleHTTPRequestHandler
from StringIO import StringIO

Tivo_MAK = '***********'
Tivo_URL = 'http://192.168.1.106'
Port = 8000


def findPID(exename):
    a = os.popen4('tasklist /FI "IMAGENAME eq '+exename+'"')
    a[0].flush()
    try:
         info=a[1].readlines()[3].split()
    except:
        info=[exename,"NotFound"]
    return info[1]



class DirectoryHandler(SimpleHTTPRequestHandler):
   
    def do_GET(self):
        global Tivo_MAK
        global Tivo_URL
            
        # create TiVo URL
        link = Tivo_URL + self.path
        print link
   
        # Send header
        self.send_response(200)
        self.send_header("Content-Type", "video/mpeg")
        self.end_headers()
         
        # Get and decode the Tivo file     
        decode = subprocess.Popen('curl.exe --digest -k -c cookies.txt -u tivo:{0:s} "{1:s}" | tivodecode -m {0:s} -- -'.format(Tivo_MAK,link),shell=True,bufsize=1000,stdout=subprocess.PIPE)
        processID = findPID('curl.exe')

        # send decoded data to the client
        while True:
            chunk = decode.stdout.read(4096)
            if not chunk:
                break
            try:
                self.wfile.write(chunk)
            except:
                os.popen('TASKKILL /PID '+ processID +' /F')
                time.sleep(1)
                break
        
         
httpd = SocketServer.TCPServer(("", Port), DirectoryHandler)
print "serving at port", Port
httpd.serve_forever()

With this running I can open VLC and pass an URL that looks like this:

http://192.168.1.100:8000/download/Auction%20Kings.tivo?Container=%2FNowPlaying&id=4707752&Format=video/x-tivo-mpeg

And the server will change the URL to the TiVo URL and run curl and tivodecode then pass the output back to VLC.

This works in VLC, but I can't get any other media player or media server to recognize the output. I think it's because they can't determine the codec to use, they will open and close the link a few times before saying it can't play. My question is what exactly is output from tivodecode and what content-type and header info should I send? I tried "video/mpeg","video/mp4","video/mpgv" but windows media player won't play the output.

[wmcbrine]

The output from tivodecode should be an MPEG-2 program stream. I believe "video/mpeg" is the correct MIME type for this. But I don't know what other software besides VLC you've tried, what its capabilities are, or what it expects.

[Allanon]

Originally Posted by wmcbrine

The output from tivodecode should be an MPEG-2 program stream. I believe "video/mpeg" is the correct MIME type for this. But I don't know what other software besides VLC you've tried, what its capabilities are, or what it expects.

I tried VLC, Windows Media Player and Serviio media server. VLC played the stream no mater what MIME was on the stream. And Serviio Media Server would only play the stream if it was designated as an internet stream and a MIME of "video/mpeg" but if the same exact URL was passed using an RSS feed it would not play. Windows Media Player would not play the stream at all. It starts the stream a few times then says it can't play it.

My goal is to stream videos from my TiVo to my Sony TV which has an UPNP/DLNA client. I would like to feed Serviio media server an RSS feed with proxy links to all the videos on the TiVo. Then I can use the UPNP/DLNA client to select videos and have them streamed using the proxy server. I don't want to store TiVo files on my computer.

Update:
I got RSS and video working with the PS3 Media Server. This media server satisfies all my media server needs so I am going to stop trying to get this proxy server working with other players and servers. I will post here the final code to my proxy server when it's complete.

[ferrumpneuma]

I can't believe that works at all telfpountee.

[Allanon]

Below is the final code that I'm using for the Tivo Proxy Server. It isn't pretty and doesn't do a lot of error checking but it works for me. You can use this code for whatever you like without restrictions.

To use just run the Tivo Proxy Server.py code then pass a URL that looks like this:

http://192.168.1.100:10000/rss

This assumes:
Server_URL = 192.168.1.100
Port = 10000

That url will return an RSS FEED to the specified Tivo's now playing list. The .tivo URLs in the RSS feed will point to the proxy server and when a .tivo file is requested the proxy server will download the .tivo file and convert it to an .mpg and stream it to the requesting application.

This has been tested with VLC and PS3 Media Server. I use this proxy server to stream videos from the TiVo to my Sony TV which has a UPNP / DNLA client which connect to the PS3 Media Server.

[ejonesss]

after i set up the server i get invalid syntax at print link

i guess print is not a valid command

[vanclute]

I just stumbled across this thread and am very intrigued to try this. I hope I'm doing something wrong because it doesn't seem to function at all for me.

I run the .py file in Terminal after editing it to include my MAK, Tivo IP, and server IP (my Mac media server which is running the .py script).

This is what I get:

users-Mac:~ user$ /Users/user/Desktop/Tivo\ Proxy\ Server.py ; exit;
/Users/user/Desktop/Tivo Proxy Server.py: line 1: import: command not found
from: can't read /var/mail/SimpleHTTPServer
from: can't read /var/mail/StringIO
: command not found/Tivo Proxy Server.py: line 4:
/Users/user/Desktop/Tivo Proxy Server.py: line 7: Tivo_MAK: command not found
/Users/user/Desktop/Tivo Proxy Server.py: line 8: Server_URL: command not found
/Users/user/Desktop/Tivo Proxy Server.py: line 9: Tivo_URL: command not found
/Users/user/Desktop/Tivo Proxy Server.py: line 10: Port: command not found
: command not found/Tivo Proxy Server.py: line 11:
: command not found/Tivo Proxy Server.py: line 12:
/Users/user/Desktop/Tivo Proxy Server.py: line 14: syntax error near unexpected token `('
'Users/user/Desktop/Tivo Proxy Server.py: line 14: `def findPID(exename):
logout

Any idea where I've gone wrong?

[Allanon]

Originally Posted by vanclute

Any idea where I've gone wrong?

Only thing I can think of is to make sure you have all the modules installed. I'm not really a Python expert and I don't have a Mac. I taught myself how to program Python so there can be numerous things wrong with the code. It works on my Windows XP computer but I have no knowledge of how to get it working on other operating systems.

Also checkout this thread where I explain some of the problems other operating systems might have:
http://www.tivocommunity.com/tivo-vb...d.php?t=496286

[wmcbrine]

Originally Posted by vanclute

Any idea where I've gone wrong?

It's treating it as a shell script rather than as Python. You can either launch it as "python scriptname", or add this as the first line:

#!/usr/bin/env python

[vanclute]

Aha good point. OK I added the line, but now I get this:

users-Mac:~ user$ /Users/user/Desktop/Tivo\ Proxy\ Server.py ; exit;
Traceback (most recent call last):
  File "/Users/user/Desktop/Tivo Proxy Server.py", line 121, in <module>
    httpd = SocketServer.TCPServer(("", Port), Handler)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 408, in __init__
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 419, in server_bind
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
socket.error: [Errno 48] Address already in use
logout

[Process completed]

I quit PyTivoX in case that was conflicting but that didn't help.

[Allanon]

Since there seemed to be some interest in this program I went ahead and made the code less OS dependent. I removed all Windows specific code such as the process ID check function and the need for curl.exe. I tested the program using Python 2.7. You will still need to place tivodecode in the same path as the program. You can get a tivodecode binary file from kmttg's Helper Tools. Also you will still need to replace the IP, port and MAK parameters located near the top of the code to reflect your computer and Tivo network settings.

[vanclute]

Originally Posted by Allanon

You will still need to place tivodecode in the same path as the program.

Aha! I missed this requirement and had not done so at all. Hopefully that, combined with your new OS-independence changes, will yield better results for me at last. I shall report back ASAP!

[vanclute]

Well rats... no joy. Here is the result:

users-Mac:~ user$ /Applications/kmttg_v0p8u/tivodecode/Tivo\ Proxy\ Server\ 2.py ; exit;
Traceback (most recent call last):
  File "/Applications/kmttg_v0p8u/tivodecode/Tivo Proxy Server 2.py", line 148, in <module>
    httpd = SocketServer.TCPServer(("", Port), Handler)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 408, in __init__
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 419, in server_bind
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
socket.error: [Errno 48] Address already in use
logout

[Process completed]

Any suggestions on where to go from here?

[Allanon]

Originally Posted by vanclute

Any suggestions on where to go from here?

Did you try changing the port number?

Edit:

I also went online to see if I can find a solution and you might try using this line of code:

Replace:

# Start Proxy Server
httpd = SocketServer.TCPServer(("", Port), Handler)

With:

# Start Proxy Server
SocketServer.TCPServer.allow_reuse_address = True
httpd = SocketServer.TCPServer(("", Port), Handler)

[wmcbrine]

Yes, change the port number. (Don't do the reuse mod.)

[bradleys]

I have the script running and I can see the rss feed. When I try to play the video in VLN I get nothing.

Looks properly formated...

"http://192.168.1.6:10000/download/Fringe.tivo?Container=%2FNowPlaying&id=2337518&Format=video/x-tivo-mpeg"




Is it possible we could have a DNLA server built into PyTiVo?

[vanclute]

ok, progress! The server is now serving on port 10001 and I can see a list of programs on the tivo! Playing one however, still fails thusly:

Video URL:  http://192.168.2.10/download/The%20O...ing&id=1337050

192.168.2.16 - - [12/Jan/2013 17:50:29] "GET /download/The%20Owl%20and%20the%20Pussycat.TiVo?Container=%2FNowPlaying&id=1337050 HTTP/1.1" 200 -

Decoding video

----------------------------------------
Exception happened during processing of request from ('192.168.2.16', 53674)
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 310, in process_request
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 323, in finish_request
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 639, in __init__
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 337, in handle
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 325, in handle_one_request
  File "/Applications/kmttg_v0p8u/tivodecode/Tivo Proxy Server 2.py", line 127, in do_GET
    decode = subprocess.Popen('tivodecode -m {0:s} -- -'.format(Tivo_MAK),shell=False,bufsize=0,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 672, in __init__
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1202, in _execute_child
OSError: [Errno 2] No such file or directory
----------------------------------------

So it's missing something... can you tell what?

So close!

Also I notice it's showing only a handful of programs, certainly nowhere near everything that's on the Tivo.

[bradleys]

Originally Posted by vanclute

ok, progress! The server is now serving on port 10001 and I can see a list of programs on the tivo! Playing one however, still fails thusly:

So it's missing something... can you tell what?

So close!

Also I notice it's showing only a handful of programs, certainly nowhere near everything that's on the Tivo.

Looks like you and I are seeing something very similar....

I am also only seeing a subset of the actual content on my TiVo in the RSS feed.

Video URL:  http://192.168.1.7/download/Fringe.tivo?Container=%2FNowPlaying&id=2337518&Format=video/x-tivo-mpeg

HOME_OFFICE.home - - [12/Jan/2013 19:33:48] "GET /download/Fringe.tivo?Container=%2FNowPlaying&id=2337518&Format=video/x-tivo-mpeg HTTP/1.1" 200 -

Decoding video

----------------------------------------
Exception happened during processing of request from ('192.168.1.6', 57892)
Traceback (most recent call last):
  File "C:\Python27\lib\SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python27\lib\SocketServer.py", line 638, in __init__
    self.handle()
  File "C:\Python27\lib\BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "C:\Python27\lib\BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "C:\Program Files (x86)\TivoProxy\Tivo Proxy Server.py", line 125, in do_GET
    decode = subprocess.Popen('tivodecode -m {0:s} -- -'.format(Tivo_MAK),shell=False,bufsize=0,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
  File "C:\Python27\lib\subprocess.py", line 679, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 896, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
----------------------------------------
/

Definately looks like some file is not being found...

[vanclute]

My first thought was that it's not finding something it needs from tivodecode, but I would have expected the error message to be a bit more specific if that were the case. I don't have any other theory at this moment. =(

[bradleys]

That was my thought as well... So I tried placing the TiVodecode files in the root directory (C:\Program Files (x86)\TivoProxy) to see if that would help. It didn't.

I am sure we are doing something simple wrong.

[Allanon]

The URL posted in the image looks different than the ones I'm getting. My URLs are formatted like this:

http://192.168.1.114:10000/download/NFL%20Football.TiVo?Container=%2FNowPlaying&id=5377217

Try removing the "&Format=video/x-tivo-mpeg" in your URL and see if it plays.

What type of Tivo do you guys have? I have a Tivo HD and I did a hack to convert the XML file from the Tivo to an RSS feed. If you guys have a Premiere then maybe the XML file is different and the hack is messing up. I will fix the code to use XMLLib and read the XML file properly.

Can someone post the XML Now Playing List from their Premiere so I can do a comparison?

Also, in VLC you can add the RSS feed as a podcast and have the Tivo Now Playing List show up in the VLC playlist. This will prevent you from having to enter the URLs manually.

[bradleys]

I have an HD machine as well. And temper this with the knowlege that I am half a bottle down on a fantastic Cab tonight, but I am still not getting a result from the VLN player using:

http://192.168.1.114:10000/download/Grimm.TiVo?Container=%2FNowPlaying&id=2979432

let me see what I get from the console.....

serving at port 10000
/download/Gold%20Rush.TiVo?Container=%2FNowPlaying&id=1972195

Video URL:  http://192.168.1.4/download/Gold%20Rush.TiVo?Container=%2FNowPlaying&id=1972195

HOME_OFFICE.home - - [12/Jan/2013 23:50:04] "GET /download/Gold%20Rush.TiVo?Container=%2FNowPlaying&id=1972195 HTTP/1.1" 200 -

Decoding video

----------------------------------------
Exception happened during processing of request from ('192.168.1.6', 50601)
Traceback (most recent call last):
  File "C:\Python27\lib\SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python27\lib\SocketServer.py", line 638, in __init__
    self.handle()
  File "C:\Python27\lib\BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "C:\Python27\lib\BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "C:\Program Files (x86)\TivoProxy\Tivo Proxy Server.py", line 125, in do_GET
    decode = subprocess.Popen('tivodecode -m {0:s} -- -'.format(Tivo_MAK),shell=False,bufsize=0,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
  File "C:\Python27\lib\subprocess.py", line 679, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 896, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
----------------------------------------

Same result from my HD, definately something I am doing wrong.

[Allanon]

In this line of code:

decode = subprocess.Popen('tivodecode -m {0:s} -- -'.format(Tivo_MAK),shell=False,bufsize=0,stdin=subprocess.PIPE,stdout=subprocess.PIPE)

Try putting in the path to tivodecode, for example if tivodecode is in c:\tivodecode then make the code look like this:

decode = subprocess.Popen('c:\\tivodecode\\tivodecode -m {0:s} -- -'.format(Tivo_MAK),shell=False,bufsize=0,stdin=subprocess.PIPE,stdout=subprocess.PIPE)

Also make sure you are running as administrator, maybe it's an authorization problem. I'm using Windows XP and run as administrator. I tried running without tivodecode and got a different error than the one you posted so I'm not sure what is causing that error.

[vanclute]

I've got a Premiere as well (actually two, but I wasn't going to broach the question of running two servers until I had at least one working!)

Try removing the "&Format=video/x-tivo-mpeg" in your URL and see if it plays.

I'm not sure what you meant there, as I don't have that in my URLs....?

I would gladly post the XML Now Playing list but I have no idea how to get it. If you can clue me in, I'll contribute what I can.

As for RSS feeds in VLC, I suspect that might be a Windows-only feature, I don't see anything relating to RSS feeds at all in VLC for Mac.

I'll try the absolute path to tivodecode, I considered that but wasn't sure of the syntax. Will give it a shot and report back!

[vanclute]

No change by adding the absolute path to Tivodecode. I think that part is at least working, it looks like maybe something about my python build is the problem. Apple includes python on the OS but maybe I need to try downloading a binary or something. I'm going to investigate that.

[Allanon]

Originally Posted by vanclute

I've got a Premiere as well (actually two, but I wasn't going to broach the question of running two servers until I had at least one working!)

You can probably run 2 or more servers at a time just use different port numbers. But your computer is probably not fast enough to decode 2 HD videos at the same time. Mine has trouble just decoding one HD video.

I'm not sure what you meant there, as I don't have that in my URLs....?

The URL Bradleys posted had that at the end.

I would gladly post the XML Now Playing list but I have no idea how to get it. If you can clue me in, I'll contribute what I can.

To get the XML file just enter the following in to your internet browser. I used the Tivo URL from your error message but if that is not correct then change the URL to what it should be:

https://192.168.2.16/TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes&AnchorOffset=0

Then it should ask for a username and password, use the word tivo as the username and your Tivo MAK number as the password. Your browser should then display the Tivo XML file. Just save it to a file and post it here or send it as a private message.

As for RSS feeds in VLC, I suspect that might be a Windows-only feature, I don't see anything relating to RSS feeds at all in VLC for Mac.

Go to your playlist and on the left side it should have a menu item that says internet. Click that and you should see a podcast option. Just add the RSS Feed URL as a new podcast. If you don't see that then maybe it is only Windows.

[wmcbrine]

Originally Posted by vanclute

No change by adding the absolute path to Tivodecode. I think that part is at least working, it looks like maybe something about my python build is the problem. Apple includes python on the OS but maybe I need to try downloading a binary or something. I'm going to investigate that.

No, that part is NOT working. And there is nothing wrong with your Python.

subprocess.Popen() expects a list, with the command and parameters separate, not a simple string:

['tivodecode', '-m', TiVo_MAK, '--', '-']

not

'tivodecode -m etc.'

It only works as a string in Windows, due to a quirk in how Popen is implemented there.

http://docs.python.org/2/library/sub...en-constructor

[Allanon]

Thanks wmcbrine for pointing that out, I would have never found that.

Alright, here is an updated version. This version includes the modification wmcbrine posted and the program now uses XMLLib to parse the XML file which might work better if the Tivo HD and Premiere XML files are different. I also added more context headers, this seems to have made the video play on more devices such as within the Chrome browser and in the Serviio Media Server.

[bradleys]

Running the RSS in Chrome I now get the error:

This page contains the following errors:

error on line 31 at column 49: xmlParseEntityRef: no name
Below is a rendering of the page up to the first error.



Trying to run in VLN and I get:

serving at port 10000
/download/Gold%20Rush.TiVo?Container=%2FNowPlaying&id=1972195&Format=video/x-tivo-mpeg

Video URL:  http://192.168.1.7/download/Gold%20Rush.TiVo?Container=%2FNowPlaying&id=1972195&Format=video/x-tivo-mpeg

HOME_OFFICE.home - - [13/Jan/2013 09:27:58] "GET /download/Gold%20Rush.TiVo?Container=%2FNowPlaying&id=1972195&Format=video/x-tivo-mpeg HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('192.168.1.6', 53074)
Traceback (most recent call last):
  File "C:\Python27\lib\SocketServer.py", line 284, in _handle_request_noblock
    self.process_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 310, in process_request
    self.finish_request(request, client_address)
  File "C:\Python27\lib\SocketServer.py", line 323, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python27\lib\SocketServer.py", line 638, in __init__
    self.handle()
  File "C:\Python27\lib\BaseHTTPServer.py", line 340, in handle
    self.handle_one_request()
  File "C:\Python27\lib\BaseHTTPServer.py", line 328, in handle_one_request
    method()
  File "C:\Program Files (x86)\TivoProxy\Tivo Proxy Server.py", line 155, in do_GET
    videoFile =  openUrl(link, 'tivo', Tivo_MAK)
  File "C:\Program Files (x86)\TivoProxy\Tivo Proxy Server.py", line 25, in openUrl
    return urllib2.urlopen(url)
  File "C:\Python27\lib\urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 406, in open
    response = meth(req, response)
  File "C:\Python27\lib\urllib2.py", line 519, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python27\lib\urllib2.py", line 444, in error
    return self._call_chain(*args)
  File "C:\Python27\lib\urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 527, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
----------------------------------------

[Allanon]

Bradleys, what type of Tivo are you using to test this program? Also, can you send me the XML file from your Tivo so I can write code that reads it?

Your Tivo video URL is different than the one from my Tivo HD. Plus the RSS Feed error doesn't happen on my Tivo. There is something different about your XML file. Do you live in the United States? Maybe non US boxes have different XML Files. I can't help until I see what your XML file looks like.