1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Streaming TiVo Videos

Discussion in 'Developers Corner' started by davidblackledge, Jan 18, 2009.

  1. davidblackledge

    davidblackledge Registered lÜser

    466
    0
    Sep 9, 2008
    NM
    I've got a surprise for you guys... well, the surprise is you guys give up too easily.

    I coded this last weekend, but wanted to package it up and do some more testing...here it is...I even provided a .bat file to run it...woohoo!

    Yes, Virginia, you CAN stream .tivo files!
    Rather simple, actually. Just claim they're a supported video format. video/mpeg, and I even tried video/mp4 at one time and that seems to work as well. It works for files fed directly from a tivo, or files I downloaded to my desktop.

    Attached is a little fun I put together with this feature, my VideoView (still needs work) and my hmo client library. It's still a little rough, but it works. (oh...not attached... too big for the forum... It's on my website at: http://David.Blackledge.com/tivo/NowPlaying.zip ) Unzip it, change the launch.txt to have your MAK, and run it. (it also contains the RSS-feed-based video listings, half of which don't work, but some do)

    In and of itself, it's just a demo and/or silly, but it leaves the door open for creating one's own UI for now playing stuff, to a degree, and merging interfaces however you like.

    It also notes when you've got a "resume" video (paused somewhere in the middle) but I've tried about 6 or 7 ways of asking the TiVo to send the video starting at that point and it hasn't cooperated... has anybody sniffed the network to see what one Tivo says to the other when he just wants to "transfer from paused point" video?

    I also have one more trick up my sleeve that I want to add, but need some help. I really want somebody to just to point me to a jar file that will do the discovery stuff for me so I can see what is tivo-related on the network. Anybody got one of those? I started reading up on zeroconf and that's a bit confusing to me, plus it doesn't appear to address the rest of the service discovery (apps) that a TiVo does.

    Enjoy, doubters ;]
     
  2. Allanon

    Allanon Member

    581
    0
    Nov 2, 2005
    Good work figuring out how to stream .tivo files, I'm sure this will excite people that store their Tivo videos on their computer in order to free space on the Tivo. But I was a little disappointed that it didn't stream HD videos in real time without pauses, SD videos streamed in real time without pauses. It seems to stream at the same speed as it takes to transfer the video back to the Tivo using Tivo Desktop.
     
  3. wmcbrine

    wmcbrine Ziphead

    10,369
    22
    Aug 2, 2003
    Nope, I still can't stream .tivo files with HME/VLC, even marking them as video/mpeg. Perhaps it's some difference on the web server side, like the TiVo wants a Content-length header or something.
     
  4. wmcbrine

    wmcbrine Ziphead

    10,369
    22
    Aug 2, 2003
    OK, here's the winning combo: in the HME call to create the stream, use video/mpeg; but in the Content-type header, use video/x-tivo-mpeg. Both must be present. Pretty messed up.

    Anyway, thanks for finding this.
     
  5. davidblackledge

    davidblackledge Registered lÜser

    466
    0
    Sep 9, 2008
    NM
    Wow... that's pretty weird... I didn't do anything special to stream the file from my computer, I just had my Factory find the file and send it along. Java must have taken care of the Content-type for me automatically in combination with whatever the TivoDesktop installation did to associate the .tivo files with that mime type.

    Allanon: get me an HD TV and I'll test that out for you :p - Not suprising that there was no speed difference, actually. Try getting your TiVo and PC hardwired into your network to boost your transfer speed.

    ----

    Let me know if anybody finds out the key that TivoConnect is looking for to send a mid-stream file. That will make things much more interesting ;]
     
  6. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    What would be REALLY nice is if we can find a way to stream RAW transport stream variety of .TiVo files, (x-tivo-raw-tts) instead of the program stream variety.

    As you guys probably know one can add &Format=video/x-tivo-raw-tts to TivoToGo transfer URLs to download the raw .TiVo file (which is what MRV transfers between S3 Tivos does). The advantage is that you get MRV type transfer speeds which are typically double TTG speeds since there is no conversion taking place. This would be especially useful to stream back HD mpeg2 to Tivos that would buffer fast enough for uninterrupted playback.

    I tried various combinations of mime type during creation and Content-Type headers but couldn't get any of them to work. Some of the combos I tried that didn't work:
    video/mpeg + video/x-tivo-raw-tts
    video/m2ts + video/x-tivo-raw-tts
    video/mpeg + video/x-tivo-raw
    video/mpeg + video/x-tivo-mpeg
    video/m2ts + video/x-tivo-tts
    (Just in case I also added tsn as a header but it didn't make any difference)
     
  7. davidblackledge

    davidblackledge Registered lÜser

    466
    0
    Sep 9, 2008
    NM
    Wow, you totally lost me, moyekj.
    Ok, I read it again, and I think I follow... let's see if I got this right:

    So what I'm streaming is the link provided in the XML file... which I guess is the kind of link that lets you download a .tivo file that would play on your computer with the right codec installed. I assume that's also the same format as what TivoToGo would give you.

    You're saying that if one supplies the special Format argument to the URL (no, I didn't know about that one for video... grr.. when are they gonna open that wiki up?) then you get another, faster format.
    That faster format is what TiVos use when sharing with each other for Multi-room Viewing (MRV).



    I see that the format argument is the same as the content-type that the XML claims the video is... which is interesting.

    Your experiments have been unable to stream the faster format.

    I'd like to recommend experimenting with using "video/mp4" for the protocol contentType argument, and probably video/x-tivo-raw-tts as the Content-Type header.
    Oh... wait a minute... are you trying to stream raw files from your computer?? That confuses me (you downloaded the raw file to your computer then turned around and tried to feed it back to TiVo?)... But you must going from your computer since going from the TiVo directly doesn't give you the opportunity to control its headers which you say you're doing... unless you're proxying the connection for some reason. Surely the TiVo sends an appropriate Content-Type header when you download with that format. Whatever it sends is the right one... at least it better be, or we're up a creek unless we do the proxying, I guess.

    If we can get that working, I strongly suspect that's a prerequisite for getting the start-in-the-middle transfer to work since that's the format the MRV is using in that circumstance.


    By the way...hopefully keary is reading this:
    I noticed when playing around that if I have a video running and try to start another video, The second one doesn't fail, but instead I think it stops at the "READY" state. What I think is important about this is: With that feature, surely one could pre-fetch the next 1GB section of the movie before the first section quits, then simply destroy the first section and play the second, already-queued section... I assume currently keary is just killing the first section and at that time starting to queue the second section. This other technique should reduce the pause between sections. (no, I haven't downloaded Keary's work yet... I think it's been less than 24 hours for goodness sakes)


    Also, can anybody recommend a good, clean sequence for ending a streamed video so the next one won't break? I eventually figured out that sometimes when I can't stream it's because something broke the feature and all further videos that would have worked just get a "Connection Failed" error. I have occasionally seen the TiVo apparently continuing to show the video after I killed it a long time ago (e.g. when I go back to TiVo central the background is temporarily my video playing for a moment).
    I was doing resource.remove() but I guess that's not enough to definitively clean up the streaming player inside the TiVo.
    Perhaps it's just a fragile underlying system like the mp3 player.
     
  8. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    David, OK I'll break it down just to clarify:

    For normal Tivo->PC TTG transfers you can get simple listing of programs available to TTG as follows:
    Code:
    https://<ip>/nowplaying/index.html
    Then you have links to download that look like for example:
    Code:
    http://<ip>/download/Two&#37;20and%20a%20Half%20Men.TiVo?Container=%2FNowPlaying&id=1683865&Format=video/x-tivo-mpeg
    Using that link you get a normal TTG transfer and an mpeg2 program stream .TiVo file. But if you substitute the Format option above as below then you get the raw Tivo mpeg2 transport stream file (which is what MRV transfers are doing which I discovered a long while ago doing network sniffing) and hence the download is quite a bit faster:
    Code:
    http://<ip>/download/Two%20and%20a%20Half%20Men.TiVo?Container=%2FNowPlaying&id=1683865&Format=video/x-tivo-raw-tts
    Currently this raw .Tivo file is pretty useless on the PC since it's encrypted and and cannot be decoded like the normal program stream variety .TiVo.
    It's this raw .TiVo file that I would like to be able to stream back to Tivo which supposedly should be a faster transfer back just as it is a faster transfer fom Tivo->PC.
    i.e. The only potential value of getting this to work is to take advantage of faster transfers between PC & Tivo. TTG transfers can vary a lot but seem to cap out around 13 Mbps or so, while MRV transfer speeds can quite easily be 30 Mbps or better. 30 Mbps is plenty to stream HD better than real time.
     
  9. kearygriffin

    kearygriffin New Member

    346
    0
    Jan 31, 2008
    I did play around with this for a while, and never could get it working quite right, so gave up. It was a few weeks ago, and streambaby itself wasn't really working very well yet, so its quite possible that I need to have another go at it. And like I think you said at the beginning of this thread, maybe I gave up too easily ;-)

    Also about closing down the stream properly, I'm not really sure. It hasn't happened to me in a while, but I was definitely seeing Tivo refuse to stream anything till it had a chance to "cool down". I haven''t been banging my tivo as much as I was in earlier testing of streambaby, so I'm not sure if I close down the streams in a way that makes the problem not appear, or if I just need to try harder to make it happen.
     
  10. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    In the hope this may be of help to someone looking into raw tivo file transfers (via HMO or HME) here's a summary of packet sniffing of an MRV transfer. (Note that I intentionally replaced last 8 digits with xxxxxxxx at end of tsn numbers).

    The trickiest part seems to be the authentication piece, namely how to come up with:
    response="4bd238b4ed6d8b3194fb224defc27f51"

    NOTE: In case it's not obvious, 192.168.1.107 is requesting a video from 192.168.1.101 in this example.

    311 43.523098 192.168.1.107 192.168.1.101 HTTP
    ----------------------------------------------------
    GET /download/Grey's&#37;20Anatomy.TiVo?Container=%2FNowPlaying&id=1169277&Format=video%2Fx-tivo-raw-tts HTTP/1.1
    Cookie: sid=45F68EE391E5B4D7
    host: 192.168.1.101:80
    Range: bytes=0-
    User-Agent: TvHttpClient
    tsn: 6480001xxxxxxxx
    Connection: close

    313 43.974932 192.168.1.101 192.168.1.107 HTTP
    ----------------------------------------------------
    "HTTP/1.1 401 Authorization Required
    Server: tivo-httpd-1:9.2a.B1-01-2:648
    WWW-Authenticate: Digest realm="TiVo DVR", nonce="39EB249B42E7E598", qop="auth"
    Content-Length: 31
    Content-Type: text/html
    Connection: close

    <h2>Authorization Required</h2>

    320 43.988079 192.168.1.107 192.168.1.101 HTTP
    ----------------------------------------------------
    ">GET /download/Grey's%20Anatomy.TiVo?Container=%2FNowPlaying&id=1169277&Format=video%2Fx-tivo-raw-tts HTTP/1.1
    Authorization: Digest username="tivo", realm="TiVo DVR", nonce="39EB249B42E7E598", uri="/download/Grey's%20Anatomy.TiVo?Container=%2FNowPlaying&id=1169277&Format=video%2Fx-tivo-raw-tts", qop=auth, nc=00000001, cnonce="tivo tcd", response="4bd238b4ed6d8b3194fb224defc27f51"
    Cookie: sid=45F68EE391E5B4D7
    host: 192.168.1.101:80
    Range: bytes=0-
    User-Agent: TvHttpClient
    tsn: 6480001xxxxxxxx
    Connection: close

    323 44.947812 192.168.1.101 192.168.1.107 TCP
    ---------------------------------------------------
    ">HTTP/1.1 200 File Follows
    Server: tivo-httpd-1:9.2a.B1-01-2:648
    Content-Type: video/x-tivo-raw-tts
    TiVo-Estimated-Length: 4869586944
    Connection: close
     
  11. kearygriffin

    kearygriffin New Member

    346
    0
    Jan 31, 2008
    I haven't done a lot of playing with HMO, but this seems to be the wrong question. The question should be how to make the tivo request a raw video from a source other than another tivo.

    The request= above is the requesting tivo "logging on" to the source TiVo using digest authentication (If I am following correctly). It is up to the source TiVo to decide if the response= key is correct. Since in our case "we" would be the source TiVo, so we could just respond in the affirmative (i.e. accept the digest authentication regardless of what it sends).
    Edit: (Actually since we would be the source, we probably wouldn't even have to request authorization in the first place, unless for some reason the TiVo was unhappy without it)

    Full Disclosure: I only have the single TiVoHD plus some disconnected Series 1s, so I've never actual seen how room-to-room transfers function.
     
  12. davidblackledge

    davidblackledge Registered lÜser

    466
    0
    Sep 9, 2008
    NM
    Interesting...thank you very much. I'm trying to play with a little bit of that right now. Would you mind doing the same thing with a transfer of a something from the "pause point" somewhere other than the beginning? I assume the "Range" header will change, but I want to know if the URL changes as well.
    Thanks in advance.
    (I'm getting slowed down by the fact that my Series 3 is refusing to respond to TivoConnect queries??? luckily I've got a HD that's not giving me any trouble)
     
  13. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I know very little about HMO as well, but a server running Tivo Desktop or pyTivo does just that currently, at least for regular mpeg2 files.

    David, I'm pretty certain only the Range field changes when starting from a pause point. This MRV capture I posted was actually from several months (maybe longer) ago and I'm not currently setup to capture more very easily but can do so if really necessary.
     
  14. ggieseke

    ggieseke Active Member

    4,031
    12
    May 30, 2008
    I wanted to look at the raw format, but changing the last part of the URL to x-tivo-raw-tts just gets me a standard .TiVo transfer. I have tried IE6 & IE7 to my S2DTs.

    Am I missing something or have they disabled that capability recently?
     
  15. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I think this only applies to series 3 Tivos.
     
  16. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
  17. ggieseke

    ggieseke Active Member

    4,031
    12
    May 30, 2008
    Oh, well. From a few other posts I've seen I was thinking that 2.7 might be skipping the transcode for MP4 transfers to an S3 TiVo, but I don't have one to play with since I use DirecTV.

    Thanks for the info.
     
  18. davidblackledge

    davidblackledge Registered lÜser

    466
    0
    Sep 9, 2008
    NM
    Ok, since I finally learned enough (on my own :/ ) about zeroconf/bonjour/jmdns, I've gotten some work done on that "one more trick" which I'm adding to the NowPlaying interface. It's really frickin cool ;] (but I'm easily amused). I need to make it prettier/friendlier, but it works without a major hitch which was a surprise.
    Why am I not telling you what it is yet? Because as soon as I do somebody will go out and do a better one, and I can't compete with just a few hours a week. I'd at least like to get the first version. (and if I'm just too paranoid and nobody will write one, then it won't make a difference because you have to wait for me to finish it ;] )

    Anyhow, a major stumbling block or two with the zeroconf stuff...I'm hoping somebody (preferably somebody from Galleon) can help me out on this.

    1) Running NowPlaying+zeroconf stand-alone using the sample server it works perfectly except that the Galleon Video service presents itself with <full.numeric.ip>.local. as its zeroconf host address while other stuff either has a <name>.local. or <number>.local. (one number from ip address with no additional dots)... my stand-alone can't look up those full ip number versions correctly (for now I'm working around it by just removing the ".local." at the end if there is an UnknownHostException, then hoping for the best)... that's making it awkward to serve Galleon's videos as "another DVR" in the NowPlaying app.
    That's the least of my problems, and I expect it to go away if I can run within Galleon and use whatever version of zeroconf it's using.

    2) Attempting to run NowPlaying from within Galleon is just wacko. I was going to try it with an older version of Galleon and including the Bonjour jmdns jar that I'd downloaded/installed, but I saw in Galleon change log that they had updated its zeroconf stuff, so I thought I'd install the newest version and see if it magically worked. It didn't. In fact, it complained that it couldn't load the native library (dll).
    That makes no sense to me since Galleon itself is presumably SERVING zeroconf data using that same library.
    I found a copy if it as a dll in C:\windows\system32, I think, and copied it into Galleon's lib folder so it would be in the load library path. I honestly don't recall exactly what the result was after that, but it was different... it just didn't work still. I believe it broke and didn't load any of the hme apps for serving at all, at least once it hit the app that made use of it. In reality going this direction was silly anyhow... I must be missing some point with Galleon itself... perhaps I have an old Jar somewhere that includes jmdns which is somehow in my classpath but isn't in the main Galleon server's classpath.
    I tested some stuff last night and found that Galleon was serving apps fine (with NowPlaying removed from its list) after a restart of the service, but in the middle of the night, it just stopped serving them...nothing in any logs. I had that dll file still in the lib folder. So I removed it and restarted the service this morning... haven't had a chance to see what the result was.

    Wild guess is the .dll I had is different from what Galleon uses, but is overriding it... it serves fine, but doesn't send out reminders or doesn't respond to tivo "who's there?" requests after a while...or whatever it is that zeroconf does.

    So does any of this mean anything to anybody? or am I still on my own?
    I guess the short question is:
    How do I correctly use Zeroconf (DNSSD.browse, etc) from within Galleon's environment?
     
  19. Allanon

    Allanon Member

    581
    0
    Nov 2, 2005
    Ever since I saw your NowPlaying demo I thought someone would write an application that streams video from one Tivo on the home network to another. This would take the place of MRV allowing you to instantly stream a video instead of transferring it. Hopeful this is what you are working on. :)
     
  20. moyekj

    moyekj Well-Known Member

    11,150
    33
    Jan 23, 2006
    Mission...
    I thought about that quite a lot a while back, but one issue is that TTG protocol is being used instead of straight MRV which means maximum sustained transfer rates are approximately cut in half compared to MRV which makes it less than ideal and virtually impossible for real time HD mpeg2 streaming. Then of course is the nagging issue of the ~1.1GB streaming buffer limit... Both of those conspire to make MRV a much better choice, especially for HD transfers.
     

Share This Page