TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Main TiVo Forums > TiVo Home Media Features & TiVoToGo
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 02-03-2013, 07:19 PM   #1
jemenake
Registered User
 
Join Date: Dec 2000
Location: San Luis Obispo, CA
Posts: 27
Any interest in a Java of tivodecode?

KMTTG, I've noticed, calls tivodecode as an external app. I'm thinking of starting work on a Java app with TTG features, but I'd like to have the decoding entirely in Java.

So, that means making a port of tivodecode to Java. Would anybody else be interested in using this if I ported it? Would anybody else be interested in *helping* to port it?
__________________
Linux - "It is now safe to turn *ON* your computer"
jemenake is offline   Reply With Quote
Old 02-04-2013, 01:09 PM   #2
rfryar
My Media, My Way
 
Join Date: Feb 2008
Location: Cottage Grove, MN
Posts: 206
I have not taken a look at the TivoDecode source, but depending on the complexity of that source it should not be a large effort to convert. Do you have a copy of the C source?

The real question is performance, while spawning an external process can be expensive in Java, the IO overhead of moving a fast native decode process into Java will probably be more expensive in the long run.

Not to knock the idea, I would be up for assisting when I have time, or at least help you debug or bounce ideas off.

Rick
rfryar is offline   Reply With Quote
Old 02-04-2013, 01:18 PM   #3
jemenake
Registered User
 
Join Date: Dec 2000
Location: San Luis Obispo, CA
Posts: 27
Quote:
Originally Posted by rfryar View Post
I have not taken a look at the TivoDecode source, but depending on the complexity of that source it should not be a large effort to convert. Do you have a copy of the C source?
Yup. It's a sourceforge project.

Quote:
The real question is performance, while spawning an external process can be expensive in Java, the IO overhead of moving a fast native decode process into Java will probably be more expensive in the long run.
Well, I'm going to need the kind of control that I can't get by spawning an external process. Specifically, I'd like to make a streaming converter, which grabs a program off of my TiVo, decodes it, and pumps it out to my iPad without (like KMTTG has to do) saving the entire TiVo file to disk and then converting it.

Quote:
Not to knock the idea, I would be up for assisting when I have time, or at least help you debug or bounce ideas off.
Fortunately, it looks like some portion of the code can just be tossed. He has some code which supports reading huge files, which is probably just supported by the stock java.io. Also, he's got some kind of wrapper for reading from the tivo file which does buffering... which may or may not even be necessary for my needs (and which may be supplied by BufferedInputStream), etc.

The tricky part is he's got some C structs which *might* exactly match headers in the file. So, he does a read() and then just sets a pointer to a certain place in the read buffer, casts it as that struct, and then starts reading values. And Java don't play that way. So, that'll take some careful looking over.
__________________
Linux - "It is now safe to turn *ON* your computer"
jemenake is offline   Reply With Quote
Old 02-04-2013, 01:26 PM   #4
moyekj
Registered User
 
Join Date: Jan 2006
Location: Mission Viejo, CA
Posts: 9,202
You still likely have to transcode on the fly to H.264 to play on iPad, and doing that in better than real time requires a beefy multi-threaded processor, and you can't do that within Java unless you're also planning on making a multi-threaded Java based encoder as well that runs fast enough to suit your needs. So if you're going to spawn off an external process for encoding anyway then it kind of defeats the purpose of your goal to stay entirely within Java.

I'm still hoping someone familiar enough with mpeg2 transport stream container can take a look at existing code and fix the remaining issues such that tivodecode can work properly for TS .TiVo files since that's the biggest hole to be filled right now in tivodecode IMO.
__________________
Roamio Pro, Elite, Premiere
Cox - Motorola CableCards & TAs
Slingbox 350 via TiVo Mini & TiVo Stream for remote viewing

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
moyekj is offline   Reply With Quote
Old 02-04-2013, 03:48 PM   #5
ggieseke
Registered User
 
Join Date: May 2008
Posts: 3,020
Quote:
Originally Posted by moyekj View Post
I'm still hoping someone familiar enough with mpeg2 transport stream container can take a look at existing code and fix the remaining issues such that tivodecode can work properly for TS .TiVo files since that's the biggest hole to be filled right now in tivodecode IMO.
That's on my ToDo List, but it will be a while before I get back to it.

I wrote my own version of tivodecode several years back just to get a feel for it. The Turing algorithm is fairly tricky, and you have to be able to parse the MPEG-2 stream down to the PES packet level just to be able to detect which packets are even encrypted (not all of them are).

It was hard enough in C with some assembly thrown in to process the bitstream faster. Can't imagine tackling it in Java.
ggieseke is online now   Reply With Quote
Old 02-04-2013, 05:54 PM   #6
Dan203
Super Moderator
 
Dan203's Avatar
 
Join Date: Apr 2000
Location: Nevada
Posts: 25,370
The other issue with tivodecode is that there is still no proper support for the TiVo-TS format used by Premiere units. So you have to make sure to download in the older, slower, PS format. And going forward there is no guarantee they will retain support for the older PS format in the Premiere software.

There is a beta with TS support but it's not 100%. It creates some errors in the decrypted files. If you're serious about doing something like this you should really look into fixing that to ensure future compatibility. The other option is to do what VideoReDo does and tap into the TiVo DirectShow filter supplied with TiVo Desktop. It can decrypt TS files 100%.
__________________
Dan Haddix
Super Moderator
Developer for VideoReDo
Dan203 is offline   Reply With Quote
Old 02-04-2013, 07:18 PM   #7
jemenake
Registered User
 
Join Date: Dec 2000
Location: San Luis Obispo, CA
Posts: 27
Quote:
Originally Posted by moyekj View Post
You still likely have to transcode on the fly to H.264 to play on iPad, and doing that in better than real time requires a beefy multi-threaded processor,
True. I grant that it may not be feasible to do it for H.264. After all, I'd need someone to have written an open-source H.264 encoder, as writing one from scratch is *far* outside of my expertise.

In the end, the only thing that comes out of it might be that KMTTG could use it and, thereby, produce decoded mpeg files without needing an intermediate .tivo file. I'd also like to make it part of a Java library which would also handle all of the connections to the TiVos (it looks like KMTTG still uses curl), parsing all of the HTML/XML, and returning the contents of each TiVo as some kind of data structure.

Even if I got that far, I probably could just use VLC in streaming/transcoding mode.... or mencoder might do it, as well. Who knows... I'm not even that far along, yet. Right now, I'll be happy if I develop a small library that would let me do something like:

Code:
  TiVo[] mytivos = findTiVos();
  ProgramData tivo0_now_playing = mytivos[0].getProgramData();
  MPEG2Stream theStream = mytivos[0].getProgramAsMPEG2(tivo0_now_playing.getNewestProgram());
Quote:
Originally Posted by moyekj View Post
and you can't do that within Java unless you're also planning on making a multi-threaded Java based encoder as well that runs fast enough to suit your needs. So if you're going to spawn off an external process for encoding anyway then it kind of defeats the purpose of your goal to stay entirely within Java.
Not entirely. Some of the external tools (it appears), don't like pipes. You have to save the entire stream to a file, run an external app, wait for it to finish, and then read the file. That's a deal-killer if you're trying to stream. Like I said, VLC can be set up into transcoding/streaming mode where it can grab a stream from one socket and transcode it and send it right out another socket... so, in a pinch, I can always use that.
__________________
Linux - "It is now safe to turn *ON* your computer"
jemenake is offline   Reply With Quote
Old 02-04-2013, 07:25 PM   #8
jemenake
Registered User
 
Join Date: Dec 2000
Location: San Luis Obispo, CA
Posts: 27
Quote:
Originally Posted by ggieseke View Post
It was hard enough in C with some assembly thrown in to process the bitstream faster. Can't imagine tackling it in Java.
Well, the nice thing is that, if you can do the decoding as it's transferring, then, suddenly, speed isn't a problem.

When I download a 500MB program from my TiVoHD, it takes 8 minutes. Then, tivodecode takes 19 seconds. If you decode at transfer-time, the bottleneck is how fast the TiVo can cough up the stream, and the decoder can now take 25x as long. Even *Java* isn't *that* slow.
__________________
Linux - "It is now safe to turn *ON* your computer"
jemenake is offline   Reply With Quote
Old 02-04-2013, 08:21 PM   #9
moyekj
Registered User
 
Join Date: Jan 2006
Location: Mission Viejo, CA
Posts: 9,202
FYI, in kmttg there is already an option called "Combine download and tivodecode decrypt" which generates decrypted mpeg file directly without intermediate .TiVo file. Personally I don't like to do it that way, but some people requested it so I did implement it.
__________________
Roamio Pro, Elite, Premiere
Cox - Motorola CableCards & TAs
Slingbox 350 via TiVo Mini & TiVo Stream for remote viewing

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
moyekj is offline   Reply With Quote
Old 02-04-2013, 08:27 PM   #10
wmcbrine
Resistance Useless
 
wmcbrine's Avatar
 
Join Date: Aug 2003
Posts: 9,087
Quote:
Originally Posted by moyekj View Post
FYI, in kmttg there is already an option called "Combine download and tivodecode decrypt" which generates decrypted mpeg file directly without intermediate .TiVo file.
pyTivo does this as well.

Quote:
Personally I don't like to do it that way
Why not?
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
wmcbrine is offline   Reply With Quote
Old 02-04-2013, 10:27 PM   #11
moyekj
Registered User
 
Join Date: Jan 2006
Location: Mission Viejo, CA
Posts: 9,202
Quote:
Originally Posted by wmcbrine View Post
Why not?
Mostly because I prefer to use VideoRedo QS Fix to do decrypt + QS Fix in 1 shot (and I can use TS downloads). But even if using tivodecode I'd rather have separate download + decrypt to separate 2 potential points of failure and to have .TiVo file with embedded metadata available as well.
__________________
Roamio Pro, Elite, Premiere
Cox - Motorola CableCards & TAs
Slingbox 350 via TiVo Mini & TiVo Stream for remote viewing

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
moyekj is offline   Reply With Quote
Old 02-05-2013, 02:40 PM   #12
rfryar
My Media, My Way
 
Join Date: Feb 2008
Location: Cottage Grove, MN
Posts: 206
Quote:
Originally Posted by jemenake View Post
Not entirely. Some of the external tools (it appears), don't like pipes. You have to save the entire stream to a file, run an external app, wait for it to finish, and then read the file. That's a deal-killer if you're trying to stream. Like I said, VLC can be set up into transcoding/streaming mode where it can grab a stream from one socket and transcode it and send it right out another socket... so, in a pinch, I can always use that.
I wrote a pipe command that will use curl to download the video, pipe to tivo decode and pipe that into ffmpeg to transcode for my phone. The major problem is that in windows the pipe buffer is ridiculously small. I was going to write a simple JAVA program just to replace the windows command kernel in the middle that has a much larger pipe buffer but never got around to it. (See the albatross thread). The problem is not so much in the tools, but in the OS under the covers.

Rick
rfryar is offline   Reply With Quote
Old 02-05-2013, 02:55 PM   #13
Dan203
Super Moderator
 
Dan203's Avatar
 
Join Date: Apr 2000
Location: Nevada
Posts: 25,370
We tried to add pipe output to VideoReDo once. It was really slooooooow because the pipe buffers are so small. Output was taking 3-4x what it normally took. We ended up scrapping it.

Dan
__________________
Dan Haddix
Super Moderator
Developer for VideoReDo
Dan203 is offline   Reply With Quote
Old 02-06-2013, 10:42 PM   #14
herbman
Registered User
 
Join Date: Apr 2008
Posts: 106
Quote:
Originally Posted by jemenake View Post

Yup. It's a sourceforge project.

Well, I'm going to need the kind of control that I can't get by spawning an external process. Specifically, I'd like to make a streaming converter, which grabs a program off of my TiVo, decodes it, and pumps it out to my iPad without (like KMTTG has to do) saving the entire TiVo file to disk and then converting it.

Fortunately, it looks like some portion of the code can just be tossed. He has some code which supports reading huge files, which is probably just supported by the stock java.io. Also, he's got some kind of wrapper for reading from the tivo file which does buffering... which may or may not even be necessary for my needs (and which may be supplied by BufferedInputStream), etc.

The tricky part is he's got some C structs which *might* exactly match headers in the file. So, he does a read() and then just sets a pointer to a certain place in the read buffer, casts it as that struct, and then starts reading values. And Java don't play that way. So, that'll take some careful looking over.
I think you can do this(ish) with filechannel, bytebuffer, some of the nio and nio2 classes. I'd love to help if I had time.. Plenty of experience with those types of operations though so feel free to PM with questions or to bounce ideas! I may take a look at the c source.
herbman is offline   Reply With Quote
Old 02-20-2013, 04:25 PM   #15
tluxon
Registered User
 
Join Date: Feb 2002
Location: Kirkland, WA
Posts: 202
Quote:
Originally Posted by jemenake View Post
...Specifically, I'd like to make a streaming converter, which grabs a program off of my TiVo, decodes it, and pumps it out to my iPad without (like KMTTG has to do) saving the entire TiVo file to disk and then converting it...
If I understand you correctly, you pretty much want to be able to stream video from your TiVo to your iPad.

This is one of the reasons I started using a bundle of HDHomeRun network tuners along with Windows Media Center on my PC. Via the Remote Potato Server, I can watch everything that's on my WMC (all recorded shows and anything in my video library) on my Nexus 7 with the Remote Media Center app. With the paid version I can watch it live and with the free version it is only delayed by a minute or so. All the transcoding is done by the CPU of the PC hosting Remote Potato Server and the quality is easily good enough to readily see the difference between SD and HD. The same app also provides for browsing the guide listings, setting up recordings (they start recording almost instantly! - no phoning home to TiVo headquarters), and deleting shows - all from my tablet over wi-fi.

What's really cool, is that if there's something I find out is on from a remote location, I can set it to record in WMC and be watching it as it's recording from my tablet in about a minute.

Now if that could somehow be done with our TiVo's it would be jaw-dropping.
tluxon is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 03:51 PM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |