Proof of concept

[greg_burns]

I've been thinking of doing this for quite awhile and finally gave it a shot.

I just "removed" the .tivo header portion of a Family Guy episode I transferred using a hex editor and saved it as a file named FG.dat

I then edited the original .tivo file in VideoRedo to remove commercials, saved that away as FG.mpg

Opened up a DOS prompt and typed copy /b FG.dat+FG.mpg FG_edit.tivo

Just transferred FG_edit.tivo back to my TiVo and ... voila ... I have a Family Guy episode with no commercials but all its information intact (and in a folder!). :up:

Now to automate this... and figure out how to alter the xml data inside the .tivo portion of the file. :cool:

Edit: We'll I've taken a stab at automating the removal of the header...

http://home.comcast.net/~greg_burns/tivoattach/screenshot.png

TiVo Attach (http://home.comcast.net/~greg_burns/tivoattach/)

[davezatz]

Sweet!

[dmets]

So could this process also be used to add Tivo metadata to video files that did not originate from a tivo. Like Family Videos - So that they could have titles and be placed in folders when transfered to a Tivo via GoBack...

[greg_burns]

So could this process also be used to add Tivo metadata to video files that did not originate from a tivo. Like Family Videos - So that they could have titles and be placed in folders when transfered to a Tivo via GoBack...

First I gotta figure out how the information is stored in the header. It is not in clear text, but it may simple be Base64 or something. I see a hash and salt combination. I assume that is just hashing the MAK and not all the metadata, but I am not sure.

That would be sweet! though. :)

[HDTiVo]

Congratulations, you are on the way to providing an enormous benefit to the TiVo World.

[gonzotek]

Good work, and good luck on the next steps!!!

[davezatz]

First I gotta figure out how the information is stored in the header. It is not in clear text, but it may simple be Base64 or something. I see a hash and salt combination. I assume that is just hashing the MAK and not all the metadata, but I am not sure.

It would be nice if TiVo could point us (YOU) in the right direction for creating custom headers until they create some sort of TiVo Desktop add-on widget to do it. I do remember early on there was a Wiki entry somewhere trying to decode that stuff, but I don't think they got very far. (Their goal was different anyhow.)

It'd be nice to add Dave's Home Video or something and end up with a folder of all my home videos. Not that I have any home videos, but maybe I'd think about getting a camera. :)

[samkuhn]

So if you copy the header back on without changing the file extension to .tivo does the Tivo still recognize the metadata? Is the editing file that has been re-merged playable on a personal computer that doesn't know the MAK?

I sent this thread to the author of VideoRedo asking if he would add the ability to retain the metadata after editing as described in this thread.

[videoredo]

This is Dan from VideoReDo. Since the text isn't in clear-text if someone can describe what it looks like and where its located we can look at automating this. I can be reached at the videoredo support email (The board would not let me post our email address).

[greg_burns]

So if you copy the header back on without changing the file extension to .tivo does the Tivo still recognize the metadata?

No. You can try this yourself by just renaming an original .tivo file to .mpg. Your Tivo will no longer show the metadata for that file, just its filename. The Tivo box must decide whether to even look for metadata based on file extension. If you do rename .tivo file to .mpg it won't even transfer.

Is the editing file that has been re-merged playable on a personal computer that doesn't know the MAK?

Haven't tried that. Can't image that it would since the part of the file that makes it a .tivo file has been reattached.

[greg_burns]

This is Dan from VideoReDo. Since the text isn't in clear-text if someone can describe what it looks like and where its located we can look at automating this. I can be reached at the videoredo support email (The board would not let me post our email address).

Well, all I am doing is opening any Tivo file up using a Hex editor. (I am using the one built-into VirtualDub, its on the Tools menu).

You quickly see a disclaimer at the start of the file in clear text\xml format.

From 0x480 to around 0x4000 is data. I assume this must be where the metadata is stored. The regular mpeg file seems to be after this point.

[jmemmott]

From 0x480 to around 0x4000 is data. I assume this must be where the metadata is stored. The regular mpeg file seems to be after this point.

I spent some time with the file format when I was learning to extract the Closed Captioning information. Adding what has been posted here to what I already know, I would expect that reusing the existing header is workable but creating something custom will be much more difficult. The file is really two parts. The first part is a header created by Tivo with a few notable objects:

Byte 0 : Fingerprint – “TiVo”
Byte 9, length 4 or Byte 11, length 2 : offset of the MPEG2 Program Stream.
Byte 28 : Start of the xml wrapper containing the metadata.
a. Copy Right Notice is in clear text.
b. The metadata follows and is encrypted with a cipher I believe is similar to Blowfish.

The offset value points to a standard (ISO/IEC 13818) MPEG2 Program Stream. Some parts of the program stream are allowed to be encrypted in 13818 and TiVo uses this. In particular, all P and B pictures as well as the audio blocks are encrypted. Everything before the MPEG2 Program Stream should be copied as metadata.

[AV_Novice]

b. The metadata follows and is encrypted with a cipher I believe is similar to Blowfish.


A quick Google of Blowfish cipher took me to a web site that claims to have a list of 150 products that use Blowfish. Tivo, Inc. is listed.

[jmemmott]

A quick Google of Blowfish cipher took me to a web site that claims to have a list of 150 products that use Blowfish. Tivo, Inc. is listed.

Blowfish is conjecture on my part but it does have characteristics that fit such as processor/memory requirements, speed, security, license requirements, etc. However, there is also evidence that they have looked at other related algorithms such as Turing. In any case, it is a modern block or stream cipher, so even if Tivo explicitly told us which one they are using, we would still not be likely to break it. The security is in the key and key expansion algorithms not the cipher and no one except Tivo knows what those are...

[greg_burns]

I noticed there is a thread on the other site (which shall remain nameless and unlinked :rolleyes: ) that is trying to determine the makeup of .tivo file as well.

"tivo file format (split from: Decrypting TiVo ToGo)"

I'm afraid I am in WAY over my head. :(

[HDTiVo]

I noticed there is a thread on the other site (which shall remain nameless and unlinked :rolleyes: ) that is trying to determine the makeup of .tivo file as well.

"tivo file format (split from: Decrypting TiVo ToGo)"

I'm afraid I am in WAY over my head. :(
What you have done so far is very valuable. It would be nice if, for example, VideoRedo just added the option to preserve the data and output to a .tivo extension.

[greg_burns]

What you have done so far is very valuable. It would be nice if, for example, VideoRedo just added the option to preserve the data and output to a .tivo extension.

Dan's (from VideoRedo) response to my email to him...

OK, Seems simple enough. Will see what we can do.

.. Dan


Let's hope he's right. :)

[endicot]

This is awesome!

Is is possible to do the reverse? That is, make up a .tivo from a homemade .mpg - adding the tivo meta data (episode information) to the mix. Is there a utility to do this easy (assuming you have your media access key)?

[greg_burns]

This is awesome!

Is is possible to do the reverse? That is, make up a .tivo from a homemade .mpg - adding the tivo meta data (episode information) to the mix. Is there a utility to do this easy (assuming you have your media access key)?

There is currently no such utility. Unless Tivo makes the software themselves to do this, the chance of one of us figuring it out looks slim. There appears to be somesort of encryption (Blowfish?) on that portion of the file.

[megazone]

TiVo has used the El Gamal and Turing algorithms, but I don't know exactly which algorithm is used on TTG.

[greg_burns]

TiVo has used the El Gamal and Turing algorithms, but I don't know exactly which algorithm is used on TTG.

OUt of curiosity, any idea what this (http://www.schneier.com/blowfish-products.html) is referring to?

[megazone]

TiVo® DVRs by TiVo Inc.
Digital Video Recorders. Use Blowfish to safely transmit data to and from the hard drive.
---
The files are encrypted locally on the drive, aren't they? Perhaps that's what this is referring to? Or maybe MRV, since I thought that used a pre-shared key/cert and a symmetric system...

[davezatz]

Here's some speculation... The decryption key is buried somewhere within that .dll (and possibly loaded into memory when called). Depending what type of encryption scheme is used, the same key may or may not be used for encryption. In theory one would need to decrypt the .tivo file to understand the syntax of the meta data, then create properly formatted unique meta data, encrypt it, and prepend it to a mpeg. This is mostly beyond what my little brain can comprehend. :)

[HDTiVo]

It sounds like the metadata is encrypted with the MAK - which is unique to each account. That means my metadata would not work - or might be scrambled - with your mpegs. However, if the MAK could be found in the metadata, it is possible replacing it with your own would work.

If it is not the case that the metadata itself is unique to the account, then a "metadata sharing" library could be created to offer known metadata to people that need it. You could create a library of your own metadata and use it as needed. Not that this is going to be very useful.

It might be that there is a universal key, or no encryption key code (like a MAK of all zeroes.) If this could be found then it would be possible to make from scratch metadata with the universal key and "clear text" that any account could use.

[davezatz]

It sounds like the metadata is encrypted with the MAK - which is unique to each account. That means my metadata would not work - or might be scrambled - with your mpegs. However, if the MAK could be found in the metadata, it is possible replacing it with your own would work.

Could be... Though I was assuming the MAK is used to authentic or verify the owner prior to decrypting with a seperate key. It's also possible a decryption key is partially composed of the MAK and partially composed of some universal string. Regardless, it's beyond my skills to analyze.

It might be that there is a universal key, or no encryption key code (like a MAK of all zeroes.) If this could be found then it would be possible to make from scratch metadata with the universal key and "clear text" that any account could use.
This is probably where need TiVo's assistance and blessing.

[greg_burns]

I built a little app to automate this process of removing the header information.

TiVo Attach (http://home.comcast.net/~greg_burns/tivoattach/)

Requires .NET 2.0 Framework (http://www.microsoft.com/downloads/details.aspx?FamilyID=0856eacb-4362-4b0d-8edd-aab15c5e04f5&displaylang=en) to be installed.

Basically, it allows you to remove the header from a .tivo file. You can then edit the .tivo file using VideoRedo (or whatever), reattach the header file back onto the edited .mpg file. Thus preserving the metadata for transferring back to the TiVo.

[lafos]

Greg, I tried your app and it works just great. I downloaded an episode of Lost, cut the ads with VideoRedoplus, then put the metafile back on. It would be great if VideoRedo could automate, since the file now has to be written teice, once after editing, and again in your utility.

Very nice.

[Grimm1]

I built a little app to automate this process of removing the header information.

TiVo Attach (http://home.comcast.net/~greg_burns/tivoattach/)

Requires .NET 2.0 Framework (http://www.microsoft.com/downloads/details.aspx?FamilyID=0856eacb-4362-4b0d-8edd-aab15c5e04f5&displaylang=en) to be installed.

Basically, it allows you to remove the header from a .tivo file. You can then edit the .tivo file using VideoRedo (or whatever), reattach the header file back onto the edited .mpg file. Thus preserving the metadata for transferring back to the TiVo.Good job Greg! :up:
I've used it on a few shows and movies and so far it has been working great!

[greg_burns]

Another use for the little app; disguising... ahem... shows on your Tivo! ;) :D

Just attach some boring .tivo headers to your more interesting mpegs and there ya go!

BTW, I posted a slightly streamlined version earlier tonight. Less button clicking. :)

[merrickw]

All we need now is a little editor for those .tivo header files ...

[davezatz]

All we need now is a little editor for those .tivo header files ...

As long as it's buried in the encryption it's not likely... even if someone broke the encryption for something harmless like editing meta data, they wouldn't want to publicize that.

I wonder if the way .tivo files are constructed will change with the next release of TiVo Desktop (and box) software... they did mention something about watermarking. Though I guess that could be added during the desktop conversion phase, rather than changing how the TiVo unit handles files.

[greg_burns]

Makes you wonder why the metadata portion is encrypted along with the "fingerprint" stuff. What is their reasoning?

You would have thought Tivo would have been aware of all the cool posibilities this would have presented for 3rd parties to create. Think of all the programs out there that work with mp3's id3 tags. Same sort of thing. :(

[MickeS]

So does this mean that if I have a .tivo file taken straight from the TiVO, and I run Direct Dump on it and create an .mpg, and then run TiVo Attach, I will have a resulting .tivo file without DRM?

[greg_burns]

So does this mean that if I have a .tivo file taken straight from the TiVO, and I run Direct Dump on it and create an .mpg, and then run TiVo Attach, I will have a resulting .tivo file without DRM?

Afraid not. The DRM portion is in the header (along with the metadata). This program just reattaches the header back onto the original freed (or perhaps a different) mpg file.

You can actually attach any .tivo file's metadata to any mpg you have lying around. Not sure why you would want to do this unless you wanted to disguise a video as something else on your Tivo box. :rolleyes:

[MickeS]

Gotcha! Thanks.

[Dan203]

This is an awesome find! :up:

I'm going to have to run some tests when I get home. :)

Dan

[greg_burns]

Don't get too excited. :) It doesn't do the holy grail of allowing modifications to the metadata. :(

[Dan203]

I don't care about modification. What I want to test is whether I can "free" a TiVo file, reattach the metadata, then transfer it to a TiVo on a different account. If that works then it'll save me a whole lot of time when one of my friends or family members misses a program because I'll no longer have to go through the hassle of creating a DVD. Instead I can just transfer the program to an external harddrive, take it over their house and have them transfer it form the drive to their TiVo.

I have some confidence that this will work because at one time I had a different MAK and while I can't actually play files which use that alternate MAK they do display their data properly in TiVo Desktop. Which suggests to me that the encryption used for the metadata is independent of the MAK and uses some sort of universal key that's hard coded into the TiVo software.

Dan

[greg_burns]

What I want to test is whether I can "free" a TiVo file, reattach the metadata, then transfer it to a TiVo on a different account.

Hmm. My gut instinct is to say this won't work. The file is only "free" until you reattach the header again. Unless running it through DSD or VideoRedo is doing something more than I am aware of. :confused:

[greg_burns]

If that works then it'll save me a whole lot of time when one of my friends or family members misses a program because I'll no longer have to go through the hassle of creating a DVD. Instead I can just transfer the program to an external harddrive, take it over their house and have them transfer it form the drive to their TiVo.

Other than getting the folder support/program details info back, what advantage does this have over just taking the freed mpg file and transferring it back onto their Tivo? Doesn't a freed .tivo file already solve this problem for you? :confused:

[Dan203]

Hmm. My gut instinct is to say this won't work. The file is only "free" until you reattach the header again. Unless running it through DSD or VideoRedo is doing something more than I am aware of. :confused:

Running it through DSD or VideoReDo does a LOT more then just remove the header. Both of those programs exploit TiVo's own DirectShow filter to actually decrypt the MPEG data. You see the way it works is simple. TiVo encrypts the MPEG data at record time using a key built into the TiVo which is essentially your MAK. In order to make it so these files could be played back in Windows Media Player TiVo wrote a DirectShow source filter which decrypts the MPEG data then feeds that unencrypted data into the DirectShow filter graph. DSD and VideoReDo are able to exploit this by inserting a filter which captures that unencrypted data after it exits the TiVo filter, which inturn results in a completely unencrypted MPEG file. When you add back the header using your method you're not reencrypting the MPEG data, you're simply adding back the metadata in an already encrypted format. Which means, provided I'm correct and this metadata is not encrypted using the MAK, the resulting file should be capable of being transferred to any TiVo regardless of the MAK.

Dan

Disclaimer: Just because the MPEG data is no longer encrypted it does NOT mean that the file can not be traced back to you. So if you think this is the perfect way for you to start uploading .tivo files to BitTorrent you better be prepared for a call from TiVo's lawyers. :)

[Dan203]

Other than getting the folder support/program details info back, what advantage does this have over just taking the freed mpg file and transferring it back onto their Tivo? Doesn't a freed .tivo file already solve this problem for you? :confused:

I guess. But with an MPEG file the only identifying information you get on the TiVo itself is the file name, which seems kind of lame. With this the show will look no different then something you actually recorded on the TiVo.

Dan

[greg_burns]

Which means, provided I'm correct and this metadata is not encrypted using the MAK, the resulting file should be capable of being transferred to any TiVo regardless of the MAK.

Thanks for explanation. Let us know how it goes.

[jmemmott]

Which means, provided I'm correct and this metadata is not encrypted using the MAK, the resulting file should be capable of being transferred to any TiVo regardless of the MAK.

My own assessment/experience is that it may be the reverse. The MAK is too small to provide the security TiVo desires and seems to have been added after the fact to allow multiple TiVo's on the same account to decode video for T2G. To do this, it is more likely that the program stream decryption key along with other information such as the serial number of the source TiVo is part of the encrypted metadata and decrypted is with the MAK. This would allow each TiVos to use built in hardware encryption on the program stream (with it own unique ROM key) and still support T2G. It also likely explains why the metadata was encrypted in the first place. TiVo did not anticipate the need to change the rest of the information when the path was only from the TiVo to the PC and needed to protect this information.

Obviously, this is best tested by taking the headers across accounts to see if they are readable but a few headers I have have accessed while trouble shooting my closed captioning software do not seem to be readable acrossed accounts...

[HDTiVo]

While you guys are doing this...take a moment and see what happens if you run a re-attached .tivo back through DSD or VideoRedo. Same account and other account. Thanks.

[Dan203]

I played with this a little more and it does appear that the metadata is encrypted with the MAK, so it looks like jmemmott was right. :(

Oh well. Still a cool find.

Dan

[greg_burns]

While you guys are doing this...take a moment and see what happens if you run a re-attached .tivo back through DSD or VideoRedo. Same account and other account. Thanks.

I just ran a re-attached .tivo back through DSD. Then I re-re-attached the output. WinDiff says the two .tivo files are identical.

Is that what you wanted tested?

[greg_burns]

I just ran a re-attached .tivo back through DSD. Then I re-re-attached the output. WinDiff says the two .tivo files are identical.

Is that what you wanted tested?

Wanted to point out the test above was starting with a re-attached freed .tivo file.


I just tried it with an original .tivo file and the results are the same!

Run original .tivo file through DSD and then reattach header. Comparing original .tivo to newly reattached .tivo and you find they are identical. (I need to do more testing to verify this, but it sure appears that way).

I do agree with what Dan is saying about DSD/VR doing a LOT more than just removing the header. I've looked at the code with Reflector (http://www.aisto.com/roeder/dotnet/). I am just not sure the end result is any different... :rolleyes:


Edit: scratch that, perhaps WinDiff isn't the best tool for this. Opening the two in a Hex editor clearly shows they are not identical. :confused:

I need to test this when I get more time.

Edit2: Used Fairdell HexCmp. Much easier to see the difference. And didn't crash my machine while doing it. :p

[HDTiVo]

That's what I want to know...you were freaking me out at the start, because your first results were implying the mpeg portion of the original .tivo was unencrypted. Keep at it and thanks. Your ultimate result may be interesting.

Let me elaborate on my curiosity.

A re-attached (RA) .tivo is substantially different from the original (O) in that the mpeg portion is supposedly unencrypted and possibly edited - or even entirely replaced. Presumably the meta data informs the TiVo DS filter what to do with the 'mpeg/DRM' part of the .tivo to make it look like a regular mpeg. So how is it that an RA even plays in any recognizable form or can be run through DSD again with any meaningful result?

Seems it should not be possible based on assumptions about how it all works. So some of our assumptions my be wrong.

[greg_burns]

So how is it that an RA even plays in any recognizable form or can be run through DSD again with any meaningful result?

Good questions.

Not sure if others have noticed; but when you play back an edited RA file on the Tivo the time bar at the bottom is off. It appears to be the length of the original recording, not the edited version. I guess the time length is included in the header as well.

Edit: of course, this doesn't explain how a regular mpeg transferred to a TiVo has the correct time bar. :confused:

[HDTiVo]

Is it possible that the MetaData is encrypted universally - with the only differences being account specific (ie. only a different MAK is buried in the MetaData?) If so, a "transplant" of the MAK could open MetaData transfer across accounts.

Suppose two account holders from identical cable systems recorded the same shows and compared the MetaData to see if there were only a few bytes that were different, and found some pattern in those bytes that led to making MetaData transplants possible?

Is it further possible that there is something (ie. the MAK) buried in the original .tivo encrypted-MPEG header portion that is the key to the mpeg part, and that the removal of that by the TiVo DS filter makes the mpeg part now universal in the "RA" .tivo?

Suppose a utility could be written that transplants MAKs in MetaData and/or mpeg headers? That would make sharing of .tivo files quite simple.

[greg_burns]

My original version of Tivo Attach ouputted a ".header" file (they were of course very small). I'll add back that feature so a test can be done on a common recording.

[Dan203]

I did some reading over on that other forum and they seem to think that the metadata is encrypted using a combination of a metadata key stored in the registry and your MAK. I also looked at both the TiVo DirectShow filter and TiVo Desktop in a HEX editor and found several references in each to Blowfish so I think that is in fact the encryption scheme used. Does anyone know of a Blowfish decryption program that we might be able to try feeding these two keys into to see if it ends up decrypting the metadata?

Dan

[HDTiVo]

My original version of Tivo Attach ouputted a ".header" file (they were of course very small). I'll add back that feature so a test can be done on a common recording.
Didn't your earlier version have a detach function that output a metadata file also. Could use .header and .metadata files.

Thanks.

P.S. also need detach to work with mpeg so can compare mpeg.header to tivo.header

[greg_burns]

Didn't your earlier version have a detach function that output a metadata file also. Could use .header and .metadata files.

No, the program used to remove what I call the "header" (i.e. the portion of the file before the actual mpeg content, which also contains the encrypted metadata info) and save it as a separate file; *.header. Then you could attach it back onto a regular .mpg file. The program currently just does this all in memory, since it seemed pointless to clutter ones system with these little .header files.

P.S. also need detach to work with mpeg so can compare mpeg.header to tivo.header

Not following you here. A regular mpeg doesn't have a "header" file (using my above definition of it). :confused:

[HDTiVo]

No, the program used to remove what I call the "header" (i.e. the portion of the file before the actual mpeg content, which also contains the encrypted metadata info) and save it as a separate file; *.header. Then you could attach it back onto a regular .mpg file. The program currently just does this all in memory, since it seemed pointless to clutter ones system with these little .header files.


Ok, I thought there were 3 parts: metadata; mpeg header info;mpeg video data

Anyway you are right about not needing it in the "production" version; but for our "academic" exercise...

[MickeS]

of course, this doesn't explain how a regular mpeg transferred to a TiVo has the correct time bar.

It probably first checks if a header exists. If it does, takes the time from there, and if it doesn't just gets the time information from the mpeg itself.

I wish TiVo would release a simple "show info" editing tool and add to TiVo Desktop. Makes sense to me, after they enabled GoBack.

[jmemmott]

A re-attached (RA) .tivo is substantially different from the original (O) in that the mpeg portion is supposedly unencrypted and possibly edited - or even entirely replaced. Presumably the meta data informs the TiVo DS filter what to do with the 'mpeg/DRM' part of the .tivo to make it look like a regular mpeg. So how is it that an RA even plays in any recognizable form or can be run through DSD again with any meaningful result?

Seems it should not be possible based on assumptions about how it all works. So some of our assumptions my be wrong.

The file as a whole is not encrypted. Within the MPEG2 Program Stream only the P and B pictures as well as the audio blocks are encrypted. There are a lot of other headers and decoding information within the stream that are in the clear. Ignoring the audio for simplification, the stream contains picture frames at about the rate of 30 fps. Periodically these are "I" pictures that contain the entire image. Mostly they are P and B pictures which contain only differential information used to construct new frames from what you already know. This is the basis of MPEG2 compression. To decrypt the tivo video, you need to demux the stream, decrypt the P and B pictures as they go through and then put the decrypted copies back in the stream where they came from. Then you end up with an mpeg2 stream that could be played by any mpeg2 compliant player ("freed").

This would appear to be what the TiVo DirectShow filter does. Since the stream is mpeg2 compliant from the start, all encrypted blocks are marked by a bit flag in the picture headers. The allows the filter to separate the unencrypted I pictures from the encrypted B and P pictures and only decrypt where it is required. If you ran a freed mpeg2 file with a tivo header back through the DirectShow filter, none of the pictures should be marked as encrypted any more and the filter should just copy the input stream to the output ==> You get back what you put in...

[HDTiVo]

Since the stream is mpeg2 compliant from the start, all encrypted blocks are marked by a bit flag in the picture headers. The allows the filter to separate the unencrypted I pictures from the encrypted B and P pictures and only decrypt where it is required.
Not that this is a huge issue because of DSD, but maybe the mpeg encryption is not MAK specific and that part of the process (DSD) can be eliminated and the .tivo can be made usable on another account with just a MAK transplant in the metadata?

[Dan203]

As someone else already said the most likely scenario is that the MPEG data is encrypted with the key taken from the crypto chip inside your TiVo. That key is then embedded in the metadata portion of the .tivo file, then encrypted again using your MAK. The TiVo filter then uses your MAK to decrypt the metdata, which inturn exposes the key from the crypto chip, which is then uses to decrypt the video.

If this is the case it should be easily tested by simply taking a .tivo file from one physical TiVo box and transplanting the header from a .tivo file from another physical TiVo. If the theory is correct then the file will no longer play because the key in the header will no longer match the key used to encrypt the video stream.

Dan

[davezatz]

As someone else already said the most likely scenario is that the MPEG data is encrypted with the key taken from the crypto chip inside your TiVo.

Hmmm I hadn't considered a specialized chip... especially since a person's MAK is the same across all TiVo units.

So has anyone actually tried transplanting header info and playing a .tivo on someone else's unit? That'd be very interesting. Assuming I'm following this conversation... :) I haven't tried it (and one of my TiVo's in on loan, and the other on the dining room floor waiting to be packaged) but could I download a .tivo file from unit 1 on my accont and then upload it to unit 2 on my account for playback?

OK, I reread some of Dan's thoughts... seems like he thinks there are two keys? A hardware encryption key and then the MAK. Hm.

[HDTiVo]

Doesn't make sense that encryption gets more specific than the MAK because .tivos play on all TiVoes under the same account. The "crypto chip" is really just a serial number, not something that does encryption work.

Transplanted metadata info won't play under another MAK. You've got to find the MAK within the metadata, replace ("transplant") it with the target account's MAK, and pray you can see everything. That's what needs to be tried.

[davezatz]

Doesn't make sense that encryption gets more specific than the MAK because .tivos play on all TiVoes under the same account. The "crypto chip" is really just a serial number, not something that does encryption work.

Transplanted metadata info won't play under another MAK. You've got to find the MAK within the metadata, replace ("transplant") it with the target account's MAK, and pray you can see everything. That's what needs to be tried.

What if you and I download the exact same show and swap headers. Essentially, just the MAK is different right? Not that the MAK part can neccessarily be read or is visible, but we should be able to play the shows on the other person's unit?

[AllAboutJeeps]

Wow. This is cool stuff.

Here is a thought. Anyone tried swapping 'header' data with someone else (on a different MAK) reattaching it, and seeing how TiVo handles it? For example, I give Mr.B my 'header' and he reattaches it to one of his files file. He then checks to see if his TiVo will read my 'header' properly.

Or, we could get two people with the same show, Lost seems popular right now, exchange their 'header' data and compare them. If the 'header' from both files is the same (which I doubt it is), it would be safe to assume the 'header' encryption is static, not MAK based.

...danny

[greg_burns]

Suppose two account holders from identical cable systems recorded the same shows and compared the MetaData to see if there were only a few bytes that were different, and found some pattern in those bytes that led to making MetaData transplants possible?

I just posted a new build of Tivo Attach. You can now choose Export Header from the File menu. This creates a *.header file that we can use to compare to others as suggested by HDTiVo.

Anyone tried swapping 'header' data with someone else (on a different MAK) reattaching it, and seeing how TiVo handles it? For example, I give Mr.B my 'header' and he reattaches it to one of his files file. He then checks to see if his TiVo will read my 'header' properly.

To do what you suggest, I will have to add back the ability to browse for the *.header you want to use during the attach. :rolleyes: :p

[greg_burns]

To do what you suggest, I will have to add back the ability to browse for the *.header you want to use during the attach. :rolleyes: :p

I just added the ability to browse for a *.header file back to the program.

[greg_burns]

Or, we could get two people with the same show, Lost seems popular right now, exchange their 'header' data and compare them. If the 'header' from both files is the same (which I doubt it is), it would be safe to assume the 'header' encryption is static, not MAK based.

If anyone wants to try this out; PM me. :cool:

I don't think Lost will be a good show for what HDTivo wanted to test (isolating the MAK in the header). Problem is that it is on a local network channel. Some of the channel info will be different for sure.

I think we would need to pick a show that is on a common channel # with common call letters.

I have dish network, which may make it easier to find a common header file.

[HDTiVo]

Wow. This is cool stuff.

Here is a thought. Anyone tried swapping 'header' data with someone else (on a different MAK) reattaching it, and seeing how TiVo handles it? For example, I give Mr.B my 'header' and he reattaches it to one of his files file. He then checks to see if his TiVo will read my 'header' properly.

Or, we could get two people with the same show, Lost seems popular right now, exchange their 'header' data and compare them. If the 'header' from both files is the same (which I doubt it is), it would be safe to assume the 'header' encryption is static, not MAK based.

...danny
Glad to see all the enthusiasm. This is what we are discussing over the last days.

I mentioned specifically earlier two people on the SAME cable system recording the same show to compare headers (I mean metadata, I mean headers, no I mean metadata...)

I like metadata because it does not confuse me regarding the mpeg header data.

[jmemmott]

What if you and I download the exact same show and swap headers. Essentially, just the MAK is different right? Not that the MAK part can neccessarily be read or is visible, but we should be able to play the shows on the other person's unit?

For what it is worth, I tried a localized variation of your proposal. I took two of my own files, different programs but recorded on the same machine = a.tivo and b.tivo. I split them into header and program stream = a.header, a.tivops, b.header and b.tivops and then swapped them as you suggested = a.header+b.tivops and b.header+a.tivops. Neither resulting file will play back although the desktop does show the metadata associated with the header portion correctly.

To me this implies that not only is the key for the program stream in the metadata, it appears to be a session key ==> each recording has a different key for the program stream. This would mean that a header can only go back on the same encrypted program stream or on a mpeg that has been "freed".

I will be intrested to see if someone else finds the same results (or if I messed something up in the process).

[HDTiVo]

What if you and I download the exact same show and swap headers. Essentially, just the MAK is different right? Not that the MAK part can neccessarily be read or is visible, but we should be able to play the shows on the other person's unit?
That should not work. It is the same as swapping .tivo files. We know now that both the DRM-MPEG and the (post DSD) MPEG play without regard to your own metadata. But neither an O nor an RA will play with your metadata on someone elses PC or TiVo; The TiVo DSF will see the wrong MAK in the header and refuse to function making both DSD and WMP useless with those .tivoes.

There is no solution yet if you have someone else's .tivo (O or RA) even if Greg were to write a function that spit out the plain DRM-MPEG without the metadata. Neither the Tivo DSF or anything else could decode that either.

The only possibility is if the metadata is not encoded differently with each MAK. Then the challenge is to find where the MAK is and replace it with the target's MAK (a MAK Transplant.) I mean here the encrypted MAK; we are not decrypting the MAK, only replaceing one encrypted MAK with another encrypted MAK; then we hope that fools the TiVo DSF into working.

One approach to finding the MAK is comparing metadata from the same show recorded on the same cable system with tivoes from two different accounts (MAKs.) If the metadata is the same except for a few bytes, then that's where the MAK probably is. Then look at those bytes in other show's metadata and see if they are always the same. That's a start anyway.

More...

[HDTiVo]

Here is a "matrix" of possibilities:

1. MetaData AND MPEG video encoded with MAK:
- Solution: DSD to MPEG; no way to save MetaData without breaking encryption which is far beyond what we are contemplating here.

2. MetaData w/ MAK, MPEG Video w/o MAK:
- Solution: DSD to MPEG; again no way to use MetaData as in #1.

3. MetaData w/o MAK, MPEG Video w/MAK:
- Solution: DSD .tivo to .mpg and ReAttach MetaData with Greg's program. Do a MAK transplant with update of Greg's program.

4. MetaData and MPEG Video w/o MAK:
- Solution: Do a MAK Transplant with Greg's updated program.



If Case 1 or 2 exist, the next try is to find a universal MAK (ie. a MAK of all zeroes) that would allow creation of plain text metadata.



Please no one be annoyed that I am writing so much theory and not doing the grunt work. I really am not in a position to do the grunt work properly.

[greg_burns]

I will be intrested to see if someone else finds the same results (or if I messed something up in the process).

We'll I just tried swapping headers on two .tivo files (not freed ones, still encrypted originals).

When I tried to play back one of the new frankenstein files it was very not very pretty. Looked a lot like that screechy audio/garbled video problem with Nero. Had to end task WMP to make it stop.

[jmemmott]

One approach to finding the MAK is comparing metadata from the same show recorded on the same cable system with tivoes from two different accounts (MAKs.) If the metadata is the same except for a few bytes, then that's where the MAK probably is. Then look at those bytes in other show's metadata and see if they are always the same. That's a start anyway.


If they are using a modern block cypher such as Blowfish, then the effects are not localized. That is one of the tests for a strong cypher. A one bit change in either the encryption key or the clear text is "randomly" scattered across the entire encrypted block so there isn't a "localized" difference to attack.

[HDTiVo]

We'll I just tried swapping headers on two .tivo files (not freed ones, still encrypted originals).

When I tried to play back one of the new frankenstein files it was very not very pretty. Looked a lot like that screechy audio/garbled video problem with Nero. Had to end task WMP to make it stop.
And for fun, how 'bout a swapped RA .tivo?

[greg_burns]

And for fun, how 'bout a swapped RA .tivo?

We'll I know that works already. But the RA files don't have the encrypted mpegs in them anymore. You can attach any header you like to any freed mpg.

[HDTiVo]

The file as a whole is not encrypted. Within the MPEG2 Program Stream only the P and B pictures as well as the audio blocks are encrypted. There are a lot of other headers and decoding information within the stream that are in the clear. Ignoring the audio for simplification, the stream contains picture frames at about the rate of 30 fps. Periodically these are "I" pictures that contain the entire image. Mostly they are P and B pictures which contain only differential information used to construct new frames from what you already know. This is the basis of MPEG2 compression. To decrypt the tivo video, you need to demux the stream, decrypt the P and B pictures as they go through and then put the decrypted copies back in the stream where they came from. Then you end up with an mpeg2 stream that could be played by any mpeg2 compliant player ("freed").

This would appear to be what the TiVo DirectShow filter does. Since the stream is mpeg2 compliant from the start, all encrypted blocks are marked by a bit flag in the picture headers. The allows the filter to separate the unencrypted I pictures from the encrypted B and P pictures and only decrypt where it is required. If you ran a freed mpeg2 file with a tivo header back through the DirectShow filter, none of the pictures should be marked as encrypted any more and the filter should just copy the input stream to the output ==> You get back what you put in...

Before VideoRedo supported .tivo, I think when I opened a .tivo with it that the audio played OK, and the top 10% of the video was OK but the rest of the video was covered by what looked like flags from various nations. I wish I could go back and see that again.

I can think of a very simple way TiVo could break Greg's ReAttach (within same MAK) if your description is correct. My personal rules for this thread prevent me from discussing such a topic though. ;)

[HDTiVo]

We'll I know that works already. But the RA files don't have the encrypted mpegs in them anymore. You can attach any header you like to any freed mpg.
Oh, right, its getting hard to remember everything. That is what I expected based on jmemmott's description of the process.

[Dan203]

OK I just discovered something very interesting! Like jmemmott I tried copying the header from one file onto the encrypted data from another file, and it did not work. The video played like HDTiVo described above when describing how VideoReDo use to handle .tivo files. Anyway I started playing around trying to figure out what might get it to work and I found something. I discovered that if you copy the offset value (i.e. the 9th bit), the "salt" and the "creation date" from the original file over to the header in the other file the video will play back correctly! What this means is two things...

1) The metadata is encrypted using a static key that is completely independent of the file itself. Most likely the key stored in the registry and your MAK.

2) The key used for decrypting the video is NOT burried in the metadata. It's some combination of your MAK, the salt and the creation date. This is not really all that important, since decrypting the MPEG data is already easily done using the DirectShow filter, but it might be important to someone looking for a way to decrypt them using a Mac or Linux so I thought I'd mention it.

I'm no crypto expert, but knowing this information and that there are definite references to Blowfish in the TiVo Desktop .exe, shouldn't we be able to figure out how to decrypt the metadata and inturn figure out the format?

Dan

[Dan203]

OK I was looking at the metadata registry key and I noticed that it changes slightly even if you reenter the same MAK. So I think it might also be using some combination of the current date and time when encoding the key stored in the registry.

Dan

[HDTiVo]

I discovered that if you copy the offset value (i.e. the 9th bit), the "salt" and the "creation date" from the original file over to the header in the other file the video will play back correctly! What this means is two things...

1) The metadata is encrypted using a static key that is completely independent of the file itself. Most likely the key stored in the registry and your MAK.

2) The key used for decrypting the video is NOT burried in the metadata. It's some combination of your MAK, the salt and the creation date.

You've made some key discoveries here involving pieces that must be preserved properly.

However, in #1 you have not established the MAK is involved yet, which is a good thing for our purposes.

In #2, some of the key could be in the metadata. I don't see how you establish the conclusions that a) the MAK is involved and b) nothing is in the metadata. Remember, you are using headers generated with the same account (MAK)

[greg_burns]

One approach to finding the MAK is comparing metadata from the same show recorded on the same cable system with tivoes from two different accounts (MAKs.) If the metadata is the same except for a few bytes, then that's where the MAK probably is. Then look at those bytes in other show's metadata and see if they are always the same. That's a start anyway.

Kinda gave this a shot. Had two .header files from two accounts. Both same show recorded same bat channel (via dish). Unfortunately, the two headers were drastically different viewed in a hex editor. Not much to gleen from that. :(

Tried replacing the date-time, fingerprint, and salt values with the same from my own header. Then attached it to a freed mpeg. The Tivo obivously didn't know what to do with it (didn't display the metadata on the screen), but it still transferred.

[davezatz]

Kinda gave this a shot. Had two .header files from two accounts. Both same show recorded same bat channel (via dish). Unfortunately, the two headers were drastically different viewed in a hex editor. Not much to gleen from that. :(

Tried replacing the date-time, fingerprint, and salt values with the same from my own header. Then attached it to a freed mpeg. The Tivo obivously didn't know what to do with it (didn't display the metadata on the screen), but it still transferred.

That is a very interesting discovery. Sorry I don't have anything more to add than that. :) I was going to ask if it mattered what types of TiVo units were used to record the two shows, but since files should work across all machines in a household the question is probably irrelevent.

[CuriousMark]

That is a very interesting discovery. Sorry I don't have anything more to add than that. :) I was going to ask if it mattered what types of TiVo units were used to record the two shows, but since files should work across all machines in a household the question is probably irrelevent.
In a good cypher system, even a small change to plaintext will result in a completey different cypher text starting with the block in which the difference occurs. This is done by design to prevent a differential attack.

CM

[davezatz]

In a good cypher system, even a small change to plaintext will result in a completey different cypher text starting with the block in which the difference occurs. This is done by design to prevent a differential attack.

It was interesting that the foreign header could not be read. I would assume the encryption and/or decryption key (MAK?) is required on the TiVo unit itself. The MAK may not be in the header at all... just on the TiVo unit and maybe buried (and obfuscated) on the PC somewhere and referred to by the .dll when decrypting. That's why breaking open the .dll and/or seeing what it loads into memory might provide useful info in addition to our analysis of responses. However, I don't have the slightest idea (or motivation) how to do it.

I don't know a ton about encryption, but I love saying Diffie-Hellman quickly at high volumes. :)

[Dan203]

However, in #1 you have not established the MAK is involved yet, which is a good thing for our purposes.

Actually I have, and Greg's test confirms it. The way I have proven that the MAK is involved with decrypting the metadata is by simply changing the MAK in TiVo Desktop. When I do that TiVo Desktop can still see the .tivo files, but lists them only by their file name because it can no longer read their metadata. Changing it back to the proper MAK immediately allows it to start displaying the metadata again.

In #2, some of the key could be in the metadata. I don't see how you establish the conclusions that a) the MAK is involved and b) nothing is in the metadata. Remember, you are using headers generated with the same account (MAK)

All I was saying is that my previous theory, that there was a machine specific key burried in, and encrypted with, the metadata is false. I proved this by taking encrypted (non-freed) files from two different TiVos, swapping their header information, and then proving that all that needs to be done to get them to play again is to put in the proper offset, "salt" and "creation date" back into the file. Both of which are in the clear and easily swapped. If there was a machine specific key burried in the metadata then this would not have worked.

Dan

[HDTiVo]

In a good cypher system, even a small change to plaintext will result in a completey different cypher text starting with the block in which the difference occurs. This is done by design to prevent a differential attack.

CM
This is the nightmare which jmemmott pointed out earlier. It leads to having to figure out how to actually break the encryption.

[HDTiVo]

It was interesting that the foreign header could not be read. I would assume the encryption and/or decryption key (MAK?) is required on the TiVo unit itself. The MAK may not be in the header at all... just on the TiVo unit and maybe buried (and obfuscated) on the PC somewhere and referred to by the .dll when decrypting. That's why breaking open the .dll and/or seeing what it loads into memory might provide useful info in addition to our analysis of responses. However, I don't have the slightest idea (or motivation) how to do it.

I don't know a ton about encryption, but I love saying Diffie-Hellman quickly at high volumes. :)
Maybe there will prove to be a way to create a "dll Switcher" which can change your key as needed with each time you play a .tivo.

[HDTiVo]

I've got an idea for another approach, which will allow preservation and editing of metadata, viewing of the modified metadata on your PC, playing of the altered .tivo on your PC and probably on your TiVo - but the original metadata (or original metadata from another recording) would appear on the TiVo. It would also allow for a freed mpeg to have plain text metadata attached. The metadata could only be used on the PC.

Its based on 1. the ability to download recordings without using TiVoDesktop and 2. the ability to view the metadata of the recording in open text while it is still on the TiVo.

Before I write this up, I want to double check with you all that both those things are possible.

[samkuhn]

Its based on 1. the ability to download recordings without using TiVoDesktop and 2. the ability to view the metadata of the recording in open text while it is still on the TiVo.


Galleon does both of these functions, I believe. Certainly the second. I am not sure if these features are electronically plumbed through the Tivo software or not though.

[davezatz]

1. You can download shows via the little TiVo webpage without having the TiVo Desktop software installed. It is not possible to playback those files in anyway without the .dll bundled with the Desktop Software being installed to decrypt the file.

2. I'm not sure... I'm also not sure at what point in the process video and metadata are encrypted.

[MickeS]

My guess it that if it was possible to easily do this, it would be included in Galleon already. :)

I think TiVo will have to provide this functionality through API to make it possible, at least if you don't want to try and break the encryption.

[greg_burns]

Before I write this up, I want to double check with you all that both those things are possible.

Sounds like you are going to go down this path (http://www.tivocommunity.com/tivo-vb/showthread.php?p=3752738&&#post3752738).

Wait... you're in that thread. Where are you going with this? :p

[greg_burns]

2. I'm not sure... I'm also not sure at what point in the process video and metadata are encrypted.

We'll it's definately not happening in Firefox. :rolleyes: So my money is on the Tivo box. :p

[davezatz]

We'll it's definately not happening in Firefox. :rolleyes: So my money is on the Tivo box. :p

Doh! In case it really wasn't clear... I meant is the metadata always encrypted when attached to the file and is the file always encrypted in this manner (on the TiVo). Or do these things happen (on the TiVo) once a file is requested. I haven't read up on any of the real hacking tytools stuff, but I wonder if there are clues related to that knowledge.

Oh yeah, I can confirm it's not happening in Opera either. ;)

[jmemmott]

Doh! In case it really wasn't clear... I meant is the metadata always encrypted when attached to the file and is the file always encrypted in this manner (on the TiVo). Or do these things happen (on the TiVo) once a file is requested. I haven't read up on any of the real hacking tytools stuff, but I wonder if there are clues related to that knowledge.

Oh yeah, I can confirm it's not happening in Opera either. ;)

As an educated guess, I suspect that encryption of the program stream is added when it is recorded. There is no evidence that the program stream passes through the tivo cpu. Instead, it seems to take a hardware path from the tuner through a proprietary encoder/encrypter and straight on to the hard disk. Encryption was added with the advent of the Series 1 DirectTivo's and has been present since. Since it only takes a handful of chips - mostly xor gates and shift registers to add the type of encryption they are using to the hardware path and it has been available for so long it doesn't make sense for them not to leverage it.

There is no evidence that the header predates T2G or that it is used in any other function such as MRV. The likely place for this to be added is when the file is transferred off the Tivo through the T2G protocol. The file has to be remuxed from the internal .ty format used on the disk to a 13818-2 format for the outside world at that time. Adding the header to the front would be a trivial extension.

[Dan203]

The best solution to making the process Greg came up with simple, and useful, is to just let DanR include the functionality in VideoReDo. With that you will be able to open the .tivo file, edit out the commercials and get back a .tivo file which is capable of being transferred back to any TiVo on your account complete with the original data.

Other then that functionality, or posssibly spoofing data to hide something dirty from the wife, I see no real use for the current functionality.

The next major step in this process would to either a) learn how to decrypt the metadata so that files can be transferred complete with metadata to TiVos on an different account or b) simply figure out the metadata format so that we can create our own data in standard MPEG files. Personally I'd prefer the second option since it would protect TiVos encryption format and would keep us from having to run afoul of the DMCA. *

Dan

* I'm assuming that the metadata does not actually need to be encrypted, since we've already discovered that the MPEG data doesn't.

[greg_burns]

The best solution to making the process Greg came up with simple, and useful, is to just let DanR include the functionality in VideoReDo. With that you will be able to open the .tivo file, edit out the commercials and get back a .tivo file which is capable of being transferred back to any TiVo on your account complete with the original data.

ACK

[davezatz]

b) simply figure out the metadata format so that we can create our own data in standard MPEG files.

This is the reason I find the topic interesting... I want to be able to properly catalog my own video. I wonder if the TiVo software folks have considered or are working something like this into a software utility or TiVo Desktop. Then again, even if they are who knows when we might ever see it... (TTG has been out over a year now).

[EwanG]

My .02 worth, but I would agree with Dan that I am much less interested in being able to trade Tivo files (at which point I suppose they'd have little choice but to change the encryption or turn off the functionality anyway) than in being able to add my own metadata to MPG files I'm going to load onto the TiVo.

This would also be a BIG help for files where the TiVo gets it wrong. The personal example is that for some reason, in TiVo's database it has the Episode Number for Dora the Explorer's "A Letter for Swiper" and "Super Map" mixed. So whenever it's recorded on the TiVo, I have to remember to tell the daughter that Goose Juice is Moose Juice... sorry Dr. Seuss moment there... to tell her the other way around.

It would be nice to transfer the episodes to the PC, fix the metadata, and then transfer back with the right names and a "Never Delete" flag.

[davezatz]

ACK

Sorry Greg, you're obsolete now... you should have patented it. ;)

[HDTiVo]

Kinda gave this a shot. Had two .header files from two accounts. Both same show recorded same bat channel (via dish). Unfortunately, the two headers were drastically different viewed in a hex editor. Not much to gleen from that. :(

That's JM's nightmare scenario. Its not possible to find and transplant the encrypted MAK without decryption* and re-encryption. This takes the "project" to a huge new level, and one that maybe should not be worked on here.


Tried replacing the date-time, fingerprint, and salt values with the same from my own header. Then attached it to a freed mpeg. The Tivo obivously didn't know what to do with it (didn't display the metadata on the screen), but it still transferred.
That shows more than JUST those items that Dan found are needed.


* Let me not assume that data distribution within the header and encryption are one and the same.

[HDTiVo]

Actually I have, and Greg's test confirms it. The way I have proven that the MAK is involved with decrypting the metadata is by simply changing the MAK in TiVo Desktop. When I do that TiVo Desktop can still see the .tivo files, but lists them only by their file name because it can no longer read their metadata. Changing it back to the proper MAK immediately allows it to start displaying the metadata again.
Its become moot, but what you did was demo that TiVo Desktop won't show metadata if the .tivo creator's MAK is not the same as the current user's. What Greg did shows that the metadata is DISTRIBUTED over the header differently with another MAK, but still has not proven it is ENCRYPTED differently. The situation makes it too tough to crack, regardless.


All I was saying is that my previous theory, that there was a machine specific key burried in, and encrypted with, the metadata is false. I proved this by taking encrypted (non-freed) files from two different TiVos, swapping their header information, and then proving that all that needs to be done to get them to play again is to put in the proper offset, "salt" and "creation date" back into the file. Both of which are in the clear and easily swapped. If there was a machine specific key burried in the metadata then this would not have worked.

Agree.


Were you not one of the folks who has seen program description info ("free text metadata") for files still on the TiVo in their Web browser? That can be done, right?

[HDTiVo]

Sounds like you are going to go down this path (http://www.tivocommunity.com/tivo-vb/showthread.php?p=3752738&&#post3752738).

Wait... you're in that thread. Where are you going with this? :p

Ok, that reminded me of what I needed to know.

Now I need time to write up my idea, which is a bit different and will take someone lots of time to develop. And remember it will not accomplish the Holy Grail of using edited metadata on your (or even another's) Tivo, but will make it possible to see it on your or anyone else's PC.

1. You can send a freed MPEG with readable metadata (even editied) to another person's PC; the MPEG can be played on the other person's TiVo with obviously no metadata.

2. You can edit your metadata and see it on your PC; You can transfer your program to your TiVo, but the unedited metadata (original or from some other show) will be seen on your TiVo.


One more question: With TiVoToCome, can a .tivo or .mpeg be sent to the TiVo from the PC without using TiVoDesktop?

[petew]

This is the reason I find the topic interesting... I want to be able to properly catalog my own video. I wonder if the TiVo software folks have considered or are working something like this into a software utility or TiVo Desktop. Then again, even if they are who knows when we might ever see it... (TTG has been out over a year now).

This might be possible though galleon, Tivo uses HTTP requests to browse the remote server which returns the program data in XML format. With current versions of Galleon if the file name is in a very specific format Galleon parses it and returns the Episode Title, Channel ID and Recording Date. These do get saved when using GoBack. It would be a relatively simple step forward to add additional information to Galleon's reply.

Data returned by Galleon for a .TiVo file:

<Item>
<Details>
<Title>Malcolm in the Middle</Title>
<ContentType>video/x-tivo-mpeg</ContentType>
<SourceFormat>video/x-tivo-mpeg</SourceFormat>
<Duration>1799000</Duration>
<CaptureDate>0x43BE2380</CaptureDate>
<Description>When Hal and Lois receive a check for $10,000 made out to Malcolm, their first instinct is to keep it, until Dewey overhears their plan and wants a piece of the pie. </Description>
<EpisodeTitle>Malcolm&apos;s Money</EpisodeTitle>
<SourceChannel>2</SourceChannel>
<SourceStation>KTVU</SourceStation>
<SourceSize>546076963</SourceSize>
</Details>
<Links>
<Content>
<ContentType>video/x-tivo-mpeg</ContentType>
<Url>http://192.168.0.20:8081/TiVoConnect/GalleonExtra%2FArchive%2FMalcolm+in+the+Middle+-+Malcolms+Money+%28Recorded+Fri+Jan+6+2006+08+30PM+KTVU%29.T iVo</Url>
</Content>
<CustomIcon>
<ContentType>video/*</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>urn:tivo:image:save-until-i-delete-recording</Url>
</CustomIcon>
<TiVoVideoDetails>
<ContentType>text/xml</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>/TiVoConnect/TivoNowPlaying/GalleonExtra%2FArchive%2FMalcolm+in+the+Middle+-+Malcolms+Money+%28Recorded+Fri+Jan+6+2006+08+30PM+KTVU%29.T iVo?Format=text%2Fxml</Url>
</TiVoVideoDetails>
</Links>
</Item>


And for a .mpg file


<Item>
<Details>
<Title>24 Season 416</Title>
<ContentType>video/x-tivo-mpeg</ContentType>
<SourceFormat>video/x-tivo-mpeg</SourceFormat>
<Duration>0</Duration>
<CaptureDate>0x43C182E8</CaptureDate>
<Description></Description>
<EpisodeTitle>Day 4: 10:00 P.M.-11:00 P.M.</EpisodeTitle>
<SourceChannel>0</SourceChannel>
<SourceStation>0</SourceStation>
<SourceSize>751568896</SourceSize>
</Details>
<Links>
<Content>
<ContentType>video/x-tivo-mpeg</ContentType>
<Url>http://192.168.0.20:8081/TiVoConnect/GalleonExtra%2FArchive%2F24+Season+4%2F%7B24+Season+416%7D%7 B+4+Apr+05%7D%7BDay+4%3A+10%3A00+P.M.-11%3A00+P.M.%7D%7BRecorded+10.45+PM+Wed+Nov+16%2C+2005%7D%7B KFOX%7D.mpg</Url>
</Content>
<CustomIcon>
<ContentType>video/*</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>urn:tivo:image:save-until-i-delete-recording</Url>
</CustomIcon>
<TiVoVideoDetails>
<ContentType>text/xml</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>/TiVoConnect/TivoNowPlaying/GalleonExtra%2FArchive%2F24+Season+4%2F%7B24+Season+416%7D%7 B+4+Apr+05%7D%7BDay+4%3A+10%3A00+P.M.-11%3A00+P.M.%7D%7BRecorded+10.45+PM+Wed+Nov+16%2C+2005%7D%7B KFOX%7D.mpg?Format=text%2Fxml</Url>
</TiVoVideoDetails>
</Links>
</Item>


So it should be possible to add generation of a correct Duration and Description fields to the galleon code. Making the shows group on the Tivo would be much harder though, since even MRV'd shows don't group if the guide data isn't present on the destination box.

[greg_burns]

One more question: With TiVoToCome, can a .tivo or .mpeg be sent to the TiVo from the PC without using TiVoDesktop?

Galleon is doing it. So it must be possible.

[gonzotek]

This might be possible though galleon, Tivo uses HTTP requests to browse the remote server which returns the program data in XML format. With current versions of Galleon if the file name is in a very specific format Galleon parses it and returns the Episode Title, Channel ID and Recording Date. These do get saved when using GoBack. It would be a relatively simple step forward to add additional information to Galleon's reply.

Data returned by Galleon for a .TiVo file:

<Item>
<Details>
<Title>Malcolm in the Middle</Title>
<ContentType>video/x-tivo-mpeg</ContentType>
<SourceFormat>video/x-tivo-mpeg</SourceFormat>
<Duration>1799000</Duration>
<CaptureDate>0x43BE2380</CaptureDate>
<Description>When Hal and Lois receive a check for $10,000 made out to Malcolm, their first instinct is to keep it, until Dewey overhears their plan and wants a piece of the pie. </Description>
<EpisodeTitle>Malcolm&apos;s Money</EpisodeTitle>
<SourceChannel>2</SourceChannel>
<SourceStation>KTVU</SourceStation>
<SourceSize>546076963</SourceSize>
</Details>
<Links>
<Content>
<ContentType>video/x-tivo-mpeg</ContentType>
<Url>http://192.168.0.20:8081/TiVoConnect/GalleonExtra%2FArchive%2FMalcolm+in+the+Middle+-+Malcolms+Money+%28Recorded+Fri+Jan+6+2006+08+30PM+KTVU%29.T iVo</Url>
</Content>
<CustomIcon>
<ContentType>video/*</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>urn:tivo:image:save-until-i-delete-recording</Url>
</CustomIcon>
<TiVoVideoDetails>
<ContentType>text/xml</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>/TiVoConnect/TivoNowPlaying/GalleonExtra%2FArchive%2FMalcolm+in+the+Middle+-+Malcolms+Money+%28Recorded+Fri+Jan+6+2006+08+30PM+KTVU%29.T iVo?Format=text%2Fxml</Url>
</TiVoVideoDetails>
</Links>
</Item>


And for a .mpg file


<Item>
<Details>
<Title>24 Season 416</Title>
<ContentType>video/x-tivo-mpeg</ContentType>
<SourceFormat>video/x-tivo-mpeg</SourceFormat>
<Duration>0</Duration>
<CaptureDate>0x43C182E8</CaptureDate>
<Description></Description>
<EpisodeTitle>Day 4: 10:00 P.M.-11:00 P.M.</EpisodeTitle>
<SourceChannel>0</SourceChannel>
<SourceStation>0</SourceStation>
<SourceSize>751568896</SourceSize>
</Details>
<Links>
<Content>
<ContentType>video/x-tivo-mpeg</ContentType>
<Url>http://192.168.0.20:8081/TiVoConnect/GalleonExtra%2FArchive%2F24+Season+4%2F%7B24+Season+416%7D%7 B+4+Apr+05%7D%7BDay+4%3A+10%3A00+P.M.-11%3A00+P.M.%7D%7BRecorded+10.45+PM+Wed+Nov+16%2C+2005%7D%7B KFOX%7D.mpg</Url>
</Content>
<CustomIcon>
<ContentType>video/*</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>urn:tivo:image:save-until-i-delete-recording</Url>
</CustomIcon>
<TiVoVideoDetails>
<ContentType>text/xml</ContentType>
<AcceptsParams>No</AcceptsParams>
<Url>/TiVoConnect/TivoNowPlaying/GalleonExtra%2FArchive%2F24+Season+4%2F%7B24+Season+416%7D%7 B+4+Apr+05%7D%7BDay+4%3A+10%3A00+P.M.-11%3A00+P.M.%7D%7BRecorded+10.45+PM+Wed+Nov+16%2C+2005%7D%7B KFOX%7D.mpg?Format=text%2Fxml</Url>
</TiVoVideoDetails>
</Links>
</Item>


So it should be possible to add generation of a correct Duration and Description fields to the galleon code. Making the shows group on the Tivo would be much harder though, since even MRV'd shows don't group if the guide data isn't present on the destination box.Thanks! I had suspected this was how it worked, but haven't had the time to dig through the code or capture the traffic to verify.

[Dan203]

This might be possible though galleon, Tivo uses HTTP requests to browse the remote server which returns the program data in XML format. With current versions of Galleon if the file name is in a very specific format Galleon parses it and returns the Episode Title, Channel ID and Recording Date. These do get saved when using GoBack. It would be a relatively simple step forward to add additional information to Galleon's reply.

Wait, wait , wait! Are you saying that Galleon can actually get your TiVo to add that metadata to a standard MPEG file when using ComeBack by simply reporting it via http when the TiVo requests the file? If so then the "Holy Grail" might actually be possible. It might even be doable via a simple TiVo Desktop plug-in.

Dan

[davezatz]

Galleon is doing it. So it must be possible.

I don't think Galleon can send files to the TiVo unless the TiVo server stuff of the Desktop is running on the PC. An easy way to check... does this feature work on Mac or Linux Galleon installs?

[gonzotek]

I don't think Galleon can send files to the TiVo unless the TiVo server stuff of the Desktop is running on the PC. An easy way to check... does this feature work on Mac or Linux Galleon installs?Galleon works without TD installed. To playback the .tivo files on the pc, you need the dll registered, but other than that retrieving the listings, uploading, and downloading are fully supported, for both .TiVo and .mpgs.

/edit: The galleon 2.2.0 source file located at: galleon-2.2.0.src\src\org\lnicholls\galleon\goback\VideoServer.java is pretty interesting. I haven't gotten started on setting up a build environment for Galleon yet, but if anyone is already doing it, they could test petew's suggestion pretty easily I think.
(this is from galleon's zipped source code (http://prdownloads.sourceforge.net/galleon/galleon-2.2.0.src.zip))

[petew]

I don't think Galleon can send files to the TiVo unless the TiVo server stuff of the Desktop is running on the PC. An easy way to check... does this feature work on Mac or Linux Galleon installs?

It works on linux, that's where my galleon is. Galleon will happily do ToGo and GoBack on a linux box. You just can't watch the .Tivo files.

[davezatz]

I emailed Leon and asked him to take a look at this thread if the topic piques his interest. He might be able to illumate some of these issues... seems like he's started exploring this path already.

Gonzo, what do you need for your build environment? I could FedEx you a fan or area rug. ;)

[petew]

I'm going to go home now and start playing!

[gonzotek]

I emailed Leon and asked him to take a look at this thread if the topic piques his interest. He might be able to illumate some of these issues... seems like he's started exploring this path already.

Gonzo, what do you need for your build environment? I could FedEx you a fan or area rug. ;)
The JDK, ant, and whatever else that's mentioned in the developer's readme in the Galleon source :). I've never done serious work in java before, and although I had the TiVo SDK examples building and running correctly (on a different computer than this), Galleon is far more complex and I need to get the pre-requisites together first(just like my current college experience ;) ). I think I know what it'll take, but I'm taking my time with this whole idea, reading up on things, studying the source seriously for the first time, and generally doing the Zen coding warm-up exercises I always seem to do before embarking on something totally new to me.

I hope Leon chimes in, I hoped to get in touch with him at some point after I've gotten myself more comfortable with things.

Good luck pete!

[davezatz]

Leon says:
Galleon has a built-in database and it would be possible to edit the data
associated with video file so that it can be passed on to TiVo as part of
GoBack. Its not a complete solution, since grouping wont work, but you'll be
able to get more than just the file name to appear on your TiVo.
So obviously he has figured out a way to communicate the info back to TiVo... at least some of it. Folders would be ideal, but any metadata tagging is pretty sweet! (He also confirmed what you guys already reported... GoBack doesn't require the official TiVo software.)

EDIT: Leon has also pointed us to some details of the internal database or adding an external one. He says no GUI, but obviously it can be populated in other ways:
http://galleon.tv/component/option,com_joomlaboard/Itemid,/func,view/catid,5/id,242/#242

EDIT2: What is probably more interesting is how he constructs and transmits the data back to the TiVo unit I'm guessing? Related to what petew posted above...?

[Dan203]

I wonder if grouping doesn't work because it's impossible to send the proper data via the http callback? Or if Leon just hasn't figured out the right XML tag or data format to make it work?

Dan

[gonzotek]

I wonder if grouping doesn't work because it's impossible to send the proper data via the http callback? Or if Leon just hasn't figured out the right XML tag or data format to make it work?

DanAs you're probably already aware grouping won't work even in TiVo's official MRV protocol, if there isn't another episode of the same series in the guide data of the receiving unit. Based on that, I'm not hopeful we can hack in anything on unmodified units.

Dave, thanks for relaying Leon's response! It is indeed related to what petew posted. The outgoing xml looks very very similar(if not identical) to the episode information the TiVo unit sends from it's xml interface.

[Dan203]

As you're probably already aware grouping won't work even in TiVo's official MRV protocol, if there isn't another episode of the same series in the guide data of the receiving unit. Based on that, I'm not hopeful we can hack in anything on unmodified units.

You're right I hadn't considered that. I wonder if we could spoof it by using data from an unrelated show? And if so if that would pick up the group name from the transferred program or the spoofed program?

I'm also starting to wonder if the metadata in the .tivo file is nothing more then encrypted XML data. It makes sense, since the .tivo files use XML for the "salt" and "created date".

Dan

[davezatz]

I'm also starting to wonder if the metadata in the .tivo file is nothing more then encrypted XML data. It makes sense, since the .tivo files use XML for the "salt" and "created date".

It's your home address and phone number. ;) Or your MAK and TiVo serial number that the show originated on. The real tagging metadata is meta to the encrypted metadata and transmitted with (in addition to? two files - xml and mpeg?) the actual mpeg... maybe... if I'm following along here.

Here's something that could confirm that... if we go back to the idea of pulling down a show WITHOUT TiVo Desktop on the system... and then use GoBack is the metadata retained on TiVo? (We may have already covered this here and I've forgotten or don't understand...) If NOT, then the TiVo Desktop software could store the external metadata in some sort of file or database the way Galleon does.

(Has Greg's program worked on a machine without the TiVo Desktop?)

[Dan203]

The metadata is maintained if you use GoBack on a .tivo regardless of whether you use TiVo Desktop or your browser to download the file. In fact I've actually moved .tivo files to a different computer and and used a different version of TiVo Desktop to transfer files back to a TiVo and the metadata is always maintained.

Dan

[HDTiVo]

Here's an outline of my idea:

Define two new file formats with extensions called ".otivo" and ".ompeg" The "o" stands for open, clear text metadata.

Both formats differ from their non-"o" counterparts only in having clear text program and related data in a defined (by us) format at the beginning of the file. Ie. perhaps the first 10KB contains this data (the "Open Header" (OH))...

A new program called Open TiVoDesktop (OTD) does basically what TiVoDesktop does now, but downloads .tivos as .otivos by taking the metadata as read in the clear via XML and writing the OH at the beginning of the file and standard .tivo to the rest of the .otivo file.

A Direct Show Filter called Open TiVo DSF (OTDSF) is written which when a .otivo is openned with a DS aware program sends everything after the OH to the regular TiVo DSF.

OTD has an edit function that lets you modify the OH of a .otivo and .ompeg.

OTD uses the OH to display program info, unlike TD which uses the encrypted TiVo
metadata.

OTD can convert a .otivo to a .ompeg, which is a DSD style mpeg with an OH at the beginning. OTD also reads and displays the OH from a .ompeg.

A DSF is written for .ompegs that filters the OH before the plain mpeg then heads down the line to the next DS step.

OTD can send back to a TiVo either an .otivo or .ompeg without the OH (ie. a .tivo or .mpeg), however if possible to use the xml trick, the OH is sent to replace the original TiVo metadata - thereby achieving edited TiVo program info (Yippeee!!!!)

OTD incorporates Greg's RA functionality to 1. output regular mpegs, with .header file also (DeAttach); 2. ReAttach a .header to a .mpeg and create either a .tivo or .otivo.

....

I am going to stop hear so people can tell me if I'm sounding nuts before I spend any more time.

[davezatz]

The metadata is maintained if you use GoBack on a .tivo regardless of whether you use TiVo Desktop or your browser to download the file. In fact I've actually moved .tivo files to a different computer and and used a different version of TiVo Desktop to transfer files back to a TiVo and the metadata is always maintained.

So then I'll agree with you the encrypted stuff must be that string of XML tagged stuff petew post. So it can obviously be sent encrypted (as TiVo does it) or unencrypted as Leon is doing it. All we need now is a light weight (lighter than Galleon) widget to ask us info about a show, populate the XML info based on our responses, and attach it to the mpeg.

[Dan203]

You wouldn't even need "Open TiVo Desktop". The real TiVo desktop has a plug-in arcitecture that would do this without having to reinvent the wheel. Only problem is the plug-in arcitecture is not documented for use with video, so it might be a little tricky to figure out. Although I know for a fact that it does work with video because TiVo uses a plug-in themselves to support standard MPEG files.

Or, again instead of reinventing the wheel, you could simply add support for your new file type to Galleon.

Dan

[Dan203]

So then I'll agree with you the encrypted stuff must be that string of XML tagged stuff petew post. So it can obviously be sent encrypted (as TiVo does it) or unencrypted as Leon is doing it. All we need now is a light weight (lighter than Galleon) widget to ask us info about a show, populate the XML info based on our responses, and attach it to the mpeg.

I'm going to play with this over the weekend and see if I can populate a .tivo file with unencrypted metadata. I'll let you all know what, if anything, I come up with.

Dan

[HDTiVo]

You wouldn't even need "Open TiVo Desktop". The real TiVo desktop has a plug-in arcitecture that would do this without having to reinvent the wheel. Only problem is the plug-in arcitecture is not documented for use with video, so it might be a little tricky to figure out. Although I know for a fact that it does work with video because TiVo uses a plug-in themselves to support standard MPEG files.

Or, again instead of reinventing the wheel, you could simply add support for your new file type to Galleon.

Dan
Whatever works is good. What do folks think of the conceptual model?

[petew]

So spent a couple of hours getting the DB2 CLP working so I could manipulate the galleon database and I can confirm <drum roll> that if you return the Program Title, Episode Title and Description in the XML fields they are saved on the Tivo.

[davezatz]

So spent a couple of hours getting the DB2 CLP working so I could manipulate the galleon database and I can confirm <drum roll> that if you return the Program Title, Episode Title and Description in the XML fields they are saved on the Tivo.

How are those XML fields transmitted? Clear text appended to the beginning of a mpeg or as a seperate file? Or something else I'm not getting? If you do append them, I assume.

Like I said, I think all we need to some sort of widget which we populate with data and tell it which mpeg file to associate with. Then upload as you normally would.

Oh yeah, good work! All of you. :)

[greg_burns]

Define two new file formats with extensions called ".otivo" and ".ompeg" The "o" stands for open, clear text metadata.

This reminds me of the new Office 12 file formats: .docx and .xlsx

These are just zip files (if you rename them to .zip you can unzip them!) that contain .xml files and other "stuff".

.tivox could be a zip file that contains the freed .mpg file and the clear text metadata .xml file

[gonzotek]

How are those XML fields transmitted? Clear text appended to the beginning of a mpeg or as a seperate file? Or something else I'm not getting? If you do append them, I assume.

Like I said, I think all we need to some sort of widget which we populate with data and tell it which mpeg file to associate with. Then upload as you normally would.

Oh yeah, good work! All of you. :)No, it's similar to the way that TiVo Desktop has always published metadata about mp3s or jpegs. When the TiVo finds the PC server, it makes a url request to it, and the PC returns an xml file describing the contents of the shared directory.

Now, an editing widget that appended the data in some standardized way would be ok, if we can then serve that file as seperate metadata and content back to the TiVo. As Dan was saying TD can take a plugin, and Galleon can be modified to do so, so we have at least two options open. And someone could always code up a standalone video server, using the code in Galleon for reference.

[davezatz]

No, it's similar to the way that TiVo Desktop has always published metadata about mp3s or jpegs. When the TiVo finds the PC server, it makes a url request to it, and the PC returns an xml file describing the contents of the shared directory.

Now, an editing widget that appended the data in some standardized way would be ok, if we can then serve that file as seperate metadata and content back to the TiVo. As Dan was saying TD can take a plugin, and Galleon can be modified to do so, so we have at least two options open. And someone could always code up a standalone video server, using the code in Galleon for reference.

OK, so if I'm understanding this right... something on the PC has to be listening and responding to requests. That something would then return an XML file... an index of *each* mpeg and associated metadata in the directory?

Hmmm... A TiVo Desktop plugin of some sort might be easier for the masses, but is Galleon easier for developers (you guys) to interact with? Because Galleon appears modular could the GoBack widget be enhanced with some fields to complete pointing to specific mpegs... which then could be thrown into the XML template?

[javahmo]

So spent a couple of hours getting the DB2 CLP working so I could manipulate the galleon database and I can confirm <drum roll> that if you return the Program Title, Episode Title and Description in the XML fields they are saved on the Tivo.

For those who dont want to manipulate the database, Galleon will guess the metadata about a video file using the following file name patterns:
http://galleon.tv/component/option,com_joomlaboard/Itemid,/func,view/catid,30/id,61/#60
Galleon actually ignores file extensions, so you could use any of these patterns with mpg files. There are some variations on these patterns supported which is descibed in the source code at: org.lnicholls.galleon.media.VideoFile.java
These patterns could be extended to include more metadata.

GoBack is implemented using a TiVo protocol called HMO:
http://tivo.com/developer/i/HmoMusicPhotosSpecv1.1.pdf

[davezatz]

Thanks for dropping by Leon! I'm mostly an observer here, but I did have a question about something you said in that referenced thread...

"Galleon would need to support some additional file formats to give TiVo the necessary information for grouping."

What exactly does that mean? :) Do you have some ideas how to making grouping work...

Ideally, I'd want to fill a compuer directory with home movies, somehow add the metadata (Galleon, TiVo Desktop plugin, mystery widget), and then move those mpegs to TiVo (manually or otherwise).

[javahmo]

Thanks for dropping by Leon! I'm mostly an observer here, but I did have a question about something you said in that referenced thread...

"Galleon would need to support some additional file formats to give TiVo the necessary information for grouping."

What exactly does that mean? :) Do you have some ideas how to making grouping work...

Ideally, I'd want to fill a compuer directory with home movies, somehow add the metadata (Galleon, TiVo Desktop plugin, mystery widget), and then move those mpegs to TiVo (manually or otherwise).

I had assumed at the time that all I had to do was add enough metadata and TiVo will do the grouping right, but I was proved wrong. If TiVo doesnt know about the show in its guide data, grouping never seems to work.

[HDTiVo]

2.3 Application Flexibility
The meta-data format is not tied to any particular organization inherent in the source
information, making it possible not only to represent "physical" containers (folders on a hard disk,
for example), but also "virtual" ones (a randomized selection of images, a "TiVo Top 40" list
based on thumb ratings, a genre/artist/album hierarchy built from ID3 tag data, and so on). This
is all up to the Server and the meta-data it decides to generate.

Are you guys saying what we see on the TV screen as program info is basically just a Web page, so the home made metadata could be in an arbitrary format and the program info screen for those shows would look like the home made format?

[javahmo]

Are you guys saying what we see on the TV screen as program info is basically just a Web page, so the home made metadata could be in an arbitrary format and the program info screen for those shows would look like the home made format?

The metadata about each show has to conform exactly to the TiVo XML specs. The format of the data that the recorder expects from Galleon is exactly the same format as that provided by the recorder on its web pages for each show.

[juanian]

I do recall a situation when we (my wife and I) had received a "new" TiVo. I had transferred some shows from our old TiVo to the new TiVo. One set of shows transferred oddly didn't nest into a folder. I figured out that all of the other shows had at least one episode from the series in the current guide data, but the odd show had been canceled, so there was nothing in the guide data for it. Looking at the TiVoVideoDetails XML for the shows, it did lack the <series> info, which contains the series title. My guess (based on this) is that the <series><seriesTitle> info is critical to grouping of shows. (Note that shows that group based on a WishList don't fall into this category.) (The more complete nesting is <showing><program><series><seriesTitle>.)

Unfortunately, this is the show detail info likely contained in the encrypted header.

Now, here's something that I haven't seen suggested yet in this thread. Since you can apparently attach an unencrypted mpeg stream after an encrypted header, there doesn't appear to be anything intrinsic to the .TiVo format that may require encryption. So, has anyone tried putting unencrypted XML info into the likely place in a header block to see what would happen? The exact length of the XML data (the complete info from TiVoVideoDetails) appears to be located at offset 0x488 (in my sample file) for 4 bytes (or 8 bytes into that section after the padded </license> area). So, maybe plopping the XML into the area (and maybe flipping some of those bits) might be a way to try this? (I'm sure someone who knows more than I about the innards of a .TiVo file can give this a stab.)

Just a thought . . .

[HDTiVo]

This reminds me of the new Office 12 file formats: .docx and .xlsx

These are just zip files (if you rename them to .zip you can unzip them!) that contain .xml files and other "stuff".

.tivox could be a zip file that contains the freed .mpg file and the clear text metadata .xml file
Yes. And putting the OH at the front of a .otivo or .ompeg is arbitrary too.

The reason I propose the DSFs in my architecture is to make the whole thing seamless to the user. No conversions back and forth to one thing and another. Click on the ".oXXXX" file and everything works in all DS aware programs just like it was a ".XXXX" file.


The metadata about each show has to conform exactly to the TiVo XML specs. The format of the data that the recorder expects from Galleon is exactly the same format as that provided by the recorder on its web pages for each show.

Its sounding like XML method is going to work - and maybe grouping is going to work also?


You wouldn't even need "Open TiVo Desktop".

The OTD functionality can certainly be implemented in any of the ways suggested that someone is comfortable with and can get to work - 1. seperate app 2. TD plug in 3. Galleon extension 4. Rewrite based on Galleon

OTD's purpose is to house all functionality in one place so the user isn't jumping from one program to another.

[Dan203]

I've been playing with this a little more and I made a couple more discoveries...

1) While the basic XML data galleon provides is sufficient, there is actually architecture in place to provide all the extended information you normally see when you press the info button on the TiVo. I'm not sure if this helps grouping or not, but it might be worth looking at some more.

2) While I can't actually extract the data directly from a pre-existing .tivo file it is possible to get it from the TiVo Server application by querying the TiVo Server using HTTP commands. It's not super elegant, but it could be used as a work around to extract info from .tivo files which are on your hard drive but no longer on your TiVo.

3) While it is possible to create a video plug-in for TiVo Desktop TiVo has absolutely no documentation on how to do it. I've sent an email to someone I know at TiVo to see if they can get me something, but if they can't we'll probably have to resort to an open desktop type application like HDTiVo suggested.

I'm also looking into the possibility of adding the meta data directly to a standard .mpg file. From what little information I've found thus far it sounds like you might be able to add proprietary information to a standard MPEG stream using a "private stream" or "user data" section of the file. If what I'm reading is correct this will allow software that knows what to do with the data to read it and use it, while all other MPEG players will simply ignore it and play the audio and video streams. If this is truly possible I think this is best solution for maximum portability of user created MPEG files with metadata. Alternatively we could just forget about the one file approach and simply allow the creation of an XML file which has the same name as the MPEG and just read the information from that. (probably a lot easier, but not as elegant)

Dan

[HDTiVo]

I'm also looking into the possibility of adding the meta data directly to a standard .mpg file. From what little information I've found thus far it sounds like you might be able to add proprietary information to a standard MPEG stream using a "private stream" or "user data" section of the file. If what I'm reading is correct this will allow software that knows what to do with the data to read it and use it, while all other MPEG players will simply ignore it and play the audio and video streams. If this is truly possible I think this is best solution for maximum portability of user created MPEG files with metadata. Alternatively we could just forget about the one file approach and simply allow the creation of an XML file which has the same name as the MPEG and just read the information from that. (probably a lot easier, but not as elegant)


Just as long as the way you are talking about doing this does not result in files the TiVo will choke on with TiVoToCome. The TiVo may be sensitive to extra streams that other mpeg playing thingys just whistle past. If necessary, could a DirectShow Filter (DSF) take care of that, eliminating any need to write back out to normal .tivo or .mpeg format?

Awesome work Dan, esp #2, query the TiVo Server for xml info from a .tivo on your PC!

[macfisherman]

I've been playing with this a little more and I made a couple more discoveries...

1) While the basic XML data galleon provides is sufficient, there is actually architecture in place to provide all the extended information you normally see when you press the info button on the TiVo. I'm not sure if this helps grouping or not, but it might be worth looking at some more.


From my tinkering, all meta data is queried separately from the mpeg stream. Also, some things must be retrieved via HTTPS and others (the MPEG streams) via HTTP. I believe a .tivo file is just a convienent container. I've been working on a perl based "TiVo Server" to run on a Linux box to serve up my local video archive. But all of this may not be needed if DLNA is also offered on Series 2 boxes. Buffalo Tech has something called "LinkStation Home Server 300GB". It is basically a NAS device that's also a DLNA device. So if I understand how these things work, the Buffalo device would show up on a DLNA capable TiVo. If TiVo doesn't support that, then I plan to put my "TiVo Server" on the LinkStation very much like those folks who have hacked their for SlimServer support.

As an aside, does anyone else believe the license for the REST/Web based interface (aka HMO) is more stringent than HME? I'd asked once before but never got a response.

[CuriousMark]

I've been working on a perl based "TiVo Server" to run on a Linux box to serve up my local video archive. But all of this may not be needed if DLNA is also offered on Series 2 boxes. Buffalo Tech has something called "LinkStation Home Server 300GB". It is basically a NAS device that's also a DLNA device. So if I understand how these things work, the Buffalo device would show up on a DLNA capable TiVo. If TiVo doesn't support that, then I plan to put my "TiVo Server" on the LinkStation very much like those folks who have hacked their for SlimServer support.
I have a KuroBox that I have been playing with in order to do just what you are talking about, but never got further than some wishful thinking. If you are looking for a partner, helper, tester, let me know and I will do my best to help out. I know absolutely no Perl, but have been reading up on Java, thinking that was the way to go.

I think DLNA is not in the cards for a series 2. It is UPNP based and very top-heavy. TiVo went with the Bonjour approach which is much lighter. Still, with enough programming effort it might be possible to support both, but changing horses mid-stream would not be easy. (OT:If there was any one thing that might win DTV back, it might be DLNA since they announced that is what their DVR will ostensibly support.) Maybe TiVo could do DLNA on the series 3 since it will hopefully have more horsepower.

Anyway, anything to give me a kick in the pants to work this project is a good thing, so how can I help? PM me and let me know what you need.

CuriousMark

[Dan203]

Also, some things must be retrieved via HTTPS and others (the MPEG streams) via HTTP.

Actually all of the metadata can be retrieved via standard http using a URL like so....

http://tivo:<MAK>@<Computer_IP>:8080/TiVoConnect/TivoNowPlaying/I<filename>.TiVo?Format=text%2Fxml

It's the longer version of the data, but it contains all the information needed for the shorter version as well.

Dan

[DUDE_NJX]

OK, so if tivo needs program guide data to create extended xml header with folder info, how come the Rocketboom nonsense gets put into its folder? :confused:

Being able to add folder info to my mpegs is #1 on my new feature list. I hope we can figure it out one day or maybe the good tivo folks will decide to enable that via the Tivo Desktop app. (fingers crossed)

[Dan203]

Good point. Does anyone know of a way to check what, if any, XML data is coming across before a Rocketboom starts to download?

Dan

[greg_burns]

Good point. Does anyone know of a way to check what, if any, XML data is coming across before a Rocketboom starts to download?

Starts to download to the Tivo box? As you probably know, you cannot download Rocketbooms to the PC. How would you check this, with a packet sniffer or something?

I tried to transfer a .tivo cartoon back the other day and it didn't group. I checked and there didn't appear to be any future episodes in the guide. I kinda assumed later it would group with the rest, but it never did (even after it showed back up in the guide.) :confused:

[CuriousMark]

It makes one wonder, is TiVo putting the IP downloads into the guide somehow? If so, could they put a generic "Home Video" program in the guide that people could use to force grouping?

CuriousMark

[morac]

The grouping must key off some meta data. The easiest thing to do would probably see what data is the same (other than the show title) in the XML file for the shows grouped together.

[AV_Novice]

The grouping must key off some meta data. The easiest thing to do would probably see what data is the same (other than the show title) in the XML file for the shows grouped together.

I don't think that is true. If it only used metadata from the files, then two shows would group even if they are no longer listed in the program guide.

Greg_Burns indicated that he transferred a cartoon back to his Tivo and it wouldn't group. I think it is safe to assume he had more than one program recorded on the Tivo that should have allowed it to group.

[greg_burns]

I don't think that is true. If it only used metadata from the files, then two shows would group even if they are no longer listed in the program guide.

Greg_Burns indicated that he transferred a cartoon back to his Tivo and it wouldn't group. I think it is safe to assume he had more than one program recorded on the Tivo that should have allowed it to group.

I would agree. I am going to play tonight. I have a some cartoons that are currently grouped that are not coming up in the guide (no future episodes). What will happen if I transfer one to the PC and immediately transfer it back? Will it group or not? :confused:

Wish I still had the one that didn't group but it annoyed me sitting there by itself so I killed it.

[EwanG]

At the risk of appearing more than a little dense, has anyone tried PM'ing one of the Tivo folks who hangs out around the forums to ask if Tivo would either:

A) Add this functionality to the next version of Tivo Desktop

or

B) Create a plugin that could be used by the current and next version

or

C) At least tell the folks here who are willing to create such a tool what they need to know

Solving the problem being discussed will not affect their ability to protect videos, or to keep their DRM intact. Rather this would be VERY useful for those of us who have a lot of our own content.

FWIW,
Ewan

[Aflat]

Another test would be to see if a recently cancelled show still groups. I know Love Monkey was cancelled, but I only have 1 episode, so it won't group. Maybe it has something to do with the transfer.

[davezatz]

Another test would be to see if a recently cancelled show still groups.

Could one test that theory by removing a channel from the lineup so it appears that there are no upcoming episodes?

[greg_burns]

Could one test that theory by removing a channel from the lineup so it appears that there are no upcoming episodes?

I had thought of doing just that, but wasn't sure if it would be a valid test. The guide data maybe already downloaded and cached or somethin'.

[petew]

A) Add this functionality to the next version of Tivo Desktop



I've already done so, anyone else can go to http://research.tivo.com/suggestions/ and add you 2 cents worth. The more that ask the more likely it might be done.

[DUDE_NJX]

Back to folders and the Rocketboom example. Tivo doesn't wait for 1+ episodes to create a folder. The first episode goes in a folder as soon as it starts the download.

[petew]

Back to folders and the Rocketboom example. Tivo doesn't wait for 1+ episodes to create a folder. The first episode goes in a folder as soon as it starts the download.

Which is consistent with auro record wish lists. An actor ARWL creates a folder with the actors name.

[AllAboutJeeps]

Another test would be to see if a recently cancelled show still groups. I know Love Monkey was cancelled, but I only have 1 episode, so it won't group. Maybe it has something to do with the transfer.

We had three or four episodes of Joey that grouped into early February when we finally watched them. Joey was cancelled in Dec I believe. Joey however did remain in my Season Pass List. Actually, we had a backlog of Apprentices that stayed grouped also long after the season finale aired, but they were recorded under a Wish List.

...danny

[greg_burns]

We had three or four episodes of Joey that grouped into early February when we finally watched them. Joey was cancelled in Dec I believe. Joey however did remain in my Season Pass List. Actually, we had a backlog of Apprentices that stayed grouped also long after the season finale aired, but they were recorded under a Wish List.

...danny

Yes, I've never seen anything become ungrouped while living on the Tivo. The question is, what happens if you transfer one of those to your PC then back to the Tivo???

[davezatz]

I had thought of doing just that, but wasn't sure if it would be a valid test. The guide data maybe already downloaded and cached or somethin'.

Could be... but I won't know until you test it. ;)

[Aflat]

I'll try it now, I'm downloading 2 episodes of Four Kings (which was cancelled, thankfully) and I'll reupload them and see what it does.

[greg_burns]

I downloaded a cartoon earlier and immediately transferred it back. It went right back into the group with the rest. AFAIK, it is not currently in the guide.

Maybe I need to delete the others from the group first...

[Aflat]

So I downloaded both episodes of Four Kings. Deleted them from the tivo, so the group was empty, confirmed that Four Kings couldn't be found in the guide anywhere, and reuploaded them to the Tivo.

They got grouped. So it doesn't appear that the guide data has to do with grouping.

[greg_burns]

So it doesn't appear that the guide data has to do with grouping.

There are people much smarter than I on this group that think that it does. I am just not sure of the details myself... :(

[davezatz]

There are people much smarter than I on this group that think that it does. I am just not sure of the details myself... :(

I'm not smarter than Greg, but I wonder if Greg's thought that somehow the info is cached might be on target... either that or once something has been grouped the XML has been updated in the file to reflect that and it is retained. After how many show uploads did the folder show back up - did it happen after just the first show?

[Aflat]

I'm not smarter than Greg, but I wonder if Greg's thought that somehow the info is cached might be on target... either that or once something has been grouped the XML has been updated in the file to reflect that and it is retained. After how many show uploads did the folder show back up - did it happen after just the first show?

Nope, it only grouped after the second show started to upload. But that is how grouping works, if you only have 1 show it won't be grouped.

I'll try to upload both shows to my other Tivo and see if they get grouped there as well.

[morac]

They got grouped. So it doesn't appear that the guide data has to do with grouping.
Guide data has a good deal to do with the grouping because the guide data is where all that information that shows up in the XML data (I'd start with SeriesId).

I once asked E. Stephen Mack (better known as TiVoOpsMgr) about an issue I had with grouping in 4.0.1b where one show refused to group and if I tried to play it, it would play a different episode of that show (the first one in the group).

This was his reponse:
Don't worry about this problem causing more corruption; the issue isn't any particular corruption, just some strange behavior for grouping if the shows are part of the same series but don't have the exact same title. This problem usually is only seen when a series has changed its title.So apparently seriesID and show title have something to do with grouping (both come originally from the guide data, but are then stored with the program). This was back in 4.0.1b and grouping most likely changed since I haven't seen this issue in quite a while now.

[AV_Novice]

Guide data has a good deal to do with the grouping because the guide data is where all that information that shows up in the XML data (I'd start with SeriesId).

I once asked E. Stephen Mack (better known as TiVoOpsMgr) about an issue I had with grouping in 4.0.1b where one show refused to group and if I tried to play it, it would play a different episode of that show (the first one in the group).

This was his reponse:
So apparently seriesID and show title have something to do with grouping (both come originally from the guide data, but are then stored with the program). This was back in 4.0.1b and grouping most likely changed since I haven't seen this issue in quite a while now.

I believe you are correct from the standpoint that the only way to populate a show's metadata is with the information from the guide. Once the show is recorded, the metadata is stored with the file, so it shouldn't matter if the show is no longer shown in the guide.

Greg_Burns uploaded two shows to his Tivo that are no longer in the guide. These shows still grouped. One theory is they were grouped using the metadata in each file. 2nd theory is there is some information cached in the Tivo that "remembered" these programs and knew to group them.

[DVDerek]

I believe you are correct from the standpoint that the only way to populate a show's metadata is with the information from the guide. Once the show is recorded, the metadata is stored with the file, so it shouldn't matter if the show is no longer shown in the guide.

Greg_Burns uploaded two shows to his Tivo that are no longer in the guide. These shows still grouped. One theory is they were grouped using the metadata in each file. 2nd theory is there is some information cached in the Tivo that "remembered" these programs and knew to group them.

If I had to harbor a guess, I'd say that when the TiVo originally grouped the files, it put metadata into the file.

How about this for a test - Take 2 different series' that are no longer in the guide but that you have on your TiVo. One of these series' should have 2 or more episodes (so that it groups) - let's call this series A - and the other should have only 1 ungrouped episode (Series B). Download the three episodes to your PC and delete them from the TiVo.

Make a copy of the series B episode so that you now have 2 of each series. Upload the 2 episodes of series B and see if they group (they likely won't). Delete them from the TiVo.

Using Greg's app, take the headers off the 4 tivo files. Re-attach the headers for series B to the shows for series A and vice versa. Re-upload the 4 episodes to the TiVo.

What groups now?

-----------------
Re-reading this now, I don't know that it would prove anything, but I'll leave it out there as an idea... I just checked and I don't have an canceled shows on my TiVo to try this with.

[greg_burns]

Not sure what this proves, but my last test was...

Pulled off Star Wars Clone Wars epidsode. Only had 1 on the Tivo (of course, no group). Not any upcoming in the guide, hasn't recorded one in months.

Once on my PC, turned around and put back on the Tivo. Now there are two (identical) and they grouped themselves together. :confused:

[AV_Novice]

Not sure what this proves, but my last test was...

Pulled off Star Wars Clone Wars epidsode. Only had 1 on the Tivo (of course, no group). Not any upcoming in the guide, hasn't recorded one in months.

Once on my PC, turned around and put back on the Tivo. Now there are two (identical) and they grouped themselves together. :confused:

That proves there is nothing added to the file in the metadata that says the file is or was part of a "group".

The Tivo reads the metadata from the file and only groups the files IF the program name is an exact match.

[greg_burns]

Yeah, but I hard another cartoon that would not group with its others when put back. And it originally was part of that group. Why didn't it group? It had the same program name. Very confusing. :confused:

[AV_Novice]

Yeah, but I hard another cartoon that would not group with its others when put back. And it originally was part of that group. Why didn't it group? It had the same program name. Very confusing. :confused:

Greg, try this.

In your browser enter the following: https://192.168.0.104/TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes
Substitute your IP address above. This command will return the xml file of the NowPlaying list. Scroll through the list and find the cartoons you are describing that won't group.

Are the Titles the same?
How about SeriesId?

Wild guess, but it may only be grouping by SeriesId and nothing else. SeriesID is probably assigned by the program guide created by Tivo when the file is recorded.

[HDTiVo]

Also wasn't there mention of a subtle change in the series name sometimes causing problems. Maybe there is an extra space or invisible character?

[greg_burns]

Greg, try this.

In your browser enter the following: https://192.168.0.104/TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes
Substitute your IP address above. This command will return the xml file of the NowPlaying list. Scroll through the list and find the cartoons you are describing that won't group.

Are the Titles the same?
How about SeriesId?

Wild guess, but it may only be grouping by SeriesId and nothing else. SeriesID is probably assigned by the program guide created by Tivo when the file is recorded.

I would love to, but I don't have that file anymore. I've only seen it happen once (so far) and that was a couple weeks ago. :(

Killed it from the Tivo as well (it annoyed me that it wasn't grouped :D).

[greg_burns]

Also wasn't there mention of a subtle change in the series name sometimes causing problems. Maybe there is an extra space or invisible character?

Yeah, but the one in question was part of a group before I took if off the Tivo. I used VideoRedo/TivoAttach to remove commercials and replace the header and put it back. Should have regrouped, but didn't. Not discounting one of those two things screwed it up, but I've done many others without that happening.

[DVDerek]

It's baffling to me that none of the TiVo-ites on this board can just tell us "This is how shows group..."

I don't think we're talking trade secrets here!

[greg_burns]

In your browser enter the following: https://192.168.0.104/TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes
Substitute your IP address above. This command will return the xml file of the NowPlaying list.

Why does this return an incomplete list? For example, I have 9 Catscratch episodes in my NPL, but this only has info for 6 of them. :confused:

TivoDestkop shows them to me.

I can also find them using this URL:
https://192.168.1.25/nowplaying/index.html

dcahoe's TivoPlayList (http://www.tivocommunity.com/tivo-vb/showthread.php?t=284443&page=1&pp=30) app (v0.48) shows them to me as well.

But the query params you gave me seems to return incomplete data. I am missing other shows as well (for example both of those Star Wars Clone Wars episodes).

Not sure the signifigance of this just at the moment...

[Aflat]

I uploaded both episodes for Four Kings onto my other Tivo, and they both grouped just fine. Still nothing in the guide about Four Kings.

[morac]

Why does this return an incomplete list? For example, I have 9 Catscratch episodes in my NPL, but this only has info for 6 of them. :confused: TiVo will only return a maximum of 128 shows at a time. So if you have more than 128 shows on your TiVo, you'll only get the latest 128 with the request specified above. You'll need to use the offset parameter to get the others. See this thread (http://www.tivocommunity.com/tivo-vb/printthread.php?t=235563) I started last year for more info.

[greg_burns]

TiVo will only return a maximum of 128 shows at a time. So if you have more than 128 shows on your TiVo, you'll only get the latest 128 with the request specified above. You'll need to use the offset parameter to get the others. See this thread (http://www.tivocommunity.com/tivo-vb/printthread.php?t=235563) I started last year for more info.

I do now recall reading that thread before. Thanks for clearing that up!

[Dan203]

I uploaded both episodes for Four Kings onto my other Tivo, and they both grouped just fine. Still nothing in the guide about Four Kings.

I noticed that some shows have a SeriesID tag in their XML data. I wonder if this is something new that's being added to transferred files so that they'll group properly even on a TiVo which does not have the proper data in the guide?

If so then that would be something we could use to group personal recordings. All we need now is a server or plug-in to generate this data for standard MPEG files. I'm becoming pretty well acquainted with the TiVo plug-in architecture, so as soon as TiVo releases an updated SDK with video plug-in support I should be able to throw something together.

Dan

[AV_Novice]

I noticed that some shows have a SeriesID tag in their XML data. I wonder if this is something new that's being added to transferred files so that they'll group properly even on a TiVo which does not have the proper data in the guide?

If so then that would be something we could use to group personal recordings. All we need now is a server or plug-in to generate this data for standard MPEG files. I'm becoming pretty well acquainted with the TiVo plug-in architecture, so as soon as TiVo releases an updated SDK with video plug-in support I should be able to throw something together.

Dan

Does this mean that you have older shows recorded that don't have a SeriesID tag? If so, how old are they. I have only had my Tivo since December, so all my shows have the SeriesID tag. If you have older shows without the tag, then that may be the answer.

There is probably some legacy support for grouping (the old way it was done before they started using SeriesID) that used active listings in the program guide.

[Dan203]

I do have shows which do not have the SeriesID. However I haven't investigated to see if there is any correlation with age or not. I'll look into it and let you know.

Dan

[Dan203]

There is probably some legacy support for grouping (the old way it was done before they started using SeriesID) that used active listings in the program guide.

I'm still a little confused as to when this happens. I know for a fact that about 2-3 months ago Cynthia transfered a bunch of recordings from a digital cable channel to another TiVo that does not have digital cable and the recordings did not group on the destination TiVo. However it's possible that those shows were actually recorded before the originating TiVo got the most recent software, so that might explain it. Or perhaps they snuck something in a little more recently that started adding this SeriesID tag.

Dan

[HDTiVo]

I'm still a little confused as to when this happens. I know for a fact that about 2-3 months ago Cynthia transfered a bunch of recordings from a digital cable channel to another TiVo that does not have digital cable and the recordings did not group on the destination TiVo. However it's possible that those shows were actually recorded before the originating TiVo got the most recent software, so that might explain it. Or perhaps they snuck something in a little more recently that started adding this SeriesID tag.

Dan
My first thought was that season passes are channel specific and about whether that might have grouping implications on a TiVo that does not get the channel another TiVo made the recording on. Now I have 24 from recorded on two different channels - with two different tivoes - stored together in the same group on one tivo. And both tivoes get both source channels for 24.

If that means anything?

P.S. Could season pass recordings be using SeriesIDs and other recordings not?

[AV_Novice]

P.S. Could season pass recordings be using SeriesIDs and other recordings not?

I don't think so. I have recordings that are season pass and others that are not season pass. Both contain SeriesID when viewing the XML data.

As I was reviewing the XML file returned by my Tivo, I noticed that one of my shows did not contain SeriesID data. I recorded the SEMA show from Las Vegas. Since this was a special program and not part of a series, it makes sense that it wouldn't have a SeriesID. I had downloaded a copy to my PC, but I still had the original copy of the show on the Tivo. I decided to upload the same show from my PC back to the Tivo and see what happened. It created a 2nd copy of the same program and immediately grouped them! I guess it doesn't have to have SeriesID to group.

Someone will probably need to have a couple of shows that they can repeatedly not get to group (when logics says they should) so we can examine the difference in the data for the shows.

[HDTiVo]

I wonder if its as simple as Tribune semi-randomly including/excluding certain parts of the metadata and TiVo having two or three ways of attempting to group?

Or what if TiVo has three tags used to group; when a show is recorded, it checks the tags in the order 1,2,3 for a match to a previous recording. Upon a match it groups. Now, suppose with TiVoToCome, the order checking happens to be reversed 3,2,1 and on the first non-match it decides not to group.

Theory.

[AllAboutJeeps]

We haven't even started talking about wishlist groupings and how they work! I wonder if you can enter a 'title' wishlist for a couple of custom named mpgs that have been transferred to the tivo and perhaps they would be grouped as a wishlist recording!?!? Just throwing it out there to confuse everyone :) .

I know a show can show up in a standard grouping AND a wishlist grouping.

...danny

[morac]

There may be more than one way to cause a show to group. For example maybe all shows with the same title on the same channel group. Also since shows recorded under the same Season Pass or Wish List group that also might be a criteria.

An interesting experiment would be to manually record shows that show up on different channels like The Simpsons or something and see if they group. You would have to make sure you don't have any season passes or wish lists for that program though.

[Dan203]

Shows on different channels do group. I know this because I get suggestions for shows on various channels that get grouped together with my Season Pass recordings.

I really think it's this SeriesID tag that is causing the grouping and that something in the most recent version of the software changed so that this information is now transmitted with transferred recordings.

We just need some sort of http server software that we can use to test that. Unfortunately I don't have the knowledge, nor the time, to write something to do that.

Dan

[videoredo]

FYI, The next beta of VideoReDo Plus will have a .tivo output option. When selected, metadata from the source .tivo file is automatically copied to the output. If no metadata is found from the source file, then the result will be a simple MPEG program stream.

Beta should be available later today (Tuesday).

[greg_burns]

FYI, The next beta of VideoReDo Plus will have a .tivo output option. When selected, meta data from the source .tivo file is automatically copied to the output. If no meta data is found from the source file, then the result will be a simple MPEG program stream.

:up: :up: :up:

[HDTiVo]

FYI, The next beta of VideoReDo Plus will have a .tivo output option. When selected, metadata from the source .tivo file is automatically copied to the output. If no metadata is found from the source file, then the result will be a simple MPEG program stream.

Beta should be available later today (Tuesday).
:up: :up: :up:
If only there were four thumbs up.

[AllAboutJeeps]

FYI, The next beta of VideoReDo Plus will have a .tivo output option. When selected, metadata from the source .tivo file is automatically copied to the output. If no metadata is found from the source file, then the result will be a simple MPEG program stream.

Beta should be available later today (Tuesday).


Once again VRD, Best money I have EVER spent on software. Keep up the awesome work!

:up: :up: :up:

[Dan203]

Once again VRD, Best money I have EVER spent on software. Keep up the awesome work!

:up: :up: :up:

Same here! :)

Dan

[videoredo]

New Beta has been posted. Please leave comments on the VideoReDo forum.

[petew]

Bad news! Even if you return the correct Title, Series ID and Program ID in the XML data from galleon. Tivo does not group the .mpg recording, conversely a .TiVo file of the same show without the extra info groups anyway.

[Dan203]

That's too bad. Although it would still be nice to upload personal programs with more data then just the name of the file.

On a related note I briefly played with hacking together a TD plug-in for this. And while I was able to get the MPEG files with my hacked data to show up on the TiVo I couldn't actually get it them to transfer. Looking at the XML data returned by TD it looks like they use a different URL format for video transfers then they do for Music/Photos, and none of helper functions in the current SDK will format the data correctly. (I couldn't even hard code them the right way, so it must require a specific function to parse it correctly) Which means this is going to be a no-go in TD until TiVo releases a new SDK.

Looks like it could be done in Galleon, but as I've said before Java is not my thing so that's not something I could put together.

Dan

[petew]

Looks like it could be done in Galleon, but as I've said before Java is not my thing so that's not something I could put together.

Dan

Leon has already put something in place. With Galleon 2.2.0 if the file name matches:

{Title}{2005-05-04}{Episode}{04.28 PM Thu Jan 26, 2006}{BBCA}{3600}{Description}.mpg

Where the first date is the First Shown Date and the second is the recorded date, the next field is the channel call sign, followed by the duration in seconds.

Max length of a filename limits the amount of data you can put into the Titles and Description, however it's a start.

[HDTiVo]

Greg:

Would you consider extending your ground breaking program Attach to take a .tivo and stip the header, outputing an .mpg?

The reason: We now have lots of non-DRM .tivo files made with Attach. Someday, if the MAK is lost or whatever, we may need to strip the header and recover the non-DRM MPEG portion.

Thanks.

[greg_burns]

your ground breaking program Attach

:p :rolleyes:

Seriously, what's preventing you from just running the new "non-DRM" .tivo file through DSD (or perhaps even VR)? It will re-remove the header and spit out the .mpg again. Don't think it will alter the file in any other way.

Now they question becomes how does DSD handle those files if you misplaced your MAK? That is your concern, right?

Well I just tried changing my MAK in TivoDestkop and ran a "non-DRM" .tivo file through DSD and it seems to have worked regardless. (Having TivoDesktop installed and DSD installed will always be a requirement though :o )

[HDTiVo]

:p :rolleyes:

Seriously, what's preventing you from just running the new "non-DRM" .tivo file through DSD (or perhaps even VR)? It will re-remove the header and spit out the .mpg again. Don't think it will alter the file in any other way.

Now they question becomes how does DSD handle those files if you misplaced your MAK? That is your concern, right?

Well I just tried changing my MAK in TivoDestkop and ran a "non-DRM" .tivo file through DSD and it seems to have worked regardless. (Having TivoDesktop installed and DSD installed will always be a requirement though :o )
OK then, I figured the .tivo header would prevent DSD from working at all. If not, then my request is less necessary, but for the "I don't have TD anymore situation..."

[jmemmott]

OK then, I figured the .tivo header would prevent DSD from working at all. If not, then my request is less necessary, but for the "I don't have TD anymore situation..."

You should be able to simply rename one of these files from .tivo to .mpg and use any mpeg2 tools you have to work on it. MPEG2 is designed to support error recovery from a corrupted program stream by skipping the corrupted part. Because the MPEG2 part of the file is already in the clear, your editors, decoders, etc. will simply skip the header as unrecognized junk and keep going until they sync on first set of program stream headers anyway.

[Killerz298]

It is really farking redicilious that Tivo doesn't support something as simple as letting us create our own damn folders and file information. I love Tivo and all, and currently have nothing better to switch to, but come on now, stop giving up stupid updates and take care of some of the bigger requests already!

[ashu]

It is really farking redicilious that Tivo doesn't support something as simple as letting us create our own damn folders and file information. I love Tivo and all, and currently have nothing better to switch to, but come on now, stop giving up stupid updates and take care of some of the bigger requests already!

Thanks for the whiny and inaccurate post.

There is a whole slew of desired features, and you're wrong in assuming the ones you want are highest on that list. I bet a lot of folks appreciate the features being added by the new update!

[Killerz298]

Perhaps YOU should check the accuracy of YOUR post!

Even if the ability to add your own metadata isn't highest on the list, I am SURE AS HELL that the "features" currently released aren't even close. Besides TTG, Goback and MAYBE the recently deleted folder, none of the "extras" that have rolled out recently are anything great. As I said, I love Tivo and wouldn't use anything else, but there are far more desirable things (one look at the suggestions thread shows this fact) that could be added instead of what has recently been offered.

Besides TTG and Goback, could you please enlighten me as to what this "slew" of great new DESIRED (I assume you mean by Tivo owners/users) features consists of? Is it the slow as crap yahoo weather, traffic, photos and movie listings? Is it the ability to design my own Lexus ES 350 from the comfort of my own couch OH YAY! Or perhaps you are talking about Rocketboom and Cnet subscriptions which while cool for some of us, really isn't a priority in terms of what people would like to see added.

So I ask you again, to what features (as desired by the Tivo community) are you referring?

[gonzotek]

Although the official TiVo Desktop software provides neither publishing arbitrary metadata nor folders, the TiVo units themselves can support both, if the software server on a pc (or mac) supports it. Galleon, for instance, offers folders and rudimentary metadata via file naming conventions. I've inspected the source of Galleon and it is very possible to introduce a new, more advanced mechanism for metadata manipulation.

So it is inaccurate to say that TiVo doesn't support letting us add our own information, or folders, because they have put the ability in place to do this. My assumption is that the majority of normal users don't need either of these features, and spending resources developing them into the TiVo Desktop code isn't, or isn't yet, cost effective to TiVo. They have left the door open to allow other developers to provide them, however. The protocols and APIs required to do this are mostly documented in TiVo's developer resource documents and Galleon is a working, open source, proof of concept for anyone looking to develop this.

Also, some other features that I've seen people very excited about:
KidZone
One-button Delete
Recently Deleted Folder
Guru Guides

[Dan203]

So I ask you again, to what features (as desired by the Tivo community) are you referring?

I believe he's referring to the recent release od TiVo Desktop 2.3 which includes the ability to auto transfer recordings from TiVo to PC and the ability to convert those recordings to formats suitable for playback on devices like iPods and PSPs.

Dan

[Killerz298]

I believe he's referring to the recent release od TiVo Desktop 2.3 which includes the ability to auto transfer recordings from TiVo to PC and the ability to convert those recordings to formats suitable for playback on devices like iPods and PSPs.

Dan
Would be a nice addition, IF it works :-) and works better than the current methods (I find them to be simple enough and works great for me)

[Dan203]

At the very least it saves a couple of steps as it can automatically download and transcode your shows without any user interaction. I don't actually have an iPod or PSP, so I can't comment on how well it works, but from what I've read it actually has less sync issues then the other freeware solutions out there.

Dan

[id242]

I enjoy having previous night's episode (or rerun) of "The Daily Show" automatically downloaded and converted to my player's video format each morning. In the morning, the show can start to transfer to my player and when I get into work, I'll have something to catch up on during lunch and breaks - then something to BS about with my co-workers during that time.

Time-Shifting *AND* Place-Shifting is much easier because of these "stupid updates" as you choose to call them.

[saberman]

Although the official TiVo Desktop software provides neither publishing arbitrary metadata nor folders, the TiVo units themselves can support both, if the software server on a pc (or mac) supports it. Galleon, for instance, offers folders and rudimentary metadata via file naming conventions. I've inspected the source of Galleon and it is very possible to introduce a new, more advanced mechanism for metadata manipulation.


I came across this today but the thread seemed to have died months ago. I am using galleon native under Windows XP and wondered if you could provide some information as the galleon forums don't seem to.

Is there a way to get galleon to refresh its view of the TiVo relieably or even to manually force it? The only way I have found to be able to do that is to delete all of the galleon datbase and that is still only a one-time refresh.

It also leads to the next problem. Is there a way to get galleon to reprocess the header information in the .TiVo files? It appearently only stores the information once -- when it downloads the file. Deleting the database to force a refresh of the TiVo's contents also deletes the file descriptions.

[juanian]

Is there a way to get galleon to refresh its view of the TiVo relieably or even to manually force it? The only way I have found to be able to do that is to delete all of the galleon datbase and that is still only a one-time refresh.
I'm assuming that "refresh its view" means getting the latest set of recorded shows from the TiVos. As far as I know. the only way to force this kind of refresh is to stop the Galleon Server and restart it. Note that, after restarting the Galleon Server, it will still take around 10-20 minutes for Galleon to complete the refresh (depending on how many TiVos you have, and how many shows are on the TiVos).

You can set the amount of time between automatic refreshes in the Galleon Preferences. (Note that the time interval selected is the time between refreshes; the time starts after the previous refresh completes.)
It also leads to the next problem. Is there a way to get galleon to reprocess the header information in the .TiVo files? It apparently only stores the information once -- when it downloads the file. Deleting the database to force a refresh of the TiVo's contents also deletes the file descriptions.True. Also, the header information is also lost if the file is renamed (or moved to a different GoBack directory (at least this occurred with an older version of Galleon)).

[saberman]

I'm assuming that "refresh its view" means getting the latest set of recorded shows from the TiVos. As far as I know. the only way to force this kind of refresh is to stop the Galleon Server and restart it. Note that, after restarting the Galleon Server, it will still take around 10-20 minutes for Galleon to complete the refresh (depending on how many TiVos you have, and how many shows are on the TiVos).

You can set the amount of time between automatic refreshes in the Galleon Preferences. (Note that the time interval selected is the time between refreshes; the time starts after the previous refresh completes.)
True. Also, the header information is also lost if the file is renamed (or moved to a different GoBack directory (at least this occurred with an older version of Galleon)).

I am using Galleon 2.3.0 native under Windows XP Pro SP2 and neither of the methods you describe cause a refresh of galleon's view of the TiVo.

Currently, Galleon shows the last recorded show on the TiVo as being from 1/27 at 10:15 PM. The refresh is set for one hour. I think more than an hour has passed since 1/27 at 10:15 PM. (This was the point that I stopped the galleon service, deleted it's database and restarted it.)

As near as I can tell from the log it only queried the TiVo three times and then stopped. "ToGo - lastChangedDate=" appears three times about an hour apart. Then it stops.

Stopping and starting the service sometimes causes the refresh to happen and sometimes it doesn't.

While I understand that caching the .TiVo file descriptions in a database speeds the file list display it seems there should be a way to get galleon to rescan the files in its "published" directories and subdirectories especially as the auto copy to PC function does not let you specify a sub directory which means that nine times out of ten the user will be moving the file.

[juanian]

Currently, Galleon shows the last recorded show on the TiVo as being from 1/27 at 10:15 PM. The refresh is set for one hour. I think more than an hour has passed since 1/27 at 10:15 PM. (This was the point that I stopped the galleon service, deleted it's database and restarted it.)Check your log file: look for the line "DEBUG [ToGoThread] ToGoThread - downloaded.size()=". This is when the download refresh is completed. You won't see the changes on the 'ToGo/Recorded' list until this line is output. This can take over 10 minutes from the start of the service to the output of this line. (Also, the 'refresh' time cycle starts at this time.)

As near as I can tell from the log it only queried the TiVo three times and then stopped. "ToGo - lastChangedDate=" appears three times about an hour apart. Then it stops.Sometimes I have found that a TiVo just stops responding to requests for program information. When this occurs, the only solution is to reboot the TiVo. (I have 3 TiVos on my network; when I had a problem with refreshes not occurring, I found that one of the TiVos was just not responding properly, and Galleon would not complete the refresh processing for the other TiVos. I guess, over time, some things just stop working in a TiVo; I now reboot all of my TiVos every week or two to avoid this problem (and other odd user interface problems).)

Stopping and starting the service sometimes causes the refresh to happen and sometimes it doesn't.
See if the above suggestions (waiting for the "downloaded.size" line, rebooting the TiVo) work.

While I understand that caching the .TiVo file descriptions in a database speeds the file list display it seems there should be a way to get galleon to rescan the files in its "published" directories and subdirectories especially as the auto copy to PC function does not let you specify a sub directory which means that nine times out of ten the user will be moving the file.Until recently, there was no way for a "non-Tivo company" program to be able to look at the encrypted descriptions contained in a .TiVo file, so the only way to maintain the descriptions (for use during GoBack) was to remember them when the program was downloaded from the TiVo. (Although there is now an unsupported way to read the encrypted descriptions, it is not likely that this functionality will be added to Galleon anytime soon (since enhancements to Galleon have stopped).) And, I also hate the fact that the descriptions and other information are lost when moving (or renaming) a file.

One other note: I am no 'expert' on Galleon, but the one thing I use Galleon for is to download (and re-upload) programs on a regular basis, so I am fairly familiar with the oddities associated with this part of Galleon.

[CuriousMark]

Until recently, there was no way for a "non-Tivo company" program to be able to look at the encrypted descriptions contained in a .TiVo file, so the only way to maintain the descriptions (for use during GoBack) was to remember them when the program was downloaded from the TiVo. (Although there is now an unsupported way to read the encrypted descriptions, it is not likely that this functionality will be added to Galleon anytime soon (since enhancements to Galleon have stopped).
Do you have link for this?

[greg_burns]

Do you have link for this?

I assume he was referring to tivodecode, but I don't think it does that.

[saberman]

Check your log file: look for the line "DEBUG [ToGoThread] ToGoThread - downloaded.size()=". This is when the download refresh is completed. You won't see the changes on the 'ToGo/Recorded' list until this line is output. This can take over 10 minutes from the start of the service to the output of this line. (Also, the 'refresh' time cycle starts at this time.)

Sometimes I have found that a TiVo just stops responding to requests for program information. When this occurs, the only solution is to reboot the TiVo. (I have 3 TiVos on my network; when I had a problem with refreshes not occurring, I found that one of the TiVos was just not responding properly, and Galleon would not complete the refresh processing for the other TiVos. I guess, over time, some things just stop working in a TiVo; I now reboot all of my TiVos every week or two to avoid this problem (and other odd user interface problems).)

It is galleon that stops requesting the information. After a full restart at the PC galleon starts the refresh cycle with: /TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes&ItemCount=20&AnchorOffset=0

I have refresh set to one hour and my log shows this command every hour for the first ten hours. After that galleon does not request the contents of the NowPlaying list from the TiVo.

See if the above suggestions (waiting for the "downloaded.size" line, rebooting the TiVo) work..)

Actual the download.size occurs more that once for one refresh. It appears that galleon requests only twenty listings at a time so after the first twenty are received there is another request with AnchorOffset=20. I will try rebooting the TiVo but, since galleon stops making the request I am not sure what that does. It is possible that galleon gets a message that the TiVo is now ready after a reboot and starts querying it again. I only have one TiVo on the network and it is galleon that stops querying it not the TiVo stop responding.

Until recently, there was no way for a "non-Tivo company" program to be able to look at the encrypted descriptions contained in a .TiVo file, so the only way to maintain the descriptions (for use during GoBack) was to remember them when the program was downloaded from the TiVo. (Although there is now an unsupported way to read the encrypted descriptions, it is not likely that this functionality will be added to Galleon anytime soon (since enhancements to Galleon have stopped).) And, I also hate the fact that the descriptions and other information are lost when moving (or renaming) a file.

I could live without the encrypted descriptions if it would at least publish the information in the file names. For example: Boston Legal - ''Dumping Bella'' (Recorded Jan 30, 2007, WABC).TiVo shows up on the TiVo as "Boston Legal" without the descriptive material in the file name. This does not require decryption.

Galleon is a very nice program but it appears to be at alpha or beta level. The primary reason I use it is to get folder support. (The auto copying is nice but it doesn't allow you to specify a target directory and it recopies everything if you purge its database to force it to rescan directories and the TiVo.)

BTW, I had a strange experience when renaming files. Galleon continued to publish the old name to the TiVo and would supply the correct file when the old name was slected on the TiVo. The only way to fix this was to delete the galleon database which forced it to rescan the directories and use the new file names.

One other note: I am no 'expert' on Galleon, but the one thing I use Galleon for is to download (and re-upload) programs on a regular basis, so I am fairly familiar with the oddities associated with this part of Galleon.

Thanks for your help.

[juanian]

It is galleon that stops requesting the information. After a full restart at the PC galleon starts the refresh cycle with: /TiVoConnect?Command=QueryContainer&Container=%2FNowPlaying&Recurse=Yes&ItemCount=20&AnchorOffset=0

I have refresh set to one hour and my log shows this command every hour for the first ten hours. After that galleon does not request the contents of the NowPlaying list from the TiVo. I'd be curious to know if there are any ToGoThread lines in the log when Galleon stops requesting information. Specifically, I'd like to see the two "lastChangedDate" lines, including the timestamp at the beginning of the lines.

It looks like the best line to mark the start of the refresh start of the refresh can be detected by looking for the log line "DEBUG [ToGoThread] ToGoThread - tivos=". (The "... AnchorOffset=0" line occurs once per TiVo, but is close enough since you have only one TiVo.)

Actual the download.size occurs more that once for one refresh. It appears that galleon requests only twenty listings at a time so after the first twenty are received there is another request with AnchorOffset=20. I will try rebooting the TiVo but, since galleon stops making the request I am not sure what that does. It is possible that galleon gets a message that the TiVo is now ready after a reboot and starts querying it again. I only have one TiVo on the network and it is galleon that stops querying it not the TiVo stop responding.I have never seen multiple "DEBUG [ToGoThread] ToGoThread - downloaded.size()=" lines for a single refresh (although there are multiple lines like "tivo.getNumShows()=" that occur during a refresh).

Newer versions of Galleon log more problems that occur, but I don't think the current version (2.3.0) will log a specific error message if there is a problem retrieving program info from a TiVo.

I'd like to see one of the log files where the TiVo stops responding to see if there is anything 'special' that is occurring when Galleon stops requesting the refresh info. Posting it here would take up a big chunk of space in this thread (and would only be useful if others want to also look at the log file), so I guess the best thing to do is PM it to me.

Note: Galleon requests program info in chunks of 20 in order to keep the requests at a manageable size. (Some TiVos can't return more info than about 80 shows at a time, and it is possible to have hundreds (or maybe even a few thousand) shows on a particular TiVo.)

I could live without the encrypted descriptions if it would at least publish the information in the file names. For example: Boston Legal - ''Dumping Bella'' (Recorded Jan 30, 2007, WABC).TiVo shows up on the TiVo as "Boston Legal" without the descriptive material in the file name. This does not require decryption.
What version of Galleon are you using? If your version still includes quotes in the program name (like Boston Legal - "Dumping Bella" (Recorded Jan 30, 2007, WABC)), you should get a newer version. (Galleon is pretty picky about the format of the file name; the current version should read the Episode name if the quotes are not there or if the episode name is enclosed in 'single apostrophies'.)

Galleon is a very nice program but it appears to be at alpha or beta level. The primary reason I use it is to get folder support. (The auto copying is nice but it doesn't allow you to specify a target directory and it recopies everything if you purge its database to force it to rescan directories and the TiVo.)
Galleon is freeware, developed and enhanced over many years by someone (Leon Nicholls) to help TiVo users. (See TiVo Galleon Developer Calls It Quits (http://www.zatznotfunny.com/2006-02/tivo-galleon-developer-calls-it-quits/) for more info.) I would assume that, if Galleon was a 'commercial' product that people purchased, maybe a few of the oddities could be ironed out. But it is difficult for an HME application like Galleon to 'control' a TiVo, especially when there are inconsistencies and insufficient tools available to make a TiVo do what a programmer would really want it to do.

BTW, I had a strange experience when renaming files. Galleon continued to publish the old name to the TiVo and would supply the correct file when the old name was slected on the TiVo. The only way to fix this was to delete the galleon database which forced it to rescan the directories and use the new file names.That sounds odd to me, since I thought Galleon re-read the list of files in the PC directory when a GoBack list is chosen from the Now Playing list, and only associated the details to a file if it found a matching filename in its database. (I guess I can only answer this by diving into that code in Galleon.)


I'm sorry for the longwinded (and delayed) answer -- I try to err on the side of being more thorough than needing to get clarifications. :D

[juanian]

Do you have link for this?TiVo File Decode (http://tivodecode.sourceforge.net/)

[juanian]

I assume he was referring to tivodecode, but I don't think it does that.Yes, I was referring to TiVoDecode (and its ability to decrypt a .TiVo file if it is given the MAK associated with the file). I would expect that the 'technologies' used also allow the header to be decrypted, although I don't think TiVoDecode makes use of this. (I seem to recall it is a different encryption, but I think it has been done.)

[broered]

What you have done so far is very valuable. It would be nice if, for example, VideoRedo just added the option to preserve the data and output to a .tivo extension.

several messages with Video ReDo tech support indicates they want nothing to do with the law suit that might result from tampering with the TiVo Metadata. Too bad, since TiVo isn't one of the best customer oriented organizations out there.

Wish there were a way to read a .tivo file display, edit, and re-write the metadata back to the same .tivo file.
think of the benefits:
correction of the run-length after removal of commercials
ability to folder-ize related shows.
and on and on....

[id242]

broered, pyTivo with MetaGenerator will do that.
http://pytivo.sourceforge.net/forum/metagenerator-release-notes-locked-t11.html

but then again, you've just replied to a post that was at least 3 years old and a lot has changed since 2007.

[wmcbrine]

What pyTivo does is to provide the metadata via XML, as the TiVo browses/requests the files. This is slightly different from messing with .TiVo files, although I don't see why that would be lawsuit territory (any more than the existing decryption functions).

Also, you don't need MetaGenerator -- pyTivo's metadata comes in simple text files. There are several programs available that will do semi-automatic lookups, like MetaGenerator, but can also just use a text editor to make the files.

Looking back through the last few (old!) messages: tivodecode does decrypt metadata now, and I believe there's even a patch for it that allows you to create a .TiVo file header from XML, to which you can apparently append an unencrypted MPEG program stream.

[id242]

woah, Thanks wmcbrine!

I've only ever used your fork of pyTivo with a txt file, i didnt know it also used XML files.

For example, here is the txt file for "The Good, The Bad And The Ugly [1966].avi.txt" inside a "Sergio Leone" directory... as listed on my TiVoHD
title : The Good, The Bad And The Ugly
movieYear : 1966
description : Scoundrels are competing for 200,000 dollars hidden in a grave. The third Italian-Western film of a trilogy from director Sergio Leone gives Western films a new (bloody) color.
isEpisode : false
vDirector : Sergio Leone
vActor : Eli Wallach
vActor : Clint Eastwood
vActor : Lee Van Cleef
vActor : Aldo Giuffr
vActor : Luigi Pistilli
vActor : Rada Rassimov
vActor : John Bartha
vActor : Livio Lorenzon
vActor : Antonio Casale
vActor : Angelo Novi
vActor : Jos Terrn

[Dan203]

several messages with Video ReDo tech support indicates they want nothing to do with the law suit that might result from tampering with the TiVo Metadata. Too bad, since TiVo isn't one of the best customer oriented organizations out there.

We do actually have the ability to preserve the metdata like that user requested, you just have to open a .tivo file, edit it, and save using the TiVo output option. We're able to do this by simply copying the header to a new file which is completely legal. What we can't do is actually decrypt the data to modify it or use it in any fashion. We have talked to TiVo several times about getting some sort of license to be able to do this, but they've never come through.

Dan

[wmcbrine]

I've only ever used your fork of pyTivo with a txt file, i didnt know it also used XML files.It doesn't (although that's something I'm looking at). What I'm saying is that XML (plain, unencrypted) is the mechanism by which the metadata is transferred to the TiVo. But it's rendered in memory and passed to the network, never saved as a file.

[bkdtv]

We do actually have the ability to preserve the metdata like that user requested, you just have to open a .tivo file, edit it, and save using the TiVo output option. We're able to do this by simply copying the header to a new file which is completely legal. What we can't do is actually decrypt the data to modify it or use it in any fashion. We have talked to TiVo several times about getting some sort of license to be able to do this, but they've never come through.Might you consider adding a command line parameter to allow copying of a TiVo file header to a new MPG file?

I think VideoRedo is excellent, but I don't find its built-in commercial detection reliable enough for my purposes. The latest version of comskip works much better for me to remove commercials, but then I have to manually run TiVoAttach which lacks CLI parameter support.

[Dan203]

No, our attach code is very embedded into our input/output routines. There is no way for us to break it out into a separate function. However it should be relatively easy for Greg to add CLI parameters to TiVo Attach, you should hit him up.

Dan

[greg_burns]

Uh, I would whip it out in an hour if I could find my source code. This was so long ago... :P

Let me look around.

May be time to dig out Reflector (http://www.red-gate.com/products/reflector/).