TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

pyTivo - Transcoding server

2M views 6K replies 596 participants last post by  mlippert 
#1 ·
pyTivo lets you stream most videos from your PC to your unhacked TiVo. It uses the HMO server protocol, so no hacking of the TiVo is required.

* It will guess if your video is 4:3 or 16:9 and pad your video if it thinks it is needed (looked good on my tests).
* It will not transcode an mpeg that is supported by your TiVo.
* It will stream TiVo recordings (.tivo).

http://pytivo.armooo.net/

* Master changes
* Subfolder changes
 

Attachments

#1,627 ·
I just installed pyTivo. It looks great. I have about 250 movies in a directory (avg size 5 gig). They show up on my tivo (Series 3) but it takes a long time to see the list (about 30 seconds) and then every page down is another 30 seconds.

pyTivo is running on Vista the movies are on a share (external hard drive). I can access the share very quickly from Vista.

What is making the listing of the movies so slow? Can I speed it up? Thanks in advance.
 
#1,628 ·
Is there a way to use the native height and width of the input file as the height and width of the output file, or does every output file has to have the height/width set in the config file?
Some of us are already doing this by setting the ffmpeg_prams as described here. It works great (for the HD). Much less server CPU for the transcoding and the HD does scaling. But the S2's can't handle this.

tlc
 
#1,630 ·
MasterCephus - one thing that you could do is allow file to episode matching.
I.E display two columns (one with episodes and one with files) and use <=> type commands to match them together.

No need to guess the naming conventions and it streamlines the process.

Of course, I'm not a developer so this is pie-in-the-sky theorizing.

If nothing else, i'd like to request remembering the last folder location and/or the file extension.
Or, just prompt for an output name for each items as you write it. Kind of what you do now. Ask for a file to match, change the extension to .txt and write that name. For a 13 episode season, you would prompt 13 times, but right now, using season stuff you would need to rename 13 files, and with the older by episode thing, it needs 5 clicks per episode or 65 total clicks. This would be only 13 clicks

phil
 
#1,631 ·
.............I've added options to:

- keep original height/width
- keep original framerate
- keep original audio sampling rate

All of these things should save CPU time on transcoding, and should probably result in a smaller video file being sent to/stored on the tivo.............
I considered also adding an option to check the bitrate of the original audio stream and try to intelligently set the bitrate of the new audio stream to match ..............
A good idea for optimizing settings for S3/HD tivos, along with others that have been posted including wgw's post #1588 and another post for ffmpeg parameters that didn't specify the video bitrate at all, rather a quality goal.

What we need is an algorithm (i.e., a function) for optimally setting ffmpeg parameters. Arriving at a consensus of how this should work may not be easy, but some suggested general guidelines are:

1. For s2 (SD) tivos, use the old pre-HD method. (pyTivo knows whether it's s2 or s3/HD by the TSN, which it gets whether you have a per-tivo section or not.)

2. Compute an intelligent s3/HD default parameter set that has a high probability of giving a fairly good result (good compromise between quality and encoding/transfer time) even if the user chooses not to configure anything.

3. Provide "bounded" configuration where the intelligent settings from (2.) are constrained by max/min limits set by the user (e.g., for bitrates, size, fps, others as appropriate).

4. Allow the user to completely determine some parameters. For some parameters such as bitrates this could be done with the bounded configuration by just choosing min and max bitrate to be equal.

The basic principle is to provide good performance and ease of use for the user who knows nothing about encoding but allow increasing optimization control in proportion to how much the user knows and is willing to control.

Even just formulating guidelines like these is controversial, so just consider these food for thought. Arriving at a consensus will probably be a messy process but a good solution will be very worthwhile.
 
#1,632 ·
What is making the listing of the movies so slow? Can I speed it up? Thanks in advance.
It should go faster after the initial view of every folder. There is a cache built into pyTivo so as long as you are leaving it running you should notice speed increases the second and subsequent times viewing that folder.

The reason for this is: When you select a folder for the first time pyTivo has to run each file in that folder through the ffmpeg process to determine of the file is a compatible video. The results of this test are stored in the cache so that subsequent requests for that folder do not require ffmpeg calls. While it is slow the first time, it ensures that only valid videos are displayed on your TiVo.

Kevin
 
#1,633 ·
FYI,This may lead to a subfolder fix on the TiVo! tlc
Cool I am always excited for any bug fixes that TiVo does.

This sounds similar to the pyTivo shares replacing the NPL list. Even the solution is the same as what we came up with. This has been present even before the 8.3 bug, but hopefully fixing this will get rid of the subfolders issue.

Thanks for the heads up!
 
#1,634 ·
What we need is an algorithm (i.e., a function) for optimally setting ffmpeg parameters. Arriving at a consensus of how this should work may not be easy, but some suggested general guidelines are:
I like the idea. But I agree it will certainly be a messy process. I am a little behind on the HD front in pyTivo, but I am working on catching up as quickly as possible.
 
#1,635 ·
I have no problems there, but we might need to add some things in there like what's new in the release, a link to previous versions, etc. Is there somewhere in Tracs where Armoo is posting the newest features in releases?
Yes, my example was only to show that the wiki can be used to display any number of things. The wiki has come a long way, but it will always need constant edits from everyone. Maintaining the wiki is just too much for one person to do.

Well you can look at the changelog and determine what was fixed or added in each release. Most releases do not add new features but fix bugs that have been found.
 
#1,637 ·
Interestingly enough, I just discovered pytivo the other day, and wanted the same thing. There weren't to be any options to do this (at least in the code branches i looked at) so i patched pytivo to do this.

I've added options to:

- keep original height/width
- keep original framerate
- keep original audio sampling rate

All of these things should save CPU time on transcoding, and should probably result in a smaller video file being sent to/stored on the tivo. on my tivoHD the limited selection of files i've sent this way have all worked, with framerates 25fps and 29.97fps, audio sample rates from 32KHz to 48KHz, and sizes 512x384, 640x480, and 960x528.

email me (danno umich edu) if you want my patches immediately; i'll work to get them submitted to the main distribution.

I considered also adding an option to check the bitrate of the original audio stream and try to intelligently set the bitrate of the new audio stream to match (you probaby want a bit more on the target, since mp2 is less efficient than other codecs), but i decided not to bother. This would save a little space but probably not any cpu time; might actually cost more cpu time.
Some of us are already doing this by setting the ffmpeg_prams as described here. It works great (for the HD). Much less server CPU for the transcoding and the HD does scaling. But the S2's can't handle this.

tlc
Thanks to both of you for responses! I will try tlc's suggestion with the ffmpeg_pram. Since the S2 still has the annoying bug where it squeezes widescreen video and adds black bars on top and bottom, I don't use it for TTCB any longer. I wonder if TiVo will fix that...
 
#1,639 ·
If somehow you got metadata from that program and files from it, I wrote a quick VB program to turn the meta data into pyTivo NPL metadata. I will distribute the source, since it needs a quick config and I dont wanna build a full vb exe generator install kit.

There is alot more data available in the meta files than is currently in the .txt files.

Does anyone have a list of keywords that PYTIVO allows in the metadata .txt files when showing in the NPL? Or are we using all of them allowed so far?

And can the PYTIVO .txt file take genre's in numbers, not words, like

vProgramGenre : 34

instead of

vProgramGenre : Comedy

Since the Tivo uses multiple genres in a numeric format. BTW, I do have a list of all Tivo Genre numbers if anyone wants it


And is there a way in those metafiles for PYTIVO to show Original Airdate, and this showing airdate? Both is available.
 
#1,641 ·
For those using MetaGenerator for entire seasons here is a good free batch renamer:
http://www3.telus.net/pfrank/

This way you can rename 1x1.txt -1x22.txt to match your desired format
Thanks KRKeegan. After messing with that PFrank program I got it to work. I just had to make the search pattern .txt and have my desired name replace it ie... .avi.txt . That did it.
 
#1,642 ·
#1,643 ·
(pyTivo knows whether it's s2 or s3/HD by the TSN, which it gets whether you have a per-tivo section or not.)
Doh! How did I miss that. That makes my HDtivosonly option redundant. I was a bit hesitant to add it but it sounded like a good idea at the time. I'll remove that setting from my mod unless someone can think of a good reason to leave it in. Let me know if you noticed any other fubars in my mod.

2. Compute an intelligent s3/HD default parameter set that has a high probability of giving a fairly good result (good compromise between quality and encoding/transfer time) even if the user chooses not to configure anything.
Along those lines, I've been wondering if a string like one of these should be added to the default S3/HD ffmpeg template to improve quality.

-b_strategy 1 -bf 2 -coder vlc -mbd rd -qmin 1 (submitted by ScottE)
or
-mbd rd -flags +trell -mv0 -cgop -cmp 2 -subcmp 2 (from WinFF)

3. Provide "bounded" configuration where the intelligent settings from (2.) are constrained by max/min limits set by the user (e.g., for bitrates, size, fps, others as appropriate).
Hmmm, food for thought. Specifying min/max parameters in the conf like this would allow for fewer parameter lines needing to be specified in the config.

audio_br= 192k, 448k
video_br= 4096k, 17408k

4. Allow the user to completely determine some parameters. For some parameters such as bitrates this could be done with the bounded configuration by just choosing min and max bitrate to be equal.

The basic principle is to provide good performance and ease of use for the user who knows nothing about encoding but allow increasing optimization control in proportion to how much the user knows and is willing to control.

Even just formulating guidelines like these is controversial, so just consider these food for thought. Arriving at a consensus will probably be a messy process but a good solution will be very worthwhile.
Everyone sure has come up a lot of good ideas for consideration in the last 24 hours.
 
#1,644 ·
Would it be possible to add support for .tmf files to pyTivo. They are native TiVo ty streams that are tar'd and have metadata in XML format added to them.
I SECOND THAT! But we all know where TMF files came from...Ahem.

Use TY files and the TXT files and run my program to put back the metadata

phil
 
#1,645 ·
If somehow you got metadata from that program and files from it, I wrote a quick VB program to turn the meta data into pyTivo NPL metadata. I will distribute the source, since it needs a quick config and I dont wanna build a full vb exe generator install kit.

There is alot more data available in the meta files than is currently in the .txt files.

Does anyone have a list of keywords that PYTIVO allows in the metadata .txt files when showing in the NPL? Or are we using all of them allowed so far?

And can the PYTIVO .txt file take genre's in numbers, not words, like

vProgramGenre : 34

instead of

vProgramGenre : Comedy

Since the Tivo uses multiple genres in a numeric format. BTW, I do have a list of all Tivo Genre numbers if anyone wants it

And is there a way in those metafiles for PYTIVO to show Original Airdate, and this showing airdate? Both is available.
maybe I am a little slow tonight, but what does your application do?
 
#1,648 ·
I don't think any of the developer have access to such a file. If anyone who uses these comes up with a solution to this I would consider doing it.
I can put up a tmf file for you if you need one to look at.

http://www.dvrpedia.com/TiVo_Recording_File_Formats

TMF files are Ty files that were extracted from a TiVo using MFS_FTP. The biggest differences is that it is a TAR ball that contains several different files. The files it contains are the individual TY chunks as they were stored together on the TiVo. It also contains a file with the show metadata for the recording.
 
#1,649 ·
Does anyone have a list of keywords that PYTIVO allows in the metadata .txt files when showing in the NPL?
.
.
.
And is there a way in those metafiles for PYTIVO to show Original Airdate, and this showing airdate?
http://pytivo.armooo.net/wiki/MetaData

originalAirDate : <value>
time : <value>

Speaking of "recorded date" (time)...

MasterCephus - Any chance of adding an option to your MetaGenerator program for setting the "time" value to the originalAirDate value? I think this would help when sorting by recorded date in the NPL (when transferred to the TiVo).
 
#1,650 ·
maybe I am a little slow tonight, but what does your application do?
I have txt file metafdat and ty files for video.

It takes the metadata in the txt files and turns it into the pytivo txt files
to hold the orig metadata.

Just a read and convert.
 
#1,651 ·
http://pytivo.armooo.net/wiki/MetaData

originalAirDate : <value>
time : <value>

Speaking of "recorded date" (time)...

MasterCephus - Any chance of adding an option to your MetaGenerator program for setting the "time" value to the originalAirDate value? I think this would help when sorting by recorded date in the NPL (when transferred to the TiVo).
Thanks

Your origairdate change is a bad idea.

I have shows that have orig air dates of 1964, and earlier. They would always sort low, even though I recorded them this week!!!!!

That is why we need both dates. OrigAirdate - 1964/mm/dd and showing date, for when it was recorded.
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top