TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Main TiVo Forums > TiVo Home Media Features & TiVoToGo
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 03-06-2007, 11:44 PM   #91
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Proposed code mods

I've modified video.py and transcode.py (from Version 167) with two goals in mind:

1. Adjust estimated file size properly for TiVo-compatible mpeg2 files that will be transfered without re-encoding. The actual bitrate reported by ffmpeg is used in the estimated size computation for those files. (For re-encoded files the already existing computation continues to be used.) Without this fix, a mpeg with bitrate of around 6000 kbps (e.g., TiVo Best Quality) would not transfer completely, and lower bitrate files (e.g., TiVo Medium Quality) would have greatly overestimated file sizes.

2. Make the video_info() function, which parses the data reported by ffmpeg, more robust to variations of format due to either ffmpeg version used or type of file. This has been tested on the pyTivo-distributed ffmpeg and on the TiVo.Net ffmpeg and on several different files.

Code mods are as follows:

In transcode.py, the tivo_compatable() and video_info() functions were modified as follows:

Code:
def tivo_compatable(inFile):
    suportedModes = [[720, 480], [704, 480], [544, 480], [480, 480], [352, 480]]
    type, width, height, fps, millisecs, bitrate =  video_info(inFile)
    #print type, width, height, fps, millisecs, bitrate

    if not type == 'mpeg2video':
        #print 'Not Tivo Codec'
        return 0

    if not fps == '29.97':
        #print 'Not Tivo fps'
        return 0

    for mode in suportedModes:
        if (mode[0], mode[1]) == (width, height):
            #print 'Is TiVo!'
            return bitrate
        #print 'Not Tivo dimensions'
    return 0

def video_info(inFile):
    cmd = [FFMPEG, '-i', inFile ] 
    ffmpeg = subprocess.Popen(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
    output = ffmpeg.stderr.read()
    durre = re.compile(r'.*Duration: (.{2}):(.{2}):(.{2})\.(.), (.+),.*bitrate: (\d+)')
    d = durre.search(output)

    rezre = re.compile(r'.*Video: ([^,]+),.*')
    x = rezre.search(output)
    if x:
        codec = x.group(1)
    else:
        return None, None, None, None, None, None

    rezre = re.compile(r'.*Video: .+, (\d+)x(\d+),.*')
    x = rezre.search(output)
    if x:
        width = int(x.group(1))
        height = int(x.group(2))
    else:
        return None, None, None, None, None, None

    rezre = re.compile(r'.*Video: .+, (.+) fps.*')
    x = rezre.search(output)
    if x:
        fps = x.group(1)
    else:
        return None, None, None, None, None, None

    millisecs = ((int(d.group(1))*3600) + (int(d.group(2))*60) + int(d.group(3)))*1000 + (int(d.group(4))*100)
    return codec, width, height, fps, millisecs, int(d.group(6))
The video_info() function uses separate regular-expression searches for video parameters where a single search was used before. It also returns an additional (6th) parameter, which is the actual bitrate.

The tivo_compatable() function no longer returns True or False. It returns 0 if the file is not TiVo compatible but returns the actual bitrate otherwise.

The select_aspect() function also needed a trivial modification because of the extra parameter returned by video_info ().


In video.py, the est_size() function is modified as follows:
Code:
        def est_size(file):
            full_path = os.path.join(path, file)
            bitrate = transcode.tivo_compatable(full_path)
           #Size is estimated by taking audio and video bit rate adding 2%
            if bitrate > 0:  # Is TiVo compatible mpeg2
              return int((duration(file)/1000)*((bitrate * 1.02 * 1000)/8))
            else:  # Must be re-encoded
	      return int((duration(file)/1000)*((4288 * 1.02 * 1000)/8))
Thus if the file is TiVo compatible, the bitrate returned by tivo_compatable will be greater than zero and it will be used instead of the constant 4288 in the computation.

This tests fine in my limited testing so far but I am definitely not an experienced Python programmer so .......
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-07-2007, 09:36 PM   #92
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Verified my code mods (previous post) work with two additional versions of ffmpeg: 7760 and 8047.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-07-2007, 10:22 PM   #93
KRKeegan
Im lost and confused
 
KRKeegan's Avatar
 
Join Date: Jul 2004
Location: Los Angeles, CA
Posts: 215
Quote:
Originally Posted by dlfl
Verified my code mods (previous post) work with two additional versions of ffmpeg: 7760 and 8047.
Sorry I have been busy the past few days.

Thank you for the help though. All of you modifications seem good, except the file size of TiVo Compatible files. There is nothing wrong with it, in fact the way you handled it is very ingenius. However, since we are just sending a whole file the best way to handle this would just be to get the exact file size and use that. This means TiVo doesnt have to get into that whole 206 issue. Not a big deal, but probably the more proper thing to do.

I will add your changes tonight and hopefully change the TiVo compatible file size too.

Thanks for you patience everyone. And you exhaustive testing
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A much better place to receive pyTivo help and updates.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A python real time video transcoding server for TiVo.
KRKeegan is offline   Reply With Quote
Old 03-08-2007, 01:05 AM   #94
KRKeegan
Im lost and confused
 
KRKeegan's Avatar
 
Join Date: Jul 2004
Location: Los Angeles, CA
Posts: 215
New Version 168

ok again another minor update to fix some small bugs.

New in this version:
- More versions of ffmpeg output supported
- TiVo compatible files are reported to TiVo as actual size.

Thanks to dlfl for the bug testing and solutions.

I made a change to your file size solution though. As previously discussed I report TiVo compatible file sizes as there actual file size. This circumvents any need to get into that whole 206 area. And it is the proper way to transfer a non-transcoded file. Who knows it may prevent issues in the future.

Future planned minor update(maybe tomorrow)
Transfer of .tivo files. Since ffmpeg can't be used to verify correct format we will just have to assume that all ".tivo" files are compatible. A reasonable assumption.

Alright night all.

Kevin
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A much better place to receive pyTivo help and updates.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A python real time video transcoding server for TiVo.
KRKeegan is offline   Reply With Quote
Old 03-08-2007, 01:11 AM   #95
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Quote:
Originally Posted by KRKeegan
.......... since we are just sending a whole file the best way to handle this would just be to get the exact file size and use that. ....
Yeah, I thought of that first but being the newbie to Python that I am, I couldn't figure out how to get the file size ! That is obviously the best way.

All I could think of was an fseek, ftell sequence. I'm guessing there is a better way to get the file size.

I don't know what's going on with Pipakin..... I've kind of gotten to like pyTivo anyway. All I do is start it with a shortcut and it's off and running -- actually simpler than TiVo.Net and serves my purposes fine.

I do prefer the ffmpeg that Tivo.Net uses ... I think it's significantly better than the current one. It handles wmv3 video (encoded with WMV9 codec). It is build 7215 and the Windows version is downloadable from this web page -- it is the one named ffmpeg-SVN-r7215-static-gpl-win32.zip Of course it is just called ffmpeg.exe in the zip. If you run "ffmpeg -i dud" you get all the build information and it is identical for this one and the TiVo.Net one, including the build date/time right down to the second. I just tested by substituting and it worked fine although I only tested it on one WMV3 video. I've tested the Tivo.Net one on a wide variety of formats.

Glad to see you back "on the job" again.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-08-2007, 01:49 AM   #96
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Quote:
Originally Posted by KRKeegan
ok again another minor update to fix some small bugs.

New in this version:
- More versions of ffmpeg output supported
- TiVo compatible files are reported to TiVo as actual size.

Thanks to dlfl for the bug testing and solutions.

I made a change to your file size solution though. As previously discussed I report TiVo compatible file sizes as there actual file size. This circumvents any need to get into that whole 206 area. And it is the proper way to transfer a non-transcoded file. Who knows it may prevent issues in the future.

Future planned minor update(maybe tomorrow)
Transfer of .tivo files. Since ffmpeg can't be used to verify correct format we will just have to assume that all ".tivo" files are compatible. A reasonable assumption.

Alright night all.

Kevin
Just tested 168 on one TiVo compatible and one WMV. Worked fine! I did notice some indenting irregularities in est_size () and corrected them before running. In this case the logic would work the same whether the last return statement was indented or not I think, i.e., whether it was in the else: clause. The first return was only indented 3 spaces (all other indents are 4) and I don't know enough about Python to know how important that might be so just made both indents 4 spaces.

Your plans to handle .tivo sound good.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-08-2007, 01:24 PM   #97
sabu
Registered User
 
Join Date: Jan 2002
Location: Harrisburg, PA
Posts: 17
Machine Requirements for running pyTivo

Is there a minimum machine spec required for running pyTivo?

Will an older, slower machine still work but just have the transfer take longer because the transcoding takes longer or could it be too slow and timeout the process?
sabu is offline   Reply With Quote
Old 03-08-2007, 02:36 PM   #98
KRKeegan
Im lost and confused
 
KRKeegan's Avatar
 
Join Date: Jul 2004
Location: Los Angeles, CA
Posts: 215
Quote:
Originally Posted by sabu
Is there a minimum machine spec required for running pyTivo?

Will an older, slower machine still work but just have the transfer take longer because the transcoding takes longer or could it be too slow and timeout the process?
Exactly, as long as you can run python2.4 and ffmpeg, which is I think just about any machine, you can run pyTivo.

You will however not be able to watch your videos while they stream live however.

It might be nice for people to post their Network, CPU speeds and the rate at which pyTivo is able to transfer an average file??

For example I have a 2.8Ghz processor, running 100Mbps wired ethernet and I can stream an average downloaded TV show at about 125% of real time.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A much better place to receive pyTivo help and updates.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A python real time video transcoding server for TiVo.
KRKeegan is offline   Reply With Quote
Old 03-08-2007, 07:27 PM   #99
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
pyTivo faster thanTiVo.Net ?

Just subjective because I haven't done any careful measurements, but it seems like pyTivo can transfer at a higher encoding bitrate than TiVo.Net. I have two PC's networked to my TiVo via a router:

1. 3 GHz Pentium 4, hyperthreaded, wired ethernet
2. 2.8 GHz Pentium 4, wireless ethernet
The TiVo is wired to the router.

With TDN, I limited bitrates to 3 kbps on #1 and 2 kbps on #2 in order to transfer slightly greater than real-time. pyTivo always encodes at 4 kbps and it has no trouble keeping ahead of real-time on either PC.

Anyone else get this impression?
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-08-2007, 08:07 PM   #100
ocntscha
Registered User
 
Join Date: Oct 2003
Posts: 148
Quote:
Originally Posted by dlfl
Just subjective because I haven't done any careful measurements, but it seems like pyTivo can transfer at a higher encoding bitrate than TiVo.Net. I have two PC's networked to my TiVo via a router:

1. 3 GHz Pentium 4, hyperthreaded, wired ethernet
2. 2.8 GHz Pentium 4, wireless ethernet
The TiVo is wired to the router.

With TDN, I limited bitrates to 3 kbps on #1 and 2 kbps on #2 in order to transfer slightly greater than real-time. pyTivo always encodes at 4 kbps and it has no trouble keeping ahead of real-time on either PC.

Anyone else get this impression?
Since you asked, I run Tivo.Net on a 2Ghz Athlon XP, Centos Linux. I've settled on leaving ffmpeg configured at 4096/kbs which gives me slightly faster than real time.
ocntscha is offline   Reply With Quote
Old 03-08-2007, 11:34 PM   #101
sabu
Registered User
 
Join Date: Jan 2002
Location: Harrisburg, PA
Posts: 17
Performance Stats

With a PIII-850 - dual processor (ffmpeg appears to be running single threaded though) and a 100MB ethernet connection, I get a 1 hour show transcoded from Xvid and transferred to the TiVo (unhacked SA Dual Tuner) in about 2.5 hours. I didn't change the bitrate or any of the other settings from the initial installed values.

I had been thinking about putting it on my file server, but I think the PII-333 in that would have to run overnight for a transfer. I may try it just for fun over the weekend.
sabu is offline   Reply With Quote
Old 03-09-2007, 12:41 AM   #102
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
"Native" TiVo proposed mod

Tested simple mods to transfer .TiVo files back. They seem to be OK:

Two mods, both in transcode.py:

1. Added two lines at top of tivo_compatable ():

Code:
def tivo_compatable(inFile):
    if (inFile[-5:]).lower() == '.tivo':
        return True
2. Added two lines at top of video_info ():

Code:
def video_info(inFile):
    if (inFile[-5:]).lower() == '.tivo':
        return True, True, True, True, True
Before I added the 2nd mod, there were times when it would scan a folder with a .tivo in it and ffmpeg would take off running and never stop.

This approach doesn't feed TiVo a duration (although it does send the file size) so all it has to display is the duration of the original program from the meta-data, I presume. Thus if the file is a small clip it will not show the true duration in the info display until after it has transfered completely and you look back in the NPL. I don't see this as much of a problem. That's the only difference I could detect compared to transfering with the TiVo beacon.

See what you think --- hope I haven't built in a bug with this.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-09-2007, 03:11 AM   #103
KRKeegan
Im lost and confused
 
KRKeegan's Avatar
 
Join Date: Jul 2004
Location: Los Angeles, CA
Posts: 215
Update thanks to dlfl

VERSION 171
Quote:
Originally Posted by dlfl
See what you think --- hope I haven't built in a bug with this.
No issues, you did a great job!

New Feature
- Thanks to dlfl pyTivo will now transfer TiVo encrypted files as well.

Known Issues:
- While transfering the green play bar at the bottom of the screen will be solid green and the duration will just get longer and longer, instead of the normal thermometer effect. This is because ffmpeg is unable to read these files and cannot determine the duration. But funcationally everything else works great.

dlfl
Thanks again for the update, you actually solved a bug that hadn't come up yet. Specifically on windows ffmpeg could get stuck on ".tivo" files. This may also suggest the need to use a mask of known ffmpeg recognized file types. It is possible that there are other file types which could equally cause ffmpeg to hiccup.

Where are versions 169 and 170?
Our ever gracious benefactor, Armooo, went through and fixed my tab mistake for me , thank you! And made an alteration to the music plugin. He is like underwear gnome of coding, secretly fixing bugs in the night.

Step 1: Steal Underwear
Step 2: - - - - - - -
Step 3: Profit!!
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A much better place to receive pyTivo help and updates.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A python real time video transcoding server for TiVo.
KRKeegan is offline   Reply With Quote
Old 03-09-2007, 09:26 AM   #104
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Masking for ffmpeg-recognized file types

Quote:
Originally Posted by KRKeegan
........ This may also suggest the need to use a mask of known ffmpeg recognized file types. It is possible that there are other file types which could equally cause ffmpeg to hiccup. ........
What frequency are you tuned to? We must be on the same wavelength ... I had exactly the same thought overnight!
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-10-2007, 05:56 PM   #105
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Bullet-proofing ffmpeg calls (??)

I've tested a code mod to transcode.py based on the following idea:

In video_info(), launch the ffmpeg cmd in the shell (i.e., non-blocking mode). Then wait for a maximum time (e.g., 4 secs) while polling the ffmpeg subprocess to see if it is done. If it isn't done after the max time, kill the subprocess and return None's. If it finishes before the 4 secs, stop waiting and proceed as before.

Here's the modified code:
Code:
    cmd = [FFMPEG, '-i', inFile ] 
    ffmpeg = subprocess.Popen(cmd, shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
    for i in range(80):
        time.sleep(.05)
        if not ffmpeg.poll() == None:
            break
#    print 'ffmpeg time = ', (i*.05), ' for ', inFile
    if ffmpeg.poll() == None:
        if mswindows:
            win32kill(ffmpeg.pid)
        else:
            import os, signal
            os.kill(ffmpeg.pid, signal.SIGKILL)
#        print 'killed ffmpeg'
        return None, None, None, None, None
    output = ffmpeg.stderr.read()
#    print output
Also, an "import time" statement was added near the top of the file.

Note the commented out debug print statements I used while testing.

I learned several things from testing:
1. The times reported for ffmpeg to run averaged about .05 sec.

2. I had one .tivo file that I knew locked up ffmpeg with the old code. So I commented out the code recently added to declare .tivo's OK just based on the file extension. Amazingly, with the new code it didn't lock up ffmpeg -- the kill clause never executed, i.e., just running it in a non-blocking process apparently made that difference. FYI, The output text returned from ffmpeg for this .tivo and another one was:
Quote:
'C:\Program' is not recognized as an internal or external command, operable program or batch file.


3, Since no file was hanging ffmpeg, I had to make sure the polling and kill code actually worked. So I commented out the for loop with the sleep delays in it. Then the ffmpeg process was indeed killled for every file before it could complete, and nothing went onto the pyTivo NPL. I also verified that the info returned for non-Tivo files was still valid, i.e., identical to what it had been with the old code for these same files.

Thus, I think this code is bullet-proof in the sense that the call to ffmpeg to get info cannot hang the program or leave ffmpeg running forever regardless of what "video" file is passed to ffmpeg.

Obviously this needs to be tested on Linux since the changes involve os functions.

Filtering based on file extensions may still make sense and definitely should be done if this mod is not adopted.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-11-2007, 03:05 AM   #106
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
proposed ffmpeg patch

I had a mpeg file that I knew should be TiVo-compatible which started re-encoding when transfered. It had been edited with VideoReDo. The ffmpeg version I use (build 7215, same as Tivo.Net) reported the fps as 59.94. However there is a line in the ffmpeg return info something like:
"Appears to be film source: 29.97..."
The proposed patch allows the fps to be corrected back to 29.97 if this line is found.

The patch is a few lines in the video_info() function in transcode.py. I have repeated the existing fps reading code just for context:
Code:
    rezre = re.compile(r'.*Video: .+, (.+) fps.*')
    x = rezre.search(output)
    if x:
        fps = x.group(1)
    else:
        return None, None, None, None, None

    # fps can come back 59.94 on VRD-edited tivo's with ffmpeg build 7215 (TDN version)
    #  but in this case there is a line that includes "...film source: 29.97..."
    if not fps == '29.97':
        rezre = re.compile(r'.*film source: 29.97.*')
        x = rezre.search(output.lower() )
        if x:
            fps = '29.97'
The version of ffmpeg currently being distributed with pyTivo does not have this problem. However the patch does no harm in that case and it helps those of us who might be using the later build (and I continue to hope the later build will become the recommended one).

Oh and BTW, the file in question played fine when transfered without re-encoding.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-11-2007, 08:55 PM   #107
ocntscha
Registered User
 
Join Date: Oct 2003
Posts: 148
I've got pyTivo going on my Linux box now. As I mentioned in a previous post the beacon was being sent out from my WAN ip address so my Tivo would never see it. I enventually came up with a solution, I changed this line in beacon.py

self.UDPSock.sendto(self.format_beacon(), ('255.255.255.255', 2190))

to this..

self.UDPSock.sendto(self.format_beacon(), ('192.168.1.255', 2190))

theorizing if I gave it the broadcast ip address of my LAN it would have to broadcast from the LAN ip address. It worked!

The web server on port 9032 is listening on both my ip addresses and the loop back interface, I'd like to just limit it to the LAN ip but haven't figured out how. Oh well, its listenting on port 9032 of my public ip but thanks to Shorewall/iptables it won't respond to anything coming in off the internet.

Anyhoo, seems pretty cool. Among other things, the biggest advantage it has over Tivo.Net is that new videos are just automatically available. Very nice.

I did have a couple issues with it though, one was a video I got off youtube, a .flv file, ffmpeg would refuse to play it, kept saying bit rate of 192 is not supported for mp2 audio. I knew I could play that in Tivo.net so I "borrowed" Pipakin's settings and got it working. Actually, the one that mattered was -ar 48000, just adding that one parameter to the ffmpeg command line lets the video transcode fine. Had another one, I think the original source was PAL and it was ending up looking horizontally squished so I started playing around with the other stuff like aspect ratio.

So, at the moment I've ended up with pretty much, if not exactly, what Tivo.Net uses.. I changed the ffmpeg line of transcode.py to this, haven't got to test a whole lot but I suspect it will do nicely, particularly since I have done a lot with Tivo.Net and have had fantastic success with it, so here's what I've got right now..

cmd = [FFMPEG, '-i', inFile, '-vcodec', 'mpeg2video', '-r', '29.97', '-b', '4096K', '-aspect', '4:3', '-s', '720x480', '-acodec', 'mp2', '-ar', '48000', '-ac', '2', '-ab', '192', '-f', 'vob', '-' ]
ocntscha is offline   Reply With Quote
Old 03-11-2007, 10:02 PM   #108
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
@ocntscha, @krkeegan

Regarding needing "-ar 48000" in ffmpeg command line:

Yep, I see the same behavior, same solution in the Windows world. Had a few .flv from google/youtube but hadn't tried them with pyTivo until now.

Can't see how this could cause problems for other files (tested only on .wmv so far). The TiVo go back specs don't say anything about audio sampling rate and 48000 is the spec for NTSC DVD.

Perhaps most relevant: If you load a TiVo recording into VideoReDo and look at the file parameters, the audio sampling rate is .... 48000 Hz.
The ffmpeg doc says default sampling rate is 44000 without a -ar option. I don't understand WHY this is a problem, but.... there it is.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Last edited by dlfl : 03-11-2007 at 10:09 PM.
dlfl is offline   Reply With Quote
Old 03-11-2007, 10:24 PM   #109
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Quote:
Originally Posted by ocntscha
I've got pyTivo going on my Linux box now. ..........
...... seems pretty cool. Among other things, the biggest advantage it has over Tivo.Net is that new videos are just automatically available. Very nice.
AND it transfers .TiVo files and TiVo-compatible MPEG2 files without re-encoding....AND one click (shortcut) gets it going ... AND ... it really does handle the end-of-transfer/file-length issue (in a way that has been fool proof in all my testing so far).

Kind of grows on you, doesn't it?
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-11-2007, 11:47 PM   #110
sabu
Registered User
 
Join Date: Jan 2002
Location: Harrisburg, PA
Posts: 17
Quote:
Originally Posted by sabu
I had been thinking about putting it on my file server, but I think the PII-333 in that would have to run overnight for a transfer. I may try it just for fun over the weekend.
Well, the PII-333 over 100Mbit Ethernet only took 6 hours to transfer a 1 hour 15 minute show. Not as bad as I thought it would be.
sabu is offline   Reply With Quote
Old 03-12-2007, 01:10 AM   #111
armooo
pyTivo Developer
 
Join Date: Feb 2003
Posts: 79
I added a bunch of the above patches and fixed one bug that should be a large speed up for anyone who's tivo dose not have a domain name. (4.5 sec per request speed up)

pyTivo
- Don't do reverse DNS lookups when logging (adds 4.5 sec to response time)
- Check for Appears to be film source: (\d+) and override fps with it
- Set an audio sample rate
- Caching video info
- Timeout for ffmpeg getting video info
armooo is offline   Reply With Quote
Old 03-12-2007, 11:51 AM   #112
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
@armooo, @krkeegan

Quote:
Originally Posted by armooo
I added a bunch of the above patches and fixed one bug that should be a large speed up for anyone who's tivo dose not have a domain name. (4.5 sec per request speed up)

pyTivo
- Don't do reverse DNS lookups when logging (adds 4.5 sec to response time)
- Check for Appears to be film source: (\d+) and override fps with it
- Set an audio sample rate
- Caching video info
- Timeout for ffmpeg getting video info
Thanks armooo ! The speedup is very welcome!

Glad to see the upgrade to build 8047 of ffmpeg too.

I think I've learned a python lesson (remember I'm a newbie to this language):
Apparently the blocking after the video_info() call of the ffmpeg subprocess comes from the ffmpeg.stderr.read () statement, not from the Popen call.

One slight problem:
"- Check for Appears to be film source: (\d+) and override fps with it" doesn't work with build 8047 -- it returns a different string than 7215, so the "...film source: 29.97" doesn't match anything.

I wonder if trying to chase this around for different builds of ffmpeg is worth it? At least for the file I've been testing with (which is 352x480) the re-encoded version has the correct aspect ratio and looks pretty good.

Also, at least on my python (windows, 2.5) the RE "(\d+)" doesn't match "29.97" because the \d doesn't match the decimal point (or period). I would have to use "(\d+)\.(\d+)" and combine the two groups. Am I missing something?

On this patch (if you decide to keep it), I still prefer gating the whole patch with
if not fps == '29.97':
because the sole purpose of the patch is to allow a chance to correct the fps to 29.97 if the usual matching returned something else. (Actually I prefer my origiinal patch, which required an explicit 29.97 match for the override.)

If you decide to keep the patch, I will be happy to supply the additional patch to make it work with build 8047 if you would like. The patches are accumulative, i.e., you don't have to remove the one for 7215 when you add the one for 8047.
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-12-2007, 01:26 PM   #113
ocntscha
Registered User
 
Join Date: Oct 2003
Posts: 148
Quote:
Originally Posted by armooo
I added a bunch of the above patches and fixed one bug that should be a large speed up for anyone who's tivo dose not have a domain name. (4.5 sec per request speed up)

pyTivo
- Don't do reverse DNS lookups when logging (adds 4.5 sec to response time)
- Check for Appears to be film source: (\d+) and override fps with it
- Set an audio sample rate
- Caching video info
- Timeout for ffmpeg getting video info
You guys rock! Getting every last little gremlin out, I love it.

I haven't actually tried this latest release but I did download and look at it. I see you've now got -ar 44100 in the ffmpeg command line. According to the ffmpeg man page 44100 is the default for that parameter anyway. Changing it to 48000 is what enables youtube .flv files to work for both dlfl and I, he details it pretty thoroughly a few posts back.

Thank you for creating pyTivo and sharing it with us.
ocntscha is offline   Reply With Quote
Old 03-12-2007, 01:44 PM   #114
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
audio sampling rate option

Quote:
Originally Posted by ocntscha
You guys rock! Getting every last little gremlin out, I love it.

I haven't actually tried this latest release but I did download and look at it. I see you've now got -ar 44100 in the ffmpeg command line. According to the ffmpeg man page 44100 is the default for that parameter anyway. Changing it to 48000 is what enables youtube .flv files to work for both dlfl and I, he details it pretty thoroughly a few posts back.

Thank you for creating pyTivo and sharing it with us.
I noticed this too, but it transcoded one of my .flv files OK with 44.1k. I wonder if the ffmpeg doc is just not correct on this -- or it's something that varies with the many builds. Still -- seems like 48k generally would be prefered since that's what TiVo files have.

This thing is really getting sweet, isn't it!
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-12-2007, 07:18 PM   #115
KRKeegan
Im lost and confused
 
KRKeegan's Avatar
 
Join Date: Jul 2004
Location: Los Angeles, CA
Posts: 215
Sorry I have been neglecting you all. And I am glad to see Armooo is more then happy to pickup the slack.

I only have one concern with the above modifications. And it is a minor one. Setting the timeout on the ffmpeg loop to .05 seconds means that even if a users computer operates faster than .05 seconds he will still be limited by this loop.

Like I said, it is only a minor concern, and probably doesn't affect anyone in any form that is noticable.

Other than that I am very excited about working out all the little bugs.

The one main bug I would still like to fix:
- Moving the LRU cache from video.py to transcode.py into the video_info() block.
OOPS: WAHOO Armooo already did this. I guess I should pay attention more. Thanks Armoo

Future Features:
- After moving the LRU cache to video_info(), add the option to scan all subdirectories on start up for video files.
- Some how add titles and descriptions and maybe other data to the videos.

What is this new dll??
Armooo, what is this new dll in the plugins/video folder?? pthreadcg2.dll??
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A much better place to receive pyTivo help and updates.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

A python real time video transcoding server for TiVo.

Last edited by KRKeegan : 03-12-2007 at 07:28 PM. Reason: oops
KRKeegan is offline   Reply With Quote
Old 03-12-2007, 09:08 PM   #116
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
New dll

Quote:
Originally Posted by KRKeegan
.......What is this new dll??
Armooo, what is this new dll in the plugins/video folder?? pthreadcg2.dll??
From having tried ffmpeg build 8047 previously, I know this dll must accompany it for it to operate. (You find out by trying to run it without the dll in the path and it tells you it can't find that dll!). IIRC, the dll is downloadable from the same site as the ffmpeg build but just up one directory.

Also: I understand your concern about the .05 sec ffmpeg wait.... who knows? It's a judgement call. The computer I tested on is no slouch (3 GHz P4, hyper-threaded) and it was averaging about .05 secs (some zeros, some .1 and an occasional .15). It was probably the cache changes Armooo made that made the difference, but my NPL list is populating MUCH faster than before -- very satisfying!
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Last edited by dlfl : 03-12-2007 at 09:14 PM.
dlfl is offline   Reply With Quote
Old 03-12-2007, 09:39 PM   #117
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
For what it's worth......

I'm going to post my proposed changes in transcode.py (starting from ver. 173) to handle the case where a tivo-compatible file has fps reported as 59.94 by ffmpeg, either Build 7215 (Tivo.Net version) or Build 8047 (current pyTivo version).

In video_info() replace:
Code:
    rezre = re.compile(r'.*film source: (\d+).*')
    x = rezre.search(output.lower())
    if x:
        fps = x.group(1)
with:
Code:
    # fps can come back 59.94 on VRD-edited tivo's with ffmpeg build 7215 (TDN version)
    #  or with build 8047 (latest pyTivo distribution) but in this case there 
    #  is a line that includes "...film source: 29.97..." for build 7215 or 
    #  one that contains "...frame rate differs from container frame rate: 29.97..."
    #  for build 8047.

    # Allow override only if it is mpeg2 and frame rate was doubled to 59.94
    if (not fps == '29.97') and (codec == 'mpeg2video'):
        # First look for the build 7215 version
        rezre = re.compile(r'.*film source: 29.97.*')
        x = rezre.search(output.lower() )
        if x:
            fps = '29.97'
        else:
            # for build 8047:
            rezre = re.compile(r'.*frame rate differs from container frame rate: 29.97.*')
            x = rezre.search(output.lower() )
            if x:
                fps = '29.97'
I'm not trying to force these on anyone and I realize mssrs. KRKeegan and Armooo may prefer to ignore or change them. I just have this strange desire to share.

Bon appetit!
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-13-2007, 10:27 AM   #118
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Installation problems ??

I posted on the Tivo.Net thread suggesting people having problems with it give pyTivo a try. So far two people (see Posts #1147 and #1151 on that thread) have responded saying they couldn't get pyTivo installed and working. I posted back suggesting they post pyTivo problems on THIS thread since I don't want to hijack the Tivo.Net thread.

BTW, is pyTivo known to work on MAC's ?
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
dlfl is offline   Reply With Quote
Old 03-13-2007, 10:31 AM   #119
Chew
Registered User
 
Chew's Avatar
 
Join Date: Jan 2003
Posts: 284
^ To answer your question from the TiVo.net thread, those three attempts to get it installed (and it failed everytime) occured just this past weekend.

I'll give it another go after work and post exactly where it gets hung up.
Chew is offline   Reply With Quote
Old 03-13-2007, 10:43 AM   #120
dlfl
Cranky old novice
 
dlfl's Avatar
 
Join Date: Jul 2006
Location: Near Dayton, Ohio
Posts: 6,069
Quote:
Originally Posted by Chew
^ To answer your question from the TiVo.net thread, those three attempts to get it installed (and it failed everytime) occured just this past weekend.

I'll give it another go after work and post exactly where it gets hung up.
Good deal. If you have problems again, please post your computer specs and a copy of the pyTivo.conf file you used. (If you didn't edit this file, that would explain pyTivo not running. The required edits are very simple however.)

You have to install python version 2.4 or later. (Why not 2.5 ?). To verify installation select the command line version from the start menu and type in something simple like: a = 'hello' and hit the return key.

I've never used the pyTivo windows installer. AFAIK there are no registry mods for pyTivo so using the Windows installer isn't important. There is a .zip file with the same version number. Just put everything in the zip file into a folder of your choice. Edit the pyTivo.conf file. Launch program by double clicking the pyTivo.py file (or a shortcut pointing to it). You may see nothing in the command window that pops up until you look at the NPL in tivo, or you may see a long warning about a Cheetah something or other not being installed -- ignore this warning.

(I've been assuming you are on a Windows system.)
__________________

"It must be swell to be so perfect and odor-free" -- Del Griffith


VideoReDo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

pyTivo users: Try
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
and
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Last edited by dlfl : 03-13-2007 at 10:51 AM.
dlfl is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 02:03 PM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |