TiVo Community Forum banner
  • TiVoCommunity.com Ambassador Program Now Open! >>> Click Here

Streambaby - new streaming application

1M views 4K replies 481 participants last post by  thewebgal 
#1 ·
I wanted to let everyone know about a new streaming video application, StreamBaby:

you can find it at:
http://code.google.com/p/streambaby/

It is originally based off of tivostream, and has the following main features:

* Random access streaming of video files. Able to seek to anywhere in the video. Also removes need for qt-faststart.
* Attempts to work around the 1.1G limit for streaming videos on the TiVo. If you reach the 1.1G limit, the video you are watching will be paused and restarted with a fresh buffer at the position you were in before.
* Ability to FF/RWD to points in the video that have not been buffered yet. When outside of the buffer, a "preview" of the movie (low-quality static images) will appear, and when play is pressed the video stream will be repositioned to start streaming at that point (ala Netflix/TiVo)
* Realtime preview generation for most file formats. Others (mpeg in particular) use autogenerated thumbnail files.
* Ability to transcode and stream video that is not inherently TiVo compatible. Supports most wmv, avi, and mkv. Preview mode and random-positioning work with most transcoded files.
* Remembers your position in the video when you stop watching and automatically starts at that position next time you watch. Works even when exiting playing via the "TiVo" button.

Please let me know what you think, or if you have any bugs/suggestions, etc.

Also I just wanted to make sure I thanked moyekj here for his work on tivostream, as well as his efforts in testing/debugging/code suggestions on Streambaby. Thanks! Thanks also to Steve C who also helped with suggestions and testing. Their efforts have hopefully helped make Streambaby a better application.

(Apologies for the repost, it was mentioned this may be a better forum for the thread)

Keary
 
See less See more
#1,550 ·
neither pyTivo nor Streambaby interpret these files correctly
I'm not seeing that here. Both LF and CR/LF endings work for me in pyTivo, under both Linux and Windows. I haven't tried old Mac style (CR-only); I wouldn't be surprised if that one failed.

I wonder if your complaining Windows users are saving as RTF or something? Windows can get very creative in how it mangles things.

I tested with Python 2.6.2 in Ubuntu and 2.5.1 in Win XP. I doubt this has changed with new Python versions. pyTivo simply relies on Python to split the file into lines (i.e., "for line in file(metadata):"), and on .strip() to remove the line endings. The metadata files are opened in text mode, but it makes no difference if I change that to open them in binary mode.
 
#1,551 ·
It massively sped up the processing (and therefore transmission) of .mkv files and other files that ffmpeg are involved in transcoding. I was wondering why "-threads 2" is the default on the pyTivoX installation (and I am assuming the StreamBaby default installation) when even on a Mac Mini (at least *my* Mini) that does not peg the processor. This ("-threads 4") at least goes 160% in the "Activity Monitor"... "-threads 2" only goes 60-90%.

Also, when I use MacPorts under Snow Leopard and compile ffmpeg with the following options:

Code:
sudo port install ffmpeg +universal +gpl +libogg +vorbis +theora +faac +faad +lame +x264 +xvid
and then add the following to the "/Users/<user>/Library/Application Support/pyTivoX/streambaby-user.ini" file:

Code:
ffmpeg.path=/opt/local/bin/ffmpeg
the speed bump is enormous--even on a Mini over just increasing the "-threads 2" option! Is there a reason to use the libraries and binaries that ship with applications such as pyTivoX?
ffmpeg docs generally say to use threads = number of processors. On my older hardware I tended to only use 2 for dual core machines, bumping it up to 4 didn't help and in some cases hurt. I haven't tested with greater than 4 on my Q6600, but I'll give it a shot and see later. The point is it will vary depending on your hardware...

As for the Snow Leopard MacPorts compile, older Apple compilers created ffmpeg binaries that would crash when the full Intel optimizations were enabled, so for a while MacPorts would disable Intel optimizations for ffmpeg. I think Apple finally fixed the compiler problem with Xcode 3.0 or 3.1 and I guess MacPorts finally caught up so what you are seeing is an ffmpeg with full Intel optimizations enabled AND the additional optimization improvements moving from Xcode 3.1 to 3.2. Not sure whether the 32 vs 64 bit has anything to do with it. Haven't looked at the MacPorts stuff in Snow Leopard yet.
 
#1,552 ·
I'm not seeing that here. Both LF and CR/LF endings work for me in pyTivo, under both Linux and Windows. I haven't tried old Mac style (CR-only); I wouldn't be surprised if that one failed.

I wonder if your complaining Windows users are saving as RTF or something? Windows can get very creative in how it mangles things.

I tested with Python 2.6.2 in Ubuntu and 2.5.1 in Win XP. I doubt this has changed with new Python versions. pyTivo simply relies on Python to split the file into lines (i.e., "for line in file(metadata):"), and on .strip() to remove the line endings. The metadata files are opened in text mode, but it makes no difference if I change that to open them in binary mode.
In testing I just confirmed that Mac EOL (CR) does mess up the pyTivo interpretation of the meta files. I will check with my users to see but they said they were using notepad.exe and I did not think notepad (mac user here--so am not sure) saved as RTF.
 
#1,553 ·
In testing I just confirmed that Mac EOL (CR) does mess up the pyTivo interpretation of the meta files. I will check with my users to see but they said they were using notepad.exe and I did not think notepad (mac user here--so am not sure) saved as RTF.
I just tested streambaby with CR, CR/LF, LF and they all seemed to work for me (at least under Linux).

The thing you need to be careful of with Notepad.exe is that it unfortunately likes to add a UTF-8 BOM marker to the beginning of text files. At least for Java, it wasn't handled automatically and at some point I had to add a bunch of code to deal with it.
 
#1,554 ·
As for the Snow Leopard MacPorts compile, older Apple compilers created ffmpeg binaries that would crash when the full Intel optimizations were enabled, so for a while MacPorts would disable Intel optimizations for ffmpeg. I think Apple finally fixed the compiler problem with Xcode 3.0 or 3.1 and I guess MacPorts finally caught up so what you are seeing is an ffmpeg with full Intel optimizations enabled AND the additional optimization improvements moving from Xcode 3.1 to 3.2. Not sure whether the 32 vs 64 bit has anything to do with it. Haven't looked at the MacPorts stuff in Snow Leopard yet.
In examining the resulting ffmpeg with x264 I am seeing all optimizations including the following during an encode. This would indicate the speed improvements seen with a pyTivo/StreamBaby transcode:

Code:
[libx264 @ 0x101034000]using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
73fps during 1st pass:
Code:
ffmpeg -y -i INPUT -threads 0 -b 2M -bt 4M -vcodec libx264 -pass 1 -vpre fastfirstpass  -flags2 -bpyramid  -an -f mp4
43fps during 2nd pass:
Code:
ffmpeg -y -i INPUT -threads 0 -b 2M -bt 4M -vcodec libx264 -pass 2 -vpre hq  -flags2 -bpyramid  -acodec libfaac -ac 2 -ar 48000 -ab 256k OUTPUT
That is a 300% improvement on firstpass and a 250% improvement on 2nd pass. (The -threads 0 argument just lets x264 use as many threads as it thinks it needs). It ends up using 180% processing time so both cores ARE in use. For those that say: "Hey ... of course it is faster -- he turned of bpyramid", I explicitely turn off bpyramid (-flags2 -bpyramid ) cos using mb-tree -- which is the default now -- is much better (in my estimation) and that default option turns off bpyramid anyway. Doing it on the command line simply reminds me it is being done.

Again, this is simply encoding -- not transcoding.. but the math should be roughly the same. This is also 64bit build (actually it is a universal build across the board).

Jann
 
#1,555 ·
I just tested streambaby with CR, CR/LF, LF and they all seemed to work for me (at least under Linux).

The thing you need to be careful of with Notepad.exe is that it unfortunately likes to add a UTF-8 BOM marker to the beginning of text files. At least for Java, it wasn't handled automatically and at some point I had to add a bunch of code to deal with it.
What I got when I saved the metafiles using Mac CR and served them up using StreamBaby on an "iMac 7,1" using pyTivo was:

(this is the filelisting)
filename1 episodeTitle :
filename2 episodeTitle :
filename3 episodeTitle :
filename4 episodeTitle :
filename5 episodeTitle :

whereas "episodeTitle :" *was* actually the words "episodeTitle :"

When you clicked into the program the screen was filled with gobblygook and then redrawn with spaces all over half of the screen -- erasing what was there.

I even quit and restarted both the TiVo and the Mac

Snow Leopard 10.6.1 was the platform. Files were saved using BBEdit with Mac CR line endings.

Not arguing, you understand, just saying what I experienced.
 
#1,556 ·
I was wondering why "-threads 2" is the default on the pyTivoX installation (and I am assuming the StreamBaby default installation) when even on a Mac Mini (at least *my* Mini) that does not peg the processor. This ("-threads 4") at least goes 160% in the "Activity Monitor"... "-threads 2" only goes 60-90%.
I actually think the Streambaby default is ffmpeg.threads=1. It's a relatively recent addition to streambaby and I didn't want to change the default behaviour.

I tried -threads 0 which I thought was "auto", but at a minimum it crashes the stock Ubuntu ffmpeg, so that was out.

I probably won't do anything about this except leave it as it is and let people change their INI to set the thread parameter as they want. (Except maybe if ffmpeg does end up with an "auto" parameter, use it)

And just as a side note, at least for streambaby when you are doing your benchmarks, you should be transcoding/encoding to MPEG-2 (not h.264).
 
#1,557 ·
What I got when I saved the metafiles using Mac CR and served them up using StreamBaby on an "iMac 7,1" using pyTivo was:

(this is the filelisting)
filename1 episodeTitle :
filename2 episodeTitle :
filename3 episodeTitle :
filename4 episodeTitle :
filename5 episodeTitle :

whereas "episodeTitle :" *was* actually the words "episodeTitle :"
.
Is it possible to post it as an attachment? (Can't remember whether I've seen anyone post a small attachment on tivocommunity or not...)
 
#1,558 ·
I would hazard to guess they're using Notepad, which can really mangle line breaks if the file doesn't originally have the only type of line break Notepad understands (CR-LF if I remember correctly - whatever the Windows standard is). Since the source config wasn't originally built on Windows, probably when they "correct" it they're getting a sequence of 3 or 4 different breaks on each line.

FWIW when I was a Windows user I found Textpad to be a much superior text editor. It's shareware, but back in the day it was still usable in its free version (although I found it useful enough that I bought a license).

On the Mac I've used both the free TextWrangler and it's paid sibling BBedit - both work just fine with streambaby and pyTivo config files. They're smart enough to recognize the type of line endings being used by the file, and sticking with that pattern when you edit it.
 
#1,559 ·
And just as a side note, at least for streambaby when you are doing your benchmarks, you should be transcoding/encoding to MPEG-2 (not h.264).
What? I thought streambaby encoded to MP4...not Mpeg2 when it had to transcode. That, I thought, is why mp4's were sent unchanged to the tivo? Is that a choice cos it is faster to encode to mpeg2 than mp4 or is it a requirement?

Jann
 
#1,561 ·
What? I thought streambaby encoded to MP4...not Mpeg2 when it had to transcode. That, I thought, is why mp4's were sent unchanged to the tivo? Is that a choice cos it is faster to encode to mpeg2 than mp4 or is it a requirement?

Jann
mpeg2 files can also be sent unchanged to Tivos. Transcoding to H.264 (currently the only video format that can be natively decrypted decoded by series 3 Tivos in mpeg4 container) takes far more CPU resources than mpeg2 and is not feasible on most consumer level computers, thus making it impossible to transcode to H.264 in real time or better. Hence mpeg2 is really the only choice.
 
#1,562 ·
FWIW when I was a Windows user I found Textpad to be a much superior text editor. It's shareware, but back in the day it was still usable in its free version (although I found it useful enough that I bought a license).
Notepad++ is far superior to Notepad for Windows. It'll handle the various line types and it's free, open source, and tabbed. About the only thing I don't like about it is the 200MB file size limit, but thankfully I don't have to deal with 200MB+ text files very often.
 
#1,564 ·
mpeg2 files can also be sent unchanged to Tivos. Transcoding to H.264 (currently the only video format that can be natively decrypted by series 3 Tivos in mpeg4 container) takes far more CPU resources than mpeg2 and is not feasible on most consumer level computers, thus making it impossible to transcode to H.264 in real time or better. Hence mpeg2 is really the only choice.
The other problem is the MP4 container itself. (the only container that we know TiVo supports for H.264 streams). Even if the PC could do real-time H.264 encoding, the MP4 container requires a complete index before the file can be played. This means the entire file would need to be transcoded before being streamed to the TiVo.
 
#1,565 ·
The other problem is the MP4 container itself. (the only container that we know TiVo supports for H.264 streams). Even if the PC could do real-time H.264 encoding, the MP4 container requires a complete index before the file can be played. This means the entire file would need to be transcoded before being streamed to the TiVo.
The HME/VLC Video Streamer can use VLC to transcode files on the fly to a H.264 or MP4 stream. Why doesn't it require a complete index before the stream can be played on the Tivo?
 
#1,566 ·
The other problem is the MP4 container itself. (the only container that we know TiVo supports for H.264 streams). Even if the PC could do real-time H.264 encoding, the MP4 container requires a complete index before the file can be played. This means the entire file would need to be transcoded before being streamed to the TiVo.
So, this opens up another can of worms... Questions follow :)

So, if StreamBaby transcodes everything to mpeg2 before sending it to the TiVo, then those of us that think storing it all in MP4 format (to avoid transcoding) are totally wrong, eh?

Also, what methodology does Streambaby use to decode the mp4 and re-encode to Mpeg2. The only reason i ask is when i start a tv show (requesting that Streambaby send it from my computer to my tivo), I see Java go to the top of activity monitor for a second, then ffmpeg.bin starts taking up time. Am I correct in thinking that ffmpeg.bin is doing the transcode from mp4 to mpeg2 (in my case)?

Does Streambaby transcode or send ANYTHING on its own without helper applications or libraries? This is asked cos I can then increase how "good" it performs by recompiling those apps and libraries specifically for my architecture. I have noticed that many, for instance, x264 libraries are not compiled to take advantage of SSE, SSE3, Cache64, etc. (yes i know that x624 is not involved...that was an example. )

Handbrake, for instance, when Andy Vandijck compiled it to be fully 64-bit, (see this at: http://www.insanelymac.com/forum/index.php?showtopic=159004 ) did not compile x264 correctly and thus it took upwards of 6-9 hours to rip a dvd...and this was ALL blamed on being 64-bit...not an invalid compile of x264.

Having said that, for those of us with the know-how, it would be nice to recompile the necessary apps and libraries to be as fast and have as many features as possible.

I know, a lot of questions, but if we can improve streambaby by improving the supporting apps, isn't that a good thing?

TIA

Jann
 
#1,567 ·
The HME/VLC Video Streamer can use VLC to transcode files on the fly to a H.264 or MP4 stream. Why doesn't it require a complete index before the stream can be played on the Tivo?
Haven't used it for a long time but I thought HME/VLC was transcoding to mpeg2 (if needed) before sending to TiVo much like streambaby and pyTivo.
 
#1,568 ·
So, if StreamBaby transcodes everything to mpeg2 before sending it to the TiVo, then those of us that think storing it all in MP4 format (to avoid transcoding) are totally wrong, eh?
No you still misunderstand. If a video file is deemed "Series 3 TiVo compliant" then no transcoding to mpeg2 takes place and the file is streamed *unmodified to TiVo. Compliant videos from what we have gathered are detailed in video compatibility Wiki page. If the video fits into those (fairly narrow) standards then it won't be transcoded, otherwise it will be transcoded to mpeg2.

*unmodified: Not quite true for case of mpeg4 files since Streambaby may move atoms to the front of the file and perhaps some other compliance measures so as to make TiVo happy, but that is essentially not modifying the core mpeg4 file (not transcoding).
 
#1,571 ·
Just a bump, since I think I might've been lost in the shuffle....
Okay, I need some streambaby assistance. I am running a Windows Home Server where all my video files are located. I have pyTivo installed on the server itself and it automatically runs and serves up my video to my 2 S3s. I have played around with streambaby a little, but the streaming seems choppy with it running on a wireless laptop. Should streambaby instead be installed directly on the server, like I have done with pyTivo to ensure the smoothest streaming? If so, is there a way to make streambaby run automatically at startup (like if the server ever needs a restart)? I am looking for seamless functionality for the spousal unit. If that is not the correct way of doing things, can someone steer me in the right direction?

Sorry about the noob questions, sometimes I need to be talked through like I'm 8. :eek:
 
#1,572 ·
Just a bump, since I think I might've been lost in the shuffle....
Here are the instructions on installing Streambaby as a windows service: (windows services autoload at start time, so streambaby would survive a reboot)

http://code.google.com/p/streambaby/wiki/WindowServiceInstallation

Note: I'm not sure what the differences are between WHS and other windows versions, but I am assuming the above will work. It's possbile you may need to play with the account the service runs as after installation, but I'm not familiar with the process off the top of my head.
 
#1,573 ·
Here are the instructions on installing Streambaby as a windows service: (windows services autoload at start time, so streambaby would survive a reboot)

http://code.google.com/p/streambaby/wiki/WindowServiceInstallation

Note: I'm not sure what the differences are between WHS and other windows versions, but I am assuming the above will work. It's possbile you may need to play with the account the service runs as after installation, but I'm not familiar with the process off the top of my head.
PERFECT! That makes it clear, thanks.
 
Top