Movie Jukebox- woof woof woof

[Justin Thyme]

With TivoToGoBack, and steadily declining prices in very large Hard drives, it is becoming more and more affordable to build a Media library in one's home.

As an experiment a few years ago (http://archive2.tivocommunity.com/tivo-vb/showthread.php?t=246588), I prototyped this using an MCE and an IODATA LinkPlayer. My observations were that

finding stuff was a huge chore on the Microsoft and IODATA solutions. Something like
huge inefficiency- although unused most of the time the server was running 24/7 spinning disks accumulating wear and tear
Mpeg4 formats (DivX, WMV) multiplied effective space over .tivo files by a factor of 2.5
DivX.com's free encoder at the time had sync problems with files recorded by a tivo (no doubt fixed by now).
The server would run no problem using an obsolete PC board stuck into a large Antec box with lots of drive bays. However, a machine with gobs more power was necessary for the encodings.


I am thinking now that this sort of thing is starting to become affordable to folks whose mental age is close to that of their 13 year old children. All the excitement, but all the danger of actually having the resources and attention span to build something with a lot of woof woof woof....

So my idea of a dream server would be a box that

Wasn't spinning disks 24/7
Be instantly available from the Tivo (competing design constraint)
has a back end database that loaded up IMDB like photos and movie info- eg.Extreme Movie Manager (http://www.binaryworks.it/extrememoviemanager/)
streams content to the Tivo like PyTivo.
ok if it were noisy, since it could be anywhere in the house.


That shouldn't be real hard- cache the entire movie directory trees and show descriptions to RAM- by the time the user makes it to the play button, you have the disk spun back up. So most of the time, it is like pyTivo running off a ramdisk- sitting in low power mode with all the disks off.

Content acquisition sucks shows off of Tivo, enhances the movie data like extreme Movie Manager (XMM) does, then compresses it to Divx.

The Tivo interface servers up virtual folders alphabetized by director, actor, user defined database filter ("wishlist" UI).

Distribute through PVRupgrade and Weaknees if they are interested in selling premade boxes ready to go.

My big question is whether my family would actually use it. My gut tells me yes, because the directories show up on the Now Playing list, so they don't have to learn ANYTHING new. The trouble with modeling it is that Mrs Thyme and I are so busy with activities that after the twins came on line, we really haven't been regular TV watchers.

[Justin Thyme]

Since I really really don't want to babysit transcodings and the latest DivX encoder produces .divx's that won't play with pyTivo, I guess I am going to use TTG's auto transcoder.

Which forces me to move from divx to something supported by the TTG converter (yeah, ok- maybe it could be hacked to work with the divx encoder, but I said, "something supported"- meaning something that wouldn't be a nest of huge hassles for civilians)

So it looks like it is either choose WMV (egads), or H.264. Although I have a video iPod, H.264 lost out, because WMV enjoys much broader support. Tough break. I wish .ogg were further along, but at this point it isn't so WMV it is. I have stayed away from WMV for now, but I guess I will learn more about MS shenanegans with their format going this way...

[etsolow]

/subscribe!

[dlfl]

Since I really really don't want to babysit transcodings and the latest DivX encoder produces .divx's that won't play with pyTivo...........
Are you talking about the DivX converter? There was a survey on the DixX community forum recently asking who preferred it vs. DrDivx and last I looked the doctor was winning by a skunk. (And it gets my vote too.) I've paid for the latest DivX codec and my encodings (via doctor) play fine via pyTivo. Is it just a couple of advanced features (e.g., qpel, packed bitstream) that cause the pyTivo problems?

pyTivo uses ffmpeg, which depends on volunteer efforts so isn't always up to speed with the very latest codec features. However it is under continued development so may catch up sooner or later. Then there is XviD. There are several free programs (e.g., autoMKV and ffmpeg) that will make XviD encodings (which also play via pyTivo).

Not that I'm against WMV....

If that's the only reason you're moving away from pyTivo, it seems ... well, insufficient. :confused: If there is a packaged process (e.g., TTG) that really handles all this with no glitches on any video, I will be pleasantly surprised.

[Justin Thyme]

I would prefer an XVid or ogg thing, but the tools are still a bit behind the curve. I am not getting married to any of these formats and I am quite leary of WMV and the arbitrary limitations their encoders and players often have.

The divx converter I had required me to strip the .tivo first, and then one out of 6 times it lost the sync on the transcode. If there is another version that does Divx, that has a much better error rate than this, then I would prefer to use it, especially if they are open source/ offer libraries for all platforms.

I do not contemplate giving up on pyTivo unless something better comes along.

On a different subject, I noted that TTG 2.4 appears to disallow transcodes at greater than 320x480. (yes, the correct xml was in both the support and plus directories). Maybe I was hallucinating on that one though.

Oh- and naturally your transcode will run hugely faster if there is no resize. Meaning if you are cable best, you want to archive to hrez 480 vrez 480. For Satellite users best, then hrez 544 vrez 480, and dvd burners 720x480.

[dlfl]

The divx converter I had required me to strip the .tivo first, and then one out of 6 times it lost the sync on the transcode. If there is another version that does Divx, that has a much better error rate than this, then I would prefer to use it, especially if they are open source/ offer libraries for all platforms.
............ .
I don't know any converter software that takes TiVo without stripping. I've always used VideoReDo to do my stripping, and usually also to top, tail and remove commercials. VRD cleans up mpeg2's, in particular with regard to maintaining audio sync. You can use its hands-off Quick Stream Fix if you don't want to edit. QSF or saving after edits takes only a few minutes per hour of video. With this process I've never had an Mpeg4 encoding (H264, Xvid, DivX or WMV) from a TiVo source that had video sync problems. But I haven't done a huge number of conversions yet, so no reliable statistics. My intuition is you need clean-up for some major fraction of TiVo recordings.

[Justin Thyme]

I use FF for my cleanup. With hundreds of films to archive, I just can't be botherred unless I am positive it will get watched a huge number of times.

It is nice if the .tivo step were integrated, and actually Prish's directshow dump has a watch feature so it is not really that big a deal. I would just like a much more turnkey workflow for novice users.

[Justin Thyme]

Perhaps the best approach on the back end would be to integrate pyTivo like functionality into FreeNAS (http://sourceforge.net/project/showfiles.php?group_id=151951), so that it is just one of many other server interfaces like FTP. This supports Spin down, and is open source so it looks like a good candidate.

Hmph. Lessee now. An auto transcoder like TivoDesktop 2.3's to do the archiving into some MPEG4 format, Xtreme Movie manager like information-mining to suck info from IMDB and other sources into a MySQL database, an interface to present the enhanced movie information to the Tivo user quickly, and ffmpeg sending streams to the Tivo. Jeez- at least 6 months to get something solid. With 4 kids- Oh say 5 years....

Rats.

[dlfl]

I use FF for my cleanup. With hundreds of films to archive, I just can't be botherred unless I am positive it will get watched a huge number of times.

It is nice if the .tivo step were integrated, and actually Prish's directshow dump has a watch feature so it is not really that big a deal. I would just like a much more turnkey workflow for novice users.
What means FF ?

[Justin Thyme]

Sorry. attempt at humor. Fast Forward.

[Justin Thyme]

OK, I have been ripping my DVDs over now, and just like with my CD collection, I think I may be able to box up all the DVDs and bury them in storage. They are much nicer on Tivo being able to continue playing a movie right where I left off a week ago. The thing that allowed me to do this now and not before is twofold. First, huge storage now allows me to keep such dvds on Tivo permanently, and secondly, PyTivo allows a transcribe/transfer from archived mp4 compressed format making access to huge libraries convenient.

Mostly I am doing the baby dvds like Baby Einstein that are pretty convenient on the Tivo, a few art and the christmas ones, but I'll get around to the rest.

I permanently keep the baby ones on the 1.5 TB tivo, but the other ones will stay on the server until needed.

This format choice is still bugging me. My criteria seem pretty simple.

Output by DVDFab Platinum & TivoConverter
PyTivo can play them
Easy transcode to Ipod
Video editors input them

There is a lot of "Can't get there from here" (without a transcode). I noticed that ULead MovieFactory 10 will eat .mp4 file wrappers around MPEG4 +aac sound, but would not eat H.264 mp4's. They may work with Ulead, and are playable but not transferable from Media Player because I have not bowed before the wmv god, so I can't use that to do the auto transcode to portable media devices, but that isn't a huge deal to me because i don't do video on the PDA and the iPod is mostly static cause the girls like to watch the same things over and over on the way to school. Anyway, other choices are possible, like XVid, which would be nice if my main way of using these were playiing using an mpeg4 capable dvd player. But for me that would be pointless. I want to be rid of individual disks.


My DVDFab Platinum settings allow me to rip as if I were going to a portable, only I use a modified .xml profile (you can add any sort you like- see parameter explanations in the read me.)

I don't know if these are best, but I am listing them in case anyone wants to go in a similar direction. The resultant Videos look fabulous on the S2 piped in from PyTivo. Looks like DVDs which it probably can't be do to the low data rate that pyTivo uses, but anyhow, much better PQ than recorded video. One gotcha- dvdfab by default includes subtitles, so go to customize in DVDFab, select subpictures and uncheck auto include subpicture.

Just drop this xml in your dvdfab profiles directory and you will get it as a portable generic option that you can set as a default. Processing is about an hour for every hour of movie on recently manufactured computers using single pass.

<profile name="generic.pyTivo" version="200" device="generic" tag="dvdfab,default" description="The profile is used to create mp4(mpeg4 + aac) files
for use in archiving family video DVDs at high fidelity, editable by ULead.">
<common>
<file format="mp4" mode="0" onefile_size_mb="300" number="1" />
<audio codec="aac" channel="2" >
<freq value="48000" list="8000,12000,24000,32000,44100,48000" />
<bitrate value="96000" list="32000,48000,64000,96000,128000" />
<volume percent="150" />
</audio>
<video>
<!-- JUSTIN THYME- based on ipod mp4- ADDED 640x480 etc, audio at 128K, file format from mp4_ipod to mp4, increased maxbitrate from 2500 to 10000 -->
<size list="720x480,640X480,480x480,320x240" />
<subtitle mode="0" />
<outfps mode="3" value="29.97" list="10,12,15,14.985,15,20,23.976,24,25,29.97,30" />
<encode codec="mpeg4" bitrate_kbps="1500" minbitrate_kbps="500" maxbitrate_kbps="10000" >
<mode value="real_time" list="real_time, high_quality" />
</encode>
</video>
</common>
<acodec name="faac">
<common profile="1" quality="100" />
</acodec>
<vcodec name="mpeg4">
</vcodec>
</profile>

It really does seem to me that MPAA is missing the boat here- it really feels to me like a repeat of MP3 ripping. One thing they could do that it is way too late for the music industry to now do is- say hey- we will release content to your format making your platform a success if you cut us in to a percentage of your hardware profits. And really just flood the market with content so that anyone who doesn't do a deal has to rely on pirate content and basically commits market suicide since it won't play the stuff they can get off the walmart/unbox site.

If content owners don't release real DVD quality content online, well tough break- it's a replay of mp3 and they cut themselves out of the action. Folks will just buy the dvds and rip them into a format that has zero protection. It really is not at all difficult for naive folks to do, and from what I see is exceptionally reliable, unlike earlier dvd ripper technology.

[Justin Thyme]

Playing back ripped dvds via PyTivo presents a really stunning picture on an S2. I am a little surprized, because my understanding was that FFMPEG was only outputting at 4Mbps. I guess I am mistaken, but I didn't think that was sufficient for DVD quality SD.

Anyway- really amazing picture quality. I should probably do a side by side comparison of DVD original and that produced through pyTivo archive of the ripped copy.

[SugarBowl]

You have babies and you find time for this?

<jealous>

[Dajad]

Justin, I'm loving where you are going with this.

As I mentioned to you, I too am looking for the holly grail format and I believe it is h.254 given that all of the following now natively support it:

- TiVo S3 (well it will some day),
- PS3
- Xbox 360
- Apple TV

Portables:

- iPods
- iPhone
- PSP
- Zune

It surprises the heck out of me that the Bittorrent crowd hasn't abandoned DivX/XVid for H.264.

Anyway, I note above that you user DVDFab to rip your DVDs to Mpeg 4. It wasn't 110% clear to me that this program can rip a DVD into the H.264 format that will be playable on the settop box devices above (eg: TiVo (some day), the Xbox 360, AppleTV and the PS3. Does it? Or do you need to somehow add in the script that you set out above for this.

I'm on the cusp of purchasing a program to do this and all I want out of the program is:

1. DVD to DVD copying
2. DVD to portable H.264 (ie: to iPods, iPhones, Zunes and PSPs); and
3. DVD to settop H.264 (360, PS3, AppleTV, TiVo S3 - some day)

Does DVDFab Platinum do all this without having to be a rocket scientist? it wasn't clear from its product description page (http://www.dvdfab.com/dvd-fab-platinum.htm) that it can do number 3 above. If not, can you recommend something that does all 3 of these? For instance, there are a number of programs reviewed, recommended here (http://topsixreviews.ca/mult.htm), but I don't know which is best and if they are better/worse than DVDFab or anything else.

OK, I guess I'd also like it to turn .tivo files into a DVD but that is optional. I mostly want to convert my DVDs for use on my portable devices and playback though my settops.

...Dale

[Justin Thyme]

Sorry Sorry for not noticing the activity on this thread.

I don't monitor thread very well. If anyone notices I haven't responded to a query in a few days, please PM me.

Yeah Dale- DVDFab does H.264 basic profile at any aspect ratio in an mp4 container. Also does something called H.264-5.5G if that notation makes any sense to you. It's a trial for a month so you can play with it. (It also does xvid/Divx in avi wrapper). Load up a dvd and click on the Generic Portable option. Other MP4 in the iPod option might float your boat.

So it explicitly supports iPod and delivers the kind of compression performance you expect from H.264. I'll be encoding everything and tossing my old movies in mpeg4 as soon as the players and software tools I use start support it.

Hope that helps.

=JT=

[Justin Thyme]

got a pm and it is a general question so will answer it here.

I just was reading your post on creating a movie jukebox and I must say I am VERY impressed. :cool:

So you are using DVDFab Platinum to rip DVDs to your server. What format are you ripping to? I want to start a similar project, but want to find a happy balance between high quality and size of a file. I would rather have a large file with high quality, than the reverse. That being said, I don't want to just rip to MPEG2. While that is easier, the file sizes are way too large. Currently, I have a few DivX/XviD files that I am using pyTivo to transcode on the fly, and that seems to work nicely. But I am looking on how to create a nice "jukebox" of my ripped DVDs.

Do you happen to have an HDTV? I don't yet, but plan to buy one in the future. I am curious on how the ripped DVDs look on an HDTV.

Any insight you could share would be appreciated.
As stated above in this thread, I am going to what DVDFab refers to as "MPeg4 + aac audio" in an .MP4 wrapper.

If you prefer xvid/DivX5 (pytivo won't play Divx6.5 files produced by the latest divx free encoder), then you can use DVDFab Platinum for that.

I haven't had any problem converting my dvds (probably done a couple hundred) except for a few that might have been dirty. But I have a lot of old series and documentary disks like Connections and Avengers. They probably aren't protected much.

The bulk of my material is "reruns". I used desktop 2.3 version which allows you to transfer and transcode 640x480 as I think I described above or in some other thread.

I recommend just downloading the dvdfab thing and trying for a month. I think you will be happy.

But hey- this advice only applies to disks you own. Anything else would be theft and I refuse to help on anything regarding that or the equally illegal trading activity. Fair enough?

[Justin Thyme]

Oops- missed this one. "Do you happen to have an HDTV? I don't yet, but plan to buy one in the future. I am curious on how the ripped DVDs look on an HDTV."

HDTV? Yes. I have a 38 inch Sony Bravia and a 58 inch Panasonic upstairs.

PyTivo delivers the ripped files in a quality comparable to non upscaled dvd coming through S-Video. When I had Tivo back working on S3 due to a TivoCorp glitch, I found that the S3 does a fabulous job upscaling, so on an S3 you can expect to see PQ comparable to an upscaling DVD player with an HDMI cable.

So- not noticably different than dvd quality with one exception I have noted on the S2. Rarely, on highly complex scenese I do see jitter. EG on one of the baby einstein disks there is an underwater scene with a huge school a fish that suddenly turn. There is a frame loss and even I think an audio drop out, but then it continues just fine. I don't know if that is due to pytivo transfer rate set too high, or DVDFab messing up, but it happens very rarely, and only on very high data rate type scenes.

Hope this helps.

[stevereis]

I have been using DVDShrink as outlined in Dave Zatz writeup from a while back: http://www.zatznotfunny.com/gtt.htm. I have encountered a few problems with some newer DVD I own but found that DVDfab does the job to extract the VOB files. Then I use DVDshrink to combine the VOBs and audio.

This makes a nice MPEG2 file that looks great but has a couple of problems. The first is that it's the raw DVD file so takes up a lot of space. However, that's secondary to the issue I have with my Humax. For some reason, it will not play back anything properly unless it's encoded as 4:3 aspect ratio. I think this is a known issue - it wraps lines around so the image is worthless. It's some kind of interpretation issue since I can use VideoReDo to fake it out and turn the file into a 4:3 by modifying the headers and it will play fine but the aspect ratio is screwed up. So, with most movies, I end up with a file I can only watch on my regular S2's.

I had bought an old lifetime S2 unit on eBay a while back and had planned to put a big disk in and use as a video server to store rips of all my movies so that any of the TiVos could access. I had held off doing this due to the Humax playback issues but with the new PC-to-TiVo transfers/transcoding in Desktop 2.5, I can rip to DivX or another format and use Desktop to have the PC transcode to a letterboxed 4:3 file that will play on any TiVo. (BTW, this is the approach taken by Unbox so that the movies play on any TiVo, including the Humax DVD units.)

BTW, I am using DivX format since my Archos media play handles this format fine. I have been using AutoGK for DivX transcoding but sounds like I need to try out DVDfab (and pay for the platinum edition).

[Justin Thyme]

Just for the record, I ruminate on DVDFab only because it is what I have some experience with. Take it with a grain of salt.

I have heard good things about other rippers and just for the record I have no idea whether DVDFab is any better than the current generation. For all I know it is worse. Anyway, it is clearly better than the tools I used 2 years ago, but if someone wants to spend some time looking for the ultimate, I'd recommend doing a little more thorough survey than what I did.