Photo module for pyTivo

wmcbrine · 12-08-2007, 01:25 AM

Edit: Go here for the latest version. It fixes most of the problems described below.

Seriously, unless you're just into historical curiosities, don't bother with the version attached to this message, or any of them except for the latest. But do note the bit about the Python Imaging Library.

------------------------------

Here's my first stab at a "Photos" module for pyTivo. Now it's got all the HMO services covered.

I wrote this against the December 4th version of pyTivo... I might do a backport as well.

To use this, you need the Python Imaging Library. Get it from your distro, or go here:

http://www.pythonware.com/products/pil/index.htm

The Good: Aspect ratios are correct, even for 16:9 (but see below). My pictures never looked right with Tivo Desktop, IIRC.

The Bad: The aspect ratio is hardwired -- I want to change it automatically for 16:9 and 4:3, but I haven't worked that out yet. In the meantime, you can edit it at the top of photo.py: if you're using a 4:3 TV, comment out the 16:9 PIXEL_ASPECT line, and uncomment the 4:3 line.

The Ugly: Performance really drags on folders with a lot of pictures, apparently due to the way the Tivo builds a list for slide-show purposes. (That's only when you do "View photo"; thumbnails are fast.) I'm not sure if I can do anything about it. In extreme cases, pictures may fail to come up at all.

In the (short!) course of developing this, I found that sending too large an image can instantly reboot the Tivo, no questions asked. But that shouldn't happen with this version.

This should support HD resolutions if you have the HD Photos HME app, but I haven't tested that. (It switches to 1:1 aspect ratio if the request is for a size larger than SD.)

wmcbrine · 12-08-2007, 01:31 AM

P.S. Supported formats... anything readable by PIL, which is practically anything. Currently it filters on .jpg, .gif, .png, .bmp and .tif, but if you want others, just add them to the "goodexts" tuple.

wmcbrine · 12-08-2007, 01:34 AM

P.P.S. Usage... just add a share to pyTivo.conf with "type = photo":

[pics]
type = photo
path = /home/wmcbrine/pictures

dlfl · 12-08-2007, 02:47 AM

Cool beans! And what are we both doing up at this ungodly hour!

wmcbrine · 12-08-2007, 02:21 PM

When I said thumbnails were fast, I obviously hadn't tried it with really large pictures.

Here's a new version that generates thumbnails faster, and caches them.

steve101 · 12-08-2007, 03:13 PM

I installed pil 1.1.6 and unzipped your files to my pytivo
folder. The photos folder shows up on the tivo and when
I select it - it starts to expand but then shows error 5-6
and that's it.

The trace text from the pyTivo window includes:
File "F:\pyTivo\plugins\photo\photo.py", line 110, in QueryContainer
local_base_path = self.get_local_base_path(handler, query)
AttributeError: 'Photo' object has no attribute 'get_local_base_path'

wmcbrine · 12-08-2007, 04:10 PM

It sounds like you're not running a recent enough pyTivo. get_local_base_path() is defined in plugin.py, but only after a certain version... I dunno when exactly, but it's in the latest.

This is why I said I might do a backport... in principle there's no reason this can't work with 2.01 or even older, but the interface for pyTivo modules has changed slightly.

dlfl · 12-08-2007, 04:23 PM

Quote:

Originally Posted by wmcbrine

It sounds like you're not running a recent enough pyTivo. get_local_base_path() is defined in plugin.py, but only after a certain version... I dunno when exactly, but it's in the latest.

This is why I said I might do a backport... in principle there's no reason this can't work with 2.01 or even older, but the interface for pyTivo modules has changed slightly.

I believe windows users are currently unable to use the versions that have the get_local_base_path function, i.e., where the use of "full" paths was introduced. This is ticket #52 on the Wiki.

wmcbrine · 12-08-2007, 06:22 PM

Quote:

Originally Posted by dlfl

I believe windows users are currently unable to use the versions that have the get_local_base_path function, i.e., where the use of "full" paths was introduced. This is ticket #52 on the Wiki.

I see that now.

I guess I'll take a look at it... Meanwhile, here's my backport of the photo module to pyTivo "d498" of November 25th.

Edit: Hmm, the whole thing works better with d498. The large folder problem kinda just goes away. Interesting.

dlfl · 12-08-2007, 07:42 PM

Quote:

Originally Posted by wmcbrine

I see that now.

I guess I'll take a look at it... Meanwhile, here's my backport of the photo module to pyTivo "d498" of November 25th.

Hmmm... Worked partly -- but no photos.

When I selected the photo share name the python window scrolled a huge barrage of messages, too many to try to copy. The page of thumbnails did show up and all the text seemed OK but no images in the thumbnails. When I selected one image, the next page showed up with a large black area where the image would have been and all the normal metadata was present (date taken, etc.) When I selected to view the image, my Tivo rebooted.

Here is the last error dump from the python command window when I selected one of the thumbnails:

Quote:

Exception happened during processing of request from ('192.168.0.158', 1332)
Traceback (most recent call last):
File "C:\Python25\lib\SocketServer.py", line 463, in process_request_thread
self.finish_request(request, client_address)
File "C:\Python25\lib\SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Python25\lib\SocketServer.py", line 521, in __init__
self.handle()
File "C:\Python25\lib\BaseHTTPServer.py", line 316, in handle
self.handle_one_request()
File "C:\Python25\lib\BaseHTTPServer.py", line 310, in handle_one_request
method()
File "C:\Program Files\pyTivod498\httpserver.py", line 38, in do_GET
plugin.send_file(self, container, name)
File "C:\Program Files\pyTivod498\plugins\photo\photo.py", line 136, in send_f
ile
pic.save(handler.wfile, 'JPEG')
File "C:\Python25\Lib\site-packages\PIL\Image.py", line 1405, in save
save_handler(self, fp, filename)
File "C:\Python25\lib\site-packages\PIL\JpegImagePlugin.py", line 409, in _sav
e
ImageFile._save(im, fp, [("jpeg", (0,0)+im.size, 0, rawmode)])
File "C:\Python25\Lib\site-packages\PIL\ImageFile.py", line 491, in _save
s = e.encode_to_file(fh, bufsize)
IOError: [Errno 9] Bad file descriptor
----------------------------------------

I'm guessing you've only tested this on linux (?). I suspect the problem may be something like the '/' vs '\' path delimiter difference?

I tried both delimiters in my share path -- no difference in behavior.

Using d498, python 2.5 and pil 1.1.6 for python 2.5

steve101 · 12-08-2007, 07:48 PM

Yeah I forgot to mention windows 2000 server.
BTW python 2.5, pil 1.1.6 for python 2.5

Guess I'll wait - I'm moving my server over to
a linux box after some hardware changes are done.

wmcbrine · 12-09-2007, 01:48 AM

OK... boy was that a pain. *&@!$^# Windows. And it's not fixed yet...

The failure was on the line

Code:

pic.save(handler.wfile, 'JPEG')

In Python terms, handler.wfile is a file-like object (it's the http output stream). File-like objects may not have all the properties of a real file, like the ability to seek. In Linux, handler.wfile evidently satisfied Image.save(); but in Windows, it didn't. That was truly bizarre. To get around it, I had to use cStringIO. On the plus side, this should make the thumbnail cache work even faster.

So, that got it mostly working in Windows. But there were still some problems: rotation didn't stick, and sometimes pictures didn't appear, or it would just pop out of slideshow mode for no apparent reason. I managed to fix the rotation, by doing a lot of backslash-to-slash replacing (it made the cache keys match), but I still got the other problems -- and then I hit a reboot. So, I'm giving up for the night. I recommend you not try to use it on Windows at this time.

(None of these problems occur in Linux, nor probably in Mac OS X, etc.)

Anyway, here's what I have right now, minus the ugly backslash-to-slash stuff. Besides the Image.save() workaround, I also reverted to using PixelShape, but doing it correctly this time.

This actually does work automatically with both 16:9 and 4:3 sets, assuming that you have the ratio set in your Tivo's video settings.

This is the "backported" version, which is now the main version, at least until I can get it performing as well under the newer pyTivo.

wmcbrine · 12-09-2007, 02:43 AM

OK, I take it back -- a Tivo reboot can still happen even with the photo module running under Linux.

I don't yet know exactly what provokes this behavior, but it's preceeded by a bogus request from the Tivo:

Code:

GET /TiVoConnect?Command=QueryContainer&Container=pics1%2Factress&Width=634&Height=432&PixelShape=40%3A33&Format=image%2Fjpeg&Rotation=1593802088 HTTP/1.0

that mixes elements from the two basic kinds of requests it normally makes (for info, and for pictures). I especially like the "Rotation=1593802088" bit.

wmcbrine · 12-10-2007, 04:38 AM

Here's my latest. 0.4 is for the most recent pyTivo release; 0.4b is for d498. They both perform acceptably on large folders now, although d498 is still faster. These also include the path kludges to make them work on Windows.

I wish I could say I'd eliminated reboots, but I can't. I probably won't post another version until I've sorted that out. If you avoid trying to advance past the last picture in a folder from the full-screen ("View photo") view, you can probably avoid reboots. But I can't promise that..

wmcbrine · 12-10-2007, 06:12 PM

AFAICT, I've fixed the reboot problem.

When the Tivo asked for a recursive list of files, get_files() was returning a list that included directory names along with their contents. This should probably be fixed in plugin.py, but in the meantime, I've included a replacement get_files() in photo.py.

I also found a more elegant way to deal with the path problems in windows. Instead of a bunch of replace()s, it's just this:

Code:

if os.path.sep == '\\':
    os.path.sep = '/'

I wasn't sure it would work, but it does. The module still seems to act a bit more flaky under Windows than Linux, for reasons undetermined, but I think I can recommend it now.

For now, this is only for the latest pyTivo.

dlfl · 12-10-2007, 08:25 PM

Worked great for me! Good job!

wmcbrine · 12-10-2007, 08:53 PM

I expect people will be annoyed that I'm posting another version less than three hours after the last one, but I managed a big performance boost in slideshow mode by caching the last recursive lookup. I'm finally happy with the speed now. But please note that this means new pictures in a folder won't show up until you trash the cache, by doing "View photo" in a different directory (or by restarting pyTivo).

Remaining issues: I need to implement the date sort modes; right now it's always alphabetical, despite what the Tivo says onscreen. Also, when scrolling backwards in thumbnail mode, the last item disappears. I see the same thing in the music module. What I don't know is whether Tivo Desktop has the same problem, or if this is a pyTivo problem I can hope to fix. Can anyone with TD answer?

reneg · 12-10-2007, 11:03 PM

Tried it out for a few minutes and it worked great. Will play with it some more later this week.

wmcbrine · 12-11-2007, 02:33 AM

Quote:

Originally Posted by wmcbrine

What I don't know is whether Tivo Desktop has the same problem, or if this is a pyTivo problem I can hope to fix.

To answer my own question: It is a pyTivo problem, and I have fixed it. However, my os.path.sep hack didn't really work as well as I'd thought. So, I'll spare you version 0.7 while I try to sort that out.

wmcbrine · 12-11-2007, 03:05 PM

OK, here's the new version, with much better handling of Windows paths, and no more missing items when scrolling through the thumbnails. But I'm still getting "error #5-6" from the Tivo sporadically when pyTivo is running under Windows (not under Linux, so far), and I can't quite figure it out. It goes away when I try to debug it.

wmcbrine · 12-12-2007, 03:02 AM

Tested OK on Mac OS X now, in case you were wondering.

wmcbrine · 12-12-2007, 05:24 AM

Much faster thumbnails, much better quality on fullscreen view, and a few more extensions added to the good list:

wmcbrine · 12-13-2007, 05:09 AM

This does all the sort modes. I think the module is functionally complete now. (Which isn't to say that it can't be improved!)

I have it sorting folders first only when "Arrange by Type" is On; otherwise they're mixed in with the pictures. Anyone who uses Tivo Desktop, please let me know if this is actually the expected behavior.

jcthorne · 12-13-2007, 10:53 AM

First I gotta say GREAT WORK! and THANKS! This is MUCH faster and easier to navigate than the photobucket app that Tivo just added in.

I got the 0.9 version working and have a question if you don't mind.

Is there a way to get this module to display the photos in a higher resolution? It does not appear to be using the HD resolution of the TivoHD at all, resizing to 6xx x 4xx or so.

If you just have not gotten that far OK, just asking in case there is something I should be setting.

It does appear to be navigating my albums directories several layers deep just fine.

When I select a photo, it shows it and offers several options. One of which is 'view slideshow' Selecting that does indeed play a slideshow but it seems to select from all photos in this diretory and others adjacent or at the same level in the tree, not just from within this folder. Is that right?

Is there a way to adjust the rate at which the slideshow advances to the next picture or to manually select next picture?

Thanks again for all your work in adding this much needed piece to pyTivo.

wmcbrine · 12-13-2007, 01:03 PM

Quote:

Originally Posted by jcthorne

First I gotta say GREAT WORK! and THANKS! This is MUCH faster and easier to navigate than the photobucket app that Tivo just added in.

Well, it should be faster; it's working over your LAN instead of the public Internet.

As for navigation, that's credit to Tivo -- the HMO picture viewer is a built-in function. My module just supplies the pictures.

Quote:

Is there a way to get this module to display the photos in a higher resolution?

Unfortunately, like the rest of the Tivo menus, the picture viewer doesn't make use of HD. Only Tivo could change that. My photo module is ready if they do.

The only way currently to get HD pictures on your Tivo is via HME. There's an HD photos HME app that comes with Tivo Desktop, and I understand that some people have gotten it to work with Galleon. (It needs an HMO server as a backend.) It should also work with pyTivo as the backend, but I haven't tested that.

It would be nice to have a Python interface to HME (and I don't mean in Jython)...

Quote:

When I select a photo, it shows it and offers several options. One of which is 'view slideshow' Selecting that does indeed play a slideshow but it seems to select from all photos in this diretory and others adjacent or at the same level in the tree, not just from within this folder. Is that right?

It should be all photos in subfolders, not folders at the same level. Anyway, this is an option you can turn off -- select "Options" off of the slideshow option.

Quote:

Is there a way to adjust the rate at which the slideshow advances to the next picture

Yes, that's also in the Options menu.

Quote:

or to manually select next picture?

In slideshow mode, you can use FF and REV almost as in a video. In "View photo" mode, FF and REV go back and forth between pictures.

P.S. Writing this was actually great fun and a breeze, except for trying to make it work right under Windows (which it still doesn't). But anyway, you're welcome.

wmcbrine · 12-14-2007, 11:31 PM

So, I finally tested it with the HD Photos HME app, and it needed a couple of tweaks. (pyTivo was only sending 10 items when no ItemCount was specified -- it should be all items; and I'd set 200x200 as the maximum size to treat as a thumbnail, but the HD app used 225x225. (My module loads thumbnails in "draft" mode, which is much faster.) So now the cutoff is 250x250.)

Also in this version, I decided to use the highest-quality resize, antialias, even for thumbnails. It looks much better, and loading in draft mode makes most of the speed difference anyway.

How to use the HD Photos app

See here:

http://www.tivocommunity.com/tivo-vb...99#post5032599

Where he says, "Make sure TiVo Desktop is running and sharing photos", substitute "pyTivo"; and where he has port 8101, make it 9032 (unless you've changed this in pyTivo.conf). Note that it requires Java 1.5; Java 1.6 (or 6, or whatever Sun is calling it today) will not work, because it doesn't like the certificate the app is signed with.

There seems to be a fear of posting the .jnlp file itself in that thread, although personally I think it's too trivial to be covered by copyright. (It's not the app, just a pointer to it, basically. Like a .ram or .m3u file.) Not to mention that it's part of a package that Tivo distributes for free anyway. But I'd just as soon not be the first to test the issue...

reneg · 12-15-2007, 04:54 AM

The .9 version works very well on my system.

From version .9 back to .10, that may confuse some folks between version .1 & .10 in the first post in the thread.

Do you know if there is any support in the Tivo to support metadata for Photos? I'd like to add descriptions to some of my photos.

wmcbrine · 12-15-2007, 01:39 PM

Quote:

Originally Posted by reneg

From version .9 back to .10, that may confuse some folks between version .1 & .10 in the first post in the thread.

Don't think of it as a decimal point. It's just a separator (like in a URL). Basically, it's version ten... the "zero dot" meaning "it isn't finished yet".

I did hesitate to call it "0.10", because I had this conversation already ten years ago.

But I'd thought that this kind of version numbering was common enough now that it wouldn't confuse anyone. And calling it "1.0" would be wrong, because there are still a few things I need to sort out before I'd feel comfortable saying "yes, this is a complete substitute for the photo function of Tivo Desktop".

Quote:

Do you know if there is any support in the Tivo to support metadata for Photos? I'd like to add descriptions to some of my photos.

There is a "Caption" tag mentioned in the HMO spec that didn't seem to do anything in the regular viewer when I tried it, but it looks like it may be honored in the HD viewer. I'll look into it further.

jcthorne · 12-16-2007, 09:19 AM

I followed the instructions in the referenced link for editing the jnlp file etc. right up to the point that it generated the .jar files. 10 of them.

Need some instructions on what I am to do with them and how to use them to get pytivo to serve HD photos.

I can see the share I named in the hdphotos.jnlp file. Selecting it starts the HD photos ap and shows the share name from pytivo.conf as a folder.

Selecting that folder yields "This folder does not contain any viewable photos" Also of note is that pyTivo shows a message "Bad request synax"

Looking at the responses from pytivo from the HD ap and the SD ap are different in one place. In the working SD ap the share name is called out as:

Container=Thornoli's%20Photo%20Albums%2F2000.....

In the non working HD ap the same share is referenced as:

Container=Thornoli's Photo Albums&....

Seems the spaces are being represented differently and pyTivo does not like the second one.

Thanks

wmcbrine · 12-16-2007, 03:04 PM

Quote:

Originally Posted by jcthorne

Need some instructions on what I am to do with them and how to use them to get pytivo to serve HD photos.

No, it looks like you did it right.

Quote:

Also of note is that pyTivo shows a message "Bad request synax"

Looking at the responses from pytivo from the HD ap and the SD ap are different in one place. In the working SD ap the share name is called out as:

Container=Thornoli's%20Photo%20Albums%2F2000.....

In the non working HD ap the same share is referenced as:

Container=Thornoli's Photo Albums&....

The "Bad request" message must be from Python's BaseHTTPRequestHandler, and the bad request itself from the HD photos app. Nothing I can really do about either of those... It looks like it's not quoting the request properly. However, I can't reproduce this. I have no problem with directories with spaces in their names. Maybe it's the single quote mark that's screwing it up?

You could try making that directory the base of the share (with a different share name), so it won't be passed on.