HME for Python

[wmcbrine]

I've mentioned this a couple times... here it is. Yes, the new HME implementations are coming thick and fast now. (Seriously, I'd been working on this for about a week when jbcooley posted his... weird.)

Here's the README:

HME for Python, v0.20
by William McBrine <wmcbrine@gmail.com>
May 6, 2012

An implementation of TiVo's HME (Home Media Extensions) protocol for
Python, as a module (hme.py), a simple server (start.py), and examples
(mostly ported from TiVo's Java HME SDK). Everything is released under
the LGPL 2.1+, except where noted. (Most of the examples are Common
Public License.)

Tested on multiple platforms with Python 2.3 through 2.7 (not
caompatible with Python 3.x). Requires only the standard library.


Quick Start
-----------

In Linux, Mac OS X, etc.:

./start.py

In Windows:

python start.py

This will serve the example apps. The default port is 9042 (NOT TiVo's
default of 7288). To see more options, run "./start.py --help".


Quick Stop
----------

In Linux, Mac OS X, etc., Ctrl-C does an orderly shutdown, un-announcing
the apps before closing. (The equivalent is SIGINT, or "kill -2".)

In Windows, with older versions of Python, you're stuck with either
Ctrl-Break, or closing the command window, neither of which does an
orderly shutdown. This may have been fixed with Python 2.7.


Config
------

You can optionally create a file named "config.ini" to specify the same
kind of options as on the command line. The contents also become part of
the context passed on to each app, so apps can store their own
preferences there. See the picture viewer for an example -- it now
checks the config for optional root path, delay and extension list
options. (You'll almost certainly want to change the path.)

Here's an example config.ini, showing all the possible keywords for the
server and the picture app. (The values shown here are purely for
illustrative purposes.)

[hmeserver]
port=7288
address=192.168.1.1
zeroconf=False
basepath=c:\hme
datapath=c:\
apps=picture clock

[picture]
path=c:\pictures
delay=2
exts=.jpg .png


Direct Text Input
-----------------

TiVo has extended HME to allow direct input of ASCII text via key
events, instead of requiring the "Ouija board" input method. This is
intended primarily for use with the Slide remote, but it also works with
other USB keyboards, as well as the network remote control interface.
TiVo has not documented it publicly, so I'll describe it here.

There are two types of direct text key events. The first type returns a
key code of KEY_UNKNOWN (0), with the value of the key encoded in the
rawcode. (To convert this rawcode to ASCII, you can use HME for Python's
new top-level function qwerty_map().) This type of event is used for
KEY_PRESS events generated by the IRCODE command from the network
interface, and is used for all KEY_RELEASE events. Case is not
distinguished (HME for Python decodes the values as uppercase), and for
symbolic values that appear on the upper half of a key, the value on the
lower half is reported.

The second type of direct text key event returns the ASCII code, plus
0x10000, as the key code. Case is preserved, and symbols are reported
exactly. This type of event is used for KEY_PRESS events generated by
USB keyboards, as well as by the KEYBOARD command from the network
interface, which currently works only on the Premiere. This second type
of event is only returned if the HME app reports that it implements
version 0.49 of the HME protocol or later; otherwise, the TiVo drops
these events, while still reporting events of the first type. (This
means that a pre-0.49 app will receive only KEY_RELEASE events for most
keys on a USB keyboard.)

After going back and forth on this, I decided to pass the codes for both
event types through unchanged, so that the app could handle them as
needed. (In my own Reversi, for events of the second type, the existing
case is respected, while it's adjusted automatically for events of the
first type.)


More Info
---------

The main documentation is in the hme module itself, as docstrings. (Open
a Python shell; "import hme"; "help(hme)".) And be sure to look over the
example apps.

You may also want to consult the HME Protocol Specification, from TiVo's
Java SDK. And the rest of the documentation from that package may be
helpful in a general way, but it can also be misleading, because (apart
from the example apps) this is NOT a port of the Java SDK; it's based on
the protocol spec.


Notes
-----

As of TiVo software version 20.2, at least, the HDUI no longer requests
the app icons. (The SDUI works as always.) Also in 20.2, there's a bug
in transparency rendering that sometimes causes black to be substituted
for white in text that's not fully opaque.

The last TiVo software revision for the Series 2, 9.3b, still has fatal
bugs in its HME engine under certain conditions, where the text
disappears from the menus (outside HME), or the entire machine locks up.
As a result, the "animate" and "effects" demos now have code to keep
them from running in that environment. You can see the same effects with
the original versions from the Java SDK.


Changes
-------

0.20 -- UTF-8 strings can now be passed directly to HME commands --
previously only ASCII and Unicode strings were accepted. Note
that HME for Python isn't checking for valid UTF-8, so in fact
you can pass anything, and it's up to the TiVo to handle it.

Automatic flush after send_key().

Added KEY_OPT_ZOOM as another alias for KEY_OPT_WINDOW.

Check for existing instances of a title before announcing it,
to avoid stepping on them; append "[2]" etc. as needed. After
the Java SDK.

Added the ability to back out to most sample apps (all that
didn't have it except "hello"), using Left, Clear and/or Pause,
depending on the app. This deviates (further) from the apps
being exact clones of the Java versions, but who cares.

Use the alternate (modern) method of loading PIL's Image
module; prefer "import hme" over "from hme import *" for
examples; minor style cleanups for examples.

In Zeroconf, use the same method of getting the default IP as
in start.py, instead of gethostbyname(gethostname()); minor
style cleanups and code tightening for Zeroconf.

Documentation cleanup.

0.19 -- HME for Python now reports itself as implementing version 0.49
of the HME protocol. This version has not been publicly
documented by TiVo, but the change is necessary in order to
properly support direct text input (including from the Slide
remote). In addition to the new keyboard handling (see above),
there's an added parameter for stream creation: a dict called
"params". None of its possible values are known.

New top-level function, qwerty_map(), to convert the rawcode
from an IRCODE-style keyboard event to its ASCII equivalent.

More key names reported by the test app, with keyboard text
reported from either kind of input.

Initialize MIMETYPES from mimetypes.types_map -- allows HME/VLC
to continue using it.

Flush after a change in stream speed -- gets sounds to play
immediately, even when not immediately followed by an event
check.

0.18 -- More automatic removal of some resources when they go out of
scope: nameless Image, TTF (named ones are cached, and may
be duplicate references to the same TiVo-side resource --
should probably rework this), Text and Stream resources. (Other
resources are still cached.) As part of these changes,
app.resources has been removed.

"Test" app: Add KEY_OPT_A, B, C, and D button names; for key
codes that aren't in the list of names, print the number.

Stupid bug: extra colon in Content-Length header. Revealed by
the new HD UI on the TiVo Premiere.

Use the mimetypes module instead of hardwiring quite so many
MIME types in start.py.

Various minor fixes for Zeroconf (mainly cosmetic).

Icons resaved as RGB for the sake of the new UI on the
Premiere, which can't handle indexed or greyscale PNGs.

0.17 -- Set up a 64K output buffer before initializing. This will cut
down on the number of packets sent, as well as forestalling any
potential "#3-5-6"-style errors, as in pyTivo (though I haven't
observed those in HME). But it also means that you have to
flush the output to be sure all commands are sent. The buffer
is flushed automatically when waiting for events; the only time
this is an issue is when sleep()ing instead of returning from
the event handler. To deal with this, I've added a sleep()
method to the Application class; just use it in place of
time.sleep(). All it does is flush the output, then sleep. You
could also do this manually. The example programs have been
modified to use the new method.

When unpacking an HME dict, if a list contains only a single
item, take it out of list form. This makes it more symmetrical
with what was done for packing in 0.8, although if any actual
one-item lists were packed, the list will be lost in that case,
too.

Added MIME types for .tivo files, and some other video types.

When the address isn't specified, to find it, skip
gethostbyname(), and just use the default route method; use
port 123 instead of 0 (the Mac doesn't like 0).

Supress Zeroconf.py's useless "NonLocalNameException".

0.16 -- Support for alpha values in colors -- apparently this was fixed
in TiVo software 9.3. (?) Reported by TCF user "Allanon". The
way this works is, wherever you previously could specify a
color number in RGB form, you can now do it as ARGB, where "A"
(alpha), the most significant byte, represents the opacity --
_except_ that an alpha value of 0 is treated as full opacity
(equivalent to 0xff), for the sake of simplicity and backwards
compatibility. (So, if you don't want to mess with alpha
values, you can continue using plain RGB values.) Otherwise,
the higher the number, the more opaque.

Removed the note about limits on the size of color resources --
fixed in 9.4 (or earlier?). A color assigned to an HD view will
now fill the whole view.

hmeserver (start.py) now responds to requests for robots.txt,
with no permission. (No good can come of trying to crawl an HME
app.)

Taiwanese TiVos append "?width=704&height=480" to their app
requests. Previously, this would make hmeserver send back a
403. Now, it ignores these parameters, so the app will work.
(And perhaps in the future, hmeserver will actually support
this undocumented feature of TiVo's SDK.)

Added MIME type for WMV video.

Minor changes to demo apps: made a few bits more Pythonic, and
fixed some erroneous spacing that was only apparent with Python
3.0.

Minor changes to the Zeroconf module: untabbed, and removed the
deprecated has_key().

0.15 -- Added clear_resource() and remove_resource() methods for Views.
clear_resource() disassociates the View from its resource,
without removing the resource. remove_resource() is kind of
trivial -- equivalent to "resource.remove(); resource = None" --
but is included for completeness.

Optional config file parsing for hmeserver -- see above.

Moved check for application class from Handler to startup.

Slightly more robust path handling for hmeserver -- works
better in Windows, and the disallowed-directory check is less
kludgy.

Renamed hmeserver.py to start.py. But note that it's still
"hmeserver" for purposes of config.ini.

Support for Python < 2.3 is dropped.

0.14 -- hmeserver now separates the app and data roots, to allow
keeping icons etc. together with their apps, while having data
elsewhere. The new command-line option "--datapath" specifies
the data root, while "--basepath" still sets the app root.
Files outside of app directories (including those in the app
root, which had previously been allowed) are now forbidden
unless the datapath is set.

The initial transparency of a View can now be set via the
"transparency" keyword parameter when creating it, instead of
being settable only by a call to set_transparency().

0.13 -- Added a new method for specifying sounds, by name. (The old
method, by number, will also still work.) The symbolic names
were a bit cumbersome, but I didn't feel that I could shorten
them... hence, this. It's similar to how it works in the Java
SDK, but without the fake ".snd" extension. See examples.

Added a "speed" attribute to the Resource class -- only
meaningful for Streams.

In hmeserver, send size information with regular files where
available; catch and report socket errors on sending regular
files; use log_error() and log_message() instead of print where
appropriate.

Added MIME types for video, since it can now be Streamed.

Added port number to logged address, for help in debugging.

Startup banner for hmeserver; "help" and "version" options.

Removed "sorted()" for compatibility with older versions of
Python (or Jython).

Some new command-line options for hmeserver, to set host, port,
path, and/or zeroconf off, and to allow specific modules to be
named. (See "./hmeserver.py --help".)

The HME server classes now depend only on the values passed
to __init__(), not the globals. In principle, you can import
hmeserver and use the classes from another program.

Skip Zeroconf functions if the module is missing.

0.12 -- Barred effects and animate from all non-S3/HD TiVos running 9.1
or 9.3. This may be overbroad, but I can't confirm that any
S2's can handle these apps with 9.x.

Made some things "private", so they won't be imported by "from
hme import *".

Added ".pyo" to the list of file types not to send.

In the picture viewer, shuffle the pictures instead of sorting
them; also, check not only for a valid directory, but for one
containing pictures.

Miscellaneous internal reworking and simplification.

Tested under more versions.

Typo: had "LPGL" instead of "LGPL".

0.11 -- Added a simple slideshow Picture Viewer to the included apps.
It depends on the Python Imaging Library, and you'll have to
edit picture/__init__.py to set ROOT to an appropriate
directory. The app automatically uses hi-def mode when
available.

Changed the Animation class (and all the "animtime" parameters
of various functions) to take seconds instead of milliseconds,
to make it more consistent with general Python usage.

Small tweak to the put_chunked() function.

0.10 -- Sending data larger than 64K was broken since 0.3. Argh.

Images without names are not cached, but Image.remove() was
still trying to remove them from the cache.

0.9 -- Added a separate set_resolution() function.

Changing the resolution with an already-visible root view
(which is always the situation, since 0.2) requires a
set_bounds() call on the root view.

0.8 -- The dict items for transition() no longer have to be in list
form, although they still _can_ be. (The values returned to
handle_init_info() are still lists, though.)

Simplified the transition demo and added an icon for it.

Print the skipped directories in hmeserver, along with the
reasons they were skipped.

0.7 -- Added support for app transitions -- the last unimplemented
part of the HME specification. (But I still have more to do to
match the Java SDK.) This also entailed support for the HME
"dict" type, which although mentioned in the specification is
not documented. A sample app is included.

0.6 -- Added set_focus() to use when changing focus, instead of
setting self.focus directly. Define a handle_focus() method for
an object if you want it to do something special on a focus
change; handle_focus() should take a single boolean parameter,
which will be False when losing focus and True when gaining it.

Shorter form for sound calls -- because in practice, they're
always ID-based.

Exit (event loop) on receiving an EVT_APP_INFO of active=false.

Flush the sent handshake; exit mainloop() if no valid handshake
is received.

0.5 -- Focus support -- set self.focus to the object you want to
handle events. It need not be a View; it can be any object.
Just add the appropriate handler as an attribute. If a handler
isn't present in the self.focus object, the app's handler is
used. But if you want to pass events on from the object's
handler, you'll have to call the app's handler explicitly.

Default key repeat event behavior is now to call the key press
handler.

Unspecified width and height in a child window need to default
to that of the parent window _minus_ the position.

0.4 -- Narrow the list of exceptions handled when importing -- this
covers non-module directories without masking real errors.

One more event handler -- handle_active(). This can be used
where startup() used to be, i.e., after the startup events.

0.3 -- Absorb all exceptions during reading or writing, allowing
orderly shutdown even if the socket is abrubtly cut off; also,
the use of the term "chunk" was not appropriate -- it should be
reserved for the components of the chunked stream.

Removed self.app_info and self.device_info; added
handle_app_info() and handle_device_info(). With the new
structure (startup(), activate, then start handling events),
something like this is necessary.

Prevent animate and effects from being run on the broken 9.1
software (the others seem OK); print more info about events in
the test app.

0.2 -- Moved root view activation to after startup(), but before event
handling; flush output before checking for input; flush input
after close. This should bring it a little closer to the Java
SDK's behavior, and makes it work under TiVo software 7.2.

hmeserver now prints app names as they're published/unregistered.

0.1.3 -- Skip reverse lookup in hmeserver (thanks armooo).

Include the app name in the Starting/Stopping HME line.

0.1.2 -- The mechanism for skipping non-app directories was broken.

0.1.1 -- Changed default port.

[Now hosted at wmcbrine.com (http://wmcbrine.com/tivo/).]

[jbcooley]

Very cool. It is amazing how much you can do with so little code. My Application.cs file only has a few more lines than your hme.py and yours seems to contain the entire protocol.

I ran this with python 2.4.3 on Vista, and got socket.error: (10013, 'Permission denied'). I haven't tried upgrading to a newer version yet. I just happened to have that version on my machine.

I'll post with more info after I upgrade python.

[jbcooley]

Even with python 2.5.1, I get the same error. Any thoughts?

[wmcbrine]

I'd guess that Vista's not allowing hmeserver to bind to port 8080. Either that, or it's not letting Zeroconf broadcast. I dunno, I don't have Vista to test with.

Edit: Oh, or allowing it to connect to 4.2.2.1. It'll only try that if it can't get a sensible IP from gethostbyname() (as it can't on my Linux system). If that's the problem, you can hardwire the IP.

With the full traceback, I could tell you which it is.

[jbcooley]

Here's the traceback:

Traceback (most recent call last):
File "hmeserver.py", line 217, in <module>
httpd = Server((HOST, PORT), Handler)
File "c:\Python25\lib\SocketServer.py", line 330, in __init__
self.server_bind()
File "c:\Python25\lib\BaseHTTPServer.py", line 101, in server_bind
SocketServer.TCPServer.server_bind(self)
File "c:\Python25\lib\SocketServer.py", line 341, in server_bind
self.socket.bind(self.server_address)
File "<string>", line 1, in bind
socket.error: (10013, 'Permission denied')

[wmcbrine]

Yeah, it's not letting the http server bind to port 8080. So you gotta either do whatever it is one does in Vista to allow that, or change the port in hmeserver.py. (It's just a line that says PORT = 8080, near the top of the file.)

What I'm wondering now is, would this apply for any port in Vista, or is it that particular port that's a problem?

[jbcooley]

I should have looked at this earlier. I have TivoDesktop running and the TivoServer.exe is sitting on port 8080. After stoping TivoServer, I was able to run hmeserver.py.

Do you have a stop key combination that ends hmeserver.py?

[smock9]

I'm just getting into all the home networking options that are available to my new Tivo HD. Does this serve a similar function as pyTivo (http://pytivo.armooo.net/)?

[wmcbrine]

TivoServer.exe is sitting on port 8080.Ah. I guess 8080 was a bad choice, then. :o

Do you have a stop key combination that ends hmeserver.py?Up there... under "Quick Stop"...

I'm just getting into all the home networking options that are available to my new Tivo HD. Does this serve a similar function as pyTivo?No.

This is only for programmers... and, maybe, avid Tic Tac Toe players.

[wmcbrine]

OK, I've chosen a new port (9042) and updated the original post.

[eb3604]

ran it last night. i ran the animation program, and at first it worked, then the screen went black, and random vertical stripes mostly green ones would appear. I got back to the home screen, but there was no text. I could go to the other screens, but no text. Live tv did not have the guide or information bar thing.

[wmcbrine]

Odd. I can only tell you that I've run my version of Animate for several hours without incident.

[armooo]

You may want to add this to your Handler, it prevents large delays if there are not valid reverse DNS entries for your tivo.


def address_string(self):
host, port = self.client_address[:2]
return host

[wmcbrine]

So, you may have noticed lines like this in the sample apps:


Font(self, size=96, style=FONT_BOLD)
Color(self, 0)


and wondered what was going on there. By the capitalized names, you'd guess that they were classes. But nowhere are the instances saved. Well, they are classes, and the instances are saved -- in the caches created by the Application class.

When I first got the resource classes together, they worked, but my code looked like this:


font = Font(self, self.default_ttf, FONT_BOLD, 36)
color = Color(self, 0xffffff)
text = Text(self, 'Hello, world!', font, color)
self.root.set_resource(text)


(and you can still do it that way). Which is fine, I guess; but the Java SDK did the same thing like this:


getRoot().setResource(createText("default-36-bold.font",
Color.white, "Hello, world!"));


and I wanted my module to be easier to use than that, much less what I had at that point.

I made a little progess by adding lots of default parameters -- for example, self.default_ttf is the default TTF object for the Font class, and white (0xfffffff) is the default Color:


font = Font(self, style=FONT_BOLD, size=36)
color = Color(self)
text = Text(self, 'Hello, world!', font, color)
self.root.set_resource(text)


But the real boost came from having the new resources cache themselves in the app. There's a cache for each type of resource (well, almost) that records each one sent to the TiVo, so it doesn't have to be sent again. That means I don't necessarily need to keep track of the return values in the app:


font = Font(self, style=FONT_BOLD, size=36)
text = Text(self, 'Hello, world!', font, Color(self))
self.root.set_resource(text)


The resource for color white is created and cached; the next time I want to use white, "Color(self)" will create a new object, but won't recreate the resource on the TiVo.

Better, but I still wanted to get rid of some of those darn "selfs". So in addition to the overall resource caching, I added records of the last color, TTF and font created. Then I made these the defaults for the Text class. That way, I could have a number of successive Text items without having to specify font or color again until I actually wanted to change them. It also meant that "Font" and "Color" could essentially be used in a declarative way -- they still return instances, but I don't have to explicitly use the return values:


Font(self, style=FONT_BOLD, size=36)
Color(self)
text = Text(self, 'Hello, world!')
self.root.set_resource(text)


And if there was no last color set, then I had it automatically make a resource for color white:


Font(self, style=FONT_BOLD, size=36)
text = Text(self, 'Hello, world!')
self.root.set_resource(text)


Text isn't cached, but since I never wanted to use that text again anyway:


Font(self, style=FONT_BOLD, size=36)
self.root.set_resource(Text(self, 'Hello, world!'))


Finally, I added a View method that takes some text directly, so I can leave out the explicit Text class call:


Font(self, size=36, style=FONT_BOLD)
self.root.set_text('Hello, world!')


And that's today's version of the 'Hello, world!' app. Note that if I wanted size=24 and style=FONT_PLAIN, I wouldn't need to invoke Font at all.

I hope this has been helpful to someone.

[armooo]

I have two patches for you.

The first makes passing app optional if you are passing the parrent to View.

The other prints a traceback if there is an exception while importing a package.

[wmcbrine]

I have two patches for you.Thanks. However...

The first makes passing app optional if you are passing the parrent to View.This was something I'd already done, and then backed out. I felt it was too hairy, and too inconsistent. (If you can omit the self (app) parameter for views, then why not for resources? But you can't.) Instead, I recommend using the child() method -- rather than:

View(self, parent=parent)

just:

parent.child()

In fact, using self.root.child(), you can avoid ever calling View directly.

The other prints a traceback if there is an exception while importing a package.I don't want to do it that way because I want to allow non-module directories to be present, and have them silently ignored. But maybe with a more specific set of exceptions?

On the other hand, perhaps I should at least be printing a list of modules successfully loaded...

[CuriousMark]

I tried 0.1.3 on my Kurobox NAS last night and it worked ok. I see the messed up animation and it leaves my TiVo in an unusual state, but a restart clears that. There are exceptions occurring when this happens, so it is probably something that can be handled with a real application with error handling built in, rather than toy sample apps. If you are interested I can post the exceptions and describe the screen errors if you would like.

On the clock sample, when running under windows XP the text stayed on screen all the time and seconds character faded nicely from one digit to the next, a very sweet effect. On the Debian box the text flashed on with each new second for about half a second and the screen cleared for the other half second. This could be because I ran apps in a different order between the two boxes and the TiVo had different resources available, or a real difference in how python is working between the two platforms.

Let me know if there is something I should be looking at here.

[wmcbrine]

I see the messed up animation and it leaves my TiVo in an unusual state, but a restart clears that.Huh. What kind of TiVo are you using, with what software version? (I only have a Series 3 with 9.2 and the simulator to test in -- no problems there.) Does the same thing not happen with the Java version from TiVo's SDK?

The Animate app is a pretty straight port of the Java version (as is the Clock app), and they're almost identical in what they send to the TiVo. AFAIK, there are only two differences: the point at which the root window is made visible, and the number of Animation resources sent -- fewer for mine, so I'd expect it to be less stressful for the TiVo. But then, the Java version removes some of them... hmm... I wonder if I'm going over a limit there? None is documented, but there probably is one. Mine only sends up to 100 (vs. up to 5000 for the Java version), but with none being removed, maybe that's still too many for some TiVos.

When to make the root visible was an issue... originally I was doing it manually at the end of startup(), which (going by simulator debug output) matched the Java version; but then I wanted it to be automatic (as it was in Java), so I put it at the end of __init__(), which means it happens before startup(). I saw no difference here, but maybe it matters on some TiVos?

If you are interested I can post the exceptions and describe the screen errors if you would like.Yes, please.

On the clock sample [etc.]More weirdness. Again, I develop on an Ubuntu system (= essentially Debian) and have never seen this. How about with the Java version?

I have noticed a difference in the way that Python networking works in Windows and Linux, and something about the way it works in Windows seems to be distasteful to the TiVo, making it report errors every so often even when the data is correct -- I saw this in pyTivo; so far, not with HME. But maybe that's what others are seeing? Although it wouldn't explain your Clock problem.

[CuriousMark]

I have not installed or tried out the Java simulator, but surely can try to do so tonight. First I will replicate what I saw and take some pictures and copy/pastes to show you what I see.

This is going to a Humax DVD burner TiVo, the slowest model out there. I recall JavaHMO making comments about having to add slowdowns to Galleon to keep it from causing problems in a very old thread here. I wonder if that might be part of it. I guess it is too soon to speculate, you will probably find the actual Exceptions more useful than my guesswork.

As I collect information I can post it here, email it, or post it on Armooo's site, whichever is most convenient for you.

BTW, I found I needed to exit most apps by hitting the TiVo button, left didn't seem to do anything. Is that expected, or am I doing it wrong and introducing problems. I will provide a narrative with my results so you can catch mess-ups on my part.

Edit: Humax SW version is 9.1

[wmcbrine]

As I collect information I can post it here, email it, or post it on Armooo's site, whichever is most convenient for you.Here is fine. Email is fine. But this isn't pyTivo, so don't put it there.

BTW, I found I needed to exit most apps by hitting the TiVo button, left didn't seem to do anything. Is that expected,Yes, it's normal. That's just the way TiVo wrote most of their sample apps; and my goal, for illustrative purposes, was to slavishly copy them. :D (I couldn't resist that one attempt at optimization in animate, but that's about the only serious change I made.)

[CuriousMark]

I did some testing, but the results were a little better, and unfortunately less revealing.

Here is my annotated PuTTy log of the test:
=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2008.01.31 20:17:06 =~=~=~=~=~=~=~=~=~=~=~=
Thu Jan 31 20:15:44 2008 Server Stops
### Start HME
Serenity:/home/mark/hme# ./hmeserver.py
Thu Jan 31 20:15:57 2008 Server Starts
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /animate/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /clock/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /effects/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /fontinfo/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /hello/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /test/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:16:11] "GET /tictactoe/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:18:00] "GET /clock/ HTTP/1.1" 200 -
Thu Jan 31 20:18:00 2008 Starting HME: clock
Thu Jan 31 20:21:11 2008 Ending HME: clock

###See pictures showing clock working correctly
###Returned to TiVo Central using TiVo button, all is good

192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /animate/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /clock/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /effects/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /fontinfo/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /hello/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /test/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:28] "GET /tictactoe/icon.png HTTP/1.0" 200 -
192.168.2.101 - - [31/Jan/2008 20:21:37] "GET /animate/ HTTP/1.1" 200 -
Thu Jan 31 20:21:37 2008 Starting HME: animate
Thu Jan 31 20:24:03 2008 Ending HME: animate

### See pictures showing mostly blank screen with occasional tall rectangle flashing briefly
### Hit TiVo button, no text on TiVo Central, had to restart TiVo
### ARGH --- Smallville was recording

Thu Jan 31 20:26:21 2008 Server Stops
Serenity:/home/mark/hme# exit
logout

Photos of the screen are here (http://s221.photobucket.com/albums/dd90/MarkEvanetich/TiVo/)

So if I run clock first, it works well, not like it did when I ran other samples first last night. The middle picture shows the last digit fading from 0 to 1, a very nice effect.

I then ran animate and the screen showed the corect animation for about a second and then blanked showing an occasional tall rectangle. I took a long sequence of shots at about 2 per second to capture one of the rare appearances of a rectangle. They were appearing every 5 to 10 seconds, in different locations on the screen, but mostly to the right side.

When I exited the TiVo had lost fonts and rebooted. Luckily Smallville is also being recorded on the other TiVo too.

I will try some more later, just thought I would share what I have now. As you can see, there were no exceptions in the code this time around.

CuriousMark

[wmcbrine]

Photos of the screen are here (http://s221.photobucket.com/albums/dd90/MarkEvanetich/TiVo/)It's password-protected.

[CuriousMark]

Sorry about that.

[CuriousMark]

Image showing clock changing seconds with a nice fade between digits. I had reported this not working before, but it works fine now, assuming clock is run before an example that messes with the TiVo.
http://i221.photobucket.com/albums/dd90/MarkEvanetich/TiVo/hmetesting006.jpg

Image showing screen during animate, after it breaks down and stops showing the moving squares.
http://s221.photobucket.com/albums/dd90/MarkEvanetich/TiVo/hmetesting015.jpg

Image showing all fonts gone after TiVo Button escape from failed animate run.
http://i221.photobucket.com/albums/dd90/MarkEvanetich/TiVo/hmetesting019.jpg

Hopefully these won't require a password.

I will try the modified the versions you emailed me tonight and let you know how they perform.

[wmcbrine]

I wonder if I'm going over a limit there?Well, I just checked with the simulator again, and the Java version of Animate sends somewhere over 576 animations before removing any (because that's where I stopped it, with no removals).* So the potential explanations are down to a) activating the root view too soon, or b) still unknown.

If you have the problem, and you want to test 'a', then take the line in hme.py that says "self.root.set_visible()", and move it down a few lines, to just under the call to "self.startup()". Also, if you can, try the Java version and tell me if you see the same problem.

* Edit: On that run. Then only 74 on the next. (And the simulator was restarted in between.) I wish I could figure out the logic behind that.

[wmcbrine]

It's become apparent from this thread that I need a Series 2 to test on. So, I've just ordered one. It should be free after rebate, but there's still the service charge. Since I only need it for this project, if you'd like to help defray the cost, please consider donating via PayPal to the address in the documentation. (That's this same username at gmail.com.)

Just ten people donating $10, or twenty donating $5, would pay for my year's subscription.

Current donations: $102.00

Thanks!

-------------------------------------------------------------------

Update on January 12, 2009 (with donations at $102):

I've met my original goal. However:

- I also spent $25 on a TiVo Desktop Plus license, just so I could study its "push" capability to try and get it into pyTivo. (A couple of the donations I've already received were actually specified as for this, so technically I haven't quite met my original goal after all.)

- My year's worth of service on the Series 2 is about to expire. I still use it almost exclusively to test my HME projects and pyTivo.

- My old laptop has just died, which means I no longer have a working Windows system (just Linux and Mac). Even before that, I didn't have a Vista system. I'm a Linux guy, so I wouldn't care for myself, but I need Windows to test on to support you, the users.

Minor points:

- My Mac is a G4 Mini running 10.4. I can't test Intel or 10.5. But I expect that doesn't matter all that much.

- Even my main Linux system has shown signs of ailing recently. I list this as a minor point because I'll replace it regardless of donations.

So, for all these reasons, your contributions are still welcome. :)

New donations: $140.00 and a Windows computer!

Thanks!

[CuriousMark]

Sorry to hear that. Even a 540 or a dual tuner might work better than my 595 so I will keep trying to help as best I can.

[wmcbrine]

eb3604 has a 2 DT and has the same trouble. So it's got to be the Series 2 hardware and/or version 9.1 (since S3's and HD's are on 9.2 and don't have the problem).

Anyway, even solving that, it probably won't be the last time having a Series 2 around would be useful for testing.

No donation too small! :)

[windracer]

Soooo .... who's going to port the Galleon apps to Python? ;) pyTiVo already supports music and photos, so all I need is the weather app and I can dump java. :up:

[wmcbrine]

I have many plans for this library, but first I want to get it working solidly on S2's, or else rule out that possibility (if the same errors appear with the Java SDK). So for now, I'm working on pyTivo while I wait for my S2 to arrive. (Of course the S2 should help with pyTivo development as well; but this is the thing I need it for.)

[refried]

Are you using a version control system (CVS, SVN, git, etc)?
Do you plan on having a public repository?

[wmcbrine]

Are you using a version control system (CVS, SVN, git, etc)?Yes, I have it in git on my own system.

Do you plan on having a public repository?If people show sufficient interest. So far, I'm not blown away by the response. That's OK; I needed it for my own purposes, and additional users are just a bonus. But it doesn't make me want to bother hosting it anywhere.

[s2kdave]

Soooo .... who's going to port the Galleon apps to Python? ;) pyTiVo already supports music and photos, so all I need is the weather app and I can dump java. :up:

Not me. :D If anything I'll componentize a couple of them that I use for the Catnip (http://www.tivocommunity.com/tivo-vb/showthread.php?t=383429) server I started working on. But I really don't want to touch the beast that is Galleon much more.

Have fun whomever wants to try and port it. :rolleyes: It would be easier/cleaner to just rewrite it.

[wmcbrine]

So, my new 540 came today. It came with 7.2, which was interesting -- my demo apps didn't come up at all; the screen just sat at "Please wait..." It turns out that, in that version, the TiVo won't send the "active: true" message until after it's already received something more than the handshake from the app. That was easily fixed -- I'm just not waiting for that anymore -- but it broke my original design to have resolution changes handled before startup(). Ah well. Anyway, that done, all the apps worked fine, including Animate and Clock. I'll see what happens once it upgrades to 9.1. There was some odd behavior with pyTivo, too, that I haven't fully characterized yet.

[wmcbrine]

OK! Here's the story: It's 9.1. I ran Animate for several hours under 7.2; no problem. Upgraded to 9.1 -- saw what others have reported. Then I tried the original Java version: exactly the same problem. So, there's another one for the 9.1 bug list. I'm surprised no one has reported it before.

I can only hope and expect that TiVo will fix it with their next software release for the Series 2, since it doesn't happen in either 7.2 or 9.2. But it happens in 9.1 regardless of the language or library used.

I'll be attaching a new version to the first post shortly.

[CuriousMark]

Thanks, glad to know it isn't just me.

[wmcbrine]

Some further experiments with my 540: I tried forcing a short pause after each command sent. That didn't fix it, although it did make a difference -- I could see a little bit more of the rectangles, and the menu text didn't disappear. I extended the pause all the way to "don't send the next command until I press a key," but it didn't improve further.

I found that Effects caused the same problem as Animate -- not too surprising, since it also has many views in constant motion. One interesting thing here: when I inserted a delay, the "Visible" test actually worked (on my S3), for the first time (i.e., that square disappeared and reappeared). This hadn't worked even in the original Java version.

Then I took out the pause, and tried reducing the number of sprites instead. That was interesting. With one sprite, no problems. Two also seemed fine. With four, the first run seemed to freeze, although there was no visual distortion -- just four rectangles stopped. At first I assumed that the TiVo had frozen, but it responded just fine to the TiVo key. So I ran the four-square Animate again, and it seemed to work OK. Then I upped it to eight. Now I could see what I'd missed with four -- it actually started out normal, and only gradually came to a halt, with some squares still moving (based on their last animation, I assume) while others had already stopped; eventually, they all stopped. This happened every time with eight, although there was still no distortion or loss of text. Then I tried scaling it back down and letting it run a long time. Even with just three, after a few hours, I got the distortion and lost text.

Anyway, there doesn't seem to be anything I can do about it, but I kinda knew that already.

Instead, I offer you the next version of HME for Python. It adds two new event handlers (they're needed now, since startup() runs before the startup events are handled), and incidentally makes use of one of them in the demos to keep you from running Animate or Effects on a 9.1 system. (The others seem to be safe.) It also suppresses any exceptions during reading or writing the stream, since the socket can close at any time, but you still might need to run cleanup(). Plus it looks neater that way.

[jbcooley]

I tried an older version of your software and only got an occasional artifact on my Series 2 240. I got similar artifacts from the java animate sample and my animate sample. None of them affected the usability of my tivo. This is all with 9.1.

[wmcbrine]

New version -- adds handle_active(), which handles EVT_APP_INFO events of "active = true"; and better import exception handling in hmeserver.py -- empty directories are silently passed over without masking errors in real apps.

[refried]

I'm trying out the test app on my Humax DVD Recorder (595) and it looks like the optional keys aren't coming back with the right key codes. Here are the key names, the spec codes, and what I actually get.

KEY_OPT_STOP 51 190
KEY_OPT_MENU 52 179
KEY_OPT_TOP_MENU 53 178
KEY_OPT_ANGLE 54 182

Odd, it looks like the vint parsing is correct and the packets coming over the wire sure use two bytes to specify the key number.

[refried]

I tried out the 1.4.0 simulator and the 1.4.1 experimental simulator. The 1.4.0 gets the key codes right. The 1.4.1 simulator can't run the test app. I get this:
WARNING: resource 2052 not found (type class com.tivo.hme.sim.SimResource$FontResource) [3]
WARNING: resource 2055 not found (type null) [3]

[wmcbrine]

Turn on the debug output; it's more interesting. The 1.4.1 simulator is seriously broken. It gets a NullPointerException when trying to create resource 2052 (the font), which in turn is why resource 2055 (the text) fails. You can see the same behavior even when running the Java SDK's "Hello World" sample app (either 1.4 or 1.4.1 version). There are other problems with 1.4.1, too. But, I guess that's why it's labelled "experimental".

As for the key codes... they are what they are. If the real TiVo is sending out codes that don't match the spec, that's regrettable. But even the spec says not to rely on the KEY_OPT_* codes.

In a way, I'm encouraged to be receiving bug reports about things that would already be apparent with the Java SDK -- because it means that people are picking up HME for Python who never tried the Java SDK.

[wmcbrine]

Version 0.5 posted -- this adds focus support, fixes a bug with the default sizing of child windows, and eliminates the need for a separate handle_key_repeat(), if you just want a repeat to work like a press.

[CuriousMark]

I found that Effects caused the same problem as Animate -- not too surprising, since it also has many views in constant motion. One interesting thing here: when I inserted a delay, the "Visible" test actually worked (on my S3), for the first time (i.e., that square disappeared and reappeared). This hadn't worked even in the original Java version.

Then I took out the pause, and tried reducing the number of sprites instead. That was interesting. With one sprite, no problems. Two also seemed fine. With four, the first run seemed to freeze, although there was no visual distortion -- just four rectangles stopped. At first I assumed that the TiVo had frozen, but it responded just fine to the TiVo key. So I ran the four-square Animate again, and it seemed to work OK. Then I upped it to eight. Now I could see what I'd missed with four -- it actually started out normal, and only gradually came to a halt, with some squares still moving (based on their last animation, I assume) while others had already stopped; eventually, they all stopped. This happened every time with eight, although there was still no distortion or loss of text. Then I tried scaling it back down and letting it run a long time. Even with just three, after a few hours, I got the distortion and lost text.

Anyway, there doesn't seem to be anything I can do about it, but I kinda knew that already.
I notified Jerry, who is passing the information along to the right people. I also played with skull and bones on my TiVo and it plays and works fine, although it never has more than one thing in motion at a time. Perhaps I need to play it for hours to see if things start getting bad.

I will try 0.5 tonight.

[wmcbrine]

Of all the demo apps, only those two cause a problem. Skull and Bones is not at all a demanding app. You can probably play it indefinitely.

Anyway, thanks for talking to TiVoJerry. I think we can drop this subject now, since it's a general HME problem, not an HME for Python problem.

[wmcbrine]

New version 0.6 refines the focus handling a bit, makes sounds a little easier, and recognizes a couple more exit conditions.

[wmcbrine]

Version 0.7 supports app transitions. I believe this is the last part of the HME protocol specification that I hadn't implemented. (Of course there's still more work to do in other respects.)

[wmcbrine]

0.8 -- The dict items for transition() no longer have to be in list form, although they still _can_ be. (The values returned to handle_init_info() are still lists, though.)

Simplified the transition demo and added an icon for it.

Print the skipped directories in hmeserver, along with the reasons they were skipped.

[tlc]

Would this support HD-res photos? HMO and pyTivo don't, right?

[wmcbrine]

If you want to write an HD photo viewer using this library, then yes. Out of the box, no. (BTW, I have an HD-related fix to post...) I'll probably write one myself, but don't wait for me.

HMO per se is neutral about resolution. The HMO photo viewer built into the TiVo is strictly SD, but the "HD Photos 2.0" HME app in TiVo Desktop uses an HMO server as the backend -- and it will even work with pyTivo. (Getting it to run outside of TiVo Desktop is a little tricky, but doable.)

Edit: Simplest possible HD photo viewer app:


import hme

class Picture(hme.Application):
def handle_resolution(self):
return (1280, 720, 1, 1)

def handle_active(self):
self.root.set_image('/your/pic/here.jpg')

[tlc]

HMO per se is neutral about resolution. The HMO photo viewer built into the TiVo is strictly SD, but the "HD Photos 2.0" HME app in TiVo Desktop uses an HMO server as the backend -- and it will even work with pyTivo. (Getting it to run outside of TiVo Desktop is a little tricky, but doable.)

Are you saying HD-res photos are possible with pyTivo and without TiVo desktop? Or would you still need the TiVo Desktop to tell TiVo to request HD photos from the HMO server?

Edit: Never mind, I get it. Tivo <-> HD photos 2.0 (HME) <-> HMO server.
HMO is neutral, but we can't get the Tivo to request hi-res.

[wmcbrine]

0.9 -- Added a separate set_resolution() function; changing the resolution with an already-visible root view (which is always the situation, since 0.2) requires a set_bounds() call on the root view.

Sorry for updating again so soon.

[wmcbrine]

0.10 -- Stupid bugs: sending data larger than 64K was broken since 0.3; images without names are not cached, but Image.remove() was still trying to remove them from the cache.

[wmcbrine]

0.11 -- Added a simple slideshow Picture Viewer to the included apps. It depends on the Python Imaging Library, and you'll have to edit picture/__init__.py to set ROOT to an appropriate directory (because I didn't want to bloat the archive with sample pictures). The app automatically uses hi-def mode when available.

- Changed the Animation class (and all the "animtime" parameters of various functions) to take seconds instead of milliseconds, to make it more consistent with general Python usage. I hope this doesn't inconvenience anyone.

- Small tweak to the put_chunked() function.

But I guess people are losing interest, judging by the number of downloads.

[gonzotek]

0.11 -- Added a simple slideshow Picture Viewer to the included apps. It depends on the Python Imaging Library, and you'll have to edit picture/__init__.py to set ROOT to an appropriate directory (because I didn't want to bloat the archive with sample pictures). The app automatically uses hi-def mode when available.

- Changed the Animation class (and all the "animtime" parameters of various functions) to take seconds instead of milliseconds, to make it more consistent with general Python usage. I hope this doesn't inconvenience anyone.

- Small tweak to the put_chunked() function.

But I guess people are losing interest, judging by the number of downloads.

I'm tracking your updates, I just don't have a lot of time to play, unfortunately. I would like to use HME for python for a few projects I have in the back of my mind, when I can find the time to devote to them.

[CuriousMark]

But I guess people are losing interest, judging by the number of downloads.

Not at all, I just am not able to spend much time on it right now. I think I am still at 0.5 for my playing around and learning about event driven programming using python and hme. I have only done procedural scripts up to this point and have a lot to learn. In addition to that I probably will put more effort into fixing pytivo on my NAS first. It isn't as stable there as it is on a PC, and I fear it may be resource bound. I am trying to learn enough about Linux to try to look into that. But that is for another forum.

This is great stuff, amazing to me really.

[refried]

But I guess people are losing interest, judging by the number of downloads.

Downloading zip files off a web forum is more effort than typing "git pull." ;)

[CuriousMark]

I then ran animate and the screen showed the corect animation for about a second and then blanked showing an occasional tall rectangle. I took a long sequence of shots at about 2 per second to capture one of the rare appearances of a rectangle. They were appearing every 5 to 10 seconds, in different locations on the screen, but mostly to the right side.

When I exited the TiVo had lost fonts and rebooted.
I can happily report that the new 9.3 software on my Humax DVD burner TiVo fixes the issues I was seeing in HME, it all works correctly now.

[wmcbrine]

I can happily report that the new 9.3 software on my Humax DVD burner TiVo fixes the issues I was seeing in HME, it all works correctly now.Well, I wish I could say the same for my 540. I still see it crashing in Animate, leaving me textless.

It works OK for the first few seconds... which might be an improvement. But fundamentally, it's not fixed. :(

[CuriousMark]

Bummer, I only tested it for about 10 or 15 seconds, I guess I should have tried for much longer. It was so nice seeing it behave correctly for the first time ever. Try it again after a day, just in case indexing is pushing it over the edge.

[wmcbrine]

Try it again after a day, just in case indexing is pushing it over the edge.Yeah, I thought of that. The thing is, it well may be performance-related as to whether it shows up or not, but it should not happen, under any circumstances. It should not even be possible to send anything over HME that ever crashes your TiVo, or kills any of its functions. The whole point of the design of HME is to keep things safely sandboxed. So it's failing in a big way.

[wmcbrine]

I will say that "Transitions" seems to be working better in 9.3; I haven't had it lose its place yet. But "Effects" still crashes immediately for me.

[wmcbrine]

Bummer, I only tested it for about 10 or 15 seconds, I guess I should have tried for much longer.So, have you tried it again?

I finally got 9.3a on my S3 today, and not surprisingly, everything works fine there, as it did before. (I did have an instance of the icons disappearing, though, requiring an hmeserver restart.) So at this point, I'm thinking to make the Animate/Effects check fail for anything that's not an S3 or HD, running 9.x -- or maybe, being optimistic, just 9.1 and 9.3.

[jbcooley]

Any chance of seeing this run on Google App Engine? I think that would be cool to be able to host your tivo apps on google servers. I wish there were a similar service I could use for my .NET SDK.

[jbcooley]

Any chance of seeing this run on Google App Engine?
Sorry, forgot the link.
http://code.google.com/appengine/docs/whatisgoogleappengine.html

[wmcbrine]

Any chance of seeing this run on Google App Engine?I'll look into that... I've tried to get it to run as CGI, but couldn't quite make that work, at least not with Apache. I think it should be possible with mod-python, but so far I haven't got that working either.

[s2kdave]

Yeah, that would be awesome if it does work. The biggest problem is that you can't open your own socket though or access the file system. So you are at the mercy of the web server that google provides. HME keeps a HTTP socket open/connected as long as you are using the app. I can see the google app engine killing the socket after a certain time period.

[jbcooley]

I can see the google app engine killing the socket after a certain time period.

This is part of the problem I had with IIS.

[s2kdave]

This is part of the problem I had with IIS.

That's the exact problem I had to solve with the Apache Tomcat Catnip extensions I made too. Any respectable web server will kill the connection after a certain time period.

[wmcbrine]

0.12 -- No big stuff; mostly internal reorganization. Here's hoping that the next release will support video streaming. :)

[jbcooley]

0.12 -- No big stuff; mostly internal reorganization. Here's hoping that the next release will support video streaming. :)

If you've got some protocol docs on that, please share. :)

[wmcbrine]

I don't. I'm hoping it's easy to reverse-engineer... or that TiVo publishes an updated spec...

[s2kdave]

I don't. I'm hoping it's easy to reverse-engineer... or that TiVo publishes an updated spec...

It's not going to be an easy reverse engineer. Swivel search uses 2 way SSL certificates (albeit self signed) and I'm willing to bet youtube will do the same. But I definitely am interested in video streaming too so if you do figure it out. please share. :-)

[PaulS]

It's not going to be an easy reverse engineer. Swivel search uses 2 way SSL certificates (albeit self signed) and I'm willing to bet youtube will do the same. But I definitely am interested in video streaming too so if you do figure it out. please share. :-)

Not necessarily. There's no reason to suspect that's the case.

Swivel Search allows proprietary comms between a TiVo unit and one or more servers at TiVo headquarters. These comms include accessing (at a minimum) guide data from Tribune that TiVo licenses, and is likely obligated to protect from intrustion. I can definitely understand why TiVo would want to lock that down.

YouTube features open API's that anyone can code against, as far as I can tell. Why would TiVo or Google need to encrypt these sessions ?

We'll find out soon enough. Someone over on the pyTivo forums is going to set up a capture and see exactly what's going on with the YouTube HME streaming.

[wmcbrine]

It's not going to be an easy reverse engineer.Then again, maybe it is. (http://pytivo.krkeegan.com/h-264-t397-45.html#3117) :D

Or maybe that's more educated guessing then reverse engineering... anyway, it works.

[s2kdave]

Sweet! you rock.

[s2kdave]

I just tested this equivalent java code and it works. There is one issue though. It has a long pause before starting the video after the please wait goes away (about 13 seconds). I'm guessing it's probably downloading the entire video before starting rather than streaming it. I tested this with the 1.4 HME SDK.

public class TestVideo extends BApplication {

@Override
public void init(IContext context) throws Exception {
super.init(context);

getRoot().setResource(createStream("http://192.168.1.114:7288/video/test.mp4"));
}

public static class TestVideoFactory extends Factory {

@Override
public InputStream getStream(String uri) throws IOException {
if (uri.endsWith("test.mp4")) {
return new FileInputStream("skatedog.mp4");
}
return super.getStream(uri);
}

}
}

[wmcbrine]

I would say it's buffering, but not downloading the whole video. You can see a similar effect in the YouTube app. I got HME to play (well, play maybe the first half of, before I got an error) an hour-long MPEG-2 video, and it still only took ten seconds or so to start.

Edit: Actually, I take it back -- the recording was only 23 minutes (only the first part of an hour-long program), and it was played back completely. The error I was seeing was due to premature closure of the second request the TiVo made for the file... why it made a second request is another issue.

[ebdavison]

I love this idea -- HME from python. I have used Tivo ToGo quite a bit in the past but it has been "broken" lately and has not been downloading anything. I would like to add this feature to your hmeserver but am not sure how to even start. Do you have an app like this already? if not, any pointers on how I would go about getting a start on this?

[wmcbrine]

TiVoToGo = HMO. It's outside the scope of HME. You may want to look at pyTivo. Or am I misunderstanding your question?

[ebdavison]

TiVoToGo = HMO. It's outside the scope of HME. You may want to look at pyTivo. Or am I misunderstanding your question?

Maybe it is my misunderstanding. I do not know the difference between HMO and HME, actually.

But, I was wanting to implement something like TivoToGo to download shows. Maybe pyTivo is the right solution, I do not know. I did not see this sort of capability in it though.

And downloads from the Tivo via the web interface are PAINFULLY slow. I mean like 12k - 250k every 30 seconds. A download of a 30 min show (approx 600Mb) has taken 28 hours so far to download 280Mb. This is using a bash script that I cooked up. galleon just quit the download after about 2mb every time I restarted the server.

[windracer]

But, I was wanting to implement something like TivoToGo to download shows. Maybe pyTivo is the right solution, I do not know. I did not see this sort of capability in it though.
It's part of the admin module. See this thread on the pyTiVo forums:

http://pytivo.krkeegan.com/release-2008-03-23-new-feature-togo-t204.html

[ebdavison]

I see the files but do now know how to download from the git hosting site. I have git installed on my linux box but that is all I know about it. Any pointers?

[wmcbrine]

Getting back on topic:

0.13 -- So as it turns out, version 0.12 (and prior) was already capable of video streaming (to an S3/HD with 9.4). :) This version focuses on improvements to hmeserver.py, allowing you to specify apps to run on the command line, along with other options.

[Allanon]

I found when using HME Python 0.13 and setting the base path to 'c:/' in windows the Tivo can't find the program's icons. It doesn't prevent the programs from working but no icon is displayed.

[wmcbrine]

From the point of view of hmeserver, the icons are just regular files; so, they have to be in the correct location under the basepath (basepath + program name + "icon.png", e.g., "c:/test/icon.png").

[Allanon]

I found my problem I edited the HMEServer.py file and changed self.basepath = basepath to self.basepath = 'c:/' so I wouldn't need to use the -b option but I didn't notice that you did other checking for the base path so if I use the -b option it does work correctly.

[wmcbrine]

hmeserver now separates the app and data roots, to allow keeping icons etc. together with their apps, while having data elsewhere. The new command-line option "--datapath" specifies the data root, while "--basepath" still sets the app root. Files outside of app directories (including those in the app root, which had previously been allowed) are now forbidden unless the datapath is set.

The initial transparency of a View can now be set via the "transparency" keyword parameter when creating it, instead of being settable only by a call to set_transparency().

[wmcbrine]

I never did properly answer the question about Google Apps.

I can see the google app engine killing the socket after a certain time period.Indeed, that's what it does:

Application code only runs in response to a web request, and must return response data within a few seconds. A request handler cannot spawn a sub-process or execute code after the response has been sent.So, I'm afraid Google Apps won't work as an HME host. :(

[wmcbrine]

Some changes that I found useful now that I'm seriously working with this library (in HME/VLC) -- mainly, config file parsing for the server (which is now called "start.py", although I still expect to refer to it as "hmeserver").

[noseph]

I am a noob to this. I see references that we are able to stream video to a Series 3 with this app. What configuration is required to accomplish this? I have MPG4s on my system under v:\video. How would I access them?

[wmcbrine]

No, you want HME/VLC (see my sig below). HME for Python is for developers.

[Allanon]

I am loading an image like this:


self.screen = View(self ,0, 0, 640, 480,visible=True,parent=self.root)

self.screen.child(0, self.height, self.width, self.height, image='TEST.PNG',visible=True,id=ID_CLIENT+1)


After the image is displayed on the Tivo I overwrite the TEST.PNG file with a new TEST.PNG file that is the same width and height. Can you show me the best way to update that image on the Tivo?

I am currently using the following code but is there a better way?

Image(self.app,name='TEST.PNG').remove()
self.screen.child(id=ID_CLIENT+1).remove()
self.screen.child(0, self.height, self.width, self.height, image='TEST.PNG',visible=True,id=ID_CLIENT+1)

[wmcbrine]

Here's what I'd do...

self.screen = View(self ,0, 0, 640, 480,visible=True,parent=self.root)becomes:
self.screen = self.root.child()

Everything else is the default, so you don't need to specify it.
self.screen.child(0, self.height, self.width, self.height, image='TEST.PNG',visible=True,id=ID_CLIENT+1)could be:img_view = self.screen.child(image='test.png')

(Same notation. Actually, wait -- you have self.height as the ypos. Were you planning to do an upwards wipe? Also, height and width aren't standard attributes of an Application instance, so I don't know how you've defined these. But, the rest still applies -- you don't need to specify visible, and you should not specify id.) We'll use the name img_view to keep track of the new view rather than trying to manipulate its id.

Now, I'm assuming here that you have a reason for wanting the image to be in a child view of self.screen -- namely, that self.screen will have its own resource assigned to it elsewhere -- otherwise, there's no reason to use two views, and both lines could be combined to:
self.screen = self.root.child(image='test.png')
For that matter, if you're not going to assign a resource to self.root, you could just do this:
self.root.set_image('test.png')
Note that the "id" parameter is meant for use with the built-in resources. I'd never use it for client-created resources.
Image(self.app,name='TEST.PNG').remove()
self.screen.child(id=ID_CLIENT+1).remove()
self.screen.child(0, self.height, self.width, self.height, image='TEST.PNG',visible=True,id=ID_CLIENT+1)could be (following the above):
img_view.remove_resource()
img_view.set_image('test.png')
I see no reason to destroy the old view and create a new one -- you can just assign the new image to the old view -- but if you did want to, it could be done like this:
img_view.remove_resource()
img_view.remove()
img_view = self.screen.child(image='test.png')
There are circumstances in which you'd replace an old view with a new one, but generally that would be for the purpose of doing a transition between them -- fade, wipe, etc.

And if you really, really need to find a child of self.screen without having given it a name at creation time, you can use the self.screen.children list:
self.screen.children[0].remove()(equivalent to img_view.remove() in the above).

[Allanon]

Thanks for the help, here is a little insight in to what I'm doing. I have a large picture that gets updated frequently that I want to display but Tivo can't display it all as one image so I broke it into 14 images that are 1024 x 240. I add all the images as children to the view so when I scale or scroll the view it moves or scales all the child images at the same time. I paired down my example in my post so it would be easier to understand. I have it working but had a feeling that I wasn't using the API correctly so I thought I would ask for the proper way to do it. Thanks again for the help.

UPDATE:

This is the code I am now using to load the images the first time:

self.screen = self.root.child()
for i in range(1,15):
self.screen.child(0, self.height*(i-1), self.width, self.height,image="TEST %i.PNG" % i)


And this is the code I am using to update the images:

for i in range(1,15):
self.screen.children[i-1].remove_resource()
self.screen.children[i-1].set_image("TEST %i.PNG" % i)


This code works good and it's much easier to read than what I had before. I can move and scale self.screen and all the images will move and scale.

[Allanon]

This code is from the Color class code found in the hme.py file:

def __init__(self, app, colornum=None):
if colornum is None:
colornum = 0xffffffff
# I set the alpha to full opacity here for the sake of the
# simulator.
colornum |= 0xff000000
self.colornum = colornum
if colornum in app.colors:
Resource.__init__(self, app, app.colors[colornum].id)
else:
Resource.__init__(self, app)
self.put(_CMD_RSRC_ADD_COLOR, 'r', struct.pack('!I', colornum))
app.colors[colornum] = self
app.last_color = self


I'm just wondering why the default code disables the alpha value making semi transparent views not work? Using an alpha value in the simulator also works so why disable it?

[wmcbrine]

Using an alpha value only works in the simulator. (See the comments just above the code you quoted.) I mask it out so you get the same results in the simulator as on a real TiVo, and so you only have to specify the three color bytes, instead of having to preface every color with an ff (full opacity) for the alpha.

Semi-transparent views work fine, BTW. You just set the transparency via the "transparency" option when creating the view, or via the set_transparency() method later -- not via a color resource. See the "test" app for an example. This is how it's done wherever you see that effect in an HME app.

Edit: I just tested this again now, and it's not working the way I remember -- which is to say, alpha in color resources apparently is working now. Perhaps this was a change in a recent TiVo software update. Or perhaps I'm misremembering... the way I remember it, when I was first writing HME for Python, I had the alpha part unused (i.e., set to zero), and everything looked normal on the TiVo, but blank in the simulator.

I'll revisit this in the next version.

[Allanon]

I commented out the colornum |= 0xff000000 code and I'm able to create a semi transparent view using an alpha value on both the Tivo HD and simulator.

[wmcbrine]

Yes, as I said, it seems to work now. Thanks for bringing it to my attention.

BTW, I should point out that, although it's supported, the TiVo really hates transparency. I quote from the HME SDK Developer's Guide:

Hardware rendering constraints include the following:

• Maximum of 8 images per scanline (a scanline is a horizontal line on the display)
• Maximum of 3 images per pixel (4 on some systems)
• Maximum of 56 bits of depth per pixel (128 on some systems)

Note: The maximum depth of 56 bits per pixel means that placing two transparent images on top of each other results in software rendering for the overlapping portions. (Transparent images — which includes any image with at least one transparent pixel or any GIF image — require 32 bits each. Two transparent images would thus require 64 bits, which exceeds the limit of 56 bits per pixel.)

Workarounds

To avoid software rendering, use opaque images instead of transparent images. Opaque images require 16 bits and are faster to decode and render. Using transparent images to create shading and texturing effects is costly in this environment and should be avoided.

This probably doesn't apply to the S3/HD, but it still does on the S2. I'm not sure how this translates to non-image resources, though.

[Allanon]

There is an odd problem with this test code:


self.screen = self.root.child()

for i in range(0,10):
self.screen.child(0, self.height*i, self.width, self.height,image="TEST %i.PNG" % i)

for i in range(0,10):
self.screen.children[i].remove_resource()
self.screen.children[i].remove()


The first loop loads all the images without a problem but the second loop stops at the remove_resource() line with a "list index out of range" error when i = 5. Another odd thing is if I set the second loop to stop at 4 so the error doesn't occur then it show that every other image was removed instead of the first 4 images.

Any idea why this might occur?

[wmcbrine]

Yeah, the problem there is a Python thing -- if you remove items from a list while you're iterating over it, it gets confused. (The remove() method of Views also removes them from their parents' "children" lists.) You can get around that by using a copy of the list -- maybe something like this:

for child in self.screen.children[:]: # [:] makes a copy
child.remove_resource()
child.remove()

[Allanon]

Thanks for the fast response, that seems to have worked. :)

[lelele]

Does Apps.tv allow python apps? I think it's self-hosted so it shouldn't matter right?

[wmcbrine]

Yeah, you host it yourself (however you want to arrange that), and just give them the URL. There's a Python app there right now: my Reversi game.

Hosting is the hard part. Since I've so far been unable to get HME working as CGI (and am increasingly pessimistic about the prospect), you pretty much have to be able to run start.py as a server and bind your own port. This means a more expensive plan. For now, I'm just running Reversi from my own PC.

[wmcbrine]

Some changes that I found useful now that I'm hosting an apps.tv game, and support for alpha values in colors.

[Joe Q]

Quick Start
-----------

In Windows:

python start.py

This will serve the example apps. The default port is 9042 (not TiVo's
7288). To see more options, run "./start.py --help".




I am a programmer but have never used python. I mostly do C/C++.


I know I am missing one little step.

From your quick start, start.py runs and is obviously sitting there waiting to be told something after it displays "Server Starts".

So,I go into the animate directory,for example, and try __init__.py but all I get from any of the apps, is this:

D:\Tivo\pytivo\hme-python-0.16\animate>__init__.py
Traceback (most recent call last):
File "D:\Tivo\pytivo\hme-python-0.16\animate\__init__.py", line 13, in <module
>
from hme import *
ImportError: No module named hme

I assume then that I am supposed to also run hme.py but when I do, all it does is simply exit back to the command line.

What am I doing wrong or what step am I missing?

Thank You

[wmcbrine]

From your quick start, start.py runs and is obviously sitting there waiting to be told something after it displays "Server Starts".No, it's just waiting for requests from a TiVo (or the simulator), which will be shown in that window. There's nothing more for you to type. Just go to your TiVo and pull up the Music, Photos & More Showcases menu.

[Joe Q]

No, it's just waiting for requests from a TiVo (or the simulator), which will be shown in that window. There's nothing more for you to type. Just go to your TiVo and pull up the Music, Photos & More Showcases menu.


Thank you.

Turns out that it was NOT properly running.

I was not sure what I was supposed to see so your reply was all I needed.


After you told me this, I made some change to the config.ini file and I am now getting the printfs of the HTTP connections.

I see the various apps on the Tivo now.




The kicker is that I did not save the config.ini and I edited it and NOW I do not get any connections.

[Joe Q]

I am not sure what I have done but the server no longer seems to be connecting to the Tivo.

When I run the server, it says that it is registering the various apps but I no longer get print statements that says it is connected.

I can see the various apps ON the Tivo but of course, when I Select them I get a file not found error since the server is not connecting anymore.

Any ideas?

I guess you can tell that I am new to this. Both the Tivo HME as well as Python.

I am trying this on a TIVO HD XL

Thanks

[wmcbrine]

Show me the exact messages, from both sides. Also your config.ini.

[Allanon]

How can I wrap text to a text window? If a new line character is added anywhere in the text string then the simulator will wrap that text correctly but the same text doesn't wrap when viewed with my Tivo HD. Am I going to have to write code to wrap the text manually?

[wmcbrine]

How can I wrap text to a text window? If a new line character is added anywhere in the text string then the simulator will wrap that text correctly but the same text doesn't wrap when viewed with my Tivo HD.I'm not sure what you're saying here. New line characters should always be processed, and are, in my experience. But as for automatic wrapping, that doesn't happen unless you set the RSRC_TEXT_WRAP flag. These are two separate issues.

[Joe Q]

Show me the exact messages, from both sides. Also your config.ini.

There is not much to show you.

Here is the ini file:
[hmeserver]
port=7288

#zeroconf=False
basepath=d:\Tivo\pytivo\hme-python-0.16
datapath=D:\Tivo\pytivo\hme-python-0.16

Here is the results from running start.py with that config.ini:
D:\Tivo\pytivo\hme-python-0.16>start.py
HME Server for Python 0.16
Skipping: picture - No module named Image
Wed Jan 14 12:45:04 2009 Server Starts
Registering: animate
Registering: clock
Registering: effects
Registering: fontinfo
Registering: hello
Registering: test
Registering: tictactoe
Registering: transition






I get the same exact resultrs with NO config.ini file

[Allanon]

I'm not sure what you're saying here. New line characters should always be processed, and are, in my experience. But as for automatic wrapping, that doesn't happen unless you set the RSRC_TEXT_WRAP flag. These are two separate issues.


I was talking about automatic wrapping. I didn't know there was a flag for it. When the text has a new line character anywhere in the text the simulator will wrap all the text automaticly as if the flag was set.

[wmcbrine]

Really? Huh. That's another bug in the simulator, then.

Joe Q: You've overlooked the part where I said "from both sides"; i.e, the TiVo side as well as the server.

[Joe Q]

Really? Huh. That's another bug in the simulator, then.

Joe Q: You've overlooked the part where I said "from both sides"; i.e, the TiVo side as well as the server.


I appreciate all the help as well as patience.

As I had said, Last night, one time I got a connection from the Server and the Tivo was displaying the clock/fontinfo/etc. as items to choose on the menu.

I just took this photo of the tivo and I see they are now gone.

So here is 'the tivo side':

[Joe Q]

After I found a bug on the SDK website where they have the wrong name for sampleS.jar, I finally was able to use the simulator and the java test sample.

With that, I could verify using the simulator.

(see http://tivohme.sourceforge.net/docs/hmesdk/01_Overview.html#wp998762)


I had all your code under this path "d:\tivo\pytivo\hme-python-0.16" but it would not work so I got your code to work by moving it all up under d:\hme and deleting my config.ini file


Your apps show up in the Java simulator now as well as on my Tivo

Here is the output of start.py
HME Server for Python 0.16
Skipping: picture - No module named Image
Wed Jan 14 18:00:35 2009 Server Starts
Registering: animate
Registering: clock
Registering: effects
Registering: fontinfo
Registering: hello
Registering: test
Registering: tictactoe
Registering: transition
192.168.1.5:5239 - - [14/Jan/2009 18:00:41] "GET /animate/icon.png HTTP/1.1" 200
-
192.168.1.5:5241 - - [14/Jan/2009 18:00:41] "GET /animate/icon.png HTTP/1.1" 200
-
192.168.1.5:5242 - - [14/Jan/2009 18:00:41] "GET /effects/icon.png HTTP/1.1" 200
-
192.168.1.5:5243 - - [14/Jan/2009 18:00:41] "GET /effects/icon.png HTTP/1.1" 200
-
192.168.1.5:5244 - - [14/Jan/2009 18:00:41] "GET /fontinfo/icon.png HTTP/1.1" 20
0 -
192.168.1.5:5245 - - [14/Jan/2009 18:00:41] "GET /fontinfo/icon.png HTTP/1.1" 20
0 -
192.168.1.5:5246 - - [14/Jan/2009 18:00:41] "GET /hello/icon.png HTTP/1.1" 200 -

192.168.1.5:5247 - - [14/Jan/2009 18:00:41] "GET /hello/icon.png HTTP/1.1" 200 -

192.168.1.5:5248 - - [14/Jan/2009 18:00:41] "GET /test/icon.png HTTP/1.1" 200 -
192.168.1.5:5249 - - [14/Jan/2009 18:00:41] "GET /test/icon.png HTTP/1.1" 200 -
192.168.1.5:5250 - - [14/Jan/2009 18:00:41] "GET /tictactoe/icon.png HTTP/1.1" 2
00 -
192.168.1.5:5251 - - [14/Jan/2009 18:00:41] "GET /tictactoe/icon.png HTTP/1.1" 2
00 -



Thanks

[wmcbrine]

It's past time I updated this. The changes are minor, but potentially important.

[jbcooley]

It's funny how these things happen. I'd just posted a (long over due) update to my .NET libraries a few days ago. I'm glad to see you're still publishing updates to your library.

[wmcbrine]

More automatic resource management, and some bug fixes and enhancements for the Premiere.

[wmcbrine]

I've started a GitHub page for HME for Python (http://github.com/wmcbrine/hmeforpython), so that, finally, you can download it without registering here. :)

[wmcbrine]

Direct text keyboard input support.

[jtseltmann]

I'm very new to mac programming and python...well mac in general. How do I go about getting this great code installed and running? I downloaded the zip file...but I don't see how to get it running? Is there install directions for dummies? i'm coming from windows programming background so...be patient!
Thanks
J

[wmcbrine]

I already put my best instructions in the README, so unless you have a more specific question, I can't help you. I can tell you that there's no need to "install" it, though. Just unzip it.

BTW, how do you know it's great code if you haven't got it working yet? :)

[ttocsmi]

I already put my best instructions in the README, so unless you have a more specific question, I can't help you. I can tell you that there's no need to "install" it, though. Just unzip it.

BTW, how do you know it's great code if you haven't got it working yet? :)

i'm intrigued by this program - it seems like it's quite handy and i can tell you (and others here) have put a lot of time and effort into its development.

i downloaded both the zeroconf.py and remote.pwy files, and can view each in IDLE. i also installed python 2.7 on my XP laptop. when i attempt to run each, a shell window briefly appears, disappears, and nothing else happens. if i try to run each file using F5 from within Idle, a "there's an error in your program: invalid syntax" box appears and nothing else happens.

any ideas as to what i'm doing wrong? thank you very much.

scott

[wmcbrine]

Well, remote.pyw is over in the other thread (http://tivocommunity.com/tivo-vb/showthread.php?t=392385)... anyway, don't run it from IDLE. You don't need IDLE, and Tk-based apps like remote.pyw won't work from within IDLE. For remote.pyw, just double-click on it from the desktop. Alternatively, you can start it by typing "python remote.pyw" at the command line.

[ttocsmi]

thanks again for your assistance. i tried double-clicking it again from my desktop - nothing.

i also tried launching from the command prompt and got the message:

file"remote.pyw", line 6
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
SyntaxError: invalid syntax

very odd.

[wmcbrine]

Yes, that certainly is odd. You've somehow got XHTML in your remote.pyw. I guarantee, it didn't come that way.

Seeing that you said "i downloaded both the zeroconf.py and remote.pwy files" suggests your error. Download the zip file.

Anyway, please stop posting about remote.pyw in the HME for Python thread. Post in the appropriate thread, or PM me if you must.

[ttocsmi]

Ok, will do. (Got the files from https://github.com/wmcbrine/tivoremote by the way)

[carlroth]

I wrote this simple HME app, and I am finding that the TiVo (or at least, the simulated TiVo) quits the application when the last (non-root) view is removed. Is this expected behavior?

[wmcbrine]

It doesn't happen on my real Premiere. Looks like another simulator bug.

[jbernardis]

In another thread, I was saying that I wish that when I requested a video from one of my tivos that pytivo processed this as a push instead of a pull. Then I started thinking that I could write an HME app hased on your module here that does just this.

I started working on it this pask weekend and have gotten quite far - far enough that I'm now actually thinking about the code for the push itself. The question I have is how do I know the identity of the tivo to which to push the video. I was thinking that I only needed to know the identity of the tivo I was talking to, but then I thought it might be nice if I could push to a different tivo also. However this requires that I discover the identities of the tivos on my network.

I see that you have the zeroconf.py module as part of your start.py, although you are using it only to advertise the applications. I'm trying to think of the best way to incorporate discovery. I don't think the apps should do it since they only exist for each instantiation and I think it would be expensive to do discovery so often. Also, you already have the zeroconf code intertwined in the start code.

I guess I could modify the start code to do the discovery, but then I would need to pass this information down into the app - either through a new method that would be stubbed out in the base class but implemented in the apps where necessary, or simply through additional parameters passed into mainloop and then into the startup method.

Am I missing something here? Can you think of any other way to tackle this problem?

Thanks

[orangeboy]

In another thread, I was saying that I wish that when I requested a video from one of my tivos that pytivo processed this as a push instead of a pull. Then I started thinking that I could write an HME app hased on your module here that does just this.

I started working on it this pask weekend and have gotten quite far - far enough that I'm now actually thinking about the code for the push itself. The question I have is how do I know the identity of the tivo to which to push the video. I was thinking that I only needed to know the identity of the tivo I was talking to, but then I thought it might be nice if I could push to a different tivo also. However this requires that I discover the identities of the tivos on my network.

I see that you have the zeroconf.py module as part of your start.py, although you are using it only to advertise the applications. I'm trying to think of the best way to incorporate discovery. I don't think the apps should do it since they only exist for each instantiation and I think it would be expensive to do discovery so often. Also, you already have the zeroconf code intertwined in the start code.

I guess I could modify the start code to do the discovery, but then I would need to pass this information down into the app - either through a new method that would be stubbed out in the base class but implemented in the apps where necessary, or simply through additional parameters passed into mainloop and then into the startup method.

Am I missing something here? Can you think of any other way to tackle this problem?

Thanks

Here's a small chunk of python code I use in my TiVoToDo project to get the TiVos on my LAN, and to put the information into a dictionary, with the TiVo name as the key, and the TSN as the item the key refers to:

TiVoToDo.py:

import config as c
import tools as t

# Find TiVos on the LAN
b = t.Beacon()
b.start()
b.stop()


tools.py

import config as c
import Zeroconf as Z

class ZCListener:
def __init__(self, names):
self.names = names

def removeService(self, server, type, name):
if name in self.names:
self.names.remove(name)

def addService(self, server, type, name):
self.names.append(name)

class ZCBroadcast:
def __init__(self):
self.rz = Z.Zeroconf()

def scan(self):
""" Look for TiVos using Zeroconf. """
VIDS = '_tivo-videos._tcp.local.'
names = []

# Get the names of servers offering TiVo videos.
browser = Z.ServiceBrowser(self.rz, VIDS, ZCListener(names))

# Give them half a second to respond.
time.sleep(.5)

# Now get the addresses -- this is the slow part.
for name in names:
info = self.rz.getServiceInfo(VIDS, name)
if info and 'TSN' in info.properties:
tsn = info.properties['TSN']
name = name.replace('.' + VIDS, '').encode('utf-8')
if isHDtivo(tsn):
c.tivos[name] = tsn

def shutdown(self):
self.rz.close()
time.sleep(.5)

class Beacon:
UDPSock = socket(AF_INET, SOCK_DGRAM)
UDPSock.setsockopt(SOL_SOCKET, SO_BROADCAST, 1)
services = []

def __init__(self):
self.log = logging.getLogger("TiVoToDo.tools.Beacon")
self.log.info('Scanning for TiVos...')
self.bd = ZCBroadcast()
self.bd.scan()

def format_services(self):
return ';'.join(self.services)

def format_beacon(self, conntype, services=True):
beacon = ['tivoconnect=1',
'swversion=1',
'method=%s' % conntype,
'identity=%s' % c.getGUID(),
'machine=%s' % gethostname(),
'platform=pc']

if services:
beacon.append('services=' + self.format_services())
else:
beacon.append('services=TiVoMediaServer:0/http')

return '\n'.join(beacon)

def send_beacon(self):
beacon_ip = c.get_broadcast()
if not beacon_ip:
beacon_ip = '255.255.255.255'
try:
self.UDPSock.sendto(self.format_beacon('broadcast'),
(beacon_ip, 2190))
except error, e:
log.debug(e)

def start(self):
self.send_beacon()
self.timer = Timer(60, self.start)
self.timer.start()

def stop(self):
self.timer.cancel()
if self.bd:
self.bd.shutdown()


This was code modified from pyTivo. Only a couple of lines were changed to get the functionality I needed.

HTH!

Edit: You may not need or want the HD model test: "if isHDtivo(tsn):", so you could delete that line and re-indent "c.tivos[name] = tsn" appropriately.

[jbernardis]

Thanks Orangeboy. I was looking through the code, and it seems to me that I could do it simply with zeroconf:zc = ZeroConf()
zc.addServiceListener(type, Listener)
...wait a few seconds...
zc.removeServiceListener(Listener)
zc.close() and the listener will have recorded the tivo names.

The problem is that I don't think I can do this in the app because the main logic already has a listener thread going and I think the two threads would be competing for the same port number (or maybe I don't fully understand this zeroconf stuff - very possible).

I think instead I will have to adapt the main logic. Right now, it registers the apps inside the Broadcast constructor - but this is also where the Zeroconf constructor is called.

I was thinking of doing the following:
1) split the Broadcast constructor into the constructor and a separate method for registering the apps.
2) adding a method for the above discovery logic (much like the scan method in pytivo)
3) changing the mainline logic to construct the Broadcast object and to invoke the discovery method before the loop that imports the apps, and then calling the new register method after this loop.

That would give me the list of Tivos which I would have to pass into the Server constructor so that it could then get passed down into the apps themselves.

[jbernardis]

The code I ended up adding to start.py is the following:

class ZCListener:
def __init__(self, names):
self.names = names

def removeService(self, server, type, name):
if name in self.names:
self.names.remove(name)

def addService(self, server, type, name):
self.names.append(name)

def doDiscovery(config):
""" Look for TiVos using Zeroconf. """
VIDS = '_tivo-videos._tcp.local.'
names = []

rz = Zeroconf.Zeroconf()
print "Attempting discovery..."
# Get the names of servers offering TiVo videos
browser = Zeroconf.ServiceBrowser(rz, VIDS, ZCListener(names))

config.remove_section(DT_SECTION)

tivoCount = 0

time.sleep(5)


# Now get the addresses -- this is the slow part
for name in names:
info = rz.getServiceInfo(VIDS, name)
if info and 'TSN' in info.properties:
tsn = info.properties['TSN']
address = socket.inet_ntoa(info.getAddress())
name = name.replace('.' + VIDS, '')
print "Discovered tivo: " + name + ' ' + address + ' ' + tsn
if tivoCount == 0:
config.add_section(DT_SECTION)
tivoCount = tivoCount + 1
config.set(DT_SECTION, 'tivo'+str(tivoCount)+'.name', name)
config.set(DT_SECTION, 'tivo'+str(tivoCount)+'.tsn', tsn)
config.set(DT_SECTION, 'tivo'+str(tivoCount)+'.address', address)

browser.cancel()
rz.close()

This was added right before the "main" routine. And then later in the code, I added:


if have_zc:
doDiscovery(config)


This was added immediately after determining if we have zeroconf available.

Two things to note here: 1) This does not work reliably on windows. It works fine after a reboot, but after that, it only works intermittently. I suspect that windows is not cleaning up the outstanding threads after the process is killed. It works flawlessly on linux. 2) Note that I decided to place the discovery information into the config. This was already available to the application logic, so it was an already-existing way to pass it through.

[orangeboy]

The code I ended up adding to start.py is the following:

class ZCListener:
def __init__(self, names):
self.names = names

def removeService(self, server, type, name):
if name in self.names:
self.names.remove(name)

def addService(self, server, type, name):
self.names.append(name)

def doDiscovery(config):
""" Look for TiVos using Zeroconf. """
VIDS = '_tivo-videos._tcp.local.'
names = []

rz = Zeroconf.Zeroconf()
print "Attempting discovery..."
# Get the names of servers offering TiVo videos
browser = Zeroconf.ServiceBrowser(rz, VIDS, ZCListener(names))

config.remove_section(DT_SECTION)

tivoCount = 0

time.sleep(5)


# Now get the addresses -- this is the slow part
for name in names:
info = rz.getServiceInfo(VIDS, name)
if info and 'TSN' in info.properties:
tsn = info.properties['TSN']
address = socket.inet_ntoa(info.getAddress())
name = name.replace('.' + VIDS, '')
print "Discovered tivo: " + name + ' ' + address + ' ' + tsn
if tivoCount == 0:
config.add_section(DT_SECTION)
tivoCount = tivoCount + 1
config.set(DT_SECTION, 'tivo'+str(tivoCount)+'.name', name)
config.set(DT_SECTION, 'tivo'+str(tivoCount)+'.tsn', tsn)
config.set(DT_SECTION, 'tivo'+str(tivoCount)+'.address', address)

browser.cancel()
rz.close()

This was added right before the "main" routine. And then later in the code, I added:


if have_zc:
doDiscovery(config)


This was added immediately after determining if we have zeroconf available.

Two things to note here: 1) This does not work reliably on windows. It works fine after a reboot, but after that, it only works intermittently. I suspect that windows is not cleaning up the outstanding threads after the process is killed. It works flawlessly on linux. 2) Note that I decided to place the discovery information into the config. This was already available to the application logic, so it was an already-existing way to pass it through.

How's this project coming along? I'd be interested in such an app, and wondering if you'll make it available through google code, or some other means?

[jbernardis]

Sure - I'll make it available, but I'm not quite done yet. Also, I'm not crazy about the idea of providing support for the code, so it would most likely be on an as-provided basis.

I am just about through the user interface - up to the point where I can make the actual push request, but I have also recently decided to add the ability to delete a video file, so I still have to do that too.

I'm also re-thinking my "need" to do discovery. I need to study the push code in pytivo, but I think the only thing I need is the TSN - not the IP address. If so, I might just statically enter the values in the config file since the TSN isn't ever likely to change.

[orangeboy]

Sure - I'll make it available, but I'm not quite done yet. Also, I'm not crazy about the idea of providing support for the code, so it would most likely be on an as-provided basis.


Great! I look forward to seeing the finished product! :up:
As far as it being "as-is", putting it in google code or some other type of distribution will give the chance for other developers to offer support, and also to make branches/plugins/whatever, as well.


I am just about through the user interface - up to the point where I can make the actual push request, but I have also recently decided to add the ability to delete a video file, so I still have to do that too.


The delete feature may indeed come in handy.


I'm also re-thinking my "need" to do discovery. I need to study the push code in pytivo, but I think the only thing I need is the TSN - not the IP address. If so, I might just statically enter the values in the config file since the TSN isn't ever likely to change.

Early versions of my TiVoToDo code had the TSNs "hard-coded" in the config file. But yes, I believe only the TSN is required for the TiVo DVRs, but an IP address (or DNS name) and port would still be required for the pyTiVo server.

[jbernardis]

ok - so as of now, the entire user interface is working and the file deletion function is working. All that remains is the actual push. The user can select files to push, and select the tivo to push to, but all I do with the information right now is print it.

I might need some help here. I had originally thought that I could just send the push request to mind, and pytivo would actually take care of the transfer, and I'm still not certain that won't work. I am having some difficulties right now though.


I took the mind.py file from pytivo and adapted it for my purposes. Right now, I am able to log in to mind, but I get a 302 back which I think indicates a redirection. The login routine is line for line extracted from the pytivo file, so I don't know if this is an issue or not. Certainly pytivo does nothing with thie return code and just proceeds.

Next my routine calls pcBodySearch and seems to get back a legitimatebody_id. So the next message sent to mind is via bodyOfferModify. This routine fails though. It tries to invoke the offer_id's replace method, but I get a message that there is no such method. I think the offer_id is getting the value of "None" but I haven't had time to analyze it yet.

I'll keep going on this, but progress has definitely slowed down.

[orangeboy]

ok - so as of now, the entire user interface is working and the file deletion function is working. All that remains is the actual push. The user can select files to push, and select the tivo to push to, but all I do with the information right now is print it.

I might need some help here. I had originally thought that I could just send the push request to mind, and pytivo would actually take care of the transfer, and I'm still not certain that won't work. I am having some difficulties right now though.


I took the mind.py file from pytivo and adapted it for my purposes. Right now, I am able to log in to mind, but I get a 302 back which I think indicates a redirection. The login routine is line for line extracted from the pytivo file, so I don't know if this is an issue or not. Certainly pytivo does nothing with thie return code and just proceeds.

Next my routine calls pcBodySearch and seems to get back a legitimatebody_id. So the next message sent to mind is via bodyOfferModify. This routine fails though. It tries to invoke the offer_id's replace method, but I get a message that there is no such method. I think the offer_id is getting the value of "None" but I haven't had time to analyze it yet.

I'll keep going on this, but progress has definitely slowed down.

Interesting. So pyTivo will not take any part of the push? I had mistakenly assumed that the request would be sent to pyTivo, and not directly to the mind server. This is what I'm currently doing in batch to initiate a push outside of pyTivo's web interface:


curl.exe -s -d "Command=Push&Container=%Container%&File=!File!&tsn=%tsn%" "http://DL380-Server:%port%/TivoConnect?"


%port% is a variable for me since I have two instances of pyTivo running. All other variables should be self explanatory. It sounds like you're going a bit deeper than just merely passing the request off to pyTivo though...

[jbernardis]

Thank you - this is exactly what I wanted - I didn't want to have to go through mind - a direct request to pytivo was what I wanted - but I didn't know there was such an interface.

Now that I can do it this way, it should be complete soon.

[jbernardis]

What is the !File! - is that the fully qualified path to the video file? Why the bangs instead of % ?

[orangeboy]

Thank you - this is exactly what I wanted - I didn't want to have to go through mind - a direct request to pytivo was what I wanted - but I didn't know there was such an interface.

Now that I can do it this way, it should be complete soon.

You're welcome! Yep, pyTivo has a listening port that I use regularly.

What is the !File! - is that the fully qualified path to the video file? Why the bangs instead of % ?

Bang is needed in Window's batch when using a FOR loop, and expands variables after the original command line is interpreted. It basically functions the same as percent.

This is my batch in it's entirety, with an explanation following:

@ECHO off
SETLOCAL ENABLEDELAYEDEXPANSION
SET tsn=%1
SHIFT

:HouseKeeping
::-----------------------------------------::
:: Clean up successfully transferred files ::
::-----------------------------------------::
FOR /f "tokens=2* delims= " %%A IN ('FINDSTR /I /L "\TV\ \Cut" "\\DL380-Server\pyTivo\transfers.txt"') DO (
SET File=%%B
CALL :Trim_File !File!
IF EXIST "!File!" (2>NUL DEL "!File!")
IF EXIST "!File!.txt" (2>NUL DEL "!File!.txt")
IF EXIST "!File!.xfer" (2>NUL DEL "!File!.xfer")
REM IF EXIST "%%~dpnB.mkv" (2>NUL DEL "%%~dpnB.mkv")
IF EXIST "%%~dpB\*.nfo" (2>NUL DEL "%%~dpB\*.nfo")
IF EXIST "%%~dpB\*.sfv" (2>NUL DEL "%%~dpB\*.sfv")
IF EXIST "%%~dpB\*.srr" (2>NUL DEL "%%~dpB\*.srr")
IF EXIST "%%~dpB\*.srt" (2>NUL DEL "%%~dpB\*.srt")
)

FINDSTR /I /L "\TV\ \Cut" "\\DL380-Server\pyTivo\transfers.txt" >> "\\DL380-Server\pyTivo\deleted.transfers.txt"
FINDSTR /I /L /V "\TV\ \Cut" "\\DL380-Server\pyTivo\transfers.txt" >> "\\DL380-Server\pyTivo\transfers.tmp"

TYPE NUL> "\\DL380-Server\pyTivo\transfers.txt"
TYPE "\\DL380-Server\pyTivo\transfers.tmp" >> "\\DL380-Server\pyTivo\transfers.txt"

SORT /R "\\DL380-Server\pyTivo\pushed.txt" /O "\\DL380-Server\pyTivo\pushed.txt"
SORT /R "\\DL380-Server\pyTivo\transfers.txt" /O "\\DL380-Server\pyTivo\transfers.txt"
SORT /R "\\DL380-Server\pyTivo\deleted.transfers.txt" /O "\\DL380-Server\pyTivo\deleted.transfers.txt"

DEL "\\DL380-Server\pyTivo\transfers.tmp"

:Loop
::-----------------------------------------::
:: Process all files until arguments empty ::
::-----------------------------------------::
IF "%~1"=="" GOTO Done

:Look
::-----------------------------------------::
:: Parse conf file looking for containers ::
::-----------------------------------------::
FOR /F "usebackq delims=[]" %%A IN (`type "\\DL380-Server\pyTivo\pytivo.conf" ^|FINDSTR "["`) DO (
FOR /F %%B IN ('ECHO "%~1" ^|FIND /C "%%A"') DO (
IF %%B EQU 1 SET Container=%%A
IF DEFINED Container GOTO Found
)
)
ECHO pyTivo container not found.
ECHO File not pushed.
GOTO End

:Found
::-----------------------------------------::
:: Container found, create variables ::
::-----------------------------------------::
SET DisplayFile=%~1
SET File=%~1
SET File=!File:*%Container%=!
SET File=!File:\=/!
SET File=!File: =+!
SET ThisPath=%~dp1
SET ThisPath=!ThisPath:~0,-1!

:MetaData
::-----------------------------------------::
:: Create metadata if there isn't any ::
::-----------------------------------------::
IF NOT EXIST "%~dpnx1.txt" (
IF NOT EXIST "%~dp1\.meta\%~nx1.txt" (
"\\DL380-Server\lan path\pyTivoMetaThis.py" -d -p "!ThisPath!"
)
)
IF NOT EXIST "%~dpnx1.txt" (
IF NOT EXIST "%~dp1\.meta\%~nx1.txt" (
>"%~dpnx1.txt" ECHO episodeTitle : "%~n1"
)
)

:Push
::-----------------------------------------::
:: Request pytivo PUSH of files using cURL ::
::-----------------------------------------::
IF /I "%tsn%" EQU "TivoS3" (
SET port=9033
) ELSE (
SET port=9033
)

IF EXIST "!ThisPath!\Push.Results.txt" DEL "!ThisPath!\Push.Results.txt"

ECHO Pushing: %DisplayFile%
curl.exe -s -d "Command=Push&Container=%Container%&File=!File!&tsn=%tsn%" "http://DL380-Server:%port%/TivoConnect?" >>"!ThisPath!\Push.Results.txt"

FOR /F "tokens=5,12,16 delims=^<^>" %%A IN ('FINDSTR /R "\<<.>\>" "!ThisPath!\Push.Results.txt"') DO (
@ECHO %date:~-10% %time:~0,8% %%A >>"!ThisPath!\Push.Results.txt"
@ECHO %date:~-10% %time:~0,8% %%B >>"!ThisPath!\Push.Results.txt"
@ECHO %date:~-10% %time:~0,8% %%C >>"!ThisPath!\Push.Results.txt"
)

:End
::-----------------------------------------::
:: File is done, go get next one ::
::-----------------------------------------::
SHIFT
GOTO Loop

:Done
::-----------------------------------------::
:: Files done. Go back from wence you came ::
::-----------------------------------------::
EXIT /B

:Trim_File
SET File=%*
GOTO :EOF


The batch would be called like this: Push.To.TiVo.bat <TiVo name> "<Full path to video file>"

The first part of the batch sets up the environment, and gathers the TiVo name I want to push to from "%1", and then "shifts" that first argument away.

The "Housekeeping" section reads a text file created by a pyTivo hack that lists successfully transferred files, and deletes a variety of different files within my "TV" and "Cut Programs" directories only. This is merely a disk maintenance step, and not really pertinent to the "push" functionality.


The "Loop" section is only there if I want to push more that one file.


The "Look" section parses through the pytivo.conf file, looking for pyTivo Share names (or container) that are found within the path of the file I'm trying to push. If a Share name can't be found, the batch ends since pyTiVo wouldn't know where to find the file to push.


The "Found" section manipulates the "File" variable derived from the file path, starting at the container name, making it "url" friendly by replacing spaces with pluses, and back-slashes with forward-slashes. As an example, with TV being the name of a Share or container:

File="C:\Videos\TV\Smallville\Clark Kent goes to Washington.mpg"
would turn into:
File="/Smallville/Clark+Kent+goes+to+Washington.mpg"

pyTivo already knows that the "TV" container is rooted at "C:\Videos\TV" from the conf file.


The "MetaData" section should be self explanatory.


The "Push" section starts off by selecting a port number based on TiVo name. I've had occasion where I needed two instances of pyTivo running to connect to an alternate mind server (NTTABs).

I log the results of the push in a text file, so some clean up is done to allow this.

Next, cURL is used to actually make the push request to pyTivo, with the results captured into the text file.

After variable substitution:

curl.exe -s -d "Command=Push&Container=%Container%&File=!File!&tsn=%tsn%" "http://DL380-Server:%port%/TivoConnect?"

would look like:

curl.exe -s -d "Command=Push&Container=TV&File=/Smallville/Clark+Kent+goes+to+Washington.mpg&tsn=TivoS3" "http://DL380-Server:9033/TivoConnect?"


The "End" section shifts the file name argument out, and goes to the Loop label to start all over.


The "Done" section will actually exit the batch file once all files have completed.


The "Trim_File" section merely drops trailing blanks from the file name, and only relates to the "Housekeeping" section.

[NatasNJ]

Not sure if this is the right place to ask but figured it can't hurt.

I installed and had pyTivo running for months on end with no problems. Well tonight I went into my list and no longer see the WHS icon. Both my TivoHD's don't see it they BOTH however see the HP WHS EX495 Server which is where my WHS pytivo videos are housed.

So I don't think it is a connection (wifi) issue since I can see and access the EX495. I tried rebooting the TIVO. No luck.
I tried logging into the EX495 and stopping/starting the Pytivo service. No luck.

[orangeboy]

Not sure if this is the right place to ask but figured it can't hurt.

I installed and had pyTivo running for months on end with no problems. Well tonight I went into my list and no longer see the WHS icon. Both my TivoHD's don't see it they BOTH however see the HP WHS EX495 Server which is where my WHS pytivo videos are housed.

So I don't think it is a connection (wifi) issue since I can see and access the EX495. I tried rebooting the TIVO. No luck.
I tried logging into the EX495 and stopping/starting the Pytivo service. No luck.

Here would be the place to ask: http://www.tivocommunity.com/tivo-vb/showthread.php?t=328459

[jbernardis]

OK - so I incorporated orangeboy's suggestion into the code, and now I can send the push request to pytivo. I even get back the response that the file is queued. However, the blue light never comes on and the push never occurs. This happened late last night and I haven't had a chance to diagnose it yet. This morning before I left for work, I logged onto my NAS, emptied the pytivo log file, fired up the HME script and requested a push. Again, I got the "queued for push" message. I wasn't able to wait to see if the push started - I had to leave for work. But I'll check this evening and report my findings. This way, the log file should contain ONLY messages dealing with my push request, so hopefully there'll be fewer messages to focus in on.

If I'm still baffled, I'll post my code, program output, and log file contents.

[jbernardis]

OK - I have put the first version of my HME application up on github. The installation instructions are in the included README file, but basically, this program is meant to be run under wmcbrine's python HME. WIth it, you can request that videos be pushed to your tivo. You can also delete files.

Be kind with your criticism. Although a programmer for 30+ years, OO programming has never been my strong suit. Also, this is my first use of Python, so I'm still learning the finer points of the language.

The source code can be found here. (https://github.com/jbernardis/pytivo-video-manager)

Thanks

[orangeboy]

Bah. I can't get ConfigParser to open config.ini. This is even in HME's "start.py". With no config.ini being available, I get "No Tivos found - exiting".

Windows XP,
python 2.6.6
python HME v0.19

[jbernardis]

I don't see config.merge in the zip file :(

My screw-up. It's there now.

[orangeboy]

My screw-up. It's there now.

No worries. I used the info out of the Readme. I'm just having issues with HME for Python, and not necessarily your app!

[orangeboy]

I added this to __init__.py close to the beginning (at line 25):


p = os.path.dirname(__file__)


and altered the Images class:

class Images:
def __init__(self, app):
self.Background = Image(app, os.path.join(p, 'background.png'))
self.CueUp = Image(app, os.path.join(p, 'cueup.png'))
self.CueDown = Image(app, os.path.join(p, 'cuedown.png'))
self.CueLeft = Image(app, os.path.join(p, 'cueleft.png'))
self.HiLite = Image(app, os.path.join(p, 'hilite.png'))
self.MenuBkg = Image(app, os.path.join(p, 'menubkg.png'))
self.IconFolder = Image(app, os.path.join(p, 'folder.png'))
self.IconVideo = Image(app, os.path.join(p, 'video.png'))


This alleviated this problem in Windows:

IOError: [Errno 2] No such file or directory: 'vidmgr/background.png'


I like what I see so far!

I ended up hard-coding the *full path* to my config.ini in Start.py to get ConfigParser to work.

[jbernardis]

I didn't have any such problems on my system. Did you organize your files differently than what wmcbrine was expectiing? On my system, vidmgr was a subdirectory of the main pyhme directory.

In any event, I felt that the changes you suggested make the code more defensive and so they have already been added to the repository.

[orangeboy]

I didn't have any such problems on my system. Did you organize your files differently than what wmcbrine was expectiing? On my system, vidmgr was a subdirectory of the main pyhme directory.

In any event, I felt that the changes you suggested make the code more defensive and so they have already been added to the repository.

I put vidmgr as a subdirectory, too. It may be an issue with embedded spaces in the path. I had problems with HME for Python's start.py finding config.ini as well. Here's the result of a Window's "Tree" command:

Sun 04/03
19:30:00=>Tree /A /F "D:\Program Files\HME for Python .19"
Folder PATH listing for volume Recovery
Volume serial number is 006E006F 7887:9D2E
D:\PROGRAM FILES\HME FOR PYTHON .19
| config.ini
| hme.py
| hme.pyc
| metadata.py
| metadata.pyc
| README.txt
| start.py
| Zeroconf.py
| Zeroconf.pyc
|
+---animate
| icon.png
| __init__.py
| __init__.pyc
|
+---clock
| icon.png
| __init__.py
| __init__.pyc
|
+---effects
| icon.png
| __init__.py
| __init__.pyc
|
+---fontinfo
| icon.png
| __init__.py
| __init__.pyc
|
+---hello
| icon.png
| __init__.py
| __init__.pyc
|
+---picture
| icon.png
| __init__.py
| __init__.pyc
|
+---test
| back1.jpg
| icon.png
| __init__.py
| __init__.pyc
|
+---tictactoe
| bg.jpg
| grid.png
| icon.png
| __init__.py
| __init__.pyc
|
+---transition
| icon.png
| __init__.py
| __init__.pyc
|
\---vidmgr
background.png
cuedown.png
cueleft.png
cueup.png
folder.png
hilite.png
icon.png
menubkg.png
pushmsgbkg.png
video.png
__init__.py
__init__.pyc


Regardless, I'm "pushing" from the TiVo interface! I like it! :up::up::up:

[jbernardis]

Regardless, I'm "pushing" from the TiVo interface! I like it! :up::up::up:

Glad you like it. I know it still has a few rough edges, but I thought I'd get it out there.

[Allanon]

Jbernardis, worked fine for me and I didn't have to modify the code, good work. I really have no use for this program since I usually just push from my computer once I download a video or use pyTivo's interface from the Tivo. I don't think I ever was at my Tivo thinking I really need to send a video to the other Tivo right now. What I really could use is a program that allows me to transfer a show from the Tivo I'm watching to another Tivo on the network. I know this could probably be done by downloading the show then pushing it to the other Tivo but that would probably take a lot longer than a MRV transfer. Is there anyway to fake a Tivo MRV file request?

[jbernardis]

Thanks for the kind words!

Regarding your request, I can't imagine how you would fake out the receiving tivo. The sending tivo - yes - but the receiving tivo is the one usually initiating the request, so to have it suddenly be presented with an inbound MRV request? I don't know how that would be done.

OTOH, you could as you suggest, write an HME app to initiate a pull to the HME server and then a push to the recipient tivo. This sounds pretty complicated, and it would take a while since the transfers would have to take place successively.

Regarding code modifications, I THINK orangeboy's issue was that the pyhme directory was not the current directory. After thinking about it, I think that would cause the symptoms he describes.

[orangeboy]

Regarding code modifications, I THINK orangeboy's issue was that the pyhme directory was not the current directory. After thinking about it, I think that would cause the symptoms he describes.

That makes sense. I was invoking start.py from a command line:


Sun 04/03
19:43:08=>"D:\Program Files\HME for Python .19\start.py"
HME Server for Python 0.19
Sun Apr 03 19:43:10 2011 Server Starts
Registering: vidmgr

[jbernardis]

I created an init.d startup script that changes into the installation directory before trying to start up hme. Basically it's the same script I use to start up pytivo - I just changed the directory and all occurrences of pytivo.py to start.py. I can post it when I get home if you're interested.

All that remains is to hook it into the actual system startup - I just wanted to be sure it was somewhat stable before I did that.

[jcthorne]

WOW. Finally got around to installing this and getting it working. THANKS!

Any chance you could enhance the app to display thumbnails and moviesheets for selection of items rather than a simple list? Tools for the creation of the moviesheets etc already exist, the app would need to display them. Let me know if this is something you could or would like to do, I am quite willing to help where I can.

[jbernardis]

If you could give me some more information on moviesheets, I'll let you know. I'm not sure what they are or what would be involved in doing what you ask.

As far as thumbnails are concerned, I'm not sure I want to get into that. I am already leery of how long it takes to process each directory as they are traversed. Adding something like this (real-time processing of a video file) could be too expensive.

Of course I guess we could generate the thumbnails separately and just display them. I can experiment with something over the next few days to see what might be possible. I'm thinking that we need to define the image size, but if I find a file in the current directory with the same basename as the video file, but with an extension of say .jpg, then I could display it, perhaps instead of the ubiquitous dot. If no such file, just the dot gets displayed.

[jcthorne]

14040

14041

Check out a tool called Thumbgen. Its popular for use in generating thumbnails and moviesheets for the WDTV Live media player and others. It auto gathers the information and graphics from various web sites and assembles the sheet and thumbnail files. The player just associates the media file name with the two jpg files using the name convention shown in the two examples attached.

The files all have to be generated before hand, not on the fly so there is little overhead. By example the processer in the WDTV Live is much less powerful than the tivo.

On the WDTV live, the thumbnails are displayed in a row across the bottom and the movie sheet is displayed for the thumbnail currently selected and changes as the arrow keys are pressed to move to a different thumbnail.

This is what I had in mind but realize its a big step. Perhaps just the thumbnail to start?

[jbernardis]

I played around with a simple thumbnail - I generated a 57x32 png file from a video snapsnot. (57x32 maintains the 16x9 aspect ratio and is as tall as I could get in the existing 32 pixel height of the current screen design).

It actually wasn't that good. First of all, the HME as I understand it only gives you an SD resolution of 640x480. This has two issues: 1) limited resolution and real estate, amd 2) the image is actually stretched to fit onto a true 16x9 display. I am not at this time interested in making such massive changes to the interface I presently have.

I might consider adding a screenshot to the video details page where I have more real estate to play with. This is more in line with your example anyway.

[wmcbrine]

First of all, the HME as I understand it only gives you an SD resolution of 640x480.That's the default, but it's not limited to that. See the "picture" example app, Reversi, and HME/VLC for programs that do HD using HME for Python. For most cases you can use the handle_resolution() function from the picture app. (1280x720 is the highest HME resolution for Series 3 and 4... this is the resolution for the text and graphics overlay, not to be confused with the background video, which is independent of the overlay's resolution.)

BTW, you also need to be aware of the "safe areas", which currently your app is not -- text should be within the title-safe area. (On my TV, your title is partly off-screen.)

Re: thumbnail images, we added the "image" keyword to the pyTivo metadata spec for the benefit of Streambaby, so you could use that. I've never heard of this "moviesheet" thing, and would be curious to know more.

If you're going to continue developing this app, maybe it should have its own thread?

[jcthorne]

I've never heard of this "moviesheet" thing, and would be curious to know more.



Here is a forum topic that discusses moviesheets as relates to the WDTV series of media players. Includes templates for many user generated formats, how to's etc. If I can help in any way, I'd be glad to.

http://forum.wdlxtv.com/viewforum.php?f=18

I mostly got involved with the WDTV Live as it was one of the few hardware media players that could play HD mp4 files with ac3 audio in the same format as Tivo. As all video is stored on our server in this format, this becomes a tivo extender box of sorts. it has a bunch of additional capability for other things as well that I really have not made use of. It does a great job of presenting and playing tivo compatible video files in our motorhome though.

[jbernardis]

If you're going to continue developing this app...

It was never my intent to continue to develop this app. This is exactly what I feared when orangeboy convinced me to post it. The source code is there - people can take it and do what they want with it. It does what I want it to do and I'm not interested in making it everything for everyone.

I WILL look into the HD resolutions, because that is what my TV's are - so thanks for that. But, for example, I will not be interested in making the app "sense" its environment and operate in SD mode for some or HD mode for others. It is what it is.

I will also investigate the use of safe areas. I noticed the safe areas on the simulator, but frankly didn't pay them much attention (again - it was working fine on MY tv's).

If I post an update to the app, I will do so in a separate thread. I had thought that all along.

[jcthorne]

No problem. I really appreciate what you have contributed so far as do others. I can full well understand not wanting to take on someone ELSES project when it was just something you needed and decided to share. Please let us know if you implement the HD interface as that would greatly enhance the look of the menus.

[jbernardis]

Question: just playing around with the HME, I was trying to play an mp3 file. The interface to the Stream class wants a URI though - not a simple file name. I tried every variant of the 'file;' URI that I could think of. I took a look at the examples, but there is no example of playing an MP3. I looked at the Tivo Java HME documentation and sample code, and tried changing the URI to the base URI of the app followed by a file path, but nothing I did seemed to work.

How do I provide a file to the Stream class???

If I am able to implement what I am thinking, this will ultimately run on my NAS in a linux environment. Right now, however, I am developing/debugging in Windows 7.

[wmcbrine]

Take a look at HME/VLC to see how it's done. (I get the feeling, based on some of the other things you did with your app, that you've never looked at it. You really should.)

The key is to define "datapath" (a command-line or config.ini option for start.py). As it says in start.py, "The default is to not allow any access outside of the app directories." I designed this as a security feature, so that one could put up an HME app without unintentionally exposing one's whole drive to the Internet. HME/VLC sets the datapath to "/" by default instead, with the idea that it will only ever be run on a LAN, and this will make configuration easier. But the whole system doesn't play well with Windows and its stupid drive letters -- basically, you can only expose one drive at a time with start.py. (However, you can have apps on one drive, and data on another.)

Here's the core part where a file path is converted to a URL by HME/VLC (in hmevlc/__init__.py):

url = url.replace(self.context.server.datapath, '', 1)
url = 'http://%s/%s' % (self.context.headers['host'],
urllib.quote(url.encode('utf-8')))

The conversion back (on receiving a request) is handled by start.py.

P.S. Oh, but when you said you "tried changing the URI to the base URI of the app followed by a file path" -- that should have worked, even without setting the datapath. I guess I'd have to see exactly what you were sending to know why it didn't work, but if you can see the "icon.png" requests in your log, those are the right form (and if you're seeing an icon, then it's working).

[lrhorer]

It was never my intent to continue to develop this app. This is exactly what I feared when orangeboy convinced me to post it. The source code is there - people can take it and do what they want with it. It does what I want it to do and I'm not interested in making it everything for everyone.
Fair enough, but the app could definitely use a little development. So far, the one thing I see that the app really needs is a "Please wait" splash when it receives a valid command. Often it takes quite a while - 30 seconds or more - to implement the received command. During that time, the user is unsure if the system is processing the command, or if the command was just lost. Lost commands are extremely common with IR control systems, so even in the very short time I have been testing this app, I have several times inadvertently entered several commands into the command buffer, not knowing whether the unit had recognized the button press, or not.

I'm also having a problem with the system locking up whenever I try to go into a particular one of the shares. I need to investigate.

Is there anyone out there who would like to pick up development of this app and maybe start a development thread?

[lrhorer]

Question: If I hard restart pyTivo - that is to say completely kill the python process and start it again, should I re-start pyhme?

[wmcbrine]

lrhorer, I think you want this thread (http://www.tivocommunity.com/tivo-vb/showthread.php?t=468466).

[Allanon]

Wmcbrine, I am using your Apples and Oranges ListView code from HME/VLC Video Streamer. It works fine except when trying to change icons from one menu to another. The only modification I made was to go fetch the icon from a web server, save it to disk, and then assign the image as a icon. Here is the code I changed:


def redraw(self):
hme.Color(self.app, 0xcfcfcf)
count = 0
for i, item in enumerate(self.items[self.startpos:self.startpos +
self.pagesize]):
self.page[i][0].remove_resource()
self.page[i][0].set_text(item['title'], flags=hme.RSRC_HALIGN_LEFT)
if 'icon' in item:
name = 'tempicon'+str(count)+'.png'
urllib.urlretrieve(item['icon'],name)
im = Image.open(name)
newim = im.resize((self.icon_width, self.bar_height))
newim.save(name, "png")
self.page[i][1].set_image(name)
count = count + 1
else:
self.page[i][1].clear_resource()
hme.Color(self.app)
self.upwin.set_visible(self.startpos > 0)
self.downwin.set_visible(len(self.items) >
self.startpos + self.pagesize)


This works fine on the first menu but when I select a menu item and go to the next menu then the icons are the same as the first menu. If there are more menu items in the second menu then it displays icons from the first menu for however many menu items were in the first menu and displays the correct icons for the remaining menu items. If I page down then page back up in the menu then the icons are shown correctly.

I tried removing and clearing the resource before setting the image but that didn't help. The only thing that seems to work is to call redraw twice then the menu icons are correct. Is there a better way to update the icon images?

Also, do you think it would be advantageous to do all the menu, text, and icon drawing to one or two large bitmaps then write them to the Tivo opposed to writing a lot of small bitmaps to the Tivo and have it position them and the text? I could just move a semi-transparent view over the bitmaps to act as a menu selection bar. The program I'm working on can have a lot of menu items with a lot of different icons.

Another question: Do you know of a way to download a bitmap from the web and send it to the Tivo without writing it to a local hard drive? It would be great if I didn't have to save the icon images to disk before sending them to the Tivo.

[jbernardis]

I looked at the sample animation applet that comes with the python HME, and it appears to be reliant of the main app processing its message loop. Am I right in saying that if I am doing a busy task, I would need to periodically yield control so that the waiting internal "send_keys" can be processed. If I don't yield control, only the initial animation would occur, and the send key event would not be processed until my task was completed.

If this is true, one thing that occurs to me would be to have a long-running animation - one that would cover the longest duration I would reasonably expect. But if my task completes before that animation completes, would I be able to stop the animation by just making its view invisible (or something along those lines)?

Otherwise, I can just use a static view for my purpose.

[wmcbrine]

This works fine on the first menu but when I select a menu item and go to the next menu then the icons are the same as the first menu.I'd prefer to leave redraw() (and the rest of ListView) alone, and build the list of icons beforehand. Anyway, I believe what's happening is that you're reusing cached names, so the HME module sees them as the same, and doesn't resend them to the TiVo.

Also, do you think it would be advantageous to do all the menu, text, and icon drawing to one or two large bitmaps then write them to the Tivo opposed to writing a lot of small bitmaps to the Tivo and have it position them and the text?In case it wasn't clear from what I said here (http://www.tivocommunity.com/tivo-vb/showthread.php?p=8549289#post8549289), I thought that the all-in-one background image "solution" that TiVo used for keyboards in Bananas was goofy. :) So no, I wouldn't think that was better. But it all depends... There are limits on the number of resources you can use, including views, but that's mainly an issue for the Series 2.

Another question: Do you know of a way to download a bitmap from the web and send it to the Tivo without writing it to a local hard drive?Sure. The hme.Image resource will take three kinds of initializers: name, file object, or actual image data. Incidentally, only the named images are cached, so using one of the other methods would solve your original problem as a side effect.

urllib will of course also allow you to pull an image directly into memory, without saving it to disk. The sticking point then might be PIL. What I did in pyTivo, to generate thumbnails in the photo plugin, was to work around that by using StringIO():

out = StringIO()
pic.save(out, 'JPEG')
encoded = out.getvalue()
out.close()

[wmcbrine]

Am I right in saying that if I am doing a busy task, I would need to periodically yield control so that the waiting internal "send_keys" can be processed.Yes. But that's what threads are for.

The "Loading..." screen in HME/VLC gives a nice example of an animation running while waiting on something else to happen, although it doesn't depend on threads, because both processes are asynchronous. Note in particular loadbar_anim() in hmevid.py. What happens is that it first checks if the animation is active, and if so, it updates it and schedules another callback. Meanwhile, the main event (loading the stream) sends its own event when it's done, which causes HME/VLC to remove the animation, setting loadbar = None. loadbar_anim() is then invoked again, as expected, at its next scheduled time, but it sees that the loadbar is inactive, so it does nothing, and no more calls to it are scheduled.

[Allanon]

Wmcbrine thanks for the reply, you were correct when suggesting the problem was due to reusing cached names. I also used your advice and now the icon images are read from the web to memory and then to the Tivo without writing to local storage. Thanks again.

[jcthorne]

I could use some help getting HME for Python running correctly under linux on my Synology NAS. If it matters, the shell is BusyBox v1.16.1 and its an x86 Intel Atom based NAS.

If I start HME from a command line using these two commands:

cd /usr/local/pythonHME/
nohup /opt/bin/python /usr/local/pythonHME/start.py

It starts and runs fine.

If I create a startHME.sh script file and run it at boot up by placing it in the /usr/syno/etc/rc.d/ directory;

`sleep 60`
cd /usr/local/pythonHME
/opt/bin/python /usr/local/pythonHME/start.py > /usr/local/pyhme.log 2>&1 &

HME for python starts but it ignores the config file. running the script from a command line yields the errors:

: not foundtc/rc.d/startHME.sh: line 1:
/usr/syno/etc/rc.d/startHME.sh: cd: line 2: can't cd to /usr/local/pythonHME
: not foundtc/rc.d/startHME.sh: line 3:

Suggestions and help?

[windracer]

Do you need to invoke a shell in Busybox like in Linux? In other words, do you need something like
#!/bin/sh
on the first line of your script so it invokes the interpreter?

[jbernardis]

The other question is what are the permissions on the directroies and what user ID is being user interactively versus at boot-up.

If you cd to /usr/local and do 'ls -ld pythonHME' what do you see

incidentally - did you make a mistake when you copied the sleep error message - your posting just says 'leep' not 'sleep'. Did you edit the file correctly when you removed the ticks?

One other thing - are you editing this file on windows and transferring to Linux, or are you editing on Linux. I have seen situations where a file transferred from a PC to Linux has CR/LF at the end of every line - and thtis has thrown off various shell interpreters.

DO you have the od command available? If you try 'od -c startHME.sh' what do you see'?

[jcthorne]

The other question is what are the permissions on the directroies and what user ID is being user interactively versus at boot-up.

If you cd to /usr/local and do 'ls -ld pythonHME' what do you see

This is the result of that query:

drwxr-xr-x 12 root root 4096 Jun 16 09:20 pythonHME

incidentally - did you make a mistake when you copied the sleep error message - your posting just says 'leep' not 'sleep'. Did you edit the file correctly when you removed the ticks?

No, that is what is so strange, without the ticks, it gives that error message with the missing s. With the tick marks the sleep command actually runs, executed in the script either at boot up or in the command line.

One other thing - are you editing this file on windows and transferring to Linux, or are you editing on Linux. I have seen situations where a file transferred from a PC to Linux has CR/LF at the end of every line - and thtis has thrown off various shell interpreters.

This may be the problem. Yes I am using notepad in windows to edit the scripts and then cp them into place on the NAS. I do not know how to edit the file from the NAS side command prompt. Any suggestions for editor on either side that I can use/install?

DO you have the od command available? If you try 'od -c startHME.sh' what do you see'?

-ash: od: not found

[jcthorne]

Do you need to invoke a shell in Busybox like in Linux? In other words, do you need something like
#!/bin/sh
on the first line of your script so it invokes the interpreter?

Tried that on the first line of the script and the result was just one more

: not foundle Storage/temp/scripts/startHME.sh: line 4:

thanks

[jbernardis]

This may be the problem. Yes I am using notepad in windows to edit the scripts and then cp them into place on the NAS. I do not know how to edit the file from the NAS side command prompt. Any suggestions for editor on either side that I can use/install?

I assume you just have a command line prompt on the NAS.

There are several editors for Linux - I use vi because I have years of muscle memory that make it second nature to me. It's quite arcane though and I wouldn't want to try to explain it.

Most if not all linux/unix environments at least have good old ed line editor - this is what we used to program with years ago.

make a copy of your file, and try the following:
ed startHME.sh
1,$s/.$//
w
q

that should strip the last character off of every line - then give it a try

[jcthorne]

ed is not found on the NAS. I did find vi and a pretty good tutorial on it online. For what little I need to do I think this will work. I opened the script with vi and there was a ^M at the end of each line. I removed them, saved the file and ran it from the command line. Executed without error, I think we are getting somewhere. I just rebooted the NAS with the script in place to run...


SUCCESS!!!

Thank you, Thank you, thank you.

pytivo, HME for python with vidmgr all started from bootup and work!

Whew.

Next step is a java runtime for harmonium but I want to complete a backup of the server before going that far. I have a Java SE for x86 embeded from Oracle that another user has expressed sucess with so....

but thanks again. to everyone. Learned alot in the last few days.

[lrhorer]

I use vi (or actually, the updated version, vim) a lot, in Linux, Solaris, and HP-UX. If you intend to use ssh to manage the files on the NAS, I suggest you get familiar with vi / vim. You could also probably implement XDMCP or an X Client solution on the NAS server along with an X server like Xming running on a Windows machine. Another alternative, however is to implement a *nix friendly Windows editor along with an integrated FTP client in order to handle file editing on the Linux box. I recommend FileZilla and Edit Plus.

[jbernardis]

Thank you, Thank you, thank you.

pytivo, HME for python with vidmgr all started from bootup and work!



Glad to have been of help

[lrhorer]

Do you need to invoke a shell in Busybox like in Linux? In other words, do you need something like
#!/bin/sh
on the first line of your script so it invokes the interpreter?
Well, he has it working, but to respond to your question, the answer is, "Yes, but No."

Unless they are doing something that I would not expect, the user login uses Busybox as its interactive shell, but init does not. Indeed, init does not employ an interactive shell. This means that inherently bootup procedures do not act in quite the same way as a user shell. That's one reason why for any service I want to start I create a run script with bash as the interpreter and call the run script from the init script. Things like nohup, environment variables, and redirection don't always work as one might expect in an init script. It also happens to be easier under Debian (or in your case, Ubuntu) to make very simple, fairly uniform modifications to the skeleton init script and then put all the details in the run script.

[jcthorne]

Irhorer,

I just wanted to say thanks for your detailed answers to my questions. I learned a lot from your posts. Some, I may not know what I learned until later but I will come back to them.

[lrhorer]

Irhorer,
That's lrhorer, if you please, not irhorer.

I just wanted to say thanks for your detailed answers to my questions. I learned a lot from your posts. Some, I may not know what I learned until later but I will come back to them.
You are very welcome. Linux is an extremely flexible and powerful operating system, but consequently it is also enormously sophisticated. Frequently it offers simple ways to skin any cat, but it also offers lots of them, which means there's a lot to know and a lot to learn.

[Iluvatar]

I have a suggestion for wmcbrine's code in pyhme. I have had issues running pyhme on startup with the full path command (eg. 'python /Applications/pyhme/start.py') which of course fails to properly read the config file. The default suggested way to run this is to change to the pyhme directory first and then run 'python start.py' but this gives difficulties when trying to start at login ( I know jcthorne had similar issues on his Synology NAS device)

My suggested code change is this which should fix my issue and allow proper full path execution:

In start.py at line 307 change

config = SafeConfigParser()
config.read('config.ini')


to

config = SafeConfigParser()

p = os.path.dirname(__file__)
config_file = [os.path.join(p, 'config.ini')]
config.read(config_file)


This fix seems to work great for me and allows me to use my startup scripts unedited with just the full path. This should work on other devices to such as jcthorne's Synology that he had trouble with getting to work.

[jbernardis]

I have a suggestion for wmcbrine's code in pyhme. I have had issues running pyhme on startup with the full path command (eg. 'python /Applications/pyhme/start.py') which of course fails to properly read the config file. The default suggested way to run this is to change to the pyhme directory first and then run 'python start.py' but this gives difficulties when trying to start at login ( I know jcthorne had similar issues on his Synology NAS device)

My suggested code change is this which should fix my issue and allow proper full path execution:

In start.py at line 307 change

config = SafeConfigParser()
config.read('config.ini')


to

config = SafeConfigParser()

p = os.path.dirname(__file__)
config_file = [os.path.join(p, 'config.ini')]
config.read(config_file)


This fix seems to work great for me and allows me to use my startup scripts unedited with just the full path. This should work on other devices to such as jcthorne's Synology that he had trouble with getting to work.

I run the delivered HME code, as is, on my Netgear readynas. I have a shell that, among other things, changes into the proper directory and then invokes "python start.py". The shell is what I invoke in my startup.

[Iluvatar]

I run the delivered HME code, as is, on my Netgear readynas. I have a shell that, among other things, changes into the proper directory and then invokes "python start.py". The shell is what I invoke in my startup.

It works for my specific situation and I just thought I would share. Since I have seen another user who had expressed the same issues I figured it may be useful.

[lrhorer]

I'm having an isue with the applications not writing to the log file. Errors like traceback calls are properly logged, but the ordinary text output is not. I daemonize the app during startup, redirecting its output to a log file like so:

#! /bin/bash
PIDFILE=/var/run/pyHME.pid
RUNDIR=/usr/share/pyhme
LOGFILE=/var/log/pyhme.log
ERRFILE=/var/log/pyhme.err
cd $RUNDIR
/usr/bin/nohup $RUNDIR/start.py > $LOGFILE 2> $ERRFILE &
echo $! > $PIDFILE
exit 0
I've tried both the above - writing to separate log files and also writing to a combined logfile by invoking 2>&1. Neither works properly

One Linux guru has suggested it might be because the app is opening a tty, rather than writing to stdout. I don't think that's the case, because

$RUNDIR/start.py > $LOGFILE 2> $ERRFILE

works just fine. The problem of course is that does not daemonize the application. Does anyone have any ideas?

I do the same thing, BTW for pyTivo, the only obvious difference being with pyTivo, I invoke python directly in the nohup call, rather than running a script with python as the interpreter. I suppose I could try the same thing here, but I was wondering if anyone else had any ideas?

Edit: I tried invoking Python, rather than using the embedded interpreter in the start.py file, but it doesn't seem to work.

[lrhorer]

Perhaps it would help to clarify a bit. I do get some info. For example, I get this from vidmgr:

192.168.1.103:42051 - - [22/Jul/2011 18:07:50] "GET /vidmgr/icon.png HTTP/1.0" 200 -
192.168.1.103:42052 - - [22/Jul/2011 18:07:55] "GET /vidmgr/ HTTP/1.1" 200 -
192.168.1.103:42052 - - [22/Jul/2011 18:07:55] Starting HME: vidmgr
192.168.1.103:42052 - - [22/Jul/2011 18:08:13] Ending HME: vidmgr
192.168.1.103:42094 - - [22/Jul/2011 18:08:17] "GET /vidmgr/icon.png HTTP/1.0" 200 -

I don't, however, get the plain text messages from vidmgr or the regular messages from HME like this:

HME Server for Python 0.19
Fri Jul 22 18:10:23 2011 PyTivo Video Manager version 0.7b starting
Loading thumbnail cache
100 thumbnails loaded from cache
Fri Jul 22 18:10:23 2011 Server Starts
Registering: vidmgr

[wmcbrine]

So, your stderr is being redirected, but not your stdout? Because that's all stdout stuff there (which you're trying to send to pyhme.log).

Probably just a question of buffering, then.

[jbernardis]

I could certainly add a flush to vidmgr to see if it was a buffering issue. I'm away for the weekend, but will be a able to try something sunday evening.

[lrhorer]

So, your stderr is being redirected, but not your stdout?
Well, sort of. By default, nohup redirects stderr to stdin, so that any output sent to those two ports should get re-directed to the same target. Also by default, nohup creates a file named nohup.out where the outputs are sent. Explicitly redirecting any or all of these will override the default. This is all seemingly working properly, as long as I don't detach the process (using "&"). With no explicit redirection, nohup creates the nohup.out file and sends stdout and stderr to it. If I explicitly redirect only stdout, then nohup.out is not created and both streams go to the file specified by the redirection. If I explicitly redirect only stderr, then nohup.put is created and stdout goes to it, while stderr goes to the specified file. If I explicitly redirect both streams, then they each go to the file(s) specifically named in the redirect, and once again nohup.out is not created. This all works as documented in the nohup man page. When I detach the process using "&", however, the stdout evaporates.

Because that's all stdout stuff there (which you're trying to send to pyhme.log).
Wel, I don't know what it is supposed to be, but if I redirect stdout to pyhme.log and stderr to pyhme.err via the following

/usr/bin/nohup $RUNDIR/start.py > $LOGFILE 2> $ERRFILE

then

HME Server for Python 0.19
Fri Jul 22 18:10:23 2011 PyTivo Video Manager version 0.7b starting
Loading thumbnail cache
100 thumbnails loaded from cache
Fri Jul 22 18:10:23 2011 Server Starts
Registering: vidmgr

shows up in pyhme.log, and

192.168.1.103:42051 - - [22/Jul/2011 18:07:50] "GET /vidmgr/icon.png HTTP/1.0" 200 -
192.168.1.103:42052 - - [22/Jul/2011 18:07:55] "GET /vidmgr/ HTTP/1.1" 200 -
192.168.1.103:42052 - - [22/Jul/2011 18:07:55] Starting HME: vidmgr
192.168.1.103:42052 - - [22/Jul/2011 18:08:13] Ending HME: vidmgr
192.168.1.103:42094 - - [22/Jul/2011 18:08:17] "GET /vidmgr/icon.png HTTP/1.0" 200 -

shows up in pyhme.err. If I redirect both to pyhme.log via the following

/usr/bin/nohup $RUNDIR/start.py > $LOGFILE

then both show up in pyhme.log, or if I don't redirect anything, then nohup shoves them both to nohup.out, all as it should be. When I detach the process using "&", then this

HME Server for Python 0.19
Fri Jul 22 18:10:23 2011 PyTivo Video Manager version 0.7b starting
Loading thumbnail cache
100 thumbnails loaded from cache
Fri Jul 22 18:10:23 2011 Server Starts
Registering: vidmgr

disappears.

Probably just a question of buffering, then.
Howso?

[lrhorer]

I could certainly add a flush to vidmgr to see if it was a buffering issue. I'm away for the weekend, but will be a able to try something sunday evening.
Hmm. Isn't this:

HME Server for Python 0.19
from HME for Python, not vidmgr? If so, how would flushing the vidmgr buffers help, and why doesn't subsequent text from vidmgr come through? What's more, I am having trouble seeing how an unflushed buffer could eliminate text on both sides (separated by several minutes) of

192.168.1.103:42051 - - [22/Jul/2011 18:07:50] "GET /vidmgr/icon.png HTTP/1.0" 200 -
192.168.1.103:42052 - - [22/Jul/2011 18:07:55] "GET /vidmgr/ HTTP/1.1" 200 -
192.168.1.103:42052 - - [22/Jul/2011 18:07:55] Starting HME: vidmgr
192.168.1.103:42052 - - [22/Jul/2011 18:08:13] Ending HME: vidmgr
192.168.1.103:42094 - - [22/Jul/2011 18:08:17] "GET /vidmgr/icon.png HTTP/1.0" 200 -

yet allows the stuff in the middle through.

[jwagner010]

Does anyone have a simple python program developed that shows a list of menu items on the Tivo that when selected executes a sub process that I can leverage?

[jwagner010]

Does anyone have a simple python program developed that shows a list of menu items on the Tivo that when selected executes a sub process that I can leverage?

I found a TIVO HME App that provides a Tivo menu based on a properties file that then executes URLs from that properties file. See URLMenu on this page http://david.blackledge.com/tivo/

urlmenu.properties should have entries like so:
My\ First\ Menu=http://url.to.be.executed.com/with/other?information
My\ Second\ Menu=http://url.to.be.executed.com/with/other?information

Problem is it is based on the the JAVA HME not Python HME. I would like to build a similar app/plugin for the Python HME but do not know Python. Does anyone know of a similar app or can point me in the right direction to try and figure it out bearing in mind I have very basic programing skills (1980's basic) and zero of that is with Python.

PS. rather than execute URLs it would be good if it can execute OS commands, and if I want a URL to be executed I can do a wget or curl.

[wmcbrine]

I would like to build a similar app/plugin for the Python HME but do not know Python.OK. Why then?

Take a look at HME/VLC for one way to do menus...

[jwagner010]

I will take a look at HME/VLC. Thanks

I want to do this for two reasons, both very similar:

1) From my Tivo I want to be able to turn on and off my lights in the house. My home automation controller has a REST interface so I want to send HTTP GET requests (preferrably through wget) to turn on and off my lights based on menu selections on the Tivo.

2) Simlar to 1) above I would like to be able to control my home audio system that also has a similar interface.

[lrhorer]

Hey, that's rather cool. It would be nice to be able to control the lights and the projector sreen in the theater.

[wmcbrine]

No, I mean, why do you want to do it in Python, if you don't know Python anyway? Why not just use David Blackledge's program?

[jwagner010]

For two reasons:

1). David's program doesn't let you pass user id and password with the URL get

2) I run the server on a nas that doesn't have java but does have python

On a seperate note I installed HMEVLC. Very cool. Has anyone connected an IP camera to this? I might try stream the mjpeg from a couple of Foscam cameras at home

[jwagner010]

On a seperate note I installed HMEVLC. Very cool. Has anyone connected an IP camera to this? I might try stream the mjpeg from a couple of Foscam cameras at home

I figured out the http to get the mjpeg stream from my camera but I cannot get vlc to run on my synology nas due to the way vlc and dbus are installed. I was told that I need to use the following syntax: "dbus-launch cvlc -I http://playlist.yahoo.com/makeplaylist.dll?id=1368162"

Any suggestions on the python code that needs to change to make this happen?

[wmcbrine]

You shouldn't need to modify the code. Just add a line to config.ini, in the hmevlc section:

vlc = cvlc

(but give the full path). I don't think you should need dbus. (?)

[jwagner010]

You shouldn't need to modify the code. Just add a line to config.ini, in the hmevlc section:

vlc = cvlc

(but give the full path). I don't think you should need dbus. (?)

I tried your suggestion but I think I need the dbus syntax. When I add vlc=/fullpath/cvlc it throws the following errors. My (very limited) understanding is the x11 error is due to this being run on a NAS

192.168.0.5:49832 - - [04/Mar/2012 00:12:13] Ending HME: hmevlc
192.168.0.5:50850 - - [04/Mar/2012 00:39:58] "GET /hmevlc/ HTTP/1.1" 200 -
192.168.0.5:50850 - - [04/Mar/2012 00:39:58] Starting HME: hmevlc
VLC media player 0.9.9a Grishenko
[00000001] main libvlc debug: VLC media player - version 0.9.9a Grishenko - (c) 1996-2009 the VideoLAN team
[00000001] main libvlc debug: libvlc was configured with ./configure '--build=i386-pc-linux-gnu' '--host=i686-linux-gnu' '--target=i686-linux-gnu' '--prefix=/opt' '--enable-v4l' '--disable-v4l2' '--enable-bonjour' '--enable-x264' '--enable-dvbpsi' '--enable-a52' '--enable-dvdnav' '--with-dvdnav-config-path=/home/slug/optware/syno-i686/staging/opt/bin' '--enable-faad' '--enable-flac' '--disable-gnutls' '--enable-mpc' '--enable-ncurses' '--enable-ogg' '--enable-png' '--disable-remoteosd' '--enable-shout' '--enable-speex' '--enable-vorbis' '--disable-alsa' '--disable-dca' '--disable-glx' '--disable-gnomevfs' '--disable-libcdio' '--disable-libcddb' '--disable-screen' '--disable-sdl' '--disable-wxwidgets' '--disable-skins2' '--disable-x11' '--disable-nls' '--disable-static' 'build_alias=i386-pc-linux-gnu' 'host_alias=i686-linux-gnu' 'target_alias=i686-linux-gnu' 'CC=/home/slug/optware/syno-i686/toolchain/gcc-4.2.1-glibc-2.3.6/i686-linux-gnu/bin/i686-linux-gnu-gcc' 'LDFLAGS= -L/home/slug/optware/syno-i686/staging/opt/lib -Wl,-rpath,/opt/lib -Wl,-rpath-link,/home/slug/optware/syno-i686/staging/opt/lib ' 'CPPFLAGS=-O2 -O2 -pipe -I/home/slug/optware/syno-i686/staging/opt/include -I/home/slug/optware/syno-i686/staging/opt/include/ncurses' 'CPP=/home/slug/optware/syno-i686/toolchain/gcc-4.2.1-glibc-2.3.6/i686-linux-gnu/bin/i686-linux-gnu-gcc -E' 'CXX=/home/slug/optware/syno-i686/toolchain/gcc-4.2.1-glibc-2.3.6/i686-linux-gnu/bin/i686-linux-gnu-g++' 'PKG_CONFIG_PATH=/home/slug/optware/syno-i686/staging/opt/lib/pkgconfig'
[00000001] main libvlc debug: translation test: code is "C"
[00000298] inhibit interface error: Failed to connect to the D-Bus session daemon: /opt/bin/dbus-launch terminated abnormally with the following error: Autolaunch requested, but X11 support not compiled in.
Cannot continue.

[00000298] main interface error: no suitable interface module
[00000001] main libvlc error: interface "inhibit,none" initialization failed
[00000300] dummy interface: using the dummy interface module...
[00000306] main access out: creating httpd
[00000306] main access out error: socket bind error (Permission denied)
[00000306] main access out error: socket bind error (Permission denied)
[00000306] main access out error: cannot create socket(s) for HTTP host
[00000306] access_output_http access out error: cannot listen on port 9028
[00000305] stream_out_standard stream out error: no suitable sout access module for `http/ps://:9028'
[00000303] stream_out_transcode stream out error: cannot create chain
[00000302] main stream output error: stream chain failed for `transcode{vcodec=mp2v,vb=2048,acodec=a52,ab=384,audio-sync,samplerate=48000,fps=30}:std{access=http,dst=:9028,mux= ps}'
[00000301] main input error: cannot start stream output instance, aborting

Any suggestions on what I need to change in the python code in order to be able to use the dbus syntax (dbus-launch cvlc -I http://urltocall) in the call to cvlc ?

[jpolk5000]

Greetings All,

I was unaware of pyHME until today....Hats off to Mr. McBride!

I've been using Python and Qt for several years, and having been a TiVo user
for over 15 years, I, like many others I'm sure, have a lot of ideas I'd like to try
to implement.

Where can I download the pyHME module?

Cheers,

[wmcbrine]

Hats off to Mr. McBride!

Thanks, but it's McBrine.

Where can I download the pyHME module?

It's linked from the first post in the thread, and in my sig...

[lrhorer]

having been a TiVo user
for over 15 years
That seems unlikely, since the TiVo was introduced in 1999. Even if you participated in one of the service trials in 1998, that would still only be a little over 13 years.

[johnh123]

OK, so i've got it running on my synology nas, and a couple of the apps work, but then I try tic tac toe and get this:

Exception happened during processing of request from ('192.168.1.112', 35176)
Traceback (most recent call last):
File "/opt/lib/python2.5/SocketServer.py", line 464, in process_request_thread
self.finish_request(request, client_address)
File "/opt/lib/python2.5/SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/pyhme/start.py", line 141, in __init__
client_address, server)
File "/opt/lib/python2.5/SocketServer.py", line 522, in __init__
self.handle()
File "/opt/lib/python2.5/BaseHTTPServer.py", line 316, in handle
self.handle_one_request()
File "/opt/lib/python2.5/BaseHTTPServer.py", line 310, in handle_one_request
method()
File "/usr/local/pyhme/start.py", line 232, in do_GET
self._page(True)
File "/usr/local/pyhme/start.py", line 186, in _page
appinst.mainloop()
File "/usr/share/pyhme/hme.py", line 1134, in mainloop
self.startup()
File "/usr/share/pyhme/tictactoe/__init__.py", line 32, in startup
File "/usr/share/pyhme/hme.py", line 1040, in set_image
self.set_resource(Image(self.app, name, f, data), flags)
File "/usr/share/pyhme/hme.py", line 735, in __init__
f = open(name, 'rb')
IOError: [Errno 2] No such file or directory: 'tictactoe/bg.jpg'


and I checked, and bg.jpg is in fact there...

any thoughts?

[wmcbrine]

"/usr/local/pyhme/start.py" ... "/usr/share/pyhme/hme.py" ... any thoughts?

I see that you've split it into two directories. I'm not sure if that's going to work, but if it is, you'll at least need this in your config.ini:

basepath = /usr/share/pyhme

Edit: Actually, as I look at it, tictactoe doesn't even check the basepath (nor the datapath), just assumes that the starting directory is the one above the app directory (in your case, /usr/share/pyhme). I guess I might do something about that... but in the meantime, you should probably start up like this:

cd /usr/share/pyhme
/usr/local/pyhme/start.py

[johnh123]

Ah, I see that. There are two directories because the way I originally installed it it went to share, then I saw that others were using local so I switched it - i'll have to try and track down any mentions of share and change to local. Thanks.

[johnh123]

OK, so I started over. Got hme running with all the apps. Have installed vidmgr there. I get this when it runs:

192.168.1.125:41301 - - [18/Apr/2012 22:23:16] Starting HME: vidmgr
Vidmgr thread entering startup
----------------------------------------
Exception happened during processing of request from ('192.168.1.125', 41301)
Traceback (most recent call last):
File "/opt/lib/python2.5/SocketServer.py", line 464, in process_request_thread
self.finish_request(request, client_address)
File "/opt/lib/python2.5/SocketServer.py", line 254, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "start.py", line 141, in __init__
client_address, server)
File "/opt/lib/python2.5/SocketServer.py", line 522, in __init__
self.handle()
File "/opt/lib/python2.5/BaseHTTPServer.py", line 316, in handle
self.handle_one_request()
File "/opt/lib/python2.5/BaseHTTPServer.py", line 310, in handle_one_request
method()
File "start.py", line 232, in do_GET
self._page(True)
File "start.py", line 186, in _page
appinst.mainloop()
File "/usr/local/pyhme/hme.py", line 1134, in mainloop
self.startup()
File "/usr/local/pyhme/vidmgr/__init__.py", line 108, in startup
raise ConfigError("No Tivos found - exiting")
ConfigError: No Tivos found - exiting

the pyhme config.ini is like so:

[hmeserver]
apps=vidmgr

[vidmgr]
exts=.mpg .mp4 .avi .wmv .m4v .mkv
descsize=16

[tivos]
tivo1.name=Basement
tivo1.tsn=xxx-xxxx-xxxx-xxxx

[pytivos]
pytivo1.config=/usr/local/pyTivo/pyTivo.conf
pytivo1.ip=192.168.1.113
pytivo1.port=9032

what next?

[wmcbrine]

That's not an HME for Python issue. Please take it to the VidMGR thread.

[windracer]

Like wmcbrine said this should be in the vidmgr thread, but it looks like you've got the wrong stuff in your ini file. The config.ini file for HME for Python should only contain the [hmeserver] section. The [vidmgr] and [pytivos] sections should be in the vidmgr.ini in the vidmgr folder and the [tivos] section should be in your pyTivo.conf.

[wmcbrine]

is now available. Only minor changes.

[wsware]

Does this work on the Tivo Premiere? I could not get anything to show up using just a clean install using the SD or HD menus. I didn't spend a lot of time on it so I can play with the config next. I just installed it and left it running then checked my Tivo a few hours later.

Also, Are there just not many HME apps out there? I seem to have a hard time finding any. Would there happen to be a list of them?

PlayOn under pyTivo is ok, but it seems like it would make a much better HME app.

[wmcbrine]

Does this work on the Tivo Premiere?

Yes.

I just installed it and left it running then checked my Tivo a few hours later.

It would make more sense to check it immediately.

There are any number of reasons why it might fail -- firewall issues, unreliable wi-fi, a brand-new TiVo or a TiVo otherwise not set up to allow network apps... and the classic, users looking in the wrong menus. :)

Also, Are there just not many HME apps out there? I seem to have a hard time finding any.

Indeed there are not.

Would there happen to be a list of them?

http://apps.tv/

[lrhorer]

Vidmgr is not on that list, adn IMO it is a terrific app. 'Blows the NPL away (for content on the server).

[cassiusdrow]

I'm seeing a small cosmetic issue with the HME app names in the "Music, Photos, & Showcases" list. On my Series 3 OLED they look like this:

Jukebox
PyTivo Video Manager
Tic Tac Toe

On my Premiere XL they look like this:

Jukebox
PyTivo\032Video\032Manager
Tic\032Tac\032Toe

It looks like the spaces are replaced with the character code escape sequence or something.

[windracer]

This is a problem with the 20.x software, not the HME apps themselves.

[cassiusdrow]

While the 20.x software may be causing the difference, this only apps with this issue are running via HME for Python. No TiVo supplied apps do this.

[wmcbrine]

The apps that do it are apps 1) with spaces in their names, that 2) are announced via Zeroconf on the LAN. It's not HME for Python. If you want to see the same effect from a TiVo-supplied app, run their Java SDK.

If you want to see HME for Python not doing it, turn off Zeroconf, set it to run on port 80, and do "Manually add a server" from the SD menus. (It won't work properly in the HD menus, unless you use the Git version of HME for Python, due to other recent changes (bugs) in how the HD menus handle HME. Version 0.21 coming soon...)

More details here: http://www.tivocommunity.com/tivo-vb/showthread.php?t=497441

[cassiusdrow]

Thank you for providing further explanation.