Suggs: hack to create suggestions

mikerr · 09-11-2008, 04:16 PM

Suggs: the new suggestion engine ! V 1.04
a hack to recreate suggestions.

This hack will run in the background and set suggestions every 30 minutes.

Basically looks at the schedule of the current tv channels,
and tries to record some of the upcoming programmes as suggestions

The threshold for choosing programmes can be set:
default is neutral ( i.e. record all except thumb-downed shows)
This default option good for generating lots of suggestions
To run this way:

./suggs.tcl

To record only those with 1 thumb up or more:

./suggs.tcl -threshold 127

To record those with "predicted relevance":

./suggs.tcl -threshold 1

this last example is closest to how tivo suggestions worked.

they will show up in tivoweb scheduled suggestions
(http://tivo/ui/suggestions )
but not in the tivo GUI suggestions menu.

To show scheduled suggestions on the tivo GUI todo list,
navigate to the todo list on your tivo, and press:
Thumbs Down, Thumbs Down, Thumbs Up, Instant Replay

Being suggestions, they never interfere with your season passes or wishlists.

- Doesn't record duplicate suggestions,
- Won't record a show as a suggestion if it is present in now playing
( e.g. use suggs.tcl -daysback 28 if you don't want another episode for 28 days)
- Gives priority to thumbed shows, but also has predicted thumbs
- Predicted thumbs based on genres
- By default only schedules one day ahead, use "-days 2" to schedule 2 days ahead
-
- Shares the blocklist.txt and blockgenres.txt from global blocklists
(E.g. block all football programmes from being recorded as suggestions by putting "Football" in blockgenres.txt , or "Children", "News" etc one per line)
- Takes account of padding

# 1.04 Some duplicate checking code
# 1.03 No stuttering!
# 1.02 - only processes channel guide data once per day
#
# 1.01 Uses tivo's database of genre thumbs

a_tivo_noob · 09-29-2008, 05:37 PM

is anyone else using this other than mikerr? sounds like just what we need now!

quick (stupid) question mikerr, but the txt file accompanying it states

Quote:

# record suggestions on current channel

- does it not change channel then or am i reading it wrong

mikerr · 09-29-2008, 05:50 PM

Quote:

Originally Posted by a_tivo_noob

is anyone else using this other than mikerr? sounds like just what we need now!

It was originally a 10 minute hack to help out when suggestions were flaky,
but now as it is the only way of getting suggestions I'll spend some time on it,

and also include it with preconfigured drives

Quote:

quick (stupid) question mikerr, but the txt file accompanying it states - does it not change channel then or am i reading it wrong

Not in the current version, no.

Basic operation is as follows:

It looks a few hours ahead into the tv guide for the current channel,
if any programme has a thumbs up it will record it as a suggestion.

Version 0.3 will look across all channels and pick from there.

a_tivo_noob · 09-29-2008, 06:17 PM

nice one - thx for clarification and keep up the good work - any ideas on when a v0.3 release will be (if its in the next week or two i'll hold off v0.2)

andonevris · 09-29-2008, 06:33 PM

The fate of my tivo rests in your hands mikerr, if I can't get suggestions working again to a reasonable level I'll be jumping ship, I hope you can pull it off.

Looking forward to your 0.3 release

Mike Rogers · 09-29-2008, 07:06 PM

mikerr you're a star.

Many thanks for this!

kitschcamp · 09-30-2008, 07:11 AM

Quote:

Originally Posted by mikerr

Version 0.3 will look across all channels and pick from there.

Any chance of making it so the script will only check and add suggestions if it isn't currently recording a program? With tivoweb, endpad and irblast and this at the same time you get "stuttering" whilst it is processing.

mikerr · 09-30-2008, 07:29 AM

The stutter is probably on first access to the database, common to many hacks:

Code:

set db [dbopen]

It doesn't stutter after the first few seconds does it?

kitschcamp · 09-30-2008, 09:00 AM

His nibbs at home has been complaining of TiVo having a stammer today, so it looks like it is at times. I've put it down to too many hacks, and this was the last one I added.

TCM2007 · 09-30-2008, 09:17 AM

Why every 30 mins? Wouldn't once a day be enough with this way of working?

mikerr · 09-30-2008, 10:52 AM

I've found they get removed from the todo list after a while,
even if there is nothing else scheduled at that time.

I should probably be sending a todolist-changed signal, but can't find it.

[edit] Oh.. might have found it:

Code:

event send $TmkEvent::EVT_DATA_CHANGED $TmkDataChanged::SCHEDULE 0

ColinYounger · 09-30-2008, 11:11 AM

I tried that event before when trying to fix the restore thumbs problem, but it didn't affect the scheduling process.

ColinYounger · 09-30-2008, 11:17 AM

Interesting link: Broad overview of suggestions. No technical detail.

kitschcamp · 09-30-2008, 11:30 AM

Quote:

Originally Posted by mikerr

I've found they get removed from the todo list after a while,
even if there is nothing else scheduled at that time.

Ah, I did wonder - I did notice one appear then disappear earlier today.

kitschcamp · 10-01-2008, 12:11 AM

Quote:

Originally Posted by mikerr

The stutter is probably on first access to the database, common to many hacks:

Code:

set db [dbopen]

It doesn't stutter after the first few seconds does it?

I was at home last night, and yes, every thirty minutes, regular as clockwork.

mikerr · 10-01-2008, 02:28 AM

Ah thanks, noted.

kitschcamp · 10-01-2008, 02:34 AM

I've turned off irblast today to see if it's just a simple case of too many things at once running. I'll soon know if it's ok

6022tivo · 10-01-2008, 05:00 AM

Just a quickie on this, so does it add "suggestions" and do they get recorded as the little tivo icon next to them (Not blank, Yellow, Green DOT?)

Do they go to the bottom of the list and not get in the way of "proper" recordings???

If so.... This is ace..

mikerr · 10-01-2008, 05:13 AM

Yes to all the above. They appear at the bottom as a suggestion, and doesn't interfere with other recordings. Tivo actually is recording it as normal suggestion.

Indistinguishable from the "real thing" apart from:

1) Pending suggestions don't appear in the TV tivo suggestions menu
(you have to use tivoweb (http://tivo/ui/suggestions) to see the todo list)

2) At present only records on the current channel, but this is just in testing.

Pugwash · 10-01-2008, 05:23 AM

mikerr, you've clearly done something TiVo have been unable to do (for whatever reason). I'm sure if you can get this working across all channels many people would shake your hand.

mikerr · 10-01-2008, 03:42 PM

New version uploaded, now covers all channels you receive.

6022tivo · 10-01-2008, 03:55 PM

Mike you are amazing.. I will upload this to mytivo in the next few days...

Thank you for all the time you have put into this...

Pugwash · 10-01-2008, 04:11 PM

Out of interest, do we need to kill the previous version's process (how?) or will replacing the file make it restart?

p.s. This is excellent work. I have suggestions in TiVoWebPlus across all channels now, which I assume means the new one is running by just copying the file over the top and running it.

Mostin · 10-01-2008, 04:12 PM

Nice one mike, i'll be in touch for a new hd soon

andonevris · 10-01-2008, 07:06 PM

Mikerr....this is VERY cool. Fired it up and started seeing suggestions filling up straight away. They look like the sort of stuff tivo would record for me when its own suggestions were working.

Thanks very much for this.....

Tivo should give you a job

kitschcamp · 10-02-2008, 01:30 AM

Hi Mike,

Does it just run once in the background then close down? I'm asking because it doesn't seem to be staying running on mine in the background, and when I check back after about an hour and run

ps ax

The process is no longer running, but there is nothing I can see in the logs to say it closed.

It has, however, added some entries to the scheduled suggestions recording, so that definitely works!

Many thanks for putting in the effort.

mikerr · 10-02-2008, 02:47 AM

Quote:

Originally Posted by ColinYounger

Interesting link: Broad overview of suggestions

Quote:

TiVo Suggestions currently works by sending all the ratings across to their server cluster from each TiVo, generate correlations between the shows, and send back the list of correlations to each TiVo

Ours don't (didn't!) work that way..the US tivos work the above way, including US S1 tivos.
TivoPony mentioned that in the suggestions announcement thread.

Our UK tivos process all suggestions on the tivo itself.

[everyone] What would your opinions be if I added something similar (upload thumbs) to this hack ?
Obvious privacy implications as I'd be uploading all your thumbs to my website for processing,
but it would produce better results.

Quote:

Originally Posted by kitschcamp

it doesn't seem to be staying running on mine in the background [...] but there is nothing I can see in the logs to say it closed.

It's possible it has closed with an error, logs are in /var/log/suggs.log

[edit] Argh ! 0.4 does only run once, due to a silly bug

The code

Code:

-console { set bg true ; set runonce true }

should be

Code:

-console { set bg true }

up with 0.5 ....

ag5002 · 10-02-2008, 02:57 AM

Quote:

Originally Posted by andonevris

Mikerr....this is VERY cool. Fired it up and started seeing suggestions filling up straight away. They look like the sort of stuff tivo would record for me when its own suggestions were working.

Thanks very much for this.....

Tivo should give you a job

Very clever and cool. If the Nero product ever comes to the UK, I assume you will be building new Tivo boxes for everyone?

Perhaps you could even take over the entire Tivo UK service.

kitschcamp · 10-02-2008, 03:44 AM

Quote:

Originally Posted by mikerr

Argh ! 0.4 does only run once, due to a silly bug

Thanks! I'll test it again now.

AMc · 10-02-2008, 03:54 AM

Thanks for adding the attachment

Is there a reason it runs every 30 minutes - would there be any harm in running it less frequently - say once a day at 3am with cron instead?