DailyMail: a daily e-mailed status report from your TiVo

sanderton · 10-28-2003, 06:04 PM

Note: the original version of DailyMail has now been superceded by tivo_onion's Jazz version, which can be found lower down the thread (here at the the time of writing.)

Ian was mentioning the other day about his daily routine for checking that his TiVo was all set up for the day's recordings. I know that I haven't the patience for that - heck, I know I haven't even got the patience to check via TiVoWeb on my coffee break. If such a system was to work for me, it would have to be more pro-active.

So I wrote this script.

It sends me an e-mail at work every day, telling me:

* If the TiVo has rebooted recently
* If the last daily call failed
* What unresolved conflicts are coming up
* What programmes are about to expire
* What the TiVo is planning to record that day

Here is an example of what it sends.

I wrote it for my own amusement, and I certainly don't want to get into a big round of bug fixing and beta testing, but I offer it here in case its of use to anyone else, or if someone wants to use it as the basis for something better.

Most of the code is adapted, filched and straight copied from various TiVoWeb modules - if I've omitted a credit in the code my apologies. I thank those authors for their great work.

To use it you'll need to edit the .tcl file - preferably on the TiVo using joe or vi, or on the PC if you have a Unix compatible text editor. If not you'll need to run it through DOS2UNIX or similar to get it back in Unix format. There are some variables at the top you'll need to set as a minimum:

- the e-mail address to send the mail to
- the IP address of your ISP's SMTP server (must be an IP as TiVo has no DNS lookup)
- your domain name

Some have also found it necessary to tweak the exact commands sent to the mail server - see the posts below for details if you are having problems; most have been incorporated into the code now. In particular, if your ISP mail server requires authentication see here

The file dailymail.tcl will need to be made executable with

chmod 755 dailymail.tcl

You need to have TiVoWeb 1.9.4 installed - DailyMail is not compatible with the TiVoWebPlus hack, but there are workarounds, see below. And an always-on internet connection. This script will not work using the TiVo's modem - it must be able to connect to the net via your network

I have a cron job set up to run it every morning.

Feel free to take this code and run with it - just please post anything cool you add or fix!

bobnick · 10-28-2003, 06:27 PM

Will dailymail automatically include the latest number of asylum seekers and the dangers of interest rates rising, or do I need to edit the tcl file?

Lysander · 10-28-2003, 06:53 PM

Stuart,

Two things.

1. Haven't you got a job? (Tivo are you listening?)

or

2. Haven't you got a life?

Some amazing stuff kicking around in that head of yours....

James

tivo_boj · 10-28-2003, 07:22 PM

Great, now to make me look thick (not hard generally)?

How do I find the IP address of my ISP's SMTP server (mail.btconnect.com)?

my domain - I take this as the @xxxx bit

as my emails address is xxxxxxxx@btconnect.com the domain is btconnect.com , True?

have not a clue how to set up a a "cron job" to run it every morning. How do I get it to send every morning?

thanks

m3geezer · 10-28-2003, 07:26 PM

how do i find the ip address of the smtp server?

look in outlook or whatever your mail client is and see what your outgoing mailserver is called, then if you run win nt/2000/xp type on a command line 'nslookup <the server name here>'

not sure about tivo but in unix 'crontab -e' allows you to edit the crontab, you want to make a timed entry for a specific time each day, under day you enter '*' to mean everyday, i think i'm right in saying 0 is sunday, 1 is monday, upto saturday which is 6, if you only want it on specific days.

maybe someone can clarify this.

bobnick · 10-28-2003, 07:31 PM

Script works just great - cheers!

(Eventually) tracked down Cron - http://www.tivocommunity.com/tivo-vb...highlight=cron

Looks v. complicated - I'll have to work out how to get it to run each morning tomorrow I think! Anyone got any tips?

BTconnect should have an IP of 193.113.154.2 - hit start, then run in windows, then type CMD and press enter. then just type ping mail.btconnect.com to find out the ip address. And yes, that's your domain - but I don't think it does anything other than form the 'From' address in the Tivo email.

randymc · 10-28-2003, 07:39 PM

I get the following error:

object not found (errNmNameNotFound)

while executing
"mfs find $guideindexdir/ChannelTable"
("uplevel" body line 2)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set channeltablefsid [lindex [mfs find $guideindexdir/ChannelTable] 0]
set channeltabledata [mfs get $channeltablefsid]..."
(procedure "init_channelindex" line 58)
invoked from within
"init_channelindex"
(file "/var/hack/tivoweb-tcl/modules/index.itcl" line 658)
invoked from within
"source /var/hack/tivoweb-tcl/modules/index.itcl"
(file "/var/hack/dailymail.tcl" line 445)

Any help?

Thanks,
rAndy

tivo_boj · 10-28-2003, 07:40 PM

Thanks, think I have got it - except for the crone bit. Tool late to worry tonight ( just got back from a 9-8 penalty shoot out?).

If anyone out there is sleepless, maybe an idiots guide exacly what I need to run on the tivo for the "cron" stuff. Got tivoweb etc etc working , but only by following idiots guides

Automan · 10-29-2003, 02:09 AM

Quote:

Originally posted by tivo_boj
Great, now to make me look thick (not hard generally)?

How do I find the IP address of my ISP's SMTP server (mail.btconnect.com)?

my domain - I take this as the @xxxx bit

as my emails address is xxxxxxxx@btconnect.com the domain is btconnect.com , True?

Try ping mail.btconnect.com from the command prompt on your PC.

Some of the big email providers can have more than one mail server (to ease the load / increased reliability). Thus it's possible when you ping a mailserver by it's DNS name you make get a diferent IP number.

Automan.

sanderton · 10-29-2003, 05:02 AM

Quote:

Originally posted by randymc
I get the following error:

object not found (errNmNameNotFound)

while executing
"mfs find $guideindexdir/ChannelTable"

rAndy

Check the version3 variable at the start of the code, should be false if v2 or true if v3. No idea about v4.

sanderton · 10-29-2003, 05:17 AM

cron's a bit complicated.

I did it like this (from memory so replace the xxx's with the real names!):

1) copy the xxx.tar.gz file to /var/hack
2) gzip -d xxx.tar.gz to get to the xxx.tar
3) cpio -idu -H tar < xxx.tar which will create a directory of xxx
4) cd xxx to that directory
5) Make everything there executable with chmod 755 *
6) There is a .sh script in the directory, run it with ./xxx.sh which will create a load of subdirectories of /var/hack
7) Copy the crontab file (the text file which sets what cron does) to /var/hack/etc with mv crontab /var/hack/etc/crontab
8) Copy the cron program itself to /var/hack/bin with mv cron /var/hack/bin/cron
9) If you've followed one of the usual hacking guides /var/hack/bin will be in your path so you can now run cron with cron &, if not use /var/hack/bin/cron &
10) Open the /var/etc/crontab file with joe or your preferred editor. The last line is an example cron job wich just modifies a file to see if its working. Delete that line, and to run dailymail.tcl every weekday at 8am add this one:

0 8 * * 1-5 /var/hack/dailymail.tcl &

11) So it survives reboots, add the line
/var/hack/bin/cron &
to rc.sysinit.author

racingclub · 10-29-2003, 08:23 AM

snds a nice script - will try it later..................

Dapper Dan · 10-29-2003, 01:46 PM

Blimey, that's good

Do you have trouble sleeping with all these thoughts that keep going through your head ?

bobnick · 10-29-2003, 03:48 PM

Thanks for the cron help - i had to change a few bits to make it work with the cron link above, but it looks like it will work - we'll find out in the morning!

Dailymail caused my Tivo to reboot this afternoon when it was run through a bash prompt - no idea why, sorry!

tivo_boj · 10-29-2003, 04:39 PM

Quote:

Originally posted by sanderton
cron's a bit complicated.

I did it like this (from memory so replace the xxx's with the real names!):

1) copy the xxx.tar.gz file to /var/hack
2) gzip -d xxx.tar.gz to get to the xxx.tar
............. etc

being really thick now - what is programme xxx, anyone know the real name and where I ge it

horwitz · 10-29-2003, 05:48 PM

Quote:

Originally posted by tivo_boj
being really thick now - what is programme xxx, anyone know the real name and where I ge it

You'll find it in the lost post of the thread pointed to here:

Quote:

Originally posted by bobnick
Script works just great - cheers!

(Eventually) tracked down Cron - http://www.tivocommunity.com/tivo-vb...highlight=cron

Looks v. complicated - I'll have to work out how to get it to run each morning tomorrow I think! Anyone got any tips?

BTconnect should have an IP of 193.113.154.2 - hit start, then run in windows, then type CMD and press enter. then just type ping mail.btconnect.com to find out the ip address. And yes, that's your domain - but I don't think it does anything other than form the 'From' address in the Tivo email.

Specifically, go here and get cron-3.0pl1_TiVo-2.tar.gz (that should tell you what "xxx" is, too).

bobnick · 10-29-2003, 05:50 PM

Read further up the thread, and follow the readme files that come with it - you'll need to change the insructions in this thread slightly (as the files go in a different place)

horwitz · 10-29-2003, 06:02 PM

Quote:

set domain yourdomain.com ;# your domain

What do I do here? I'm sending the email to a hotmail.com address (and, as such, am using their mail server). I assume this has nothing to do with my Windows domain nor with my ISP.

p.s. This sounds great -- exactly what I do by hand and wish I had a quick tool for!

tivo_boj · 10-29-2003, 06:15 PM

Thanks,

got it now, just configuring files to see if I can get the cron job working

sanderton · 10-29-2003, 06:36 PM

Quote:

Originally posted by horwitz
What do I do here? I'm sending the email to a hotmail.com address (and, as such, am using their mail server). I assume this has nothing to do with my Windows domain nor with my ISP.

You need to set this to be the internet domain that you are sending from and which appears in your outgoing e-mails, often something like accountname.isp.com.

The mailserver needs to be the IP addess of your ISP's outgoing SMTP server, not hotmail. Check your e-mail client's preferences to see what the name of it is, then (as suggested above) use ping to find out the IP address.

sanderton · 10-29-2003, 06:38 PM

Quote:

Originally posted by bobnick
Read further up the thread, and follow the readme files that come with it - you'll need to change the insructions in this thread slightly (as the files go in a different place)

My files are where I've said above - and that agrees with the locations in the readme??

bobnick · 10-29-2003, 06:57 PM

The version of cron I linked to above seems to want things in different places:

Quote:

The main crontab file should be placed at /var/hack/etc/crontab. Log files,pid file, spool, etc, will show up under /var/hack/cron/.

I presume there's a couple of versions of cron floating around. The changes don't seem to be major - just keep an eye on the files you download!

Or perhaps I've screwed everything up and I'm not going to get a mail in the morning

sanderton · 10-29-2003, 07:13 PM

Er, that's where I said to put them??

Quote:

The main crontab file should be placed at /var/hack/etc/crontab.

Quote:

7) Copy the crontab file (the text file which sets what cron does) to /var/hack/etc with mv crontab /var/hack/etc/crontab

cron itself can go anywhere you like.

horwitz · 10-29-2003, 08:21 PM

error

Quote:

couldn't open socket: network is unreachable
while executing
"socket $mailserver 25"
(procedure "sendmail" line 3)
invoked from within
"sendmail $mailserver $domain $recipient "Daily status e-mail" $message"
(file "./dailymail.tcl" line 512)

The only changes I made:

Quote:

#set variables
set mailserver 198.144.192.41 ;# the IP address of your ISP's SMTP server
set domain rawbw.com ;# your domain
set recipient jahorwitz@hotmail.com ;# the email address to send to
set hoursahead 36 ;# number of hours ahead to look for enclosed info
set version3 true ;# change to true if you are on v3
set uktivo false ;# change to true if you are a colonial

I have a (US) series 1 v3.0 and 198.144.192.41 is smtp.rawbw.com (which succesfully responds to pings). Help, please ...

tivo_boj · 10-30-2003, 03:04 AM

run the cron, got this this morning

cron (10/30-08:00:00-210) MAIL (mailed 85 bytes of output but got status 0x0001
)
*system* (10/30-08:08:00-201) RELOAD (/var/hack/etc/crontab)

in the cron log file

Any Clues

UPDATE

pinged my mailserver but timeout (ie did not recieve ping). Checked mail account is up OK. Is there some setting i need to do on my PC (Windows xp) to be able to ping?

sanderton · 10-30-2003, 04:23 AM

Quote:

Originally posted by horwitz

couldn't open socket: network is unreachable

That would be the clue! It seems that your network configuration does not allow the TiVo to open a connection to the SMTP server. You'd have to look at how your network is set up with firewalls etc.

sanderton · 10-30-2003, 04:25 AM

Quote:

Originally posted by tivo_boj
run the cron, got this this morning

cron (10/30-08:00:00-210) MAIL (mailed 85 bytes of output but got status 0x0001
)
*system* (10/30-08:08:00-201) RELOAD (/var/hack/etc/crontab)

in the cron log file

Any Clues

UPDATE

pinged my mailserver but timeout (ie did not recieve ping). Checked mail account is up OK. Is there some setting i need to do on my PC (Windows xp) to be able to ping?

Does the script work if you run it manually?

edit: I noticed this morning that the dailymail cron job failed on one of my TiVos, after I reconfigured it to get its listings via the phone instead of the network. Maybe TCL defaults to eth0; I'll have to get my manuals out!

bobnick · 10-30-2003, 05:49 AM

Quote:

Originally posted by sanderton
cron itself can go anywhere you like.

Oh, that's good to hear - sorry to have alarmed you!

The distro linked to in this thread is also a zip file that needs to be unzipped on a PC before it gets sent over to Tivo - no biggie, but I just wanted to warn the absoloute beginners not to get too confused!

Thanks again for a brilliant utility

bobnick · 10-30-2003, 06:25 AM

Hmm, having a few problems getting the script to run through cron.
Cron's log just says :
cron (10/30-11:00:00-258) CMD (/var/hack/dmn.tcl &^M)
with no error messages (I have a couple of versions of DM - this one is called dmn.tcl)

going to a bash promt and typing /var/hack/dmn.tcl & gives a response of [1] 264 and drops me back to the prompt. No mail was sent.

Tried again, without the ampersand, and got a long error message:

PHP Code:


			
no such object: {CONFLICT err=errTmActiveLockConflict}

    while executing

"dbobj $station fsid"

    ("foreach" body line 10)

    invoked from within

"foreach channel $channels2 {

            regexp {([0-9]*)/(.*)} $channel junk id subid

            set channelobj [db $db openidconstruction $id $subi..."

    ("uplevel" body line 2)

    invoked from within

"uplevel $body"

    invoked from within

"transaction {uplevel $body}"

    (procedure "RetryTransaction" line 5)

    invoked from within

"RetryTransaction {

         foreach channel $channels2 {

            regexp {([0-9]*)/(.*)} $channel junk id subid

            set channelobj [db $db ..."

    (procedure "init_channelindex" line 37)

    invoked from within

"init_channelindex"

    (file "/var/hack/tivoweb-tcl/modules/index.itcl" line 658)

    invoked from within

"source /var/hack/tivoweb-tcl/modules/index.itcl"

    (file "/var/hack/dmn.tcl" line 444)

no such object: {CONFLICT err=errTmActiveLockConflict}

    while executing

"dbobj $station fsid"

    ("uplevel" body line 12)

    invoked from within

"uplevel $body"

    invoked from within

"ForeachMfsFileTrans fsid name type $nowshowingdir "" 10 {

                set rec [db $db openid $fsid]

                set expdate [dbobj $rec get ExpirationDate]

                set expsecs..."

    (procedure "getnpexpire" line 7)

    invoked from within

"getnpexpire $hoursahead"

    (file "/var/hack/dmn.tcl" line 493)

Perhaps the first script was still running? There wasn't anything in PS when I checked.
I tried again, and got a few "retrying after errTmActiveLockConflict ..." messages, and then the mail was sent, along with a [1]+ Exit 1 on the screen.

I tried again with an ampersand, and the script worked fine, and got a [1] Done.

Tried it again, and just got [1] 272 on the screen. After a while, the mail was sent.

Is there any reason you can see as to why the first cron-called script didn't work? Everything else appears fairly normal. What does the ampersand do?

sanderton · 10-30-2003, 07:00 AM

The ampersand backgrounds the script, which is why you don't get error messages when you try that.

The error is errTmActiveLockConflict, which just means that another process was accessing that part of the database at the time the script was running. Normally you'd code it to retry the access in a few moments in that instance, but that particular bit of the script calls a procedure from TiVoWeb, here for some reason the usual retry code doesn't seem to be working. In that instance the only think to do is to try again and hope the databse is clear at that point.

You could probably make it less likely to happen by changing the cron job time to an odd number of minutes as if you fire it off at 8.00 its quite likely that a recording is just starting up, whoich triggers loads of database accesses.

Oh, and the ^M in the cron message suggests that it's been edited on a PC and not DOS2UNIXEd - I think you can juts delete all the ^Ms at the end of each line in crontab.