Endpad and Cron

1843 Views 25 Replies 5 Participants Last post by RichardJH, Nov 21, 2011

kev160967

Discussion Starter · Oct 29, 2011

Hi,
I have a recurring problem where endpad stops working every few days. If I go to hackman it thinks it is still running, but when I hit the stop button it tells me it has encountered a stale PID. I can then start it and it will run again for a few days. I came up with the idea of stopping and starting it each day using cron, but it doesn't appear to work. Here are the lines from my crontab:

# shut down endpad
0 2 * * * /var/hack/endpad.tcl -stop >> /dev/null &

# start endpad up again
0 3 * * * /var/hack/endpad.tcl -s 1 -e 5 -auto >> /var/hack/cron/eps.log &

So at 2am it stops it, which works okay if it is running, then at 3am it should start it again.

The problem is that by 5am (when daily mail runs, from same crontab file) it has stopped again. Any ideas what's going on? The following is the recording of the startup I directed to eps.log:

EndPad v1.5.4 (c) 2005 Stuart Anderton
Strictly no commercial distribution
See file header for terms of use and distribution
Switching to background...

Not sure what other logs might have anything useful in them? Endpad.log and endpad.msg have both been empty since I changed this - only thing I can see is the initial cron kill being accepted

Kev

1 - 20 of 26 Posts

TCM2007

· #2 · Oct 30, 2011

I'd try to find out why EndPad is crashing, rather than this workaround.

Can you post the log?

kev160967

Discussion Starter · #3 · Oct 30, 2011

Okay, I've removed the crontab entries and started it again. Next time it fails I'll post the error from hackman and the logs. Are there any particular logs you think would be helpful? Under logs on tivoweb I have an endpad.log and an endpad.msg. I don't remember seeing anything in either of these in the past though.

Kev

TCM2007

· #4 · Oct 30, 2011

EndPad.log should have useful info.

kev160967

Discussion Starter · #5 · Nov 11, 2011

Okay, lasted for longer than usual this time. There's not much in the log though. Here it is:

Thursday 00:55:01 : No recording in progress
Thursday 00:55:01 : Next end padding setting due 04:00
Thursday 00:55:01 : Not time to add start padding yet
Thursday 00:55:01 : Next start padding setting due 03:38
Thursday 00:55:01 : Next wake up will be 01:25
Sleeping, checking every minute for kill signal...............................
Thursday 01:25:01 : Woken up
Thursday 01:25:01 : No recording in progress
Thursday 01:25:01 : Next recording is {The Cramp Twins} on CNTOO (Suggestion) at 03:40
Thursday 01:25:01 : No end padding required:
Thursday 01:25:01 : No recording in progress
Thursday 01:25:01 : Next end padding setting due 04:00
Thursday 01:25:01 : Not time to add start padding yet
Thursday 01:25:01 : Next start padding setting due 03:38
Thursday 01:25:01 : Next wake up will be 01:55
Sleeping, checking every minute for kill signal...............................
Thursday 01:55:01 : Woken up
Thursday 01:55:01 : No recording in progress
Thursday 01:55:01 : Next recording is {The Cramp Twins} on CNTOO (Suggestion) at 03:40
Thursday 01:55:02 : No end padding required:
Thursday 01:55:02 : No recording in progress
Thursday 01:55:02 : Next end padding setting due 04:00
Thursday 01:55:02 : Not time to add start padding yet
Thursday 01:55:02 : Next start padding setting due 03:38
Thursday 01:55:02 : Next wake up will be 02:25
Sleeping, checking every minute for kill signal.................

Also, here's the error message I got when I tried to stop endpad from hackman. At this point Dailymail had reported endpad wasn't running, but hackman thought that it was. I pressed the stop button and after the usual text I got this:

A stale PID file was found and has been deleted. This means that this script terminated unexpectedly last time. Loading now. The errors logged (if any) before the last termination were: No instance of EndPad running

Any ideas?

See less See more

AMc

· #6 · Nov 11, 2011

If endpad isn't auto starting from a boot then it might be a symptom of something else?
What's your uptime? Are there any other clues in the main logs at the same(ish) time.

kev160967

Discussion Starter · #7 · Nov 11, 2011

Endpad starts okay from a boot, it's just every so often it will stop for no apparent reason. That's why I tried the cron approach, since it would invariably run for a day or so each time before it stopped working - by stopping and starting every night I was hoping to avoid the issue

spitfires

· #8 · Nov 11, 2011

Hmm, nothing wrong there. Other than "tcp_keepalive" messages is there anything in your kernel log between 01:55 and 02:25 which might suggest a problem? (Or in the 'messages' log?)

Edit: oops I see your endpad log was for Thursday so that would probably be "Okernel" and "Omessages"

kev160967

Discussion Starter · #9 · Nov 11, 2011

spitfires said:
Hmm, nothing wrong there. Other than "tcp_keepalive" messages is there anything in your kernel log between 01:55 and 02:25 which might suggest a problem? (Or in the 'messages' log?)

Edit: oops I see your endpad log was for Thursday so that would probably be "Okernel" and "Omessages"

Already rolled of the end of the log, unfortunately - Okernel goes back to 6am on Thursday. Next time it happens I'll grab a segment of the kernel log for the appropriate time. I'll also scan any other logs for events in that timeframe.

kev160967

Discussion Starter · #10 · Nov 11, 2011

Interesting (well maybe). Tivoweb.log rolled over at 02:13 on Thursday - looking at the log entry, tivoweb restarted at that point.

Okay, my bad - it seems that the tivo rebooted on Thursday morning. Now as far as i can recall, in the past endpad failing hasn't aligned with the tivo rebooting. That being said, i didn't think it had this time, so who knows. To back it up though, I've noticed the tivo is more prone to rebooting since moving to altepg (and endpad failures became more common), but prior to that I can state as a fact that the tivo went for months without rebooting but endpad was failing (very) occasionally. Not as much, by far, but occasionally.

I guess there could be two issues here then - an intermittent/rare problem not tied to a reboot, and a more frequent problem tied to a reboot. Has anyone any thoughts on why endpad wouldn't start after a reboot, but hackman would think it was started?

I'll continue to monitor for endpad failures and reboots and see how they tie together. Since going to altepg it's rare to see endpad stay up for more than a couple of days at a time, so it shouldn't take long

Thanks folks,
Kev

See less See more

TCM2007

· #11 · Nov 11, 2011

Your TiVo rebooted, but EndPad should recover cleanly from that if it's in rc.sysinit.author - which it's not because you've done your cron stuff. If it was set up conventionally it would have restarted.

In this particular case, it seems your attempt to fix the problem has caused it!

I think you need to put everything back to a standard setup and keep checking for errors!

When was your daily call; 2.23 perchance?

You could use cron as belt and braces; don't stop EndPad but just try to start it without the -auto argument. If it's not running it will start, if it is running it will do nothing.

kev160967

Discussion Starter · #12 · Nov 11, 2011

TCM2007 said:
Your TiVo rebooted, but EndPad should recover cleanly from that if it's in rc.sysinit.author - which it's not because you've done your cron stuff. If it was set up conventionally it would have restarted.

In this particular case, it seems your attempt to fix the problem has caused it!

I think you need to put everything back to a standard setup and keep checking for errors!

When was your daily call; 2.23 perchance?

You could use cron as belt and braces; don't stop EndPad but just try to start it without the -auto argument. If it's not running it will start, if it is running it will do nothing.

I didn't actually take endpad out of rc.sysinit.author when I added it to cron. There wasn't a conflict so I didn't see the need - sysinit was there for reboots, cron for daily maintenance.

This is the relevant part of my rc.sysinit.author:

/var/local/TivoWebPlus/tivoweb &
/var/hack/endpad.tcl -s 1 -e 5 -auto >> /dev/null &
/var/hack/suggs.tcl -threshold 1 <</dev/null

The tivoweb bit worked, but not endpad. Also I notice that suggs doesn't have an ampersand so is being started synchronously, which doesn't sound right - it's after the endpad though, so probably not relevant? Interesting to know if it too didn't start. Are there any bits of that that you would change? Is suggs still needed?

kev160967

Discussion Starter · #13 · Nov 11, 2011

TCM2007 said:
You could use cron as belt and braces; don't stop EndPad but just try to start it without the -auto argument. If it's not running it will start, if it is running it will do nothing.

Just to be explicit, endpad is no longer in crontab, and still is in sysinit

TCM2007

· #14 · Nov 11, 2011

Is that copy and pasted, because youve got << and >> mixed up, and no space

kev160967

Discussion Starter · #15 · Nov 21, 2011

Tivo's been annoyingly

)) stable since I brought this up, but i did a manual reboot over the weekend, and endpad did not start up (although hackman seemed to think that it did??! and again reported the stale PID when I stopped it).

TCM2007 said:
Is that copy and pasted, because youve got << and >> mixed up, and no space

It was copy and pasted - though it was strange, but I assumed I'd copied that from a set of installation instructions. Going into /etc/rc.d, and manually executing rc.sysinit.author does start endpad though, so I'm still not sure what's going on.

On the face of it, endpad only seems to start from an interactive bash shell. From both a startup script and from cron it fails. I'll comment out the suggs line from sysinit.author, and see if that makes a difference

Kev

kev160967

Discussion Starter · #16 · Nov 21, 2011

This is the end of my rc.sysinit file - does anything look out of order:

Code:

if [ ! "$vmstat" = "" ]; then
  echo "Starting memory statistic gathering"
  vmstat 10 &
fi

#echo "[ ! -f /etc/rc.d/rc.sysinit.author ] || /etc/rc.d/rc.sysinit.author"
/etc/rc.d/rc.sysinit.tpm

echo "rc.sysinit is complete"
source /etc/rc.d/rc.net

# Call the Tivo Package Manager Startup Scripts
#/etc/rc.d/rc.sysinit.tpm

/bin/bash </dev/ttyS3 >& /dev/ttyS3 &
/sbin/tnlited 23 /bin/bash -login &
/sbin/tivoftpd
/var/hack/bin/cron >> /dev/null &

# Set the path
PATH=/bin:/sbin:/tvbin:/devbin:/var/hack:/var/hack/bin
export PATHbash-2.02#

I'm not sure what the commented out line near the top do with rc.sysinit.author is doing. The first bit, I think, is saying if the file doesn't exist, but I'm not sure what the second half does.

Should there be something in rc.sysinit that calls rc.sysinit.author, or does the tivo boot sequence do it automatically?

Kev

See less See more

kev160967

Discussion Starter · #17 · Nov 21, 2011

Just rebooted again. Here is my current rc.sysinit.author - I added the echo line to see if it would appear in the log on a reboot. It doesn't appear to.

Code:

echo "running rc.sysinit.author"
/var/local/TivoWebPlus/tivoweb &
/var/hack/endpad.tcl -s 1 -e 5 -auto >> /dev/null &
#/var/hack/bin/cron >> /dev/null &
#/var/hack/suggs.tcl -threshold 1 <</dev/null

Here's the log from a reboot I did after adding the echo line

Code:

Nov 21 09:58:29 (none) kernel: Scanning for phase4 repair scripts 
Nov 21 09:58:30 (none) kernel: MCP startup complete 
Nov 21 09:58:30 (none) kernel: rc.sysinit is complete 
Nov 21 09:58:32 (none) kernel: Warning: kernel-module version mismatch 
Nov 21 09:58:32 (none) kernel: ^I/lib/modules/ax88796.o was compiled for kernel version 2.1.24-TiVo.1 
Nov 21 09:58:32 (none) kernel: ^Iwhile this kernel is version 2.1.24-TiVo-2.5 
Nov 21 09:58:32 (none) kernel: ax88796 version 20030223-0 
Nov 21 09:58:32 (none) kernel: ax88796: updated tlb entry 2 to 1M 
Nov 21 09:58:32 (none) kernel: ax88796: using MAC address 00:0B:AD:69:C6:67 
Nov 21 09:58:32 (none) kernel: ax88796: driver installed 
Nov 21 09:58:32 (none) kernel: 
Nov 21 09:58:33 (none) kernel: IP struct was not filled in! 
Nov 21 09:58:33 (none) kernel: sa is: 0x20000 0xc0a8c806 0x7ffffce4 0x7ffffd00 
Nov 21 09:58:33 (none) kernel: sa.sin_addr = 0x7ffffc48 a sockaddr is 16 bytes 
Nov 21 09:58:33 (none) kernel: route.tivo forgot to specify route netmask. 
Nov 21 09:58:41 (none) kernel: EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended 
Nov 21 09:58:44 (none) last message repeated 2 times
Nov 21 09:59:01 (none) kernel: Microcode version 00034 
Nov 21 09:59:04 (none) kernel: Initialize with 1 live caches 
Nov 21 09:59:04 (none) kernel: mediaswitch: resetting adjustments 
Nov 21 09:59:04 (none) kernel: tivoblaster.C::ReadTivoFormatKey(), Line : 356, Found 350 ms. Prefix. 
Nov 21 09:59:05 (none) kernel: osd buffer size = 1080000 
Nov 21 09:59:30 (none) kernel: mediaswitch: returning -1 from standin tune after tuning to ch -3 with adjust 0 
Nov 21 09:59:30 (none) kernel: tuning:lock 1 
Nov 21 09:59:31 (none) kernel: Done with this packet

Starting to think the author file isn't getting executed?? But then what would be starting tivoweb?

Another funny, which may be a clue - after the reboot my tivo's filesystem was read/write. I'd specifically set it to read only before rebooting (since I had a suspicion something was doing this at a reboot).

I'm going to try changing the order of endpad and tivoweb in the rc.sysinit.author file, and rebooting

Kev
Kev

See less See more

kev160967

Discussion Starter · #18 · Nov 21, 2011

Stranger and stranger - I commented out the tivoweb startup from rc.sysinit.author, but when I rebooted tivoweb was up and running. Also, the path reported by "ps -ax" isn't the one from the author file. Note also that the pid (112) is lower than things like the ftp daemon and cron, which are started at the end of rc.sysinit:

Code:

  109  ?  S    0:00 Mcp event
  110  ?  S    0:00 tcphonehome
  112  ?  R    0:14 tivosh /var/local/tivoweb-tcl/httpd-tt.tcl
  115  ?  S    0:00 tcphonehome
  116  ?  S    0:00 PipeListener
  117  ?  S    0:00 FsMpStream
  118  ?  S    0:00 PhoneHome event hd
  120  ?  S    0:00 /bin/bash
  123  ?  S    0:01 /sbin/tnlited 23 /bin/bash -login
  126  ?  S    0:00 /sbin/tivoftpd
  128  ?  S    0:00 /var/hack/bin/cron
  142  ?  S    0:00 dbgc-mcp
  143  ?  S    0:00 dbgc-mcp
  144  ?  S    0:00 PipeListener

Any idea what might be starting tivoweb? I can't see anything else in rc.d.

Kev

See less See more

TCM2007

· #19 · Nov 21, 2011

[ ! -f /etc/rc.d/rc.sysinit.author ] || /etc/rc.d/rc.sysinit.author

is the line that runs rc.sysinit.author.

No idea a) why it's commented out and b) why it's been edited to be in an echo statement.

Both of those things will stop rc.sysinit.author running.

So EndPad is only being started by Hackman.

I'm afraid I've no experience of using that,.

See less See more

kev160967

Discussion Starter · #20 · Nov 21, 2011

Should this be in /etc/init.d/TivoWebPlus:

Code:

bash-2.02# cat TivoWebPlus
#!/bin/sh
RUN_ONCE=0
if [ "$1" == "start" ]; then
    if [ $RUN_ONCE -eq 1 ]; then
        mount -o remount,rw /
        rm -f /etc/rcS.d/S90TivoWebPlus
        mount -o remount,ro /
    fi
/var/local/tivoweb-tcl/tivoweb
fi

The file /etc/rcS.d/S90TivoWebPlus has the same contents as above??

See less See more

1 - 20 of 26 Posts

This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.