pre-alpha: Season Pass / Wishlist / Logo state backup & restore tool

[angra]

The volume of people mentioning a need of such a thing has motivated me to get off my duff enough to package the tool I had made. This tools is intended for making backups, from running tivos, of important state information. The tool backs up and restores:

season passes
wishlists
logo assignments

This software is AS-IS with no warranty. It will probably make your tivo turn into a doorstop. Please don't run this (especially the restore!) if you are not prepared for your tivo drive to be rendered useless. This means backups, being prepared to make new backups and restore from backup, etc.

That said, I'm interested in hearing feedback from anyone who runs this. I don't have time to maintain this, so I can't promise timely fixes. I will not be offended at all if other people decide to take this code and "fork" it, so to speak, if I don't respond to requests quickly enough to suit.

to use, unpack and read the README.

you can PM me here or mail me at danspam@envy.gtf.org.

good luck and enjoy.

(edited 1/29/03 to clarify description)

Note below that AGW has posted a modified version with important corrections.

(eidted 2/24/03 to point to AGW bugfixed release)

[zaknafein]

Hey, nice. I assume this is geared for online backup?

[angra]

Yes, sorry I was not explicit about this.

This script is intended for an operating Tivo.

[mrtickle]

Thanks!

The readme says that if you do a backup and restore without deleting wishlists/sps, you end up with two of everything. But what about logos, should you delete them too or will the restore overwrite existing ones? I see it's easy enough to comment out the "restore_logos" line so I'll probably do that.

Oh one other thing, which software version did you use it with? Hopefully 2.5 and above, we have 2.5.5 here.

TIA

[angra]

Well I'm currently on 3.0.

I _think_ I did some of this with 2.5, but I don't remember for sure. I am reasonably sure that nothing major changed in the SP/wishlist structure between those versions though.

The logo stuff won't duplicate - it sets values in records that already exist, whereas the SP/WL stuff creates new records.

[SteakMan]

Has anyone tried it on a DTiVo?

I'll make a backup tonight, but I don't know when I'll have time to try a restore.

So, since this essentially re-creates entries instead of restoring, it might be adapted to adjust your SP lineup offline :)

-SteakMan-

[angra]

yes, that could definitely work.

if you feel adventurous, the backup file is pretty easy to read and you could probably tweak it.

It doesnt completely automate the process though - you'd still have to delete your SPs on the tivo first. You could modify the restore script to delete existing SPs first though if you were so inclined.

[mrtickle]

Originally posted by angra
Well I'm currently on 3.0.

I _think_ I did some of this with 2.5, but I don't remember for sure. I am reasonably sure that nothing major changed in the SP/wishlist structure between those versions though.

The logo stuff won't duplicate - it sets values in records that already exist, whereas the SP/WL stuff creates new records.

Righto - thanks!

[agw]

I've just tried this script on my UK TiVo running 2.5.5 and unfortunately, while it picked up the wishlists and logos no problem, it didn't pick up the season passes.

This is the entire restore_seasonpasses proc that it writes:

proc restore_seasonpasses {} {
global new_tms
global new_tms_fs
global newids
global oldids
#
# SEASON PASSES
#
set new_tms ""
set new_tms_fs ""
}

I've had a little play with the script and changing the line that does the ForeachMfsFile for the season passes to use the directory "/SeasonPass" instead of "/SeasonPass/User" appears to do the trick and fill in the season passes.

This directory name corresponds with the directory name for the season passes in TiVoWeb.

I've just bought a couple of 120Gb drives so I'll let you know if the script loads the season passes back on after the upgrade :-) It'll be SO cool if it does. The thought of putting those back in by hand gives me the shivers... especially having to redo the priorities. It's a shame TiVoWeb doesn't give you an interface for setting the priorities because it's a nightmare doing it on the TiVo... too sloooooooooooooooow...

[agw]

Just to clarify before everyone points out that MFSTools will carry season passes over... this TiVo has already been upgraded once and MFSTools refuses to produce anything smaller than a 160 hour backup for it (I've used the -s option, no difference - ditto the -v(? can't remember) option that looks at stream sizes and not file sizes).

So I've taken a backup of a friends 40 hour 2.5.5 (I have my original discs but they're ancient) and I'll be restoring THAT onto the 120 Gb discs and expanding it.

Hence the reason why a season pass backup and restore script will be so useful.

[angra]

sounds great! I'm glad you were able to find and correct the problem.

I'll try to put together a modified version that reacts properly to UK tivos.

[jodell]

angra,

Any chance exporting "thumbs" could be added? I just added tivo #2 to my household and I want to move the new tivo to the living room. I will never get it done if i have to manually restore all the SP, wishlists and thumbs.

Thanks,
Jeff

[angra]

hi, I tried to do thumbs, and I could reliably restore the thumbs themselves but it never seemed to impact the suggestions engine.

The code is in the utilities as posted, just commented out, but as I said it doesn't quite work. Shows, actors, etc, will all be thumbed appropriately, but the prediction engine won't use this info properly.

[mrtickle]

Originally posted by agw

I've had a little play with the script and changing the line that does the ForeachMfsFile for the season passes to use the directory "/SeasonPass" instead of "/SeasonPass/User" appears to do the trick and fill in the season passes.

This directory name corresponds with the directory name for the season passes in TiVoWeb.


Interesting, thanks for that. I think the /User structure was from software 3.0 onwards, which we don't have of course.


I've just bought a couple of 120Gb drives so I'll let you know if the script loads the season passes back on after the upgrade :-) It'll be SO cool if it does. The thought of putting those back in by hand gives me the shivers... especially having to redo the priorities. It's a shame TiVoWeb doesn't give you an interface for setting the priorities because it's a nightmare doing it on the TiVo... too sloooooooooooooooow...

Putting mine back by hand I wouldn't mind at all. What would annoy me is losing all my Hibernating Season Passes, I have loads of these which are not currently airing and which CANNOT be recreated, unless you use angra's wonderful script! I definitely do NOT want to use wishlists for these! The last time I thought it would be a "good idea" to use wishlists (big mistake) it took me a full year to get the SPs back.

[LJ]

Originally posted by angra
hi, I tried to do thumbs, and I could reliably restore the thumbs themselves but it never seemed to impact the suggestions engine.How long did you wait after importing to check for new suggestions? Looking in some of the test scripts in \tvlib\tcl\tv it looks like TiVo will take at least a few minutes to respond to a change in thumbs.

Also I wonder if there's an event that needs to be sent to make TiVo see a change in thumbs if the change is made from a script rather than using the UI? For example ui.itcl sends a 'data changed' event when a season pass is updated - maybe a change in thumbs has it's own 'data changed' event?

[angra]

I waited several hours - I think the last test I ran I waited about 12. It was long enough to populate the suggestions list, but it was filling it up with stuff on the restored system that bared no resemblance to the original one.

you may be (probably are) right about the special event that needs to be sent.

i went from having time to work on this stuff to not, about 4-6 months ago. I had some good tivoweb mods 3/4 finished too :/. Anyway I just dont have time to continue working on it seriously, especially since testing the thumbs engine stuff is very disruptive to my tivoing.

[StanSimmons]

Originally posted by agw
I've just tried this script on my UK TiVo running 2.5.5 and unfortunately, while it picked up the wishlists and logos no problem, it didn't pick up the season passes.

This is the entire restore_seasonpasses proc that it writes:

proc restore_seasonpasses {} {
global new_tms
global new_tms_fs
global newids
global oldids
#
# SEASON PASSES
#
set new_tms ""
set new_tms_fs ""
}

The same is true for US DTiVo's still running v2.5.

[agw]

Well, I ran the restore script and unfortunately it failed. The first part of restore restores the logos, which calls the file generated by backup, which in turn calls set_station_logo, which takes the tmsid it gets passed and tries to translate this to an fsid with station_tms_to_fsid.

This function returns -1 if the tmsid it gets passed isn't in the MFS directory "/StationTms". The backup and restore were both done with the same cable package, so I'd have expected them to both have the same sets of stations - but regardless, sometimes this function was getting to the error return of -1. I didn't look too hard to see why as it wasn't really the problem.

The problem is that set_station_logo doesn't test the value it gets back from station_tms_to_fsid, and so it falls over when it gets the "unknown" return. Ideally it should just puts stderr and skip that station.

I changed my version of backup-support to do this and then the next problem I hit was that backup had generated a call to set_station_logo with only one parameter. In fact, it generated a few lines like this:

set_station_logo 31611

as opposed to the more usual:

set_station_logo 21599 65539

This stops the script because set_station_logo is expecting two parameters.

I had a look and for me, TMS 31611 is "Living (Reduced)". It has no LogoIndex value associated with it, which is presumably why it's missing from the backup script output.

The backup script needs to test the logo index to see if it's empty and if it is, don't write the line out.

I've not had a look at the season passes and wishlist restores yet but I'll look soon. It looks like most of the errors are just missing checks for error conditions. If you went through and put those in everywhere then you'll probably cover most of the problems that would break the scripts.

I've attached (I hope!) the changes I made to backup-settings and backup - these just set the season pass directory based on the version number (nicked from TivoWeb), skip stations with no LogoIndex when generating the output from backup and skip missing stations when restoring the backup. I'm a TCL newbie, I usually write C++, so please feel free to fix anything rubbish. I've marked the changes with # agw comments. There really aren't a lot of them.

[angra]

I'm glad people are finding and correcting problems, and posting fixes. I've made a reference to your modified release in the top post so people will hopefully grab that.

[mrtickle]

Yep! Many thanks for this bit too:

if {$version3} {
set seasonpassdir "/SeasonPass/User"
} else {
set seasonpassdir "/SeasonPass"
}

:D

[havanahjoe]

I did a backup of season passes and wishlists on a DirecTivo running version 3.1. I am now trying to restore this info on the same Tivo which is now running version 2.5.1. I got some errors at first, about the logo restoration, but since I don't need this I just erased all the logo entries on my backup file.
I ran the restore again and it starts working but it then gives me the following error:
object not found (errNmNameNotFound)

while executing
"mfs find $guideindexdir/Tms.key"
(procedure "tms_to_fsid" line 4)
invoked from within
"tms_to_fsid $tmsid"
("uplevel" body line 2)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set fsid [tms_to_fsid $tmsid]
} "
(procedure "rebuild_series" line 10)
invoked from within
"rebuild_series {SH080955 Version:63 ServerId:ATSH080955 ServerVersion:0 {Genre:
375 384 111 116 105 124} ThumbData:268633087 {Title:{The X-Files}} TmsI..."
(procedure "restore_seasonpasses" line 12)
invoked from within
"restore_seasonpasses"
(file ".//restore" line 27)

I think the problem is that it can't find the entry for this show in the guide... Is that it? It's very possible since I just reinstalled everything a few hours ago so the guide won't be completely populated for a few days.

Does the script need to find the shows in the guide?

[havanahjoe]

I just took a closer look at this and the problem is getting the station ID. I guess that 3.1 handles these in a different way, and now with 2.5 it can't find them. If this is the problem, can anything be done to fix it?

Another thing.. Does the script have to finish without errors for the changes to take effect? I have some wishlists that are spawned out by the script before season passes but they don't appear on the season pass manager.

[agw]

Originally posted by havanahjoe
I did a backup of season passes and wishlists on a DirecTivo running version 3.1. I am now trying to restore this info on the same Tivo which is now running version 2.5.1.

<snip>

I ran the restore again and it starts working but it then gives me the following error:
object not found (errNmNameNotFound)

while executing
"mfs find $guideindexdir/Tms.key"


I've had a look at the script and it hard-codes guideindexdir to GuideIndexV2. Looking at TiVoWeb's index.itcl it appears that this is another thing that changed between V2 and V3 - on V2, which you're trying to restore to, it's /GuideIndex.

I've attached a modified backup-settings.itcl that sets guideindexdir to the right value. It's untested! But it should certainly fix the problem of going for the wrong directory.

As for whether the script has to finish without errors before changes take effect - I would expect that all changes up to the point where the error breaks the script will take effect. With the wishlists it just creates themes for them, so they should be there... but because you wouldn't have been able to add any season passes the associated season pass wouldn't have been created. I would have *expected* (but I'm probably wrong!) to see the wishlists in the "Search by wishlist" menu on the TiVo.

[havanahjoe]

Ok, that worked, I don't get that error anymore. But now I get this:

rebuilding series: SH080955 Version:63 ServerId:ATSH080955 ServerVersion:0 {Genr
e:375 384 111 116 105 124} ThumbData:268633087 {Title:{The X-Files}} TmsId:SH080
955
found prebuilt 7681
or:SeasonPass Version:7 MaxRecordings:5 {Series:@$fsid} {Station:@$station_id} P
riority:0 ShowStatus:0
doing it
minival=$fsid
id=7681
doing it
minival=$station_id
id=4004
invalid attribute: ShowStatus
while executing
"dbobj $new_obj set $type $subval"
("foreach" body line 3)
invoked from within
"foreach subval $val {
if {$index==0} {
dbobj $new_obj set $type $subval
} else {
dbobj $new_obj add $type $subval ..."
("uplevel" body line 21)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set new_obj [db $db create $type]
for {set i 1} {$i<[llength $object]} {incr i} {
set element [lindex $object $i]
..."
(procedure "object_rebuild" line 5)
invoked from within
"object_rebuild {SeasonPass Version:7 MaxRecordings:5 {Series:@$fsid} {Station:@
$station_id} Priority:0 ShowStatus:0}"
(procedure "restore_seasonpasses" line 15)
invoked from within
"restore_seasonpasses"
(file "./restore" line 27)

[agw]

Referring back to the ever-handy TiVoWeb scripts it appears that the ShowStatus field in the season pass record is version 3's new name for the FirstRun field.

I've grepped for "version3" in the TiVoWeb stuff and I think you'll probably be OK with the entry you're trying to insert, as long as you or the script just changes the text "ShowStatus" to "FirstRun". In this case it appears that only the name changed - the value assigned to the name is valid under both versions.

However I'm a little unsure as to how the TiVo will react to entries that have the same name between V2 and V3, but different valid sets of values. Going from V2 to V3 should be OK as you'd imagine (or rather, hope) that the valid V3 values would be supersets of the V2 values. However there's no reason to expect V2 software to accept all values that are valid under V3. The script could certainly be changed to normalise V3 values back to valid V2 ranges of values - but I've no idea what any of those ranges or values would be, or even if it's a problem at all. Some brave soul will have to try it and see what happens, if they haven't already.

Attached is another version of backup-support.itcl that has in place a mechanism for renaming these sub-types. I've only put in support for renaming between "FirstRun" and "ShowStatus" for the object type "SeasonPass" but it can easily be expanded to cover any other examples that crop up.

I've not tested this with a live script but I have tested the gobbet that does the name swapping, and for your example it did the job and didn't knacker any of the other names. I've put in a trace to screen to tell you that the name is being changed.

I would recommend that you clear out all season passes before running the restore script. I suspect that it will try to restore the "Priority" field in the season pass. This appears to be a number, zero-based, that determines the priority of the season pass. The TiVo might not be happy if more than one season pass ends up with the same priority. It might be an idea for the restore script to find the highest priority in the existing season passes on the machine and use that as a base for the new passes it inserts, and so ensure they remain unique - but it doesn't do this at the moment. Clearing out all existing season passes would probably be enough to avoid this - and like before, it might not even be a problem, the TiVo might be quite happy with it.

Good luck! :)

[havanahjoe]

We're making progress!! But there are still problems:

rebuilding series: SH080955 Version:63 ServerId:ATSH080955 ServerVersion:0 {Genr
e:375 384 111 116 105 124} ThumbData:268633087 {Title:{The X-Files}} TmsId:SH080
955
found prebuilt 7681
or:SeasonPass Version:7 MaxRecordings:5 {Series:@$fsid} {Station:@$station_id} P
riority:0 ShowStatus:0
doing it
minival=$fsid
id=7681
doing it
minival=$station_id
id=4004
Renaming sub-object ID ShowStatus for SeasonPass to FirstRun
The value assigned to this ID - 0 - remains unchanged...
rebuilding series: SH080955 Version:63 ServerId:ATSH080955 ServerVersion:0 {Genr
e:375 384 111 116 105 124} ThumbData:268633087 {Title:{The X-Files}} TmsId:SH080
955
found prebuilt 7681
or:SeasonPass Version:6 MaxRecordings:5 {Series:@$fsid} {Station:@$station_id} P
riority:1 ShowStatus:0
doing it
minival=$fsid
id=7681
doing it
minival=$station_id
id=4006
Renaming sub-object ID ShowStatus for SeasonPass to FirstRun
The value assigned to this ID - 0 - remains unchanged...
rebuilding series: SH229827 Version:59 ServerId:ATSH229827 ServerVersion:0 {Genr
e:369 371 105 124} ThumbData:268633087 {Title:{South Park}} TmsId:SH229827
found prebuilt 13274
or:SeasonPass Version:9 MaxRecordings:5 Priority:2 {Series:@$fsid} {Station:@$st
ation_id} ShowStatus:1
doing it
minival=$fsid
id=13274
doing it
minival=$station_id
id=4057
Renaming sub-object ID ShowStatus for SeasonPass to FirstRun
The value assigned to this ID - 1 - remains unchanged...
can't read "newids": no such variable
while executing
"lindex $newids [lsearch $oldids 38442]"
(procedure "restore_seasonpasses" line 27)
invoked from within
"restore_seasonpasses"
(file "./restore" line 27)

I deleted all the season passes I had already created, about 10, and ran the script but got this error after a while. It created 3 season passes and stopped. Sorry about all the problems.. :(

[agw]

No worries about the problems - I'm impressed with your peserverance :)

The restore script runs the "restore_wishlists" function first and that runs through all of your wishlists, creating new identifiers for each of them and building up two global arrays, once called oldids that contains the original ID numbers for these wishlists and the other called newids that contains the ID number as it is now.

The line that is going over is where the restore_seasonpasses function, again in the script generated by backup, is searching oldids to find the element number that contains the old wishlist ID number and then getting the value for this from the newids array.

These arrays are global and the restore_wishlists function is obviously meant to run before restore_seasonpasses so that it can fill these arrays in. Have you changed the output from the backup script to empty out the restore_wishlists function, or modified restore to not call restore_wishlists before calling restore_seasonpasses?

I think the function to rebuild the wishlists will always try to create them, so it might be an idea to clean out the wishlists from the "Search by wishlist" menu before running the script.

If you haven't changed the restore script or the output from the backup file then it might be an idea if you attached the output from the backup script to a message so that I can see what's going on with your version of it.

[angra]

agw's analysis is right on.

if it can't find the "newids" variable it sounds like you're trying to run just the season pass restores. This is an intuitive thing to do, but won't work. Season Passes that are actually auto-record wishlists contain what amounts to a symbolic link inthe database object to the wishlist databse object. In order to restore these, the script has to be able to find the w/l object. The first lines in the restore script create the translation table variables (newid, etc), so if they don't exist at all, I suspect that's the trouble.

I considered putting in an option to clear out existing data before doing the restores (which would alleviate the temptation to run just one portion), but I was pretty convinced I would have accidentally destroyed all my settings during testing several times.

Thanks again to AGW for taking the ball and running with this one, and good luck with getting everything up & running havanahjoe.

[havanahjoe]

That could be the problem, I actually removed everything from the wishlist function in my output from the backup, because it had already created them. I'll use the original file, but this reminds me that if I use the restore file without any modifications, I get two different errors. The first one is caused by this line:

TmkLogger: <134>Feb 26 21:24:05 tcl[261]: Tcl created pool of 1458176 bytes

that was stored when the backup script ran. So I just erased it.

But then I'm getting problems with the logo Id's:

no value given for parameter "logoindex" to "set_station_logo"
while executing
"set_station_logo 12444 "
(procedure "restore_logos" line 3)
invoked from within
"restore_logos"
(file "./restore" line 25)

I got this errror the very first time I tried restoring the backup, so I removed all the entries in the restore logos function and that's when I started finding the other problems you guys have already helped me with.

The logos are not really something that I'm worried about, but I'm posting this error in case someone does care about them.

I'll remove the logos and try the script again and post my results.

[havanahjoe]

Well I got farther down the restore but I got another errror.

The script created all my wishlists, 6 season passes, 1 auto-recording wishlist and stopped with this:

rebuilding series: SH524467 Version:43 ServerId:ATSH524467 ServerVersion:0 {Genr
e:375 105 124} ThumbData:268566912 {Title:{John Doe}} TmsId:SH524467
can't read "db": no such variable
while executing
"db $db create Series"
("uplevel" body line 2)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set new_obj [db $db create Series]
for {set i 1} {$i<[llength $object]} {incr i} {
set element [lindex $object $i]
..."
(procedure "rebuild_series" line 23)
invoked from within
"rebuild_series {SH524467 Version:43 ServerId:ATSH524467 ServerVersion:0 {Genre:
375 105 124} ThumbData:268566912 {Title:{John Doe}} TmsId:SH524467}"
(procedure "restore_seasonpasses" line 31)
invoked from within
"restore_seasonpasses"
(file "./restore" line 27)

I'll attach my original backup file in case it helps.

Thanks again guys!

[agw]

The thing with the missing logoindex value in the logos shouldn't be a problem any more, I covered that in the first modification. Was it created with angra's original backup script? You can just remove those lines from the output it created - or skip the whole thing, as you've done, if you're not worried about logos.

The problem you hit is hopefully nice and easy to fix - there were a couple of functions where the global "db" wasn't declared :) The season pass you're adding must be one that isn't already in the guide, one of Mr Tickle's "hibernating" season passes.

Attached is another fix to backup-support.itcl to put the declarations in... untested I'm afraid but it *should* be alright... good luck!

[havanahjoe]

I don't remember if I used the latest file from this thread to do the backup or maybe I used a version that I downloaded before.

Anyway, the script restored more season passes and wishlists now! 9 Season Passes and 1 Auto Recording Wishlist

But... Another error...

rebuilding series: SH163817 Version:74 ServerId:ATSH163817 ServerVersion:0 {Genr
e:371 376 105 124 384} ThumbData:268633087 {Title:{3rd Rock From the Sun}} TmsId
:SH163817
found prebuilt 12188
or:SeasonPass Version:10 MaxRecordings:5 Priority:10 {Series:@$fsid} {Station:@$
station_id}
doing it
minival=$fsid
id=12188
doing it
minival=$station_id
id=-1
can't open object (errDbNotFound)

while executing
"db $db openid $id"
("uplevel" body line 41)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set new_obj [db $db create $type]
for {set i 1} {$i<[llength $object]} {incr i} {
set element [lindex $object $i]
..."
(procedure "object_rebuild" line 9)
invoked from within
"object_rebuild {SeasonPass Version:10 MaxRecordings:5 Priority:10 {Series:@$fsi
d} {Station:@$station_id}}"
(procedure "restore_seasonpasses" line 64)
invoked from within
"restore_seasonpasses"
(file "./restore" line 27)

[mrtickle]

Originally posted by agw

However I'm a little unsure as to how the TiVo will react to entries that have the same name between V2 and V3, but different valid sets of values. Going from V2 to V3 should be OK as you'd imagine (or rather, hope) that the valid V3 values would be supersets of the V2 values. However there's no reason to expect V2 software to accept all values that are valid under V3. The script could certainly be changed to normalise V3 values back to valid V2 ranges of values - but I've no idea what any of those ranges or values would be, or even if it's a problem at all.


I'm guessing, but with V2 we have "First Run only" and "First Run & repeats". I understand that with V3 you have a third option which turns off the 28-day rule and records everything including duplicates? So yes I think the valid V3 values ought to be supersets.

Originally posted by agw , later on :)

The problem you hit is hopefully nice and easy to fix - there were a couple of functions where the global "db" wasn't declared :) The season pass you're adding must be one that isn't already in the guide, one of Mr Tickle's "hibernating" season passes.

Attached is another fix to backup-support.itcl to put the declarations in... untested I'm afraid but it *should* be alright... good luck!

Cool ta!

[TedZeus]

Hey AGW, it looks like there still might be some missing references. I got around the problem by moving

set db [dbopen]
to before
source $source_dir/backup-support.itcl

in the backup script.

I was able to get backups working, however when I then try a restore the TiVo core dumps and reboots. Here's what it spits out:
Dumping mempool to /tmp/BlockFailure.190

To view the blocks, run:
$TIVO_ROOT/devbin/poolview.tcl <app-with-symbols> /tmp/BlockFailure.190

In the UI that comes up, find your leaked block by address (see above)
This will help you identify the type and ownership of the blocks.

Common causes for leaks:
- Circular refs. Redefine ownership without circular dependency
- Explicit Malloc or GetChunk without Free or ReturnChunk
- Use of non-TmkCore objects, without using delete operator (TmkLock for example)

Tmk Assertion Failure:
BlockFailure, line 2073 ()
Tmk Fatal Error: Thread tivosh <190> died due to signal -2
1aa90ac 1aa79c8 1aa2290 1cda374 1ce64ec 1d37004 1d4d9d4 1d46cc8 1d3f7f8 1d3e724
1d36388 1d53470 1d3b060 1d48968 1d363d8 1d53470 1d5457c 1ce648c 1ce61d4 1800134

I'm still looking into it though.

This is on a Phillips SA running 3.x

[TedZeus]

Okay, fixed the reboot problem by getting rid of the Thumbs data in the backup file. There were more than 4000 entries. :) Then it complained about the logo data, so I dropped that too. Got my SPs and WLs back.

The reason why I had to do the restore was because I managed to corrupt my DB by trying to boot off the offline partion (while trying to recover from my botched network driver upgrade and not wanting to pull the drive)

[agw]

Ah-ha... I think you might have downloaded the latest backup-support.itcl and skipped over the first zip file that I posted up. That one had a backup script with the set db statement in the right place. I hope! It's my fault, I should really be posting up all the changed files as a set each time, sorry about that.

The other thing the modified backup script did was fix the problems when generating the rebuild logos function with stations that have no logo ID. In havanahjoe's case that was every single station. In my case it was maybe a third of them.

I hope the reboot is down to the thumbs thing. I've not noticed a reboot here yet, but then my runs of the script have been limited to simple cut-down test runs so far.

The havanahjoe problem is the same one that stops the backup logo from working for all stations - station_tms_to_fsid doesn't know about all of the stations. The original logo backup gets the stations from the channels attached to the /Setup directory, but the restore for logos and season passes tries to work backwards from the tms to the fsid by looking at the /StationTms directory. I don't know why, but there are quite a few stations that are in the channel lineup but aren't in /StationTms. Or at least, there are on my machine and also on havanahjoe's.

I *think* the fix is to go through the channels under /Setup to get the fsid but when I was testing it I realised my attempt didn't work :) Unfortunately it's now quite late here in the UK and as the TiVo has started recording a three hour program that I don't want to take any chances with, I'm off to bed and I'll try again tomorrow morning.

Thanks Mr Tickle for pointing out the stuff about V2 not having the 28 day thing that V3 has. Presumably I ought to be able to find the V3-only value in TiVoWeb in the code that lets you create season passes - once I know it then I can look for it and change it on restores of V3 backups onto V2 machines. I'll definitely do that tomorrow.

One day these scripts WILL work! ;)

[TedZeus]

Yes, I'm confident that the reboot problem was related to trying to import the thumbs. I think a 700K backup file is a bit overwhelming even for my 32MB unit.

One bit of concern though. I was able to complete the restore (40 SPs and a bunch of WLs) and all seemed well. However when I looked at the To Do list a number of programs were missing. They were in the SPs. The season passes looked okay but they had no upcoming episodes, even though they really did. In total there were 12 munged SPs (and another handful of dormant ones that I couldn't tell for certain). I was able to readd those SPs and delete the bad ones. I did a backup before and after fixing to try and pinpoint what was different. Hopefully I can post the results tomorrow.

[angra]

yeah the thumbs stuff doesn't work right anyway, I definitely dont reccomend enabling that for now.

oops on the forgotten "global db" - I did not have any hibernating series when I started, so that was a case I never got to debug :/

agw: if any of my ugly code is too obfuscated or confusing to work with, drop me a PM. I'll be happy to do my best to help out your efforts cleaning up junk I dropped in here :)

[havanahjoe]

agw:

After reading your post, I checked the guide on my Tivo and discovered that my local channels have disappeared! I had them with 3.1, and now they are gone. So I did some investigating and found out that the locals for about 10 cities are disappearing and some modifications need to be done in order to get them back.

So, because the locals aren't on the guide, and are not recognized by the Tivo, that is why I get the last error. That season pass is recorded on one of those local channels. So I am now trying to get the locals back, and if that doesn't work I'll just delete that season pass from my backup and run the restore again.

I'll post my results later

[havanahjoe]

Well I finally got my local channels back so I ran the restore script again and it completed without any errors! :D

Thanks for all the help! And sorry about all the problems.. :p

agw, if all the mods you made are going to work for everyone, then you should upload a new package that contains the latest files.

Thanks again

[agw]

It's good news about the restores! I've bundled up the modifications into a new complete archive which I hope I'll remember to attach to this posting. This is just what was there before plus:

1. If you are restoring a backup from a V3 machine onto a V2 machine then the "Record All" option (which isn't on V2 machines) is turned into "Record First Run & Repeats". I hope. I've written a trace to the screen when this happens.

2. I'd accidentally left two lines in set_station_logo commented out in previous mods to backup-support.itcl. Sorry about that! They're back in again now.

I think the problem that TedZeus hit is because the function that inserts the season pass into the MFS database doesn't tell the TiVo that it's done it. At least, it doesn't raise an event like TiVoWeb does. I'm on shakey ground with the event statement, at the moment I'm just guessing at its purpose, but it seems reasonable. I would hope, however, that this wouldn't have any long-term ill effect. It might be that you won't see recordings scheduled for the season pass until after the next call home and resulting updated guide data? I've not tried to transplant the event raising code from TiVoWeb into the script, on the principle that if it isn't (very) broken then don't fix it, and that there was probably a good reason why the event raising isn't there in the first place. HOWEVER I'm just guessing here, so if munged restored season passes stay munged after a call home then it obviously is quite serious and the script will have to change.

I've abandoned the idea of revising station_tms_to_fsid because (a) it doesn't seem to be a problem after all and (b) the code is already in TiVoWeb.

It occurred to me that these scripts aren't a million miles from becoming a TiVoWeb module. I think as the scripts seem to be on the verge of settling down, it might be an idea to begin using them as the basis for a TW module.

Instead of backup generating a script that contains function calls to recreate the backed up data directly, the TW version of backup can create a script that will just build up an in-store list of the backed up data when run. The backup file would also record the version number of the TiVo that generated the backup. The backup file would end up being just the data being backed up, which I think long-term will be a good thing.

The TW version of restore will run the script to load the data into store, present it to the user for final validation and modification and then use angra's functions and existing TiVoWeb functions to write the data into the MFS database. It would remove a lot of the redundancy between these scripts and TiVoWeb, it'll make it easier to validate the data before making changes to the MFS database and it could open the door to adding more functionality - such as changing the priority on restore, maybe making up new season passes from scratch for shows that you have never received (if you know TiVo's SH number for the show and the channel you expect it to air on) or changing the channel that the season pass is attached to before doing the restore (so if you have a channel line-up change and a hibernating season pass gets zapped by it, you can restore just that pass from backup and change the channel for it at the same time).

I'll start on this now but it will take a little while to do. The first version certainly won't do anything more than the shell version does now.

[Snoopy]

/var/hack#/var/hack# ./restore infile
> <166>Mar 3 07:34:31 tcl[253]: Tcl created pool of 1458176 bytes
bash: syntax error near unexpected token `<166>'
< free software; you can redistribute it and/or modify
<rms of the GNU General Public License as published by
/var/hack# #the Free Software Foundation; version 2.
/var/hack# #See <http://www.fsf.org/copyleft/gpl.txt>.
/var/hack# #
/var/hack# #This program is distributed in the hope that it will be useful,
/var/hack# #but WITHOUT ANY WARRANTY; without even the implied warranty of
/var/hack# #MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
/var/hack# #GNU General Public License for more details.
/var/hack# #
/var/hack# #You should have received a copy of the GNU General Public License
/var/hack# #along with this program; if not, write to the Free Software
<., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
/var/hack# retrying after errTmBackgroundHoldoff ...
bash: retrying: command not found
/var/hack# invalid command name "restore_logos"
bash: invalid: command not found
/var/hack# while executing
> "restore_logos"
> (file "./restore" line 25)
> /var/hack#

[agw]

The problem is that when you did ./backup > infile the script generated a gash line, the "TCL created pool of yadda yadda bytes" bit. havanahjoe hit the same thing, if you scroll back about a dozen messages you should see it.

infile is actually a TCL script that gets loaded and run, but the load fails because this message gets in the way.

If you edit infile and remove the line (presumably from the bottom) then you should fare better.

[Snoopy]

Oh well, I had most of them written down just in case. I am in the process of just re-creating the wishlists and season passes. Hopefully the next time around a better version will exist. I'll keep checking back. Thanks for all your help.

Direct Tivo Sony T60.

[agw]

I don't know about better but there'll certainly be a different version available when the TiVoWeb module is finished.

The TiVoWeb version should be capable of running without you having to remove everything first (it won't restore passes & wishlists if they're already there), it should show and let you remap missing channels and it should let you restore a subset of the passes and wishlists in a backup file. It'll definitely show you the content of a backup file before you restore from it.

It won't do anything with logos though. Maybe a future version.

A reasonable version should be ready by the end of next week, there or thereabouts.

[Snoopy]

Originally posted by agw
It's good news about the restores! I've bundled up the modifications into a new complete archive which I hope I'll remember to attach to this posting. This is just what was there before plus:

1. If you are restoring a backup from a V3 machine onto a V2 machine then the "Record All" option (which isn't on V2 machines) is turned into "Record First Run & Repeats". I hope. I've written a trace to the screen when this happens.

2. I'd accidentally left two lines in set_station_logo commented out in previous mods to backup-support.itcl. Sorry about that! They're back in again now.

I think the problem that TedZeus hit is because the function that inserts the season pass into the MFS database doesn't tell the TiVo that it's done it. At least, it doesn't raise an event like TiVoWeb does. I'm on shakey ground with the event statement, at the moment I'm just guessing at its purpose, but it seems reasonable. I would hope, however, that this wouldn't have any long-term ill effect. It might be that you won't see recordings scheduled for the season pass until after the next call home and resulting updated guide data? I've not tried to transplant the event raising code from TiVoWeb into the script, on the principle that if it isn't (very) broken then don't fix it, and that there was probably a good reason why the event raising isn't there in the first place. HOWEVER I'm just guessing here, so if munged restored season passes stay munged after a call home then it obviously is quite serious and the script will have to change.

I've abandoned the idea of revising station_tms_to_fsid because (a) it doesn't seem to be a problem after all and (b) the code is already in TiVoWeb.

It occurred to me that these scripts aren't a million miles from becoming a TiVoWeb module. I think as the scripts seem to be on the verge of settling down, it might be an idea to begin using them as the basis for a TW module.

Instead of backup generating a script that contains function calls to recreate the backed up data directly, the TW version of backup can create a script that will just build up an in-store list of the backed up data when run. The backup file would also record the version number of the TiVo that generated the backup. The backup file would end up being just the data being backed up, which I think long-term will be a good thing.

The TW version of restore will run the script to load the data into store, present it to the user for final validation and modification and then use angra's functions and existing TiVoWeb functions to write the data into the MFS database. It would remove a lot of the redundancy between these scripts and TiVoWeb, it'll make it easier to validate the data before making changes to the MFS database and it could open the door to adding more functionality - such as changing the priority on restore, maybe making up new season passes from scratch for shows that you have never received (if you know TiVo's SH number for the show and the channel you expect it to air on) or changing the channel that the season pass is attached to before doing the restore (so if you have a channel line-up change and a hibernating season pass gets zapped by it, you can restore just that pass from backup and change the channel for it at the same time).

I'll start on this now but it will take a little while to do. The first version certainly won't do anything more than the shell version does now.


I have tried the latest release with a new backup and restore file just to see. I need to reset guide data but can't yet. Can anyone tell me how to get a good backup using the latest files as of this writting.

Errors:
invalid command name "232"
while executing
"232"
(file "infile" line 1)
invoked from within
"source [lindex $argv 0]"
(file "./restore" line 22)
/var/hack#s:

[agw]

It sounds like you have some junk at the start of your file - if you attach "infile" to another message I can have a look at it.

[Snoopy]

Originally posted by agw
It sounds like you have some junk at the start of your file - if you attach "infile" to another message I can have a look at it.

Outfile/Infile attached

[agw]

Okey dokey - attached is your outfile.txt minus the gibber line at the start. It should load and run OK now. Whether it works or not is another matter! :)


Good luck,
Andrew.

[Snoopy]

Now THAT rocks!

Thanks for your help. It worked perfectly on both Season Passes and Wishlists.

Question: When creating an outfile, it takes an amazingly long time to create the file due to logo info. Is there a way to tell the restore > infile command to ignore the processing of logo info unti such time as I choose to figure out just how additional logos will help me? This would save alot of time on the backup and create less room for error. Thanks again!

[angra]

just comment out the logo function calls in "backup" and "restore".

I think they are "dump_logos" and "restore_logos", or something along those lines.

[Snoopy]

Originally posted by angra
just comment out the logo function calls in "backup" and "restore".

I think they are "dump_logos" and "restore_logos", or something along those lines.

It takes no time to restore. On exporting though, through the backup command, I would like it to not even try to backup the logo info. Something like command "backup > outfile -logos" is what I was looking for.

[angra]

nope, that's not in there.

you have the source, you can mod the program all you like.

[Snoopy]

Thanks for your help in getting the outfile prepared for import. It works beautifully. I am clearing and deleting everything as we speak via the Tivo reset options. Do you have any time line on which to expect a backup utility for TivoWeb?

[agw]

Hopefully by the end of next week if all goes well. It should be finished a little sooner than that but it'll need testing etc.

[mrtickle]

But is the current version working perfectly? Why not get this one completely bug-free first, otherwise things will get very confusing... :)

[agw]

The current script is pretty much bug free as far as I know, in that it's perfectly possible to backup and restore season passes etc. with it. It's difficult to tell when only three people have used it with any success, but the nature of the thing is such that you don't expect lots of people to use it every day - you usually have an extraordinary reason to be in possession of a blank TiVo and need to restore everything onto it - so it could take ages to build up a good test sample.

The problem with the junk line at the start that some people have hit can be fixed by having backup create the file and write to it directly, rather than writing to stdout and have the user redirect it. But it's not a trivial change, in that you have to change the lines that write output, and there are a few of them. If you hit the problem it stops the script before any writes are made so it's not a machine killer. You just need to edit out the line at the start of the file.

The drawbacks with the current script are that you need to be comfortable with running scripts from the command line, at the moment you may also need to be comfortable with editing the output, and because there is no interaction it's an all-or-nothing operation with no provision for the channel lineup changing between the backup and the restore and no provision for restoring just a subset of the passes and wishlists you backed up.

All things considered I thought the command-line scripts were as good as they needed to be given the kind of user that would be likely to be running them, and that it was a reasonable time to start on the TiVoWeb version.

[mrtickle]

Fair enough, thanks for the summary. I still intend to be running it myself soon, but I need to sort out the logistics first - watch all my recordings (nearly done!), back up all my other scripts/hacks etc. Also my only "window" of opportunity when my tivo isn't recording anything for a decent number of hours and I have free time is Saturday afternoons :).

Thanks for pointing out the channel lineup issue. I hadn't realised the implication of that - it means I can't take a backup now and lock it away ready, if there were any lineup changes between now and the day I did the restore I could have problems. If I understand it correctly, on the day I do this I should make a daily call to make sure my lineup is completely up to date, then run the backup script (and copy the file off safely!), then blast everything away and restore.

[Snoopy]

Originally posted by agw
The current script is pretty much bug free as far as I know, in that it's perfectly possible to backup and restore season passes etc. with it. It's difficult to tell when only three people have used it with any success, but the nature of the thing is such that you don't expect lots of people to use it every day - you usually have an extraordinary reason to be in possession of a blank TiVo and need to restore everything onto it - so it could take ages to build up a good test sample.


I have used it with success. The only thing that was strange to me is that the import appeared to work when I tested it before reloading my Tivo HD image. I tried to re-import immediately after I got my Tivo back up and running and I still got errors unrelated to the first line which DO need to be removed as stated in the previous post. The reason for the errors was simple. I simply didn't have any guide info so the only thing that imported initially was wishlists. It is best to wait until the guide data fills up over 3 days before doing a full import. I waited and it worked perfectly after the guide data filled up. Thanks to everyone that has worked on this command line script.

[agw]

If the script doesn't find the series you're importing then it will add it in, it carries a copy of it in the backup file. The guide data shouldn't really matter. The only thing it expects to find on the TiVo is the channel guide.

Having said that I expect that if you re-run the guided setup then it is probably a good idea to wait eight hours for the indexing to finish before restoring season passes.

[Snoopy]

Originally posted by agw
If the script doesn't find the series you're importing then it will add it in, it carries a copy of it in the backup file. The guide data shouldn't really matter. The only thing it expects to find on the TiVo is the channel guide.

Having said that I expect that if you re-run the guided setup then it is probably a good idea to wait eight hours for the indexing to finish before restoring season passes.

I have confirmed that this is not the case. I ran it first and got nothing but wishlists. I ran it 12 hours later and got wishlists and SOME season passes but it errors out at the end. I ran it again 12 hours later and all is well. It won't add everything back in unless one waits at least 24 hours. I would add a recommendation in the readme to wait 36 hours before re-importing the data. I feel certain you will disagree but that is my experience.

[agw]

I do believe you - I'm hardly in a position to disagree, not having tried it for myself. I'm in no hurry to re-run guided setup to try it either :)

What error did you get after 12 hours?

Do you have any hibernating season passes - passes for shows that are not currently being aired?

[angra]

I have a spare drive specifically set aside for testing these sorts of things.

I'm willing to run a test or two to try and figure out what issues are coming up, at some undetermined time in the near future :).

Any specific information about the errors that came up will probably be helpful, since, if possible, I will try to make sure that I create the same conditions.

If only I had a spare tivo, I could run the test tonight :)

[Snoopy]

Originally posted by agw
I do believe you - I'm hardly in a position to disagree, not having tried it for myself. I'm in no hurry to re-run guided setup to try it either :)

What error did you get after 12 hours?

Do you have any hibernating season passes - passes for shows that are not currently being aired?

HMM I knew I should have made a copy of that error but I didn't. I feel quite certain there are "hybernating season passes for shows that are not aired" because after all, that is the nature of things when you do a backup. Obviously the time you restore is going to be LATER than the original backup. However, the course of events as described in my previous post, does not point to "hybernation". It does point to data filling in. It is not necessary that you agree with me at this point. I am sure I will be using the web version next time with a whole different set of issues. I have all I need for now. Thanks.

[havanahjoe]

I've added some new season passes and wishlists and wanted to back up everything the way it is now. So I downloaded the latest file and ran the backup script, only to find that it dumps the wish list and logo info, but the season pass section is empty! I'm now running 2.5.2 on my Hughes box.

Any ideas?

[havanahjoe]

I just read a post about removing User from the Season Pass path. That did it. Why does the script have this as the default path? Is this the way Season Passes are stored on UK Tivos?

Maybe this detail should be added to the README file..

[angra]

The original script had season pass hard coded to the v3.0+ location for season passes, since that's what the original author used as a development platform.

I believe that AGW added sufficient checks in his bugfixes release, but I'm not sure.

I think he's working on a totally new approach though, so I believe that continued work on the original approah is more or less suspended.

[havanahjoe]

That's great! If this first version works wonders, I can't wait to see the next one!

Great work AGW! Thanks again!

[agw]

The current backup does set the correct season pass directory and I've tested it on a UK 2.5.5 TiVo - it *should* be working alright. However, as angra mentions there is a TiVo Web version in the works that should hopefully address one or two of the points with the command-line version. The TiVo Web version uses the same season pass directory as the rest of TiVo Web, so it should avoid the problem you're describing.

I hope it lives up to expectations! :) It's pretty much finished, just testing it now. Looks alright so far. It should be ready about mid-week, if everything goes well.

I only have a UK 2.5 TiVo so it will need testing with the US v3 TiVo's before it can be used in anger.

[agw]

Mid-week obviously didn't happen :) It was working fine, I was wiping and restoring season passes all over the place, but then I thought I'd better test it with a virgin program guide - so after running clear program data & todo list I ran the restore and I had the same result that TedZeus had with angra's script... a bunch of season passes showed as "No upcoming episodes" even though some of them had two broadcasts a day. I left them for a couple of days but they didn't sort themselves out, even after I removed them and restored them again.

It turned out that the TiVo Web function that translates from the TmsId (which uniquely identifies a series across all TiVos) to the FSID (which uniquely identifies the series on your TiVo) was returning orphan records for some series. angra's script was using a copy of the TiVo Web function, so it would have done the same thing to TedZeus. It works fine on established TiVo's, I think the orphans are only there for a while after you restore program guide data.

Anyway, the module now checks for orphans, shows you them and skips any season passes for them when restoring. It also has code in there to get the FSID the long way, which can take several minutes (in theory half an hour!) to run... but it keeps you informed while it's doing it, and it isn't something that should happen too often. Ideally the tms_to_fsid function should skip over orphans, but I figured it was easier to go with the work around in the short term.

I'm currently clearing program guide and todo lists again and then I'll do a daily call for the program guide. After that I'll wait a few hours and do a clean restore, and then keep an eye on it. I expect it to work fine. If this turns out to be the case then I'll put up the module and documentation (such as it is) over the weekend for beta testing by any brave soul who cares to try it out :)

[Snoopy]

Is this going to be a new release of Tivo Web. If so, I would like to report a problem recording individual shows. If only ONE tuner conflicts with a show, it does not allow Tivo to record. I would like to have the option to go ahead and record despite the conflict for individual shows. Thanks for all your fine work.

[agw]

I'm afraid it's just an add-on module for TiVo Web - it steers well clear of any changes to existing modules, it just uses some of the existing TiVo Web functions.

[Snoopy]

Originally posted by agw
I'm afraid it's just an add-on module for TiVo Web - it steers well clear of any changes to existing modules, it just uses some of the existing TiVo Web functions.

Who is the Author of TivoWeb?

[agw]

For better or for worse I have a version of the TiVo Web backup & restore module for people to try. It can be found on http://www.boygenius.co.uk/tivo - sorry about the domain name, Dexter's Lab is to blame.

The module is still very much a work-in-progress and has two known glitches - the first is that it can be slow when loading a large backup file and the second is that it really eats up memory and can run out of store when restoring large backup files. It takes steps to try to figure out how much memory is left and stop before it uses it all up, but the code for this is experimental and I'm not convinced that it is working as intended.

The module has been used to backup and restore two UK 2.5.5 TiVo's - my own (several times!) and mrtickle's, who was kind enough to test early versions of the module for me. It has not been tested on US TiVo's at all.

My season pass list runs to about 45 items and it doesn't hit any out-of-memory problem. However mrtickle's is around 160 items and he did hit the problem when restoring. It can be worked around, but it is a pain. I'll be getting the store requirements down in the next version.

[Snoopy]

Backup Checklist

Cannot continue - less than 16K free in the pool on taking snapshot theme
Forcing an 'INTERNAL SERVER ERROR' to stop the module now
INTERNAL SERVER ERROR
--cut here--
action_backup_create_write '' 'set "fname" "/tivoweb-tcl/backups/settings";set "submit" "Create";'
invalid command name "stopthisscriptnow"
while executing
"stopthisscriptnow"
(procedure "check_store" line 17)
invoked from within
"check_store "on taking snapshot theme""
(procedure "snapshot_add_theme" line 11)
invoked from within
"snapshot_add_theme $fsid $content $full_info"
("uplevel" body line 5)
invoked from within
"uplevel $body"
invoked from within
"ForeachMfsFileTrans fsid name type "/Theme" "" 20 {
set theme [db $db openid $fsid]
set fields [dbobj $theme attrs]
set content [construct..."
(procedure "take_snapshot_theme" line 8)
invoked from within
"take_snapshot_theme 1"
(procedure "take_snapshot_for_backup" line 3)
invoked from within
"take_snapshot_for_backup"
(procedure "create_backup" line 18)
invoked from within
"create_backup $chan $fname"
(procedure "::action_backup_create_write" line 9)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[havanahjoe]

Great module! The backup worked great on my Hughes DirecTiVo with v2.5.2. I'll let you know what happens when I use the restore function.

Thanks agw!

[agw]

Okey dokey - version 1.00.0003 is on http://www.boygenius.co.uk/tivo - this version is identical to 1.00.0002 except I took the store checks out of the backup. The backup didn't cause a problem with mrtickle's 160 season pass TiVo. It was the load and restore that caused problems.

The store check is far from perfect but I've left it in place on the restore for the time being until I stop it from hogging so much store. So you might still get the "Forcing an INTERNAL SERVER ERROR" message when you load and restore.

I've noticed that these can occasionally be cleared by doing a Restart | Quick Reload in TiVo Web.

[Combat Medic]

OK, on my HDVR2 it right away dies. Is it possible to run the commands in the backup file manually one at time?

-Mike

[agw]

Originally posted by Combat Medic
[B]OK, on my HDVR2 it right away dies. Is it possible to run the commands in the backup file manually one at time?


What message are you getting? Also, are you running the shell backup script or the TiVo Web backup module?

The backup file generated by the former writes directly to the database, so it is possible to edit the file to restore just the bits you're after (although if you restore a wishlist season pass then you also have to restore the associated wishlist in the same session).

If it's the latter then the backup file just loads the season passes into store. You can reduce the number of items that are loaded into store by trimming down the list of season passes, wishlists and series - but if you're getting an out of memory error it is just as likely that the code to figure out how much memory is left isn't working properly. I should probably just take that bit out, I've a feeling it's going to be more trouble than it's worth.

[Combat Medic]

I'm using the TiVoWeb version. I get the error that the code trips to protect the TiVo from crashing. It doesn't look like it processed any data at all. Can you give me a pointer as to what I need to comment out to remove the memory check?

Thanks
Mike

[agw]

OK - I've removed the call to check the store. I would put it on the web site but the swining thing is down at the moment, so I've attached it to this posting.

[Combat Medic]

Originally posted by agw
OK - I've removed the call to check the store. I would put it on the web site but the swining thing is down at the moment, so I've attached it to this posting. OK, I've got a new error which means I'm making progress right? :) BTW, thanks for all of your help and this wonderful tool.


INTERNAL SERVER ERROR
--cut here--
action_backup_restore_read '' 'set "fname" "/var/hack/tivo_settings";set "submit" "Load";'
can't read "restore_station(84194)": no such element in array
while executing
"set data $restore_station($fsid)"
(procedure "station_is_mapped" line 8)
invoked from within
"station_is_mapped $station"
(procedure "automap_restore_sps" line 28)
invoked from within
"$script"
invoked from within
"time {$script}"
(procedure "agtime" line 2)
invoked from within
"agtime {automap_restore_sps}"
(procedure "load_backup" line 32)
invoked from within
"load_backup $chan $fname"
(procedure "::action_backup_restore_read" line 10)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--


-Mike

[agw]

Okey dokey - I can see the problem. The channel that Scrubs, Fear Factor and ER are on wasn't in the original list of stations that was saved with the backup file.

If you are just loading this onto the same TiVo that you used to create the backup then stopping and restarting TiVo Web may get the station into TiVo Web's list.

I'll make two changes to the module:

1. When I create the backup file I'll check that the station exists in the TiVo Web list rather than assume it's there. I should really have done that in the first place. Sorry about that!

That will stop the problem from happening from now on. For backup files made already that may have missing stations I shall also:

2. Detect missing stations when I read the backup file in and create a dummy channel called "Missing" for them. That way you can use the channel remapping part of the module to map them onto the real channel. You can probably get a good channel list by stopping and restarting TiVo Web. I'll look into making the appropriate call in TiVo Web to get it to rebuild it at the start of a backup. I'm assuming that the channel was added after TiVo Web was last restarted.

I don't have the time right now to make the changes - I'm working now (or I'm supposed to be), then I'm out tonight and there is a server move at work over the weekend - so I might not have a good version of the module until Sunday. However if you want to have a play in the meantime, I've kludged the backup file you sent to add the missing channel. This one loads up fine on my UK TiVo and I remapped the missing channel to UKFOOD(!) and restored those three season passes for it, so it should be alright on your machine :)

[Combat Medic]

It worked wonderfully, thanks for the help.

-Mike

[mrtickle]

I should post here too - just to say that this is a superb module! It solved my problem and I'm sure many people will find it invaluable in the future :D

[agw]

Jolly good, glad it's been useful! :)

Sorry about the delay with the update. I've put the next version onto www.boygenius.co.uk/tivo/ - this one creates much smaller backups by missing out the channels that don't have season passes on them (thereby saving time and memory when restoring) and it copes with backup files created with prior versions that have season passes with missing stations (such as combatmedic's).

It also checks to see whether the station for each season pass is in the backup file as the file is being created. If the station is missing from TiVo Web's list of channels then it gets added to the end of the channel list in the backup file. This means that the channel number will be wrong (I usually get that from the TiVo Web channel list)... but in the grand scheme of things it doesn't matter, it makes no difference to the restore.

Edited to add...

I should point out that this latter case is not the norm - I expect TiVo Web to usually have the full list of channels and for none to be missing when the backup is taken. Normally the channel numbers in the backup file will be correct as of the time the backup was taken. The change just copes with the odd occasion where this isn't the case.

[mrtickle]

Yep. I think when I did it, because I was starting from scratch and doing Guided Setup again, I made different decisions in the "Channels I Receive" list and left a few more channels unticked that I realised were complete rubbish. When I then restored from the SP backup taken on my old tivo image, your module tried to put those channels back again :D. But it gave me the choice which was cool.

[agw]

It won't put the channels back, it just tells you that they're missing from the current lineup and lets you map them onto your current channels. Or rather, map season passes for those channels onto channels you currently have. Handy (I hope!) for those occasions where a channel and all of the season passes for it gets deleted and then put back in under a different name.

I don't know how often that happens in the States but here in the UK it felt like it was happening once a week when we had the old digital TV service. It used to drive me mad! :)

[Snoopy]

INTERNAL SERVER ERROR
--cut here--
action_backup_create_write '' 'set "fname" "/tivoweb-tcl/backups/settings";set "submit" "Create";'
can't read "snapshot_station(65208)": no such element in array
while executing
"puts $fd " load_channel {$fsid} \{$snapshot_station($fsid)\}""
(procedure "create_backup_write_channels" line 13)
invoked from within
"create_backup_write_channels $chan $fd"
(procedure "create_backup" line 26)
invoked from within
"create_backup $chan $fname"
(procedure "::action_backup_create_write" line 9)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[mrtickle]

Originally posted by agw
It won't put the channels back, it just tells you that they're missing from the current lineup and lets you map them onto your current channels. Or rather, map season passes for those channels onto channels you currently have. Handy (I hope!) for those occasions where a channel and all of the season passes for it gets deleted and then put back in under a different name.

I don't know how often that happens in the States but here in the UK it felt like it was happening once a week when we had the old digital TV service. It used to drive me mad! :)

Yep. When I say "tried to" I meant "tried to persuade me" :). I know it will be useful. But the channel in question wasn't one of those I ever had a SP for!

[agw]

Yes, the previous versions were a bit eager about mapping channels regardless of whether they had a season pass on them :)

This version should be better behaved though, and as an added bonus it should produce smaller backup files, take less time to load and so on. If the swining thing works that is.

To that end I've attached a test version of backup.itcl - it's not on the web site, I can't be arsed to package it all up if it's not going to do the trick. I've tightened up the code that removes unused stations from the list of stations to backup - if Snoopy wouldn't mind downloading it and giving it a go I'd be grateful.

[Snoopy]

Originally posted by agw
if Snoopy wouldn't mind downloading it and giving it a go I'd be grateful.

Yes, a little faster too.

Backup Checklist
Copy existing backup file to '/tivoweb-tcl/backups/settings.old' Done
Overwrite backup file '/tivoweb-tcl/backups/settings' Done
Write header Done
Store layout version number 1 Done
Store TiVo version number 2.5.2-01-1-011 Done
Store details for 15 channels (for reference only, may contain duplicates) Done
Store details for 12 wishlists Done
Store details for 33 season passes Done
Store details for 22 series Done
Return to Backup Menu

Nicely done. I would add something at the beginning that says "CONGRADULATIONS on successfully installing this backup utility for TivoWeb" "please be patient" "This process may take a couple minutes to complete". "You will status messages once the process is finished".

[agw]

Ah, I'm just glad it works! :) Thanks for checking it, I'll update the website version.

The original intention with the checklist thing was that it would come up as each stage completed. However I made it a table to make it prettier, and of course they have to finish before the browser will display any part of them... so it doesn't quite do the job it was originally intended for. But it looks nice ;)

I've only got 40-odd stations and the same number of passes, so the backup is over in no time on my machine - I didn't realise there was a problem with the backup going slowly. I'll change the module at some point to stop showing the checklist as a table, that way it'll give the user a little more feedback as it's going along.

[Snoopy]

I'm happy to help anytime. Feel free to pm for my MSN as well. Take care.

[Snoopy]

running in to problems doing the restore. I have included the error and the settings file created by the latest version.

It seems the WishList comes in fine but the Season Passes are left out.

INTERNAL SERVER ERROR
--cut here--
action_backup_perform_restore '' 'set "submit" "Restore";'
can't read "restore_theme(99674)": no such element in array
while executing
"set result $restore_theme($fsid)"
(procedure "get_theme" line 5)
invoked from within
"get_theme $theme_fsid $use_restore"
("Wishlist" arm line 3)
invoked from within
"switch -exact [agextract $data agwSPType] {
Series {
set series_fsid [agextract $data Series]
set series [get_series $series_fsid $use..."
(procedure "get_sp_name" line 2)
invoked from within
"get_sp_name $sp 1"
(procedure "perform_restore_sps" line 18)
invoked from within
"perform_restore_sps $chan"
(procedure "::action_backup_perform_restore" line 14)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[agw]

Okey dokey - sorry for the late reply, I was on holiday and didn't get back until last night.

I've had a look at the backup file and basically the wishlist item with the ID number 99674 is missing from the backup file.

Could you check on TiVoWeb and see what the wishlist is for the 7th season pass, and then check to see whether TiVoWeb can see that wishlist properly please?

You may be able to get a dump of the wishlist on the machine from which you took the backup if you go into TiVo Web and then (assuming your tivo has the name 'tivo' - change if not) enter this URL:

http://tivo/object/99674

You should get a dump of the information for the theme. You should also see the wishlist under

http://tivo/mfs/Theme

- it's this list that gets dumped to the backup file so if it's missing from there then that's the reason it's missing from the file.

This is assuming you still have the original TiVo that the backup was taken from. If you don't then I'm afraid the only recourse for the time being is to remove the 7th season pass from the backup file so that the file can be made to load. I'll change it to check and automatically ignore season passes with missing wishlists, but it'd be nice to see what caused it first.

[Snoopy]

hmmm,
I am not sure what you mean by dumps and so forth. What I did was a backup of the original tivo configuration with your latest software. That backup is still in it's original state in a file called "settings" which I sent you. I did a "clear and delete everything" by using the remote. This helped me get my local guide data back. It does however, remove the "to do list" as well as the stored "season passes" and "wishlists". Knowing this was to happen before hand, I did a backup using your latest tivoweb backup module. What I sent you was the setting file your software created and the error it gave me upon trying to restore the data. it restored the wishlist in tact but did NOT restore the season passes. The error it gave was also sent. Can you clarify what I need to do to get the data back given 'my' senario.

[agw]

Okey dokey, no worries. In point of fact it doesn't actually restore anything until after it has loaded the backup file and you've clicked the restore button - the failure happens during the file load, so no wishlists have been restored by that point. Any wishlists on your TiVo were left there after the operation you carried out.

I've attached a modified version of your settings file that has the affected season pass commented out. I've loaded the file onto my machine and it loads fine, there should be no problems with it - although I'm afraid you'll have to add the missing season pass back in by hand. Unfortunately I can't give any decent clue as to what it might have been, other than it was originally the 7th season pass in the list.

I'll change the module to check the wishlist for each season pass as it writes it to the backup to make sure that it exists, and similarly change the restore part to cope with missing wishlists on loading the file so that any files already made (a) don't have to be modified by hand and (b) show why the season pass can't be restored. I don't have a lot of time at the moment but it should be done within the week, probably at the weekend.

[hoopsbwc34]

Originally posted by Snoopy
Nicely done. I would add something at the beginning that says "CONGRADULATIONS on successfully installing this backup utility for TivoWeb" "please be patient" "This process may take a couple minutes to complete". "You will status messages once the process is finished".

Does this tool only work with TivoWeb and a hacked system? I'd like to use some tool to revert to an older version of software for my HDVR2, and still be able to keep my season passes etc.

Does a tool like this exist now (similar to mfs_tools)?

Thanks

[mrtickle]

The original scripts don't need TiVoweb - read back from the start of this thread.
They were developed into a module which runs on TiVoweb.
HTH

[havanahjoe]

Well I'm back in this thread with some more problems for you agw. I hope you can help me again.. :D

I'm trying to restore a backup I made using the tivoweb module, but my TiVo reboots every time. I have 44 unmapped channels, and when I try to map the ones that affect the season passes I backed up, the TiVo reboots. If I try to select the season passes to restore, it reboots. If I try to restore everything without configuring anything first, it reboots.

So I decided to use the command line program that I had used before, and I am getting this error:

<155>May 1 20:28:57 Database[122]: Duplicate index: /Server/ATSH524467:4:0:0

commit failed (0x00030012)

while executing
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set new_obj [db $db create Series]
for {set i 1} {$i<[llength $object]} {incr i} {
set element [lindex $object $i]
..."
(procedure "rebuild_series" line 24)
invoked from within
"rebuild_series {SH524467 Version:49 ServerId:ATSH524467 ServerVersion:0 {Genre:
375 105 124} ThumbData:268566912 {Title:{John Doe}} TmsId:SH524467}"
(procedure "restore_seasonpasses" line 31)
invoked from within
"restore_seasonpasses"
(file "./restore" line 26)

Sometimes I get a retrying after TMFBackgroundholdoff (or something like this) and after many of this messages, the TiVo reboots. And it happens with the same season pass, which is scheduled for an existing channel. What I'm thinking is that I have local channels for LA, and I also have the networks on 381,383,387 and 389 (which are the same channels) and have noticed that the tivo maps all the season passes to the LA channels (when I first configured them for the 380 channels). So maybe this is the problem, the script is finding to channels that are the same and it doesn't know what to do.

Now, I don't think that the Tivoweb module problems are related to this same double channel issue, but I may be wrong.

So, agw, if you have any suggestions for me. Please let me know.

Thanks

edit: I forgot to mention that I AM using different backup files for the tivoweb module and the command line program.

[agw]

Sorry for the delay in replying - the forum decided to disappear at just the wrong moment! :)

I'd like to say I have some idea as to what the problem is with your backup but nothing definite comes to mind. If the TiVoWeb module runs out of memory then it'll reboot the TiVo so at first sight it appears to be that - in which case you can work around it by editing the backup file to contain *just* enough information to restore the ones you're interested in. That will take a little patience though because there are no comments in the backup file to help with this. I intend to write a program to mangle these files one day but at the moment I'm afraid it's a case of editing it by hand.

However - the problems you're having with the command-line program would imply that there's more to it than just running out of store. The command line program searches the MFS database to try to find the series - if it's there then it keeps a note of its identifier and uses it, but if it's not then it creates the series from scratch. The error you're getting is saying that it is trying to create a series but it's already there - which means that it tried to find the series, failed to find it, but when it tried to create it the TiVo said the series was already there (i.e. it shouldn't have failed to find it).

As to why this might be - hum. Dunno. I seem to remember that figuring out if a series already exists on the TiVo is a little fraught. The way I went in the end was to use a mix of what the original backup script did, which was to try to figure backwards from the TMS identifier for the series to the FSID using a function that is present in TiVoWeb, but unused; and also by looking up the series in Server using the ServerId - but this ID seemed to change between versions though so it wasn't always reliable. I did this because the experimental tms_to_fsid function could occasionally miss series that were actually there, so it sounds a little like you're running foul of that.

In the month that has passed since you originally posted the problem have you managed to work past it? If so, what did you do?

[havanahjoe]

Well.. I just deleted the series that were giving me problems and I didn't have to create them again since its the end of the season. The problem with my TiVo is that I always have a mess with all the local channels, I keep adding some new ones and deleting the others, so that is probably part of the problem since the error I posted here is for one of the shows that I record of the local channels. I also was using a kind of old backup so I had to recreate many shows by hand, all because I was too lazy to make newer backups. But other than this error everything worked flawlessly. Thanks again for a great tool agw.

[Snoopy]

The last one did not work well

[Willin]

Most of my season passes have been hosed recently. I backed them up, cleared out the guide data, restored, and all is well. I'm so glad I didn't have to recreate my 92 season passes by hand. Great work.

[agw]

Sorry for the delay, again! Busy time at work :)

I think fundamentally the module is alright, it's just that it doesn't expect the list of season passes that it gets from the TiVo to refer to a wishlist that *isn't* in the list of wishlists that it gets from the TiVo. Snoopy managed to get a backup with this so I was going to add a sanity check for it, just in case.

Hopefully I'll finally get around to this at the weekend.

[Snoopy]

We all appreciate it.

[troycarpenter]

I had to wipe out my guide data and to-do list earlier today. Before the wipe out, I had saved my backup data, and the restore at the time said it would not do anything since all the data was still there. Now after the system wipe, I am trying to do a restore and I am getting the following error with the same backup file:


INTERNAL SERVER ERROR
--cut here--
action_backup_restore_read '' 'set "fname" "/var/hack/tivoweb-tcl/backups/settings";set "submit" "Load";'
Error: infinite loop in binary search (2 1 1 1)
while executing
"error "Error: infinite loop in binary search ($bottom $top $current $slen)""
(procedure "get_recordindexsearch" line 41)
invoked from within
"get_recordindexsearch $tmskeyfsid $startoffset $maxoffset $recsize $incrsize $search 0 11"
(procedure "tms_to_fsid" line 38)
invoked from within
"tms_to_fsid $tmsid"
(procedure "get_series_fsid" line 12)
invoked from within
"get_series_fsid $restore_data"
(procedure "automap_restore_series" line 8)
invoked from within
"$script"
invoked from within
"time {$script}"
(procedure "agtime" line 2)
invoked from within
"agtime {automap_restore_series}"
(procedure "load_backup" line 28)
invoked from within
"load_backup $chan $fname"
(procedure "::action_backup_restore_read" line 10)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--


I have tried the old non-tivoweb scripts and I get some error about not finding procedure "restore_logos". If I comment that one out, it just goes down the line on each procedure saying it can't be found.

The guide data shows up on the channels, but there is no data related to future shows. If I do a search by title, there are no programs listed.

Do I need to wait until after the indexing occurs before I can do the restore?

Help!

[agw]

The first thing to mention is that the tivoweb module and the old backup script create and use different backup files, they're not interchangable - hence the errors when running the old restore script.

There are alternatives to the function that you're having problems with and they're already built into the script. I'd been warned by the author that the function was a bit experimental; despite this I'm unfortunately calling it in such a way that an error will stop the module, as you saw. It'll probably be better if I make it fall back to the alternative methods to find the series if tms_to_fsid breaks. At the moment it only falls back if tms_to_fsid says it can't find the series I'm looking for.

I'm changing one or two other things in the module this weekend so I'll add this in at the same time.

However, putting that aside, have you left the TiVo to index the new guide data? This can take about eight hours or so. When I was testing the restore module I used to leave the background re-index running overnight and then do the restore the next morning. If the TiVo is indexing the guide data in background then it won't be in a fit state to restore onto.

[agw]

Hum. Just read the last two paragraphs of your message properly!

Yes, you need to wait until the reindex finishes. I should read messages through twice before replying to them :)

[troycarpenter]

I figured it had to wait until the shows were indexed. Thanks for the quick reply.

And I have the same problem about reading and understanding before I start to reply...

Troy

[agw]

I've uploaded the latest version of the backup module to http://www.boygenius.co.uk/tivo/

This version puts up a 'please wait' message while the backup snapshot is being taken, it copes with season passes that refer to wishlists that don't exist and it will silently (I hope) ignore errors in tms_to_fsid - if tms_to_fsid errors the module falls back to alternative methods for searching for the series details.

[rbiro]

Having just hosed my Tivo and restored its image, the season pass and wishlist restoration was wonderfully simple!

A million thanks!

But speaking of wishes, how about adding the state of Thumbs Up/Down in the file?

With wishlist, season pass and thumbs, you would define the "personality" of any tivo.

[angra]

the original script from which the module started did include thumbs functionality but it was, basically, commented out.

I could restore the thumb data properly, but somehow the restored data never seemed to influence the suggestions engine. I tried a bunch of things but never quite got it to work. About this time, the whole suggestions engine started to suck horribly for me, so I more or less gave up.

In reality, the answer is probably not far off of what's in the original script, but it does take a pretty large amount of time to do tests...anyone really interested can probably make it work if they have more drive than I did.

[Snoopy]

It is been my experience that thumbs data is pretty useless at least for me. Ideally, I would like to have much greater control over what gets recorded than I do but thumbs up thumbs down falls WAY short in my view.

In the interest of a good stable backup, I would like the option to turn this thumbs up/down business on or off during the backup process if it is ever added as I feel that it will complicate an already shakey process. Just my two cents.


A RANT about season passes and wishlists (posted in main forum):

It bugs me that these are separate items. It would be great if we could add it all into the wishlists. BTW, AGW has put together a wonderfull backup utility at the tivocommunty.com that is more stable now. It backs up both season passes and wishlists in great form.

The problem with season passes to begin with is that they are seasonal and channel specific. I realize this is the point of seasosn passes, but a much more workable solution seems possible to me. Season passes are wonnderfull except that next season, you have to remember to re-add the item. Channels change. Time slots change. This is why I would like to see wishlists get a little more complex and do away with season passes all together.

1) What about having similar titles start comming up when you start typing them in as they do with season passes?

2) What about adding an option to KEEP the most current "date" of a particular show, in addition to the option for "first run" which we already have. In this way we would not be tied to a particular channel or time slot, and could still see the latest stuff. Better yet, keep a log of what we have actually "watched" assuming this is possible, and record over and over again, the ones we have just not gotten around to watching.

Point is there is really no need for both. One list could take care of both season passes and wishlists and would not have to be channel specific.

I am just ranting here but would be currious to hear others comments about the matter.

[Crispin]

I just got the following error from latest version:

INTERNAL SERVER ERROR
--cut here--
action_backup_create '' ''
can't read "default_savedir": no such variable
while executing
"file isdirectory $default_savedir"
(procedure "get_default_backup_dir" line 10)
invoked from within
"get_default_backup_dir"
(procedure "::action_backup_create" line 2)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

This error doesn't happen when the tivoweb-tcl/backups directory exists (which the module seems to create when it generates this error).

[agw]

Ack! You're right, I've checked back through the archives and that bug's been in there since pretty much forever. Sorry about that. I'll post up another version later on Saturday. However, as you point out, once the directory has been created it won't run that line so you shouldn't see it again.

I've missed a few of the other posts regarding thumbs etc.

I'm afraid I don't use thumbs up/down and, as angra has pointed out, they were commented out from his original script because the restored thumbs didn't seem to affect the suggestions; so I figured I'd just stick with SP and wishlists for the time being.

I have a July 25th deadline closing in at work - lots to do and only six weeks left! - so basically I can only really do bug fixes until then. Rbiro is at least the third person to ask for backups of the thumbs though, so I'll certainly look into it and see if it can be made to work in principle.

[agw]

Latest version is now on www.boygenius.co.uk/tivo/ - this one fixes the crash that you get if the module has to create the backups directory.

[sanderton]

Thanks for this agw, it works beautifully and I feel much more secure hacking around on my Tivo with 180 SPs!

I've been looking at the problem of editing the order of season pass priorities quickly and easily. One approach I was considering was to use something very ike your scripts to create a text file of the SPs, load that into a PC side application which could have the necessary user interface to manipulate the priorities (hard to do in a web page via TiVoWeb directly), then load the edited details back in.

Priority would be the only thing changed.

From your experience in creating the backup module, can you see any difficulties in that?

(BTW, searching this forum for "Tivoweb season pass back up" fails to find this thread unles you use the heavy duty search option)

[agw]

As it happens I've been thinking about doing a small utility along those lines that worked on the backup files - mainly to help people out if their backup file is so large that the module runs out of memory when it's trying to do the restore. The utility could be used to break a large backup file up into two or more smaller ones that could each be restored separately, or if one particular pass or series is causing a problem it could be commented out of the backup without the user having to rummage through the file with vi or joe.

Having said that, while it'd be possible to use the backup and restore module and this utility to change priorities, I don't think it'd be for the faint of heart. They could potentially have to delete and restore a lot of season passes. If the restore failed because it ran out of memory then, while it's always been the case so far that the restore can be restarted one way or another and run to completion, it's still a bit more fraught and non-trivial than people might be hoping for. That's not to say I won't put it in there. It just might be a bit more wing-and-a-prayer than most people would be comfortable with.

[agw]

I've implemented a few suggestions regarding getting backup files on and off the TiVo. Previously you had to use FTP to get the files on and off the TiVo - now, after the backup has been made, you're given a link to the file so that you can save it through the browser.

In a similar vein there is a new option to upload a backup file onto the TiVo.

Finally there is a rudimentary file browser on the main backup menu. It allows you to see the directories and files on your TiVo and either download a raw copy of them (as per saving backups, above) or view their content as formatted text.

This version can be downloaded from www.boygenius.co.uk/tivo/

Let me know if anything goes wrong!

[rbiro]

You have a typo in the save link

http://tivo/backup/var/hack/tivoweb-tcl/backups/settings/

You have an extra backup/ at the root.

It should be:

http://tivo/var/hack/tivoweb-tcl/backups/settings/

When I try to save it, I get cannot find file. Clicking on the link opens the file.

A little more testing shows that it works under IE6, but not under Mozilla 1.5

[agw]

I tested it with IE6 and Opera 7, neither of which have a problem with the module.

However I have just tried it with Mozilla Firebird and I get the same result as you - you can click on the link and it'll show the file, but if you right-click the link and try to save the file Mozilla says it has moved.

The /backup at the start of the link is correct, it's meant to be there - it tells TiVoWeb to pass the URL to the backup module for processing. And the link itself is alright, Mozilla is fine with you clicking on it. It just won't save it.

I have no idea why. Perhaps the trailing / on the URL is throwing Mozilla? I had to put it there, otherwise TiVoWeb gives an error 401 if the URL has a full-stop / period at the end of it, i.e. if the URL looked like it finished with a filename that had an extension.

The effect of the trailing / is to make the URL look like it is referring to a directory on the server. Maybe Mozilla isn't too happy about saving a directory, whereas IE and Opera don't care and just trust the server to dish up something they can write to a file? I don't know.

I'll have a go and see if there is some other way of getting past the "." problem without breaking downloads to Mozilla. In the meantime, if you're using Mozilla, you can work around its refusal to save the file by clicking the link and then going into view source (Ctrl+U) and saving the file there.

[Snoopy]

I like the idea of being able to back the file up to my hard drive instead of first having to back it up to the tivo and then pull it off with some FTP software or something. However, I loaded the new version and don't see that option. I see that you can upload from the PC, but not download to the pc?

[agw]

It's a bit rough and ready I'm afraid. It is implemented through a link on the page to the backup file - most browsers allow you to right-click links and save the file that they're pointing to. In this case there are two places where you can get at the file - at the end of the "Create Backup" page and from the "Browse Backup Files" page. There isn't an explicit option to download a file.

I did notice with IE that it insists on saving the file with an HTML extension and it saves it with DOS line-ends rather than UNIX line-ends. However neither of these matter with the backup files, either style of line end is fine and the extension is neither here nor there.

[MisterEd]

I had the same problem until I realized how it worked. Immediately after the backup is complete look at the bottom of the screen you are on. There is a link that says "Copy the backup file by saving this link:" Right click on the file name (./backups/settings) right after that and use SAVE AS.

VERY easy to miss :)


Originally posted by Snoopy
I like the idea of being able to back the file up to my hard drive instead of first having to back it up to the tivo and then pull it off with some FTP software or something. However, I loaded the new version and don't see that option. I see that you can upload from the PC, but not download to the pc?

[MisterEd]

I am unable to download from any link on the "BROWSE BACKUPS" page however it does (seem) to work on the page immediately after download. I get "Cannot open /backups/settings" from any link on the BROWSE page.

Also, on the backup page, when I "SAVE AS" or just click on it, it comes down as "settings.htm". I have attached a copy of what I get. Does it look correct to you?? (see attached file) I'd hate to "rely" on a backup that won't restore. :)

Hmmm, I attached the file and it didn't show up. In any case, IE saves it as "SETTINGS.HTM" and here is a clip of a little bit of it:

# agw TiVo Web backup file - do not edit on or before this line # Layout: 1 # The latest version of backup.itcl can be found somewhere under # http://www.boygenius.co.uk/tivo # This is executable code - TiVo Web runs it to load the data into store. If # you must edit it then please be careful. # The load procedure proc load_file_content {} { # Store the date that the backup was taken load_backup_date "Mon Dec 29 2003 12:44:55" # Load the layout version load_layout_version "1" # Load the TiVo software version running on the TiVo that was backed up load_version {3.1.0-01-1-001}

There are no line breaks in the actual file, just continuous text.



Originally posted by agw
It's a bit rough and ready I'm afraid. It is implemented through a link on the page to the backup file - most browsers allow you to right-click links and save the file that they're pointing to. In this case there are two places where you can get at the file - at the end of the "Create Backup" page and from the "Browse Backup Files" page. There isn't an explicit option to download a file.

I did notice with IE that it insists on saving the file with an HTML extension and it saves it with DOS line-ends rather than UNIX line-ends. However neither of these matter with the backup files, either style of line end is fine and the extension is neither here nor there.

[agw]

The settings.htm thing is what I'd expect. Because it has an extension of HTM when you double-click it in Windows it'll get opened in a browser window. In HTML carriage returns and line feeds are both displayed as space characters - so all the lines look like they run into one another, even though they don't really.

If you change the name to "settings.txt" and then double-click it you'll see it'll open in notepad and then, depending on whether the line-end characters have been converted by IE or not, it'll either show up with a little square where the line ends should be or it'll be split into separate lines. Or alternatively, if the "Browse Backups" thing was working, you could look at the formatted version of the file and see it was all split into separate lines in there.

Either way it's no problem. If you want to put your mind at ease then upload the file into TiVoWeb and run a restore on it. If it has a problem it'll say so, if it's alright then you'll get to the screen where it says what is in the file and you can abandon the restore at that point (click the "Return to Backup Menu" link, this will free up the memory that holds all the information that was read in from the file).

On the failure to open files from "Browse Backups" - the code that shows the file is the same as the code that shows files in the "Create Backup" option, so I wouldn't ordinarily expect a problem. However it might be that the path you're browsing isn't an absolute path, it might just be the word "backups" and be relative to the TiVoWeb directory, instead of "/var/hack/tivoweb-tcl/backups" - i.e., it might not start with a "/". In this case I suspect the code will fail to find the correct directories. I'll have to look into this - in the meantime if you clear out the starting directory (just rub out the directory it suggests) then it'll show you the root of the TiVo's file system and then you should be able to navigate from there to the backup file and see the contents correctly. If that doesn't work then let me know.

[MisterEd]

Thanks for the quick reponse AGW. You are correct. If I actually bowse from ROOT and find the backups directory it works fine (never thought of that). In my case Tivoweb-TCL folder is off ROOT and not in /var/hack (where it should be).

Ed

[agw]

Well version 9 was short-lived :)

Version 10 is now in www.boygenius.co.uk/tivo/ - this version has the following changes:

1. The website mentions the problem and the work-around with Mozilla. I did experiment with changing the "."'s to %2e's but TiVoWeb wasn't fooled and still gave an error.

2. If TivoWeb is using a relative path to the source directory then the backup module figures out the proper absolute path and uses that instead. Please let me know if this causes any problems. I tested it quite heavily (well, as heavily as you can in an afternoon) on my UK 2.5 TiVo, trying various combinations of paths, and the code coped properly with all of them - but I only have the one TiVo and other TiVo versions may behave differently.

3. The code was writing the text "File:<br>" to the console when you browsed a directory. It's unlikely that people will have seen this unless they started TiVoWeb from the command line. I've taken this out.

4. Reworded the link to the backup file on the Create Backup page to make it more obvious that you can save the backup file using the link.

[agw]

Another version is now on www.boygenius.co.uk/tivo/ - the changes are:

1. Fixes a bug in channel remap that could crash the module.

2. Allows *any* channel to be remapped during a restore, not just the channels that couldn't be found on the new TiVo.

This allows people to work around the (hopefully) unlikely situation where Tribune assign the same station ID to two different stations. If this were to happen then when you restored the backup image some season passes might be restored to the wrong channel. With this version you can map the erroneous channel to the correct channel and the season passes will then restore properly. This remapping is just setting up mappings for the restore - it doesn't make any changes to channels on the TiVo itself.

[agw]

Another version has been posted to www.boygenius.co.uk/tivo/ - this version adds experimental support for Australian TiVos and adds the version of the backup module to the comments block at the top of the backup file for diagnostics.

[agw]

I wasn't aware that it didn't work on TiVoWebPlus - I've had emails from people using the module with TWP so I'd assumed it worked alright with it. I've not installed TWP yet though so I can't say that it does for sure. I'll have to get around to installing it at some point.

What problems have you had with it?

[RC3105]

us dtivo 2.5.01, twp rc5, backup module v11

couldn't restore a sp backup 24 hours after a clear & delete. kept coming back with "channel not found" errors

edit: hrm, 12 seems to be working better. course, it's been a couple of days since the c&d and I've allready restored half the sp's by hand...

[agw]

When you say errors, are these local stations that haven't come through yet? What error message are you getting?

The backup module uses the list of stations built by TiVoWeb when it first starts up. If the station list is incomplete when TiVoWeb starts up then the module will think the missing stations no longer exist and will ask you to map them to stations that do exist.

Do other modules in TiVoWeb see the missing stations?

Are all of the stations missing or just some of them?

If the stations are all on the TiVo then if you stop and restart TWP do they appear properly in the backup module?

[RC3105]

shoulda saved the error page as mht (doh!) 26 season passes, only 1 for a local channel and it wasn't the 1 that showed restorable with v11

12 seems to be working a little better several days & many manually restored sp's later...

we should revisit this when the twp final version goes up

[Cspot]

Running (trying to at least) the current version .0012, and getting the following error with 44 sp's:

Taking snapshot of season passes - please wait

INTERNAL SERVER ERROR
--cut here--
action_backup_create_write '' 'set "fname" "/var/hack/tivoweb194/backups/settings";set "submit" "Create";' invalid attribute: 0x40019
while executing
"dbobj $dbobjid attrtype $field"
(procedure "construct_record_content" line 5)
invoked from within
"construct_record_content $series $seriesfields"
("uplevel" body line 13)
invoked from within
"uplevel $body"
invoked from within
"ForeachMfsFileTrans fsid name type $seasonpassdir "" 20 {
set sp [db $db openid $fsid]
set fields [dbobj $sp attrs]
set content [construct..."
(procedure "take_snapshot_sp" line 9)
invoked from within
"take_snapshot_sp 1 1"
(procedure "take_snapshot_for_backup" line 4)
invoked from within
"take_snapshot_for_backup"
(procedure "create_backup" line 18)
invoked from within
"create_backup $chan $fname"
(procedure "::action_backup_create_write" line 9)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[agw]

The latest version is now on www.boygenius.co.uk/tivo - this one fixes the invalid attribute 0x40019 error that cspot was getting.

I know I didn't reply to the posting here but he'd also emailed me and I just replied to the emails - I wasn't ignoring anyone! :-)

[Fofer]

Any reports back from TiVoWebPlus users? Curious to know if its safe to try yet.

[agw]

I had a casual go with TWP about a week ago (created backup file, deleted half-a-dozen passes, restored them - this wouldn't have tested restoring series, just restoring passes) and that worked fine. Certainly no problems picking up the list of stations from TWP and matching them up against the stations in a backup file.

[MisterEd]

Just did a restore (v .0013 and got this:

--cut here--
action_backup_perform_restore '' 'set "submit" "Restore";'
commit failed (0x30012)

while executing
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
# Create the object - this starts off empty...
global global_backup_nowrite
set global_backup_nowrite 0
if {!$global_backup_n..."
(procedure "create_object" line 13)
invoked from within
"create_object $chan $object_id $object"
(procedure "conditional_restore" line 10)
invoked from within
"conditional_restore $chan $fsid Series restore_series created_item"
(procedure "conditional_restore_series" line 4)
invoked from within
"conditional_restore_series $chan [agextract $sp Series]"
(procedure "perform_restore_sps" line 25)
invoked from within
"perform_restore_sps $chan"
(procedure "::action_backup_perform_restore" line 14)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[agw]

When it says "Commit Failed 0x30012" it's usually because the series it is trying to restore already exists. The reason the module is restoring a series that already exists (and therefore doesn't need to be restored) is because the series isn't in the index but it is in the database. This is usually because there's been a recent guided setup or deletion of guide data and the TiVo is still building its indexes when you tried to run the restore.

The indexing can take a day or two - if you leave it for twenty-four hours and run the restore again then it should go through with no problem.

[Snoopy]

What about putting warning on restore screen so that people know they need to wait 24hours for the TiVo to index itself?

Something like:
WAIT... AT LEAST 24 HOURS BEFORE ATTEMPTING TO RESTORE SEASON PASSES TO A CLEAN TiVo


It seems like this is such a regular issue that a warning would help. Also the most likely users of a backup are going to be those that have newly formatted TiVo's or are restoring from a clear and delete everything. It only makes sense that without the warning, people will inevitably try to restore and thereby get that error.

[agw]

I was thinking it'd be nice if the module could tell whether the indexing has been done, preferrably by reading some value off the database or, failing that, perhaps by scanning through the log for tell-tale signs of indexing in progress. It'd be better if it came off the database though. I'll have to look into it at some point.

[MisterEd]

Yup ... that makes sense. Just did a new re-image. Thanks.Originally posted by agw
When it says "Commit Failed 0x30012" it's usually because the series it is trying to restore already exists. The reason the module is restoring a series that already exists (and therefore doesn't need to be restored) is because the series isn't in the index but it is in the database. This is usually because there's been a recent guided setup or deletion of guide data and the TiVo is still building its indexes when you tried to run the restore.

The indexing can take a day or two - if you leave it for twenty-four hours and run the restore again then it should go through with no problem.

[MisterEd]

How do I know indexing is fully complete so I can upload my backup file ?? It always says "indexing" (been 3+ days) and as of now I have a bit less then 2 weeks of program data. (S1 DSR6000 DTivo). It probably would have been easier to just manually reenter the SP's.

[agw]

Sorry for the delay in replying.

3+ days does seem a bit excessive. You can check the /var/log/tvlog log file for the "Sweep done (eSucceeded)" line to see if the indexing has finished - but if the TiVo is telling you that it's still indexing then it probably really is.

Unfortunately the module can't work properly without an index of the series that the passes are for. Each pass refers back to the ID number of the series that it's for and this can't be (easily) figured out without the index.

If it's any consolation, which I imagine it isn't, the TiVo itself can't do much with the series until they've been indexed.

[pbolya]

Great TWP module. I have a problem with using it for HR10-250 HD TiVo's. The HD TiVo's introduced new channel numbering scema where the local ota channels are named channel.version e.g. sat local SD NBC is 3 where ota HD NBC is 3.1 Since the backup.itcl sorts the channels with -int (2 times) and -integer (2 times) the module fails when encounters local ota channels. I have replaced these 4 lsort operators with -real (to sort like floating point numbers) and it works perfectly.

There are several people in several different forums that have this problem and also there is a development effort to get tivowebplus 1.1 out which incorporates this channel numbering and fails the backup.itcl regardless of the Tivo make/model or software version.

Is it possible to fix this and post a new version on www.boygenius.co.uk so people with hr10-250 an/or tivowebplus 1.1 (only pre release version available at this moment) can use this excellent module?

I have tested the above change on HD TiVo's with version 3.1.5, 3.1.5d and 3.1.5e (all possible software combinations).

Thanks,
Peter

[agw]

Version 1.00.0014 on www.boygenius.co.uk/tivo/ implements pbolya's suggestion & should hopefully fix the problem with decimal channel numbers.

[pbolya]

Originally posted by agw
Version 1.00.0014 on www.boygenius.co.uk/tivo/ implements pbolya's suggestion & should hopefully fix the problem with decimal channel numbers. Yes it does. Thanks.

[kb7sei]

I get this while doing a restore. I have attached the backup file.

It was generated on a GXCEBOT DTivo on OS 2.5.2.

I am attempting to restore to an SD-DVR80 on OS 4.0.1b.

It loaded the file just fine, I remapped USA to the proper channel, don't know why it was wrong. I checked everything else, and it seemed fine. Then I tried to run the restore. The 1 wishlist restored fine, but it dies on the SPs. If I can provide any other info, let me know. The log message was in tvlog, and I downloaded and installed the latest version on both machines. They are running TivoWebPlus 1.0-final.

It's not critical that this work. There are only 30 of them, and tivoweb makes it easy. I just thought I'd try it to see how it does. It's entirely possible that my trying to move 2+ OS versions is the cause of the problem.

--------

Restored 1 wishlists

Created new Series (fsid 172023)
Setting Version 140
Setting ServerId ATSH446604
Setting ServerVersion 0
Setting Genre 375
(adding to Genre - 380)
(adding to Genre - 105)
(adding to Genre - 124)
Setting ThumbData 268566912
Setting Title 24
Setting TmsId SH446604

INTERNAL SERVER ERROR
--cut here--
action_backup_perform_restore '' 'set "submit" "Restore";'
commit failed (0x30012)

while executing
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
# Create the object - this starts off empty...
global global_backup_nowrite
set global_backup_nowrite 0
if {!$global_backup_n..."
(procedure "create_object" line 13)
invoked from within
"create_object $chan $object_id $object"
(procedure "conditional_restore" line 10)
invoked from within
"conditional_restore $chan $fsid Series restore_series created_item"
(procedure "conditional_restore_series" line 4)
invoked from within
"conditional_restore_series $chan [agextract $sp Series]"
(procedure "perform_restore_sps" line 25)
invoked from within
"perform_restore_sps $chan"
(procedure "::action_backup_perform_restore" line 14)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[agw]

It's saying the series that it's trying to restore (in this case '24') is already on the Tivo. It checks to make sure that the series doesn't exist before it does the restore and should only restore those series that aren't already there.

Have you waited for the background indexing to finish? The module uses the indexes that this process creates to check to see whether the series, season passes etc. are there before it tries to restore them. Generally when this problem crops up its because the background indexing is still going on when the restore was attempted.

[kb7sei]

The info screen says the indexing is completed, and I checked tvlog to make sure that message you mention in the readme is in there. Is there anything else I should check?

[agw]

No, that sounds alright. If you could set a season pass up for 24 by hand, then take a backup of it and post it here - then I can compare the information for the series from the old and new machines, see if there's anything obvious in there.

[kb7sei]

24 is still a little too far out for me to get a SP for it. I'll update in a couple days when the box lets me do it. :)

[agw]

Okey dokey - in the meantime, the backup file you uploaded appears to be empty, as in it's 0 bytes long :-) Could you re-attach it to that posting again please? Alternatively feel free to email it to me, address is on the website (I'd rather not type it out, I'd like to avoid getting spammed on it for as long as possible).

Also, if you go into TivoWeb and then click on MFS, SERVER, then click stop on the browser (there are hundreds of entries in there), go to the address bar and add /ATSH446604 after the /Server bit - (e.g. http://tivo/mfs/Server/ATSH446604) and go to that. Do you get the series entry for 24? If so, could you cut and paste the content here, or send it to me via email please?

[kb7sei]

Attachment updated.

Reply from TivoWeb:
---

PATH: /Server/ATSH446604
object not found (errNmNameNotFound)

---

[agw]

Could you add a season pass on the new Tivo for any of the series in the original backup file - e.g. ER, Malcolm in the Middle etc. - and post it here please?

[kb7sei]

Attachned. For ER.

[agw]

What happens if you search for /Server/ATSH446604:4:0:0 instead? As in http://tivo/mfs/Server/ATSH446604:4:0:0

[kb7sei]

That worked.

----

Series 171785/10 {
Version = 1
ServerId = ATSH446604
ServerVersion = 0
Genre = 375 380 105 124 141
Title = 24
TmsId = SH446604
IndexPath = /Server/ATSH446604:4:0:0
}

[agw]

Okey dokey, I can see what's going on. I've just got back in, it's very late here, so I won't be fixing it tonight :) I should have something tomorrow - if you don't mind I'll PM you a test version to try? I won't be able to reproduce your problem with my UK Tivo and so I can't test that the fix works myself, I can only test that it doesn't break current behaviour.

[kb7sei]

Sounds good. Thanks for looking into it. I'll be happy to test the fix for you.

[agw]

Version 1.00.0015 of the module is now on the server under www.boygenius.co.uk/tivo/

This version fixes the problem that causes it to miss Series records, which could later cause a commit failed (0x30012) error during restore.

There is also a slight change to the wording in the preamble to the restore to stop implying that the restore trace would be removed in a future version. It won't be :)

[codemonkey2k5]

I am trying to back up my Season pass's with version 1.00.0014 on my series 2 DirecTivo running version 4.0.01. When I click on the backup button, I get the following error. "Cannot open /usr/local/tivoweb-tcl/backups/settings"

Any Ideas?

[kb7sei]

Originally posted by codemonkey2k5
I am trying to back up my Season pass's with version 1.00.0014 on my series 2 DirecTivo running version 4.0.01. When I click on the backup button, I get the following error. "Cannot open /usr/local/tivoweb-tcl/backups/settings"

Any Ideas?


Is that directory writable? Many users of 4.0 on DTivo have a read-only filesystem, with the exception of /var. Telnet in a post the output from "mount".

You should probably use the latest version as well. 1.00.0015 is out and available on the authors website. I believe it fixes a bug I found on 4.x DTivo systems, so you may be having problems with that as well after you get past opening the file.

[eschuckmer]

I am trying to do a restore using version 1.00.0015. But I get the following error.

I have used previous versions of this module successfully. Any idea what the problem might be?

BTW, I made sure the volume was rw accessible. And also tried reinstalling everything to the /var/hack directory. But I ended up with the same result.



action_backup_restore_read '' 'set "fname" "/hack/tivowebplus/backups/settings";set "submit" "Load";'
can't read "arr(1083374)": no such element in array
while executing
"set result $arr($station)"
(procedure "extract_station" line 6)
invoked from within
"extract_station $snapshot ::snapshot_station"
("Series" arm line 11)
invoked from within
"switch -exact $restore_type {
Series {
set restore_fsid [agextract $restore Series]
set snapshot_fsid [agextract $snapshot Serie..."
(procedure "sp_matches" line 4)
invoked from within
"sp_matches $restore_data $snapshot_sp($snapshot_fsid) $restore_type $override_TmsId"
(procedure "find_matching_sp" line 6)
invoked from within
"find_matching_sp $restore_data """
(procedure "automap_restore_sps" line 73)
invoked from within
"$script"
invoked from within
"time {$script}"
(procedure "agtime" line 2)
invoked from within
"agtime {automap_restore_sps}"
(procedure "load_backup" line 32)
invoked from within
"load_backup $chan $fname"
(procedure "::action_backup_restore_read" line 10)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"

[agw]

First off, sorry for the delay in replying!

The function that fails is one that is checking each of the season passes and wishlists in the backup file against the ones that you already have on your Tivo. It takes a snapshot of all of the season passes and the stations that they're on and compares that snapshot against the values from the backup file.

It looks like it's failing when it tries to get the station that a current season pass says it's on. If everything else is working, in particular if Tivoweb's routine to read the list of current stations is working, then that would mean that there's a season pass on your Tivo that is on an invalid station. That wouldn't normally happen - when Tivo delete or change stations from a lineup they usually delete any season passes that were attached to that station.

I think the first thing to do is to figure out which season pass is causing the problem, and see if there is anything special about it. If you take a backup of the Tivo and either send the file to me (address on the web page), or PM it to me, or post it to this group, then I may be able to see which season pass is referring to a station that the backup module can't see. Having an invalid station on a season pass shouldn't (cross fingers) cause a problem when making a backup.

[dgilbert]

I’m trying to restore a backup onto a 6.2 tivo which was created on a 4.0.1b tivo. The wishlists are created correctly and the first season pass is created, then the following error occurs:
action_backup_perform_restore '' 'set "submit" "Restore";'
can't read "TmkEvent::EVT_DATA_CHANGED": no such variable
while executing
"event send $TmkEvent::EVT_DATA_CHANGED $TmkDataChanged::SEASON_PASS $agwMap"
(procedure "perform_restore_sps" line 35)
invoked from within
"perform_restore_sps $chan"
(procedure "::action_backup_perform_restore" line 14)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}" If I repeat the restore, the next sp will be created and then the error. Has anyone else had this problem?

[ThurstonX]

I’m trying to restore a backup onto a 6.2 tivo which was created on a 4.0.1b tivo. The wishlists are created correctly and the first season pass is created, then the following error occurs:
action_backup_perform_restore '' 'set "submit" "Restore";'
can't read "TmkEvent::EVT_DATA_CHANGED": no such variable
while executing
"event send $TmkEvent::EVT_DATA_CHANGED $TmkDataChanged::SEASON_PASS $agwMap"
(procedure "perform_restore_sps" line 35)
invoked from within
"perform_restore_sps $chan"
(procedure "::action_backup_perform_restore" line 14)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}" If I repeat the restore, the next sp will be created and then the error. Has anyone else had this problem? Yeah, I had the same prob. Per the general suggestion on the "other" forum for 6.2 support of dealing with tcl scripts, I commented out all lines (may have been only one) that contain "TmkEvent" re-ran it and the mod worked fine.

The following may or may not be relevant:
After inserting all my backed up TMFs, 6.2 doesn't recognize that they are there to the extent that a season pass will record the same show again as if it weren't there. All inserted shows play fine. Haven't asked about this yet, but I imagine it's something to do with those missing "TmkEvent" calls ms_ftp (again, the only way to get it to work), and *not* a prob with this mod.

Just had to throw it out there ;-)

HTH.

Hmmmm, better change that sig :-)

[dgilbert]

Yeah, I had the same prob. Per the general suggestion on the "other" forum for 6.2 support of dealing with tcl scripts, I commented out all lines (may have been only one) that contain "TmkEvent" re-ran it and the mod worked fine.

ThurstonX and agw,
Thanks for your quick responses. I commented out the event call and the backup module worked perfectly with only one minor issue. Without the event call, the tivo does not seem to process all the new SPs (the To Do List remained empty). I'm not sure if this would have resolved itself eventually, but I did a re-order of the SPs (moved one item up and down in priority) and this forced the tivo to reanalyze all the SPs. After that, the To Do List was filled and everything was fine. Thanks again!

[agw]

I've uploaded a test version of the backup module that ignores the event errors to

http://www.boygenius.co.uk/files/backup-15a.zip

It should behave as per normal with old Tivo's and still work with the new ones that don't like the event call. If someone with a 6.2 Tivo could give it a go and check that it works then I'd be grateful.

This version addresses another problem that someone wrote to me about a while back. Season passes with TuningPreferences attached couldn't be restored. As far as I can tell these record your choice concerning which language a program is recorded in. TuningPreferences are keys to other items in the database and, because previous versions of the module didn't know what the key pointed to, they would have refused to restore passes that had TuningPreferences (the module doesn't restore season passes that have keys to items it doesn't know about).

Unfortunately the UK is unlikely to ever have TuningPreferences. I don't want to write code to save and restore TuningPreferences if I can't ever test it. Given this, and given that they seem to be pretty much optional, I just remove the TuningPreferences from the season pass when restoring it.

[tivoupgrade]

I've uploaded a test version of the backup module that ignores the event errors to

http://www.boygenius.co.uk/files/backup-15a.zip

It should behave as per normal with old Tivo's and still work with the new ones that don't like the event call. If someone with a 6.2 Tivo could give it a go and check that it works then I'd be grateful.

This version addresses another problem that someone wrote to me about a while back. Season passes with TuningPreferences attached couldn't be restored. As far as I can tell these record your choice concerning which language a program is recorded in. TuningPreferences are keys to other items in the database and, because previous versions of the module didn't know what the key pointed to, they would have refused to restore passes that had TuningPreferences (the module doesn't restore season passes that have keys to items it doesn't know about).

Unfortunately the UK is unlikely to ever have TuningPreferences. I don't want to write code to save and restore TuningPreferences if I can't ever test it. Given this, and given that they seem to be pretty much optional, I just remove the TuningPreferences from the season pass when restoring it.

It seems to work fine on 6.2. Can you please elaborate on the code you added/modified to handle the events (I am looking through the mod right now) as I'm interested in making similar modifications to other modules within TiVoWebPlus to ignore the EVT_DATA_CHANGED issues on 6.2.

Thx

[Fozzie]

I'm running v1.00.0015 on my UK SA Tivo (S/W 2.5.5) and often seem to be getting a reboot when backing up season passes. Once the Tivo has rebooted, the backup works fine. If I then leave it a while (a couple of weeks say), Tivo reboots the next time I try and backup. I don't have any other problems with Tivo rebooting. I also don't recall having this problem previously.

Is anyone else having this problem or have any ideas what might be causing it?

Thanks.

[agw]

It seems to work fine on 6.2. Can you please elaborate on the code you added/modified to handle the events (I am looking through the mod right now) as I'm interested in making similar modifications to other modules within TiVoWebPlus to ignore the EVT_DATA_CHANGED issues on 6.2.

Thx

Thanks for testing it - I'll update the version number and put it onto the site within the next couple of days.

To stop the event error I just made the call within a catch block to suppress the error on 6.2. So the line

event send $TmkEvent::EVT_DATA_CHANGED $TmkDataChanged::SEASON_PASS $agwMapbecomes


catch {event send $TmkEvent::EVT_DATA_CHANGED $TmkDataChanged::SEASON_PASS $agwMap}



I'm running v1.00.0015 on my UK SA Tivo (S/W 2.5.5) and often seem to be getting a reboot when backing up season passes. Once the Tivo has rebooted, the backup works fine. If I then leave it a while (a couple of weeks say), Tivo reboots the next time I try and backup. I don't have any other problems with Tivo rebooting. I also don't recall having this problem previously.

Tivoweb is probably running out of memory. When it runs out of memory it reboots the Tivo. The backup module is a memory hog, it sets up a whole pile of arrays in store when it does a backup or a restore and those eat up memory. They get released when it's finished doing the backup or restore but in this case it doesn't look like there's enough store left to get that far. The reboot won't hurt a restore, you can just restart it and it'll pick up where it left off, but it'll stop the backup from completing.

You can work around it by increasing the amount of store that Tivoweb allocates for itself. You can do this by editing tivoweb in the tivoweb directory and changing the line export TIVOSH_POOLSIZE=2916352 to export TIVOSH_POOLSIZE=3244032, and then restarting tivoweb.

[Fozzie]

Tivoweb is probably running out of memory. When it runs out of memory it reboots the Tivo. The backup module is a memory hog, it sets up a whole pile of arrays in store when it does a backup or a restore and those eat up memory. They get released when it's finished doing the backup or restore but in this case it doesn't look like there's enough store left to get that far. The reboot won't hurt a restore, you can just restart it and it'll pick up where it left off, but it'll stop the backup from completing.

You can work around it by increasing the amount of store that Tivoweb allocates for itself. You can do this by editing tivoweb in the tivoweb directory and changing the line export TIVOSH_POOLSIZE=2916352 to export TIVOSH_POOLSIZE=3244032, and then restarting tivoweb.
I thought that may be the case. Thanks for the tip; I'll make the changes and see how things go.

[agw]

New version on the website - www.boygenius.co.uk/tivo/ - this one suppresses the event handling error under 6.2 and ignores TuningPreferences when restoring season passes.

[davidlallen]

I have downloaded backup 1.00.0016 and I am planning to include it in tivowebplus 1.2.1 when it is released. However, I have never used backup before, and when I test it on my TCD 540 with software version 7.1, it does not appear to work:

Taking snapshot of season passes - please wait
INTERNAL SERVER ERROR
--cut here--
action_backup_create_write '' 'set "fname" "/var/davea/tivowebplus/
backups/settings";set "submit" "Create";'
can't scan path (TV_NM_NAME_NOT_FOUND)
while executing
"mfs scan $dirName -start $prefix -count $count"
invoked from within
"if { [catch {mfs scan $dirName -start $prefix -count $count} batch] } {
global errorCode errorInfo
if { $errorCode == "errNmNameNotFound" ..."
("uplevel" body line 2)
[...]
"take_snapshot_theme 1"
(procedure "take_snapshot_for_backup" line 3)
invoked from within
"take_snapshot_for_backup"
(procedure "create_backup" line 18)
invoked from within
"create_backup $chan $fname"
(procedure "::action_backup_create_write" line 9)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"


Any thoughts?

[agw]

The line that it's going over on isn't in my code, it's in something I'm calling. Unfortunately the actual call is hidden somewhere in the "[...]" bit in the dump, but my guess would be this line, which is called in the last of my functions listed in the dump:

ForeachMfsFileTrans fsid name type "/Theme" "" 20

I'm guessing that in version 7.1 Tivo changed the name of the wishlists table in MFS to something else.

This line was copied from a function in ui.itcl called action_wishlists, which presents a table of wishlists to the user. Did ui.itcl have to change to cope with version 7.1 and, if so, what did it change to?

[davidlallen]

I'm guessing that in version 7.1 Tivo changed the name of the wishlists table in MFS to something else.

This line was copied from a function in ui.itcl called action_wishlists, which presents a table of wishlists to the user. Did ui.itcl have to change to cope with version 7.1 and, if so, what did it change to?

Well, the good news is the latest ui.itcl has the same Foreach.../Theme line as your code. The bad news is, that code doesn't work either in 7.1. The wishlist button fails with a similar traceback; it starts with TV_NM_NAME_NOT_FOUND and has the Foreach.../Theme line in the middle. I have searched over in ddb for TV_NM_NAME_NOT_FOUND and the only reference is to a bug in index.itcl which was for a different problem.

I don't know quite enough about mfs to find out what the corresponding name is now; can you help, or point me at specific links where I can learn more?

[agw]

Well, the good news is the latest ui.itcl has the same Foreach.../Theme line as your code. The bad news is, that code doesn't work either in 7.1.
I don't know quite enough about mfs to find out what the corresponding name is now; can you help, or point me at specific links where I can learn more?

I'm not an authority on it I'm afraid, but if they've just renamed the folder then it shouldn't be too bad. They've done this in the past, in particular between version 2 and version 3.

You could check for a simple rename of the folder and fields by running TivoWeb (Plus) and picking the MFS option off the main menu. You won't have a Theme entry there but you'll see lots of others. Click on ones that look hopeful (if there's one called "Wishlist" then definitely give it a go) and see if your wishlists are listed in any of them :)

In my Theme folder I have an entry for each wishlist which is roughly named after the wishlist's topic. When you click on each entry you get fields like Version, Actor & ActorOp (for actor wishlists), KeywordPhrase & KeywordPhraseOp (for keyword wishlists), ThemeType, Name, IndexPath and IndexUsedBy & SeasonPass (if there's an SP for the wishlist).

If it's just the case that things are laid out in roughly the same way as previous Tivo's, but they have different names under 7.1, then it won't be too bad to cope with it in the module once the new names have been figured out. On the other hand if they've scrapped the old scheme and completely reworked it then it could be a bit of a pain. What new features were introduced in 7.1? Were any of them to do with wishlists?

[davidlallen]

If it's just the case that things are laid out in roughly the same way as previous Tivo's, but they have different names under 7.1, then it won't be too bad to cope with it in the module once the new names have been figured out.

I figured it out with some help from rbautch. (His suggestions were similar to yours but I happened to see his first.) The problem appears to be that I had never created any wishlist. I created a wishlist entry, and both the twp wishlist and the backup function work fine. Then I deleted the wishlist entry, and they both still work fine.

I am guessing -- but now I can't prove it -- that the Themes mfs directory must have not existed until I created my first wishlist.

Anyway, my problem is solved, but I will put an entry into the twp faq.

[rbautch]

Perhaps a check for Theme can be added so this doesn't cause trouble for people who haven't set up any wishlists.

[agw]

New version on http://www.boygenius.co.uk/tivo/

a) Module is now more forgiving if you don't have any wishlists.

b) Some users were running Tivoweb in such a way that they never saw the main menu, and so when they got into the backup menu they weren't told what it was going to backup until they ran it. The backup menu text now mentions season passes & wishlists.

[rbautch]

I would like to set up a cron job to make regular backups of my season passes and wishlists. Initially I thought it would be pretty easy to create a tcl script to do this by copying/pasting the appropriate code from the backup.itcl module. After examining the code in backup.itcl, I quickly realized I was in over my head. Can someone point me in the right direction, or help me create this script? TIA...

[Fofer]

Great idea, rbautch! Wow, I love how you push the envelope.

Someone please help rbautch! Hopefully the fruits of his labor will make their way into the next version of his script. (http://www.tivocommunity.com/tivo-vb/showthread.php?t=250680&page=1&pp=50)

:o :D :)

[agw]

I would like to set up a cron job to make regular backups of my season passes and wishlists. Initially I thought it would be pretty easy to create a tcl script to do this by extracting the appropriate code from the backup.itcl module. After examining the code in backup.itcl, I quickly realized I was in over my head. Can someone point me in the right direction, or help me create this script? TIA...

The backup script relies on various bits of tivoweb - if you didn't want to duplicate code then one way would be to get enough of the tivoweb environment up and running and then just call the function in the backup script that does all the work.

I've attached a quickie script that does this - if you unzip it and put it into your tivoweb (plus?) directory then you can run it using:
tivosh ./write_backup.tcl full-path-to-your-backup-file-here
It works for me but it's not exactly been exhaustively tested :) You may need to set up a big TIVOSH_POOLSIZE before running it, especially if the backup reboots your Tivo:
export TIVOSH_POOLSIZE=3244032
tivosh write_backup.tcl /var/hack/tivoweb-tcl/backups/autobackup
One final note - the script assumes that it's running in the tivoweb folder, i.e. that the backup module is pathed as "modules/backup.itcl", and the script puts up some gibberish output on stdout - you may want to redirect this.

Edited 13/8/05: uploaded new version (850 bytes) of write_backup.zip to work with TWP

[rbautch]

Thank you so much! I'll try it out this weekend.

[rbautch]

When it loads the index module, I'm getting this error:
can't read "::version": no such variable
while executing
"if {$::version >= 6} {
set guideindexdir "/GuideIndexV3"
} elseif {$::version >= 3} {
set guideindexdir "/GuideIndexV2"
} else { I'm using the latest index.itcl file that's in the 1.2.1 release. Any ideas why the ::version variable is not being set? Is it set in some other module that we're not loading here?

If I manually set the ::version variable, it gets much further loading things up, and then finally errors with this: Taking snapshot of season passes - please wait<br>
no such object: CONFLICT err=0x30007
while executing
"dbobj $sub fsid"
(procedure "construct_record_content" line 19)
invoked from within
"construct_record_content $sp $fields"
("uplevel" body line 4)
invoked from within
"uplevel $body"
invoked from within
"ForeachMfsFileTrans fsid name type $seasonpassdir "" 20 {
set sp [db $db openid $fsid]
set fields [dbobj $sp attrs]
set content [construct..."
(procedure "take_snapshot_sp" line 9)
invoked from within
"take_snapshot_sp 1 1"
(procedure "take_snapshot_for_backup" line 4)
invoked from within
"take_snapshot_for_backup"
(procedure "create_backup" line 18)
invoked from within
"create_backup stdout $write_backup_filename"
(file "write_backup_modded.tcl" line 82) Just to be sure, I tried making a backup through TWP and it worked. Then I tried the script again with TWP running and I got the following error, and then it rebooted. Dumping mempool to /tmp/BlockFailure.15035

To view the blocks, run:
$TIVO_ROOT/devbin/poolview.tcl <app-with-symbols> /tmp/BlockFailure.15035

In the UI that comes up, find your block by address (see above)
This will help you identify the type and ownership of the blocks.

Tmk Assertion Failure:
DumpArenaAndBlocksAndDie, line 1472 ()
Tmk Fatal Error: Thread tivosh <15035> strayed!
Paste the following into a shell to get a backtrace...

bt -t /tvbin/tivoapp <<END_OF_BT
read 0x2aaa8000 /lib/ld.so.1
read 0x2ab04000 /lib/libutil.so.1
read 0x2ab48000 /lib/libdl.so.2
read 0x2ab8c000 /lib/libpthread.so.0
read 0x2abe8000 /lib/libm.so.6
read 0x2acb0000 /lib/libc.so.6
0x013b3e3c 0x013b3c90 0x013b3f84 0x00f58234 0x00f9312c 0x00fe4088 0x00fe58d4
0x00fbc168 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fecc98 0x00fb7a28
0x00f8eb18 0x00f93f84 0x00fb7a28 0x00f8eb18 0x00f96cac 0x00fb7a28 0x00f8eb18
0x00fecc98 0x00fb7a28 0x00f8eb18 0x00f5b310 0x00f593f4 0x00fb7a28 0x00f8eb18
0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644
0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28
0x00f8eb18 0x00fd99ec 0x00fdd23c 0x00f58150 0x00612c48 0x00403090 0x2acc13fc
END_OF_BT

Tmk Fatal Error: Thread tivosh <15035>: assertion failure I assume this was because of the poolsize.

[agw]

When it loads the index module, I'm getting this error:
can't read "::version": no such variable

OK, I've fixed this - I'm afraid I use vanilla Tivoweb so I didn't hit the problem with the new version variable. I've updated the ORIGINAL attachment rather than reposting it - I've gotten tied into knots with different versions of things in different attachments before.

This version also sets dtivo, which caused problems with the version of index.itcl that I have in my TWP (which is somewhat older than yours, I'll get the latest version in the morning).

As for the second bit...

no such object: CONFLICT err=0x30007
while executing
"dbobj $sub fsid"
(procedure "construct_record_content" line 19)
invoked from within
"construct_record_content $sp $fields"
("uplevel" body line 4)
invoked from within
"uplevel $body"
invoked from within
"ForeachMfsFileTrans fsid name type $seasonpassdir "" 20 {


The seasonpass directory in MFS is one of the things that got renamed in version 3. How did you set $::version? It should have been set with the same line that's in httpd-tt. $::seasonpassdir is set in index.itcl based on the value of $::version (in TWP, $::version3 in TW), so if you got that wrong then it'd try to read from the wrong MFS folder.

So in short - try the new version, reattached a couple of postings back :)

[agw]

Jeez man, am I just seeing things or are you changing your posting quicker than I can reply to it? :)

Then I tried the script again with TWP running and I got the following error, and then it rebooted. Dumping mempool to /tmp/BlockFailure.15035

To view the blocks, run:
$TIVO_ROOT/devbin/poolview.tcl <app-with-symbols> /tmp/BlockFailure.15035

In the UI that comes up, find your block by address (see above)
This will help you identify the type and ownership of the blocks.

Tmk Assertion Failure:
DumpArenaAndBlocksAndDie, line 1472 ()
Tmk Fatal Error: Thread tivosh <15035> strayed!
Paste the following into a shell to get a backtrace...

bt -t /tvbin/tivoapp <<END_OF_BT
read 0x2aaa8000 /lib/ld.so.1
read 0x2ab04000 /lib/libutil.so.1
read 0x2ab48000 /lib/libdl.so.2
read 0x2ab8c000 /lib/libpthread.so.0
read 0x2abe8000 /lib/libm.so.6
read 0x2acb0000 /lib/libc.so.6
0x013b3e3c 0x013b3c90 0x013b3f84 0x00f58234 0x00f9312c 0x00fe4088 0x00fe58d4
0x00fbc168 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fecc98 0x00fb7a28
0x00f8eb18 0x00f93f84 0x00fb7a28 0x00f8eb18 0x00f96cac 0x00fb7a28 0x00f8eb18
0x00fecc98 0x00fb7a28 0x00f8eb18 0x00f5b310 0x00f593f4 0x00fb7a28 0x00f8eb18
0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644
0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28 0x00f8eb18 0x00fed644 0x00fb7a28
0x00f8eb18 0x00fd99ec 0x00fdd23c 0x00f58150 0x00612c48 0x00403090 0x2acc13fc
END_OF_BT

Tmk Fatal Error: Thread tivosh <15035>: assertion failure I assume this was because of the poolsize.

I don't know. However I do know that if you suck up all the memory in a TCL script then tivosh reboots the machine, that the tivoweb shell script sets TIVOSH_POOLSIZE to a large number before it starts the tivoweb TCL scripts and that by default that variable is empty if you don't set it... so what I would do would be to put the write_backup.tcl in a shell script and set TIVOSH_POOLSIZE to the same value that you have got for tivoweb directly before running write_backup.tcl - e.g.:

Create write_backup.sh and set the content to:

export TIVOSH_POOSIZE=3244032
tivosh write_backup.tcl $1


Then do:

chmod 755 write_backup.sh

so you can execute it and then finally run it with

./write_backup.sh /var/hack/tivowebplus/backups/autobackup

[rbautch]

My bad. Didn't think you were online. The larger poolsize fixed it, and now successfully makes a backup. Thanks so much for helping me with this! :up:

[Fofer]

rbautch, how about a cron job that also makes regular backups of your Channel Prefs? (Channels Received and Favorite Channels.) That'd be cool too. :)

[rbautch]

Not a bad idea, but channelprefs aren't updated nearly as often as season passes and wishlists. I've had the same channelprefs for 3 years. It's easy enough to make a backup with TWP. Still, I may play around with it now that agw has shown me the light to starting TWP processes with a tcl script. Thanks again, agw.

[agw]

No worries, glad it works - good luck with the scripts! :-)

If you're planning on doing a few of these scripts then it might be an idea to split out the common initialisation code from httpd-tt.tcl into standalone scripts that get sucked into the things that need them (httpd-tt plus your cron scripts) with TCL source statements. That way you don't end up duplicating code and your cron scripts should become smaller, easier to maintain and hopefully more straight-forward to write.

[rbautch]

Great idea...thanks again!

[rbautch]

Tivoweb is probably running out of memory. When it runs out of memory it reboots the Tivo. The backup module is a memory hog, it sets up a whole pile of arrays in store when it does a backup or a restore and those eat up memory. They get released when it's finished doing the backup or restore but in this case it doesn't look like there's enough store left to get that far. The reboot won't hurt a restore, you can just restart it and it'll pick up where it left off, but it'll stop the backup from completing.

You can work around it by increasing the amount of store that Tivoweb allocates for itself. You can do this by editing tivoweb in the tivoweb directory and changing the line export TIVOSH_POOLSIZE=2916352 to export TIVOSH_POOLSIZE=3244032, and then restarting tivoweb. I had several restores fail and then cause a reboot. I fixed it by increasing the poolsize. Now that backup has been fully integrated into TWP, does it make sense to increase the poolsize in tivoweb on the official release?

By the way, this is by far the most useful module. I just bought 3 new Tivos, and backup saved me hours of data entry (I had 67 season passes). Keep up the great work!

[adamski]

I'm using the latest version of season pass backup/restore (backup.itcl module version 1.00.0017) on a uk TiVo 2.5.5 lba48 200gb

The story is, I made the backup shortly before the previous hd went south, the full tivo.bak I made, simply would not load, so I had to take a copy off my second TiVo (the freeview one) and load it up on my Sky one. I cleared everything down and tried to restore my old season passes. I get the following error:


INTERNAL SERVER ERROR
--cut here--
action_backup_restore_read '' 'set "fname" "/var/hack/tivowebplus/backups/settings";set "submit" "Load";'
can't open object (errDbNotFound)

while executing
"db $::db openid $fsid"
("uplevel" body line 2)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set series_data [db $::db openid $fsid]
set series_fields [dbobj $series_data attrs]
set content [con..."
(procedure "get_series_fsid" line 25)
invoked from within
"get_series_fsid $restore_data"
(procedure "automap_restore_series" line 8)
invoked from within
"$script"
invoked from within
"time {$script}"
(procedure "agtime" line 2)
invoked from within
"agtime {automap_restore_series}"
(procedure "load_backup" line 28)
invoked from within
"load_backup $chan $fname"
(procedure "::action_backup_restore_read" line 10)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--


I contains about 104 sp's, so I really don't want to reload by hand!!

Can you help?

[gfb107]

**** Update: I was able to make a backup by reverting to version 1.00.0015 ****
I'm getting the same error messages trying to create a backup on by DSR6000R (DirecTiVo S1) running OS 3.1.0c2-01-1-001.

I'm using TiVoWebPLus 1.2.1 and backup.itcl 1.00.0017
Help please.

Here's the errors:Taking snapshot of season passes - please wait

INTERNAL SERVER ERROR
--cut here--
action_backup_create_write '' 'set "fname" "/var/hack/tivowebplus/backups/settings";set "submit" "Create";'
can't open object (errDbNotFound)

while executing
"db $db openid $stationfsid"
("uplevel" body line 2)
invoked from within
"uplevel $body"
invoked from within
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {
set station [db $db openid $stationfsid]
set fields [dbobj $station attrs]
set content [construct_record_content ..."
(procedure "take_snapshot_station" line 8)
invoked from within
"take_snapshot_station 1"
(procedure "take_snapshot_for_backup" line 2)
invoked from within
"take_snapshot_for_backup"
(procedure "create_backup" line 18)
invoked from within
"create_backup $chan $fname"
(procedure "::action_backup_create_write" line 9)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[lee espinoza]

I am getting this when I try to upload a backup setting.htm from my old tivo

I get The connection was reset


The connection to the server was reset while the page was loading.








* The site could be temporarily unavailable or too busy. Try again in a few
moments.

* If you are unable to load any pages, check your computer's network
connection.

* If your computer or network is protected by a firewall or proxy, make sure
that Firefox is permitted to access the Web.


from Firefoxs


what should I do?
SD-DVR40 running OS 6.2.01.2.351 with TiVoWebPLus 1.2.1 and backup.itcl 1.00.0017

[boody]

My HR10-250 crashes when I try to do a restore (load) on a backup created on the same unit. Any ideas?

[lee espinoza]

The backup module is a memory hog. If Tivoweb runs out of memory, which is can do during a large backup or restore, then it will reboot the Tivo. This will not usually harm a restore - you can just load the backup file and the restore will skip those season passes it restored before the reboot - but it will stop you from taking a full backup.

You can allocate more memory to Tivoweb to work around this problem. Edit the tivoweb file in the Tivoweb directory and change the line:
export TIVOSH_POOLSIZE=2916352

to:
export TIVOSH_POOLSIZE=3244032

Then stop Tivoweb and restart it.

[rbautch]

The backup module is a memory hog. If Tivoweb runs out of memory, which is can do during a large backup or restore, then it will reboot the Tivo. This will not usually harm a restore - you can just load the backup file and the restore will skip those season passes it restored before the reboot - but it will stop you from taking a full backup.

You can allocate more memory to Tivoweb to work around this problem. Edit the tivoweb file in the Tivoweb directory and change the line:
export TIVOSH_POOLSIZE=2916352

to:
export TIVOSH_POOLSIZE=3244032

Then stop Tivoweb and restart it. If you used the Zipper or my enhancement script to install TWP, the poolsize is already increased.

[rbautch]

Even with TIVOSH_POOLSIZE=3244032, my HR10-250 reboots everytime a backup is made. My other S2 tivos run it just fine. Can/should the memory be increased further?

[Crispin]

Hi, I have a problem trying to restore a backup onto my UK Tivo, I did the backup before I moved house when I was using NTL digital cable, and now I'm on freeview I want to restore it as best I can:

action_backup_restore_read '' 'set "fname" "/var/hack/tivoweb-tcl/backups/settings-2006-04-21";set "submit" "Load";'
can't read "arr(3081)": no such element in array
while executing
"set result $arr($station)"
(procedure "extract_station" line 6)
invoked from within
"extract_station $snapshot ::snapshot_station"
("Series" arm line 11)
invoked from within
"switch -exact $restore_type {
Series {
set restore_fsid [agextract $restore Series]
set snapshot_fsid [agextract $snapshot Serie..."
(procedure "sp_matches" line 4)
invoked from within
"sp_matches $restore_data $snapshot_sp($snapshot_fsid) $restore_type $override_TmsId"
(procedure "find_matching_sp" line 6)
invoked from within
"find_matching_sp $restore_data """
(procedure "automap_restore_sps" line 73)
invoked from within
"$script"
invoked from within
"time {$script}"
(procedure "agtime" line 2)
invoked from within
"agtime {automap_restore_sps}"
(procedure "load_backup" line 32)
invoked from within
"load_backup $chan $fname"
(procedure "::action_backup_restore_read" line 10)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"

Any ideas ?

[agw]

Right, sorry I've not been replying to messages but for whatever reason the board wasn't sending me emails when new messages were being posted. I thought things were a bit quiet :)

Hi, I have a problem trying to restore a backup onto my UK Tivo, I did the backup before I moved house when I was using NTL digital cable, and now I'm on freeview I want to restore it as best I can:

action_backup_restore_read '' 'set "fname" "/var/hack/tivoweb-tcl/backups/settings-2006-04-21";set "submit" "Load";'
can't read "arr(3081)": no such element in array
while executing
"set result $arr($station)"
(procedure "extract_station" line 6)
invoked from within
"extract_station $snapshot ::snapshot_station"



I've downloaded your backup file and loaded it onto my tivo without any problems, so on the face of it there doesn't seem to be anything intrinsic in the file that will cause this error.

There is a known problem with tivoweb in that it only builds its list of television stations when it first starts up. Once that list is built it is never changed. This is usually fine, but if Tivoweb was started when the tivo had an incomplete list of stations - for example, right after a guided setup - then the stations list will be incomplete.

It's possible that the problem you hit was due to having a flakey list of television stations, and that restarting Tivoweb will clear it.

However your message was unfortunately posted about a fortnight before I happened upon it, so I'm guessing you've either cleared the problem yourself or worked around it?

[Crispin]

I've downloaded your backup file and loaded it onto my tivo without any problems, so on the face of it there doesn't seem to be anything intrinsic in the file that will cause this error.

There is a known problem with tivoweb in that it only builds its list of television stations when it first starts up. Once that list is built it is never changed. This is usually fine, but if Tivoweb was started when the tivo had an incomplete list of stations - for example, right after a guided setup - then the stations list will be incomplete.

It's possible that the problem you hit was due to having a flakey list of television stations, and that restarting Tivoweb will clear it.


Sadly, restarting TivoWeb was the first thing I tried, and it didn't help then, and having just tried again it still fails :(

When I first tried to load the backup file, tivoweb did have an old list of channels (the normal 5 analogue ones here in the UK), but the backup file loaded fine. When I realised the problem I restarted tivoweb, so it now has the correct list of FreeView channels, but I get the error everytime I try to load it.

I haven't managed to work around the problem yet, so any help will be greatly appreciated!

[agw]

Sadly, restarting TivoWeb was the first thing I tried, and it didn't help then, and having just tried again it still fails :(

When I first tried to load the backup file, tivoweb did have an old list of channels (the normal 5 analogue ones here in the UK), but the backup file loaded fine. When I realised the problem I restarted tivoweb, so it now has the correct list of FreeView channels, but I get the error everytime I try to load it.

I haven't managed to work around the problem yet, so any help will be greatly appreciated!

Ok, the problem isn't obvious straight away. I've added a couple of traces to the module to show the data it's working with at the time of the crash. The test module should be attached to this posting. Could you download it, stick it on the Tivo, restart Tivoweb and then see what happens when you load the backup file please?

The version number for the module should show as 1.0.0.0017-test if you've loaded it correctly.

When you load the backup file the normal table showing the results of the load will be prefaced with lots of lines line this:


data is Version int 45 MaxRecordings int 5 Series fsid 563685 Station fsid 705 Priority int 68 agwSPType string Series
setting result to arr (705)
result is agwChannelNum int 102 TmsId string 17154 Name string {{BBC2 British Brdcstg Corp.}} CallSign string BBC2 agwMap fsid {} agwWasMissing bool 1

If you could copy those lines, up to and including the first few lines of the crash message, and paste them here then I'd appreciate it.

I'm afraid there may be a little toing and froing with trace and test versions - if you'd rather mail me at the address on http://www.boygenius.co.uk/tivo/ then it might save clogging up this thread with test cruft :)

[Crispin]

As a follow up (in case anyone else has the same problem), it turned out I had a season pass that was mapped to channel that wasn't in the list of channels I currently receive (although it was working perfectly!). After I deleted that season pass the restore worked perfectly.

[tivo4ever]

Just thought I'd add a note of thanks for this thread. I tried restoring my season passess after a clear and delete and it kept rebooting. I updated to:

export TIVOSH_POOLSIZE=3244032

and tada....it works...


thanks!

[smokie]

I *think* I have the same problem as Crispin, having just had to reload my TiVo from a 3 year old image (but managing to backup the SPs before putting the new disk in). I definitely have a different TV package from then.

How can I edit the backup file before running the restore? Not a biggie but it would be nice to fix...

[speed_phreak]

Agw,

First I appreciate your efforts, this backup script is a very important piece to the hacking of a tivo ;)

I am having issues backing up season passes with TWP 2.0RC1 and the version of your script that is included (1.00.0018 ).

It backs up wishlists, but not my SP's. I have over 20 SPs. It says its taking a snapshot of my season passes and I almost believe it, but they don't get written to the backup file. The web interface says this:

Copy existing backup file to '/TivoWebPlus/backups/settings.old' Done
Overwrite backup file '/TivoWebPlus/backups/settings' Done
Write header Done
Store layout version number 1 Done
Store TiVo version number 6.2-01-2-301 Done
Store details for 0 channels (for reference only, may contain duplicates) Done
Store details for 4 wishlists Done
Store details for 0 season passes Done

The backup file says this:

# agw TiVo Web backup file - do not edit on or before this line # Layout: 1 # Created by module version 1.00.0018 # The latest version of backup.itcl can be found somewhere under # http://www.boygenius.co.uk/tivo # This is executable code - TiVo Web runs it to load the data into store. If # you must edit it then please be careful. # The load procedure proc load_file_content {} { # Store the date that the backup was taken load_backup_date "Tue Jan 23 2007 18:52:45" # Load the layout version load_layout_version "1" # Load the TiVo software version running on the TiVo that was backed up load_version {6.2-01-2-301} # These are the channels used by the season passes in the backup # The list is for reference only and doesn't reflect the full # list of channels in the lineup at the time of the backup # Load the wishlists on the TiVo at the time of backup load_theme {50261} {Version int 2 GenreFilterPath int {1 368} ThemeType int 5 Name string Movies/Action/Adventure IndexPath string /Theme/Movies_Action_Advent~50261 IndexUsedBy fsid 50264 SeasonPass fsid 50264} load_theme {50345} {Version int 2 GenreFilterPath int {1 371} ThemeType int 5 Name string Movies/Comedy IndexPath string /Theme/Movies_Comedy~50345 IndexUsedBy fsid 50346 SeasonPass fsid 50346} load_theme {50423} {Version int 2 GenreFilterPath int {1 383} ThemeType int 5 Name string Movies/Romance IndexPath string /Theme/Movies_Romance~50423 IndexUsedBy fsid 50424 SeasonPass fsid 50424} load_theme {57169} {Version int 2 GenreFilterPath int {9 382} ThemeType int 5 Name string Themes/Reality-based IndexPath string /Theme/Themes_Reality-based~57169} # Load the season passes on the TiVo at the time of backup }

I would be happy to gather any additional information I can to help you troubleshoot this...

Thanks in Advance!!!

[agw]

I've just downloaded the RC1 version and I can reproduce the problem here on my UK Tivo. I'll have a look into it. It may take a while, work is pretty full-on at the moment.

[agw]

I *think* I have the same problem as Crispin, having just had to reload my TiVo from a 3 year old image (but managing to backup the SPs before putting the new disk in). I definitely have a different TV package from then.

How can I edit the backup file before running the restore? Not a biggie but it would be nice to fix...

Sorry, I didn't get any notifications that there were new messages in here - only saw this after a PM about another problem.

Crispin's problem was that the Tivo itself had a season pass on it that wasn't mapped to any channel, not that the backup was bad. Once he removed it via the Tivo he could load the backup. IIRC the backup module was getting confused when it was trying to figure out whether a season pass needed to be restored or not - it'd see this SP with no channel and stop in its tracks.

If your backup contains SP's that were for channels that you no longer receive then you will be asked to map them to your existing channels before the restore goes ahead. You don't need to edit the backup file.

[speed_phreak]

I've just downloaded the RC1 version and I can reproduce the problem here on my UK Tivo. I'll have a look into it. It may take a while, work is pretty full-on at the moment.

Well then, I am glad it is not just a US/directivo/6.2 thing... That will make it easier for you to fix, when you have time of course ;)

Thanks again!

[BTUx9]

nope... it's broken across the board (I should know, being the one who broke it :rolleyes: )
Pretty sure I messed it up with the new channel data structures, but haven't had a chance to look into it further.

I may get a chance soon, as I'm doing some work with the channel routines right now.

[agw]

nope... it's broken across the board (I should know, being the one who broke it :rolleyes: )


Tut tut :)

It's the channels and the SPs that aren't coming through. Got Friday off work to wait around for a washing machine repair-man, I'll look at it then.

[BTUx9]

OK, I think I found the problems with the backup module (some global references slipped through the cracks)

The modified code is in CVS at http://tivowebplus.cvs.sourceforge.net/tivowebplus/TivoWebPlus/modules/backup.itcl?view=log

I'm hoping some ppl will test it out and report back, especially the restore function

p.s. I also made the "view backup" more readable

[speed_phreak]

OK, I think I found the problems with the backup module (some global references slipped through the cracks)

The modified code is in CVS at http://tivowebplus.cvs.sourceforge.net/tivowebplus/TivoWebPlus/modules/backup.itcl?view=log

I'm hoping some ppl will test it out and report back, especially the restore function

p.s. I also made the "view backup" more readable

BTU,

I think you did good!

Backup with your backup.itcl code edit:

Copy existing backup file to '/TivoWebPlus/backups/settings.old' Done
Overwrite backup file '/TivoWebPlus/backups/settings' Done
Write header Done
Store layout version number 1 Done
Store TiVo version number 6.2-01-2-381 Done
Store details for 17 channels (for reference only, may contain duplicates) Done
Store details for 1 wishlists Done
Store details for 28 season passes Done
Store details for 27 series Done


Looks good to me!

As far as a restore goes, can i do a clear and delete settings (i think that is what it is called, NOT clear and delete everything) from the tivo interface? Would that be the easiest way to remove all season passes? or would I also be blowing up my hacks?

[BTUx9]

that wouldn't affect hacks, but I wouldn't do that if I were you... if there's a problem, I'd hate for you to have to restore that many things manually.
Eventually, somebody will use the restore (hopefully on a spare machine), and if it blows up, I'm sure I'll hear about it :D

[tward_biteme1]

Generally how long should it take to restore about 33 wishlists and about 60 or so season passes?

Just seemed to be going very slow.. Was late so I gave up and went to bed....

[kitschcamp]

Oh, it can take absolute ages! I did it with 100-odd season passes and it was a good hour or so.

[tward_biteme1]

Oh, it can take absolute ages! I did it with 100-odd season passes and it was a good hour or so.

I just didn't seen anything scrolling by or anything that would lead me to believe it was still going...... excpet for the progress bar and icon animation that explorer has....

I'll start it again when I have the time and it's now past my bedtime!

[tward_biteme1]

It sat for over an hour and then came up with Page Not Found.

Is there a way to restore running something via Telnet?

I must be doing something wrong.....

[tward_biteme1]

What would cause me to not see anything happening after I click restore? It tells me how my Season Passes and how many Wishlists there are to restore, but when I click restore, it looks busy, but I can't tell if it is doing anything...

Maybe I have some networking issues....
<------------------------------------>
"Once you're happy with what is going to be restored just click the 'Restore' button at the bottom of the summary page. The restore does not take very long, and it tells you what it is doing. Scroll down the page it produces and keep scrolling until you see the 'Return to backup summary' link. At this point you should be done.

Note that if you don't click 'Restore' then NOTHING IS RESTORED. Loading a backup file doesn't do anything except load the backup file. It's a three step process - load, configure and restore. "

[agw]

When I've run restores here they're fairly quick, as in seconds for my list of 60-odd passes.

Whether you see anything or not while its restoring depends on your browser. As the restore progresses information is sent to your browser, but it might not be willing to display anything on screen until the Tivo says that the page is complete, which won't happen until the restore is complete.

Regardless, I guess one approach would be to only tick off a handful for restore - one or two at first maybe? - and just go with those. Time it and see how long it takes. If they work then tick off a few more and restore those. It'd be a bit tedious, but probably less tedious than entering them all in again from scratch.

[tward_biteme1]

Even with TIVOSH_POOLSIZE=3244032, my HR10-250 reboots everytime a backup is made. My other S2 tivos run it just fine. Can/should the memory be increased further?

Never saw an answer to this....

My Philips DSR704 was zippered and I am trying to restore.

Everytime I hit "Load" after picking the backup file it reboots...

Can it be/Should it be increased more?

[BTUx9]

For those who are interested, I believe I've got the backup module working under TWP 2.0
It hasn't been released in a bundle yet, but can be grabbed at http://tivowebplus.cvs.sourceforge.net/tivowebplus/TivoWebPlus/modules/backup.itcl?view=log

p.s. re: reboots, most reboots on earlier versions of TW/TWP ARE caused by running out of memory... increasing the poolsize MAY help, but you may end up with more thrashing to swap, depending on your system... TWP 2.0 doesn't have the same memory issues and shouldn't reboot the tivo even if it does

[tward_biteme1]

For those who are interested, I believe I've got the backup module working under TWP 2.0
It hasn't been released in a bundle yet, but can be grabbed at http://tivowebplus.cvs.sourceforge.net/tivowebplus/TivoWebPlus/modules/backup.itcl?view=log

p.s. re: reboots, most reboots on earlier versions of TW/TWP ARE caused by running out of memory... increasing the poolsize MAY help, but you may end up with more thrashing to swap, depending on your system... TWP 2.0 doesn't have the same memory issues and shouldn't reboot the tivo even if it does


Couple of dumb questions:

Where can I get 2.0?

All I see is 1.3.1.

Is it hard to upgrade?

If I upgrade, will I have to do a new backup in order to restore to another DTivo?

[BTUx9]

Couple of dumb questions:

Where can I get 2.0?

All I see is 1.3.1.

http://thomson.tivo.googlepages.com/tivowebplus

Is it hard to upgrade?

well, a couple of things:
1) TWP 2 should NOT be installed on top of an older version (bad things happen)
2) if TWP 2 is being run from somewhere readonly, you need to set the env. var TWP_DATA_DIR to somewhere writable (I use /var/TWP)
3) all the config files are now found in the config subdir of TWP_DATA_DIR, and are created the first time TWP is run (so you have to run it before modifying them... yes, a bit backwards, but it's necessary)
4) if the directory name isn't TivoWebPlus, the update feature won't work properly (but this will probably be fixed in a future release)

If I upgrade, will I have to do a new backup in order to restore to another DTivo?
the backup format didn't change, so no

[tward_biteme1]

Sorry if I'm being dense, but:

1) TWP 2 should NOT be installed on top of an older version (bad things happen)

But I should be able to use the Update as long as the directory is TivoWebPlus?

the backup format didn't change, so no

So if I am only having reboot problems with the restore, I only really need to upgrade on the Restore machine for now.

[BTUx9]

I BELIEVE you can use the update to go to v2.0, though I've never tried it, personally
and yes, you'd only need to update the restoring box

[luder]

BTU,

I think you did good!

Backup with your backup.itcl code edit:

Copy existing backup file to '/TivoWebPlus/backups/settings.old' Done
Overwrite backup file '/TivoWebPlus/backups/settings' Done
Write header Done
Store layout version number 1 Done
Store TiVo version number 6.2-01-2-381 Done
Store details for 17 channels (for reference only, may contain duplicates) Done
Store details for 1 wishlists Done
Store details for 28 season passes Done
Store details for 27 series Done


Looks good to me!

As far as a restore goes, can i do a clear and delete settings (i think that is what it is called, NOT clear and delete everything) from the tivo interface? Would that be the easiest way to remove all season passes? or would I also be blowing up my hacks?


If you have hacks and using the clear delete all method would restore to factory and would have to rezipper

[luder]

It sat for over an hour and then came up with Page Not Found.

Is there a way to restore running something via Telnet?

I must be doing something wrong.....

Try checking telnet and try typeing twp if it tells you that it's running.. see if sync; reboot
(keep in mind there is a space between ; and reboot)

[luder]

For some reason my unit keeps saying
/var/TWP/backups/_var_TWP_backups_settings.htm is not a TiVo Web backup file
:confused:

[BTUx9]

I already answered that on DDB, luder

[luder]

I already answered that on DDB, luder
opps i must of slip thanks btux9 it's running at 100%
i give BTUx9 :up: :up: :up: and a :up:

[tward_biteme1]

I BELIEVE you can use the update to go to v2.0, though I've never tried it, personally
and yes, you'd only need to update the restoring box

I gave it a try, I assume the directory is case sensitive (TivoWebPlus), because it stated it couldn't find the directory...

It is tivowebplus on my machine...
I tried to rename the directory without success..

Was too late and had to go to bed, so will try to get that directory renamed...

[tward_biteme1]

Ok, I must be really stupid.

How can I rename tivowebplus to TivoWebPlus at the bash prompt?

[BTUx9]

umm... remount root r/w, perchance?

BTW, remember what I said about running r/o... you'll want to set TWP_DATA_DIR before twp gets called (usually means in author, but you COULD set it in /test.conf)

if you want it to still get auto-run at boot, you could put tivowebplus in as a symlink to TivoWebPlus

[tward_biteme1]

Thanks for the support there BTUx9!

Got it upgraded to 2.0 via the Upgrade module.

Ran the restore and bingo... 69 season passes and 30 Wishlists restored in about a minute or two....

Again, thanks for all the work you do on TWP!

[rbautch]

On my 6.3b tivo I can backup SPs just fine, but when I try to load the backup and restore it, I get this error:INTERNAL SERVER ERROR
--cut here--
action_backup_restore_read '' 'set "fname" "/var/tmp/backups/settings";set "submit" "Load";'
can't read "arr(9415)": no such element in array
while executing
"set result $arr($station)"
(procedure "extract_station" line 6)
invoked from within
"extract_station $snapshot MOD::snapshot_station"
("Series" arm line 11)
invoked from within
"switch -exact $restore_type {
Series {
set restore_fsid [agextract $restore Series]
set snapshot_fsid [agextract $snapshot Series]
if {[are_same_serie..."
(procedure "sp_matches" line 4)
invoked from within
"sp_matches $restore_data $snapshot_sp($snapshot_fsid) $restore_type $override_TmsId"
(procedure "find_matching_sp" line 6)
invoked from within
"find_matching_sp $restore_data """
(procedure "automap_restore_sps" line 73)
invoked from within
"$script"
invoked from within
"time {$script}"
(procedure "agtime" line 2)
invoked from within
"agtime {automap_restore_sps}"
(procedure "load_backup" line 35)
invoked from within
"load_backup $chan $fname"
(procedure "MOD::action_backup_restore_read" line 10)
invoked from within
"$cmd $p1 $p2 $p3"
(procedure "do_action" line 19)
invoked from within
"do_action $action $chan $part $env 1"
--cut here--

[BTUx9]

rather brave, restoring on 6.3 with a module that I don't think has been updated for the peculiarities of 6.3's SPs

[rbautch]

rather brave, restoring on 6.3 with a module that I don't think has been updated for the peculiarities of 6.3's SPsI live out there on the edge. :)

[MessyMarvin22]

After finishing with zippering, I am trying to restore my season passes using TWP and I get the following error...does anyone have any ideas?

Thanks,
MM

INTERNAL SERVER ERROR
--cut here--
action:backup_restore_read
path:
env:fname /var/TWP/backups/settings submit Load
can't scan path (errNmNameNotFound)

while executing
"mfs scan $mfsdir -start $name -count $count"
invoked from within
"transaction {
while {$i<$blen} {
foreach {id name type} [lindex $batch $i] break
if {$prelen && $prefix!=[string range $name 0 $prelen]} {return bReak..."
(procedure "ForeachMfsFileTrans" line 1)
invoked from within
"ForeachMfsFileTrans fsid name type $seasonpassdir "" 20 {
set sp [db $db openid $fsid]
set fields [dbobj $sp attrs]
set content [construct_record_cont..."
(procedure "take_snapshot_sp" line 8)
invoked from within
"take_snapshot_sp 0 1"
(procedure "take_snapshot_for_restore" line 4)
invoked from within
"take_snapshot_for_restore"
(procedure "load_backup" line 18)
invoked from within
"load_backup $chan $fname"
(procedure "MOD::action_backup_restore_read" line 10)
invoked from within
"$cmd $chan $path $env"
--cut here--

[agw]

Was this attempt at a restore made before the TiVo had finished indexing after a guided setup? If so then you will need to wait for a while - a day should be more than enough - and then try again.

The error seems to imply that it's having a problem reading the list of current season passes. TiVo have changed the path to the season passes at least once before, there's a possibility they may have done so again. The season pass folder is in a global shared amongst all modules and set up by TWP - if you have waited for the indexing to complete, rebooted after the indexing just to make sure and you still have a problem then either a later version of TWP or regressing to the last stable release (as appropriate) may address the problem.

[BTUx9]

unless you are running s/w <v6.3, it's best that it DOES bomb out, in that I'm fairly certain that restore corrupts the SP list on all versions >=6.3

[Uncle Spanky]

Sorry for the delay in replying.

3+ days does seem a bit excessive. You can check the /var/log/tvlog log file for the "Sweep done (eSucceeded)" line to see if the indexing has finished - but if the TiVo is telling you that it's still indexing then it probably really is.

Unfortunately the module can't work properly without an index of the series that the passes are for. Each pass refers back to the ID number of the series that it's for and this can't be (easily) figured out without the index.

If it's any consolation, which I imagine it isn't, the TiVo itself can't do much with the series until they've been indexed.

I've having a similar problem, but I have found the "Sweep done (eSucceeded)" line in the log file over 18 hours ago, but I am still receiving the commit error ? The wish lists loaded correctly, but all of the SPs fail (60 total).

Any ideas ?

[agw]

Blimey, first posting in over a year! What error messages are you getting when it fails (you don't need to reproduce all 60, one or two will do). Which version of TivoWeb are you running?

With the original Tivoweb (and possibly TWP, I've not tried) you need to restart it after the Tivo has finished reindexing everything so that it can get the correct channel list. Have you restarted Tivoweb? If Tivoweb is showing the full channel list in other places then it's probably not that.

[Uncle Spanky]

Thanks for responding mate - I wasn't sure if this thread was even being monitored anymore :)

Here's the stroy:
The original unit (Hughes DTV SD40) was getting tuner #2 pixelation, so I replaced the unit with a used box. The original unit was zippered about 2 years ago, running 6.2a-01-2-351 and TWP 2.1b2; The backup module is 1.00.0018, and the latest backup was taken a few days ago.

I pulled the HD, and put it in the new(used) box which is the same model number from Hughes. Activated with DTV, and noticed that the Service Number didn't populated, and the NPL wouldn't show up. No worries, tried to run the 51killer.tcl, and it wouldn't complete, saying that "no changes were completed". This wasn't due to a DOS error or keystroke problem because I use Putty. I was forced to perform a Clear and Delete to get to service number to populate, and the NPL to show up correctly.

I set the network settings, and reconfigured the wireless adaptor, and all looked good. Started TWP and tried a restore. The first few times through I got the errDB problem, and found that if I copied the backup.itcl from /modules to /libs it fixed that problem and the restore module read the file in, and indicated the following:

Number of season pass channels in lineup at time of backup 20
Number of wishlists in backup 6
Number of unique series in backup 44
Number of season passes in backup 55

Looks good. Ran the restore, and got the following result (tail):

Created new SeasonPass (fsid 2212014)
Setting Version 8
Setting MaxRecordings 5
Setting Priority 47
Setting Series dbobj2943
Setting Station dbobj2944
Restored Series season pass Lost

Created new Series (fsid 2212015)
Setting Version 52
Setting ServerId ASER-Formula One Racing|
Setting ServerVersion 0
Setting Genre 5
(adding to Genre - 320)
(adding to Genre - 124)
(adding to Genre - 121)
(adding to Genre - 137)
(adding to Genre - 341)
Setting ThumbData 268566912
Setting Title Formula One Racing

INTERNAL SERVER ERROR
--cut here--
action:backup_perform_restore
path:
env:submit Restore
commit failed (0x30012)
while executing
"transaction {uplevel $body}"
(procedure "RetryTransaction" line 5)
invoked from within
"RetryTransaction {

Since i couldn't get this to work I reran tweak.sh to reload zipper, and configured the box from scratch.

When I ran restore today, it actually added two SPs (30 Rock and Lost) before crashing when trying to add F1 racing (which I already added as a SP manually). In the meantime i have recreated about 40 SP manually, but If the restore problem can be resolved, I'd still like to restore the SPs for shows that aren't in the 10 day guide.

I'm going to try to modify the backup file to only restore programs that I haven't added to the SP manually, and see what happens.

Please let me know if you need any more info -

cheers

sb