Running out of data

[Muzza]

Hi all.

I have got up this morning and changed the TiVo time to reflect daylight savings which went fine but I notice that I have run out of guide data on some channels but have several days worth on others.

I am not receiving any errors relating to the service being unavailable etc.

Anything going on that I should be aware of or has something died at my end?

I didn't check whether the missing data was actually populated before I did the time change so can't be sure if it happened since then or not. :(

[Muzza]

This fault looks related to the other thread entitled - "Guide Data from Dev not indexing"

Guess I'll keep an eye on that one. :rolleyes:

[tonys]

Same problem here. BTW, how do I change the Tivo time for daylight saving?

[DJC]

Yep, I'd say related to other thread... I've got the same problem.

I'm on it.. seems to be the dev systems first issue, and it could have happened at a worse time... murphys law.

I've recreated the last 2 days slices and am currently clearing and deleting my
programme data (and todo) to see what whether that helps.

[brucer]

FWIW, the guide data issue (no listings for several channels) is not related to the dev emulator, I have the same issue using LOADGUIDE (while Ed's emulator is down).

[DJC]

brucer ~ What slice are you loading? is it the dev slice nzdev.slice or the nztivo.slice

(this info might be useful, cause I know nothings changed on the dev system, has no time of late)

[Muzza]

# This is for the server emulator on minnie.
127::210.48.107.133:80:::

Does that help?

[MrGadget]

For what its worth this seems to be accompanied by errors in Tivo web - such as the following when viewing the West Wing in the Grid module:

Episode Num Orig.Air Date Channel Date
INTERNAL SERVER ERROR
--cut here--
action_series '/2148869' ''
can't read "nowshowingdir": no such variable
while executing
"ForeachMfsFile fsid name type $nowshowingdir "" {
if {[catch [do_seriesrow $chan $fsid $objectid 1] error]} {
puts "do_seriesrow $fsid $objectid ..."
(procedure "::action_series" line 34)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

or Top Ten: The Eurythmics (C4 3pm today)

INTERNAL SERVER ERROR
--cut here--
action_showing '/2128501/59' ''
can't read "cache_sp_key": no such variable
while executing
"lsearch $cache_sp_key "$seriesfsid|$stationid""
(procedure "::action_showing" line 621)
invoked from within
"::action_$action $chan $part $env"
("eval" body line 1)
invoked from within
"eval {::action_$action $chan $part $env}"
--cut here--

[brucer]

DJC, I'm using this url: http://210.48.107.133/slice

[donm]

Hi All

I'm using the emulator on orac and have the same issue. I reverted to my serial connection and did a manual download and indexing from the orac FTP address. All seems good except TV one and two which have no guide data.

Any clues what is happening?

Don

[tonys]

I've done a clear guide data, forced a daily call, seemed to get a good download, tivo's finished indexing and there's holes all over the guide data!
Any progress on this issue? btw I'm using port 80 on the development emulator.

to repeat an earlier request, how do I correct the Tivo clock for daylight saving??

[waynez]

I'm having the same problem, Tivo is showing missing information all over the place. However a twist in the tale ( tail? )

via Tivo Web, Whats On, everything is showing as being there, no gaps all looks good, I can schedule programs to record and the tivo is changing to record the shows so whats the go with that?????

keep up the good work though guys, :D

[Wanted]

This website seems to display what my TiVo is seeing.

http://mr.geek.nz/epg/online/program_listing.php?hour=22&date=20051002

If I look at Whats on in TiVo Web I can see data there too. That's weird but sounds like a good clue to what is wrong.

[MrGadget]

to repeat an earlier request, how do I correct the Tivo clock for daylight saving??

Open up a telnet window and type:

settime [YYYYMMDDhhmm]
(obviously replacing the [YYYYMMDDhhmm] with the year/month/day etc - eg settime 200510021855)
then type:
settime -rtc

[zollymonsta]

Open up a telnet window and type:

settime [YYYYMMDDhhmm]
(obviously replacing the [YYYYMMDDhhmm] with the year/month/day etc - eg settime 200510021855)
then type:
settime -rtc

you can alternativley run fixtivotime if you have it running.. and then issue a settime -rtc

[MrGadget]

you can alternativley run fixtivotime if you have it running.. and then issue a settime -rtc

or get really smart and use the attached script to avoid the problem all together...

[tonys]

Think that'll move me firmly outside my comfort zone! I'm gradually getting happier messing with Tivo settings via Putty, I can do the timestamp thing, force calls and indexing etc, even managed to change emulator, but I've no idea how to add scripts...........anyone want to give me some guidance?

ran the settime procedure, so all well there now, at least wifey can do manual recordings while we're waiting to resolve the guide data issue.

Thanks all!

PS Where would I find fixtivotime? Had a look in hack/bin and a few other directories and can't find it.

[Wibble]

PS Where would I find fixtivotime? Had a look in hack/bin and a few other directories and can't find it.

it's in /hack/bin/fixtivotime on my 1.3.

Anyway here it is:

#!/bin/sh
# Stop time warp messages
touch /var/timestamp
# Set the time to time server with appropriate timezone offset
/tvbin/settime `rtimetz -v 207.126.98.204 207.126.98.204 129.6.15.28 131.107.1.10`

# Now set hardware clock
/tvbin/settime -rtc

[neilp]

But I had ftp'd a slice manually earlier on, and just got the updated slice announced on theis thread by jaidev.

Funny thing was the previous slice had a later timestamp than the current one, didn't just seem to be an hour off due to daylight saving but considerably more. Spotted it when my ftp client complained and asked if I really wanted to overwrite the existing newer version.

Sorry cant advise the actual timestamp since it was overwritten, but I wondered if this may be related in some way to current problems?

Apologies if this is a red herring.... :confused:

[DJC]

DJC, I'm using this url: http://210.48.107.133/slice

Thats the top level URL the slice file is which... must be nzdev.slice??

[DJC]

OK, this problem is proving to be more problematic to track down :mad: than first thought.

This is the first problem with the dev system to date, and its proving to be a difficult one. The answer is probably simple its just a matter of time, the testing cycle of creating and loading the slices is long unfortunately. I want to get to the bottom of it so that it can never re-occur.

To buy a bit of time, in the interim I've posted a modified version of jaidev's slice (cheers jaidev good redundant source) into both of the dev emulators and tested it, it loads ok; so that will bring data back, but will probably confuse season passes in the short term. Watch out for that. [Dev system has turned into an ed-ulator for the time being]

Will use this thread to update on status...

[sadrake]

Just force a daily call to pickup the temp slice and it did not download any data?? :confused:

[DJC]

Just force a daily call to pickup the temp slice and it did not download any data?? :confused:

I've pushed the slice number forward again try again...

[sadrake]

Yep that did it, downloaded and imported , as you expected season passes arent working.

[zollymonsta]

Downloading here now too.

May I suggest that if there are shows you MUST have, set up a manual recording for them until the data is fixed. I do this for shows that I don't want to miss anyway :)

[Muzza]

Just forced a call and the data seems to be reappearing again. Nice work. Shame about the season pass senario but I'll get over it. :D

Good luck on the permanent fix.

By the way - did anyone notice that the forums are an hour behind...haha

[slangnz]

I've pushed the slice number forward again try again...

Looks good to me. Tivo GUI now has program information. Web GUI however has duplicated information.... I didn't do a 'clear guide data' before hand...

Also, does this fix also help the old emulator? (I've only just moved over to the dev emulator.)

Terminologoy suggestion:
Old emulator (64.151.65.200:80): Prod (Production)
Dev Emulator (210.48.107.133:80): UAT (User Acceptance Testing)
Other one.. (210.48.107.133:800): Dev (Development)

Sorry, that's just the way I think of things... may help others, and can be integrated in DNS when we get images that support DNS.

Cheers -

[DJC]

....shows a lot of red #’s from Sunday onwards...

Could it be to do with daylight saving in some way??? Probably not but…


The prize goes to Mr Gadget; definitely a daylight savings change issue.

This is a result of the database abstration layer. Feeding information into the database needs to correct for timezone on the way in - and way out, particularly when data straddles the change day the time zone correction changes. Seem's obvious now when looking at the symptoms.

Anyway, I'm onto the issue the next slice should be back up off the main database later tonight (Tue) [after testing].

/DJC

[MrGadget]

Oooohh goody... what's the prize?
I'll settle for a fully working Tivo (unless there's a trip for two somewhere nice...)
:)

[bcolloty]

The prize goes to Mr Gadget; definitely a daylight savings change issue.

This is a result of the database abstration layer. Feeding information into the database needs to correct for timezone on the way in - and way out, particularly when data straddles the change day the time zone correction changes. Seem's obvious now when looking at the symptoms.
/DJC

Any developer thats ever had to deal with Daylight savings knows what a huge pain in the *** it is. Used to be the absolute bane of my life when i worked on a metering/billing project - some of our data would come in adjhsyed, sone unadjusted and all of it needed to end up in the system with the right offset - not even to mention how do you compare days that have 50 1/2 hour periods with those that have 48 :(.

This does make me wonder about what the "correct" daylight savings settings should be for the dev emulators.
Should we have DS turned on in Guided Setup and configured for Auckland as per the directions on the NZTivo site?
OR DS turned off and let the system update the clock when it gets the first slice post changeover?
OR should we be setting the time manually after the change using SETTIME?

DJC - I imagine the above vonfig will have some impact on the success of your changes when we "fall back" in March - can you let us know how you've got things set up at some point.

Thanks for all the great work BTW - I for one certainly appreciate the efforts that all you guys put into supporting the local Tivo community.

Cheers,

Brendon

[MrGadget]

Should we have DS turned on in Guided Setup and configured for Auckland as per the directions on the NZTivo site?

Not sure where on that site you are looking but the imagewithscreenshots1.4 (http://www.nztivo.net/index.php/ImageWithScreenShots1.4) says ""Daylight Savings" select "no" and press "select" button"

Same is repeated on the Cable GS (http://www.nztivo.net/index.php/Cable_GS) page.

I understand this to be the correct settings with either manual intervention or script to fix the DS issue.

[bcolloty]

I understand this to be the correct settings with either manual intervention or script to fix the DS issue.

Sweet - I admit i was working from memoryand it's been a few months since I did the setup - what was confusing me was the "Keeping Time" (http://www.nztivo.net/index.php/Post_Setup#Keeping_Time) entry under Post Setup on the site - I was assuming that the chages made there would actually deal with the Daylight Savings issue as well as just setting the time.

However I'm pretty sure that when I forced a daily call against the dev emulator on Sunday morning the time did get updated on the Tivo - no manual update or script required.?!?!

Cheers,

Brendon.

[MrGadget]

However I'm pretty sure that when I forced a daily call against the dev emulator on Sunday morning the time did get updated on the Tivo - no manual update or script required.?!?!

Yeah - that will generally fix it - but of course there is a lag between DS change & daily call - so long as there is nothing you want recorded during that time there is no issue.
Thompson created a script to deal with this - see the thread here (http://www.forums.oztivo.net/showpost.php?p=2184&postcount=4) - Note - two traps if you are using this - make sure it is saved with Unix line endings (not windows) and make the script file executable ( eg chmod +x Daylight_Savings.sh).

[jaidev]

I understand this to be the correct settings with either manual intervention or script to fix the DS issue.

Correct, TiVo's Daylight savings is broken for the southern hemisphere...

[Wanted]

DOWN with daylight saving!!

[DJC]

OK

a) The problem was with the daylight savings switch. It was an issue in the code (python) that inserts the data into the database not recognising this shift. Essentially the daynum calcs were then left one hour out levaing any days first show, if it started in the first hour, in the previous daynum.

This was why some channels showed up when the slice was loaded. Those channels that didn't have shows in the first hour were ok. The rest confused tivos indexing scheme.

b) The emulators are ntp'd and track well, so nothing to do there, and yes the daily call works well for syncing; I never change my tivo time.

c) The slicing system date adjustment has been made (-13 hours) so its back to normal operation, the next problem I expect is in 6 months time (-12 hours).

I have stored away the transitional data from this change so I have something to test and work with. So I'll do that as time permits to ensure we have the best chance of a smooth transition next time. Its added to the list of developments to be done.

d) Slices have been restored and from yesterday and pushed to the emulator, its a full zero referenced slice containing all days. Today's slice will start being build as usual around midday and will be business as usual.

e) On my tivo I did a "clear and delete prog data and todo list" this morning and reloaded the slice. I can report that all is good.

The question remains as to whether tivos will just tick along and bring themselves back inline, or whether a clear and delete of guide data is a better option?

I suggest a clean out might be a good option. If you choose to do this then backup your season passes first! restore them after the event.

f) As the guru's Jaidev and Ed have declared we are generally looking to move to this new system as a platform for moving forward. (Just unfortunate that the opportunity presented itself on daylight savings weekend)

Theres plenty of potential in this new platform with its database abstration including web interfaces, more augmenting agents, mulitple feeds of data, custom headends including FTA mixes an so on (and therefore lots of work too)

g) A new development emulator has been created so the current systems on orac are now no longer considered development. There was an earlier post on naming etc. I suggest:-

Original emulator (64.151.65.200:80): "The Ed-ulator" - Backup production (always pays to have some redundancy)

Orac:-
Main Emulator (210.48.107.133:80): Production Emulator
Test Emulator (210.48.107.133:8000): UAT (User Acceptance Testing)

We will be testing headends and stuff on another emulator, there are a few game testers over on the dev forum so we'll enlist their help. Once things look good we will move any updates to the UAT emulator for wider public testing.

We still recommend GS'ing against the port 8000 emulator [for logos] and then dropping back to the port 80 emulator for general running.

I'll be splitting slice production so that we have a stable production slice and an option to work with development slices. But, most of the slicing work is fairly complete. What remains is the completion of web interfaces and community input to the data.

Thanks for your patience through this one.

Cheers/DJC

[jaidev]

The question remains as to whether tivos will just tick along and bring themselves back inline, or whether a clear and delete of guide data is a better option?

I suggest a clean out might be a good option. If you choose to do this then backup your season passes first! restore them after the event.


Great work DJC...
They pretty much should after the data for the previous slice runs out since there maybe some duplicate data if the current slice is loaded due to differing daynums..
Let's say it will prob take a few days for your TiVo to sort it self out if you don't do a 'Clear and Delete of Guide Data / ToDo list..' which I recommend..

And as DJC says backup your season passes first!..

[tonys]

Will the 'new' slice have the 'nonag' dummy guide data in it? Having done a clear prog data over the w/e, I'm expecting the nagging to start again soon!

Thanks to all responsible for keeping our Tivo alive!

[DJC]

Your tivo should never nag on the new system; unless I've missed something.

No need for NoNag slices or channels.

[tonys]

Good job! TY!

[sadrake]

To be back to normal i did a "clear and delete prog data and todo list" after backing up my season passes, made the daily call and got the new data, restored my season passes and ran an index.

What i found was that my season passes were busted, and wernt picking up episodes, so I deleted one and re created and it then went off and found all the episodes it needed to record. A minor inconvenice to get back to normal, just have to redo the remaining 15 season passes.

Im using the test emulator due to issues with port 80 and my ISP's transperant proxy even using the Tivo Proxy...

Test Emulator (210.48.107.133:8000): UAT (User Acceptance Testing)

Looks like everything has returned to normal. ;)

[MrGadget]

What i found was that my season passes were busted, and wernt picking up episodes, so I deleted one and re created and it then went off and found all the episodes it needed to record. A minor inconvenice to get back to normal, just have to redo the remaining 15 season passes.
Yes, I had the same issue. As you state, not a major issue given that most of my recordings are done via Wishlist, and at least the Season Passes were still listed making memory recall not required (thankfully!). Could it be something do do with the ed-ulator slice given in the interim?
Hope no-one got caught out by this and missed their favourite program :eek:

Must also pass my thanks onto those involved with fixing this wee issue and providing some guide data in the mean time. As a user I really do appreciate the effort of those with smarter minds than my own :o

[zollymonsta]

I believe that the season passes will start working once the old data runs out and the new data comes up.

[MrGadget]

I believe that the season passes will start working once the old data runs out and the new data comes up.
No... (it all comes flooding back now...) - I did a clear & delete data before loading the 'fixed' slice - so I had no old data... Something else changed....

[Wanted]

I didn't do anything to mine but wait for a fix, and my season passes are busted too. I manually created one for a show that is already there and it creates a duplicate (Same name, channel etc). I have 35+ I believe. That's because I have a season pass and a Wishlist for each program so it will take a while/an effort to create them all again.

[Bonzo]

I had the same problem with season passes not working, even though there was plenty of guide data. and I found this on oztivo.net.

http://minnie.tuhs.org/twiki/bin/view/FAQ/MyTivoSaysIHaveNoGuideData

It worked for me and now everything is back to 100%.
Note you might have to replace the 'fixtivotime' with

touch /var/timestamp

if you are running the 1.4 image.

[MrGadget]

Don't have the same issue here - my clock is not warped at all.

[JECastor]

Yes I have had the same problem with no indexing occuring and system information stating the my connection had failed (now states that connection was successful) on both my TIVO's and a complete re-image and GS to :80 did not fix it. It also stopped endpadplus from working. The thread above worked. I also used settime -rtc. So I now have data back whne you select a channel and endpadplus is working. Now am waiting to see if indexing, etc will come right.

[MrGadget]

Are the 'powers that be' aware of this post http://forums.oztivo.net/showthread.php?p=5743#post5743
(Linked to from here in case they are subscribed to this thread...)

Merry Christmas Tivo'ers :)

[DJC]

Handled on that thread.

[brian]

Hi y'all. I'm on LOADGUIDE, and have run out of data for the last 36 hours or so. Anyone else having problems?

Brian:confused:

[MrGadget]

I'm on the emulator - and no issue here...

[DJC]

Hi y'all. I'm on LOADGUIDE, and have run out of data for the last 36 hours or so. Anyone else having problems?

Brian:confused:

Sounds like you've missed a slice or two, which is the problem with loadguiding ... suggest emulator.

[brian]

has never been a problem - it has been utterly reliable for over 18 months - something must have changed. Maybe I am the last loadguider out here? Brings images of Japanese soldiers unaware that WW2 finished 20 years previously...

Will be interested to hear if here are any other loadguiders left before I make the change

B

[Wibble]

has never been a problem - it has been utterly reliable for over 18 months - something must have changed.


I am on LOADGUIDE as well, though I run it manually (long story)
There is a difference between the new slices on orac and the old slices.
If you skip a few day and run LOADGUIDE again, sometimes the data you had for today
or tomorrow 'disappears' and you are left with data starting a couple of say in the future.
It's not an issue if you run LOADGUIDE everyday and the emulator which downloads every day will also hide that problem.

[brian]

(as soon as we find out what's normal anyway...)

With no further action on my part, normal service has returned. I suspect I owe Jaidev more thanks than usual! Thanks!!

B