missig guide data?

**timmy** · 16-03-2005, 03:04 PM

umm no. ur the slice expert dude...

i'm working on the guide data just using some basic SQL to pull forward data where it doesn't exist but has show up the last two weeks.

cool?

TiM

**thomson** · 16-03-2005, 06:16 PM

Originally Posted by timmy

I'm working on the guide data just using some basic SQL to pull forward data where it doesn't exist but has show up the last two weeks.

Hmm... an example of the problem you may encounter is as follows... if you predict information for TV2 next Wednesday and give the StationDay a ServerId of 1234500... then a few days later when you get the real schedule for that day and you then give the StationDay a differnet ServerId then there is likely to be two schedules in the database for TV2 for Wednesday - and I have no idea which would be used...

The trick is to use a number which can be predicted (such as ChannelId+DayNum) so that when it comes time to update it you simply overwrite it (using a later version number).

The other thing to watch is that you probably want to alter the ProgramId for the predicted shows otherwise they will not be recorded as they will be the same as the previous weeks Id, and TiVo will not record the same programme twice within a 28 day period.

**thomson** · 18-03-2005, 11:09 AM

Code:

GAP: Station TV1     Fri 18 Mar 16:05 - 19:00 (2 hours, 55 minutes)
GAP: Station TV1     Sun 20 Mar 01:35 - 01:00 (-35 minutes)
GAP: Station TV1     Sun 20 Mar 02:10 - 01:35 (-35 minutes)
GAP: Station TV1     Sun 20 Mar 02:00 - 02:10 (10 minutes)
GAP: Station TV1     Sun 20 Mar 02:00 - 06:00 (3 hours)
GAP: Station TV1     Sun 20 Mar 08:00 - 12:00 (3 hours)
GAP: Station TV1     Mon 21 Mar 06:00 - 09:00 (3 hours)
GAP: Station TV2     Fri 18 Mar 22:00 - 01:00 (3 hours)
GAP: Station TV2     Sat 19 Mar 07:00 - 10:00 (3 hours)
GAP: Station TV2     Sun 20 Mar 01:20 - 01:00 (-20 minutes)
GAP: Station TV2     Sun 20 Mar 02:40 - 01:20 (-140 minutes)
GAP: Station TV2     Sun 20 Mar 02:00 - 06:00 (4 hours)
GAP: Station TV2     Mon 21 Mar 18:00 - 21:30 (3 hours, 30 minutes)
GAP: Station TV3     Sat 19 Mar 07:00 - 10:30 (3 hours, 30 minutes)
GAP: Station TV3     Sat 19 Mar 13:00 - 16:00 (3 hours)
GAP: Station TV3     Sun 20 Mar 05:30 - 05:30 (-60 minutes)
GAP: Station TV3     Mon 21 Mar 15:00 - 18:00 (3 hours)
GAP: Station Prime   Sun 20 Mar 02:00 - 02:00 (-60 minutes)
GAP: Station Prime   Mon 21 Mar 21:45 - 00:20 (2 hours, 35 minutes)
GAP: Station C4      Sun 20 Mar 04:00 - 04:00 (-60 minutes)

**thomson** · 21-03-2005, 02:37 PM

The attached image is the output from the GDchecker TivoWeb module.

Code:

GAP: Station TV1     Fri 25 Mar 13:25 - 15:40 (2 hours, 15 minutes)
GAP: Station TV2     Mon 21 Mar 18:00 - 21:30 (3 hours, 30 minutes)
GAP: Station TV3     Mon 21 Mar 15:00 - 18:00 (3 hours)
GAP: Station TV3     Thu 24 Mar 15:00 - 18:00 (3 hours)
GAP: Station TV3     Fri 25 Mar 06:00 - 09:10 (3 hours, 10 minutes)
GAP: Station TV3     Fri 25 Mar 21:15 - 00:40 (3 hours, 25 minutes)
GAP: Station Prime   Mon 21 Mar 21:45 - 00:20 (2 hours, 35 minutes)
GAP: Station C4      Fri 25 Mar 21:00 - 01:00 (4 hours)

**thomson** · 06-04-2005, 12:34 PM

Code:

GAP: Station TV1     Thu  7 Apr 03:30 - 06:00 (2 hours, 30 minutes)
GAP: Station TV1     Thu  7 Apr 15:05 - 18:00 (2 hours, 55 minutes)
GAP: Station TV1     Fri  8 Apr 00:40 - 03:00 (2 hours, 20 minutes)
GAP: Station TV1     Sat  9 Apr 03:00 - 06:00 (3 hours)
GAP: Station TV1     Sun 10 Apr 09:00 - 12:00 (3 hours)
GAP: Station TV1     Mon 11 Apr 06:00 - 09:00 (3 hours)
GAP: Station TV2     Thu  7 Apr 09:10 - 12:00 (2 hours, 50 minutes)
GAP: Station TV2     Thu  7 Apr 15:30 - 15:31 (1 minute)
GAP: Station TV2     Mon 11 Apr 01:25 - 03:25 (2 hours)
GAP: Station TV2     Mon 11 Apr 12:00 - 15:00 (3 hours)
GAP: Station TV3     Sun 10 Apr 18:00 - 21:30 (3 hours, 30 minutes)
GAP: Station TV3     Mon 11 Apr 06:00 - 09:00 (3 hours)
GAP: Station Prime   Wed  6 Apr 15:00 - 18:00 (3 hours)
GAP: Station Prime   Thu  7 Apr 15:00 - 18:00 (3 hours)
GAP: Station Prime   Fri  8 Apr 18:00 - 21:40 (3 hours, 40 minutes)

**Wanted** · 06-04-2005, 01:25 PM

Bring on the predicted data. I missed Firefly last night because of a gap in TV3 listing..... now it's personal

**SeanOffShotgun** · 06-04-2005, 06:42 PM

Originally Posted by Wanted

Bring on the predicted data. I missed Firefly last night because of a gap in TV3 listing..... now it's personal

That's why this Guide Data Checker is useful. Using it I knew a few days ahead that TV3 had gaps (1 channel among many).

In the current circumstances everybody should be using this module.

Instead of ~moaning~, do what I did and offer to help.

**thomson** · 07-04-2005, 06:36 AM

Originally Posted by Wanted

Bring on the predicted data. I missed Firefly last night because of a gap in TV3 listing.....

Actually, the predicted data would not solve the problem as it is usually implemented as a duplicate of the previous weeks slice, so if there was a gap in that... then there would be a gap in the predicted data. And you would only be using the predicted data if you had not been able to contact the emulator to receive the real data (which may also have gaps in it).

The program that creates the slice needs to be improved so that there are no gaps in the data. There is a chance that the pa rsers/scrapers are the ones at fault, but the other sources of XML-TV data seem to contain the missing information which leads me to believe it is the slice generator which is getting confused.

For example TV1 always has gaps, but the following script will pa rse the TVNZ site fairly well (for TV2 just replace tvone with tv2 in the URL). It does not work out the duration of the episodes, but that can be deduced from the start time of the following show. [The optional parameter to this function is the day (eg thursday)]

Code:

TV1() {
  TMP=/tmp/TV_TVNZ_TV1.tmp

  if test -n "$1"; then
    day="&date=`date -d $1 +%d%m%Y`"
  else
    day=""
    # TVNZ site is a little broken, seems to display yesterdays if
    # todays is not available, so hardcode it.
    day="&date=`date +%d%m%Y`"
  fi

  lynx -dump -nolist -width=1024 "http://tvnz.co.nz/view-preempt/tvone_epg_skin/channelDesc=tv_one$day" >$TMP

  cat $TMP |\
  sed "s,\',',g;s,\&amp;,\&,ig" |\
  while read line; do
    if test `expr "$line" : '.* [0-9][0-9]:[0-9][0-9]$'` -ne 0; then
      title=`echo "$line" | sed 's, \([0-9][0-9]:[0-9][0-9]\)$,,'`
      time=`echo "$line" | sed 's,.* \([0-9][0-9]:[0-9][0-9]\)$,\1,'`
      hour=`echo "$time" | sed 's,:.*,,'`
      read desc
      desc="`echo "$desc" | sed 's,[0-9]*$,,'`"

      echo "$title|$time|TV1|$desc"
    fi
  done
}