Oddity in channel per day guide data?

[markb]

I've just noticed an oddity in the data of the channel per day xml files.

There is a movie tonight on Seven Brisbane. It's "Super Trooper" starting at 11pm and runs for 125 minutes. The oddity is that it is listed in the Seven_Bris_2008-09-06.xml.gz file starting at 11pm, duration 60 mins, then also in the Seven_Bris_2008-09-07.xml.gz starting at 00am, duration 85 mins. I.e. it is split as 2 independent programs for each day. I would expect that the channel day files should contain all program data for programs *starting* that day. I.e. based on start time with the correct duration.

I guess this is the way the oztivo channel/day files are always built? But why would anyone want it to work the way it currently works with the program split as distinct entities? The duration is meaningless that way.

[petestrash]

It is an effect of one of our fetchers, our source for that city has changed from the usual 6am-6am schedules to midnight to midnight schedules. So when we now get the data, we only get data till midnight. At that point we do not know if the duration is supposed to go past midnight or stop at midnight so a showing is created that ends at midnight. The next fetch gets the following days data and programs that run past midnight are shown as a separate show by our source, so a second show is created.

We do not not know what is meant to be continued and what is meant to be a second episode, but the system makes a prediction and joins the one's that it thinks belong together.

This one seems to have slipped through the automated system. We rely on users to look at the guide and correct any errors that are found. It appears no one has fixed this one.

Peter.

[markb]

Thanks for replying.

This one seems to have slipped through the automated system.

But why doesn't that logic at least just simply join any adjacent programs having exactly the same title as that program did?

I'm asking because I am likely to implement that simple change on my input filtering to protect myself from this dangerous problem. I'd like to know what issue I am missing?

[petestrash]

We do not not know what is meant to be continued and what is meant to be a second episode, but the system makes a prediction and joins the one's that it thinks belong together.

I think you missed this part of my last reply

Peter.

[markb]

I think you missed this part of my last reply

No I didn't. I was asking about this "system" logic. Perhaps I need to be more explicit.

Surely the logic merely looks at the title (+ subtitle + desc?) and if it is the same as the previous program on that channel, and the start time is <= the stop time of the previous show, then it is most likely the same program? The "Super Troopers" movie I quoted would/should have been fixed by this simple logic but it wasn't, so it seems the logic must not be like that? I'm asking why why the logic can't be that simple? Am I missing something?

I build a single xmltv file for all my channels + 8 days I am interested in. I just wrote a program that filters this single file according to the logic I state above and found in fact that there are many such oddities in the guide data (21 split programs across 6 channels in 8 days for Brisbane). E.g. in Brisbane, the "Today" program is indicated as starting 5.30am for 30 mins then "starts again" at 6am every day?! I'll leave this filter in place to protect myself but am just pointing this out because I think it must be an error. There are occasional programs (e.g. "Two and a Half Men") which really are split episodes concatenated together and my logic is joining them up as a single program (since they have exactly the same title, subtitle, and desc) but that's not really a problem I guess and avoids the far worse problem of my auto timers getting set for a short duration.

[petestrash]

- Our data is primarily for TiVo owners and the TiVo's logic usually works around most of these issues.

- Our data comes from a variety of sources each which with there own unique problems

- We are a group of volunteers all of us have fulltime jobs, family responsibilities etc. There are many issues on our never ending todo list with varying priorities. Our system is constantly evolving and hopefully better with each evolution.

We are aware of this recent issue and a better fix will be implemented when we get to it.

Peter.

[markb]

Peter, sorry but I am not trying to be antagonistic. I'm new and just trying to understand how the logic works, or is supposed to work.

- Our data is primarily for TiVo owners and the TiVo's logic usually works around most of these issues.

Ok, well this explains a lot. So what you are saying is that this is not really a problem for Tivo users (which I am not) so that is why such an issue can exist without much concern here. Of course that is understandable and a quite fair given this is oztivo.net site! Thanks for replying here to me.

[petestrash]

It's pretty rare for a movie to to be aired across midnight, so it's not a priority. Series are handled ok anyway.

It is on our to-do list to make the join happen correctly more often.

Peter.

[markb]

It is on our to-do list to make the join happen correctly more often.

Well if anybody wants a copy of the program I created to fix this then please contact me. Here's the usage:

pc:~/Data/src/xml2tgd ./fixoztivo.py -h
Usage: fixoztivo.py [options, see -h] [file|stdin(-)]

Program to fix split program entries in oztivo XMLTV data. Will take a single
XMLTV file as an argument, or on standard input, and produce a converted file
on standard output. The input file can contain multiple channels and/or
multiple days of data. Assumes that programs are time ordered for each channel
although the channels can be interspersed. Will find adjacent programs of the
same title + subtitle + description and will delete the 2nd (and any other
adjacent duplicates) while extending the first programs duration. Best if
input file contains multiple days so that programs split across midnight are
corrected.

Options:
  --version      show program's version number and exit
  -h, --help     show this help message and exit
  -v, --verbose  output informational user messages

It's robust because it manipulates the xml as clean objects using python's elementtree. Python will run on all platforms.

[petestrash]

Thanks, the data needs to manipulated prior to it being made available in XML.

But if you want post the attachment here, I'll pass it on to Kevin as it is on his todo list.

It made speed up it's implimentation

Peter.