community guide data effort..

[brucer]

until we have a better solution, why don't those of us with some time and/or programming skills volunteer to look after a particular channel or channels, grab what we can and manually fill in the rest from other sources like skywatch magazine, tv guide etc.. Jaidev, Timmy.. would you be happy to share the current grabber stuff (or least point us in the right direction) so we can start tinkering with it? we could then upload our data somewhere central so a slice can be created..just a suggestion - what do you think?

[timmy]

yo,

have an even better idea right now. just thought of it. i know the nerdthings.com xmltv people have a grabber.
http://210.56.39.237/xmltv/mytvlistings.asp

previously i think it only had saturn data but they appear to have sky data. i just did a weeks grab and it was 1.5mb, it's xml so bloated of course but the data looks good. i'm just busily coding up a parser for it to dump it into my database. ETA 30m

will see how their data is and if its crud we can think of alternatives.

now back to work...

[timmy]

ok i wrote an xml parser and mapped their channels to ours. The good news is we have data for the forseable future, the bad is it isn't perfect. certainly alot better than what i can provide right now.

i had to usehis saturn listings for a few channels, newsaus, prime, tv3 as they had no sky data. that means he probably isn't grabbing on a daily or weekly basis as these are the channels that get updated frequently and usually have less than 1 week of data. The saturn data appears to be crappy, prime had one day i think, ugh.

i added tv1,tv2 from tvnz using my own grabber so that data will be good.

anyway, slice will be made shortly i imagine. will be tested first before we replace the one we put up this morning.

cheers ears

[ehintz]

Now we're talkin'... This is the sort of thing I was blathering on about a month or so back, concentrate first on getting something to convert XMLTV data into tivo slices, and let others concentrate on the non-tivo specifics such as getting XMLTV data in the first place... Once we have a good XMLTV-Tivo solution, then we can also devote effort to the XMLTV problem, and more folks working on a problem will yield a faster and more feature rich solution. Unfortunately my coding skills (or lack thereof) are pretty clearly not yet up to the task-I can make slight mods to others tools, but significant mods as of yet elude me.

Anyway, my observations so far:

XMLTV2Tivo and Parsexmlguide.pl (both from Tivo Canada) don't work well at all with NZ XMLTV sources, either not getting data or yielding corrupt slices. WKTivoGuide makes good slices, but doesn't deal with XMLTV data. Ergo, a XMLTV-WKTivoGuide tool will be a GoodThing to have(and is presumably what you've just done). The other option of course would be modifying one of the TCan jobbers to deal with non-Can data, but given their dodgy output I think you've chosen the smarter course.

Of course, we're basically now reliant on MFs scrapers, but since he's scraping multiple sites the odds are better we'll get data. And if MF drops we can always concentrate on mods to the older tv_grab_nz2 script (I've been able to mod it and get decent data). And in the meantime others will be feverishly working towards the same goal. Seems like a sweet solution all around. Timmy, you da man. :D