Daily call failed since swapping to optus

[ian]

I've been running two tivos for years, with no problems.

But a week ago I swaped from adsl1 (exetel) to adsl2 (exetel) - which is provided by optus.

Daily calls haven't worked since..

First thing I did was follow http://minnie.tuhs.org/twiki/bin/view/Network/BypassTransparentProxyServer and test for a proxy server.

However, both
http://www.lagado.com/proxy-test
and
http://www.lagado.com/tools/cache-test

didn't think there was a proxy

and

C:\tracetcp ftp.monash.edu.au:ftp
C:\tracetcp ftp.monash.edu.au:http

also didn't think there was ie


>tracetcp ftp.monash.edu.au:ftp

Tracing route to 130.194.11.222 [ringtail.its.monash.edu.au] on port 21
Over a maximum of 30 hops.
1 2 ms 2 ms 2 ms 192.168.34.138 [home.gateway]
2 Destination Reached in 51 ms. Connection established to 130.194.11.222
Trace Complete.

>tracetcp ftp.monash.edu.au:http

Tracing route to 130.194.11.222 [ringtail.its.monash.edu.au] on port 80
Over a maximum of 30 hops.
1 2 ms 2 ms 2 ms 192.168.34.138 [home.gateway]
2 Destination Reached in 52 ms. Connection established to 130.194.11.222
Trace Complete.
>

I've been reading MelissaH 's long posts (and her new one) and have tried a fair bit of that, ie -


[TiVo [p0] ~]# netstat -atn
Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN
tcp 0 132 192.168.34.1:23 192.168.34.139:1480 ESTABLISHED
tcp 0 0 0.0.0.0:23 0.0.0.0:* LISTEN
tcp 8784 0 127.0.0.1:8000 127.0.0.1:1031 CLOSE_WAIT
tcp 8433 0 127.0.0.1:8000 127.0.0.1:1026 CLOSE_WAIT
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
[TiVo [p0] ~]#


I can wget as well ie


[TiVo [p0] ~]# wget -v http://minnie.tuhs.org -O direct_test.html
--21:25:43-- http://minnie.tuhs.org/
=> `direct_test.html'
Resolving minnie.tuhs.org... done.
Connecting to minnie.tuhs.org[131.244.9.101]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7,388 [text/html]

0% [ ] 7,388 0.00B/s ETA 00:00

Last-modified header invalid -- time-stamp ignored.
21:25:44 (0.00 GB/s) - `direct_test.html' saved [7388/7388]

[TiVo [p0] ~]#

Though I am getting invalid time stamps.

But the local hosts one is failing ie


[TiVo [p0] ~]# wget -v http://localhost:8000 -O proxy_test.html
--21:29:20-- http://localhost:8000/
=> `proxy_test.html'
Resolving localhost... done.
Connecting to localhost[127.0.0.1]:8000... connected.
HTTP request sent, awaiting response...

- and didn't come back.

DNS is NOT working - its getting minnie out of cache, but nothing else ie


[TiVo [p0] ~]# wget -v -t 1 http://www.abc.com.au
--21:31:46-- http://www.abc.com.au/
=> `index.html.1'
Resolving www.abc.com.au... failed: Host not found.
[TiVo [p0] ~]#



Thoughts anyone? The tivo's have been running for years, so it's got to be something to do with the isp... Though I did change one thing locally, my home network used to be 10.0.* is now 192.168.* as exetel are putting a 10.0.* in their route (!) and I change mine just to make sure that wasn't causing the problem (tivo's picked it up fine, and I can telnet to them , and use tivoweb).

[petestrash]

What does 'route' show?

do you know what software version you are using? the newer releases are better able to cope with these sorts of issues automatically.

Peter.

[ian]

Hi Peter

From /etc/oztivo_version - "1.6.2-20070128" which I think is the latest.

This is doing a route on the tivo -


[TiVo [p0] ~]# traceroute -n 131.244.9.101
traceroute to 131.244.9.101 (131.244.9.101), 30 hops max, 40 byte packets
1 192.168.34.138 1.028 ms 0.857 ms 0.816 ms
2 220.233.1.149 36.171 ms 35.603 ms 35.699 ms
3 220.233.0.241 35.839 ms 37.520 ms 42.611 ms
4 10.0.1.2 34.545 ms 35.337 ms 34.605 ms
5 220.233.2.34 35.421 ms 34.594 ms 36.457 ms
6 203.202.84.65 36.363 ms 37.340 ms 37.918 ms
7 61.88.240.117 210.637 ms 247.539 ms 222.010 ms
8 59.154.4.2 49.383 ms 53.625 ms 51.376 ms
9 203.22.124.33 55.339 ms 54.020 ms 53.246 ms
10 203.13.70.21 56.109 ms 57.047 ms 54.463 ms
11 203.13.70.226 57.566 ms 59.712 ms 53.276 ms
12 203.144.10.94 54.483 ms 55.658 ms 54.664 ms
13 192.48.109.2 55.522 ms 54.375 ms 53.267 ms
14 * * *
15 * *


And this is from the PC -


tracert 131.244.9.101

Tracing route to minnie.tuhs.org [131.244.9.101]
over a maximum of 30 hops:

1 35 ms 1 ms 1 ms home.gateway [192.168.34.138]
2 37 ms 35 ms 38 ms 149.1.233.220.exetel.com.au [220.233.1.149]
3 37 ms 36 ms 35 ms 241.0.233.220.exetel.com.au [220.233.0.241]
4 34 ms 35 ms 35 ms 10.0.1.2
5 48 ms 34 ms 36 ms 34.2.233.220.exetel.com.au [220.233.2.34]
6 37 ms 35 ms 36 ms Vlan366.o2gsc76f03.optus.net.au [203.202.84.65]

7 50 ms 50 ms 49 ms 61.88.240.117
8 48 ms 49 ms 51 ms uecommltdvic.14vfc76f06.optus.net.au [59.154.4.2
]
9 55 ms 53 ms 55 ms briswi-lsr1.onthenet.net [203.22.124.33]
10 57 ms 53 ms 55 ms gcva-gw-fe4-0.OntheNet.net [203.13.70.21]
11 53 ms 53 ms 53 ms fast1-1-gcva-sr1-gw2.OntheNet.net [203.13.70.226
]
12 54 ms 56 ms 53 ms bond-gw1.OntheNet.net [203.144.10.94]
13 54 ms 56 ms 53 ms bdr0-hssi0.mgt.bond.edu.au [192.48.109.2]
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.


Does that help?

[petestrash]

Can you just try 'route' on the TiVo, not traceroute.

also clear tclient.log and force a daily call, then post the results here.

Peter.

[ian]

route -


[TiVo [p0] ~]# route
route_info, afname=inet, options=33
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.34.0 * 255.255.255.0 U 32767 0 0 eth0
default 192.168.34.138 0.0.0.0 UG 0 0 0 eth0
[TiVo [p0] ~]#


192.168.34.138 IS my adsl router.

With the tclient.log - I found a tclient file (no .log) in /var/log, is that the one you mean?

If so, blanking it and running a 'phone now' from tiviweb resulted in there being nothing in the file...

Ian

[ian]

And a manual wget works OK ie


wget http://131.244.9.101/tivo-static/listings/VI03001_13913-13927.slice.gz

returns the slice file..

[petestrash]

route is fine.

I have never seen an empty tclient log file, after forcing a daily call. what was the status shown in TiVoWebPlus?

I think your main issue may be DNS resolution. Try the wget again using minnie.tuhs.org instead.

Peter.

[petestrash]

Ian,

Have you had any PC issues since since swapping to ADSL2? or is it only the TiVo that is having issues.

Peter.

[ian]

So the file I should be looking in is /var/log/tclient ? Yep, empty.

From tivoweb, it gets up to 'connecting...' and will eventually time out and say failed.

DNS is definitely down (which I mentioned in my first post), but it obviously has the address for minnie.tuhs.org saved ie


[TiVo [p0] /tmp]# wget http://minnie.tuhs.org/tivo-static/listings/VI03001_13913-13927.slice.gz
--09:47:23-- http://minnie.tuhs.org/tivo-static/listings/VI03001_13913-13927.slice.gz
=> `VI03001_13913-13927.slice.gz.2'
Resolving minnie.tuhs.org... done.
Connecting to minnie.tuhs.org[131.244.9.101]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 79,928 [text/plain]

0% [ ] 79,928 0.00G/s ETA 00:00

Last-modified header invalid -- time-stamp ignored.
09:47:27 (0.00 GB/s) - `VI03001_13913-13927.slice.gz.2' saved [79928/79928]

[TiVo [p0] /tmp]#


As for other adsl issues, I've had trouble with my VPN to work - though I suspect it is at the work end as the server is stuffed.. And it has worked since I swapped, it just keeps on dying.. The work server is being replaced soon, so hopefully that will be fixed.

Apart from that, just the tivo stopping to download..

And thanks a lot for the help!

Ian

[petestrash]

So the file I should be looking in is /var/log/tclient ? Yep, empty.

From tivoweb, it gets up to 'connecting...' and will eventually time out and say failed.

I'm a bit miffed, TiVo should be writing to /var/log/tclient each time you try a daily call. I've never seen an empty tclient.

tclient is the log we rely on to troubleshoot daily call issues.

not sure where to go from here.

Peter.

[ian]

I could wipe the disk an do a clean install... Could anything be locking the tclient file?

[ian]

I tell a lie, the tivo just needed to be rebooted and then it was writing to the tclient file... One of my unix friends told me to do that, as he said it might be a locking problem.

OK, tclient file attached (renamed to tclient.txt) (I did a reboot, the a forced phone call which failed..)

Ian

[petestrash]

Feb 5 19:49:24 (none) comm[150]: Uploading HTTP Header for modLog of /var/log/svclog: POST /tivo-service/mlog.cgi HTTP/1.0^M Content-Length: 13706^M ^M
Feb 5 19:52:21 (none) comm[150]: XferRqst timeout waiting to read

This is what we typically see when a newer transparent proxy is involved.

Peter.

[petestrash]

Try tracetcp again with a server further from you.

EG.
C:\>tracetcp ftp.download.com:ftp
and
C:\>tracetcp ftp.download.com:http

Doesn't matter if it doesn't complete. Post the first few lines here if different.

Peter.

[ian]

Interesting, it is a bit different. Works for http, doesn't for ftp.



>tracetcp ftp.download.com:ftp

Tracing route to 216.239.120.101 [c18-gdl-ftp3.cnet.com] on port 21
Over a maximum of 30 hops.
1 22 ms 2 ms 2 ms 192.168.34.138 [home.gateway]
2 * * * Request timed out.
3 * * * Request timed out.
Terminate Event Occurred.

>tracetcp ftp.download.com:http

Tracing route to 216.239.120.101 [c18-gdl-ftp3.cnet.com] on port 80
Over a maximum of 30 hops.
1 2 ms 2 ms 2 ms 192.168.34.138 [home.gateway]
2 39 ms 37 ms 35 ms 220.233.1.149 [149.1.233.220.exetel.com.au]
3 37 ms 37 ms 37 ms 220.233.0.241 [241.0.233.220.exetel.com.au]
4 38 ms 37 ms 38 ms 10.0.1.1
5 39 ms 37 ms 37 ms 220.233.2.38 [38.2.233.220.exetel.com.au]
6 39 ms 37 ms 39 ms 203.166.92.57 [359-ge-0-0-0.GW5.SYD2.ALTER.NET
]
7 40 ms 37 ms 37 ms 210.80.33.189 [0.so-0-2-0.XT1.SYD2.ALTER.NET]
8 188 ms 187 ms 185 ms 210.80.50.249 [0.so-4-3-0.IR1.LAX12.ALTER.NET]

9 189 ms 186 ms 186 ms 152.63.48.65 [0.so-5-0-0.IL1.LAX9.ALTER.NET]
10 191 ms 190 ms 190 ms 152.63.112.53 [0.so-6-0-0.XL1.LAX15.ALTER.NET]

11 191 ms 188 ms 188 ms 152.63.116.21 [0.so-6-0-0.BR1.LAX15.ALTER.NET]

12 265 ms 301 ms 303 ms 204.255.168.30
13 232 ms 232 ms 231 ms 12.127.3.218
14 * * * Request timed out.
Terminate Event Occurred.



So I suppose that means a proxy issue? How do I get around it? Or something wrong with my router setup, as it doesn't seem to be getting out of that? Though I haven't changed that - though it probably has downloaded a stack of new settings from the new isp connection...



Ian

[petestrash]

Try the C:\>tracetcp ftp.download.com:ftp command a few more times to see if you can get a better result.

Also ask exetel if there is a transparent proxy on port 80 of your ADSL2 service.

Peter

[ian]

Tried it (C:\>tracetcp ftp.download.com:ftp) a few times, it never works.

A tracert does ie


>tracert ftp.download.com

Tracing route to ftp.download.com [216.239.120.101]
over a maximum of 30 hops:

1 12 ms 1 ms 1 ms home.gateway [192.168.34.138]
2 76 ms 38 ms 36 ms 149.1.233.220.exetel.com.au [220.233.1.149]
3 46 ms 39 ms 39 ms 241.0.233.220.exetel.com.au [220.233.0.241]
4 44 ms 38 ms 36 ms 10.0.1.1
5 51 ms 35 ms 36 ms 38.2.233.220.exetel.com.au [220.233.2.38]
6 42 ms 41 ms 37 ms 359-ge-0-0-0.GW5.SYD2.ALTER.NET [203.166.92.57]

7 54 ms 49 ms 35 ms 0.so-0-2-0.XT1.SYD2.ALTER.NET [210.80.33.189]
8 190 ms 191 ms 206 ms 0.so-4-3-0.IR1.LAX12.ALTER.NET [210.80.50.249]
9 192 ms 199 ms 198 ms 0.so-5-0-0.IL1.LAX9.ALTER.NET [152.63.48.65]
10 198 ms 196 ms 187 ms 0.so-6-0-0.XL1.LAX15.ALTER.NET [152.63.112.53]
11 190 ms 187 ms 196 ms 0.so-6-0-0.BR1.LAX15.ALTER.NET [152.63.116.21]
12 195 ms 200 ms 187 ms 204.255.168.30
13 235 ms 228 ms 228 ms 12.127.3.218
14 230 ms 224 ms 226 ms tbr1.dlstx.ip.att.net [12.122.10.49]
15 229 ms 220 ms 220 ms gar10.dlstx.ip.att.net [12.122.100.41]
16 237 ms 228 ms 225 ms 12.87.121.22
17 232 ms 223 ms 223 ms c18-gdl-ftp3.cnet.com [216.239.120.101]

Trace complete.



With exetel, they are unlikely to answer the question if I post it, as they aren't exactly the best at customer relationships (they abused my for raising my initial problem with adsl2 (which turned out to be that they had given me an incorrect ip address...) so I suspect that they aren't going to answer...

Assuming there is one, how do I get around it? And why would it be on port 80 if it is the ftp tracetcp that is failing?

Ian

Ian

[Darren King]

With exetel, they are unlikely to answer the question if I post it, as they aren't exactly the best at customer relationships

Assuming there is one, how do I get around it?

Well you are in a catch 22. Best way to get around it is to get the OzTiVo server (minnie.tuhs.org) removed from their transparent proxy. If you reckon they won't help you then life with a TiVo gets a little difficult.

[petestrash]

From your tclient it appears to be a transparent proxy issue.

If it is a transparent proxy, then we need the IP address from exetel to work around it, or even better for them to exclude minnie.tuhs.org from it.

I don't know why tracetcp is failing for FTP, there should be no proxy for ftp traffic, only http.

Peter.

[ian]

I've posted a support request for exetel - but they are the 'zero service' isp ie great plans, good quotas, speeds good, but no customer support..

I'll post something up at whirlpool...

There is no way around it if we don't know what it is? Would it be one of the ones on the tracert? If we do find out what it is - is there a way around it that doesn't involve them (exetel) doing anything (as there is no chance of that)?

I mean, it does work when I do a manual wget of the slice, and then load it in and do an index update.. If that works, why can't the auto one work?

Ian

[ian]

Anybody have any suggestions? Exetel haven't responded (what a surprise) and I don't really want to junk my two tivos after all these years of using them..

Can someone point me to where in the code the download happens? As (surely:)) given the manual download works, can't I just change the script to do something like that? Or is it failing in the bit where it has to figure out WHICH file to download? - yep, I figure it isn't going to be that simple...

And I suspect the 'transparent' proxy is the 10.0.1.1 in the tracert.

Ian

[petestrash]

I don't have any experience in disassembling tivoapp which controls the download. perhaps you should try @ dealdatabase.com instead.


If you suspect 10.0.1.1 being the proxy. then add the following line to /hack/etc/emuProxyZA.conf

proxyIP=10.0.1.1

Reboot and force another call.

Peter.

[ian]

Just tried that, call still failed... I'm in a brand new 12 month contract with Exetel, so am stuck....

Should I try every tcpip address that comes up in the tracert in the proxyIP field?

Ian

[petestrash]

You can try it.

But being a "transparent" proxy the actual IP may not show in tcpip.

We still do not have a sure fire way of unmasking transparent proxies.

Does ntpdate -v -b 132.163.4.103 work for you?

Peter.




Peter.

[ian]

Yep, it worked ie -


[TiVo [p0] ~]# ntpdate -v -b 132.163.4.103
oztivo ntpdate $Revision: 1.21 $
Our postcode is 03000. Found matching timezone Australia/Melbourne
Offset 39600 for timezone EST, daylight saving is 1
Connecting to time server 132.163.4.103
Setting the local time to Sun Feb 17 15:43:34 2008
[TiVo [p0] ~]#

And that is the correct post code and date/time.

Thoughts?

Ian

[petestrash]

Just checking, some people also have the ntp call blocked.

But you are ok.

Peter.

[ian]

One thought I've had is to point the tivo at an external proxy server. Not much point running one on my PC (as it then will go through the same path), but I'm currently looking around the web for an open one.. Then I'll give it a go..

Do you think that's a good idea? Can I just add it as proxyIP= ?

Ian

I just tried it, and when I tried the phone function came back pretty quickly with a "Failed. Service unavailable", so I suspect 1) it isn't using the external proxy, or 2) the external proxy isn't working , or 3) the external proxy doesn't like the request...

Ian

[petestrash]

I don't think it would make much difference, because the call is still going through port 80, so will still be stopped by the exetel/optus proxy.

Peter.

[ian]

I just tried a different proxy - you never know, exetels proxy might only doing certain things and might not always be 'on' - and it worked! Data came down, and I'm currently 62% through Loading Data!

So maybe this approach will help other people.. I agree it might not last, but at least it's something.

I've been corresponding with exetel again today, hopefully a better solution soon.. Given that using an external proxy fixed it, you'd agree that it's definitely a proxy issue?

Ian

[petestrash]

Yes, but it may be Optus's not Exetels.

Peter.

[ian]

I suspect so to (that it is an optus proxy) as exetel say they don't have one (which may, or may not, be true). Doesn't matter for the moment, as it is working. So let anyone else with a proxy issue know that they should try this as well..

Ian

[petestrash]

Can you tell us which proxy you have found sucessful.

Peter.

[ian]

I don't want to put it up here as I don't want the address too public :-) but i'm happy to pm it to anyone who needs it.

Alternatively, I got it from here http://proxy-list.org/en/index.php - generate a list and try a few until it works....

- and I wouldn't trust an open proxy for any of my other stuff, but who cares if someone is listening in to the tivo slices! :-)

[petestrash]

Thats cool,

Thanks,

Peter.

[ian]

had it happen again - but swapped to another proxy and it worked fine...

Ian

[petestrash]

Unfortunately that's the problem with open proxies, they can disappear at anytime.

Hopefully you get at least another couple of Months, a small price to pay for having automatic downloads :).

Peter.