Daily call fails - service unavailable

[gley]

Hi. First time post and humbly looking for help!
A couple of weeks ago, my TiVo froze up. I had to disconnect power and restart. Since then, daily calls have failed with the message "service unavailable". I re-imaged and tried again - initial setup completes, but subsequent daily calls still fail. I changed to a different ADSL gateway (Netcomm NB6Plus4W) and had my ISP (aaNet) remove me from their transparent proxy. I can ping and telnet the TiVo and get TiVoweb up. The wget tests listed in the forum seem to work okay (I'm not too literate in reading the results, but the web page saved, etc.) but STILL no daily call success. Sorry to bother you guys, but I think I've just about reached the end of the road with what I can do unassisted. Any ideas?

Thanks in anticipation.

Geoff

[petestrash]

Still sounds like a proxy issue.

Force a call using TWP and then copy the part of tclient.log that show the time you made this last call.

Post the results here.

Peter.

[gley]

Thanks for your help, Peter. Attached is the log file.

Regards,
Geoff

[petestrash]

The problem lies here:
Sep 3 10:08:22 (none) comm[155]: Uploading HTTP Header for modLog of /var/log/svclog: POST /tivo-service/mlog.cgi HTTP/1.0^M Content-Length: 8812^M ^M
Sep 3 10:11:19 (none) comm[155]: XferRqst timeout waiting to read

which looks like a proxy issue.

Peter.

[gley]

Thanks again for your help. Any suggestions as to what I might do about it? The tests suggested seem to show no proxy (they did before my ISP removed me from it) so I'm at a bit of a loss as to how I should proceed.

Regards,
Geoff

[petestrash]

If your ISP admits to a proxy being in place and can give you it's address, we can work around it. Without that info you are stuck.

I am trying to come up with newer tests that can prove the existance of a proxy and show it's details. The current tests are not reliable.

Peter.

[petestrash]

There is another test I would like to try.

This uses your PC, not the TiVo.

Download WinPcap_4_0_1.exe (http://www.winpcap.org/install/bin/WinPcap_4_0_1.exe) to any location and install it.

Then download tracetcp-0.99.4beta.zip (http://optusnet.dl.sourceforge.net/sourceforge/tracetcp/tracetcp-0.99.4beta.zip) to any location and then unzip it into your root directory (c:\).

Open a command prompt and make sure you are in the root directory (cd \), then run the following commands.

C:\tracetcp ftp.monash.edu.au:ftp

C:\tracetcp ftp.monash.edu.au:http

You will get a number of lines displayed over a period time after each command. Once complete copy all the outputs and post them here.

Peter.

[gley]

Peter, seems you were right. It shows a proxy. What next?

Geoff

[petestrash]

I'm glad I'm not going nuts here...

Ok, send your ISP the tracetcp file and ask them to exclude minnie.tuhs.org (131.244.9.101) from the proxy as it is blocking your TiVo.

Peter.

[petestrash]

Let us know how you get on, if they are stubborn there are other things we can do.

But I would much rather get the issue fixed for all members rather then fixing each one individually.

Peter.

[gley]

Hi again Peter,

Thanks for being so generous with your time. What do you make of this response?

"Good Afternoon

Thank you for your email.

That traceroute does not show a proxy.
The only proxy we run operates on port 80 which is web traffic only

Regards,
Andrew Wallace
Technical Support
aaNet Communications"

Regards,
Geoff

[Darren King]

I'd be opting out of the proxy. aaNet appears to use some dodgy proxy servers if you go by the comments posted on Whirlpool. A search with the word proxy showed this list of threads:

http://forums.whirlpool.net.au/forum-search.cfm?s=181015&r=9097003

At lest one of them has the procedure needed to opt out.

[petestrash]

That traceroute does not show a proxy.
The only proxy we run operates on port 80 which is web traffic only

Doh...

The traceroute does show a proxy and it is on port 80 as they say.

It is the proxy on port 80 you need to opt out of as Darren has already suggested.

As well as that process, try the following:

Enter the following commands at the putty prompt:

echo " " >> /var/hack/etc/emuProxyZA.conf
echo "proxyIP=proxy3.mel.dft.com.au" >> /var/hack/etc/emuProxyZA.conf
echo "proxyPort=80" >> /var/hack/etc/emuProxyZA.conf
reboot

Your TiVo will reboot, then see if you can do a daily call.

Peter.

[gley]

Thanks Peter,

Here's my latest correspondence:

"You are not being proxied, the traffic on port 80 goes to the proxy which is also a router but as your IP is in the exclusion list it just routes your traffic and does not send it through the proxy cache."

Anyway, I did as you suggested re the .conf file (thanks for putting it in vege language) and the call seemed to go through OK. Then the TiVo crashed during the night (irretrievably) so I have reimaged again and made the updates again. The initial setup has gone well, so I'm just waiting until tomorrow to see if all is as it should be. Looks as though the solution is at hand.

I'll let you know the outcome.

Regards,
Geoff

[petestrash]

Hope it works ok for you. If it does then it proves your ISP has not excluded you or minnie from the proxy, despite what they have said.

Peter.

[gley]

Hi again,

Well, after re-imaging and being very patient over the weekend, I now have a different problem. My TiVo thinks it's June 16 1938. If I reset the time it sets it back after a call. Additionally, forced calls seem to fail during the "loading data" phase. The setup calls worked fine (or seemed to).

Thanks for your continued help.

Geoff

[petestrash]

Make sure port 37 is open on your router.

This is used to set the time

Peter.

[gley]

Port 37 appears to be open, according to the check on www.canyouseeme.org. Is there another test you suggest? Could this be something else with my ISPs implementation? Maybe my TiVo has a fault (would a flat battery do it)?

Geoff

[Darren King]

Maybe my TiVo has a fault (would a flat battery do it)?

A flat battery would only cause the TiVo to lose the current set time/date when mains power is removed. So long as the power is connected and you simply issue the reboot command it will hold the time.

I'm assuming you have tried rebooting a couple of times and forced another daily call? TiVo's do have a weird time on occasions after initial completion of Guided Setup that is fixed by a couple of reboots after indexing has finished.

Of course yes the TiVo *may* have a dud clock chip but very doubtful. Not had one with this problem yet.

If you are still not convinced then a new CR2032 battery will soon tell you. Same battery as used on a PC motherboard so not that hard to get or expensive.

[petestrash]

try:
ntpdate -v 132.163.4.103

and post your result.

Peter.

[gley]

Thanks for the suggestions. I have rebooted countless times, but the date remains at either 1938 or 2000.

[TiVo [p0] ~]# ntpdate -v 132.163.4.103
oztivo ntpdate $Revision: 1.21 $
Our postcode is 03000. Found matching timezone Australia/Melbourne
Offset 36000 for timezone EST, daylight saving is 0
Connecting to time server 132.163.4.103
Setting the local time to Wed Sep 12 12:49:41 2007

But again, as soon as a call is made, once "setting clock" is displayed, we're back to 1938.

Geoff

[petestrash]

can run the ntpdate command again as above, then force a daily call, then post your otclient log here.

Peter.

[gley]

Thanks Peter, I'll do that. Last night I ran the guided setup again (out of frustration) but still have the same problem. Should I leave it for a while before I do as you suggest, or is it okay to do it now?

Geoff

[gley]

It's been going 24 hours now, so I just did it (bad idea?).

TiVo Web doesn't actually show that the process completes, it just stops on "Loading Data... 98%, 1 min". Please find the log file attached. Thanks and regards.

Geoff

[petestrash]

Have you run the update-oztivo -y command since re-imaging ?

Peter

[gley]

Yes. First thing I did.

[petestrash]

can you try:

find . / -name ntpdate*

Peter

[petestrash]

Are you still using the settings from post 13 since you rebooted. If you are, enter the following commands (in red) at the putty prompt:

[TiVo [p0] ~0]# echo "ntpIP=132.163.4.103" >> /var/hack/etc/emuProxyZA.conf
[TiVo [p0] ~0]# reboot

Once your tivo is rebooting, you can close putty.

Once back up, try the ntpdate command again and then paste the results here.

Peter.

[gley]

Thanks again for your generosity, Peter. Here is the result from "find . / -name ntpdate*":

[TiVo [p0] ~]# find . / -name ntpdate*
/bin/ntpdate
/bin/ntpdate.orig

Geoff

[gley]

And here is the result after "echo "ntpIP=132.163.4.103" >> /var/hack/etc/emuProxyZA.conf" then "reboot"

[TiVo [p0] ~]# ntpdate -v 132.163.4.103
oztivo ntpdate $Revision: 1.21 $
Our postcode is 03000. Found matching timezone Australia/Melbourne
Offset 36000 for timezone EST, daylight saving is 0
Connecting to time server 132.163.4.103
Setting the local time to Fri Sep 14 00:53:33 2007

Regards,
Geoff

[petestrash]

ok, try a daily call now.

Peter.

[gley]

Call causes time to reset to 1938

[petestrash]

Can you try:
rw
echo "132.163.4.103" > /etc/ntpdate_local_ip
ro
ntpdate -v -b

Then paste the results here, if successful try another daily call.

If still no good post the contents of your /var/hack/etc/emuProxyZA.conf file.

Peter.

Peter.

[gley]

Peter,

That's done it! Thank you SO much for your help. It's very kind of you to spend so much time helping strangers. Can you suggest what may have been the cause? Since I had been operating fine for months, is it safe to assume that my ISP made a change of some sort?

Thanks again. Hope you have a great weekend.

Regards,
Geoff

[petestrash]

Your ISP has added a transparent proxy which was blocking the request for time servers from minnie, so when the command ntpdate was run during the daily call no servers were specified.

We have gotten around this by specifing a default if no others were found.

I am surprised that you were still able to download slices ok.

Peter.

[gley]

So, if they had really removed me from the proxy as they said they had, we could have avoided all this? Did they need to exclude minnie as well? Sorry if these are dumb questions, but I have a support incident open with them, and I'd like to get it sorted so that others don't have to go through the same thing. How would you describe to them what needs to happen at their end (and why) so that they don't just brush me off again? I'll leave you at peace after this!

Thanks and regards,
Geoff

[petestrash]

Yes if you were excluded this would not be necessary.

Your ISP needs to exclude minnie.tuhs.org( 131.244.9.101) from their port 80 transparent proxy.

Can you also try the following commands on your TiVo.

traceroute -p 80 ftp.monash.edu.au
traceroute -p 21 ftp.monash.edu.au

Peter.

[Darren King]

but I have a support incident open with them, and I'd like to get it sorted so that others don't have to go through the same thing.

Good luck with that. I doubt they will lift a finger.

[gley]

I fear you may be right, Darren. They just say that my connection no longer uses the proxy. Attached are the traceroutes you asked for, Peter.

Geoff

[petestrash]

Your routing looks clean now.

Hopefully no more surprises.

Peter.

[gley]

All seems to be working fine, Peter. Case closed, I think. Thanks again for your help.

[petestrash]

Great.

Peter.

[gley]

Missed some of my usual shows this week. On investigation, TiVo had not made a successful call since September 27. So following the procedure from last time, I established that I am now on proxy4 (rather than proxy3.mel.dft.com.au). How annoying! Especially after my ISP maintains that I am NOT being proxied and that none of this could POSSIBLY be their fault. Anyway, I have edited the emuProxyZA.conf file to reflect the change and everything now works again. I have communicated my displeasure to my ISP (aaNet) and will post their response if it's worth reading.

[gley]

Just one more thought... If, as I anticipate, aaNet are less than helpful, is there a way I can have the TiVo alert me after ONE failed call rather than the default number?

[petestrash]

Only by writing a program to do that for you.

But realistically checking your TiVo once a week each Saturday should give a weeks worths of data.

Glad you have learned enough to solve to problem yourself, I haven't heard anyone else who is being proxied need to adjust their setting due to a change of proxy.

So hopefully it's rare issue.

Peter.

[gley]

They tell me that the proxy is subject to change at any time for load balancing, which I suppose means that my TiVo is subject to fail to connect at the same time. Here is the tech support reply. This one has an air of finality to it. I would be interested in your comments. Thanks and regards, Geoff.

"Unfortunately the problem you are experiencing is not on our end, as you demonstrated when adding the proxy server to the conf file it allows it to work. The proxy server itself can resolve MTU issues which could be causing the problem here, basically the Proxy connects to a large pipe for it's data so MTU conflicts with servers outside our network wont be an issue, but if you bypass the proxy then you're being routed straight onto that connection and thus problems with MTU, setup of your hardware and things like that can cause the problems you are experiencing. Unfortunately it is not something within our control to resolve as the issue isnt coming from our end. As explained to you you are simply having your data routed directly to you it's not passing through the proxy server first. When you add the proxy server to the conf file it puts your connection back to data going through the proxy server first."

[gley]

Finally, this:

"Unfortunately there is nothing more that we can do to resolve the issue here, numerous things can be causing the problem issues from the server you are connecting to for example, as you are getting an uncached connection to the internet it is not a problem coming from our end. I would suggest talking to the people that run the server for your TiVo or you can also try with the aaNet forums. Adding the proxy server is mearly just relieving you of issues continuing to happen. As stated it is not a problem being caused on our end. If you require any further assistance as suggested try talking to the people that run the server or the aaNet members forums"

[Darren King]

LOL! Full marks to them for shifting the blame.

You have to remember that it is not in the interests of all these el cheapo ISP's like aaNet, TPG and others to not proxy as it saves them a heap in international traffic data costs. It maximises their bottom line which is good for them but not for the end user, and is partly the reason why these ISP's can offer lots of monthly data allowance as the proxy offsets this.

There are still proxy issues with aaNet. You only have to go to www.whirlpool.net.au and do a search for "proxy" in their aaNet forum area to see that there are people not enjoying the experience of being behind an aaNet proxy.

[petestrash]

Unfortunately the only options are:
1. aaNet to exclude minnie from all it's proxies
2. Stay with aaNet and keep fiddling with the conf file.
3. Leave aaNet for a better ISP.

Peter

[gley]

Option 3!

I can't even be bothered replying to this email. They have still never suggested why it might have suddenly stopped working, or (as I asked in my last - and final - email) why it works through other ISPs. The "cracked record" response is totally infuriating! Anyone suggest a reasonably priced ISP who won't give me this grief?

"I cannot tell you why it stopped working at all as it's something that is beyond our control to cause. As it has been explained to you we have no issues that would cause problems with this type of connection, you are getting a direct connection to the server for your TiVo it is not going through the proxy at all. When you add the proxy server to the conf file it fixes what ever technical issue you have which is not from our end as if it was even that would not work. Unfortunately there is nothing more I can do to assist you, I recommended you speak with the support people for the server and to consult the aaNet members forums as it is clearly an issue that is not coming from our end as has been explained."

[JasonM]

gley,

aaNet have been as helpful as we can in relation to your issue - however you seem not to understand what we are saying.

1. You had a problem with TiVo and asked to be removed from the Transproxy.
2. aaNet granted your request - and you were assigned an IP address that is part of the non-intercept list.
3. You still experienced problems with your TiVo and asked our support team
4. Our support team provided you technical advice in relation to your data(tcptrace, etc).
5. You stated to us that when you set a proxy in the TiVo config, it works.
6. Evidently the issue is not with our proxy clusters, and never has been.

You are a clean direct connect to the internet - there is nothing aaNet is doing that can prevent you from being able to access any site, HTTP or otherwise.

You need to investigate other reasons why your system is not working - but that is not something that we can help you with. We provide an internet connection - not a TiVo support service. If you feel you need to find another provider to resolve your issue then that is your choice.

Jason

[gley]

I wonder if Jason read anything else here...

I don't fully understand all the techical stuff here, but I know this: my TiVo suddenly stopped working with no changes made at my end. My TiVo and router worked when tested at a friend's house on their (non aaNet) ADSL connection. So how can aaNet continue to claim it's "not their end"? Maybe they're right and I just "don't understand". Here's a little more of their helpful dialogue.

"You need to investigate other reasons why your system is not working - but that is not something that we can help you with. We provide an internet connection - not a TiVo support service. If you feel you need to find another provider to resolve your issue then that is your choice."

You can be assured I'll be doing just that.

[petestrash]

Perhaps a PM to Jason, if he is involved with aaNet.

The only reason the TiVo stopped working was they changed the cache server (that you are not supposed to be using anyway) so the name in our bypass area was wrong.

If you are direct you should not be going to any of the pooled cache servers.

Unfortunately aaNet still do not seem to understand the problem.

They have told you that you are now direct, but you had problems only when they changed proxy servers. When you reconfigured for that server it worked again.

It is obvious to everyone, but them that you are still being proxied regardless of what they claim.

I don't have any solution for you other then what I have said a couple of posts up.

Peter.

[gley]

So here's how I resolved the problem permanently: dumped aaNet.

I wouldn't really be too upset with them if not for the arrogant and dismissive tone of their "support" (which really just amounted to repeating the same thing over and over while ignoring my questions). The TiVo works just fine now (via Netspace) and I even get a news server. So, to borrow from Jason's point #6, "Evidently the issue is with aaNet's proxy clusters, and always was".

Darren, you were right on the money!

Thanks again for your help, Peter.

Geoff