Short Outage - Early Friday morning - Anyone else?

[JB]

I'm just attempting to check if one of my routers is developing a problem.

Did anyone else have a short outage about 3.00am this morning (Friday 27th May)? Especially in the north west.

I realise this may have been local to me or even the router itself but I'd be interested to know if anyone else had similar.

[jameshurrell]

Yep I monitor three lines, two in shropshire, one in bedfordshire, all three had a short outage in the early hours. Looks like an idnet blip because there is a line on the same exchange in bedfordshire with a different isp that was unaffected.

[Rik]

Are the lines 20CN or 21CN?

[Glenn]

I had the same in Berkshire

[Ray]

I had a brief loss of PPP at about 2:30am.

[tehidyman]

I am in Cornwall, presume this is the same phenomenon.

[Ray]

Yes, that's exactly the same as my graph.

[Glenn]

and mine too

[dmgeurts]

Same here in Hertfordshire from 2:20 till 2:35AM. My own monitoring showed it wasn't BT as it could reach the IDnet PE (first hop outside my network when doing a traceroute). I'm suspecting IDnet had a routing issue last night.

[Bill]

And another... South Oxon, FTTC.

[sobranie]

Ditto here on (01326)

[Bill]

Looking at the router log I get repeated instances of

Code Select Expand

  May 27 02:25:50  daemon  pppd[2622]: PPP: Start to connect ...
  May 27 02:25:50  daemon  pppd[2622]: PPP server detected.
  May 27 02:25:50  daemon  pppd[2622]: PPP session established.
  May 27 02:25:50  daemon  pppd[2622]: Using interface pppewan_1
  May 27 02:25:50  daemon  pppd[2622]: Connect: ppp_ewan_1 <--> eth0
  May 27 02:25:50  daemon  pppd[2622]: Couldn't increase MTU to 1500.
  May 27 02:25:50  daemon  pppd[2622]: Couldn't increase MRU to 1500
  May 27 02:25:50  daemon  pppd[2622]: Couldn't increase MRU to 1500
  May 27 02:25:50  daemon  pppd[2622]: PPP LCP UP.
  May 27 02:25:54  daemon  pppd[2622]: Remote message: CHAP authentication success, unit 4846
  May 27 02:25:57  daemon  pppd[2622]: Received bad configure-nak/rej:  03 06 d4 45 35 78 81 06 3e 06 26 7d 83 06 3e 06 26 7d
  May 27 02:26:00  daemon  pppd[2622]: Received bad configure-nak/rej:  03 06 d4 45 35 78 81 06 3e 06 26 7d 83 06 3e 06 26 7d
  May 27 02:26:03  daemon  pppd[2622]: Received bad configure-nak/rej:  03 06 d4 45 35 78 81 06 3e 06 26 7d 83 06 3e 06 26 7d
  May 27 02:26:06  daemon  pppd[2622]: IPCP: timeout sending Config-Requests
  May 27 02:26:06  daemon  pppd[2622]: Couldn't increase MTU to 1500.
  May 27 02:26:06  daemon  pppd[2622]: Couldn't increase MRU to 1500
  May 27 02:26:06  daemon  pppd[2622]: Connection terminated.
  May 27 02:26:06  daemon  pppd[2622]: Doing disconnect

Authentication server problem?

[jameshurrell]

Are the lines 20CN or 21CN?

Hi Rik, two lines are 20CN and one is 21CN. All had the outage.

[dmgeurts]

Looking at my PPPoE logs I noticed the following:

Code Select Expand

ppp: [wan_link0] Name: "bras-red10.l-nws"

instead of:

Code Select Expand

ppp: [wan_link0] Name: "telehouse-gw4"

Next to that I saw that while my PPPoE connection bounced a few times I was given wrong IP addresses. Incidentally the IP address I was given belongs to BT (ripe query on 81.146.178.2). So all in all something went wrong on IDnet's or BT's network (or the Radius realm forwarding) disrupting the connectivity to IDnet's gateway machines, hence connecting me to BT's network instead:

Code Select Expand

May 27 01:23:35 ppp: [wan] 172.16.68.167 -> 81.146.178.2
May 27 01:23:35 ppp: [wan] IPCP: LayerUp
May 27 01:23:35 ppp: [wan] IPCP: state change Ack-Sent --> Opened
May 27 01:23:35 ppp: [wan] IPADDR 172.16.68.167
May 27 01:23:35 ppp: [wan] IPCP: rec'd Configure Ack #7 (Ack-Sent)
May 27 01:23:35 ppp: [wan] IPADDR 172.16.68.167
May 27 01:23:35 ppp: [wan] IPCP: SendConfigReq #7
May 27 01:23:35 ppp: [wan] 172.16.68.167 is OK
May 27 01:23:35 ppp: [wan] IPADDR 172.16.68.167
May 27 01:23:35 ppp: [wan] IPCP: rec'd Configure Nak #6 (Ack-Sent)
May 27 01:23:35 ppp: [wan] IPADDR 0.0.0.0
May 27 01:23:35 ppp: [wan] IPCP: SendConfigReq #6
May 27 01:23:35 ppp: [wan] COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
May 27 01:23:35 ppp: [wan] IPCP: rec'd Configure Reject #5 (Ack-Sent)
May 27 01:23:35 ppp: [wan] IPCP: state change Req-Sent --> Ack-Sent
May 27 01:23:35 ppp: [wan] IPADDR 81.146.178.2
May 27 01:23:35 ppp: [wan] IPCP: SendConfigAck #183
May 27 01:23:35 ppp: [wan] 81.146.178.2 is OK
May 27 01:23:35 ppp: [wan] IPADDR 81.146.178.2
May 27 01:23:35 ppp: [wan] IPCP: rec'd Configure Request #183 (Req-Sent)
May 27 01:23:35 ppp: [wan] COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
May 27 01:23:35 ppp: [wan] IPADDR 0.0.0.0

[esh]

Looks like a total mess. Here is some excerpts.

Code Select Expand


May 27 02:23:10 mpd: AUTHPROTO CHAP MD5
May 27 02:23:10 mpd: Using authname
May 27 02:23:29 last message repeated 7 times
May 27 02:23:36 mpd: Name: "ESR8.Miltonkeynes3"
May 27 02:23:51 mpd: MESG: Authentication failure
May 27 02:23:59 mpd: MESG: Authentication failure
...
May 27 02:27:25 mpd: Name: "ESR8.Miltonkeynes3"
May 27 02:27:25 mpd: Using authname "<<omitted>>@gw5"
May 27 02:29:27 mpd: ENDPOINTDISC [LOCAL] 74 65 6c 65 68 6f 75 73 65 2d 67 77 32
May 27 02:29:27 mpd: Name: "telehouse-gw2"
May 27 02:29:59 mpd: Name: "ESR8.Miltonkeynes3"
May 27 02:29:59 mpd: ENDPOINTDISC [LOCAL] 74 65 6c 65 68 6f 75 73 65 2d 67 77 32
May 27 02:29:59 mpd: MP MRRU 1524
May 27 02:29:59 mpd: Name: "telehouse-gw2"
May 27 02:30:31 mpd: Name: "ESR8.Miltonkeynes3"
May 27 02:31:24 mpd: Name: "telehouse-gw4"
May 27 02:31:47 mpd: COMPPROTO VJCOMP, 16 comp. channels, no comp-cid
May 27 02:31:47 mpd: IPADDR 0.0.0.0
May 27 02:31:59 mpd: Name: "ESR8.Miltonkeynes3"
May 27 02:32:00 mpd: ENDPOINTDISC [LOCAL] 74 65 6c 65 68 6f 75 73 65 2d 67 77 32
May 27 02:32:00 mpd: Name: "telehouse-gw2"
May 27 02:32:28 mpd: Name: "ESR8.Miltonkeynes3"
May 27 02:32:28 mpd: Name: "telehouse-gw4"
May 27 02:32:30 mpd: <<IP>> is OK
May 27 02:32:30 mpd: IPADDR <<IP>>
May 27 02:32:46 check_reload_status: rc.newwanip starting


[Rik]

Hi Rik, two lines are 20CN and one is 21CN. All had the outage.

Thanks, James, that eliminates one variable.

[Rik]

I understand that a brief 'blip' caused connections to fail over to a second radius server, this became briefly over-loaded, leading to the inability to connect for a short period. That cleared itself and most people had re-connected within 15 minutes or so. There may be one or two who have stale sessions which a router re-boot should clear.

[Steve]

 Mine never reconnected without a reboot,although it was repeatedly trying for 4 hours.

[Rik]

Mine was fine, I wouldn't have known anything had happened if it wasn't for this thread.

[esh]

Are these radius servers under BT's authority?

[Rik]

I'm not sure, tbh. There definitely are BT radius servers, but whether they're reciprocated on the IDNet side I don't know.

[Steve]

I know that there was an issue with the MK BRAS yesterday but that was at midday.

[Rik]

There's 11 MUXes down in Yorkshire this morning due to a cable fault.

[Steve]

Or missing cable 

[Rik]

 

Shame if they thought they were getting copper and ended up with fibre.

[esh]

BT seem to have been up to an abnormally high amount of maintenance lately (anti-maintenance? unmaintenance?)

[Rik]

Trouble making?

[DorsetBoy]

BT seem to have been up to an abnormally high amount of maintenance lately (anti-maintenance? unmaintenance?)

They are trying Quantisation  on 20CN apparently .

[Rik]

They'll try anything except spending money.

[DorsetBoy]

They'll try anything except spending money.

I'm glad that I don't have any of this going on any more, LLU is soooooo much better 

[Rik]

 

Unless it's with Talk Talk. 

[DorsetBoy]

Unless it's with Talk Talk.  

I think their customers are OK till something goes wrong, that's when they regret changing over. Thankfully I have someone else to ear bend should the need arise.

Just noticed that it is 11 months today on LLU without a single ISP/supplier fault, the one issue was a fibre break due to a lorry taking out lines in an accident.

[cavillas]

ANything to do with This  Some problems going on.

[Rik]

Not as far as I know, Alf, this was a specific BT-to-IDNet problem.

[esh]

I had to endure Opal/TalkTalk for a year somewhere, that was pretty bad. I think it averaged about 1-3 hours of downtime per week, with every other month having a 6 hours+ fault.

[Rik]

Ouch.

[pctech]

I'm not sure, tbh. There definitely are BT radius servers, but whether they're reciprocated on the IDNet side I don't know.

I think the BT Radius servers check the bit after @ in the username (the realm) and direct the traffic toward the relevant ISP's hostlink provided of course that ISP has a contract with BT for that particular user's xDSL circuit.

That is my understanding anyway.

[.Griff.]

Only just spotted this.

Same thing happened on my FTTC connection. I just assumed it was me! 

[joe]

I wasn't aware of it but checking back now I appear to have suffered similar for three nights 25-27 May (Brentwood Exchange):-

25th: <a title="Broadband Ping" href="http://www.thinkbroadband.com/ping/share/c0fe872dbdf14b62011a51948ba69ed4-25-05-2011.html"><img alt="My Broadband Ping - bj" src="http://www.thinkbroadband.com/ping/share-thumb/c0fe872dbdf14b62011a51948ba69ed4-25-05-2011.png" /></a>

26th: <a title="Broadband Ping" href="http://www.thinkbroadband.com/ping/share/dd9b8d33540ab5adf820f257b07b777b-26-05-2011.html"><img alt="My Broadband Ping - bj" src="http://www.thinkbroadband.com/ping/share-thumb/dd9b8d33540ab5adf820f257b07b777b-26-05-2011.png" /></a>

27th: <a title="Broadband Ping" href="http://www.thinkbroadband.com/ping/share/2445a798da7ed174ad593c2081f10b39-27-05-2011.html"><img alt="My Broadband Ping - bj" src="http://www.thinkbroadband.com/ping/share-thumb/2445a798da7ed174ad593c2081f10b39-27-05-2011.png" /></a>

[DorsetBoy]

You can't post HTML links in the forum, you need to choose the "forum" code from the list at TB.


This is what you want.

Code Select Expand

27th: [url=http://www.thinkbroadband.com/ping/share/2445a798da7ed174ad593c2081f10b39-27-05-2011.html][img]http://www.thinkbroadband.com/ping/share-thumb/2445a798da7ed174ad593c2081f10b39-27-05-2011.png[/img][/url]


25th:




26th:




27th:

[Technical Ben]

Are those disconnects from you setting the router up? Or is it a sign someone trying to communicate in morse code Dorset? 

[joe]

Thanks for the advice and corrrection of my links DorsetBoy. TB has changed its links since the last time I up/ld a chart.

[DorsetBoy]

Are those disconnects from you setting the router up? Or is it a sign someone trying to communicate in morse code Dorset? 

What are disconnects    ....... they aren't mine Ben, I just corrected the links Joe posted earlier.

[JB]

Having started this thread, I apologise for not having contributed again. Family matters got in the way.

I finally found that my router was going faulty, in that after being on for many hours, it seemed to lose internet connectivity. The link to the DSLAM in the local exchange was OK but the internet layer above that seemed to be failing. It was only a cheap standby router and it is now packaged up for return to Amazon.

Like many people in the similar thread, I experienced the drop out at 3am in the morning a day or two ago, and yesterday afternoon like others, experienced what seems to have been some form of DNS error, lasting about 10 minutes.

Thanks to everyone for replying.