Looks good

System looked good all day.

22.04 was watching system with reasonably low ping times when suddenly all pings timing out.
Checked Draytek, it was doing 8000 packets every 10 sec on one line and 5000 packets every 10 sec on the other. Definatly need policy based routing!

Checked at 20.25
Longest ping times I've ever seen on the system mainly 50-80mS, but not too many ping timeouts. Draytek reports about 3000 packets every 10 seconds on both "Be" lines. I think this must be a user downloading a big file.

Checked OK 08.00
Checked OK 10.00
Checked OK 11.51

Checked system throughout the day, all running well. Mainly single digit mS times in ping (Loadbalancing over both Be lines).

Checked system 20.20
WAN1 had stopped, WAN2 still going, (Load balancer made users experience impossible) used remote power cycle, System running OK.

09.00 Tue checked system, WAN1 (only WAN line) had failed. It restarted with remote power cycle.
10.30 arrived on site, discovered WAN2 switched off and disconnected.
Had to log into WAN2 router and reconfigure it, System now loadbalancing WAN1 and WAN2.

Checked service most of the day - all OK, by 21.30 line so busy some pings failing, not too bad generally get 20 or 30 pings through before you get a failure.

Checked Mon 07.30 OK
Checked Mon 12.00 WAN1 failed, remote power cycle restored it.

Checked OK 20.01 watched off and on over the evening.
Checked 22.48 WAN1 line failed, used remote power cycle, returned back to 4000 packets per second.

Checked OK 18.14 (WAN1 only) 4000 packets every 10 seconds. Ping responses mainly single digit - ie customer experience good.

Checked OK at 22.30
Checked OK at 12.18

Checked system 19.00, it had been using just one "Be" line (other one would not return with remote power cycle). The one working "Be" line had failed, remotely power cycled it - it returned.
(re booted Loadbalancer). System back working on one line.

Sat morning one "Be" line doing about 4000 packets every 10 seconds. Some timeouts.

Checked system Fri night, Spitfire line failed under load, one "Be" line down. Ran system on one "Be" line. When checking to www.bbc.co.uk there were ping timeouts every 4-5 seconds, so users would have been getting a slow service.
There are two failure modes of "Be" routers, one where the router comes back after power cycling - one where it does not. At the moment we have the latter. Neither are acceptable.

I think the failures are down to the routers. I rang "Be" tech support to check if either failure mode is known / common by tech support, thinking they might be able to indicate whether it is worth replacing the "Be" router with the same model (this never normally happens) or with a more reliable brand of router (this happens whenever the line is under a lot of stress).

One thing occurs to me that if the BBCi player becomes popular amongst Charing cross residents we will be have to switch back to some sort of policy based routing.

Checked system again this morning seemed to be running OK.

I understand there were some problems this last week. When a "Be" Internet line went down and would not reset.
Kugan left the system load balancing between a 16Mb ADSL and an SDSL.
Both the Be lines and he spitfire line are proving troublesome.
I checked the system and had to reset it again.

System running OK, about 2000 packets every 10 sec on WAN1 same on WAN2.

Checked system, WAN2 running at about 2000 packets every 10 seconds. WAN1 not passing packets. Rebooted Loadbalancer with WAN1 WAN2 fixed 50% 50% rather than auto weighting.
WAN1 still not passing packets. Rebooted WAN1 WAN2 Routers, system running normally.

System checked OK, running at about 1000 packets every 10 seconds for WAN1 & WAN2

Checked system, looks OK, running about 1000 packets every 10 seconds out of WAN1 & WAN2

Graham phoned to say user reported system running slowly. He has checked Internet access to web pages, he said it seemed fast.

Quent checked the system, WAN1 and WAN2 running about 1000 packets every 10 seconds. Looks fine.

Rebooted Loadbalancer because of imbalance between WAN1 and WAN2, set auto weighting.
System still putting more traffic down WAN1 than WAN2.
Not rebooted WAN Routers this moring.

07.55 checked the system, Internet seemed to respond OK, but as last night much more traffic on WAN1 than WAN2.

Checked System at about quater past midnight. Stats shows data passing on WAN2 but not on WAN1. Since the Loadbalancer can not detect failure of WAN1 this would have given poor to no service to the users. Number of packets passed by the interface showed that the system must have been down for several hours. Reset the system using the new Router reset device. System back up again about 25 past midnight.

Fitted device so can remotely reboot WAN1 & WAN2.
Checked Loadbalancer by logging in and doing ping to www.bbc.co.uk
All three WAN ports failed ping test.
Had to reboot Loadbalancer to set it back to balancing between WAN1 and WAN2.
Load balancer started pinging out on WAN1 (as expected) failed to ping out on WAN2 (as expected because Be lines both in same subnet).
Loadbalancer would not ping out on WAN3 for a few minutes. Eventually WAN3 pings restarted without rebooting the router. It could be too much load on WAN3 made it fail.

Checked System, no packets out of WAN1 or WAN2, rebooted Loadbalancer to use WAN3 only.
Logged in again. WAN 3 now passing 2000 packets every 10 sec.
Could not ping from WAN1,2 or 3. Could be that WAN3 is too slow to respond in time as so busy?

WAN1 Rx passing approx 1,400 packets every 10 seconds. Up time 3 days 12 hrs. 43,620,505
WAN3 Rx passing approx 2,500 packets every 10 seconds. Up time 3 days 12 hrs 47,096,512
Tx and Rx fairly well matched on each line.
Only been loadbalancing between WAN1 and WAN2 (16Mb lines)

WAN1 Rx passing approx 1,400 packets every 10 seconds. Up time 2 days 22 hrs. 36,184,505
WAN3 Rx passing approx 2,500 packets every 10 seconds. Up time 2 days 22 hrs 39,251,666
Tx and Rx fairly well matched on each line.
Only been loadbalancing between WAN1 and WAN2 (16Mb lines)

System working well.

System working well

System working well

System working well.

Charing Cross Staff Internet checked all day yesterday and this morning, all seem to be working, and working fairly fast.