So last night I am made aware that the RC NUT website was down, ok, I checked the webserver .. no response, so I sent out a e-mail to support and didn’t tough much of it .. surely it would be up in 5-10 minutes

30 minutes later and no show, I started to get annoyed, told support to get a grip on things and since it was very late, I went to bed.

This morning I woke up early, about 6:00 AM to check it out, site was up, but no DB connectivity .. I have my DB on a separate server (thats the proper way a H-Sphere install is done) .. so I ping the DB server .. no luck, I then checked the IP of the webserver and .. i’ll be dammed, I was on a US server, not a UK server, I ping the MySQL server by name, and sure enough, it was there, but also in the US.

I changed my config files and removed the old US ip, the site was fully up .. except the site was 13 days old !!

Now, for 99% of people out there, this is not a biggie, but for a online shop with sometimes dozens of orders in one day .. this is a catastrophe .. over 100 order records are lost and all since we do inventory by the website stock level, a messed up inventory as well.

Upon further investigation, I found out that Relio’s (webhost) server provider in the UK seems to have decided not to pay their bandwidth bill, and the bandwidth provided cut them off, in essence rendering companies like Relio’s UK servers .. dead (notice, it was not Relio that did not pay their bill, but their server provider, UK Easily)

So I called the bandwidth provider (Uk Grid … all those UK names, they confused the hell out of me today), explain to them our situation and after some negotiations, they agreed to open up traffic to the server for 3 hours to allow me (and others) to access and backup their data out, that was commendable .. those guys where true stars in doing that, after I got the database back, I restored it and the website was up to date … gosh was I relieved.

Why was my site 13 days old to start with ? .. well .. simple, Relio backups my data every day to another server, and every week or two they backup that server to another server in the US .. ok, at first glance, not a bad setup .. but this experience taught me .. not good enough, I need more.

So, since Relio’s UK servers where dead in the water .. and no access to the backup server was possible, they had to do an emergency recovery from the US backups.

I will give Relio credit, to restore every UK account in the US servers, move DNS, remap ip’s, etc .. etc in less than 6 hours .. thats not bad, they did a good job, but it goes to show, no matter how good your first Tier is .. somebody up the ladder is bound to mess it up !!.

Thats the problem with the Web Hosting market these days, too many idiots working on it, we put out livelihoods in the hands of people who later turn around and mess it up for you .. and does not matter how big or small the company is, your are always at risk.

What is the solution ? well, nothing below 50K of hardware and a few K’s a month in fees, so sometimes .. you just have to pay up and shut up.

Thats what I am doing next, first I am going to hire a local hosting company (when I mean local, I mean 3 streets over), I feel the threat of a 300lbs+ guys walking into the office and breaking your nose is a good incentive to make sure they dont screw with you .. that can only happen if they are local, second, I am going VPS (Virtuozzo) .. my idea is to find a setup I can create a Virtuozzo Failover Cluster with a backup residing off-site.. third, I am setting up a mirror of all this on a VPS server in the US, with my DNS residing also offsite (easily re-routable) so if something happens to RIPE itself (The IP consortium of Europe) .. I can still be up, fourth … dont know fourth yet .. but it’s sure to pop up while we doing this.

Overboard, overblown, over-budget ? .. of course .. but piece of mind, i’ve been in this business for 10 years .. I have seen some huge companies working with a setup that would make anybody cringe, promises of redundancy are usually all hogwash, backups many times are simply not practical, yes, they will back your data, they just cant extract it back, customers are at huge risks all the time of loosing all their data, and for some, thats instant bankruptcy, I have try to give away control on how my data is stored and managed, but it’s proven once again, if I want it right, I have to do it myself.

Popularity: 100% [?]

I am so fed-up with BT .. everything I ever got from them fails.

Last year I decided to drop SKY .. they where charging me about £60 per month for SKY/+/HD .. so I decided, since now I had the AppleTV, to drop them and just get a PVR for Freeview instead.

Until I learned about BT Vision .. great concept, basically a Freeview PVR, with on-demand via Broadband .. the idea is great, so since I needed to have Broadband with BT in order to sign for Vision, I dumped Bulldog and signed for BT Vision (Broadband included)

Well .. thats when my problems started, I got from BT the following

- BT Vision Box

- BT Wireless Hub

- BT Vision Phone (VOIP)

I have now gone thru 4 BT Vision boxes .. these are PVR’s that can record from Freeview and on-demand .. they simply fail, Hard Drive and software failures plague the damm thing, on demand shows rarely work properly (fail half way) .. etc etc

I am on my second Phone .. the first one’s speakerphone was a dud, crackling and loss of sound made it impossible to use, and since I do allot of conference calls at home, I must have a speaker phone

And . I am on my 3rd Wireless Hub .. the hubs wireless is a nuisance .. they drop constantly .. so replacement number 3 is now being used .. it’s version v1.5 (new style) .. we shall see if it holds

But to make matters really worst .. I am now on day 6 of virtually no Broadband .. and BT does not seem to have a fix anytime soon ..

Since Sunday night, my broadband stopped working .. occasionally it would come back to live, at speed raging from 126K to 512K .. quite ways from the 8MB I am suppose to get, I called BT, they say our account was suspended by error (??) .. apologized and said it would be back to norm soon .. that was Monday ..

But thats just BS .. we are able to connect if we do the following .. take the phone off the hook, let it sit for a minute until a audible warning is heard and try to connect to broadband .. presto .. it connects .. at very slow speeds .. but it connects .. at least for a few minutes .. then it goes down again.

Another indication of trouble, the phone line is incredibly noisy when broadband is trying to connect, there is definitely something wrong between broadband and the phone line

Anyway .. 6 days now, 3 calls a day to them to piss them off, and still, no fix .. they have done the whole “lets go thru the troubleshooting steps” routine a few times now .. it just does not work .. I even try a different router .. simply does not bloody work .. but BT .. they still oblivious, not a clue and not a single engineer visit to troubleshot.

Dang it…. yes .. I am pissed off.

Popularity: 42% [?]

I have been looking for a new design for the blog non stop for 3 days now.

What happen was, I upgraded (finally) from Wordpress 1.5.x to 2.3.x .. in the process I lost my old (and tired) design .. so I went on the prowl for a new theme I can modify to my taste

Problem is, there are hundreds of themes out there, and most of them .. well, they plain suck.

I think I found something that will work for me, it’s a Theme called Purple Fever .. it only costed $49 .. I have paid, just waiting now for the download link.

Then the task of modifying it will begin …

Popularity: 62% [?]

I’m testing a few new designs for the blog..

Popularity: 31% [?]