There have been allot of discussions lately on WebHosting related forums about “Grid Hosting”
So that got me thinking, how different is a “Grid” from a “Cluster” ?
My first reaction to the whole thing was to get more info on what exactly are hosts calling a “Grid” .. on my Gfx background, we recognize a Grid as an array of computers, linked together to form a “super-computer” for processing power, somehow why would I need to link 1000 servers to offer processing power to a website was not really evident to me, what could you possibly be processing that required that much number crunching ?
The way big players like ebay and others do it (putting it in lame terms) is to have many computers serving the flow of traffic, this is called “load balancing” where traffic is routed to the least overloaded server, so this way you have 100 servers distributing ONE site, but they are not combining the power of 100 servers to process that site, instead each server processes a chunk of the traffic on it’s own.
As as it turns out, thats very much like “Grid” is working as well, you basically get more servers (In this case, VPS’s) to help process your traffic, this is not a super-computer (for Processing) as a “Grid” is usually called.
So that got me thinking, how hard would it be to do it with Relio ?
Apparently, not hard at all .. we already have most of the tools fabricated at the H-Sphere end, just need to invest into the rest of the hardware.
The solution is to use a NAS/NFS load balanced solution, all the data resides on the NAS, and the servers are then load balanced in the front, you can have as many web and mail servers as one needs for processing power (of traffic), should one server go down, the others keep the sites up, since the data also does not reside on the servers, it doesn’t really matter.
Here is a schematic of how it works for H-Sphere currently.
Neato
This system allows the Host to add more slave servers as it’s clients needs arise .. so should a site grow exponentially .. one just adds more slave servers, isn’t that the promise of GRID ? unlimited growth with no disturbance to the site ?
BUT, I see how this system could be ever better ..
1 – Add NAS Redundancy, everything is fine and redundant, but should the NAS fail, your whole network is down, one should have a second NAS on a real (or near) time sync, so should one go down, the other keeps the whole setup up.
2 – Make it VPS centric .. if one could define VPS’s as the mailservers and webservers for one user, then as the user grows, one could simply replicate THAT vps into any additional slave server (with a bit of development I am sure) .. the flaw I see is, if in order for ONE user to grow, I need to replicate a whole server, then all the other users on that server are getting a free ride ..
There is another solution, which is to move the growing user to it’s own webserver/mailserver and replicate those as needed, thats a bit more expensive, but it’s a solution neither the less.
My perfect scenario.
I’d love to see us be able to deploy a VPS as a shared account, basically, the user resides on it’s own VPS, but manages his account from within H-Sphere, just like a shared account would, no need for a second CP or additional software, and if we could make all the services for that account to be distributed (1 VPS for mail, 1 for web, 1 for databases) .. then one would replicate those vps’s for load balancing, and then increase copies of those vps’s as growth demanded.
I am sure with time and development, H-Sphere will do just that, or close, for now, I am pricing to try and deploy the existing load balanced system, can I cal it a “GRID” ? .. not sure, but it’s all about marketing.
On a side note: This whole boils down to money .. One needs to be able to charge clients for additional resources, how are people offering 1TB of traffic and hundreds of GB’s of storage for $20 is mind boggling .. it’s not even the matter of “can you do it” .. I’m sure with lots of volume, you can, but why ?, why would I want to make 100K profit on a 10Million Turnover ? call me greedy then, but I want to make 100K profit, on a 500K turnover .. otherwise, I am in jeopardy of running a business with very limited capital, and one slip, and it’s Chapter 11 time.
Update: Ok, I just learned that NAS can do fail-over .. do thats one strike off the wish-list
Technorati Tags: GRID, H-Sphere, Relio, WebHosting, website
Hey Carlos,
With time and development, H-Sphere might offer something simpler than what you’ve described. Vlad Miloushev, CEO of 3tera, posted on Web Hosting Talk that H-Sphere grids are in planning. With this system, you won’t need NAS or firewalls – just AMD or Intel servers. Data from VPSes will automatically be mirrored through the grid. If any physical server crashes, any VPSes using its resources will automatically be relocated to other machines. And if a customer’s site grows, his VPS can be resized – no need for slave servers. So don’t go and buy all that hardware for your load balanced system!
-Isabel 🙂
I have looked closely into 3tera (BTW, congrats on your new job) 😉
My problem with it, was I didnt understood the pricing, example, I saw you pay per GB of ram used, and for how long it was used.
This is of course diferent that buying x amount of servers or VPS’s with specifications XYZ and have a fixed cost per month
But, I will assume 3tera must have some sort of pricing like that.
I was recently invited by one of our clients to take part on a training session with 3tera, since they will be footing the 1,000$ per seat bill, I will gladly join in, and I’m sure I’ll get allot more info then.
But if you have more details, do let me know, I’d love to know more.
Also, with Applogic, do you provide the hardware, or does 3tera ?
3tera sells only software; you provide the hardware. You need a gigabit switch, and 2+ AMD or Intel servers. There’s a quick overview of the config at http://www.3tera.com/applogic-hw.html.
In terms of pricing, 3tera charges hosting providers $x per GB RAM per hour. Let’s say you’ve set up a grid with 10 servers, each containing 1GB RAM. While the max RAM on your grid is 10 GB, this is not what you pay. If you’ve created 8 VPSes and assigned 3GB RAM to all customers combined, your billing will be based on 3GB. If any customers cancel, sign up for additional VPSes, or decide to resize their current applications (either up or down), the rate you pay will be accordingly adjusted.
Interesting …
the folks at thegridlayer just contacted me to see if we can do some testing on deploying h-sphere on their grid using the 3tera solution, I’m getting more and more excited with the idea of running on a grid environment.
Not becuase of scalability, well, I lie, not JUST becuase of scalability, but also the ability to re-create a downed/hacked server within minutes (well, at least start the re-initialize process), instead of hours.
I believe tradicional hosting is eventually doomed, so new ways to provide such a service to the client are needed, the price wars are no longuer a way to retain customers, reliability is key, expecially as websites are increasingly going from just a calling card, to actual lifelyhoods, the percentage of people who depends on their sites to be up is on the rise, plus the masses are becoming more aware of downtime and less tolerant about it .. this looks like a step on the right direction.
Humm
Ok, I see virtuozzo is to come out with a solution to re-create downed VPS’s.
So it begs the question .. if I combine Virtozzo VPS’s with failover, and a NAS/Load Balancer with H-Sphere .. what need to I have for 3TERA ?
Ok, more research is needed …
The Virtuozzo solution I believe is going to be a heartbeat+drdb solution along with using the VPS checkpointing feature of Virtuozzo (currently used for online migrations).
The scenario is every VPS host has a slave standing by waiting for the master to fail. Every disk write is mirrored and I believe VPS memory will also be mirrored (though unconfirmed). In this event even if the master is rebooted quickly the slave takes over for a few minutes and then the master resumes when it can.
This is as close to a 99.999% uptime VPS as we’re going to get I think.
Hi Matt
Thats great to know, and brings back my question.
If one setups the h-sphere p.servers on a virtuozzo VPS with that kind of failover .. and use a NAS+Load balancer for and active-active load balancing/scalability … then you endup with a perfect system, capable of scaling up on demand, with fantastic uptime.
Wich begs the question, why pay 3TERA to use their system ?
It would be my opinion that you shouldn’t pay them for the setup you describe. Of course I think deep down you knew this since you had to ask it twice 🙂
3Tera got the HA / VPS clustering idea in your head and you developed your own method for performing a similiar task using already existing technology.
🙂
I now need to find a reliable but somewhat affordable NAS solution that allows fail-over
So Virtuozzo *is to* come out with a failover solution, and Matt *believes* it will incorporate heartbeat+drdb/checkpoint. 3tera, on the other hand, already has production-ready (and more efficient) technology 🙂
Yes, you can set up H-Sphere on master/slave Virtuozzo VPSes and use NAS+Load balancer for and active-active load balancing/scalability. But NAS/LB equipment are not free, and neither are Virtuozzo licenses. I’m guessing this (not yet possible) configuration will cost more than running AppLogic. Would love to see an actual cost comparison.
I wonder if OpenVZ will also do failover .. 🙂
Regardless, it’s indeed a numbers game, and thats my task now, to see how much both solutions would cost.
The downtime the other day with Thegridlayer was a bit of a letdown, and 12 hours to find it was a full logs partition for me was way too long.
I’m going to be speakign with Vladimir and The guys at thegridlayer this week, and come up with some pricing and information on running a load balanced H-Sphere solution, we were to do that this wek, but I got a bit busy .. i’ll then compare it with a straight forward H-Sphere + Virtuozzo + NAS solution .. lets see how the numbers add up 🙂
Re: OpenVZ / failover – http://wiki.openvz.org/HA_cluster_with_DRBD_and_Heartbeat
Isabel you are no longer an independent voice since your recent decision to become part of 3Tera. You have a distinct agenda to promote 3Tera and it’s products and for them to succeed in this industry. With your role defined I will define myself. I am a consumer of SWSoft products and am integrated as a partner into their organization. I have no stake in SWSoft except that I believe based on previous performance that they are exceptional engineers and hold a decent percentage of the hosting market with their products. It’s also worth noting they’ve been around for a while and have been very resilient and managed to grow despite competition. It should also be noted the 3Tera CEO has a history of sinking companies even with millions of dollars in funding.
That being said, 3Tera was first to market. I will give you that. The thing is, AppLogic is using Xen as the backend and Virtuozzo is definitely a better VPS backend for the “grid” application that AppLogic provides. It allows easy execution of commands inside a VPS (return status checking and all), checkpointing of memory, and shared application pages. There are other benefits, but those are clear benefits that will be difficult to argue against.
Do you really think as soon as AppLogic was released SWSoft didn’t have a “war room” meeting about how they were going to nullify the possible threat and how they would go about designing a system that would exceed AppLogic’s technology functionality? I’m going to bet they did.
As a provider I have a choice of who to go with and with the information I listed above I have made a choice. SWSoft did lay out their HA direction (I believe you were at the summit also, covering it as a reporter, and were privy to the same conference). From what I understand with private talks with SWSoft execs the direction shown is not exactly what they are going to release and that the final product will be very low cost while utilizing the unique features that Virtuozzo has over Xen.
The first to market is not always the best as has been proven hundreds (thousands? more?) of times in the free market.
It was a nice to try to spin my comment. How exactly do you know that 3Tera is more efficient? Which product have to benchmarked it against to provide it is *more* efficent? Saying “3Tera is efficient” does not require proof or a competing product. But please.. tell us what it is *more* efficient than? I also don’t think you can do a price comparision against a product that does not exist yet. And before you try to spew back with more biased marketing FUD please see my statement regarding the first to market. If you’re trying to win over customers… what you are doing is definitely not the proper way. I can say before I was not partial to one or the other. Now I definitely have a slant… if you haven’t noticed, it’s not towards AppLogic.
Hi Matt
While I agree with your points, Isabel is right on the fact of cost .. whats cheaper ? a H-Sphere + VZ + NAS (And I call upon H-Sphere becuase it’s the only way to do a proper load balanced scalable hosting environment (outside of PEM), I call on VZ for the upcoming failover solution).
Plus one will need 2 NAS systems for failover of that system as well.
I think that will be a factor for many people, but before I get ahead of myself, I will compile those numbers first.
Funny you say SWSoft has a “war room” meeting, when I was made aware of Applogic at PSoft (H-Sphere) I did the same and was pleasantly suprised to see we had the capability of deploying what people call a “grid” already, just needed a little bit more redundancy on some of the nodes (basicaly H-Sphere can load balance web and mail, so if one fails, you have another to ride on, it also allows for scalability, you need to grow just add more slave nodes, but the same does not happen for database, windows and exchange, so the ability to run those on a VZ node and be able to restore them within minutes of a failure, makes for as close to 100% uptime as one can hope).. thats were I see VZ coming in. 🙂
But I also trust Isabel to be a professional .. I’m sure she needs to sell her bacon, but the truth is, their solution is here and ready, now it’s a matter to find out if it’s cost efficient and reliable.
And sorry, I am posting this while putting up our XMAS tree, so my grammar migth be off tonight 😉
Hi Matt,
My apologies. I am absolutely NOT an independent voice – nor have I been since Web Hosting Magazine went bust in 2002. I should have made more of an effort to clarify my position: I blog on TheWHIR as a volunteer and do event recaps for Keith at PingZine as a friend. I am not a paid writer for either publication.
I also did not intend to “spin” your comment. I’m very sorry if you felt misrepresented. The only point I wanted to make was, plans aren’t the same as products. And by “more efficient”, what I meant was merely that 3tera’s software-only solution might cost less than LB/NAS hardware plus Virtuozzo licenses.
By the way, I thought the Virtuozzo HA methodology you mentioned was from a recent announcement. I didn’t realize you heard it at the SWSoft event back in June (I was invited to attend Serguei’s friend, not a reporter). If SWSoft plans to nullify 3tera, they’d better get moving!
As for whether they’ve had war room discussions about such plans, I’m having dinner with Serguei later this week. I will get the scoop and give you an update.
I personally think there’s lots of room for SWSoft and 3tera to work together. Many hosting companies might be interested in a Plesk grid – just as many of them run cPanel within Virtuozzo. Like 3tera, cPanel is also a much smaller company than SWSoft, but it seems in no danger of being nullified?
Last but not least, while I disagree with your assessment of Vlad’s track record, I will argue that neither of our opinions matter in determining 3tera’s success. Did you know that Dmitri Simonenko, the original founder of Plesk, used to be an used car salesman before he built the control panel we know and love? If we could predict future outcomes based only on history, stock picking would be a whole lot easier 🙂
Plesk was built by a used cars salesman ?
That explains allot 😀
I agree with you Isabel that there is space for companies to work together (heck, on the setup I descrive here I am using 2 competiting products), I personally dont like to see the only solution at the moment for thegrilayer is cPanel .. i’d liek to see them offering our CP as well, so in the end it is up to the consumer to choose wich route to go.
I’ll update here later in the week with numbers, of how much to do the solution I defined above, or how much to do it with Applogic.
I still vote for Solaris containers on the front end(T2000 for web, x4200/4600 for the stuff that’s heavier/less threaded) with x4500/Thumpers on the back end serving up disks as iSCSI and/or running your DB’s directly on it and just serving the data back to your front-end.
You can mirror them against each other and you’ve got raid6+1 nfs over iSCSI, blazing fast speeds, and resiliency with containers hosting your hsphere.
As a disclaimer, I work for nobody that would see any benefit from said solution, I’ve just been working on Solaris a LOT lately, and the technology they’re coming out with is VERY impressive. They seem to just “get it”.
correction, you can do nfs shares, or iSCSI, not nfs OVER iSCSI. The iSCSI would be presented as raw volumes, the underlying FS would be zfs.
Hi Tim
H-Sphere does not suport solaris anymore, but the storage containers, those are sweet 🙂
I’ve allways been a big fan of SUN .. they do it right.
nullmind: You can run centos/rhel inside of a container, and I see that h-sphere does support those 🙂
Read up on brandZ if you’re interested at all.
Well, we’re actually currently building an IBM Blade based solution, with fiber channel connected NetAPP cluster, to which we plan to move our hsphere cluster (and are seriously playing with using vmware to manage the whole thing dynamically). IBM have some amazing tools for rapid/dynamic scaling and failover with blade servers – we’ll soon be offering a 15 second hardware SLA for dedicated servers as a result 🙂
Steve
Thats great, make sure you move my UK servers to those blades 😀
I had a conference with the 3TERA reps, and I have now come to the conclusion it’s a great system to offer our clients (PSoft/H-Sphere).
Now, if you can afford a NAS/Netapp and load balancers, H-Sphere does everything Applogic sets out to do, just more expensive (but hey, it’s your hardware) and a little more work to do the initial setup (but not much).
But there are many people out there who simply cannot afford the costs to deploy such a system (like the one you are about to deploy Steve) .. so for them, the Applogic/H-Sphere combination can be a great one, as you dont need to purchase the NAS and Load balancers and it’s easy to move around if need be.
But of course, we still need to make sure H-Sphere and Applogic properly speak with each others, thats the plan for January.
We’d certainly be interested in testing applogic too – we’ve been looking at it for hsphere and other apps, but just cant decide against just doing it with more ‘standard; methods, as the BG/hour model is a tough one to sell customers (not all, but most)
Steve
It’s really simple Steve
If I have the means ($$$), I’d go the blade/netapp route .. it’s just rock solid, tested and proven environment.
Enterprise clients will certainly be interested on that solution.
Now for hosts, who cant afford big investments, the Applogic route is a very apealing one.
The BG/Hour model is indeed a bit of a complicated solution, but if you sell it as an alternative to a big upfront cost, people will buy into is as well.
When you do deploy your system, let me know, I’ll take the ferry over and check it out 🙂
Willdo, we have to build a new datacentre to accomodate it, but we’ll be finished in about 8 weeks 🙂
Building a new DC ?, now you just showing off 🙂
My understanding is that the plans for VZ to natively support DRBD+heartbeat/failover are no more. They are currently looking for alternative options due to DRBD being unreliable in production. Alternative ideas are welcomed by the VZ team.
3tera left far too many unanswered questions for me. They’re relying on a gigabit private network to do all load balancing/failover/disk transfers. They were never able to answer what happens when you have any sort of disk intensive app like a DB. That backend network will get choked in a big hurry in such a setup. Also, seeing the multitude of problems LT has had with their setup already, I’d say that 3tera is in it’s infancy at best. They were less than forthcoming with any *real* documentation as well, which was a major turn off for me. If you have anything further you can share nullmind as far as materials that would be cool.
TimC
No I agree, there are many unanswered questions, and we must all take a momento to realize it is on it’s infancy.
But I put it down to bits on what exacly does ..
1 – Deploys VPS’s with a firewall (iptables)
2 – deploys VPS’s with a load balancer
3 – deploys vps’s for web, mail and db, wich can be clustered using the load balancer running on vps 2
4 – deploys vps wich can be used an NAS storage
And all these VPS’s can be replicated for reliability and re-build in case of failure
So it’s really not that complicated on what it does.
You can basically do most if not all of it yourself with h-sphere as it stands..
1 – deploy a server with iptables and make it the firewall
2 – deploy another and install PEN load balancer on it
3 – deploy mail and web servers .. load balance those with PEN
4 – deploys a NAS system and a secondary NAS for redundancy
What 3TERA is proposing to do, is to do it quicker and with less resources, you basically have a gui were you can drag and drop server’s (VPS) to be deployed and then select to “initialize” those same vps servers.
I am excited not so much on Applogic, but with the fusion of it with H-Sphere .. who already supports a load balanced system .. so if I can drop vps’s on the fly and tie them with the h-sphere system to be managed by it, then it can be quite exciting.
Now, you raise a very good point about the DB .. here is the catch .. you CAN deploy 2 MYSQL servers and load balanced them with Applogic .. but Applogic (from what I gather) does not manage the load balancing of mysql itself .. you ned to use something like EAC or any other mysql HA manager .. it doesn’t do the HA managing of mysql itself, in the end, Applogic just gives you the blank canvas of a series of load balanced servers (VPS) .. in the end it’s up to you to make sure those all speak with each others properly.
Thats were I see such a value in H-Sphere managing it.
I am going to be working on a live system in January, and try to make it to work to my expectations .. and they are high 🙂
So, then the question becomes, why not just use Xen instead? While you do have a bit more “up-front” configuration, you save a TON of money in the long run, and can upgrade at will instead of waiting for 3tera to test and build patches for you.
Or use vmware and get much if not more of the same management and provisioning tools.
Tim/Steve
That was my question on the beginning of these posts, why use 3TERA, why not just use VZ ? or why use VPS at all (I am still on the fence of using a whole server vs vps, or even better, like Steve said earlier, use blades instead)
If you are a savvy admin with a good team, and got the resources, you have near to no use for Applogic, but we all know thats not the case of most of the hosting market, most people just want a turnkey system at litle cost, thats were I see this come in handy, and H-Sphere is in the market of making it as close to turnkey as possible, so I MUST look to this as a good solution to some of our clients.
But lets go back to my initial coments here . is this a GRID ? .. no, not quite, I used to think of a GRID as a cluster of computers tied in to create a single unique processing power (Super computer).. but it is probably what the industry will call it from now on, so lets roll with it, but putting it to basics, it’s just load balancing HA clustered servers.
My biggest problem is there seems to be absolutely NO HA at this point. Sure, that’s what they’re shooting for but their current implementation is less HA than a single dedi with absolutely no hardware redundancy internally. A POS whitebox freebsd system I threw together is currently at 3xx days uptime and counting. I think their best public grid uptime is 20 days…
Tim
I dont understand why you think there is no HA
If you deploying more than one copy of the VPS (say web), load balance them and make sure the VPS’s are in different hardware, and deploy more copies as you growth of that node demands it, then you have your HA.
What makes you say otherwise ?
If you have to bring the whole grid down for maintenance once a week, where is the HA?
Tim
Why in the world would you have to bring down the grid once a week for maitenance, if thats true, it’s ridiculous ..
Did they told you why you had to bring it down, and to do what exacly ? I’ll try and get some more info on that.
layered tech has been doing it for “patches” on a pretty regular basis. At least they were when vlad was trying to sell his product on WHT. I guess I don’t know what their current status is.
Tim
Thats different, and recently they had a major outage due to some full logs, wich failed to impress.
This is still on it’s infancy, and one needs to understand ad be prepared for that, I expect to spend months testing this technology before even beeing sure if it’s worth to invest on, but I do like to get to an exciting new product in the beginning, when it’s still young enought so that it can be molded to your needs, and the developers are willing to work to meet your expectations.
Lets talk again in, say 3 months, and see where it stands then 🙂
Fair enough, though I still think you’d be better off with Solaris running sun’s cluster suite :>
Of course I would 🙂
Lets make this clear, as I said earlier, if one has the capital, heck go for a nice blade system like Steve is doing with a nice Netapp for storage (or those nice Sun clusters)
Applogic is a solution (I expect) for the masses who cant afford the big upfront investment.
You don’t need a budget for sun clusters, it’s all free. You pay for support if you want/need it, the software itself is 0$