Posted on

Quest for Resilience: Multi-DC Masters

This is a Request for Input. Dual MySQL masters with MMM in a single datacentre are in common use, and other setups like DRBD and of course VM/SAN based failover solutions are conceptually straightforward also. Thus, achieving various forms of resilience within a single data-centre is doable and not costly.

Doing the same across multiple (let’s for simplicity sake limit it to two) datacentres is another matter. MySQL replication works well across longer links, and it can use MySQL’s in-built SSL or tools like stunnel. Of course it needs to be kept an eye on, as usual, but since it’s asynchronous the latency between the datacentres is not a big issue (apart from the fact that the second server gets up-to-date a little bit later).

But as those who have tried will know, having a client (application server) connection to a MySQL instance in a remote data-centre is a whole other matter, latency becomes a big issue and is generally very noticeable on the front-end. One solution for that is to have application servers only connect to their “local” MySQL server.

So the question to you is, do you now have (or have you had in the past) a setup with MySQL masters in different datacentres, what did that setup look like (which additional tools and infra did you use for it), and what were your experiences (good and bad, solutions to issues, etc). I’m trying to gather additional expertise that might already be about, which can help us all. Please add your input! thanks

Posted on

Decommissioning old servers, saving money…

Of course it’s not quite that simple. I’ve just decomissioned an old Red Hat 7.1 box (hosted dedicated server) that had been in service since 2002, so about 7 years. Specs? Celeron 1.3GHz, 512M, 60GB HD. Not too bad in the RAM and disk realm. It did a good job but goodness am I glad to be rid of it!

Not having that box online is safer for the planet, although it (perhaps amazingly considering the age of some of the externally facing software components) has never been compromised – I consider that mostly luck, by the way, I’m not naive about that. But it’s not easy to move off old servers, it’s generally (and also has been in this case) a lot of work.

Of course hosting has moved on since 2002, places like Linode offer more for less money/month. Of course they virtualise (Xen based in this case) and that’s not been my favourite (particularly for DB servers but depending on the use it really comes down to how you set up the whole infra). It’s a different environment, so different “rules” apply for the optimal setup. The feature/pricing model of the hosting(/cloud) provider actually has more than a little bit to do with that. Distributing tasks like MX relaying, DNS, moderate MySQL tasks, web server, across different virtual machines, with added redundancy across different data-centers, works very well for many use cases. And the funniest thing… more servers, with distributed redundancy, the net cost per month is actually lower than that one single server!

There a many aspects to consider, and I’m intending to write more about that in future posts. I just found it an interesting experience, dealing with this (personal, not even business) server. We handle with these technical environments all the time in our work, but it’s not quite the same perspective. It’s not all technical/financial issues, there’ more to it.