Posted on 5 Comments

Brian’s stab at “the death of replication”

I think Brian is right with his The Death of Replication story…. replication is very useful for many cases, but RDBMS (commercial and FLOSS) have done a fairly dismal job at getting multiple writers to scale decently economically. And modern companies just don’t have piles of cash to throw at this, nor does that really scale on the web scene (speed of deployment, etc).

In this context, I’m not sure the new memory based MySQL storage engines coming out are so relevant, they might be fixing the wrong thing in the wrong place. They’ll have their place for now, but it’s not moving us forward really.

What needs to be fixed is distributed writes. And economically!

Posted on 5 Comments

5 thoughts on “Brian’s stab at “the death of replication”

  1. what would also be nice is if you could integrate the binary log replication into other systems, like what facebook is doing with memcached, and I would do with SOLR.

  2. So you want the mysql replication system to be able to replicate arbitrary commands?

  3. I’m not sure it’s a general need, but for us we need to keep multiple data sources synchronized (solr, memcached, and other mysql schemas ), and are using tools like mule and talend to do this, and then using other replication tools to push it out to their slaves.

    having other things being able to read the binlog would mean we could update those systems in a simpler fashion and speed our time to market, as well as remove 1/2 different systems from the architecture.

    Regards
    Ian

  4. I have a library I wrote that has simple reading of the binary log (its public, though far from finished). The problem is writing to the binlog… at this point only an engine can do that.

    I’ve seen someone hack up blackhole to pass executing messages. That is not that hard to do… for that matter we could simplify this through the new gearman stuff I am about to announce next week.

  5. hi, we are using

    http://citeseer.ist.psu.edu/563485.html

    to do decentralized-distributed writes. our system is in testing still and will not be in production until september, but all indications so far are awesome. our data auto re-locates when we add new resources and we can group data together to facilitate joins, foreign keys etc if we need to.

    our site is danga based with mysql as the persistent data store.

    just thought you might be interested,

    -Shane

Comments are closed.