Tag: migration

Posted on by

Choosing Whether to Migrate to Another Database: Uber

Uber Engineering explains the technical reasoning behind its switch in database technologies, from Postgres to MySQL: https://eng.uber.com/mysql-migration/

These things are always an interesting read, because it looks at one company’s decision making process and operational steps.

At Open Query we’re not a fan of migrations – it doesn’t matter from what brand to what brand, migrations tend to be expensive and painful.  Any application tends to be more suited to a particular brand – because of design and implementation choices. This is neither good nor bad, it’s just a fact that has to be acknowledged when considering these things.

Similarly, infrastructure (what hardware there is and how it’s set up and connected) tends to be dependent on the brand choice, as different databases have different needs in that space, particularly when looking at replication and clustering as well as with larger data stores.

Consequently, moving to another brand is (most likely) going to be less optimal – unless the original was really not optimal (for the original brand it ran on).  Commonly happens when either an attempt was made to keep the application brand agnostic (this tends to work out very badly for performance and scalability), or if the schema design, infrastructure or application was developed by people with experience in a different brand – then some of their design decisions might have been suited for that brand they’re familiar with, but not the brand they were actually deploying on in this particular instance.

So in our opinion, there’s no such thing as a “better database”.  What works best in a particular case depends on many things, including the practical needs, but also any pre-existing hardware and codebase as well as experience/expertise of developers and other people.  For instance, if a desired brand cannot be efficiently deployed on the existing hardware but there is no budget to replace that infrastructure, it would be folly to move to that brand as it just wouldn’t perform effectively – and quite likely cause significant troubles (beyond the migration process itself).

The fact that Uber moved from one brand to another doesn’t mean that that’s the best choice for anybody else.  Every situation is different.  There are valid reasons for choosing to go the difficult path of database migration, but it needs to be very carefully considered.  In the case of Uber, they choose to use a special “schemaless” layer on top of MySQL, which would completely change the way things work.  Certain schemas and query constructs are more optimal in one brand than another, but when you add such extensive abstraction layers on top, it gets even more complex.

Posted on by
Posted on by

Motivation to Migrate RDBMS

http://www.itnews.com/article/3004953/use-oracles-database-watch-out-for-this-dec-1-deadline.html

Companies that use a standard edition of Oracle’s database software should be aware that a rapidly approaching deadline could mean increased licensing costs.

Speaking from experience (at both MySQL AB and Open Query), typically, licensing/pricing changes such as these act as a motivator for migrations.

Migrations are a nuisance (doesn’t matter from/to what platform) and are best avoided as they’re intrinsically painful, costly and time-consuming. Smart companies know this.

When asked in generic terms, we generally recommend against migrations (even to MySQL/MariaDB) for the above-mentioned practical and business reasons. There are also technical reasons. I’ll list a few:

  • application, query and schema design tends to be most tuned to a particular RDBMS, usually the one the main developer(s) are familiar with. Features are used in a certain way, and the original target platform (even if non deliberate) is likely to execute most efficiently;
  • RDBMS choice drives hardware/network architecture. A migration should also include a re-think of this, to make optimal use of the database platform;
  • it’s quite rare (but not unheard of!) for an application to perform better on another platform, without putting a lot of extra work in. If extra work is on the table, then the original DB platform should also be considered as a valid option;
  • related to other points: a desire to migrate might be based on employees’ expertise with a particular platform rather than this particular application’s intrinsic suitability to that platform. While that can be a valid reason, it should be recognised as the actual reason as there are obviously cost/effort implications in terms of migration cost and other options such as training can be considered.
Nevertheless, a company that’s really annoyed by a vendor’s attitude can opt for the migration route, as they may decide it’s the path of less pain (and lower cost) in the long(er) term.

We do occasionally guide and assist with migrations, if after review it looks like a viable and sensible direction to take.

Posted on by
Posted on by

Mixing databases usually not optimal

Dan McKinley (Etsy) wrote an [IMHO] insightful article Why MongoDB Never Worked at Etsy.

First off, it’s important to realise that it’s not a snipe at MongoDB – it’s a fine tool.

The lessons are related to mixing multiple databases in a deployment (administration and monitoring overhead) and the acknowledgement that issues of schema design, scalability and maintenance need attention regardless of which brand or technology you pick for your database. That comes back to the old insight that migrations are rarely worth it (regardless of what you migrate to what).

I think these are indeed important considerations as they have a major impact on the ongoing costs of your entire environment (production as well as development and testing) – these days we encounter the “we’re doing this part of our application using MongoDB” approach quite often, so it’s useful to read about and learn from other people’s experience.

With MongoDB there is a particular extra issue to consider, and Dan McKinley also mentions it in his post. NoSQL databases are often also schema-less. However, to keep your data manageable when it grows to significance, you do need to structure it somehow – that is, you need to make sure that (and I’ll just use generic terminology here) in a specific set of records each record contains the required fields. If you don’t, at some point things become unmanageable (or your data ends up as a pile of unusable bits).

Thus, you’re dealing with some form of schema, whether you call it that or not. And you might deal with it in application logic or through some toolkit, rather than in the database itself, but it can’t just be ignored or disregarded. And that’s critical, as often going to a schema-less database is presented as a “then you don’t need to worry about that” change. You do need to “worry” about it: you can pick where the most suitable place is for your needs. If you look at it in that way, you can make an appropriate choice for the particular application at hand.

Posted on by