Tag: Open Query

Posted on by — 1 Comment

Visiting Darwin/NT

From this Tuesday 12 May until Sunday 17 May I’m going to be in Darwin (Northern Territory, Australia), teaching custom MySQL training days for a medical research institute.
DarLUG has been extinct for a while, but perhaps there are some local Linux/OSS people reading this? Please do drop me a line if you’d like to catch up while I’m there!
I’ve only flown through Darwin before, never visited/stayed… and I love the tropics (I lived in Cairns for a while) so this shall be a joyous few days! Oh the tough things we have to do in business. I’m also doing a day trip to Litchfield… Kakadu will have to wait until another time (perhaps with Phoebe) as I’m told it’s best visited over multiple days. It’s extra cool that this trip is happening this year since 2009 marks Charles Darwin‘s 200th birthday.
Posted on by — 1 Comment
Posted on by — 1 Comment

Joomla Day Brisbane

After the morning and afternoon tutorials today by Andrew Eddie (Joomla dev lead), tomorrow is Joomla Day – Brisbane Joomla Users Group where I’ll be doing talk as well.

I’ve already noticed that Joomla users are a slightly different crowd. Joomla is a pretty powerful CMS with many modules/extensions, just like Drupal which runs the Open Query web site. I’m not sure the two even compete directly although there might be some overlap. It occurred to me that Joomla might be what I would call an “enabling technology” on the web, just like PHP and MySQL have been since 1995. It has a very easy entry, which of course is both good as well as bad. Again that’s quite similar to the M and the P…. love it or hate it.

I’m kinda agnostic on the subject of CMSses, there’s quite a few out there and I think that most of the main ones have a valid place or niche in the market where they thrive. That makes sense. They shouldn’t each aim to be everything for everybody…. plugin architectures can somewhat guard against that, but there’s more to it than that of course.

Anyway, I think it’s interesting to observe these things, and how different applications have a (slightly) different userbase with different perspectives and objectives.

Posted on by — 1 Comment
Posted on by

Call for Papers: Open Source Developers’ Conference 2009 – Brisbane

This year the fabulous OSDC conference is 25-27 November 2009 and returning to Brisbane (Bardon Centre, Mt.Cootha which is a great venue tucked into a rainforest setting). Stephen Thorne leads the organisation team for this event.

The call for papers is now open, until June 30th. For full details, see OSDC 2009 call for papers.

We’ll definitely be there, particularly since it’s very close to Arjen’s house. And perhaps we’ll get some talks accepted on MySQL and other topics. We’ll definitely be submitting some proposals!

Posted on by
Posted on by

MySQL SSD experiments and a request

Open Query too is exploring utilising SSDs in a MySQL infrastructure, but we wouldn’t be us if we didn’t also try some alternative perspective on it. Right now we’re running some comparative tests against various spinning HD setups in the same box, using the same controller, so we’re looking for differences rather than absolute speed.

The results so far are interesting, but the selection of SSDs we have available is limited (never enough toys!) So, a request: do you have an SSD, it’d be great if we could run our test tool on it for a bit. It won’t take long, but naturally the box shouldn’t be used for something else while the test is running. We can either log in remotely, or exchange code and results over email. Simply contact us through our site’s contact form, and we’ll sort things out! Thanks.

If you work for a vendor and would like to have your gear put through a bit of real world stress, please let us know also. Our reference architecture will definitely contain brand/model information as the performance and other aspects of SSDs varies widely.

Posted on by
Posted on by

Move from LiveJournal to Open Query blog

I’m shifting away from LiveJournal. It lacks ability to search and otherwise peruse archived blog posts. And of course it’s only me, while Open Query has more people.

From now on the posts will be at https://openquery.com.au/blog/ and this is aggregated to Planet MySQL as a group blog. You may have already seen Walter posting from his seat at the MySQL Conf. All posts and comments from my LJ blog have been migrated to WordPress, thanks to magic performed by young Akash Mehta. Unfortunately the comment threading can’t be exported.

The full export means that my personal posts are now also present at Open Query, although I may move those elsewhere later. The existing blog entries on LJ will stay for a while at least, although I do have to pay for the LiveJournal subdomain to keep the URLs alive.

A little sidenote on the export… it actually to considerable effort and script hacking and possibly even some screen scraping. Consider this is my own data… Remember, LJ used to have a perfect export, which actually enticed one to stay put. Funny, isn’t it. But that was in the Danga days, and we’re two owners down the line from that (SixApart and then the spinoff-sale to the Russians). Lock-in makes people want to leave more.

Posted on by
Posted on by

Open Query at MySQL Users Conference 2009

I’m not personally there this year, but Walter Heck will be. In case you haven’t met Walter yet, photo enclosed 😉

Photo of WalterHe’s a techie, like you, and he’d love to meet you and hear how you’re using MySQL and surrounding technologies and what things might make your life easier in terms of application architecture, development, deployment, maintenance, and so on.

This may or may not fit with the services that Open Query provides, but the key point is to listen, not sell. If there’s a good match, of course that’s fine too!

As a sidenote, just to pre-empt the inevitable question of “Arjen why are you not here?”: flying over to the US has always been fine, but coming back is a pile of grief every time taking multiple weeks to recover. I’ve never had that issue with for instance trips to Europe. That’s not just grief in terms of personal discomfort or lost work time, but it means my daughter really loses out on her papa for about 3 weeks.

Before I made the decision to note make the trip this year, I did enter various items into the session submission process months back – they didn’t get accepted, but I’m not unique in that respect. Many well known speakers from previous years did not get a slot this year. No worries. There’s plenty of things going on in the surroundings, including Sheeri’s game day and community conference and Percona’s performance sessions, as well as all the excellent encounters in the corridors and in the evenings.

Having been absent in 2007 as well, I would like to add a little bit of outside perspective on the event… while for this week everything appears to be focused on the conference (at least when you look at Planet MySQL), most MySQL users around the world will of course not be at the conf, and have the same needs like the rest of the year. I believe it’s really important to tell about insights and developments all year round, as well as being active in the various interaction media (such as forums, user groups and events).

Anyway, have a great time at the conference, and do have a chat with Walter!

Posted on by
Posted on by

The problem with April Fools in the MySQL/web space…

…is that truth is stranger than fiction. Reality does not appear any more plausible than plain nonsense.

We were discussing this yesterday on #ourdelta (Freenode IRC) in the context of How MySQL really executes a query by Baron. Antony Curtis noted that if he’d write a truthful post on that topic, people would think it was made-up regardless of the day of the year!

Another proof of the premise: Baron has now put a giant banner above/below his post, explaining that it was a joke. Apparently that’s necessary?

I tend to come up with neat ideas for April Fools throughout the year, neglect to write them down, and come the day I have a blank. But, given the above, there’s another option: you just write a truthful story, still leaving people wondering whether it’s for real. I reckon the main issue is probably with the rest of the year, where lots of people still write nonsense, and the truth remains weird as usual. At least on April 1st you know to question. I hope.

Posted on by
Posted on by

Predictive caching in a MySQL-backed infrastructure

Sounds a bit far fetched (pun intended ;-), but we’re doing it. This is not inside of the MySQL server, but rather the overall application design. Let me run you through the logic…

Some key aspects to scaling are: not doing unnecessary queries, and caching what you can. Just a quick baseline. The fastest query is the one you don’t do, or the one you’ve already done before – the latter being caching.

A simple yet brilliant example of this is the Youtube trick where a script reads the relay log, converting updates into appropriate selects and running them so that the InnoDB cache will have the blocks in memory when the slave SQL thread executes the actual update. Maatkit now has a tool for this, so it’s publically available. It’s not quite predictive, but it’s a neat trick anyway that sometimes comes in handy. Search engines use similar tricks.

Extending on this, with certain applications you actually tell what is likely to happen next, sometimes for a particular user and often for many users. Individual user behaviour may sometimes appear random, but as a group it can be highly predictable. The analysis needs to be done properly though, otherwise averaging will make certain interesting behavioural patterns disappear.

Anyway, if you can identify these patterns you can take appropriate measures, such as do some queries so they get cached, and/or schedule other relevant actions (so it’s more than just caching, but it’s a reasonably suitable name anyway). This allows the app to deal with higher peak load, as well as improving response time for individual user.

I might do a talk or article on the predictive caching concept some time, as I appreciate that the short description may appear a bit abstract or obscure. But I assure you it’s entirely practical and real.

It’s one example of how Open Query helps its clients scale well, by design. We focus on preventing emergencies, which includes not just scenarios where stuff fails (and does a safe failover), but also the “oh dear we suddenly have so many more users than a minute ago” type of happening, which should actually be an occasion to enjoy, not stress about.

Posted on by