|
Eric -> Rather dramatic performance change (8/4/2008 12:40:39 AM)
|
Performance has been lagging quite badly lately, as I am sure that may of you have noticed. I have been mulling various ways to improve this including a migration to some gargantuan hardware. However, before doing that I wanted to try out what I hoped would be a neat trick. As of this posting I have JUST turned this on, and so far it seems to be working pretty well. Here is the gist: - The main database server now replicates in real time to a parallel database server. (It happens to be the older database server which I pulled out of rotation a year ago.) The latency between these servers is pretty minimal--the replicant is typically just a few transactions (5-10 seconds) behind the live database server.
- I have coded the web server so that logged in users (e.g. registered users who can do read/write operations) continue to access the same database server that we have been using for the past year.
- And, to improve performance, 'guest' users all query the read-only replicated server.
In effect this allows me to spread the load across two database servers. There are two potential sets of issues: - In theory replication can be somewhat fragile if the two servers have trouble seeing each other. It can also make my life more difficult if/when I want to deploy database changes. I will be wrapping my head around this over the coming days.
- My traffic allocation may well be unbalanced. For example, before I deployed this change the main database server was at a rather heavy 50% CPU utilization. Now it is at 2%. Meanwhile the read-only server has gone from unused to a rather drastic 70% CPU. (It is not as beefy a server as the main server.) So the load is nowhere near balanced. THis makes sense since 75-90% of the traffic these days is from 'guests' coming to read the wine reviews that we all generate.
Anyway, it need to tweak things to better balance this (whether by just targeting known user agents such as web crawlers, using a random number generator or some IP algorithm to assign some guest to the RO server and some to the main server etc.) The fear is that I may well have dramaticlly improved the experience for one set of users while torching it for another.
All that said, the approach here appears to be quite promising. I will likely still be pursuing much more enterprise grade hardware, 64-bit migration, AND looking at some segmentation across two or more databases servers.
|
|
|
|
