The most straightforward installation is pretty easy using tomcat and your package manager. The dependencies will be met automatically.

apt-get install tomcat6

On CentOS/RedHat

You need to grab some alternative repo’s to make this possible

Eg.

rpm -Uvh http://download.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm

Then you can install the package from yum

yum install yum-priorities ant tomcat6 tomcat6-admin

cd /usr/src/
mkdir sun-java
cd sun-java

Now it gets a little trickier. Sun used to permit direct downloads; but they now have a stupid session validation in place – so download the binary via your PC and upload it to the machine.

You need both the Linux JDK and JRE.

The commands would have been:

wget -O jdk.rpm.bin http://download.oracle.com/otn-pub/java/jdk/6u29-b11/jdk-6u29-linux-x64-rpm.bin
wget -O jre.rpm.bin http://download.oracle.com/otn-pub/java/jdk/6u29-b11/jre-6u29-linux-x64-rpm.bin

You can alternatively use OpenJDK

wget http://jpackage.org/jpackage50.repo -O /etc/yum.repos.d/jpackage50.repo
yum install -y java-1.6.0-openjdk

Once you’ve uploaded the binaries

chmod +x *.bin
./jre.rpm.bin
./jdk.rpm.bin
ln -s /var/lib/tomcat6 /usr/share/tomcat6

Then the remaining steps

Then drop in your respective selection of solr

mkdir /usr/src/solr
cd /usr/src/solr
wget http://mirrors.ukfast.co.uk/sites/ftp.apache.org/lucene/solr/3.6.1/apache-solr-3.6.1.tgz
tar xvfz apache-solr-3.6.1.tgz
cd apache-solr-3.6.1
cp dist/apache-solr-*.war /var/lib/tomcat6/webapps/solr.war
mkdir -p /var/lib/tomcat6/solr

Then add the Magento solr configuration

INSTALL_DIR="/var/lib/tomcat6/solr"
touch $INSTALL_DIR/solr.xml
CORES=( "staging" "development" "live" )
for CORE in "${CORES[@]}"; do
  mkdir -p $INSTALL_DIR/$CORE/conf $INSTALL_DIR/$CORE/data 
  cp -par /usr/src/solr/apache-solr-3.6.1/example/solr/conf/* $INSTALL_DIR/$CORE/conf
  cp -par /home/path/public_html/lib/Apache/Solr/Conf/* $INSTALL_DIR/$CORE/conf
done

Then set up the cores

cat > /var/lib/tomcat6/solr/solr.xml << EOF
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true" sharedLib="lib">
  <cores adminPath="/admin/cores">
    <core name="staging" instanceDir="staging" config="solrconfig.xml" schema="schema.xml" />
    <core name="development" instanceDir="development" config="solrconfig.xml" schema="schema.xml" />
    <core name="live" instanceDir="live" config="solrconfig.xml" schema="schema.xml" />
  </cores>
</solr>
EOF

Then finally, clean up permissions and restart solr

chown -R tomcat6:tomcat6 /var/lib/tomcat6/solr
/etc/init.d/tomcat6 restart

Then in Magento, you’ve now got 3 possible independent cores you can use for your store environments.

• staging/solr
• development/solr
• live/solr

Varnish isn’t the be-all and end-all of Magento performance. Its great to offset load from bots & window-shoppers – but it shouldn’t be your first port of call to actually making your store faster.

In fact, implementing Varnish should be the last performance modification to your store. Only drop it in once you are seeing the page load times Magento is capable of delivering without it (Eg. <600ms page load times).

Your store still needs to be fast

As Varnish still requires at least a single page load to prime the cache, it means your un-cached performance still needs to be very good. A vast majority of unique URLs (layered navigation hits, search queries etc.) will never really end up being served from Varnish unless either:

a) Your TTLs are so high, that a search query from 4 days ago is still valid today
b) The footfall on the site is so vast that the URLs are populated in a very short amount of time

You also have to consider that not every store lends itself to Varnish. Any site that encourages users to create a personal session (eg. log in, add-to-cart etc.) early on in their customer journey will mean that Varnish will be ultimately redundant.

For example, private shopping sites encourage user login from the on-set, however, in doing this, it means that Varnish never really has non-unique content that is cache-able. So your hit rates will be drastically low and there will be no benefit at all from using Varnish.

Fresh content or higher hit rates

Using Varnish effectively is all about striking a balance between stale content and the amount of visitors on your site.

If you’ve got a busy site – odds are you can get away with lower TTLs and still have a high Varnish hit rate – and also continue to have low TTLs – thus, fresher content. So your stock/price changes are reflected quickly and the cache is continually primed from the volume of footfall.

If you’ve got a low-traffic site – then you’re going to have to make a compromise. Either increase your TTLs to ensure a higher hit rate – or have up-to-date content. You can’t quite have both. Yes, you could run a crawl/spider tool continuously – but the resources this would consume, and sheer volume or URLs that can be crawled (usually in the tens of thousands for small stores) means that its simply not effective. So usually, smaller stores would benefit more from a good FPC extension and having a highly optimised server configuration.

But of course I can use Varnish even when users are logged in, what about cache-per-user or ESIs?

ESIs

ESI’s are an excellent utility to be able to keep content in the cache, and still be able to have dynamic blocks on the page. But to be used effectively, you need to minimise the amount of callbacks to the bare minimum. There’s a little head-start module you can use as a basis for this process – just be sure you tighten up the security holes in it, its very insecure by default – there is no restrictions on what layout handles you can/cannot load

Each time the Magento bootstrap is loaded, it comes at a performance penalty of around 200ms – before it even loads a collection/renders a block etc. So if you’ve got more than 3x ESIs, odds are that you’ve ended up with slower page load times using Varnish+ESIs for dynamic content, than just bypassing Varnish and passing the request directly to Magento itself.

So to really use ESI’s effectively, you have to be able to combine multiple requests in a single request.

For example, a category view page listing 20 products needs to show accurate stock levels. So you use ESI’s for each block on the page. That would be 20x ESI stock requests. Whilst stock requests are very lightweight, running 20x of them simultaneously would crush performance. So instead, you could serve the whole block/collection of 20 products and just get that 1x request. But loading and rendering the collection is probably the slowest element on the page anyway – so you’ve not gained much.

Using ESI’s effectively needs proper planing and execution, or you’ll have a slower site than not using Varnish at all.

Cache-per-user

Then there is the alternative of using a user-specific cache. This is a bad idea unless you’ve got a very low-traffic site. Your hit rate will be dreadfully low – as the odds on a visitor hitting the same page they’ve already been to are very low. And for each customer, that 6Kb page will be occupying more and more space in your Varnish storage bin.

For example, if you’ve allocated 1GB to Varnish. With a typical site where users view 8 pages per visit, on average 6 of those pages will be unique. So that’s 28 visitors per 1MB of storage. Then factor in your images, CSS and JS – these (thankfully) will be common, but will still probably occupy a good 7-800MB of your available storage. This leaves you with 200MB of storage remaining, enough cache for 5,600 unique visitors.

Well, I don’t care, I just want Varnish

Okay, then you’ll need to do the following:

1. Install an SSL terminator to sit before Varnish (eg. stud/pound/nginx)
2. Install Varnish on the server
3. Ensure you configure X-Forwarded-For correctly
4. Install a Varnish module on your store
5. Set up your Varnish VCLs to exclude 3rd party extensions

As the first 3 points are beyond the scope of this answer, I’ll leave that to yourself to handle. Point 4 is child’s play and with point 5 – continue reading.

The most important thing about Varnish implementation is to ensure that you never cache content that should never be cached.

Eg.

• Payment gateway callbacks
• Cart overview
• Customer my account overview
• Checkout (and respective Ajax calls)

etc.

For the core Magento URLs, there is a fairly standard list of URIs that you can escape in Varnish:

admin|checkout|customer|catalog/product_compare|wishlist|paypal

But you also need to consider any custom/3rd party extensions you are also running that have custom routes, routers and namespaces. Unfortunately, there isn’t an easy way to know what URLs from these extensions can and cannot be cached. So you need to evaluation each on a case-by-case basis.

As a rule, whenever we are configuring Varnish, we’ll start by identifying the respective routes, routers and namespaces that they might occupy and go from there. We do this via SSH:

grep -Eiroh "<frontName>.*</frontName>" community | sed "s/<frontName>//gI;s#</frontName>##gI" | sort -u
grep -A10 -ir "<rewrite>" community | grep "<from>"
grep -A5 -ir "<routers>" community 
grep -Eiroh "<frontName>.*</frontName>" local | sed "s/<frontName>//gI;s#</frontName>##gI" | sort -u
grep -A10 -ir "<rewrite>" local | grep "<from>"
grep -A5 -ir "<routers>" local 

This won’t give you a definitive list of URLs – but it will almost certainly give you a starter.

We cannot stress how important it is to never cache content that isn’t supposed to be cached. The results could be catastrophic.

In summary

As with any other Magento server performance optimisation, implemented and tuned correctly can really yield benefits. But merely dropping in the software without properly configuring it is not only going to make your store no faster, but potentially slower, more insecure and less reliable.

Our client we are currently working with has a requirement that first response from the web server must come in under 200ms in the UK. Currently under 2 dedicated web servers under load balancer and 1 db server, we are coming in at 800ms.

The site at the moment has less than 5 customers, 2 products, 4 categories, there is no frontend to the site at the moment, it is style free and image free.

It is also being run on EngineX with Varnish.

Can anyone give me any advice on web server setups? Why is ours coming in slowly? What can you recommend to optimize this? Need to get 400% quicker!

Here’s what we had to say

You won’t achieve those figures without the aid of either Varnish or FPC (or both). I would certainly hope that figure doesn’t also have to include static content (whenever you decide to add it) – as its near impossible to achieve (short of having little to no images/js/css).

We are coming in at 800ms
It is also being run on Nginx with Varnish

You’ve got Varnish configured wrong.

A properly configured installation of Varnish will deliver <100ms page load times (we see closer to <10ms).

In fact, for Magento, you should expect to see something like this,

When a customer is not logged in …
Ie. Not having created a unique session (add-to-cart/wishlist, logging in etc.)

--1.2s--------0.8s-----------------0.6s----------------0.1s--------------0.08s----
  Uncached    Mage default cache   Partial FPC cache   Total FPC cache   Varnish

When a customer is logged in …
Ie. Having created a unique session (logged in, items in cart etc.). At this point Varnish will likely be off. And if you’ve chose to use ESI’s – depending on the reverse calls, it can either maintain a similar page load time as the FPC cache (due the bootstrap overheads) – or actually increase page load times beyond being uncached.

--1.4s--------0.8s-----------------0.6s--------------
  Uncached    Mage default cache   Total FPC cache   

Its not a case of tuning Varnish – its a case of – “you are not actually caching anything at all”.

The ideal Magento server configuration files

There isn’t one, well, not quite.

We operate over 400 servers, all purely Magento stores – of varying sizes and capacity. And it is rare that the configuration files we have on one – will match that of another. That is because not all businesses are alike.

Bottlenecks can form due many different reasons:

1. High numbers of visitor concurrency, with active sessions
2. Victims of ‘bad’ crawl bots, generating necessary, unvaluable load
3. High proportion of layered navigation hits
4. High numbers of search queries
5. High volume of transactions per hour
6. Poorly built template
7. Too many/slow/bulky 3rd party extensions
8. Outdated inbound links leading to high proportion of 404 hits
9. Network interface capacity at limit
10. Large/complex catalogue (lots of products/categories/attributes)

So with stores all across this spectrum, each have a different approach to more optimum performance.

To solve the issues outlined above; we’ll deliberately avoid just stating “more/better hardware”

1. Use a FPC beyond that of Varnish
2. Filter out/block bad bots at the network edge – or redirect all requests to Varnish regardless of cookie state/URL
3. Change stock layered navigation to SOLR, make layered navigation filters dependant
4. Change stock search to SOLR
5. Distribute MySQL load across Master/Slave configuration – only do this when you’ve guaranteed browsing load is absorbed by Varnish/FPC
6. Re-build the template
7. Strip them out
8. Monitor access logs continuously and rewrite URLs at Nginx/Varnish prior to delivery. If doing it at Nginx level – ensure Varnish is caching 301/302 redirects.
9. Split off static content to a CDN, or improve connectivity
10. Add more hardware (well, we had to say it at some point)

So with this in mind – you’ll see there probably isn’t going to be an Nginx config file, PHP fcgi pool config file, MySQL config file or Varnish config file that are going to be the same. Couple that with hardware changes itself; available memory, I/O performance (HDD and Network) and CPU – and you’ll find there is subtle variations that lead to the 400% performance gain you desire – but no one quick answer you’ll readily find on-line.

You could copy+paste the Peer 1 sponsored Magento white paper on peformance (we wouldn’t recommend it); hope that the settings don’t exceed your available memory, thread limits, TCP/IP states, I/O capacity and lead to lesser performance than a vanilla Apache/mod_php configuration.

So lets continue on.

The ideal Magento server stack

This is more likely to get you closer to reality. A good example to demonstrate this is to show how a dedicated Magento OS is configured, MageStack

Take the separate sub-components and you’ve got a list of the most optimum/critical software, when configured properly, to run a Magento store. Notably:

Firewall, DOS filter, Load Balancer, Varnish, Nginx, PHP, Redis, Memcached, MySQL

So when you ask:

What is the Best Magento Server Setup?

What is your goal exactly?

1. High availability
2. Reliability
3. Simplicity of administration
4. Performance
5. Scalability

Enough lecturing, how would we do it

To partially mirror the answer given on server fault down a similar vein. You’ve got 3 servers at your disposal – so first orient them as optimally as possible. We’ll avoid a highly-available solution as that is far beyond the scope of this answer.

The sub-components required for a multi-server configuration are:

• Firewall
• Load Balancer
• Web Server
• MySQL Server
• Common Storage

So we’ll multi-purpose some of the systems. PCI-DSS compliance dictates a role, for each server. So with 5 roles and 3 servers – you’ll be in breach immediately. MageStack gets round this by using virtualisation – you could do the same.

Server 1: Load Balancer + Web server
Server 2: Web server
Server 3: Database server

Without low-latency and significant network bandwidth (>1Gbps, <125µs), rather than having common storage – its better for you to merely store the store root directory on each machine and replicate the data, either in real-time using ionotify or lapsed using a cron job. Again, we’ll avoid network file systems like NFS, or replicated block devices like Gluster or DRBD – as vast tuning and decent network bandwidth is required.

Varnish needs to sit as close to the front as possible. But Varnish cannot decrypt SSL. So combine it with an SSL terminator; Nginx, Pound, Stunnel, Stud etc. The built in load balancer in Varnish isn’t great – but would be adequate for a 2 server set up.

Nginx + PHP-FPM is fine, but don’t drink too much of the Nginx kool-aid. It will perform almost identically to a traditional Apache/mod_php configuration, here’s some good reading on why not to use Nginx. Nginx is good, very good infact, but its certainly not a bottleneck of a Magento store – and given its complexity and lack of native Magento support. Most novice system administrators would benefit from using Apache/mod_php over anything else. This may seem like an archaic recommendation over using PHP-FPM – but our performance tests have shown performance is only ~7% faster with the Nginx solution – when properly configured. The tuning and experience required for a high-performance, reliable Nginx/PHP-FPM set-up is fairly vast to get it to outperform Apache/mod_php. Whichever you choose to use, is your call.

The database server is simple, MySQL. But as mentioned earlier, if you have a high converting site, a Master/Slave configuration is advised. Whether you should follow this approach can be determined by reading this article.

Then your peripheral back-end caches, Memcached and Redis. On smaller stores, storing sessions in one Memcache instance and the fast backend cache in another will yield good performance benefits. We don’t advocate storing the cache tags in a slow backend – as it causes more problems than it gives. So with a Memcached set-up, you’ll have to forfeit cache tagging. Instead, we use a configuration like this.

Redis isn’t native to Magento, but with the extension from Colin Mollenhour – its a better solution than Memcache, supports cache tags, session storage and even persistent cache storage – its not quite as volatile as Memcache. But it does have its drawbacks. We’ve found on large scale production stores (>500 orders/hour, >30k uniques/hour) that the cache (and tags) can fill up very quickly and once the point of saturation has been hit, the LRU mechanism fails somewhat (despite different settings) and causes a massive immediate bottleneck. So it is prudent to regularly prune old records manually.

So what hardware should be used for what?

Web servers: Fastest CPU, most CPU cores, ratio of 2GB RAM/ Core
DB server: Fast CPU, fastest disk I/O, most RAM

So when multi-purposing your 3 machines, the best layout would be:

Server 1: SSL Terminator -> Varnish -> Nginx/Apache > PHP
Server 2: Nginx/Apache > PHP, Redis, (MySQL Slave)
Server 3: MySQL

As to the specific configuration of each application. Well, that’s down to your hardware specifications, your store complexity, your type and nature of visitor and the sheer volume of visitors.

Follow each point until you finally reach a solution.

Enable PHP Errors

This is key to most issues. For security or other reasons, PHP error display could likely be disabled by default by your PHP configuration.

You can enable errors with a more permanent solution, or something merely more temporary.

Permanent solution

For Apache/mod_php users

In your document root .htaccess file – just drop this at the top.

php_flag display_startup_errors on
php_flag display_errors on
php_flag html_errors on
php_flag  log_errors on
php_value error_log  /home/path/public_html/var/log/system.log

For Nginx/fastcgi users

In your Nginx virtualhost configuration, in either the final location .php { directive, or in the fastcgi_params file (if you have one specified)

fastcgi_param PHP_VALUE  display_startup_errors=on;
fastcgi_param PHP_VALUE  display_errors=on;
fastcgi_param PHP_VALUE  html_errors=on;
fastcgi_param PHP_VALUE  log_errors=on;
fastcgi_param PHP_VALUE  error_log=/home/path/public_html/var/log/system.log;

Temporary/Universal solution

For any platform

Edit the Magento bootstrap index.php in your document root and uncomment the following line:

#ini_set('display_errors', 1);

Enable Developer Mode

When you’ve had an error and suddenly hit the “Error Report” page, and been given a seemingly useless error string like 1184257287824 – you’ve got a few options.

Permanent solution

For Apache/mod_php users

In your document root .htaccess file – just drop this at the top.

SetEnv MAGE_IS_DEVELOPER_MODE true

For Nginx/fastcgi users

In your Nginx virtualhost configuration, in either the final location .php { directive, or in the fastcgi_params file (if you have one specified)

fastcgi_param MAGE_IS_DEVELOPER_MODE true;

Temporary/Universal solution

Edit the Magento bootstrap index.php in your document root and either make the if statement always true, or enabled for your specific IP.

if (isset($_SERVER['MAGE_IS_DEVELOPER_MODE']) || true) {
  Mage::setIsDeveloperMode(true);
}

or

if (isset($_SERVER['MAGE_IS_DEVELOPER_MODE']) || $_SERVER['REMOTE_ADDR'] == 'my.ip.add.ress') {
  Mage::setIsDeveloperMode(true);
}

Revert theme to default

Its possible that either your theme or package is responsible for this issue. Reverting back to a vanilla Magento theme is a quick way to find out.

**This comes with the caveat that some modules may be dependant on certain theme features*

Rather than change anything via the admin, it is much simpler to merely rename the offending directories.

Via SSH

mv ./app/design/frontend/myBrokenTheme{,.tmp}
mv ./skin/frontend/myBrokenTheme{,.tmp}

Or via your FTP client, traverse and rename your package to something else. eg. myBrokenTheme.tmp

If this resolves your issue

Then you need to dig a bit deeper as to what part of the template is problematic. So restore your package and attempt the following, testing between each.

Essentially, the process is to gradually enable directories as you traverse down the file tree – until you can find the offending file.

1. Rename the layout directory to .tmp
2. Rename the template directory to .tmp

Then if either yields a fix, rename all files within the layout directory to .tmp – (for the SSH users ls | xargs -I {} mv {} {}.tmp or rename 's/^/.tmp/' *)

Then gradually enable each file 1 by 1 until resolved.

If this doesn’t resolve your issue

There is potential that your base/default or enterprise/default directories have become contaminated – and are best replaced with a known clean version.

You can do this by downloading a clean build of Magento and replacing your directories as necessary. Via SSH you can do this:

cd /home/path/public_html/
mkdir clean_mage
cd clean_mage
MAGENTO_VERSION=1.7.0.0
wget -O magento.tgz  http://www.magentocommerce.com/downloads/assets/$MAGENTO_VERSION/magento-$MAGENTO_VERSION.tar.gz
tar xvfz magento.tgz
cd /home/path/public_html/app/design/frontend
mv base{,.tmp}
cp -par /home/path/public_html/clean_mage/magento/app/design/frontend/base .
cd /home/path/public_html/skin/frontend
mv base{,.tmp}
cp -par /home/path/public_html/clean_mage/magento/skin/frontend/base .

You can also take the opportunity to diff the two directories if you want to verify any changes.

diff -r base base.tmp

NB. This method will cause more errors during the process, as module dependency dictates the existence of specific files. Unfortunately, its par for the course.

Disable local modules

By default, Magento defines the PHP include path to load classes in the following order

Local > Community > Core

If a file is in Local – load it and do no more.
If a file is in community – load it and do no more.
If a file can’t be found anywhere else – load it from the core.

Again, rather than disable modules via the Magento admin. It is more practical to do this at a file-level.

Typically, to disable a module the “proper” way, you would edit the respective ./app/etc/modules/MyModule.xml file and set <active>false</active> – however, this doesn’t actually prevent a class from loading.

If another class extends a given class in a module (ignoring any Magento dependency declarations), it will still be loaded – regardless of whether the extension is disabled or not.

So again, the best means to disable an extension is to rename the directory.

Begin by disabling local

Just rename the directory via FTP, or use the following SSH command

mv ./app/code/local{,.tmp}

Then disable community

mv ./app/code/community{,.tmp}

If the issue is resolved from either

Then it is a case of understanding which module in particular the error stemmed from. As with the example given above for the package diagnosis, the same process applies.

So restore the X directory and attempt the following, testing between each.

Essentially, the process is to gradually enable directories (modules) one-by-one until the error re-occurs

1. Rename all the modules in the directory to .tmp (for the SSH users ls | xargs -I {} mv {} {}.tmp or rename 's/^/.tmp/' *)
2. Gradually enable each module one-by-one, by removing .tmp from the file name

If the issue is not resolved

Then it is possible the core itself is contaminated. The main Magento PHP core consists of

./app/code/core
./lib

So again, rename these directories and copy in a clean variant. Assuming you already downloaded a clean version of Magento as above, via SSH, you can do this:

cd /home/path/public_html/app/code
mv core{,.tmp}
cp -par /home/path/public_html/clean_mage/magento/app/code/core .

Then if the issue still isn’t resolved, replace the lib directory too

cd /home/path/public_html
mv lib{,.tmp}
cp -par /home/path/public_html/clean_mage/magento/lib .

At this point, your Magento store will be nothing more than a vanilla installation with a modified database.

Some models are actually still stored in the database (Eg. order increment) – so at this point, it becomes a case of manually making those edits. So far, all the steps above have been reversible with no lasting damage. But if we were in import a clean Magento database too – it could prove reversible (short of restoring a backup).

Seek help from a professional

The above tasks are easy complete, even by amateurs, but beyond this point; if you are still experiencing problems – we would suggest seeking professional help.

The guide above serves to get you on your way to identifying an error; not to fixing the resultant error – but is the standard debug procedure at Sonassi.

Can Magento natively handle read-only slaves

Magento is natively capable of splitting off reads/writes to different database servers (with the exception of a few broken releases, eg. EE 1.11) – allowing you to offset select load to an additional (or more) server(s); and forwarding all the update/write queries to a single master.

When should I do it

This is a more appropriate question. With dedicated Magento operating systems like MageStack – it is becoming more common for in-built server side advanced caching techniques to be available and easily used (such as Varnish front end caching and Redis back end caching).

Historically, Magento has never been bound by MySQL – but rather PHP. But as Varnish and Full Page Caching (FPC) are used more frequently, the burden of repeated tasks (category/product loads, frequent searches) is suddenly absorbed and PHP becomes less of a burden. In fact, it only really comes into play to generate the content initially, or complete non-cachable scenarios (add to cart, order completion etc.); for the purpose of explanation we’re deliberately ignoring administrative load.

We have always stood by the fact that MySQL isn’t an areas of concern for most retailers, as seen both here and here. But if your in the region of processing hundreds of orders per hour, not single or double digits – it will soon become an areas for optimisation.

Ultimately for smaller stores (<25k daily unique visitors)

Your efforts would be far better focused on simply finding an appropriate host who can suggest the right hardware to be on from the offset and that has configured the machine in the most optimal fashion for your store. Don’t waste your time pursuing Master/Slave or Master/Master configurations – which will yield no performance benefit and will ultimately require continual attention and advanced MySQL knowledge.

Ultimately hardware sizing and selection will have a bigger part to play than MySQL optimisation.

But for larger stores

As your store starts to grow, converting or transactional load becomes more of a burden with the repeated task of completing complex inserts and updates. The addition of each new order will trigger the decrement of catalogue stock, callbacks from payment gateways and updates from EPOS/ERP systems. Combine this with the associated cache purge of the respective products/categories and you’ll soon see MySQL load disproportionately increase.

Multi-master is never a solution we recommend or consider as a viable option, but Master/Slave can yield benefits (we stress, on Enterprise-size stores) by offsetting read load to secondary/tertiary nodes.

But I still want to do it

First configure your slaves. We’re big advocates of the Percona utilities and MySQL branches – they have an ideal tool for taking hot backups of your existing DB – innobackupex. There is a good write up here.

On the master

Replace $TIMESTAMP or tab complete.

mysql
> GRANT REPLICATION SLAVE ON *.*  TO 'repl'@'$slaveip' IDENTIFIED BY '$slavepass';
> quit;
innobackupex --user=username --password=password /path/to/backupdir
innobackupex --user=username --password=password /
       --apply-log /path/to/backupdir/$TIMESTAMP/

rsync -avprP -e ssh /path/to/backupdir/$TIMESTAMP TheSlave:/path/to/mysql/
scp /etc/mysql/my.cnf TheSlave:/etc/mysql/my.cnf

On the slave

/etc/init.d/mysql stop
mv /path/to/mysql/datadir /path/to/mysql/datadir_bak
mv /path/to/mysql/$TIMESTAMP /path/to/mysql/datadir
chown -R mysql:mysql /path/to/mysql/datadir
sed -i 's#server-id=1#server-id=2#g' /etc/mysql/my.cnf
/etc/init.d/mysql start
cat /var/lib/mysql/xtrabackup_binlog_info
> TheMaster-bin.000001     481

mysql
> CHANGE MASTER TO MASTER_HOST='$masterip', MASTER_USER='repl', MASTER_PASSWORD='$slavepass', MASTER_LOG_FILE='TheMaster-bin.000001', MASTER_LOG_POS=481;
> START SLAVE;

Then once your slave is operational, in practice, it only takes a few additional lines of code to achieve.

In ./app/etc/local.xml

<default_read>
  <connection>
    <use/>
    <host><![CDATA[host]]></host>
    <username><![CDATA[username]]></username>
    <password><![CDATA[password]]></password>
    <dbname><![CDATA[dbname]]></dbname>
    <type>pdo_mysql</type>
    <model>mysql4</model>
    <initStatements>SET NAMES utf8</initStatements>
    <active>1</active>
  </connection>
</default_read>

Our Manchester office will be shut from 21st December 2012 until 7th January 2013. Team members will still be checking their respective and common email inbox’s – but only on a semi-regular basis.

The team here at Sonassi wants to thank all our customers for making our 2012 so brilliant – and we look forward to continuing being the best Magento agency in the UK during the years to come. Have a very Merry Christmas and a Happy New Year.

The Dock Office

The Dock Office sits beside Trafford Road in Salford. It is a Grade II listed building, designed by Harry Fairhurst and Son, was built in 1925. Beside the building is the former gateway, also by Fairhurst, that once provided access to the docks. The words “Manchester Docks” once ran across the top of the gateway. On the right hand side of the entrance gateway you will find a plaque commemorating the opening of the Manchester Ship Canal by Queen Victoria on the 21st of May 1894. On either side are sculptural motifs of a ship’s keel projecting out from the stonework.

It is a huge 36,000 square feet site, so plenty of space for expansion for us; vertically spanned across 4 stories.

So with character in spades, excellent condition and classically refurbished, the Dock Office was an ideal choice for Sonassi. Now its just time for the important decision, how to lay the office out … Our first office at the Dock Office is 500 square feet, so we’ve got to get creative.

Some initial concepts

Concept 1

The main idea was to create an element of division in the office between the work area and break area. So my first idea was to follow an outdoor-indoor design, with a raised garden style deck for the work area and a grassy break out space.

I thought of using a pool table (both for fun!) and to also serve as a meeting table, with a big 3 seater sofa behind.

Concept 2

Following a similar trend of the separation of work and rest areas, I came up with a slightly simpler concept that gave a lot more room overall, added additional storage and a kitchen area. Sheer curtains were used to divide the entry to the office and work area, so the office would still be light and airy with a sense of separation.

Concept 2 was the obvious choice, time to start decorating.

The blank canvas

Starting out with the bare room, it is 9m x 6m, and thankfully, the designs I drew above were to scale. But you can get a feel for the blank canvas that we’ll be working from – and you’ll certainly appreciate why we’ve chose to decorate the room a bit before just throwing some chairs and tables in there.

We’ve got 4 bright white walls, a freshly laid carpet, 3 cast iron radiators and three huge north west facing windows (no glare from the sun at any point!).

Wireframing an office

Well, we spend all day approving wireframes, mocking up wireframes – its seemed the logical choice to map out the room too. So I bought a few hundred metres of string and some push pins and went away with my tape measure mocking up the space to scale.

I even threw my trusty laptop down on one of the wire-framed desks for good measure.

With the raw layout of the room mapped out, I had a really good chance to visualise the concept and make sure the spacing was appropriate. It gave a real feel to the overall impression of the room and brought the whole idea to life.

Now I had to start finding furniture to do the job, so a trip down to Ikea was in order.

Construction begins

After a few nights on-line and a few trips to various retailers, I had all the furniture arranged – the small items were on site already, but the bigger items were just waiting for the courier to come drop them off.

The colour scheme was set to follow our main company branding, so some bright, lively greens and strong greys were ready to be brushed on the walls.

Testing out the colours

With the curtains up, most of the furniture assembled, walls painted, its starting to resemble something like the original designs. Behind the scenes, we had been busy pre-wiring the office with structured Cat6 for the phone and data points around the office. It only actually took about 20 hours to complete the decorating from start to finish.

The finished product

Guiding you up three stories of 1925 original art-deco class; brass banisters, polished original granite stairs and cast iron spindles. Leading you to our mahogany and brass floor level signage – top floor, for top class developers.

So now enter the break out space and kitchen area …

Freshly ground coffee, straight from the beans. So we’ve got espresso, lattes and cappuccinos on tap. The bright orange life ring might not make the most sense, but its in-fitting with the history of the building. Most offices in the old Dock Office have some kind of tribute to Salford Quay’s former existence as Manchester’s largest docks. We got the life ring as a moving in gift.

Now, its time for business – so enter the developers zone. 5 workstations, kitted out with a pair of BenQ LED backlit 1920×1080 displays, Microsoft keyboard & mice, high performance PCs (no Mac users here!), hugely adjustable posturepedic chairs, precision mouse mats and of course, some drinks coasters.

So come down and meet us

Our second office, the Dock Office should serve us well as we grow more over the coming months – but for now serves as an excellent, practical work space, relaxing chill out area and the birthplace of many Magento stores and Magento servers.

So don’t hesitate to come down and talk about your project – the coffee is on us.

We have long been advocates of using SVN – but times have changed and so has the style of the way we work – which is what makes Git such an appealing choice for us. So if you’re coming from SVN too, some things worth knowing are:

1. Repositories are de-centralised – With SVN, you have 1 master repository in a central location and everything is checked in/out of this location; with Git, its different. Each copy of the project tree (ie. your working copy) has its own repository – the .git sub-directory of the project tree root.
2. Revisions are no longer decimal numbers – With SVN, your revisions are numbered sequentially with an integer. Due to the distributed nature of Git, and its potential to scaling to hundreds of thousands of revisions, the revisions are identified by a SHA1 hash. You can still short-cut your way through the tree though, HEAD (the latest revision), HEAD^ (the latest revision’s parent), HEAD^^ or HEAD~2 (the latest revision’s parent’s parent); etc.
3. Hierarchy – Perhaps the biggest adjustment you’ll make is that SVN has a folder based hierarchy, a trunk isn’t anything special, its just a folder, the same with your tags or branches. In fact, with SVN you can check-out a single folder – which gives it a great advantage. Where Git, is just a URL – which identifies its repositories location (be it local or remote), within it, it automatically contains your master, branches and tags (not your local branches, but we’ll come back to this later).

NB. Its worth noting – this guide is not to be a fully fledged Git tutorial – there are plenty of those around and we’ve even listed the ones we prefer in the resources section below. This guide will serve as an insight into Git, traversing from SVN to Git and if you ever work with Sonassi as your development team – our rulebook and work-flow for collaborative development.

Contents

• Getting started
• Comparable Git:SVN functions
• The rules
• The Hierarchy
• The Work-flow
• Resources

Getting started with Git

The best way to start anything is to introduce yourself, so be sure to set your name and email

git config --global user.name "Sonassi"
git config --global user.email "contact@sonassi.com"

And its always nice to have colours enabled

git config --global color.diff auto
git config --global color.status auto
git config --global color.branch auto

At this stage, we’ll assume you’ve already got Git installed (if not read, How do I install Git?) – and we’ll follow the same practice of setting up a repository like we did in our Magento SVN guide. For the purpose of simplicity, we’ll also assume you are using a remote repository, like BitBucket, GitHub or Springloops.

So start by entering your root directory for your project,

cd /home/sonassi/public_html/
git init

If you are checking out an existing repository, use:

git clone git@bitbucket.org:sonassi/sonassi.git

Whereas if it is empty, then use:

git remote add origin ssh://git@bitbucket.org:sonassi/sonassi.git

If it is an empty repository, Git will set up the current directory. If the repository contains files, it will download them to a sub-directory (named from the repository name). Eg. ./sonassi. It is safe to move the contents of this directory to your current directory (including the .git directory it will create).

Lets start by ignoring the non-essential files

cat > .gitignore << EOF
.gitignore
.htaccess
app/etc/local.xml
cron.php
cron.sh
downloader
errors/
includes
index.php
index.php.sample
install.php
js
lib
LICENSE*
media
pear
php.ini.sample
RELEASE_NOTES.txt
robots.txt
shell
var
EOF

Then add the files we want to version control

git add app skin js lib
git commit -m "Initial Commit"
git push origin master

That's it! We've created the repository, committed the initial files and pushed that to our remote repository. Now we've nailed the basics, we can start to understand how Git should fall into your daily work-flow.

Comparable SVN:Git Functions

If you're coming from SVN, this look-up table will serve as a helpful reference to show you the equivalent Git command.

Rules

The rule that we follow; to ensure a clean, stable and scalable development, for anything other than short-lived changes, is to branch early - merge & commit often. Most changes, should be branched off into a local branch, the change made, a commit made and when complete - can be merged back into the master and pushed back to the repository. The exclusion being when you need to make a quick single-file change (eg. a minor CSS edit).

1. Merge from origin/master at the start of the work day
2. Commit your edits to your local repository at the end of the day
3. The database name must be suffixed with _branch_purpose/name, Eg. _live or _stag
4. Never copy/rename (Eg. creating a .bak file) a file to create a restore point for temporary edits
5. The staging site must only be used for short-lived changes and previews of branch releases
6. Live/production site pulls must be authorised by the relevant party only after the staging release has been approved
7. Always give descriptive commit comments with ticket number/bug ID references where necessary
8. If possible, use a hub system to isolate live pulls
9. The staging site must never be left in an inconsistent state - all changes should either be committed or reset
10. When you are finished with a branch, delete the branch, its files, its database and associated users
11. Never push to the remote repository master, unless you have sufficient time to test and approve the changes for live use
12. When creating a branch, do not give it an arbitrary name - ether name it after a specific ticket that is to be resolved (eg. bugfix1999) or if it is a long-term personal branch, use your name-companyname (eg. ben-sonassi)

Hierarchy

In practice, our development, staging and live Magento VCS environment would be set up like this.

You'll notice that the staging site and live site actually are both master, we define staging as being the final pre-live test environment. So after all your local branch testing, you test once more on the staging site; once that is approved; you can then pull on the production/live site. Remember rule #5.

To create the hierarchy is very straightforward, it begins by creating quick clones of the live site; we've wrote a few scripts to automate the process, so that's what we'll use.

Creating/refreshing the staging site

We clone the live site to create the mirrored staging environment. But we deliberately exclude some of the content (logs, sessions etc.) and we create a symbolic link for the media directory (to save on disk space usage). Just remember, with this set up, if you delete an image/product via the admin on the staging site - it will remove the file from the live site.

cd /home/sonassi
LIVE_DIR="public_html/"
mkdir subdomains
rsync -axHPS --delete $LIVE_DIR subdomains/staging --exclude="var/log/*" --exclude="var/cache/*" --exclude="var/session/*" --exclude="var/tmp/*" --exclude="media" --exclude="errors/*" --exclude="*.tgz" --exclude="*.gz" --exclude="app/etc/local.xml"
ln -s /home/sonassi/public_html/media /home/sonassi/subdomains/staging/media

Then edit the database connection details to suit in ./subdomains/staging/app/etc/local.xml. Then we'll dump the live database ready for import on the staging site - we have wrote a script which can dump the live database faster than the normal process and without causing table-level locks (ie. without impacting the live site at all - read the full guide here). After the dump, we also used sed to find and replace all he URLs to be the new staging URL.

cd /home/sonassi/public_html
wget -O mage-dbdump.sh sys.sonassi.com/mage-dbdump.sh
chmod +x mage-dbdump.sh
./mage-dbdump.sh
mv ./var/db.sql ../subdomains/staging/var/db.sql
mv ./mage-dbdump.sh ../subdomains/staging/
cd ../subdomains/staging/
sed -i 's/www.mydomain.com/staging.mydomain.com/g' ./var/db.sql
./mage-dbdump.sh --restore

Now, we've got an operational staging site, which is a clone of the live site, and also a working directory for master. This now gives us the final preview point before any changes are made live - and a little environment for tiny changes to be made.

Work Flow

Short lived or tiny changes

The only exception to making direct edits to the staging site is when you have a change that can be executed very quickly and you know the definitive output. A good example is if you need to make a quick CSS correction and you know the exact code (or close enough) to make the change. So you edit the relevant file(s), test the output, commit your changes with an appropriate comment, then push the changes to origin master. After approval from the live site maintainer, you can then perform a pull on the live site.

In practice, the code execution would be like this:

cd /home/sonassi/subdomains/staging
git status (you need to make sure that no-one has left the staging site in an inconsistent state - breaking rule #9)
nano ./skin/frontend/mypackage/default/custom.css (you then make some changes to the CSS)
git add ./skin/frontend/mypackage/default/custom.css (contrary to SVN, even if a file is under version control, you still have to state if the change should be part of this commit by "adding" the file)
git commit -m "Edited line +33 of custom.css to resolve ticket #1555"
git push origin master

cd /home/sonassi/public_html
git reset --hard origin master
git pull origin master

You shouldn't have to reset the live site's working directory - but we just want to be extra cautious to remove any temporary edits some naughty developers may have done.

So in summary, make sure you remember:

1. NEVER edit the live site files directly - ever!
2. NEVER make a change on the staging site, unless it takes less than a few minutes to complete

Branching out & merging

Branching

Branching out is a very straightforward practice, the first step is to set up the new branch environment. Thankfully, the process is near identical to creating a staging environment, with two additional commands at the end:

cd /home/sonassi/subdomains/mybranch
git branch mybranch
git checkout mybranch

This command then changes the current working directory to be an instance of your new branch. You can make changes in here, commit as frequently as you desire, then whenever necessary, either merge master into your branch - or if you are complete, fold your branch into master.

Remember, that branches are local to your own repository - so no-one else will be able to use, see or edit your branch unless you push the branch to the origin repository. But, unless you have a compelling reason to share a branch, its unlikely you'll need to do this.

Merging

Merging, assuming no conflicts happen, are very painless to carry out. There are two ways you can go about a merge,

1. Merge another branch into your branch
2. Merge your branch into another branch

It doesn't actually matter which you are doing, but sometimes one approach may seem more appropriate over another.

For example, if you have an actively developed branch, and you need to bring it up to date with the master - then you would execute

cd /home/sonassi/subdomains/bugfix1999
git fetch origin master
git merge master

Or, as another example, you have finished with your branch and intend to fold it into the master - then you would execute

cd /home/sonassi/subdomains/bugfix1999
git commit -am "Final changes to bugfix1999- to resolve ticket #1999, prior to folding into master"
git checkout master
git merge bugfix1999
git branch -D bugfix1999
git push origin master

Conflict resolution

Git is very sophisticated with auto-merging and will only fall back to typical conflict resolution as a last resort, but once you get your head around the <<<<<<<< notation - it will soon make sense, and you'll no longer have that sinking feeling when you've hit a conflict. A common file for conflicts to exist is likely to be a stylesheet, so we'll use style.css as our example.

Auto-merging style.css
CONFLICT (content): Merge conflict in style.css
Automatic merge failed; fix conflicts and then commit the result.

First thing to do is make sure you don't panic, resolving a conflict manually isn't any where near as hard as you might think it is. The second step is to open the conflicted file and scroll down to the conflict, you'll notice that Git has added a number of left chevrons to denote where the conflict(s) has happened.

• HEAD represents the current working directory/branch you have selected - ie. your code.
• master represents the current remove directory/branch - ie. someone else's code (similarly, this could be a branch or anywhere else you are merging from)

<<<<<<<< HEAD
body {
 background:blue;
 font-size:28px;
 font-weight:normal;
}
=======
body {
 background:red;
 font-size:48px;
 font-weight:bold;
}
>>>>>>>> master

So you have to make the educated decision as to what the correct output should be; which may be ...

1. Entirely the other code
2. Entirely your code
3. A combination of the two.

The key here is to make a decision and edit the file accordingly - making sure you remove the conflict notation (<<<<<<<<,======= and >>>>>>>>) as necessary. So in our example, we've decided that we actually need a bit of both, leaving:

body {
 background:blue;
 font-size:48px;
 font-weight:bold;
}

Then, after resolving your conflict(s), its worth committing those changes before actually continuing with your work.

Your daily routine

Getting into the habit of pulling down the changes from the remote repository in the morning and pushing your changes (or at the very least, committing your changes) in the evening is important to ensure you have a (relatively) conflict free and up-to-date working copy of the site. The last thing you want to do is go 3 days without a pull, suddenly finding dozens of conflicts that you've got to fiddle your way through.

So, print this PDF off, stick it on your wall - and live by it - the Git daily work-flow guide.

If you already having an existing branch that you are working on, then the first thing to do is merge the latest changes from master into your branch. You shouldn't have to commit any changes in your branch because you should have done that the night before!

git pull origin master
git merge master

If you have any conflicts, work through them, if not - you can carry on with the development for the day. Be sure to regularly commit your changes throughout the day (with meaningful commits). Then as it comes to the end of the day, you'll need to make sure any new files you've added to the repository have been added to version control and then make a final commit for the end of the day.

git add a_new_file
git commit -am "Addition of a_new_file, and commit of existing files for EOD 29/07/2012"

Do not merge your changes to the remote repository's master unless you have sufficient time to test and gain approval on the staging site.

Undoing a file deletion from a historical commit

Find the last commit that affected the given path. As the file isn't in the HEAD commit, this commit must have deleted it; then checkout the version at the commit before.

git rev-list -n 1 HEAD -- path/to/file
git checkout revision^ -- path/to/file

Or in one command, if $file is the file in question.

file="path/to/file"
git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

Quick and dirty restore points

The Git index is a great resource for making checkpoints in your code, that don't really require a commit exactly. If you are not familiar with the Git index, it is the "placeholder" in which changes are stored after a git add and before a git commit (see diagram). Normally, there isn't any time between add and commit - but you can actually use the index to your advantage.

The index can be used exactly the same way you would rely on the undo history in your PHP editor to provide a way to restore back to a previous point in the code. So if you are ultimately quite happy with your progress on a specific file - or you want to try some slightly different/risky code, then before you proceed you can quickly add that file to the index.

git add ./app/code/community/Sonassi/MyNewExtension/etc/config.xml

Then if you do happen to make a change and you want to restore back to the previous "working" version of the file quickly, you can just checkout the file.

git checkout ./app/code/community/Sonassi/MyNewExtension/etc/config.xml

This way, you can add files to the index repeatedly throughout the day without muddying your commit history.

Amending previous commits

git commit –amend

One unmentioned commit management feature is git commit –amend which would allow you to update the last commit with new edits. If you’re familiar with git rebase -i squashing, then this is like squashing your index into the last commit. You could also amend with the working files by using git commit –amend -a or providing specific files on the command line.

If you have made a mistake and want to remove the last commit - provided you still haven't pushed it yet, it is simply a case of running a single command

git reset --soft HEAD~1

This will not undo any changes in the files, but rather just remove the last commit from the repository. If you swapped --soft for --hard, it would perform the same action, but also remove all your changes too.

Rebase-ing

In Git, there are two main ways to integrate changes from one branch into another - the merge and the rebase. Rather than clone someone else's article, there is a fantastic and clear explanation of rebasing - what it does, when to do it and importantly, when not to do it. You can find the article here

Resources

This article provided an insight into Git management and work-flow - but is only a taster for what can be done with Git. It also wouldn't have been possible to write it without the excellent resources that people have taken the time to write; so here are some great websites that you can also use to supplement your knowledge.

The Simple Guide Guide - http://rogerdudler.github.com/git-guide/
Git Crash Course - http://git.or.cz/course/svn.html
The Definitive Git Website - http://git-scm.com/
Kent Nguyen's Git Practices - http://kentnguyen.com/development/visualized-git-practices-for-team/
Oliver Steel's Git Workflow - http://osteele.com/archives/2008/05/my-git-workflow
Git for Beginners - The definitive practical guide - http://stackoverflow.com/questions/315911/git-for-beginners-the-definitive-practical-guide
Joe Maller's Web Focused Git Work Flow - http://joemaller.com/990/a-web-focused-git-workflow/
Vincent Driessen's Successful Branching Model - http://nvie.com/posts/a-successful-git-branching-model/
Naked Startup's Simple Daily Git Work Flow - http://nakedstartup.com/2010/04/simple-daily-git-workflow

We’re big advocates of branch often and merge often – but this also means updating the DB relatively frequently too (if there has been DB changes on master/trunk). Usually, its just an excuse to go get a fresh cup of coffee whilst you while away the 2 minutes it will take to dump and however long it then takes to re-import into a new DB. But, unfortunately, there is only so much coffee we should be drinking in the day, so speeding up this process is a must.

Typically, a staging or development branch, doesn’t actually required all the data the live site has (sales_quotes, logs, reports, dataflow history), so we gain a big speed advantage by simply ignoring these tables.

Please note. This script is not to serve as a means to backup a database – as it removes data from tables that we deem non-essential when branching/mirroring a store for development.

The features

• Using the script is really easy
• Significantly faster dumps (on a 3GB DB, total time is 0m46s vs 2m10s)
• Auto-population of username, password, database name and host name
• Support for CE and EE
• No table-level locks – so safe to use on a production/live site
• You can use it to re-import the dump too

Dump a database

Running the script is really easy, it will automatically populate the username, password, database name and host from your local.xml file.

cd /path/to/my/magento/store
wget -O mage-dbdump.sh sys.sonassi.com/mage-dbdump.sh
chmod +x mage-dbdump.sh
./mage-dbdump.sh

So in practice, it looks a little like this. Execution on a 3GB database takes just under 50 seconds.

cd /home/sonassi/public_html
wget -O mage-dbdump.sh sys.sonassi.com/mage-dbdump.sh
chmod +x mage-dbdump.sh
./mage-dbdump.sh
########################################	
	
 MYSQL DUMP COMPLETE
	
 Backup Location: ./var/db.sql
 
########################################	

Restore a database

cd /home/sonassi/public_html
wget -O mage-dbdump.sh sys.sonassi.com/mage-dbdump.sh
chmod +x mage-dbdump.sh
./mage-dbdump.sh --restore
Are you sure you want to restore ./var/db.sql to mage_live? [y/N]: y
########################################	
	
 MYSQL IMPORT COMPLETE

########################################	

You can download the file here.

There can be a number of reasons why you would choose to drop all tables in a database to clear it out, rather than dropping the whole database and simply re-creating it, so we wrote a quick bash script to automate the process for you.

Running the script is really easy, it will automatically populate the username, password, database name and host from your local.xml file. It even has 2 prompts to make absolutely sure that you want to run it!

cd /path/to/my/magento/store
wget -O mage-dbpurge.sh sys.sonassi.com/mage-dbpurge.sh
chmod +x mage-dbpurge.sh
./mage-dbpurge.sh

So in practice, it looks a little like this. Execution on a 3GB database takes just under 40 seconds.

# cd /home/sonassi/public_html
# wget -O mage-dbpurge.sh sys.sonassi.com/mage-dbpurge.sh
# chmod +x mage-dbpurge.sh
# ./mage-dbpurge.sh
Are you 100% sure you want to purge $DBNAME? [y/N]: y
Are you 110% sure you want to purge $DBNAME? [y/N]: y
########################################	
	
 MYSQL DB PURGE COMPLETE
 
########################################

You can download the file here.