Simple Magento performance/load testing with Mage-Perftest

Mage-Perftest is a simple Linux command line tool to test the performance of your Magento store, it can perform a number of clever operations which makes it far more suitable than siege or ab. Its not a replacement for fully fledged simulation tools (like jMeter), but it is simple to set up and run.

You can use it for a number of different things,

  1. PHP TTFB (time to first byte) performance testing
  2. Whole page (including static content & assets) performance testing
  3. Load/stress testing
  4. Concurrency testing
  5. Repeat testing
  6. Site cache priming/crawling

It also has the ability to,

  1. Use sessions during crawling
  2. Supports keepalives
  3. Bypass Varnish cash (to test actual server performance)
  4. Read and parse Magento sitemaps (eg. <loc>http://www.example.com</loc> format)
  5. Use fixed and random seeds when parsing sitemaps 1. for either completely random tests, or fixed random tests
  6. Simulate traffic over a defined period of time (eg. 1000 unique visitors in a single hour)

What it doesn't do,

  • Take into account CSS/JS render time (this is deliberate, the tool is there to test server performance, not browser)
  • Replicate exact customer behavior (eg. add to cart, login, purchase) - although you could somewhat simulate this by making a sitemap that consists of these URLs

Downloading Mage-Perftest

There are two pre-compiled binaries,

Using Mage-Perftest

First download the binary and make it executable

wget http://sys.sonassi.com/mage-perftest
chmod +x mage-perftest

Then execute it to see the available options

~#  ./mage-perftest

      -a [0-9]+      Abort/discard requests if CPU load percent is above defined limit
      -b             PHP benchmark only (no assets downloaded)
      -c [0-9]+      Concurrency level (# users)
      -d [0-9]+      Duration (minutes) runs test over period by setting delay
      -D             IP Address (force DNS resolution to different IP)
      -f             Ignore font files in full page test (woff|woff2|eot|ttf|otf)
      -g             Enable gzip request headers
      -l [0-9]+      Load average limit - pauses test when load avg. exceeds
                      the defined limit (local testing only). [Default: 30]
      -r [0-9]+      Test repetitions
      -R             Randomise sitemap requests
      -s [URL|FILE]  URL/Path for XML sitemap (use with -r to limit URLs crawled)
      -t [0-9]+      Time delay (seconds) between requests (use with -c and/or -z)*
                      automatically, overrides -t setting (use with -r)
      -u [URL]       URL to test
      -v             Bypass Varnish (if possible)
      -x             Use fixed seed with random requests
      -z             Load test - run requests without waiting between responses*

      * May mean final figures are skewed/meaningless

Examples of use

Single URL, full page test (including static content)

./mage-perftest -u www.example.com

Single URL, full page test (including static content, without fonts)

./mage-perftest -u www.example.com -f

Single URL, PHP TTFB only (excluding all content)

./mage-perftest -u www.example.com -b

Use sitemap for URLS, selects 10 seeded random URL from sitemap, full page test (including static content)

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 10

Use sitemap for URLS, selects 10 seeded random URL from sitemap, PHP TTFB only (excluding all content)

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 10 -b      

Use sitemap for URLS, selects 10 completely random URL from sitemap, full page test (including static content)

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 10 -R

Single URL, PHP TTFB only (excluding all content), 25 concurrent users

./mage-perftest -u www.example.com -c 25 -b

Use sitemap for URLS, selects 100 completely random URL from sitemap, runs PHP load test (ie. every URL is requested at the same time)

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 100 -b -z -R

Use sitemap for URLS, selects 100 completely random URL from sitemap, runs PHP load test over the course of 10 mins (eg. 1 request every 6 seconds)

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 100 -b -R -d 10 -z

Simulate load of 1000 unique visitors per hour, viewing 8 pages per visitor (1000 * 8 = 8000 requests), ( 1 hour = 60 minutes)

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 8000 -R -d 60 -z

Prime caches, use sitemap for URLS, selects 100 URLs from sitemap, crawls 1 request every 3 seconds

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r 100 -b -t 3 -z

Prime caches, use sitemap for URLS, selects all URLs from sitemap, crawls over the course of 1 day

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r auto -b -d 1440 -z

Prime caches, use sitemap for URLS, selects all URLs from sitemap, crawls over the course of 1 day and pauses test if load average of 5 is reached

./mage-perftest -u www.example.com -s www.example.com/sitemap.xml -r auto -b -d 1440 -z -l 5

Generate Sitemaps on MageStack

You can also get pretty creative with the content you might need in your crawler/load-tester. Using the following command, it parses the web server access logs on MageStack to find all the accessed URLs, then sorts them by the most frequently accessed (apart from requests from the server itself eg. mage-perf) and outputs it to a file called mageperf_sitemap.xml.

This way, you can performance test the URLs that are most commonly hit (or prime your cache for the common URLs).

apt-get install -y ack-grep
DOMAIN="www.example.com"
ack-grep "^(?=.*$DOMAIN)(?"'!'"172.16.[0-9.]+).*GET (/[^s]+)" /microcloud/logs/web/nginx-access.log --output='<loc>$1</loc>' | sort | uniq -c | sort -nr > mageperf_$DOMAIN_sitemap.xml
./mage-perftest -u $DOMAIN -s mageperf_$DOMAIN_sitemap.xml -r auto -b -d 1440 -z