I have been using the IOZone benchmarking tool to test the IO ability of EC2 running CentOS 4.

In the last post I showed the 3D surface area chart showing how as the file size grows, the io performance degrades, quite sharply as the file migrates from CPU cache to memory cache to disk.

I redid the charts as what Excel calls contour charts, but remind me of heat maps.

The change was striking, suddenly you can easily see the boundary that IOZone has found for
various file and record sizes. The other standout feature was the appearance of holes or cool spots in the charts at specific file and record sizes.

I went back and specifically tested the file and record combination, for example to do read, write and random read/write on a 16M file with 1M record size I used this command:

iozone -R -r 1m -s 16m -i 0 -i 1 -i2

I have used similar settings for the throughput test as well.

Comments:

The whole reason for stepping back and running the benchmark tools is that the results are suggesting that choosing the appropriate column datatype, rowsize, tablesize and memory buffers is potentially going to have a large impact on your databases performance.

Look at these contour maps and you will start to see what I mean. The presence of a ridge as the filesize increases for a band of record sizes is also very interesting.

Previous Articles:

IOZone Benchmark vs EC2 – Part 1

Resources:
http://s3.amazonaws.com/dbadojo_benchmark/iozone_heatmap_writes.JPG
http://s3.amazonaws.com/dbadojo_benchmark/iozone_heatmap_reads.JPG
http://s3.amazonaws.com/dbadojo_benchmark/iozone_heatmap_random_writes.JPG
http://s3.amazonaws.com/dbadojo_benchmark/iozone_heatmap_random_reads.JPG
Zipped Excel Spreadsheet for IOZone Benchmark data and charts

Have Fun
Paul

Here are some pretty surface area graphs from the EC2 benchmark, the stepping down indicates from CPU cache to Memory cache, the last cliff is down to disk once the file was larger the available memory.

As I mentioned yesterday I was running off a IOzone benchmark on EC2 to see how the disk performs, after reading about it in this online benchmark article.
There are a couple of nice features with this benchmark

1. Output is saved in a format ready for surface area graphs in Excel
2. Gnuplot options available as well
3. It tests stride size to see if there are any stripe boundary or IO library issues.

Another first is you can download my results in this file iozone_benchmark_ec2.zip served from Amazon S3 (Right click SAVE AS)

The IOzone documentation[PDF] which is short, but detailed.

Installing IOzone:

1. wget http://www.iozone.org/src/current/iozone-3-283.i386.rpm
2. rpm -Uvh iozone-3-283.i386.rpm
3. export PATH=$PATH:/opt/iozone/bin

Running a benchmark:

Note: File should be larger than available memory. -g 2G indicates 2 Gigabyte file.

1. iozone -Ra -g 2G > iozone.out

The full size graphs can be found via Amazon S3

http://s3.amazonaws.com/dbadojo_benchmark/iozone_ec2_write.GIF
http://s3.amazonaws.com/dbadojo_benchmark/iozone_ec2_read.GIF
http://s3.amazonaws.com/dbadojo_benchmark/iozone_ec2_random_read.GIF
http://s3.amazonaws.com/dbadojo_benchmark/iozone_ec2_random_write.GIF

Have Fun

Paul

Andy, a reader of the blog left a comment asking if I could run some benchmarking of EC2.

If someone takes the time to comment, making the effort to respond is always worthwhile. Feedback drives most conversation, business and innovation.

So I went off and google’ed the most appropriate and easiest benchmarking tool.

http://www.tux.org/pub/benchmarks/
http://oss.sgi.com/LDP/HOWTO/Benchmarking-HOWTO.html
http://www.coker.com.au/bonnie++/
http://www.acnc.com/benchmarks.html
http://portal.acm.org/citation.cfm?id=71309 IOBench

I settled on bonnie and bonnie++, both may seem a little long in the tooth given when they were developed but they serve the need to test the raw speed of both the root partition and /mnt partition which comes when you run an EC2 virtual machine or Amazon Machine Image (AMI).

If you want to see other web posts on benchmarking EC2, I found a couple of good articles as well.

DeCare Systems has a bunch of articles on EC2 this one has information on using a java benchmarking tool, Javolution
http://blog.decaresystems.ie/index.php/2007/01/29/amazon-web-services-the-future-of-datacenter-computing-part-1/

Other articles on benchmarking on EC2:
http://paul-m-jones.com/blog/?p=238

Comments:

1. Bonnie required less dependent packages then bonnie++
2. Both tools were easy to install and run.
3. Both tools saturated IO and therefore bypassed any issues with caching when the filesize was sufficiently large

I will followup with some more benchmarks and analysis of the results in the next couple of days and then it is back onto MySQL and Oracle.

Installing bonnie on CentOS 4.4

1. Download bonnie: wget http://www.tux.org/pub/benchmarks/Disk_IO/bonnie.tar.gz or from Google Code
2. Install GCC: yum install gcc
3. Compile: gcc -O2 -o bonnie bonnie.c
4. Run with 100M file: ./bonnie -d /mnt/bonnie -s 100 -m centos4
5. Run with 1G file: ./bonnie -d /mnt/bonnie -s 1024 -m centos4

Results for Bonnie

100M file:

              -------Sequential Output-------- ---Sequential Input-- --Random--
             -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
centos4  100 25464 53.0 166352 47.1 189359 55.5 25127 53.7 412332 52.3 36552.4 45.7
centos4  100 25038 52.6 216190 61.2 193490 54.8 24317 52.2 418285 49.0 34320.0 51.5
centos4  100 25535 53.4 123481 37.4 188139 57.0 25472 54.5 417667 48.9 72301.4 90.4
centos4  100 25118 52.7 130512 39.5 191710 54.3 25546 53.6 576862 62.0 80402.0 100.5
centos4  100 24205 52.9 183853 53.9 223497 61.1 24852 51.9 400162 54.7 35898.3 35.9

1 Gig file:

             -------Sequential Output-------- ---Sequential Input-- --Random--
             -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
centos4  1024 23573 50.0 28648  6.2 19739  4.5  6892 12.1 409836 38.7 38105.0 38.1

Installing bonnie++ on CentOS 4.4

1. Download: wget http://www.coker.com.au/bonnie++/bonnie++-1.03a.tgz
2. Install dependencies: yum install compat-gcc-32-c++.i386 gcc-c++.i386 libstdc++.i386
3. Configure bonnie++: ./configure
4. Make bonnie++: make
5. Run on /mnt: ./bonnie++ -d /mnt/oracle -s 3000 -n 1 -m centOS4 -x 3 -r 1500 -u oracle
6. Run on /: ./bonnie++ -d /home/oracle -s 3000 -n 1 -m centOS4 -x 3 -r 150

Results for Bonnie++

/ mountpoint:

name,file_size,putc,putc_cpu,put_block,put_block_cpu,rewrite,rewrite_cpu,getc,getc_cpu,get_block,get_block_cpu,seeks,seeks_cpu,num_files,seq_create,seq_create_cpu,seq_stat,seq_stat_cpu,seq_del,seq_del_cpu,ran_create,ran_create_cpu,ran_stat,ran_stat_cpu,ran_del,ran_del_cpu
centOS4,3000M,13469,28,59124,15,19772,1,21629,38,51205,1,254.5,0,1,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
centOS4,3000M,17203,36,57555,15,20025,1,22490,40,49618,0,247.9,0,1,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
centOS4,3000M,23918,49,54411,14,19845,1,23120,41,52089,1,246.4,0,1,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

/mnt mountpoint:

name,file_size,putc,putc_cpu,put_block,put_block_cpu,rewrite,rewrite_cpu,getc,getc_cpu,get_block,get_block_cpu,seeks,seeks_cpu,num_files,seq_create,seq_create_cpu,seq_stat,seq_stat_cpu,seq_del,seq_del_cpu,ran_create,ran_create_cpu,ran_stat,ran_stat_cpu,ran_del,ran_del_cpu
centOS4,3000M,23847,48,45251,12,15149,2,19438,38,41982,5,199.8,0,1,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
centOS4,3000M,23189,49,42246,11,16938,4,21007,41,52733,1,183.9,0,1,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
centOS4,3000M,24195,49,44167,11,19923,2,20465,40,47364,1,185.0,0,1,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++