Prewarm your EBS backed EC2 MySQL slaves

This is the story of cold blocks and mismatched instances and how they will cause you pain and cost you money until you understand why.

Most of the clients that we support run on the Amazon cloud using either RDS or running MySQL on plain EC2 instances using (Provisioned IOPS) PIOPS EBS for data storage.

As expected the common architecture is running a master with one or more slaves handling the read traffic.

A common problem is that after the slaves are provisioned (normally created from an EBS snapshot) they lag badly due to slow IO performance.

Unfortunately what tends to be lost in the “speed of provisioning new resources” fetish is some limitations in terms of data persistence layer (EBS).

If you are using EBS and you have created the EBS volume from snapshot or created a new volume you have to pre-warm the EBS volume otherwise you will suffer a bad (I mean seriously bad) first usage penalty.  Bad? I am talking up to 50% performance drop[1]. So that expensive PIOPS EBS volume you created is going to perform like rubbish every time it reads/writes a cold block.

The other thing which also tends to happen is mixing up the wrong instance (network performance) with the PIOPS EBS. This the classic networked storage, the network is the bottleneck. If your instance type has limited network performance, having a higher PIOPS than the network can handle means you are wasting money (on PIOPS) you can’t use. A bit like in the old days (of dedicated servers and SAN storage) where the SAN could deliver 200-300Mbytes per sec, but the 1 Gigabit network could only do 40-50Mbytes per sec.

Here is the real downside, using the cloud you can provision new resources to handle peak load (in the case more MySQL slaves to handle read load) as fast as you can click, or faster using API calls, or even automagically, if you have some algo forecast the need for additional resources. But… the EBS is all cold blocks, so these new instances will be up and available in minutes but the IO performance will be poor until you either pre-warm or the slave gets around to writing/reading all blocks.

So the common solution is to pre-warm the blocks using dd to read the EBS device (and warm the block) to /dev/null

eg: sudo dd if=/dev/xvdf of=/dev/null bs=1M

Consider how long this will take for any reasonable sized DB (200GBytes) using an instance with 1 Gigabit network.

200Gigabytes read at 50Mbytes/sec  = 200,000 Mbytes/50 = 4000 secs = 3600 (1hr) + 400 (6 mins 40 secs) =~ more than 1 hr.

So you or your algo provisioned a new EC2 instance for the database in minutes but either your IO will be rubbish for an extended period, or you wait more than 1 hr per 200GB to have the EBS pre-warmed.

What are the solutions?

  1. Forecast further in advance depending on the size of your db (or any other persistent storage layer eg NoSQL etc)
  2. Use ephemeral storage and manage the increased risk of data loss in the event of instance termination.
  3. Break your DB or your application into smaller pieces aka micro services.[2]
  4. Pay more $ and have your databases stay around longer so waiting for a instance to be ready in the beginning is not a problem.

As you can expect, most businesses are happy with option 4. Pay more, leave instances around like they were dedicated servers (base load). Amazon is happy too.

Option 3 whilst requiring some thought (argh) and additional complexity is where the real speed of provisioning, dare I say it, agile nature of the cloud will bear the most fruit.

[1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-prewarm.html

[2] http://martinfowler.com/articles/microservices.html

 

Oracle RAC on EC2 redux

I was reading some RSS feeds the other day and noticed that Jeremy Schneider over at Ardent Performance Computing was working on getting Oracle RAC working on Amazon EC2.
http://www.ardentperf.com/2011/03/04/byo-oracle-rac-on-ec2/
He looks to have solved the whole Virtual IP issue by using another instance. Nice solution!

When I get a spare moment (don’t believe for a minute that the lack of posts means I am not busy) it would be good to take the scripts and get the whole Oracle RAC working in Amazon EC2 finally!

Other News:

I have had the chance to play with some columnar databases, Vertica and Ingres VectorWise and the performance is good. I used the TPC-H benchmark to test to a scale 20 (small only due to a lack of disk space). So the results are nothing like the recent Scale 100 benchmarks that Ingres VectorWise did but useful nonetheless. Sadly I can’t publish any scripts or results as the IP is owned by my current employer.

Currently I am focused on improving my skills in predictive modeling and analytics. This is using the data rather than just supporting/recovering and hand-holding the data a.k.a. being a DBA.

Listening to trance, in the zone and most definitely Having Fun!

Paul

OurDelta MySQL on EC2 – install

Summary:

Arjen would give me an earful if I got this wrong or poorly worded.

“OurDelta produces enhanced builds for MySQL, with OurDelta and third-party patches, for common production platforms” from http://ourdelta.org/about

Over the next series of articles I am going to put the many additions to the MySQL 5.0 baseline through their paces on Amazon EC2.

Using a base CentOS 4.4 I had lying around on Amazon S3 I had Ourdelta installed and secure in no time flat.

This is outline for the installing Ourdelta onto CentOS 4

Install:


yum install yum-plugin-protectbase
rpm --import http://mirror.ourdelta.org/deb/ourdelta.gpg
rpm -Uvh http://mirror.ourdelta.org/yum/CentOS/OurDelta-release-0.0.4-1.noarch.rpm
mkdir downloads
cd downloads/
wget http://mirror.ourdelta.org/yum/CentOS/4/i386/RPMS/MySQL-OurDelta-server-5.0.67.d7-44.el4_7.i386.rpm
wget http://mirror.ourdelta.org/yum/CentOS/4/i386/RPMS/MySQL-OurDelta-shared-5.0.67.d7-44.el4_7.i386.rpm
wget http://mirror.ourdelta.org/yum/CentOS/4/i386/RPMS/MySQL-OurDelta-test-5.0.67.d7-44.el4_7.i386.rpm
wget http://mirror.ourdelta.org/yum/CentOS/4/i386/RPMS/MySQL-OurDelta-client-5.0.67.d7-44.el4_7.i386.rpm
wget http://mirror.ourdelta.org/yum/CentOS/4/i386/RPMS/MySQL-OurDelta-devel-5.0.67.d7-44.el4_7.i386.rpm
yum localinstall MySQL-OurDelta-*
/usr/bin/mysql_secure_installation
ps -ef|grep mysql
mysql -u root -p

Comments:

Thats it. Straightforward install.
I am getting lazy in my old age I used a pipe combo to generate the install commands using this:

history |grep -v history|awk ‘{ print $2″ “$3” “$4 }’

The next article will cover the upgrade of any existing MySQL 5.0 release to OurDelta.
I have saved this Ourdelta MySQL install as an AMI.

If people are interested in making this a public AMI please post a comment.

Have Fun

Paul

Sysbench fileio vs small EC2 with EBS

Background:

I got a request from a reader to repeat the sysbench fileio benchmarks against the new Amazon EBS (Elastic Block Store).

As mentioned elsewhere as this is similar to network attached storage (NAS) it will be constrained by the speed of network.

The past series on sysbench fileio on EC2

Sysbench fileio vs EC2 Part 1
Sysbench fileio vs Large EC2 Part 2
seeker io benchmark vs EC2

Results:

As expected the network (1 Gigabit) is the upper limit of throughput (MB/s).

What this means is the local disk (not /mnt) is still the fastest disk available in terms of requests per sec and throughput (MB/s).

Adding an addition EBS volume as a RAID 0 was marginally better for single threaded reads but for many threads both the read performance was better.
The write performance was much better for single and many threads.

The additional flexibility of being able to attach and detach volumes, take snapshots is the big plus for using EBS.

At this point I made no attempt to optimize the RAID 0 chunksize. It would require more work, however at the end of the day, 90-100 MB/s is maximum.

How to use:

Same as before, stick your intensive IO portions on / and anything readonly on either /mnt or on Amazon EBS.

I am hanging out for an instance which provides infiniband network speeds or better. A bunch of clustering technology (shared nothing and shared disk) rely on good network bandwidth.

Have Fun

p.s. I had written this up about a month and half ago however delayed publishing as it isn’t a great result i.e. network disk is constrained by network bandwidth.

How to use EC2 mountpoints

It has been quiet around here. My workload has been high and I have been spending my down time reading, rather than experimenting and hence publishing.

Frederic commented in a recent post about Sysbench fileio benchmark.
If you want to see whole set of posts use EC2 Benchmark

There were three questions and my reply was going to be long. So now it is a post.

Questions:

  1. /dev/sda/ : do you mean that it would be better to separate the tables of the database or simply use /dev/sda (if 10GB is enough for DB storage)
  2. I found info about db caching (http://dev.mysql.com/doc/refman/5.0/en/query-cache.html), but don’t know what you are talking about file caching. Is there any link you could share about this principle.
  3. what kind of filesystem do you use on /mnt and what is the size of the partition created.

Answers:

  1. Given what the IO benchmarks are saying when they bypass the OS caching mechanism i.e. /dev/sda is faster than /dev/sdb for writes. Use /dev/sda for your binlogs and anything you want to have fast write access. Many databases are 95-98% reads so putting them on the /dev/sdb (/mnt) with OS caching is going to fine.
  2. There are many levels of caching. Google for Facebook MySQL presentation. Many websites which need to scale use Memcache upfront of the database. This means that most reads from the database are served from Memcache. MySQL also has a query cache as you found. File caching is just OS caching. This is normally on by default on most linux flavours. Essentially there is no point in wasting any free RAM, so the OS will cache any physical reads so that next time they can be served from memory rather than disk.
  3. I was just using the standard ext3 file system. You can run Logical Volumes on top of it. There are people who will argue for the pros and cons. It was more just what I was familiar with at the time. I was not stressing about the file system type, more about the difference in performance of the two virtual storages available.

Have Fun

Paul

Sysbench fileio vs XLarge EC2 Part 3

Background:

If you have been following the series. I have gone back to checking the raw IO performance of all the Amazon EC2 instances.
Sysbench fileio vs EC2 Part 1
Sysbench fileio vs Large EC2 Part 2
seeker io benchmark vs EC2

In the meantime Amazon has finally announced that they are providing persistent storage for all EC2 instances. This has removed a bunch of complexity from running a database on any EC2 instance.

Results:

The percentage improvement of the extra large instance over the large instance was pretty minor. It is still much faster than the small instance. There was some better results for the higher threads, more files random reads and random writes.

Comparison to seeker benchmark:

Seeker benchmark got the following for the extra large instance:

/ (/dev/sda1) : Results: 1547 seeks/second, 0.65 ms random access time
/mnt (/dev/sdb): Results: 1167 seeks/second, 0.86 ms random access time

The seeker benchmark were again widely different when there was no space used on /mnt
compared to when there was.
The Xen virtual machine is not cheating, but it is not showing a realistic result when /mnt (/dev/sdb) is empty.

Comments:

All the results were double-checked by running iostat -x whilst the benchmarks were running. Confirming the results for both sysbench and seeker.

My recommendation, based on these results remains the same as before:

  1. Place files which require good raw read and write performance (without cache) on /dev/sda
  2. Use linux file cache or database caching for most reads. But you know that already right.
  3. Don’t rely on good read or write performance without caching on /mnt.
  4. iostat is your friend. You can check your requests/sec, wait queues and disk utilization on your application, not just a benchmark tool.

If/When I get accepted to the persistence storage beta, I will rerun these tests again. It should be interesting.

Raw Results:



Specs EC2 xlarge instance

[root@ip-10-251-146-159 sysbench-0.4.8]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 270
stepping : 2
cpu MHz : 2004.542
cache size : 1024 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips : 5013.04
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor : 1
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 270
stepping : 2
cpu MHz : 2004.542
cache size : 1024 KB
physical id : 1
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips : 5013.04
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor : 2
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 270
stepping : 2
cpu MHz : 2004.542
cache size : 1024 KB
physical id : 2
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips : 5013.04
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor : 3
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 270
stepping : 2
cpu MHz : 2004.542
cache size : 1024 KB
physical id : 3
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips : 5013.04
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

[root@ip-10-251-146-159 sysbench-0.4.8]# free
total used free shared buffers cached
Mem: 15731980 581388 15150592 0 12724 217576
-/+ buffers/cache: 351088 15380892
Swap: 0 0 0
[root@ip-10-251-146-159 sysbench-0.4.8]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 1.2G 8.3G 12% /
/dev/sdb 414G 199M 393G 1% /mnt
[root@ip-10-251-146-159 sysbench-0.4.8]# uname -a
Linux ip-10-251-146-159 2.6.20-1.3002.fc6xen #1 SMP Mon Aug 13 14:21:21 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux
[root@ip-10-251-146-159 sysbench-0.4.8]# cat /proc/meminfo
MemTotal: 15731980 kB
MemFree: 15150832 kB
Buffers: 12752 kB
Cached: 217624 kB
SwapCached: 0 kB
Active: 163920 kB
Inactive: 71036 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 4568 kB
Mapped: 3464 kB
Slab: 19628 kB
SReclaimable: 11984 kB
SUnreclaim: 7644 kB
PageTables: 832 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 7865988 kB
Committed_AS: 14628 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 68 kB
VmallocChunk: 34359738299 kB
[root@ip-10-251-146-159 sysbench-0.4.8]#

sysbench_fileio.sh

[root@ip-10-251-146-159 ~]# ./sysbench_fileio.sh
sysbench v0.4.8: multi-threaded system evaluation benchmark

1 files, 131072Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 767647 Read, 0 Write, 0 Other = 767647 Total
Read 11.713Gb Written 0b Total transferred 11.713Gb (199.91Mb/sec)
12794.04 Requests/sec executed

Test execution summary:
total time: 60.0004s
total number of events: 767647
total time taken by event execution: 58.0262
per-request statistics:
min: 0.0001s
avg: 0.0001s
max: 0.0391s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 767647.0000/0.00
execution time (avg/stddev): 58.0262/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Done.

Operations performed: 1000000 Read, 0 Write, 0 Other = 1000000 Total
Read 15.259Gb Written 0b Total transferred 15.259Gb (384.1Mb/sec)
24582.41 Requests/sec executed

Test execution summary:
total time: 40.6795s
total number of events: 1000000
total time taken by event execution: 2600.3658
per-request statistics:
min: 0.0002s
avg: 0.0026s
max: 0.5825s
approx. 95 percentile: 0.0027s

Threads fairness:
events (avg/stddev): 15625.0000/2.68
execution time (avg/stddev): 40.6307/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 593313 Write, 0 Other = 593313 Total
Read 0b Written 9.0532Gb Total transferred 9.0532Gb (154.51Mb/sec)
9888.47 Requests/sec executed

Test execution summary:
total time: 60.0005s
total number of events: 593313
total time taken by event execution: 58.4663
per-request statistics:
min: 0.0001s
avg: 0.0001s
max: 4.8524s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 593313.0000/0.00
execution time (avg/stddev): 58.4663/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 525171 Write, 0 Other = 525171 Total
Read 0b Written 8.0135Gb Total transferred 8.0135Gb (127.25Mb/sec)
8144.28 Requests/sec executed

Test execution summary:
total time: 64.4834s
total number of events: 525171
total time taken by event execution: 4124.9270
per-request statistics:
min: 0.0001s
avg: 0.0079s
max: 64.4652s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 8205.7969/17156.01
execution time (avg/stddev): 64.4520/0.04

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...
sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 1024Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 951836 Write, 0 Other = 951836 Total
Read 0b Written 14.524Gb Total transferred 14.524Gb (247.86Mb/sec)
15862.76 Requests/sec executed

Test execution summary:
total time: 60.0044s
total number of events: 951836
total time taken by event execution: 3837.2266
per-request statistics:
min: 0.0004s
avg: 0.0040s
max: 6.8536s
approx. 95 percentile: 0.0058s

Threads fairness:
events (avg/stddev): 14872.4375/49.38
execution time (avg/stddev): 59.9567/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 16384Kb each, 2048Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 5029 Read, 0 Write, 0 Other = 5029 Total
Read 78.578Mb Written 0b Total transferred 78.578Mb (1.3095Mb/sec)
83.81 Requests/sec executed

Test execution summary:
total time: 60.0043s
total number of events: 5029
total time taken by event execution: 59.9890
per-request statistics:
min: 0.0001s
avg: 0.0119s
max: 1.2131s
approx. 95 percentile: 0.0361s

Threads fairness:
events (avg/stddev): 5029.0000/0.00
execution time (avg/stddev): 59.9890/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 14103 Read, 0 Write, 0 Other = 14103 Total
Read 220.36Mb Written 0b Total transferred 220.36Mb (3.6463Mb/sec)
233.36 Requests/sec executed

Test execution summary:
total time: 60.4337s
total number of events: 14103
total time taken by event execution: 3854.5087
per-request statistics:
min: 0.0003s
avg: 0.2733s
max: 0.9353s
approx. 95 percentile: 0.5471s

Threads fairness:
events (avg/stddev): 220.3594/1.71
execution time (avg/stddev): 60.2267/0.13

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 33598 Write, 0 Other = 33598 Total
Read 0b Written 524.97Mb Total transferred 524.97Mb (8.7416Mb/sec)
559.46 Requests/sec executed

Test execution summary:
total time: 60.0542s
total number of events: 33598
total time taken by event execution: 59.9642
per-request statistics:
min: 0.0001s
avg: 0.0018s
max: 6.7853s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 33598.0000/0.00
execution time (avg/stddev): 59.9642/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...

(last message repeated 63 times)
Done.

Operations performed: 0 Read, 226 Write, 0 Other = 226 Total
Read 0b Written 3.5312Mb Total transferred 3.5312Mb (46.928Kb/sec)
2.93 Requests/sec executed

Test execution summary:
total time: 77.0549s
total number of events: 226
total time taken by event execution: 4365.1668
per-request statistics:
min: 0.7881s
avg: 19.3149s
max: 56.0941s
approx. 95 percentile: 45.2706s

Threads fairness:
events (avg/stddev): 3.5312/0.85
execution time (avg/stddev): 68.2057/5.70

sysbench v0.4.8: multi-threaded system evaluation benchmark



iostat -x


[root@ip-10-251-146-159 ~]# iostat -x 1 |egrep "sda1|sdb"
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sdb 0.04 0.00 0.05 0.01 0.77 0.08 13.80 0.00 5.31 4.00 0.02
sda1 2.13 39.92 5.01 7.66 194.36 380.66 45.37 0.11 8.59 0.68 0.86
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 9.00 0.00 2.00 0.00 88.00 44.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 15.84 29500.00 1162.38 2992.08 37196.04 259936.63 71.52 83.01 19.98 0.16 65.35
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 88.89 0.00 12263.64 0.00 391886.87 0.00 31.96 0.75 0.06 0.06 75.15
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 75.25 0.00 12295.05 0.00 393188.12 0.00 31.98 0.67 0.05 0.05 66.93
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 83.17 0.00 11434.65 0.00 365592.08 0.00 31.97 0.72 0.06 0.06 71.68
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 74.75 0.00 11310.10 0.00 361632.32 0.00 31.97 0.69 0.06 0.06 69.09
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 84.85 0.00 11898.99 0.00 380412.12 0.00 31.97 0.73 0.06 0.06 72.73
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 100.00 7.00 12185.00 3.00 389640.00 80.00 31.98 0.76 0.06 0.06 76.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 95.05 0.00 12287.13 0.00 392776.24 0.00 31.97 0.71 0.06 0.06 70.89
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 108.00 0.00 12829.00 0.00 410080.00 0.00 31.97 0.75 0.06 0.06 74.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 100.00 0.00 12552.00 0.00 401216.00 0.00 31.96 0.72 0.06 0.06 72.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 105.05 0.00 12593.94 0.00 402650.51 0.00 31.97 0.77 0.06 0.06 77.17
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 73.27 0.99 12486.14 1.98 399112.87 23.76 31.96 0.71 0.06 0.06 71.29
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 103.03 0.00 12582.83 0.00 402262.63 0.00 31.97 0.74 0.06 0.06 73.54
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 75.25 0.00 12286.14 0.00 392681.19 0.00 31.96 0.80 0.07 0.07 80.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 84.00 0.00 12086.00 0.00 386304.00 0.00 31.96 0.84 0.07 0.07 82.40
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 97.03 0.00 11920.79 0.00 381021.78 0.00 31.96 0.78 0.07 0.07 78.42
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 106.00 0.00 12314.00 0.00 393760.00 0.00 31.98 0.69 0.06 0.06 68.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 78.79 1.01 12480.81 2.02 398965.66 24.24 31.96 0.72 0.06 0.06 71.52
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 98.00 0.00 12184.00 0.00 389536.00 0.00 31.97 0.77 0.06 0.06 76.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 104.00 0.00 12159.00 0.00 388672.00 0.00 31.97 0.80 0.07 0.06 78.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 101.01 0.00 11946.46 0.00 381963.64 0.00 31.97 0.76 0.06 0.06 76.36
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 92.93 0.00 11934.34 0.00 381575.76 0.00 31.97 0.75 0.06 0.06 74.75
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 88.24 0.00 11488.24 0.00 367184.31 0.00 31.96 0.78 0.07 0.07 77.25
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 78.79 1.01 11486.87 2.02 367321.21 24.24 31.97 0.86 0.07 0.07 85.66
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 93.07 0.00 11892.08 0.00 380039.60 0.00 31.96 0.79 0.07 0.07 78.81
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 92.93 0.00 12095.96 0.00 386585.86 0.00 31.96 0.79 0.07 0.06 78.38
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 89.11 0.00 12023.76 0.00 384285.15 0.00 31.96 0.80 0.07 0.07 80.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 76.77 0.00 12563.64 0.00 401583.84 0.00 31.96 0.83 0.07 0.07 82.83
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 104.95 0.00 12185.15 0.00 389607.92 0.00 31.97 0.82 0.07 0.07 82.38
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 96.00 10.00 12295.00 45.00 393088.00 440.00 31.89 0.73 0.06 0.06 72.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 128.00 0.00 14030.00 0.00 448480.00 0.00 31.97 0.69 0.05 0.05 68.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 110.89 0.00 13545.54 0.00 433013.86 0.00 31.97 0.71 0.05 0.05 71.29
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 117.17 0.00 13342.42 0.00 426408.08 0.00 31.96 0.69 0.05 0.05 68.69
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 92.93 0.00 13596.97 0.00 434715.15 0.00 31.97 0.66 0.05 0.05 65.86
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 79.21 0.00 13179.21 0.00 421196.04 0.00 31.96 0.70 0.05 0.05 69.70
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 114.00 1.00 14216.00 2.00 454560.00 24.00 31.97 0.70 0.05 0.05 70.40
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 112.00 0.00 13892.00 0.00 444128.00 0.00 31.97 0.72 0.05 0.05 71.60
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 108.00 0.00 13439.00 0.00 429568.00 0.00 31.96 0.60 0.04 0.04 59.60

iostat -x at five second intervals

[root@ip-10-251-146-159 ~]# iostat -x 5 |egrep "sda1|sdb"
sdb 0.04 0.00 0.05 0.01 0.73 0.08 13.80 0.00 5.31 4.00 0.02
sda1 6.14 66.66 540.68 10.24 17316.42 615.22 32.55 0.22 0.39 0.07 4.02
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 100.80 2.00 12478.00 0.40 398976.00 19.20 31.97 0.74 0.06 0.06 74.40
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 111.60 0.20 14267.40 0.40 456108.80 4.80 31.97 0.64 0.05 0.05 64.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 116.60 0.00 14198.80 0.00 453900.80 0.00 31.97 0.74 0.05 0.05 74.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 257.49 0.40 23116.97 1.40 741703.79 14.37 32.08 55.29 2.39 0.04 96.69
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 281.96 1.60 25049.90 0.40 803738.68 16.03 32.09 63.47 2.53 0.04 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 252.00 0.20 22304.40 0.40 715628.80 4.80 32.08 63.48 2.85 0.04 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 274.65 0.20 23675.65 0.40 759632.73 4.79 32.08 63.29 2.67 0.04 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 287.80 0.20 24654.80 0.40 791142.40 4.80 32.09 63.29 2.57 0.04 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 286.40 4.60 24286.40 1.60 779385.60 49.60 32.09 63.37 2.61 0.04 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 304.60 0.20 25854.40 0.40 829440.00 4.80 32.08 63.36 2.45 0.04 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 294.00 0.00 25450.40 0.20 816780.80 1.60 32.09 63.30 2.49 0.04 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 76.40 0.80 6545.80 5196.00 210112.00 164884.80 31.94 17.08 1.44 0.07 87.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.40 0.00 6342.20 0.00 201396.80 31.76 0.88 0.18 0.14 87.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 12910.42 0.00 409373.95 31.71 0.74 0.06 0.06 73.91
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 12959.40 0.00 411294.40 31.74 0.73 0.06 0.06 72.64
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 13295.00 0.00 421681.60 31.72 0.73 0.05 0.05 71.44
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 13015.80 0.00 412952.00 31.73 0.75 0.06 0.06 74.56
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10343.51 0.00 328059.08 31.72 0.82 0.08 0.08 80.48
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.60 0.00 4327.25 0.00 137195.99 31.71 1.62 0.08 0.21 91.06
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 6437.33 0.00 204322.55 31.74 2.03 0.51 0.13 84.87
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10635.80 0.00 337425.60 31.73 0.79 0.07 0.07 78.40
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.60 0.00 10701.00 0.00 339233.67 31.70 0.78 0.07 0.07 78.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10719.96 0.00 340003.19 31.72 0.77 0.07 0.07 76.89
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.60 0.00 10713.80 0.00 340265.60 31.76 0.77 0.07 0.07 76.56
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10963.73 0.00 347836.47 31.73 0.80 0.07 0.07 79.68
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2322.55 0.00 73606.39 31.69 0.95 0.05 0.41 95.17
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.60 0.00 5976.25 0.00 189649.50 31.73 0.85 0.28 0.14 84.55
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 13295.60 0.00 421867.20 31.73 0.75 0.06 0.06 74.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 12889.00 0.00 408760.00 31.71 0.73 0.06 0.06 72.24
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10618.00 0.00 336894.40 31.73 0.83 0.08 0.08 81.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10827.00 0.00 343294.40 31.71 0.79 0.07 0.07 78.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10549.10 0.00 334728.66 31.73 0.79 0.07 0.07 77.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.20 0.00 10795.80 0.00 342539.20 31.73 0.78 0.07 0.07 77.60
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.60 0.00 9832.00 0.00 311672.00 31.70 0.78 0.07 0.08 77.44
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 3510.20 0.00 533.20 0.00 32654.40 61.24 3.60 10.66 1.80 96.16
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 461.52 0.00 67.13 0.00 4181.16 62.28 3.53 51.02 14.88 99.88
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 2078.49 0.40 4928.88 3.19 166224.70 33.72 44.53 2.56 0.20 97.77
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 184.37 0.00 6050.90 0.00 193478.16 31.98 49.01 13.42 0.17 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 643.31 0.00 20252.89 0.00 647453.89 31.97 48.47 2.39 0.05 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 569.94 0.00 19219.44 0.00 614432.06 31.97 48.60 2.53 0.05 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 484.20 0.00 15684.00 0.00 500665.60 31.92 48.27 2.51 0.06 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.20 0.00 6.40 32.00 43.54 3312.00 5000.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 563.27 0.00 18312.57 0.00 585363.67 31.97 48.18 5.45 0.05 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 615.60 0.00 19248.80 0.00 615324.80 31.97 49.01 2.55 0.05 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 656.29 0.00 21067.86 0.00 673494.61 31.97 48.40 2.30 0.05 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 651.00 0.00 21388.40 0.00 683654.40 31.96 48.52 2.27 0.05 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 676.15 0.00 21180.16 0.00 677162.32 31.97 48.56 2.29 0.05 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 638.40 0.00 21072.60 0.00 673747.20 31.97 48.46 2.30 0.05 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 6479.60 0.00 3066.60 0.00 131396.80 42.85 86.42 19.62 0.30 92.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 298.80 0.00 56.80 0.00 4963.20 87.38 87.00 1606.92 17.61 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 2.80 0.00 7.60 0.00 376.00 49.47 32.10 3630.84 131.58 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 2743.49 0.00 265.73 0.00 21862.93 82.27 29.09 208.17 3.65 96.99
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 236.00 0.00 50.60 0.00 4420.80 87.37 85.77 1609.39 19.76 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 7.00 0.00 0.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 7.00 0.00 0.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 6.99 0.00 0.00 99.84
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 7.00 0.00 0.00 100.04
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 6.99 0.00 0.00 99.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1497.80 0.20 156.91 1.60 13317.84 84.78 75.93 716.88 6.29 98.76
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1978.64 0.00 184.23 0.00 14987.62 81.35 84.12 391.55 5.27 97.09
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 2081.40 0.00 216.60 0.00 18302.40 84.50 142.87 651.16 4.51 97.60
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1890.20 0.20 210.60 1.60 17993.60 85.37 108.64 554.66 4.66 98.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1490.00 0.00 155.80 0.00 13212.80 84.81 129.51 839.51 6.31 98.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1430.40 0.00 136.40 0.00 11590.40 84.97 103.09 687.65 7.19 98.08
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1106.40 0.00 110.80 0.00 9492.80 85.68 114.16 1151.88 8.89 98.48
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1937.60 0.00 226.20 0.00 19289.60 85.28 97.60 427.02 4.36 98.56
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 743.31 0.20 73.45 1.60 5202.40 70.66 88.49 986.88 13.42 98.84
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1491.00 0.00 157.40 0.00 13427.20 85.31 88.07 562.37 6.25 98.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1046.40 0.00 96.80 0.00 8260.80 85.34 102.64 990.85 10.15 98.24
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 810.82 0.00 88.78 0.00 7445.29 83.86 130.90 1160.75 11.18 99.24
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1122.95 0.00 112.97 0.00 9708.58 85.94 119.94 1295.33 8.76 99.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 744.40 0.20 87.80 1.60 7441.60 84.58 102.39 1090.86 11.27 99.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 744.51 0.20 96.61 1.60 8183.63 84.55 100.76 1341.66 10.24 99.16
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1497.99 0.00 164.26 0.00 13352.61 81.29 100.77 622.15 6.00 98.63
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 4397.41 0.20 429.74 1.60 36188.42 84.17 88.42 207.36 2.21 95.01
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 3777.64 0.00 426.75 0.00 36083.03 84.55 108.43 260.46 2.24 95.57
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 4892.80 0.20 483.20 1.60 40792.00 84.39 90.34 172.50 1.95 94.16
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 4049.50 0.20 425.45 1.60 36117.03 84.85 106.13 259.25 2.25 95.71
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 3583.60 0.00 373.00 0.00 31508.80 84.47 116.98 305.78 2.56 95.52
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 4599.60 0.20 483.03 1.60 40980.44 84.81 99.17 210.96 1.96 94.53
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 5242.57 0.20 568.88 1.61 48151.00 84.62 85.44 131.88 1.66 94.54
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 348.70 0.00 31.94 0.00 667.47 20.90 101.35 3325.60 30.98 98.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 6922.65 0.20 732.26 1.60 61361.12 83.78 130.94 174.46 1.28 93.55
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 6137.33 0.20 650.70 1.60 55326.15 85.00 105.11 171.18 1.42 92.69
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 8535.00 0.40 878.20 3.20 74152.00 84.40 88.46 97.92 1.02 90.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 6773.25 0.20 713.37 1.60 60146.91 84.29 125.94 185.50 1.30 92.77
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 4784.40 1.20 526.20 33.60 44696.00 84.81 48.61 94.11 1.79 94.56
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 2.00 6.80 0.60 217.60 20.80 32.22 1.05 165.30 135.14 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.00 13.40 0.40 428.80 11.20 31.88 1.00 59.07 72.46 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.40 9.40 44.20 40.20 1414.40 396.80 21.46 36.11 429.97 11.85 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.20 135.53 0.40 4336.93 12.77 32.00 1.00 7.34 7.33 99.64
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.60 151.90 0.40 4860.92 16.03 32.02 1.00 6.58 6.57 100.04
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.20 144.40 0.40 4614.40 12.80 31.96 1.00 6.88 6.87 99.52
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.60 1.20 155.40 0.40 4972.80 12.80 32.00 1.00 6.33 6.41 99.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.20 52.40 0.40 1676.80 12.80 32.00 1.02 19.48 18.94 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.00 36.47 0.40 1167.13 11.22 31.96 1.01 27.43 27.17 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.80 1.00 38.20 0.80 1222.40 14.40 31.71 1.01 25.85 25.64 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.80 1.20 137.72 0.60 4400.80 14.37 31.92 1.01 7.28 7.19 99.48
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 11.00 2.20 1152.60 0.60 36876.80 22.40 32.00 26.63 20.87 0.86 99.44
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.00 1.20 163.00 0.40 5222.40 12.80 32.04 63.36 389.43 6.12 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.40 1.20 155.60 0.40 4979.20 12.80 32.00 63.12 401.14 6.41 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 3.20 1.20 143.80 0.40 4601.60 12.80 32.00 62.66 436.97 6.93 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.80 1.60 134.40 0.40 4300.80 16.00 32.02 63.00 465.77 7.42 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.80 1.20 159.80 0.40 5113.60 12.80 32.00 63.37 398.41 6.24 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.80 155.00 1.40 4960.00 25.60 31.88 63.39 400.14 6.39 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.20 152.80 0.60 4883.20 14.40 31.93 62.52 408.45 6.52 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 3.20 1.20 153.80 0.40 4921.60 12.80 32.00 62.53 406.60 6.49 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.20 145.80 0.40 4665.60 12.80 32.00 62.59 429.42 6.84 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.80 1.20 138.72 0.40 4439.12 12.77 32.00 61.47 443.82 7.17 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 1.20 1.20 150.80 0.40 4825.60 12.80 32.00 62.25 408.74 6.61 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.40 2.21 103.41 2288.76 3309.24 72536.55 31.71 40.54 18.06 0.39 93.82
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.20 0.00 725.50 0.00 23056.57 31.78 0.98 1.38 1.36 98.33
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.20 0.00 3763.86 0.00 119460.24 31.74 0.91 0.24 0.24 91.08
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.00 0.00 63.87 32.00 1.00 454.40 500.00 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.20 0.00 2.61 0.00 83.37 32.00 1.00 409.23 384.62 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.40 0.00 0.80 0.00 11.18 14.00 5.19 1947.00 1250.00 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.20 0.00 2.00 0.00 52.80 26.40 1.59 2577.20 500.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.60 0.00 2.40 0.00 72.14 30.00 1.00 475.67 416.67 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.80 0.00 3.59 0.00 111.78 31.11 1.00 271.11 277.78 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.00 0.00 3.81 0.00 120.24 31.58 1.00 260.21 263.16 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.80 0.00 2.79 0.00 86.23 30.86 1.00 307.43 357.14 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.00 0.00 2.60 0.00 81.60 31.38 1.00 408.00 384.62 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.80 0.00 3.80 0.00 118.40 31.16 14.13 262.95 262.53 99.76
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 4.39 0.00 140.52 32.00 50.81 4022.00 227.27 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 4.81 0.00 153.91 32.00 34.36 9330.17 208.33 100.20
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.79 0.00 75.05 26.86 11.43 11746.57 357.14 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 2.01 0.00 3.82 0.00 123.69 32.42 51.05 5032.63 263.16 100.40
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.99 0.00 95.81 32.00 45.40 7306.40 333.33 99.80
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.60 0.00 83.20 32.00 31.07 12140.92 384.62 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 1.60 0.00 51.20 32.00 19.51 16378.50 625.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.20 0.00 70.40 32.00 10.86 21505.82 454.55 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 4.60 0.00 1.60 0.00 41.60 26.00 21.67 18743.00 625.00 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.20 0.00 70.40 32.00 49.72 4714.18 454.55 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.20 0.00 70.40 32.00 46.99 9282.91 454.55 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 5.40 0.00 195.20 36.15 46.16 14913.33 185.19 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 3.60 0.00 110.40 30.67 33.73 12728.67 277.78 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 2.80 0.00 89.60 32.00 24.99 11474.00 357.14 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.00 0.00 4.60 0.00 140.80 30.61 11.80 9089.22 217.39 100.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 8.58 0.00 0.60 0.00 14.37 24.00 2.13 546.67 706.67 42.32
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.40 1.40 3.21 80.16 46.22 1.75 2077.78 153.78 27.74
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 1.60 0.00 0.40 0.00 15.97 40.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

[root@ip-10-251-146-159 ~]#

Is EC2 useful as a database server

Plenty of people have been excited by the prospect of Amazon EC2 and the ability to scale out your databases as load increases from your original configuration. I noticed Morgan Tocker and Carl Mercier are going to be presenting on this topic at the upcoming MySQL Conference

However almost immediately people are worried about the lack of persistent of data across instance terminations.
In a sense people are wanting dedicated hosting services instead of what EC2 really is.

You need to think of Amazon EC2 in a electricity generation metaphor.
Coal, Nuclear Fission and Gas provide the base load electricity which is on 24×7.
Gas and Hydro can act as peak load generation as well. So at peak periods when the requirement for more electricity is higher, electricity generators can switch on these extra resources to cope.

Amazon EC2 is the same, it is there to service peak load.
Either you are using Amazon EC2 as a base load server or you are using a dedicated hosting service to provide base load. You add additional server resources during peak periods as required.
As a dedicated hosting service EC2 is not actually the cheapest option out there. There are plenty of dedicated hosting providers who will give you and your application and database, cheaper base load capacity. That said, many people choose to run both application and database servers on EC2 as base load servers and the uptime of these instances is good.

What this means is that using EC2 as base load means you must implement additional protections for your data to provide persistence. This may be in the form of clustering technologies and replication technologies or both. So running EC2 as a base load database server adds complexity. This is why numerous companies have sprung up as a result of this complexity. They are essentially providing a method for companies to pay for someone else to deal with it.

The hidden value here is, in adopting a more thorough attitude to data persistence and redundancy, your database is more robust. So if or when your dedicated hosting provider has an outage, your architectural design is already in a position to handle it.

The danger is, you see any ongoing performance issue (a demand for addition base load) as solved by throwing hardware at it. Rather than reviewing whether the demand is justified or whether it can be reduced through tuning the application, database or architecture.

Update: Added Carl as co-presenter at the MySQL conference.

Have Fun

Paul

Sysbench fileio vs Large EC2 Part 2

Overview:

I reran the sysbench fileio benchmarking tool on a large EC2 instance. This is basically the middle range offering from Amazon. The large instance runs a 64 bit OS of your choice and safely sits in the commodity PC/Server range (with a little extra memory).

If you missed the first article, I ran the same sysbench fileio benchmark on the small EC2 instance

Install:

  1. Follow the instructions to install sysbench from that article.
  2. Grab to shell script which will run the same tests.
  3. Run baby run.

Summary:

  1. The root partition mountpoint still is much much better than the /mnt
  2. The large instance outperforms the small instance by a large margin.
  3. There was no improvement in the /mnt performance between small and large EC2 instance.


Results:

Raw Results:
(with EC2 large instance CPU/Memory and disk specification)


[root@ip-10-251-159-159 ~]# uname -a
Linux ip-10-251-159-159 2.6.16.33-xenU #2 SMP Wed Aug 15 17:27:36 SAST 2007 x86_64 x86_64 x86_64 GNU/Linux
[root@ip-10-251-159-159 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 767M 8.6G 9% /
/dev/sdb 414G 199M 393G 1% /mnt
[root@ip-10-251-159-159 ~]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 270
stepping : 2
cpu MHz : 2004.540
cache size : 1024 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips : 4009.78
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

processor : 1
vendor_id : AuthenticAMD
cpu family : 15
model : 33
model name : Dual Core AMD Opteron(tm) Processor 270
stepping : 2
cpu MHz : 2004.540
cache size : 1024 KB
physical id : 1
siblings : 1
core id : 0
cpu cores : 1
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm cmp_legacy
bogomips : 4009.78
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

[root@ip-10-251-159-159 ~]# free
total used free shared buffers cached
Mem: 7864320 218748 7645572 0 3664 37612
-/+ buffers/cache: 177472 7686848
Swap: 0 0 0

[root@ip-10-251-159-159 ~]# cat /proc/meminfo
MemTotal: 7864320 kB
MemFree: 7645572 kB
Buffers: 3680 kB
Cached: 37596 kB
SwapCached: 0 kB
Active: 29264 kB
Inactive: 15376 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 7864320 kB
LowFree: 7645572 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 28 kB
Writeback: 0 kB
Mapped: 6584 kB
Slab: 9344 kB
CommitLimit: 3932160 kB
Committed_AS: 9304 kB
PageTables: 604 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 124 kB
VmallocChunk: 34359738243 kB

[root@ip-10-251-159-159 ~]# cat /etc/fstab
# Default /etc/fstab
# Supplied by: Amazon EC2 public image
/dev/sda1 / ext3 defaults 1 1
/dev/sdb /mnt ext3 defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0

Test on /dev/sda1

[root@ip-10-251-159-159 bench]# ./sysbench_fileio.sh
sysbench v0.4.8: multi-threaded system evaluation benchmark

1 files, 131072Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 785793 Read, 0 Write, 0 Other = 785793 Total
Read 11.99Gb Written 0b Total transferred 11.99Gb (204.63Mb/sec)
13096.48 Requests/sec executed

Test execution summary:
total time: 60.0003s
total number of events: 785793
total time taken by event execution: 57.9662
per-request statistics:
min: 0.0001s
avg: 0.0001s
max: 0.0600s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 785793.0000/0.00
execution time (avg/stddev): 57.9662/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Done.

Operations performed: 1000000 Read, 0 Write, 0 Other = 1000000 Total
Read 15.259Gb Written 0b Total transferred 15.259Gb (394.52Mb/sec)
25248.99 Requests/sec executed

Test execution summary:
total time: 39.6055s
total number of events: 1000000
total time taken by event execution: 2531.7466
per-request statistics:
min: 0.0001s
avg: 0.0025s
max: 0.0387s
approx. 95 percentile: 0.0027s

Threads fairness:
events (avg/stddev): 15625.0000/2.60
execution time (avg/stddev): 39.5585/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 623302 Write, 0 Other = 623302 Total
Read 0b Written 9.5108Gb Total transferred 9.5108Gb (148.35Mb/sec)
9494.70 Requests/sec executed

Test execution summary:
total time: 65.6474s
total number of events: 623302
total time taken by event execution: 64.1263
per-request statistics:
min: 0.0001s
avg: 0.0001s
max: 12.6994s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 623302.0000/0.00
execution time (avg/stddev): 64.1263/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 460201 Write, 0 Other = 460201 Total
Read 0b Written 7.0221Gb Total transferred 7.0221Gb (119.83Mb/sec)
7668.98 Requests/sec executed

Test execution summary:
total time: 60.0081s
total number of events: 460201
total time taken by event execution: 3838.8304
per-request statistics:
min: 0.0001s
avg: 0.0083s
max: 59.9959s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 7190.6406/10250.84
execution time (avg/stddev): 59.9817/0.03

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...
sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 1024Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 890007 Write, 0 Other = 890007 Total
Read 0b Written 13.58Gb Total transferred 13.58Gb (231.76Mb/sec)
14832.37 Requests/sec executed

Test execution summary:
total time: 60.0044s
total number of events: 890007
total time taken by event execution: 3837.5238
per-request statistics:
min: 0.0003s
avg: 0.0043s
max: 15.4482s
approx. 95 percentile: 0.0056s

Threads fairness:
events (avg/stddev): 13906.3594/53.51
execution time (avg/stddev): 59.9613/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 16384Kb each, 2048Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 8349 Read, 0 Write, 0 Other = 8349 Total
Read 130.45Mb Written 0b Total transferred 130.45Mb (2.1739Mb/sec)
139.13 Requests/sec executed

Test execution summary:
total time: 60.0086s
total number of events: 8349
total time taken by event execution: 59.9863
per-request statistics:
min: 0.0001s
avg: 0.0072s
max: 1.4351s
approx. 95 percentile: 0.0127s

Threads fairness:
events (avg/stddev): 8349.0000/0.00
execution time (avg/stddev): 59.9863/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 8953 Read, 0 Write, 0 Other = 8953 Total
Read 139.89Mb Written 0b Total transferred 139.89Mb (2.3145Mb/sec)
148.13 Requests/sec executed

Test execution summary:
total time: 60.4417s
total number of events: 8953
total time taken by event execution: 3859.6431
per-request statistics:
min: 0.0062s
avg: 0.4311s
max: 1.0842s
approx. 95 percentile: 0.6042s

Threads fairness:
events (avg/stddev): 139.8906/1.52
execution time (avg/stddev): 60.3069/0.17

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 25372 Write, 0 Other = 25372 Total
Read 0b Written 396.44Mb Total transferred 396.44Mb (6.3057Mb/sec)
403.57 Requests/sec executed

Test execution summary:
total time: 62.8695s
total number of events: 25372
total time taken by event execution: 62.8041
per-request statistics:
min: 0.0001s
avg: 0.0025s
max: 6.2624s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 25372.0000/0.00
execution time (avg/stddev): 62.8041/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 22442 Write, 0 Other = 22442 Total
Read 0b Written 350.66Mb Total transferred 350.66Mb (5.8438Mb/sec)
374.00 Requests/sec executed

Test execution summary:
total time: 60.0046s
total number of events: 22442
total time taken by event execution: 3839.7357
per-request statistics:
min: 0.0013s
avg: 0.1711s
max: 40.0358s
approx. 95 percentile: 0.0841s

Threads fairness:
events (avg/stddev): 350.6562/8.07
execution time (avg/stddev): 59.9959/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...


On /mnt

[root@ip-10-251-159-159 bench]# cd /mnt
[root@ip-10-251-159-159 mnt]# mkdir bench
[root@ip-10-251-159-159 mnt]# cd bench/
[root@ip-10-251-159-159 bench]# cp ~/bench/sysbench_fileio.sh .
[root@ip-10-251-159-159 bench]# ./sysbench_fileio.sh
sysbench v0.4.8: multi-threaded system evaluation benchmark

1 files, 131072Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 9686 Read, 0 Write, 0 Other = 9686 Total
Read 151.34Mb Written 0b Total transferred 151.34Mb (2.5224Mb/sec)
161.43 Requests/sec executed

Test execution summary:
total time: 60.0009s
total number of events: 9686
total time taken by event execution: 59.9755
per-request statistics:
min: 0.0003s
avg: 0.0062s
max: 0.1398s
approx. 95 percentile: 0.0100s

Threads fairness:
events (avg/stddev): 9686.0000/0.00
execution time (avg/stddev): 59.9755/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 10849 Read, 0 Write, 0 Other = 10849 Total
Read 169.52Mb Written 0b Total transferred 169.52Mb (2.8097Mb/sec)
179.82 Requests/sec executed

Test execution summary:
total time: 60.3318s
total number of events: 10849
total time taken by event execution: 3850.1830
per-request statistics:
min: 0.0077s
avg: 0.3549s
max: 0.7746s
approx. 95 percentile: 0.4958s

Threads fairness:
events (avg/stddev): 169.5156/3.16
execution time (avg/stddev): 60.1591/0.10

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 15936 Write, 0 Other = 15936 Total
Read 0b Written 249Mb Total transferred 249Mb (4.1233Mb/sec)
263.89 Requests/sec executed

Test execution summary:
total time: 60.3881s
total number of events: 15936
total time taken by event execution: 60.3480
per-request statistics:
min: 0.0001s
avg: 0.0038s
max: 0.8638s
approx. 95 percentile: 0.0002s

Threads fairness:
events (avg/stddev): 15936.0000/0.00
execution time (avg/stddev): 60.3480/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 16206 Write, 0 Other = 16206 Total
Read 0b Written 253.22Mb Total transferred 253.22Mb (4.1737Mb/sec)
267.12 Requests/sec executed

Test execution summary:
total time: 60.6696s
total number of events: 16206
total time taken by event execution: 3881.9322
per-request statistics:
min: 0.0001s
avg: 0.2395s
max: 60.6637s
approx. 95 percentile: 0.0002s

Threads fairness:
events (avg/stddev): 253.2188/1423.67
execution time (avg/stddev): 60.6552/0.02

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...
sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 1024Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 14826 Write, 0 Other = 14826 Total
Read 0b Written 231.66Mb Total transferred 231.66Mb (3.828Mb/sec)
244.99 Requests/sec executed

Test execution summary:
total time: 60.5170s
total number of events: 14826
total time taken by event execution: 3855.8588
per-request statistics:
min: 0.0004s
avg: 0.2601s
max: 1.7562s
approx. 95 percentile: 0.7735s

Threads fairness:
events (avg/stddev): 231.6562/6.36
execution time (avg/stddev): 60.2478/0.08

sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 16384Kb each, 2048Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 8373 Read, 0 Write, 0 Other = 8373 Total
Read 130.83Mb Written 0b Total transferred 130.83Mb (2.1804Mb/sec)
139.54 Requests/sec executed

Test execution summary:
total time: 60.0027s
total number of events: 8373
total time taken by event execution: 59.9801
per-request statistics:
min: 0.0003s
avg: 0.0072s
max: 0.4734s
approx. 95 percentile: 0.0107s

Threads fairness:
events (avg/stddev): 8373.0000/0.00
execution time (avg/stddev): 59.9801/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 10034 Read, 0 Write, 0 Other = 10034 Total
Read 156.78Mb Written 0b Total transferred 156.78Mb (2.5952Mb/sec)
166.09 Requests/sec executed

Test execution summary:
total time: 60.4118s
total number of events: 10034
total time taken by event execution: 3855.7836
per-request statistics:
min: 0.0066s
avg: 0.3843s
max: 1.0552s
approx. 95 percentile: 0.5357s

Threads fairness:
events (avg/stddev): 156.7812/2.78
execution time (avg/stddev): 60.2466/0.10

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 12932 Write, 0 Other = 12932 Total
Read 0b Written 202.06Mb Total transferred 202.06Mb (3.3463Mb/sec)
214.16 Requests/sec executed

Test execution summary:
total time: 60.3847s
total number of events: 12932
total time taken by event execution: 60.3509
per-request statistics:
min: 0.0002s
avg: 0.0047s
max: 3.6623s
approx. 95 percentile: 0.0002s

Threads fairness:
events (avg/stddev): 12932.0000/0.00
execution time (avg/stddev): 60.3509/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 13200 Write, 0 Other = 13200 Total
Read 0b Written 206.25Mb Total transferred 206.25Mb (3.4138Mb/sec)
218.48 Requests/sec executed

Test execution summary:
total time: 60.4171s
total number of events: 13200
total time taken by event execution: 3864.5972
per-request statistics:
min: 0.0003s
avg: 0.2928s
max: 3.7448s
approx. 95 percentile: 2.3343s

Threads fairness:
events (avg/stddev): 206.2500/5.68
execution time (avg/stddev): 60.3843/0.01

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...

Seeker io benchmark on small EC2

Frederic posted an interesting link to the seeker io script in the sysbench fileio vs small EC2 article.

He was concerned that the seeker was showing difference numbers to sysbench fileio, especially on the /mnt mountpoint.

I read the thread and there was another comment made concerning how Xen could use a sparse file image to improve the io performance. I tested that theory out and found as soon as I created a reasonable sized file on /mnt, the performance dropped significantly to less than 20% of the empty mountpoint performance.

I appreciate the feedback I receive via comments. Most of the feedback I have derive purely from Google web analytics and sitemap tools.

Results:



[root@domU-12-31-35-00-04-61 ~]# sudo hdparm -t /dev/sda1

/dev/sda1:
Timing buffered disk reads: 216 MB in 3.01 seconds = 71.68 MB/sec
[root@domU-12-31-35-00-04-61 ~]# sudo hdparm -t /dev/sda2

/dev/sda2:
Timing buffered disk reads: 380 MB in 3.00 seconds = 126.51 MB/sec


[root@domU-12-31-35-00-04-61 ~]# ./seeker /dev/sda1
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda1 [1025MB], wait 30 seconds..............................
Results: 278 seeks/second, 3.60 ms random access time

./seeker /dev/sda2

Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda2 [152688MB], wait 30 seconds..............................
Results: 5741 seeks/second, 0.17 ms random access time

Add a file to /mnt to test the sparse Xen image idea

dd if=/dev/zero of=/mnt/zerofile bs=1024M

[root@domU-12-31-35-00-04-61 mnt]# ls -l
total 7347232
drwx------ 2 root root 16384 Mar 16 12:38 lost+found
-rw-r--r-- 1 root root 710 Mar 16 23:37 randomfile
-rw-r--r-- 1 root root 7516192768 Mar 16 23:44 zerofile
[root@domU-12-31-35-00-04-61 mnt]# cd
[root@domU-12-31-35-00-04-61 ~]# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 1032056 885984 93592 91% /
/dev/sda2 153899044 7539888 138541532 6% /mnt
[root@domU-12-31-35-00-04-61 ~]# ./seeker /dev/sda2
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda2 [152688MB], wait 30 seconds.............................
Results: 1101 seeks/second, 0.91 ms random access time
[root@domU-12-31-35-00-04-61 ~]# ./seeker /dev/sda2
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda2 [152688MB], wait 30 seconds......................
Results: 582 seeks/second, 1.72 ms random access time
[root@domU-12-31-35-00-04-61 ~]# ./seeker /dev/sda2
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda2 [152688MB], wait 30 seconds.....................
Results: 177 seeks/second, 5.62 ms random access time
[root@domU-12-31-35-00-04-61 ~]# ./seeker /dev/sda2
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda2 [152688MB], wait 30 seconds.........................
Results: 881 seeks/second, 1.13 ms random access time
[root@domU-12-31-35-00-04-61 ~]# w
23:50:30 up 4:07, 1 user, load average: 0.62, 3.77, 3.39
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root ttyp0 203-19-70-133.pe 19:47 0.00s 0.01s 0.00s w
[root@domU-12-31-35-00-04-61 ~]# ./seeker /dev/sda2
Seeker v2.0, 2007-01-15, http://www.linuxinsight.com/how_fast_is_your_disk.html
Benchmarking /dev/sda2 [152688MB], wait 30 seconds....................
Results: 355 seeks/second, 2.82 ms random access time

Sysbench fileio vs EC2 Part 1

Overview:

Peter Zaitsev’s recent article about Evaluating IO subsystem performance for MySQL spurred my interest in doing something similar on EC2.

I have covered running sysbench against MySQL on EC2 however not specifically used sysbench to test IO. Rather I had used bonnie++ and iozone to do that.

I don’t have a lot of respect for the EC2 small instance. Whilst it was reasonable in the middle of 2006 when Amazon EC2 was launched, you can (even in Australia) pick up Dual and Quad Core CPUs with enough memory for 32 bit OS for a reasonable amount nowadays.
The main problem as I see it for my specific interest is the lack of grunt in network and IO bandwidth. Having the idea of Grid or Utility or the latest “Cloud” computing falls apart when you have to hit disk or worse need to synchronize across a slow network.

So I was mildly surprised when the small instance was able to match and out-perform Peter’s benchmarks on the local disk at least.

Summary:

Using Peter’s benchmark description as a key, I reran his tests on a small EC2 instance. The local disk matched and outperformed his benchmark with no changes to any settings.
Unfortunately the mounted filesystem /mnt was much much worse.

If you are using MySQL or any IO intensive application, I would do the following:

  1. Make your image specify as large a local mount as possible. You will need the raw IO speed for anything which requires fast reads and fast writes.
  2. If you can take the risk use the Linux file cache. Especially if you have to use /mnt.
  3. Treat the raw IO performance of /mnt as slow disk, even in some respects like a backup location. It has a requests/sec and io transfer rates little better than USB 1.0 High speed. Certainly not even in the realm of Firewire.
  4. Get as much into memory as possible. This means narrowing table and indexes sizes or going with the more expensive large and extra large instances.

Install:

  1. wget sysbench from sourceforge.
  2. yum install gcc (or apt-get gcc) if not already installed
  3. configure, make and install sysbench
  4. Run this simple shell script sysbench_fileio.sh to replicate Peter’s benchmark.
  5. Free feel to publish your results, either here as a comment or on your own blog. Link or drop me a note and I will link to your benchmark results and we can tally up and see for whom the IO tolls.

Documentation (explains the various options)

http://sysbench.sourceforge.net/docs/#fileio_mode

Configuration:

Amazon small EC2 instance, running CentOS 4.4



[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 855M 8.6G 9% /
/dev/sda2 147G 188M 140G 1% /mnt
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/fstab
# This file is edited by fstab-sync - see 'man fstab-sync' for details
/dev/sda1 / ext3 defaults 1 1
/dev/sda2 /mnt ext3 defaults 1 2
/dev/sda3 swap swap defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# uname -a
Linux domU-12-31-38-00-20-02 2.6.16-xenU #1 SMP Mon May 28 03:41:49 SAST 2007 i686 athlon i386 GNU/Linux
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/redhat-release
CentOS release 4.4 (Final)
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 65
model name : Dual-Core AMD Opteron(tm) Processor 2218 HE
stepping : 3
cpu MHz : 2600.000
cache size : 1024 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
bogomips : 5201.75

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# ipcs

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status

------ Semaphore Arrays --------
key semid owner perms nsems

------ Message Queues --------
key msqid owner perms used-bytes messages

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# free
total used free shared buffers cached
Mem: 1740944 533224 1207720 0 217020 263060
-/+ buffers/cache: 53144 1687800
Swap: 917496 0 917496

Results:

Raw Results:



wget http://optusnet.dl.sourceforge.net/sourceforge/sysbench/sysbench-0.4.8.tar.gz

yum install gcc sysstat

tar -xzvf sysbench-0.4.8.tar.gz
cd sysbench-0.4.8
./configure --without-mysql
make
make install

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 855M 8.6G 9% /
/dev/sda2 147G 188M 140G 1% /mnt
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/fstab
# This file is edited by fstab-sync - see 'man fstab-sync' for details
/dev/sda1 / ext3 defaults 1 1
/dev/sda2 /mnt ext3 defaults 1 2
/dev/sda3 swap swap defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# uname -a
Linux domU-12-31-38-00-20-02 2.6.16-xenU #1 SMP Mon May 28 03:41:49 SAST 2007 i686 athlon i386 GNU/Linux
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/redhat-release
CentOS release 4.4 (Final)
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 65
model name : Dual-Core AMD Opteron(tm) Processor 2218 HE
stepping : 3
cpu MHz : 2600.000
cache size : 1024 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
bogomips : 5201.75

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# ipcs

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status

------ Semaphore Arrays --------
key semid owner perms nsems

------ Message Queues --------
key msqid owner perms used-bytes messages

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# free
total used free shared buffers cached
Mem: 1740944 533224 1207720 0 217020 263060
-/+ buffers/cache: 53144 1687800
Swap: 917496 0 917496

http://sysbench.sourceforge.net/docs/#fileio_mode

Sysbench FileIO test options

[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# sysbench --test=fileio help
sysbench v0.4.8: multi-threaded system evaluation benchmark

fileio options:
--file-num=N number of files to create [128]
--file-block-size=N block size to use in all IO operations [16384]
--file-total-size=SIZE total size of files to create [2G]
--file-test-mode=STRING test mode {seqwr, seqrewr, seqrd, rndrd, rndwr, rndrw}
--file-io-mode=STRING file operations mode {sync,async,fastmmap,slowmmap} [sync]
--file-extra-flags=STRING additional flags to use on opening files {sync,dsync,direct} []
--file-fsync-freq=N do fsync() after this number of requests (0 - don't use fsync()) [100]
--file-fsync-all=[on|off] do fsync() after each write operation [off]
--file-fsync-end=[on|off] do fsync() at the end of test [on]
--file-fsync-mode=STRING which method to use for synchronization {fsync, fdatasync} [fsync]
--file-merged-requests=N merge at most this number of IO requests if possible (0 - don't merge) [0]
--file-rw-ratio=N reads/writes ratio for combined test [1.5]


Peter Zaitsev's 120M prepare file command

sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=1 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M prepare

sysbench v0.4.8: multi-threaded system evaluation benchmark

1 files, 131072Kb each, 128Mb total
Creating files for the test...

Running the test with the 120M file

sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M --file-test-mode=rndrd run

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 590164 Read, 0 Write, 0 Other = 590164 Total
Read 9.0052Gb Written 0b Total transferred 9.0052Gb (153.69Mb/sec)
9836.03 Requests/sec executed

Test execution summary:
total time: 60.0002s
total number of events: 590164
total time taken by event execution: 59.3742
per-request statistics:
min: 0.0001s
avg: 0.0001s
max: 0.0980s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 590164.0000/0.00
execution time (avg/stddev): 59.3742/0.00

Rerunning with 64 threads

sysbench --num-threads=64 --test=fileio --max-time=60 \
--max-requests=1000000 --file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M --file-test-mode=rndrd run
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 745086 Read, 0 Write, 0 Other = 745086 Total
Read 11.369Gb Written 0b Total transferred 11.369Gb (194.02Mb/sec)
12417.46 Requests/sec executed

Test execution summary:
total time: 60.0031s
total number of events: 745086
total time taken by event execution: 3837.4047
per-request statistics:
min: 0.0003s
avg: 0.0052s
max: 0.1228s
approx. 95 percentile: 0.0325s

Threads fairness:
events (avg/stddev): 11641.9688/8.86
execution time (avg/stddev): 59.9594/0.02

Running in single thread with random write i.e. --file-test-mode=rndwr


sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=1 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M --num-threads=1 --file-test-mode=rndwr run

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 93104 Write, 0 Other = 93104 Total
Read 0b Written 1.4207Gb Total transferred 1.4207Gb (24.227Mb/sec)
1550.55 Requests/sec executed

Test execution summary:
total time: 60.0457s
total number of events: 93104
total time taken by event execution: 59.9639
per-request statistics:
min: 0.0000s
avg: 0.0006s
max: 1.8278s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 93104.0000/0.00
execution time (avg/stddev): 59.9639/0.00

with 64 threads --num-threads=64

sysbench --test=fileio --num-threads=64 --max-time=60 \
--max-requests=1000000 --file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M \
--file-test-mode=rndwr run

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 109175 Write, 0 Other = 109175 Total
Read 0b Written 1.6659Gb Total transferred 1.6659Gb (24.115Mb/sec)
1543.37 Requests/sec executed

Test execution summary:
total time: 70.7379s
total number of events: 109175
total time taken by event execution: 4329.6293
per-request statistics:
min: 0.0000s
avg: 0.0397s
max: 70.7328s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 1705.8594/6223.80
execution time (avg/stddev): 67.6505/3.06

Cleaning up for test with multiple files

sysbench --test=fileio --num-threads=64 --max-time=60 \
--max-requests=1000000 --file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M --file-test-mode=rndwr cleanup
sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...

Preparing files

sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M --num-threads=64 --file-test-mode=rndwr prepare

sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 1024Kb each, 128Mb total
Creating files for the test...

rerunning with 128 files to total of 128M

sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M --num-threads=64 --file-test-mode=rndwr run

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 126633 Write, 0 Other = 126633 Total
Read 0b Written 1.9323Gb Total transferred 1.9323Gb (25.607Mb/sec)
1638.85 Requests/sec executed

Test execution summary:
total time: 77.2695s
total number of events: 126633
total time taken by event execution: 4561.7451
per-request statistics:
min: 0.0002s
avg: 0.0360s
max: 28.6922s
approx. 95 percentile: 0.0069s

Threads fairness:
events (avg/stddev): 1978.6406/28.82
execution time (avg/stddev): 71.2773/5.32

Preparing the 128 files to a total of 2Gig test

sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=2G --num-threads=1 --file-test-mode=rndwr prepare

The run

sysbench --init-rng=1 --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=2G --num-threads=1 --file-test-mode=rndrd run

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 14094 Read, 0 Write, 0 Other = 14094 Total
Read 220.22Mb Written 0b Total transferred 220.22Mb (3.6701Mb/sec)
234.89 Requests/sec executed

Test execution summary:
total time: 60.0034s
total number of events: 14094
total time taken by event execution: 59.9868
per-request statistics:
min: 0.0001s
avg: 0.0043s
max: 0.2521s
approx. 95 percentile: 0.0129s

Threads fairness:
events (avg/stddev): 14094.0000/0.00
execution time (avg/stddev): 59.9868/0.00

2Gig file test with 64 threads

sysbench --init-rng=1 --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=2G --num-threads=64 --file-test-mode=rndrd run

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 24639 Read, 0 Write, 0 Other = 24639 Total
Read 384.98Mb Written 0b Total transferred 384.98Mb (6.3875Mb/sec)
408.80 Requests/sec executed

Test execution summary:
total time: 60.2719s
total number of events: 24639
total time taken by event execution: 3847.3337
per-request statistics:
min: 0.0023s
avg: 0.1561s
max: 0.5804s
approx. 95 percentile: 0.3490s

Threads fairness:
events (avg/stddev): 384.9844/0.94
execution time (avg/stddev): 60.1146/0.08

2 Gig file test single thread random writes --file-test-mode=rndwr

sysbench --init-rng=1 --test=fileio --max-time=60 \
--max-requests=1000000 --file-num=128 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=2G --num-threads=1 \
--file-test-mode=rndwr run

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 52960 Write, 0 Other = 52960 Total
Read 0b Written 827.5Mb Total transferred 827.5Mb (13.748Mb/sec)
879.89 Requests/sec executed

Test execution summary:
total time: 60.1893s
total number of events: 52960
total time taken by event execution: 60.1396
per-request statistics:
min: 0.0001s
avg: 0.0011s
max: 2.1815s
approx. 95 percentile: 0.0001s

Threads fairness:
events (avg/stddev): 52960.0000/0.00
execution time (avg/stddev): 60.1396/0.00


2 Gig file test 64 threads random writes --file-test-mode=rndwr

sysbench --init-rng=1 --test=fileio --max-time=60 \
--max-requests=1000000 --file-num=128 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=2G --num-threads=64 \
--file-test-mode=rndwr run

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 38767 Write, 0 Other = 38767 Total
Read 0b Written 605.73Mb Total transferred 605.73Mb (10.038Mb/sec)
642.45 Requests/sec executed

Test execution summary:
total time: 60.3424s
total number of events: 38767
total time taken by event execution: 3861.3838
per-request statistics:
min: 0.0001s
avg: 0.0996s
max: 4.8871s
approx. 95 percentile: 0.8721s

Threads fairness:
events (avg/stddev): 605.7344/14.70
execution time (avg/stddev): 60.3341/0.00

Rerunning the whole test set on /mnt

cd /mnt
vi sysbench_fileio.sh
chmod 700 sysbench_fileio.sh
./sysbench_fileio.sh

sysbench v0.4.8: multi-threaded system evaluation benchmark

1 files, 131072Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!


Time limit exceeded, exiting...
Done.

Operations performed: 9782 Read, 0 Write, 0 Other = 9782 Total
Read 152.84Mb Written 0b Total transferred 152.84Mb (2.547Mb/sec)
163.01 Requests/sec executed

Test execution summary:
total time: 60.0092s
total number of events: 9782
total time taken by event execution: 59.9964
per-request statistics:
min: 0.0003s
avg: 0.0061s
max: 0.1835s
approx. 95 percentile: 0.0100s

Threads fairness:
events (avg/stddev): 9782.0000/0.00
execution time (avg/stddev): 59.9964/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!

Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 10799 Read, 0 Write, 0 Other = 10799 Total
Read 168.73Mb Written 0b Total transferred 168.73Mb (2.7943Mb/sec)
178.83 Requests/sec executed

Test execution summary:
total time: 60.3856s
total number of events: 10799
total time taken by event execution: 3852.0646
per-request statistics:
min: 0.0098s
avg: 0.3567s
max: 0.9908s
approx. 95 percentile: 0.6044s

Threads fairness:
events (avg/stddev): 168.7344/4.29
execution time (avg/stddev): 60.1885/0.11

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 15274 Write, 0 Other = 15274 Total
Read 0b Written 238.66Mb Total transferred 238.66Mb (3.9774Mb/sec)
254.55 Requests/sec executed

Test execution summary:
total time: 60.0029s
total number of events: 15274
total time taken by event execution: 59.9853
per-request statistics:
min: 0.0003s
avg: 0.0039s
max: 0.3117s
approx. 95 percentile: 0.0106s

Threads fairness:
events (avg/stddev): 15274.0000/0.00
execution time (avg/stddev): 59.9853/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 13360 Write, 0 Other = 13360 Total
Read 0b Written 208.75Mb Total transferred 208.75Mb (3.4615Mb/sec)
221.54 Requests/sec executed

Test execution summary:
total time: 60.3062s
total number of events: 13360
total time taken by event execution: 3847.7447
per-request statistics:
min: 0.0003s
avg: 0.2880s
max: 60.2694s
approx. 95 percentile: 0.0175s

Threads fairness:
events (avg/stddev): 208.7500/1157.46
execution time (avg/stddev): 60.1210/0.07

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...
sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 1024Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 14728 Write, 0 Other = 14728 Total
Read 0b Written 230.12Mb Total transferred 230.12Mb (3.8192Mb/sec)
244.43 Requests/sec executed

Test execution summary:
total time: 60.2541s
total number of events: 14728
total time taken by event execution: 3846.4353
per-request statistics:
min: 0.0004s
avg: 0.2612s
max: 1.5574s
approx. 95 percentile: 0.5732s

Threads fairness:
events (avg/stddev): 230.1250/10.67
execution time (avg/stddev): 60.1006/0.03

sysbench v0.4.8: multi-threaded system evaluation benchmark

128 files, 16384Kb each, 2048Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 8496 Read, 0 Write, 0 Other = 8496 Total
Read 132.75Mb Written 0b Total transferred 132.75Mb (2.2124Mb/sec)
141.60 Requests/sec executed

Test execution summary:
total time: 60.0016s
total number of events: 8496
total time taken by event execution: 59.9901
per-request statistics:
min: 0.0003s
avg: 0.0071s
max: 0.2733s
approx. 95 percentile: 0.0107s

Threads fairness:
events (avg/stddev): 8496.0000/0.00
execution time (avg/stddev): 59.9901/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 9497 Read, 0 Write, 0 Other = 9497 Total
Read 148.39Mb Written 0b Total transferred 148.39Mb (2.4566Mb/sec)
157.22 Requests/sec executed

Test execution summary:
total time: 60.4044s
total number of events: 9497
total time taken by event execution: 3854.1290
per-request statistics:
min: 0.0094s
avg: 0.4058s
max: 1.1867s
approx. 95 percentile: 0.7207s

Threads fairness:
events (avg/stddev): 148.3906/3.90
execution time (avg/stddev): 60.2208/0.12

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed: 0 Read, 12236 Write, 0 Other = 12236 Total
Read 0b Written 191.19Mb Total transferred 191.19Mb (3.1862Mb/sec)
203.91 Requests/sec executed

Test execution summary:
total time: 60.0056s
total number of events: 12236
total time taken by event execution: 59.9917
per-request statistics:
min: 0.0003s
avg: 0.0049s
max: 0.3623s
approx. 95 percentile: 0.0117s

Threads fairness:
events (avg/stddev): 12236.0000/0.00
execution time (avg/stddev): 59.9917/0.00

sysbench v0.4.8: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 64

Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.

Operations performed: 0 Read, 12251 Write, 0 Other = 12251 Total
Read 0b Written 191.42Mb Total transferred 191.42Mb (3.1792Mb/sec)
203.47 Requests/sec executed

Test execution summary:
total time: 60.2109s
total number of events: 12251
total time taken by event execution: 3847.4821
per-request statistics:
min: 0.0039s
avg: 0.3141s
max: 1.8187s
approx. 95 percentile: 0.6672s

Threads fairness:
events (avg/stddev): 191.4219/8.33
execution time (avg/stddev): 60.1169/0.04

sysbench v0.4.8: multi-threaded system evaluation benchmark

Removing test files...