Overview:
Peter Zaitsev’s recent article about Evaluating IO subsystem performance for MySQL spurred my interest in doing something similar on EC2.
I have covered running sysbench against MySQL on EC2 however not specifically used sysbench to test IO. Rather I had used bonnie++ and iozone to do that.
I don’t have a lot of respect for the EC2 small instance. Whilst it was reasonable in the middle of 2006 when Amazon EC2 was launched, you can (even in Australia) pick up Dual and Quad Core CPUs with enough memory for 32 bit OS for a reasonable amount nowadays.
The main problem as I see it for my specific interest is the lack of grunt in network and IO bandwidth. Having the idea of Grid or Utility or the latest “Cloud” computing falls apart when you have to hit disk or worse need to synchronize across a slow network.
So I was mildly surprised when the small instance was able to match and out-perform Peter’s benchmarks on the local disk at least.
Summary:
Using Peter’s benchmark description as a key, I reran his tests on a small EC2 instance. The local disk matched and outperformed his benchmark with no changes to any settings.
Unfortunately the mounted filesystem /mnt was much much worse.
If you are using MySQL or any IO intensive application, I would do the following:
- Make your image specify as large a local mount as possible. You will need the raw IO speed for anything which requires fast reads and fast writes.
- If you can take the risk use the Linux file cache. Especially if you have to use /mnt.
- Treat the raw IO performance of /mnt as slow disk, even in some respects like a backup location. It has a requests/sec and io transfer rates little better than USB 1.0 High speed. Certainly not even in the realm of Firewire.
- Get as much into memory as possible. This means narrowing table and indexes sizes or going with the more expensive large and extra large instances.
Install:
- wget sysbench from sourceforge.
- yum install gcc (or apt-get gcc) if not already installed
- configure, make and install sysbench
- Run this simple shell script sysbench_fileio.sh to replicate Peter’s benchmark.
- Free feel to publish your results, either here as a comment or on your own blog. Link or drop me a note and I will link to your benchmark results and we can tally up and see for whom the IO tolls.
Documentation (explains the various options)
http://sysbench.sourceforge.net/docs/#fileio_mode
Configuration:
Amazon small EC2 instance, running CentOS 4.4
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 855M 8.6G 9% /
/dev/sda2 147G 188M 140G 1% /mnt
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/fstab
# This file is edited by fstab-sync - see 'man fstab-sync' for details
/dev/sda1 / ext3 defaults 1 1
/dev/sda2 /mnt ext3 defaults 1 2
/dev/sda3 swap swap defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# uname -a
Linux domU-12-31-38-00-20-02 2.6.16-xenU #1 SMP Mon May 28 03:41:49 SAST 2007 i686 athlon i386 GNU/Linux
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/redhat-release
CentOS release 4.4 (Final)
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 65
model name : Dual-Core AMD Opteron(tm) Processor 2218 HE
stepping : 3
cpu MHz : 2600.000
cache size : 1024 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
bogomips : 5201.75
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
------ Semaphore Arrays --------
key semid owner perms nsems
------ Message Queues --------
key msqid owner perms used-bytes messages
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# free
total used free shared buffers cached
Mem: 1740944 533224 1207720 0 217020 263060
-/+ buffers/cache: 53144 1687800
Swap: 917496 0 917496
Results:
Raw Results:
wget http://optusnet.dl.sourceforge.net/sourceforge/sysbench/sysbench-0.4.8.tar.gz
yum install gcc sysstat
tar -xzvf sysbench-0.4.8.tar.gz
cd sysbench-0.4.8
./configure --without-mysql
make
make install
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 9.9G 855M 8.6G 9% /
/dev/sda2 147G 188M 140G 1% /mnt
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/fstab
# This file is edited by fstab-sync - see 'man fstab-sync' for details
/dev/sda1 / ext3 defaults 1 1
/dev/sda2 /mnt ext3 defaults 1 2
/dev/sda3 swap swap defaults 0 0
none /proc proc defaults 0 0
none /sys sysfs defaults 0 0
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# uname -a
Linux domU-12-31-38-00-20-02 2.6.16-xenU #1 SMP Mon May 28 03:41:49 SAST 2007 i686 athlon i386 GNU/Linux
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /etc/redhat-release
CentOS release 4.4 (Final)
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 65
model name : Dual-Core AMD Opteron(tm) Processor 2218 HE
stepping : 3
cpu MHz : 2600.000
cache size : 1024 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8legacy ts fid vid ttp tm stc
bogomips : 5201.75
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# ipcs
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
------ Semaphore Arrays --------
key semid owner perms nsems
------ Message Queues --------
key msqid owner perms used-bytes messages
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# free
total used free shared buffers cached
Mem: 1740944 533224 1207720 0 217020 263060
-/+ buffers/cache: 53144 1687800
Swap: 917496 0 917496
http://sysbench.sourceforge.net/docs/#fileio_mode
Sysbench FileIO test options
[root@domU-12-31-38-00-20-02 sysbench-0.4.8]# sysbench --test=fileio help
sysbench v0.4.8: multi-threaded system evaluation benchmark
fileio options:
--file-num=N number of files to create [128]
--file-block-size=N block size to use in all IO operations [16384]
--file-total-size=SIZE total size of files to create [2G]
--file-test-mode=STRING test mode {seqwr, seqrewr, seqrd, rndrd, rndwr, rndrw}
--file-io-mode=STRING file operations mode {sync,async,fastmmap,slowmmap} [sync]
--file-extra-flags=STRING additional flags to use on opening files {sync,dsync,direct} []
--file-fsync-freq=N do fsync() after this number of requests (0 - don't use fsync()) [100]
--file-fsync-all=[on|off] do fsync() after each write operation [off]
--file-fsync-end=[on|off] do fsync() at the end of test [on]
--file-fsync-mode=STRING which method to use for synchronization {fsync, fdatasync} [fsync]
--file-merged-requests=N merge at most this number of IO requests if possible (0 - don't merge) [0]
--file-rw-ratio=N reads/writes ratio for combined test [1.5]
Peter Zaitsev's 120M prepare file command
sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=1 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M prepare
sysbench v0.4.8: multi-threaded system evaluation benchmark
1 files, 131072Kb each, 128Mb total
Creating files for the test...
Running the test with the 120M file
sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M --file-test-mode=rndrd run
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 590164 Read, 0 Write, 0 Other = 590164 Total
Read 9.0052Gb Written 0b Total transferred 9.0052Gb (153.69Mb/sec)
9836.03 Requests/sec executed
Test execution summary:
total time: 60.0002s
total number of events: 590164
total time taken by event execution: 59.3742
per-request statistics:
min: 0.0001s
avg: 0.0001s
max: 0.0980s
approx. 95 percentile: 0.0001s
Threads fairness:
events (avg/stddev): 590164.0000/0.00
execution time (avg/stddev): 59.3742/0.00
Rerunning with 64 threads
sysbench --num-threads=64 --test=fileio --max-time=60 \
--max-requests=1000000 --file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M --file-test-mode=rndrd run
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 745086 Read, 0 Write, 0 Other = 745086 Total
Read 11.369Gb Written 0b Total transferred 11.369Gb (194.02Mb/sec)
12417.46 Requests/sec executed
Test execution summary:
total time: 60.0031s
total number of events: 745086
total time taken by event execution: 3837.4047
per-request statistics:
min: 0.0003s
avg: 0.0052s
max: 0.1228s
approx. 95 percentile: 0.0325s
Threads fairness:
events (avg/stddev): 11641.9688/8.86
execution time (avg/stddev): 59.9594/0.02
Running in single thread with random write i.e. --file-test-mode=rndwr
sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=1 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M --num-threads=1 --file-test-mode=rndwr run
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 0 Read, 93104 Write, 0 Other = 93104 Total
Read 0b Written 1.4207Gb Total transferred 1.4207Gb (24.227Mb/sec)
1550.55 Requests/sec executed
Test execution summary:
total time: 60.0457s
total number of events: 93104
total time taken by event execution: 59.9639
per-request statistics:
min: 0.0000s
avg: 0.0006s
max: 1.8278s
approx. 95 percentile: 0.0001s
Threads fairness:
events (avg/stddev): 93104.0000/0.00
execution time (avg/stddev): 59.9639/0.00
with 64 threads --num-threads=64
sysbench --test=fileio --num-threads=64 --max-time=60 \
--max-requests=1000000 --file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M \
--file-test-mode=rndwr run
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 0 Read, 109175 Write, 0 Other = 109175 Total
Read 0b Written 1.6659Gb Total transferred 1.6659Gb (24.115Mb/sec)
1543.37 Requests/sec executed
Test execution summary:
total time: 70.7379s
total number of events: 109175
total time taken by event execution: 4329.6293
per-request statistics:
min: 0.0000s
avg: 0.0397s
max: 70.7328s
approx. 95 percentile: 0.0001s
Threads fairness:
events (avg/stddev): 1705.8594/6223.80
execution time (avg/stddev): 67.6505/3.06
Cleaning up for test with multiple files
sysbench --test=fileio --num-threads=64 --max-time=60 \
--max-requests=1000000 --file-num=1 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=128M --file-test-mode=rndwr cleanup
sysbench v0.4.8: multi-threaded system evaluation benchmark
Removing test files...
Preparing files
sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M --num-threads=64 --file-test-mode=rndwr prepare
sysbench v0.4.8: multi-threaded system evaluation benchmark
128 files, 1024Kb each, 128Mb total
Creating files for the test...
rerunning with 128 files to total of 128M
sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=128M --num-threads=64 --file-test-mode=rndwr run
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 0 Read, 126633 Write, 0 Other = 126633 Total
Read 0b Written 1.9323Gb Total transferred 1.9323Gb (25.607Mb/sec)
1638.85 Requests/sec executed
Test execution summary:
total time: 77.2695s
total number of events: 126633
total time taken by event execution: 4561.7451
per-request statistics:
min: 0.0002s
avg: 0.0360s
max: 28.6922s
approx. 95 percentile: 0.0069s
Threads fairness:
events (avg/stddev): 1978.6406/28.82
execution time (avg/stddev): 71.2773/5.32
Preparing the 128 files to a total of 2Gig test
sysbench --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=2G --num-threads=1 --file-test-mode=rndwr prepare
The run
sysbench --init-rng=1 --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=2G --num-threads=1 --file-test-mode=rndrd run
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 14094 Read, 0 Write, 0 Other = 14094 Total
Read 220.22Mb Written 0b Total transferred 220.22Mb (3.6701Mb/sec)
234.89 Requests/sec executed
Test execution summary:
total time: 60.0034s
total number of events: 14094
total time taken by event execution: 59.9868
per-request statistics:
min: 0.0001s
avg: 0.0043s
max: 0.2521s
approx. 95 percentile: 0.0129s
Threads fairness:
events (avg/stddev): 14094.0000/0.00
execution time (avg/stddev): 59.9868/0.00
2Gig file test with 64 threads
sysbench --init-rng=1 --test=fileio --max-time=60 --max-requests=1000000 \
--file-num=128 --file-extra-flags=direct --file-fsync-freq=0 \
--file-total-size=2G --num-threads=64 --file-test-mode=rndrd run
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 24639 Read, 0 Write, 0 Other = 24639 Total
Read 384.98Mb Written 0b Total transferred 384.98Mb (6.3875Mb/sec)
408.80 Requests/sec executed
Test execution summary:
total time: 60.2719s
total number of events: 24639
total time taken by event execution: 3847.3337
per-request statistics:
min: 0.0023s
avg: 0.1561s
max: 0.5804s
approx. 95 percentile: 0.3490s
Threads fairness:
events (avg/stddev): 384.9844/0.94
execution time (avg/stddev): 60.1146/0.08
2 Gig file test single thread random writes --file-test-mode=rndwr
sysbench --init-rng=1 --test=fileio --max-time=60 \
--max-requests=1000000 --file-num=128 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=2G --num-threads=1 \
--file-test-mode=rndwr run
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 0 Read, 52960 Write, 0 Other = 52960 Total
Read 0b Written 827.5Mb Total transferred 827.5Mb (13.748Mb/sec)
879.89 Requests/sec executed
Test execution summary:
total time: 60.1893s
total number of events: 52960
total time taken by event execution: 60.1396
per-request statistics:
min: 0.0001s
avg: 0.0011s
max: 2.1815s
approx. 95 percentile: 0.0001s
Threads fairness:
events (avg/stddev): 52960.0000/0.00
execution time (avg/stddev): 60.1396/0.00
2 Gig file test 64 threads random writes --file-test-mode=rndwr
sysbench --init-rng=1 --test=fileio --max-time=60 \
--max-requests=1000000 --file-num=128 --file-extra-flags=direct \
--file-fsync-freq=0 --file-total-size=2G --num-threads=64 \
--file-test-mode=rndwr run
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 0 Read, 38767 Write, 0 Other = 38767 Total
Read 0b Written 605.73Mb Total transferred 605.73Mb (10.038Mb/sec)
642.45 Requests/sec executed
Test execution summary:
total time: 60.3424s
total number of events: 38767
total time taken by event execution: 3861.3838
per-request statistics:
min: 0.0001s
avg: 0.0996s
max: 4.8871s
approx. 95 percentile: 0.8721s
Threads fairness:
events (avg/stddev): 605.7344/14.70
execution time (avg/stddev): 60.3341/0.00
Rerunning the whole test set on /mnt
cd /mnt
vi sysbench_fileio.sh
chmod 700 sysbench_fileio.sh
./sysbench_fileio.sh
sysbench v0.4.8: multi-threaded system evaluation benchmark
1 files, 131072Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 9782 Read, 0 Write, 0 Other = 9782 Total
Read 152.84Mb Written 0b Total transferred 152.84Mb (2.547Mb/sec)
163.01 Requests/sec executed
Test execution summary:
total time: 60.0092s
total number of events: 9782
total time taken by event execution: 59.9964
per-request statistics:
min: 0.0003s
avg: 0.0061s
max: 0.1835s
approx. 95 percentile: 0.0100s
Threads fairness:
events (avg/stddev): 9782.0000/0.00
execution time (avg/stddev): 59.9964/0.00
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 10799 Read, 0 Write, 0 Other = 10799 Total
Read 168.73Mb Written 0b Total transferred 168.73Mb (2.7943Mb/sec)
178.83 Requests/sec executed
Test execution summary:
total time: 60.3856s
total number of events: 10799
total time taken by event execution: 3852.0646
per-request statistics:
min: 0.0098s
avg: 0.3567s
max: 0.9908s
approx. 95 percentile: 0.6044s
Threads fairness:
events (avg/stddev): 168.7344/4.29
execution time (avg/stddev): 60.1885/0.11
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 0 Read, 15274 Write, 0 Other = 15274 Total
Read 0b Written 238.66Mb Total transferred 238.66Mb (3.9774Mb/sec)
254.55 Requests/sec executed
Test execution summary:
total time: 60.0029s
total number of events: 15274
total time taken by event execution: 59.9853
per-request statistics:
min: 0.0003s
avg: 0.0039s
max: 0.3117s
approx. 95 percentile: 0.0106s
Threads fairness:
events (avg/stddev): 15274.0000/0.00
execution time (avg/stddev): 59.9853/0.00
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
1 files, 128Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 0 Read, 13360 Write, 0 Other = 13360 Total
Read 0b Written 208.75Mb Total transferred 208.75Mb (3.4615Mb/sec)
221.54 Requests/sec executed
Test execution summary:
total time: 60.3062s
total number of events: 13360
total time taken by event execution: 3847.7447
per-request statistics:
min: 0.0003s
avg: 0.2880s
max: 60.2694s
approx. 95 percentile: 0.0175s
Threads fairness:
events (avg/stddev): 208.7500/1157.46
execution time (avg/stddev): 60.1210/0.07
sysbench v0.4.8: multi-threaded system evaluation benchmark
Removing test files...
sysbench v0.4.8: multi-threaded system evaluation benchmark
128 files, 1024Kb each, 128Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
128 files, 1Mb each
128Mb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 0 Read, 14728 Write, 0 Other = 14728 Total
Read 0b Written 230.12Mb Total transferred 230.12Mb (3.8192Mb/sec)
244.43 Requests/sec executed
Test execution summary:
total time: 60.2541s
total number of events: 14728
total time taken by event execution: 3846.4353
per-request statistics:
min: 0.0004s
avg: 0.2612s
max: 1.5574s
approx. 95 percentile: 0.5732s
Threads fairness:
events (avg/stddev): 230.1250/10.67
execution time (avg/stddev): 60.1006/0.03
sysbench v0.4.8: multi-threaded system evaluation benchmark
128 files, 16384Kb each, 2048Mb total
Creating files for the test...
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 8496 Read, 0 Write, 0 Other = 8496 Total
Read 132.75Mb Written 0b Total transferred 132.75Mb (2.2124Mb/sec)
141.60 Requests/sec executed
Test execution summary:
total time: 60.0016s
total number of events: 8496
total time taken by event execution: 59.9901
per-request statistics:
min: 0.0003s
avg: 0.0071s
max: 0.2733s
approx. 95 percentile: 0.0107s
Threads fairness:
events (avg/stddev): 8496.0000/0.00
execution time (avg/stddev): 59.9901/0.00
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 9497 Read, 0 Write, 0 Other = 9497 Total
Read 148.39Mb Written 0b Total transferred 148.39Mb (2.4566Mb/sec)
157.22 Requests/sec executed
Test execution summary:
total time: 60.4044s
total number of events: 9497
total time taken by event execution: 3854.1290
per-request statistics:
min: 0.0094s
avg: 0.4058s
max: 1.1867s
approx. 95 percentile: 0.7207s
Threads fairness:
events (avg/stddev): 148.3906/3.90
execution time (avg/stddev): 60.2208/0.12
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 1
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
Done.
Operations performed: 0 Read, 12236 Write, 0 Other = 12236 Total
Read 0b Written 191.19Mb Total transferred 191.19Mb (3.1862Mb/sec)
203.91 Requests/sec executed
Test execution summary:
total time: 60.0056s
total number of events: 12236
total time taken by event execution: 59.9917
per-request statistics:
min: 0.0003s
avg: 0.0049s
max: 0.3623s
approx. 95 percentile: 0.0117s
Threads fairness:
events (avg/stddev): 12236.0000/0.00
execution time (avg/stddev): 59.9917/0.00
sysbench v0.4.8: multi-threaded system evaluation benchmark
Running the test with following options:
Number of threads: 64
Extra file open flags: 16384
128 files, 16Mb each
2Gb total file size
Block size 16Kb
Number of random requests for random IO: 1000000
Read/Write ratio for combined random IO test: 1.50
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random write test
Threads started!
Time limit exceeded, exiting...
(last message repeated 63 times)
Done.
Operations performed: 0 Read, 12251 Write, 0 Other = 12251 Total
Read 0b Written 191.42Mb Total transferred 191.42Mb (3.1792Mb/sec)
203.47 Requests/sec executed
Test execution summary:
total time: 60.2109s
total number of events: 12251
total time taken by event execution: 3847.4821
per-request statistics:
min: 0.0039s
avg: 0.3141s
max: 1.8187s
approx. 95 percentile: 0.6672s
Threads fairness:
events (avg/stddev): 191.4219/8.33
execution time (avg/stddev): 60.1169/0.04
sysbench v0.4.8: multi-threaded system evaluation benchmark
Removing test files...
DBA Dojo 
I didn’t know that /mnt was not as efficient as root partitionSo is it the same for/dev/sdb/dev/sdc/dev/sdd/dev/sdeon large, xlarge instances ?Is there anything special we could do when using bigger instances ?
That is the next installment.The large and extra large instances have more CPU and memory and more allocated space… and slightly better network performance.I am running some sysbench benchmarks over the weekend on the 64 bit instances.
First I discovered via your post that there was a difference between using /dev/sda1 instead of /dev/sda2 for your files (disavantage for /dev/sda2)So I looked for such an information on the web and found a post with the opposite affirmation !http://www.linuxinsight.com/how_fast_is_your_disk.html#comment-1142First I would like to understand why the performances are different with the two mount points and second what is true. Or did I missed some points in both posts ?
Thanks for the link.I did a quick test on a small EC2 instance and the requests/sec were better for /etc/sda2 to begin with. However if you notice the next comment with regards to xen sparse images. I created a zero’ed 7G file and the requests/sec dropped dramatically.I will post the results this week for that and the large instance
Paul,For these results it is not only numbers which need to be looked but also if they make sense considering all the setup. The test done uses very small set and it heavily relays on O_DIRECT being actually direct IO with no buffering/caching which may not be the case with virtualization.If you have any hesitations on how it works you can test things out with large file (say 10x amount of memory)
I’m having problem compiling sysbench on 64 bit centos. Any pointer on how to do that?