File Benchmark

When a collega asked for best way to restore 2 million small .jpeg files I got curious how some common Linux filesystems would behave with huge amount of small files. I wrote a script to do some simple testing and started it. It took a while to complete (over a week) but here are the results.

Stats

Nodename	Filesystem	File IO, 8G file								Files			Sequential, Files/second & % time in CPU						Random, Files/second & % time in CPU
		Sequential						Random		K Files	Max Size	Min Size	Create	%Cpu	Stat	%Cpu	Delete	%Cpu	Create	%Cpu	Stat	%Cpu	Delete	%Cpu
		Write	CPU%	ReWrite	CPU%	Read	CPU%	Read	CPU%	K Files	Max Size	Min Size	Create	%Cpu	Stat	%Cpu	Delete	%Cpu	Create	%Cpu	Stat	%Cpu	Delete	%Cpu
Sisko	xfs	57885	12	23572	4	51591	6	11	0	2000	5555	4444	216	3	2306	11	294	2	232	5	51	0	33	0
Sisko	ext2	48950	8	22345	4	51228	5	106	9	2000	5555	4444	28	99	2605	8	13696	13	28	99	34	58	61	99
Sisko	ext3	49669	22	22766	5	54352	6	103	3	2000	5555	4444	11	99	2355	8	13794	40	11	99	25	67	30	78
Sisko	reiserfs	47908	15	22401	5	48886	6	109	1	2000	5555	4444	321	3	57	0	141	1	274	2	51	0	67	0

quark	ext2 32G file	140621	18	84856	15	355649	31			2000	5555	4444	36	99	5785	19	83050	78	38	99	47	58	85	99
quark	ext3 32G file	139824	33	99184	21	352668	31			2000	5555	4444	7475	32	130	0	540	1	7729	33	127	0	319	0
quark	ext4 32G file	210851	43	102500	20	357825	30			2000	5555	4444	8295	32	138	0	580	1	8219	31	136	0	350	1
quark	xfs 32G file	238197	33	93512	16	355984	31			2000	5555	4444	2754	27	5167	24	4879	39	3512	36	115	0	134	1

Lab201	ufs									20	5555	4444	51	1	8510	99	62	1	51	1	8948	97	63	1
mccoy	ufs									2000	5555	4444	77	9	1267	33	32	1	75	12	121	3	27	1

TestSetup:

The test system was one I currently was working on in the lab, I just added an ide disk (OS running on SATA).
Software used for testing was Bonnie++
Note: Was going to test jfs also but mkfs.jfs gave me segfault so I couldn't

Nodename: Sisko
OS: OpenSuse 10.0
CPU: 2 x Opteron 246
Motherboard: Tyan K8SE or S2892
Mem: 2G (4x512Mb) DDR400 ECC
Disks: 1x Seagate ST3120026A 120G UDMA100
Disk structure: One single 20G partition used for this test, rest of disk unused

For some Solaris Comparison I did also check UFS:
NodeName: Lab201
OS: Solaris 9
System: Sun 220R
CPU: 2x450MHz
Mem: 1G
Disk: 1x18G

NodeName: mccoy
OS: Solaris 9
System: Sun Netra T1
CPU: 1x500MHz
Mem: 512Mb
Disk: 3x18G striped (raid0)
NodeName: quark
OS: Centos6.0
System: Dell PE2950
CPU: 2x2x2.66MHz
Mem: 8192Mb
Disk: 4x2T raid5

Test script: TestIo.sh
Test script wrapper: TestIo_at.sh
Test log: TestIo.log

While doing some research I did find someone else that done some testing, see Filesystem performance tweaking with XFS on Linux for the results there.
Copyright (c) Peter Sjöberg @ Techwiz, all rights reserved.