Post your performance results

Brand new ReadyNAS NV, running two 400GB WD "YR" drives in X-Raid. CIFS access, Share security, fast CIFS writes on.

Journaling enabled: 11.42 Mb/S write, 23.29 Mb/S read
Journaling disabled: 16.68 Mb/S write, 24.20 Mb/S read

Computer: home built with Asus P4P800 motherboard, 2G memory, Pentium IV 3.2G, on board 3Com 3C940 GigE NIC. I did not disable any services because I want to see how it runs under my normal conditions.

Network: D-Link DGL-4300 Router, newly installed Cat5e cables and connectors, all Cat5e patch cables, Leviton Cat5e structured wiring cabinet. PC and ReadyNAS autonegotiated to 1Gb/S.

yoh-dah wrote:
Processing TCP packets is a huge part of the overhead as well as copying the packets in and out of network file protocol, local file system, and RAID layers.

Can someone test this out for me? The NV+ supports an FTP server. Let's see what FTP can do (both get and put). If the problem is NFS overhead, then FTP shouldn't have the same problem. FTP has overhead too, but it's small compared to NFS.

Also, you can eliminate the local file system overhead by copying to /dev/null:

dd if=/mnt/file.bin of=/dev/null bs=<rsize> count=<large>

Where *large* means something that will exceed the cache. I'm assuming the NV+ is mounted on /mnt. To do a write test, use /dev/zero. Of course, this is all done under Linux.

yoh-dah wrote:
Also, make sure when you're comparing write performance to use RAID 5, as the parity generation will add another level of overhead.

Ugh, shouldn't parity generation be done by hardware? NV+ is a hardware RAID system, isn't it?

It also depends on what you want to test. If you want to see how fast the controller can access the drives, you should use RAID0. RAID0 is your theoretical max performance. This will tell you how much overhead you have in RAID5 by comparing the results of the two tests. Using RAID0 will also tell you how much of the physical SATA disk bandwidth you're using.

To eliminate the network throughput as a potential bottleneck, use one SATA drive with RAID0, and ftp a large file into the system. This should theorectically flood the bandwidth of that single disk drive.

My goal is to determine where the bottleneck is. Doing the above tests should be able to tell us...

--jc

ReadyNAS NV Rev B

2 x HGST 7K250 250GB HDDs

Firmware: RAIDiator™ v3.00c1-p2 [1.00a025]
Memory: 256 MB [2.5-3-3-7]

Online, 95% of 227 GB used - X-RAID (Expandable RAID), 2 disks

Access: CIFS, FTP, HTTPS
Performance: Enable Write Cache, Disable Full Journaling, Enable Fast Writes


PC

Intel D875PBX
Pentium 4 Northwood 3.0 GHz HT
2 x 1GB DDR RAM Dual Channel 2-3-3-6

NIC

Intel PRO/1000 CT
Auto Neg 1000 Mbps
Default Settings
Intel PROSet 11.1.0.19

Switch

NetGear GS105
Cat5e cables

Results

NV Jumbo On, Direct, NIC @16k

256K_Write 20.589032
256K_Read 20.875454

NV Jumbo On, Direct, NIC @9k

256K_Write 20.501461
256K_Read 20.939234

NV Jumbo On, Direct, NIC @4k

256K_Write 15.570781
256K_Read 20.919339

NV Jumbo On, Direct, NIC JF Disabled

256K_Write 7.799292
256K_Read 21.535115

NV Jumbo Off, Direct, NIC JF Disabled

256K_Write 14.291266
256K_Read 24.301739

NV Jumbo Off, via Switch

256K_Write 15.708299
256K_Read 24.55822

NV Jumbo On, via Switch, NIC @9k

256K_Write 20.244404
256K_Read 20.87902


These seem to be consistent with others in that enabling JF helps writes but slows reads. The interesting result is 'NV Jumbo On, via Switch, NIC @9k' which considering the switch is not supposed to support JF gives the same results as direct. Furthermore I can surf the net just as quickly it seems via my 100BaseT Draytek ADSL router. Since I have done the majority of the copying to the NV I think I will stick with JF off for the improved read, but it might be worthwhile enabling JF as and when I need to do any more bulk copying.

I have two HGST T7K250 250GB HDDs on order to bring the RAID upto 1TB, as well as 1GB stick of Patriot RAM, so I'll post back with updated performance tests once these are installed.

Here's a few measurements I made some time ago:

ReadyNAS X6 rev.B w/ four 300GB HDs; journaling off, jumbo off

100Mbit network:

Internet === Linksys WAG354G modem+router+switch (100Mbit)
                    ^  ^  ^
                    |  |  |__ ReadyNAS X6 rev.B
                    |  |  
(cable ~20m=65ft)-> |  |_____ PC#1 (W2K 100Mbit)
                    |
                    |__ Netgear FS108 switch (100Mbit)
                        ^
                        |__ PC#2 (W2K 1Gbit)

PC#1: write: 8.9 MBps; read: 8.4 MBps
PC#2: write: 8.6 MBps; read: 6.9 MBps

Got a 1Gbit network working for a while:

Internet === Linksys WAG354G modem+router+switch (100Mbit)
                ^
                |__ DLink DGS-1005D switch (1Gbit)
                    ^  ^  ^
                    |  |  |__ ReadyNAS X6 rev.B
                    |  |
(cable ~20m=65ft)-> |  |_____ PC#1 (W2K 100Mbit)
                    |
                    |__ Dlink DGS-1005D switch (1Gbit)
                          ^
                          |__ PC#2 (W2K 1Gbit)

... but then I experienced a few network problems (switch incompatibilities after upgrading to RAIDiator 3.0?; see Network errors - b6 issue? for details) and I had to come back to my old 100Mbit network :cry:; here's the performance tests I made while everything was working:

PC#1: write: 9.7 MBps; read: 9.0 MBps
PC#2: write: 14.8 MBps; read: 28.4 MBps

Going toward 1Gbit network was definitely cool! Waiting Xmas for my buying two *new* switches ;)…

Luc.

PC:
Dell XPS 400
CPU Pentium D 2.80Ghz
Memory 3.25G

Network Equipment:
Dlink DGS-1008D* ( 10/100/1000 No JF support? The D-Link user manual says otherwise but it's on the Infrant HDW Incompatibility list. )
* Will post results after getting a D-Link DGS-1005D that supports JF.

Network Topology:
Dell ------ D-Link------Infrant
Intranet-----|

NAS Configuration:
Infrant ReadyNAS NV+
Version: RAIDiator v3.01c1-p2 [1.00a032]
Memory: 256MB [2.5-3-3-7]

RAID
X-RAID Redundant
Disk Write Cache enabled
Full Data Journaling Disabled
Fast CIFS writes Enabled

Disks
4-500GB Hitachi "Deathstars" ( HDS725050KLA360 )


Results:

1000Mbit JF Off
Sectors: 2048000
Write MBps: 15.670103
Write Iops: 62.680412
Read MBps: 28.604001
Read Iops: 114.416004


Sectors: 8192000
Write MBps: 15.180526
Write Iops: 60.722104
Read MBps: 28.140746
Read Iops: 112.562983


1000Mbit JF On
Sectors: 2048000
Write MBps: 7.444973
Write Iops: 29.779891
Read MBps: 23.555285
Read Iops: 94.22114


Sectors: 8192000
Write MBps: 7.320481
Write Iops: 29.281924
Read MBps: 24.238073
Read Iops: 96.952293

Thanks
EDM

edmebba wrote:

PC: Dell XPS 400 CPU Pentium D 2.80Ghz Memory 3.25G Network Equipment: Dlink DGS-1008D* ( 10/100/1000 No JF support? The D-Link user manual says otherwise but it's on the Infrant HDW Incompatibility list. ) * Will post results after getting a D-Link DGS-1005D that supports JF. Network Topology: Dell ------ D-Link------Infrant Intranet-----| NAS Configuration: Infrant ReadyNAS NV+ Version: RAIDiator v3.01c1-p2 [1.00a032] Memory: 256MB [2.5-3-3-7] RAID X-RAID Redundant Disk Write Cache enabled Full Data Journaling Disabled Fast CIFS writes Enabled Disks 4-500GB Hitachi "Deathstars" ( HDS725050KLA360 ) Results: 1000Mbit JF Off Sectors: 2048000 Write MBps: 15.670103 Write Iops: 62.680412 Read MBps: 28.604001 Read Iops: 114.416004 Sectors: 8192000 Write MBps: 15.180526 Write Iops: 60.722104 Read MBps: 28.140746 Read Iops: 112.562983 1000Mbit JF On Sectors: 2048000 Write MBps: 7.444973 Write Iops: 29.779891 Read MBps: 23.555285 Read Iops: 94.22114 Sectors: 8192000 Write MBps: 7.320481 Write Iops: 29.281924 Read MBps: 24.238073 Read Iops: 96.952293 Thanks EDM

Got the D-Link DGS-1005D and got the same numbers, any sugestions? I do not see a direct way to set host system to use Jumbo frames, according to the Users guide I just need to make sure the Intel Pro Set utility is being used, which it is. The Intel host system card is a Intel Pro/1000 PL. The host system that is running Iometer and the Ready NAS are plugged directly into the D-Link DGS-1005.

EDM

edmebba wrote:

edmebba wrote: PC: Dell XPS 400 CPU Pentium D 2.80Ghz Memory 3.25G Network Equipment: Dlink DGS-1008D* ( 10/100/1000 No JF support? The D-Link user manual says otherwise but it's on the Infrant HDW Incompatibility list. ) * Will post results after getting a D-Link DGS-1005D that supports JF. Network Topology: Dell ------ D-Link------Infrant Intranet-----| NAS Configuration: Infrant ReadyNAS NV+ Version: RAIDiator v3.01c1-p2 [1.00a032] Memory: 256MB [2.5-3-3-7] RAID X-RAID Redundant Disk Write Cache enabled Full Data Journaling Disabled Fast CIFS writes Enabled Disks 4-500GB Hitachi "Deathstars" ( HDS725050KLA360 ) Results: 1000Mbit JF Off Sectors: 2048000 Write MBps: 15.670103 Write Iops: 62.680412 Read MBps: 28.604001 Read Iops: 114.416004 Sectors: 8192000 Write MBps: 15.180526 Write Iops: 60.722104 Read MBps: 28.140746 Read Iops: 112.562983 1000Mbit JF On Sectors: 2048000 Write MBps: 7.444973 Write Iops: 29.779891 Read MBps: 23.555285 Read Iops: 94.22114 Sectors: 8192000 Write MBps: 7.320481 Write Iops: 29.281924 Read MBps: 24.238073 Read Iops: 96.952293 Thanks EDM Got the D-Link DGS-1005D and got the same numbers, any sugestions? I do not see a direct way to set host system to use Jumbo frames, according to the Users guide I just need to make sure the Intel Pro Set utility is being used, which it is. The Intel host system card is a Intel Pro/1000 PL. The host system that is running Iometer and the Ready NAS are plugged directly into the D-Link DGS-1005. EDM

EDM, please open a new thread and we can discuss there.

Minor update

Changes:

ReadyNAS NV Rev B

2 x HGST 7K250 250GB HDDs
2 x HGST T7K250 250GB HDDs

Firmware: RAIDiator™ v3.01c1-p3 [1.00a032]

Online, 37% of 681 GB used, X-RAID (Expandable RAID), 4 disks

All streaming services disabled (was HSMS, uPnP)

Results:

Immediately after expansion

Jumbo OFF, via Switch

256K_Write 11.754537
256K_Read 21.188849

Which I found disappointing since I had assumed that a stripe set would be faster than a mirror, at least for reads. Thinking perhaps that the expansion had not best optimised the test file I deleted it and allowed IOMeter to recreate it.

256K_Write 15.476495
256K_Read 25.818728

Much better, and a slight improvement from 2 disks

NOTE: there was also a reboot of NAS and PC after deleting the test file, so this might have had some influence too.

NV Jumbo On, via Switch, NIC @9k

Run 1

256K_Write 18.467976
256K_Read 20.927655

Run 2

256K_Write 20.036544
256K_Read 21.021096

I did two runs as I saw some quite large access times for the first run. Run 2 is better but no real change from 2 disks.

I also discovered that my switch does support Jumbo Frames, which is :D

As before since reads far outnumber writes in quantity and priority, I am running with JFs disabled.

...and ran some informal tests.

Asus P5W DH motherboard with onboard Marvell Yukon Ethernet
Core 2 Duo E6600 CPU
2 GB DDR-800 RAM
XP Pro/SP2 latest patches
Grisoft AVG active

Infrant ReadyNAS NV
RAIDiator™ v3.00c1-p2 [1.00a025]
Patriot 1024 MB [2.0-2-2-6]
Volume C: Online, 26% of 1088 GB used
X-RAID (Expandable RAID), 4 disks Seagate ST3400620AS
FTP, NFS,
Journaling disabled
CIFS Fast writes enabled
Disk write cache enabled
CIFS,HTTP, uPNP on, other protocols, streaming and discovery services off
UPS online

IOMeter test parameters per the sticky.

Baseline with existing Netgear 100Mbit switches (2) between test machine and NAS

256K_Read
Total I/Os per Second 37.34
Total MBs per Second 9.34
Average I/O Response Time (ms) 26.7719
Maximum I/O Response Time (ms) 187.9842
% CPU Utilization (total) 3.66%
Total Error Count 0

256K_Write
Total I/Os per Second 42.72
Total MBs per Second 10.68
Average I/O Response Time (ms) 23.4049
Maximum I/O Response Time (ms) 226.314
% CPU Utilization (total) 4.80%
Total Error Count 0


Replaced with 2 Netgear GS608 1000Mbit switches (no Jumbo frames)

256K_Read
Total I/Os per Second 137.18
Total MBs per Second 34.29
Average I/O Response Time (ms) 7.2889
Maximum I/O Response Time (ms) 192.7529
% CPU Utilization (total) 7.76%
Total Error Count 0

256K_Write
Total I/Os per Second 76.99
Total MBs per Second 19.25
Average I/O Response Time (ms) 12.9875
Maximum I/O Response Time (ms) 493.9347
% CPU Utilization (total) 6.14%
Total Error Count 0


Enabled Jumbo frames on ReadyNAS and test system (9014 byte)

256K_Read
Total I/Os per Second 102.92
Total MBs per Second 25.73
Average I/O Response Time (ms) 9.7157
Maximum I/O Response Time (ms) 209.1619
% CPU Utilization (total) 6.55
Total Error Count 0

256K_Write
Total I/Os per Second 102.47
Total MBs per Second 25.62
Average I/O Response Time (ms) 9.7576
Maximum I/O Response Time (ms) 160.2585
% CPU Utilization (total) 3.59%
Total Error Count 0

Checked ReadyNAS to verify network error counts were zero after each run.

I disabled jumbo frames for everyday use. Read perfomance is more important for me.

Just to ensure my post is preserved in the sticky, see my testing in another thread: http://www.infrant.com/forum/viewtopic.php?t=7520

Summary:

18 MB/s IOTEST write performance with journaling off, no jumbo frames, and GE switch (Netgear GS108)
The ReadyNAS+ compares very favourably to nForce4 Ultra RAID performance in RAID1 or RAID0+1 configuration for SATA
ReadyNAS beats the nForce4 Ultra PATA RAID controller.
ReadyNAS beats my desktop WinXP system as CIFS fileserver
ReadyNAS compares very favourably to offline storage systems like DLT tape drives

An additional observation is that with Retrospect up and running, I have never had such a good backup solution as the NV+ and Retrospect.