NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.

Forum Discussion

SuperJT's avatar
SuperJT
Aspirant
Aug 02, 2016

NFS Performance issues

I work for Netapp in the NFS support area.

I puchased this device almost a year ago and for the most part have been happy to have it just as a storage device.

Recently I tried to put a VM on the device.  Mounted nfsv3, and default mount parameters.  The mount is fine, however performance is absolutely terrible.  I took a packet trace to see if it potentially could be network loss.  There were a few packets that appear to be loss, but VERY little and they do not correspond with the latency that i'm seeing.

 

I am seeing upwards of 20 seconds for a single write call.  I can send the trace out to you, but I dont see a way to attach it to this post.  I downloaded gcc5.4 and attempted to untar.  The untar process took 3min 54s to complete.  The file was stored on the filer, so this would be a read and write.  This was intentional to gauge performance. 

 

I need assistance to see what I can do to get better performance out of this.  I dont expect the world, but I do expect to be able to install updates on a windows vm in under 10 hours.  (Local disk it is ~15 minutes).  If there is anything / logs / traces / additional tools to run, please let me know.

8 Replies

Replies have been turned off for this discussion
    • StephenB's avatar
      StephenB
      Guru - Experienced User

      SuperJT wrote:
      I have 6x 3tb WD red drives

      They seem unlikely to be the botteneck.   With single redundancy they should be giving you 100 MB/sec for both reads and writes.

       

      You could try nastester, that will take NFS out of the equation: http://www.808.dk/?code-csharp-nas-performance

       

      Maybe download the logs and double-check the smart stats.

      • mdgm-ntgr's avatar
        mdgm-ntgr
        NETGEAR Employee Retired

        Did StephenB's post resolve your problem. If not, could you send me your logs (see the Sending Logs link in my sig)?

  • Apart from what mdgm mentioned (definitely worth a read), can you please let us know whether you have NFS async enabled or disabled?

    Go to the settings page of the share --> Network Access --> NFS --> Advanced.

     

    This can have a very significant performance impact - running with or without async.

     

     

    Thanks

    • SuperJT's avatar
      SuperJT
      Aspirant

      Hi all, 

       

      Thanks for the response.  I looked over the KB's listed.

      1) I am running 6.5.0 (1 minor version below current i believe)

      2) ASYNC is not enabled (I'll play with this to see if this improves performance)

      3) Bit-Rot Protection is not enabled.

      4) Snapshots are enabled, set for 'smart' for management and are set to daily.

       

      edit:

      Additional outputs:

      mount options

      jtownsen@superjt:~$ mount | grep vm
      192.168.1.79:/data/vm_store on /var/lib/kimchi/nfs_mount/vm_store type nfs (rw,relatime,vers=3,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.79,mountvers=3,mountport=59523,mountproto=udp,local_lock=none,addr=192.168.1.79)

      top.

       

      Tasks: 212 total,   1 running, 211 sleeping,   0 stopped,   0 zombie
      %Cpu(s):  0.2 us,  0.8 sy,  0.0 ni, 85.3 id, 13.7 wa,  0.0 hi,  0.1 si,  0.0 st
      KiB Mem:   3931280 total,  3245088 used,   686192 free,      312 buffers
      KiB Swap:  2094844 total,        0 used,  2094844 free,  1827016 cached
      
        PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                                                                       
       3656 root      20   0     0    0    0 D   1.7  0.0   0:23.35 nfsd                                                                                          
       1504 root      20   0     0    0    0 S   1.3  0.0  49:32.66 md127_raid5                                                                                   
       3655 root      20   0     0    0    0 S   0.7  0.0   0:12.35 nfsd                                                                                          
       1443 root       0 -20     0    0    0 S   0.3  0.0   1:28.83 kworker/2:1H                                                                                  
       2835 root       0 -20 2228m  17m 3120 S   0.3  0.5  16:10.14 vpnserver                                                                                     
       2954 root      35  15 1830m  71m 6596 S   0.3  1.9   6:21.11 Plex Script Hos                                                                               
       2957 admin     20   0 2363m 137m 5188 S   0.3  3.6  14:43.74 sabnzbd                                                                                       
       3543 admin     20   0  130m 7376 3084 S   0.3  0.2   0:02.97 apache2                                                                                       
      14297 root      20   0     0    0    0 S   0.3  0.0   0:01.15 kworker/u8:5                                                                                  
      14549 root      20   0     0    0    0 S   0.3  0.0   0:00.13 kworker/2:1                                                                                   
          1 root      20   0 45720 5052 2488 S   0.0  0.1   0:06.58 systemd    

       

      • SuperJT's avatar
        SuperJT
        Aspirant

        I enabled async on the volume.. Huge perf increase. 95 minutes to 4 minutes. I feel like this is indicating my disks are a problem.

         

        root@superjt:/var/lib/kimchi/nfs_mount/vm_store# time tar xf gcc-5.3.0.tar.gz 
        
        real	95m11.482s
        user	0m9.828s
        sys	0m48.948s
        root@superjt:/var/lib/kimchi/nfs_mount/vm_store# rm -rf gcc-5.3.0/
        root@superjt:/var/lib/kimchi/nfs_mount/vm_store# time tar xf gcc-5.3.0.tar.gz 
        
        real	4m7.290s
        user	0m8.152s
        sys	0m30.148s
        root@superjt:/var/lib/kimchi/nfs_mount/vm_store# 
        
        

NETGEAR Academy

Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology! 

Join Us!

ProSupport for Business

Comprehensive support plans for maximum network uptime and business peace of mind.

 

Learn More