[18373207] Duo Slow NFS Reads, Jumbo Frames and TCP window

The key piece to determine from this most recent data captured at the client is that there are no Jumbo Frames (JF) on the wire. This is an upload of data from the client to the server. In this case the only packets originating from the NAS are simple TCP ACK segments. The NAS would have no reason to use a JF, since all traffic from that end is below the 1500 MTU.
The client is not sending JFs either.

I did measure client upload TCP data rate at 15.13 MBps with an MTU of 1500. This is not the SMB (CIFS) payload, which would be slightly lower and difficult to determine because of the header size and embedded Andx. Although this trace represents less that 6 seconds of the data upload, it is enough to see the bottlenecks and make some projections.

The largest delays in this transfer is caused by the times between the end of the client TCP stream and the ACK from the NAS.

		               From NAS          From Client
Packet count           3125               69657
ACK delay >1000uS      1317 (54%)            23 (< .01%)
delay >500 <1000uS     1317 (42%)            33 (< .01%)

The increased ACK times from the NAS are to be expected because this is an upload and the NAS must calculate chacksums before the ACK. However, since the data is originating from the client as a stream of 1500 MTU packets, and there are no packets retransmitted, the NAS does not have to employ SACK as we saw in you first trace of the download from the NAS. I would have expected these dealy times to be in the order of 300 to 500 microseconds.

I did some what-if scenarios to determine the influence of these ACK times. The first case is if all ACK delays were 1000 uS or less, for both the client and server. This increases the TCP data upload rate from 15.13 MBps to 22.20 MBps. If the ACK times were kept to 500 uS or less, your transfer rate would have been 32.17 MBps.