NETGEAR is aware of a growing number of phone and online scams. To learn how to stay safe click here.
Forum Discussion
how to optimize a temp folder for maximum write/read performance ?
I have a fairly large plex media server/library running on my 528x, upgraded to 16gig ram. I am interested in utilzing an existing, or if necessary creating a specialized temporary folder dedicat...
I thought about using /run or /dev/shm (same thing?), which appear to use up to max of 8 out of 16g as tmpfs+swap.
While this might work perfect for normal streaming transcodes, I fear it would be insufficient to cover long or multiple dvr shows/movies which could cause out of memory problems.
reading more on plex forums and google, it looks likes actually at some point the streaming transcode now *requires* sizeof media_file+100meg free space in order to start a transcode.
So this means for plex transcoding bluray or 4k rips, this could potentially be 50+ gig per user that is streaming simultaniously.
I knew I should have gone with 64 gigs of ram, LOL.
for reference, here are 2 bonnie++ results: 528x 16g ram, 8x 8tb raid-6
main media share
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
sauron 31840M 93 97 256015 46 140674 41 606 95 411216 37 468.0 39
Latency 159ms 125ms 754ms 16456us 629ms 121ms
Version 1.97 ------Sequential Create------ --------Random Create--------
sauron -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 1962 14 +++++ +++ 4038 41 4503 32 +++++ +++ 5517 60
Latency 69957us 728us 258ms 36138us 8us 60330us
1.97,1.97,sauron,1,1516794851,31840M,,93,97,256015,46,140674,41,606,95,411216,37,468.0,39,16,,,,,1962,14,+++++,+++,4038,41,4503,32,+++++,+++,5517,60,159ms,125ms,754ms,16456us,629ms,121ms,69957us,728us,258ms,36138us,8us,60330usexternal usb3 SSD - ntfs format
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
sauron 31840M 414 99 150494 15 25991 4 652 99 230739 17 1438 89
Latency 21255us 33084us 55335us 16644us 18269us 248ms
Version 1.97 ------Sequential Create------ --------Random Create--------
sauron -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 9591 38 +++++ +++ +++++ +++ 22659 82 +++++ +++ +++++ +++
Latency 3103us 967us 8191us 1476us 18us 6771us
1.97,1.97,sauron,1,1516790413,31840M,,414,99,150494,15,25991,4,652,99,230739,17,1438,89,16,,,,,9591,38,+++++,+++,+++++,+++,22659,82,+++++,+++,+++++,+++,21255us,33084us,55335us,16644us,18269us,248ms,3103us,967us,8191us,1476us,18us,6771usi'll try formatting the ssd with different filesystems and post them later.
I'm wondering if you are working to remove a real bottleneck, or a hypothetical.
Are you finding that Plex can't keep up with the real-time transcoding now? If so, how do you know that it's the disk i/o, and not the CPU?
i
good questions.
I suppose the main problem I am trying to resolve is;
- I can record from my network tuner (hdhomerun quattro) fine.
- I can stream regular bluray/dvd/other media fine.
But when I try to do both, I tend to get screwed up recordings and unstable streaming...
DVR recording is highly write intensive, it is taking a raw OTA tv signal in mpeg2 ts format and recording it to disk continuously while whatever show or movie is being recorded. It is not really CPU bound as it is a just a constant write load, which may also include up to 4 simultanious recordings.
Then there is a post-processing commercial removal after each recording has ended, which is moderately more cpu intensive (scans the recorded file looking for commercial markers and re-writes the new stream).
Then there is the normal media streaming from server to client, which is often either a full audio/video transcode, or audio only or video only transcode, which plex does in like 60 second chunks spread across multiple transcoder threads.
Of which there can be multiple people streaming at the same time.
There is also other background and system processing competing for cpu and io at various times, not to mention the overhead of dual redundancy, checksums, and file creation/deletion time on a large and full btrfs file system.
Now obviously that can be a lot going on, and very well may be I need more cpu, which would be either 628x or newer or competitor models, or a dedicated server/pc for plex.
I can't just pop out and replace the cpu and since I am trying to optimize and work with what I have, which is extra ram, I want to reduce or cushion as much IO from hitting the disks, as I possibly can.
TeknoJnky wrote:
- I can record from my network tuner (hdhomerun quattro) fine.
- I can stream regular bluray/dvd/other media fine.
But when I try to do both, I tend to get screwed up recordings and unstable streaming...
This suggests that it'd be best to have a dedicated volume for DVR, to prevent the disks from thrashing during seeking. You don't need a lot of throughput, since an over-the-air ATSC stream is limited to about 20 megabits (80 megabits for 4 -> 10 MB/s). Though an SSD might prevent any issues when you are recording 4 streams at once.
Either way, a USB 3 connected disk should be fast enough, especially if it is formated with ext (a native linux format).
8 GB of ramdisk would give you about 1 hour of DVR recording time (assuming I got the math right), which wouldn't be enough if you are recording shows simultaneously (or a movie).
TeknoJnky wrote:
Then there is a post-processing commercial removal after each recording has ended, which is moderately more cpu intensive (scans the recorded file looking for commercial markers and re-writes the new stream).
If that is done by decoding/processing the video itself, it would be somewhat worse than a single transcode. But I think these markers are in the transport stream itself - if so, it is a remuxing operation that requires no decoding of the audio/video. The disk load should be the same as copying a large file. It could cause disk thrashing with a mechanical disk if there are other users streaming or other DVR streams still being recorded. Though a USB 3 SSD should be able to handle simultanous recordings + ad removal ok.
TeknoJnky wrote:
Of which there can be multiple people streaming at the same time.
My 526x manages one 1080p transcode comfortably, but can't quite keep up with one 4K transcode. That suggests that it's CPU ceiling is either 2 or 3 1080p simultaneous transcodes. But I haven't tested that - generally my media isn't transcoded unless it is going over the internet (and maybe not then).
Though improving the disk throughput with a ramdisk or SSD might help some, I think the CPU simply won't keep up with > 3, no matter how fast the disks are.
A dedicated server that has GPU acceleration might be worth looking into, if this is a common use case for you.
NETGEAR Academy
Boost your skills with the Netgear Academy - Get trained, certified and stay ahead with the latest Netgear technology!
Join Us!
