Storage Help (NAS+Media Streaming+RAID+Automatic Backup)

Aloise wrote:
1. I was thinking about Netgear ReadyNAS 516 (6-Bay). Any suggestions? Any models other than this?

An RN316 is another option. Though the RN516 has a faster processor. The RN716 is faster still, but is more than most home users need.

Aloise wrote:
2. One mandatory feature for me is to have the ability to mirror my data on backup dedicated HDs. I know that there's a tradeoff concerning performance vs security (redundancy). I know very little about RAID and thought that maybe RAID6 would be a reasonable option. I also know that if I fill my NAS with 12Tb, 6Tb would be for useful data and 6Tb for backup (in a nutshell). Does it make sense?

If you have 6x2TB with RAID-6 (dual-redundancy) then you get 8 TB of storage, and 4 TB of redundant data. This allows the RAID array to be rebuilt if two drives fail. This is not a backup though. Backups ideally would be written to a different device (another NAS, USB drives, or perhaps a cloud server). Backups protect against device failure, user error, theft, disaster, etc. RAID only protects against 1 (or 2) failed disks. It is really more about availability (your next question) than protection against loss.

Aloise wrote:
3. Also, my personal desire it that, once a drive has failed, I would like to hotswap it with a new one so that the NAS rebuilds the information and no data loss occurs. Is it feasible or am I dreaming?

Yes, it does work that way.

Aloise wrote:
4. I know these NASes don't work with SAS HDs. So I'll have to stick to SATA Enterprise Class HDs, such as Seagate Constellation or similar. Higher RPM and internal memory would increase disk I/O. Both Netgear and QNAP sell either diskless or loaded with enterprise class HDs. Any comments/suggestions on this?

Personally I use Home NAS drives (I use WD30EFRX at present, though the Seagate ST4000VN000 is also worth looking at). You can of course use enterprise drives if you wish. The home NAS drives are faster than gigabit ethernet for sustained file transfer.

BTW, you can also use SSD drives with this device. Though the new 1 TB SSDs are not on the hardware compatibility list at this point. I don't think I'd use SSDs in a RAID array though - you'd be better off with jbod and backups.

Aloise wrote:
5. Are these NASes able to stream media "from factory", of course through my wifi router or will I need to install tons of apps/software to do it? Any light on this?

What you need depends on your playback devices. You'd need an app for IOS, you might not for Android. I don't stream FLAC or RAW, so I don't know specifically about those. Plex is a DLNA server that is an add-on for the NAS - it has the ability to transcode the media if that is needed for playback. Streaming over the internet is more challenging. The openVPN server in your router would help there.

Aloise wrote:
6. I have an Apple TV box. May I also use it as another option to stream media (videos, audio,photos) from the NAS to my TV?

The short answer here is that none of the NAS (ReadyNAS or its competition) work seamlessly with the Apple TV. That's Apple's fault.

There is an iTunes server, but it can't do home sharing (Apple keeps that to themselves). You can't directly stream to the Apple TV from the NAS, but you can use airplay (assuming the IOS player supports it). Getting airplay pairing to work on the NAS is a bit more difficult than getting airplay pairing to work on your router.

The easiest answer in practice is to put your media library on the NAS, but run the real iTunes on a PC/Mac. Then you can watch the media on the AppleTV and your IOS devices (using the built-in media players).

Thanks for the answers. It's always great to learn from who really knows about it.

I've done some recent research and many people advised me that RAID may not be the best option as it may fail and during data rebuild if the box hangs, data is gone.
So, I'm still thinking of getting a NAS, but now I'm thinking not to use RAID. Is it worth to have a NAS and not using RAID capabilities? ReadyNAS can work like that?

In this case, using a 516 or a 314 for DATA, what option would you suggest for BACKUP? Getting another enclosure for BACKUP so the final setup would be... like... 1 NAS for DATA and 1 NAS for BACKUP?
In such scenario, is it possible to connect each other and run some kind of backup software like SyncBack Free to schedule automatic and periodic backup jobs?

One of my biggest concerns is related to reliability/security. I'm still trying to find the optimum setup for DATA + BACKUP with all the requirements mentioned before (media streaming for heavy content - FullHd videos, RAW photos... and accessible virtually from anywhere on the planet) and, of course, securely stored behind some sort of firewall or any other security solution.

Aloise wrote:
I've done some recent research and many people advised me that RAID may not be the best option as it may fail and during data rebuild if the box hangs, data is gone.

If there's a fire, data is gone. If someone steals your NAS, data is gone. If you aren't paying attention and accidentally delete your files, data is gone. If your NAS is mapped as a network share and you click on a CryptoLocker link in an email, data is gone. Etc, etc.

RAID is for data availability (it allows you to have access to your data even while faulty drives are being replaced) and speed. As knowledgeable people say over and over here in the forum (some even have it in their signatures), RAID is not a substitute for a good backup strategy.

That said, it's easy to see that with the size of today's inexpensive drives and the correspondingly enormous size of 6-bay RAID arrays, hard-drive bit error rates that used to seem completely safe -- like "1 non-recoverable read error in 10^14 (100 trillion) bit reads" -- are now uncomfortably close to "100% chance of a non-recoverable read error somewhere on your RAID array". And the advice you've seen certainly has some basis in reality: When one drive in a RAID array fails, a number of factors conspire to make errors on a second drive more likely.

For this situation, the 4-bay and larger ReadyNAS devices support RAID6, which provides two-disk redundancy. With RAID6, your data is safe unless three drives simultaneously fail. The cost is one drive's worth of storage space: With RAID5, a 6x4TB array provides 20TB of storage, but with RAID6 that array will provide "only" 16TB.

Aloise wrote:
So, I'm still thinking of getting a NAS, but now I'm thinking not to use RAID. Is it worth to have a NAS and not using RAID capabilities? ReadyNAS can work like that?

Yes, the ReadyNAS can be configured for "JBOD" ("Just a Bunch Of Disks"). In a JBOD setup, each drive is its own volume; when one fails, the data on it is gone and cannot be retrieved, but the data on all the other drives is unaffected. JBOD could be ok for a backup of your data, but I wouldn't use it for primary storage.

Aloise wrote:
what option would you suggest for BACKUP? Getting another enclosure for BACKUP so the final setup would be... like... 1 NAS for DATA and 1 NAS for BACKUP? In such scenario, is it possible to connect each other and run some kind of backup software like SyncBack Free to schedule automatic and periodic backup jobs?

There are all sorts of options. If you have a second NAS configured identically to the first, you can make it a mirror of your main NAS; if you ever need to take your main NAS out of service, you can replace it with the backup with essentially zero downtime. If you use external USB drives or a NAS that's not configured as an exact mirror, you can choose to backup some portions of your main NAS on a different schedule than others, make hourly/daily/weekly/monthly snapshots, setup an elaborate system of rotating backup sets, etc.

The ReadyNAS devices include backup functionality sufficient for most simple backup schemes; there are addons for more complicated schemes. I backup my NAS to external USB drives using a combination of rsnapshot and the built-in backup features.

Aloise wrote:
One of my biggest concerns is related to reliability/security. I'm still trying to find the optimum setup for DATA + BACKUP with all the requirements mentioned before (media streaming for heavy content - FullHd videos, RAW photos... and accessible virtually from anywhere on the planet) and, of course, securely stored behind some sort of firewall or any other security solution.

For CPU- and memory-intensive tasks like video transcoding, the 516 looks like the right device to me, although I'm far from knowledgeable about those new OS6 boxes. I'm sure you'll get more dependable advice from someone who actually owns one of them.

For backups, any NAS or collection of USB drives large enough to hold all your data will do.

Aloise wrote:
I've done some recent research and many people advised me that RAID may not be the best option as it may fail and during data rebuild if the box hangs, data is gone.

RAID sometimes fails, and that is one reason you need a real backup. That doesn't mean RAID isn't useful - and everything fails eventually. Though as fastfwd says, you don't need to use it with the NAS.

Whether you use RAID or not, the ideal solution is to
(a) have 2 identical NAS (primary and backup), and to back up one to the other automatically (which the NAS can do).
(b) Also maintain an off-site copy of your data. USB drives stored in another location is one approach, cloud backup (such as CrashPlan) is another.

(a) gives you a fast recovery if the primary NAS fails. (b) preserves your data if something catastrophic happens, and provides additional protection against user error (or something like CryptoLocker). Having 3 copies of all your data on different devices in different places is good protection against loss.

Firstly, thanks once again for your replies. I appreciate and value both your time and information.

I'm thinking of getting a 516 for DATA and maybe a 314 or another 516 for BACKUP.'

DATA
Although RAID6 is an option, I am not confortable with it as I am sure I'll have to get some assistance in case of HDD failure. So, I'm thinking of going JBOD on it.
I'd prefer to keep backing up the data on this DATA NAS to my BACKUP NAS. In case of an HDD failure on my DATA NAS, I'll have the option to swap the faulty drive and restore data from my BACKUP NAS. Sounds more simple to me...
In addition, I'm wondering that in a JBOD setup it's equally easy to access and stream my media files in my wifi LAN. I'm just not sure if I'll have to install additional software to stream such content to my devices (TV, iPad, android smartphone, PC, etc.)

BACKUP
No RAID will be used here as well.
I only have to design the solution so that I'll be able to have the same capacity on DATA and BACKUP NAS. That's why having two 516 NASes sounds more accurate for me than having a 516 for DATA and a 314 for BACKUP, even with different single HDD capacities on each NAS (i.e.: 6x2TB on DATA NAS and 4x3TB on BACKUP NAS)

HDDs
I've done some research about enterprise class SATA HDDs and found better reference on Seagate Constellation HDDs than on WD (reds, blacks, greens, whatever, etc...)
Other than reliability, my concern here is about I/O. I've read that for faster I/O it would be better to install more HDDs with smaller capacity (i.e.: 600Gb... 1TB at most) but for this, I'll have to buy a bigger enclosure and that's not my point. So, it seams quite a good balance to have a 516 filled with 2 or 3Tb HDDs, but I'm not sure if they are either compatible to ReadyNAS boxes (516, 314 ,etc.) and even if they are on the list of approved/compatible drives by Netgear (what would invalidate any support I might need from them). Here are the models that I'm talking about:

Seagate Constellation - 2TB - SATA - 6Gb/s - ST2000NM0033 (without cryptography)
Seagate Constellation - 2TB - SATA - 6Gb/s - ST2000NM0053 (with cryptography)
Seagate Constellation - 3TB - SATA - 6Gb/s - ST3000NM0033 (without cryptography)
Seagate Constellation - 3TB - SATA - 6Gb/s - ST3000NM0053 (with cryptography)

In case these drives are either not compatible or non compliant, which other Enterprise Class HDDs (2Tb and 3Tb) would fit RreadyNAS boxes (516, 314, etc.)?

Using jbod will have no affect on streaming. ReadyNAS DLNA, plex, and the itunes server are built in (though the itunes server is not as useful as you might want).

The main impact of jbod is that you need to balance the storage yourself (since you'd have multiple volumes). This is easier if you use larger drives. That is, 4x3TB is easier to keep in balance than 6x2TB.

All these drives will saturate gigabit ethernet for large transfers, so I wouldn't worry about the I/O. That story is more complicated than capacity anyway (it depends on the number of platters, caching, load etc). If you want to maximize I/O for small/random transfers, I suggest you add in a 1 TB SSD and place the small files / database on there (note there is no 1 TB SSD currently on the HCL, hopefully they will add in more SSDs soon).

You can browse the disk compatibility list yourself here: http://kb.netgear.com/app/answers/detail/a_id/20641

Aloise wrote:
Although RAID6 is an option, I am not confortable with it as I am sure I'll have to get some assistance in case of HDD failure.

Why so sure? The process is very straightforward: With the NAS on, remove the failed drive and insert a new replacement. Open Frontview and follow the resync progress, reboot if/when you're told to.

Aloise wrote:
So, I'm thinking of going JBOD on it. I'd prefer to keep backing up the data on this DATA NAS to my BACKUP NAS. In case of an HDD failure on my DATA NAS, I'll have the option to swap the faulty drive and restore data from my BACKUP NAS. Sounds more simple to me...

Really?

If one or two drives in your DATA NAS fail and you have a RAID6 array, up-to-date data will remain safe and continuously available. Full restore to replacement drives is automatic. Your backup NAS isn't involved in the process at all; it can remain safely away from you and the errors that you, like all humans, make. Also, if any -- or even all -- of the drives in the BACKUP NAS fail while your DATA NAS is rebuilding, your data is still safe (and if "only" one or two of the BACKUP NAS drives failed, your data is not only safe but is also still backed up).

On the other hand... If one or two drives in your DATA NAS fail and you have a JBOD array, data from the most recent backup is safe, but it's unavailable until the restore is complete. Data newer than the most-recent backup is gone forever. Restore to a replacement drive is a manual process, and until you restore the DATA NAS drive, you'll only have one copy of your data -- on the BACKUP NAS, which you'll be manually accessing, while stressed, with filesystem commands containing wildcards.

Aloise wrote:
In addition, I'm wondering that in a JBOD setup it's equally easy to access and stream my media files in my wifi LAN.

As StephenB said, you'll have to split your media collection into drive-sized pieces. If you have a big collection, that could be awkward. Also, your streaming software would have to be able to access a collection split across multiple volumes; not all programs can do that.

Aloise wrote:
I've read that for faster I/O it would be better to install more HDDs with smaller capacity

For JBOD, that is not true. For RAID5 or RAID6, it is also not true if you're only trying to get good streaming performance over WiFi or gigabit Ethernet with modern SATA drives.

I agree. Though if the NAS itself fails, data recovery of RAID-5 or RAID-6 is probably more difficult than JBOD - especially if you don't have a device that lets you mount all the disks. Some people prefer JBOD or RAID-1 for that reason.

However, since you are also buying a backup NAS, you already have that covered - if the main NAS fails, you just switch to the backup.

Ok, Guys

Considering all valuable information that you provided, I'll reconsider using RAID on my DATA NAS...
Let''s focus on hardware first:

1. Is there a problem to use a 516 for DATA and 314 for BACKUP? (I wish there was a 514 model as 516 seams to have way to many slots - and is more expensive as well - but 314 seams not to have enough processor power for what I want).
2. I know this is a Netgear ReadyNAS forum, but as we all know, there are two other considerable competitors out there: Synology and QNAP. Any comments about these vendors? Honestly, is Netgear ReadyNAS the best choice for 4-bay/6-bay NASes (such as 516 or 314)?
3. Is RAID6 the best option for DATA NAS in terms of safety+reliability+speed? I know that, compared to RAID1 or RAID5, RAID6 allows you to replace up to two faulty HDDs in case of failure, but I don't have further reasons to elect the best RAID method/type.
4. Considering NOT to use RAID on my BACKUP NAS. Is there any setup other than JBOD (Just a Bunch Of Disks) that I could use? I thought that, other than RAID, the remaining setup option would be JBOD...
5. Finally, ST3000NM0033 (Seagate Constellation ES.3 3TB 7200 RPM 128MB cache SATA 6.0Gb/s 3.5 internal hard drive) seams to be a great option in terms of enterprise class SATA HDD. I've read previous comments about SSDs but in case I go SATA, do you agree it's a good choice?
6. I've checked Netgear compatibility List and found that ST3000NM0033 nor any other SATA HDD with 128Mb of cache appears there... I'm thinking of sending an email to their support to check this out as the list seams not to be updated for a while...

Aloise wrote:
1. Is there a problem to use a 516 for DATA and 314 for BACKUP?

No, except that you can store more data on the 516 than the 314 can hold.

Aloise wrote:
3. Is RAID6 the best option for DATA NAS in terms of safety+reliability+speed?

Yes.

Aloise wrote:
4. Considering NOT to use RAID on my BACKUP NAS. Is there any setup other than JBOD (Just a Bunch Of Disks) that I could use?

You could use RAID0, which joins all the disks into a single array. It won't be automatically expandable like a RAID5 or RAID6 array, though, and if any one drive fails, the entire array will be unusable.

Aloise wrote:

5. Finally, ST3000NM0033 (Seagate Constellation ES.3 3TB 7200 RPM 128MB cache SATA 6.0Gb/s 3.5 internal hard drive) seams to be a great option in terms of enterprise class SATA HDD. I've read previous comments about SSDs but in case I go SATA, do you agree it's a good choice? 6. I've checked Netgear compatibility List and found that ST3000NM0033 nor any other SATA HDD with 128Mb of cache appears there... I'm thinking of sending an email to their support to check this out as the list seams not to be updated for a while...

Six of those drives in RAID6 will give you 12TB of storage for around $1680, or $140/TB.

Six of the NAS-rated ST4000VN000 drives in RAID6 will give you 16TB of storage for $1020, or $64/TB; if you only want 12TB, you can get that with five ST4000VN000s in RAID6 for $850, or $71/TB (or, since you'll have a backup NAS, you might even want to do it with four ST4000VN000s in a RAID5 array for $680, or $56/TB).

The ST4000VN000s run cooler, run quieter, and use less power than the ES drives. They're faster than your gigabit Ethernet, so you won't see any measurable difference in data-access speed versus the ES drives, and you can use the money you save to buy 100 octane race gas for your car.