Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Re: Single Instance Storage (Block level deduplication)
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2005-10-10
08:23 AM
2005-10-10
08:23 AM
Single Instance Storage (Block level deduplication)
First post, and a new user 😄 Chuffed to bits with my X6 so far...
I've migrated all my data from my W2K3 Server. I have had the benefit of Single Instance Storage as the server also hosted RIS to build my PC's. I have a lot of development I386's which contain a lot of repeated files. Would be great to have the same, transparent support to remove data redundancy.
I've migrated all my data from my W2K3 Server. I have had the benefit of Single Instance Storage as the server also hosted RIS to build my PC's. I have a lot of development I386's which contain a lot of repeated files. Would be great to have the same, transparent support to remove data redundancy.
Message 1 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2005-10-10
09:50 AM
2005-10-10
09:50 AM
Re: Single Instance Storage
Hi - um, care to give more details for those of us who aren't really sure what you're talking about so we can follow along when Infrant responds? 🙂
-brendan
-brendan
Message 2 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2005-10-10
05:35 PM
2005-10-10
05:35 PM
Re: Single Instance Storage (Block level deduplication)
Single Instance Storage (SIS) allows the server to maintain one copy of a file for multiple "people."
The easiest example is in MS Exchange. If you send an attachment to 30 people, the server used to need to keep 30 different copies of the same file. With SIS, it can keep just the one and use if for everyone.
The easiest example is in MS Exchange. If you send an attachment to 30 people, the server used to need to keep 30 different copies of the same file. With SIS, it can keep just the one and use if for everyone.
Message 3 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2005-10-10
06:27 PM
2005-10-10
06:27 PM
Re: Single Instance Storage (Block level deduplication)
Renegade59 wrote: Single Instance Storage (SIS) allows the server to maintain one copy of a file for multiple "people."
The easiest example is in MS Exchange. If you send an attachment to 30 people, the server used to need to keep 30 different copies of the same file. With SIS, it can keep just the one and use if for everyone.
How would this be applicable in a NAS environment? I would think this is more application dependent as the app would need to create appropriate "symlinks" to the data.
Message 4 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2005-10-11
08:01 AM
2005-10-11
08:01 AM
Re: Single Instance Storage (Block level deduplication)
In my environment I have a lot of repeated files from developing unattended Windows 2K3 and XP installs. If you've ever looked in the I386 directory of a Windows install CD you'll see thousands of files.
If you take it a stage further and compare the I386 directory of Windows 2003 Server, Windows 2003 Advanced Server, Windows 2003 Web Server and then multiply that by the different licence types like Volume, Retail and OEM you'll find very few files differ and that thousands are identical.
Single Instance Storage (is a server service), when used with Microsoft's RIS is a way of replacing the physical discrete (ie files) data with (I guess) a Unix like symbolic entry. This cuts down on the amount of *real* disk being used.
My other major uses are DJ'ing and in the future VJ'ing. I see both uses requiring repeated data when editing music / video.
If you take it a stage further and compare the I386 directory of Windows 2003 Server, Windows 2003 Advanced Server, Windows 2003 Web Server and then multiply that by the different licence types like Volume, Retail and OEM you'll find very few files differ and that thousands are identical.
Single Instance Storage (is a server service), when used with Microsoft's RIS is a way of replacing the physical discrete (ie files) data with (I guess) a Unix like symbolic entry. This cuts down on the amount of *real* disk being used.
My other major uses are DJ'ing and in the future VJ'ing. I see both uses requiring repeated data when editing music / video.
Message 5 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2005-10-11
09:00 AM
2005-10-11
09:00 AM
Re: Single Instance Storage (Block level deduplication)
I'm not sure how you would even go about creating such a service on a storage device. The problem is how do you determine when files are non-unique? md5 isn't accurate enough, file name isn't accurate enough, size is not applicable. It starts to become something that is far more resource intensive, just try running noclone, dedupe or any other software on a PC that is supposed to find duplicate files for you. Its a lot to expect of a system that is optimized to read and write files, not file contents.
I don't believe I have ever come across an enterprise storage system that supports single instance, I'm not aware of many applications that even support this outside of Exchange.
I don't believe I have ever come across an enterprise storage system that supports single instance, I'm not aware of many applications that even support this outside of Exchange.
Message 6 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2008-10-31
12:26 AM
2008-10-31
12:26 AM
Re: Single Instance Storage
A revival of an old topic, but it now seems to be a hot topic in the world of Enterprise NAS - It goes by the name of block level deduplication. Is the Readynas powerful enough to run a service like this? It would be absolute boon to the device, where only the deltas in file changes are required.
Message 7 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2008-11-13
07:42 AM
2008-11-13
07:42 AM
Re: Single Instance Storage (Block level deduplication)
I agree, especially for VM shops!!! We have lots of dup'd data in our Win2k3 server installs.
Message 8 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2008-11-13
08:50 AM
2008-11-13
08:50 AM
Re: Single Instance Storage (Block level deduplication)
sis support would be awesome.
for some more info, wiki has a decent explaination.
http://en.wikipedia.org/wiki/Single_instance_store
for some more info, wiki has a decent explaination.
http://en.wikipedia.org/wiki/Single_instance_store
Message 9 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2009-11-02
01:33 PM
2009-11-02
01:33 PM
Re: Single Instance Storage (Block level deduplication)
One day, sometime in the future the Readynas will have dedupe.... 😮
Now ZFS has block level dedupe.....http://blogs.sun.com/bonwick/en_US/entry/zfs_dedup
Now ZFS has block level dedupe.....http://blogs.sun.com/bonwick/en_US/entry/zfs_dedup
Message 10 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2010-02-23
12:34 PM
2010-02-23
12:34 PM
Re: Single Instance Storage (Block level deduplication)
has anyone thought of or tried running open solaris and zfs with dedupe, on a pro/3200 via virtual box?
I realize it would be slower than using native, but for data that is dedupe friendly it might be worth it.
I realize it would be slower than using native, but for data that is dedupe friendly it might be worth it.
Message 11 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2010-11-24
12:10 PM
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2011-01-12
09:37 AM
2011-01-12
09:37 AM
Re: Single Instance Storage (Block level deduplication)
Or netgear can make support for LESSFS which is inline dedup in linux.
http://www.lessfs.com/wordpress
http://www.lessfs.com/wordpress
Message 13 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2011-04-12
03:49 AM
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2011-05-24
12:10 AM
2011-05-24
12:10 AM
Re: Single Instance Storage (Block level deduplication)
TeknoJnky wrote: has anyone thought of or tried running open solaris and zfs with dedupe, on a pro/3200 via virtual box?
I realize it would be slower than using native, but for data that is dedupe friendly it might be worth it.
Would be slower.
Message 15 of 16
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2011-09-30
02:51 PM