RAID of USB drives?

Good day, all.

I'm wanting, if it's possible, to do a mirrored RAID of two 2TB drives in USB enclosures attached to my DebianPogodockstarsheevaplug.

Have any of you done this? Somehow, I can see it being just the kind of thing Petergunn would have tackled with aplomb. =D

Thanks,

FZR

Instead of RAID, have you thought of doing an rsync from one drive to the other on a regular basis via cron? Not as "immediate" as an actual RAID, but far less USB and CPU killing.

I agree, I settled for occasional rsyncs here, too. It's questionable in my view to do any sort of software RAID if you don't have two completely separate channels. Having two RAIDed devices hanging off of one USB hub causes more problems than it resolves.

Another alternative would be to install a hardware raid device that supports a USB interface.

The more I think about it, the more a cron'd rsync job sounds like it'd be perfect.

That's what I'll go with.

All the best,

FZR

i'm doing the exact same thing. from my cron file:

/usr/bin/rsync -av --log-file=/home/toliver/rlog.txt --temp-dir=/media/harddrive98/tmp --port=55555 --delete /media/harddrive45/ /media/harddrive98

runs at 2am nightly.

i even have it emailing me reports!

Another solution would be the following:
There are external enclosures for 2 drives that do Raid1, using a Silicon Image Sil5744 chipset for example. One could use one of those and connect it to the dockstar via USB. This should work as intended. Well it won't be any faster than a single HDD, but it will probably be better than using two USB drives and pure linux software raid. I have one of these enclosures here but couldn't test it yet.

ingmar_k, by all means let us know how it turns out when you get around to trying it.

Will do. ;)

OK, little Update on my part:
Will try to revive my poor old dockstar via JTAG tomorrow (its NAND got erased due to a typo). Should I succeed, I will try the external Sil5744 enclosures as soon as possible. I have two Hitachi/Fabrik Duo Pro enclosures here. If these work, any with the mentioned chipset should. So, wish me luck.

If you get it working, another option is rdiff-backup. It's like rsync but it does offsets for your file changes. So if you delete a file and x amount of time later you realize your mistake it might still be in the backup depending on how full that TB drive gets.

I've got a two nas systems I built. The primary is 2.3TB and the backup is 5.5TB. The nice thing is I can restore with nearly no issue and if one fails out I can just point everyone over to the backup without issue.

Just another option.

gzader Wrote:
-------------------------------------------------------
> If you get it working, another option is
> rdiff-backup. It's like rsync but it does offsets
> for your file changes. So if you delete a file
> and x amount of time later you realize your
> mistake it might still be in the backup depending
> on how full that TB drive gets.
>
> I've got a two nas systems I built. The primary
> is 2.3TB and the backup is 5.5TB. The nice thing
> is I can restore with nearly no issue and if one
> fails out I can just point everyone over to the
> backup without issue.
>
> Just another option.


Interesting. So it uses diff for backup. I suppose one can think of it like a huge local subversion repository, right?

Nice hint anyway. Just had a look at the website and I think I'm going to use that for some of my files in the future. Looks like a good idea for valueable files. Would be over the top for your media files, I guess, but for personal documents and stuff it should really be a good way to backup those important files.

Took longer than expected, but now my dockstar is finally working again. So, I'll test some other stuff today and will probably find the time to test the external enclosures tomorrow. ;-)

Took much longer than anticipated, but finally had the time to test this out.

I had compiled my kernel with support for the chipset, so that was pretty sure going to work. What was a major pain in the you know what was the maintenance under windows. I mean the actual setup of a Raid1 array, and even before that updating the firmware.

Besides that I can use the Sil5744 Raid1, connected via USB, without a problem. Works just like it should.

The downside is that I don't think that there's the Silicon Image Steelvine Manager for Linux. I don't know yet how to manage your Sil5744 Raid under Linux.

Oh and another thing: It seems like one can't just issue a command to spin down the drives. I'll see if setting the idle timer in most current drives firmware works. At least the "green" drives should do it, I hope.



Edited 1 time(s). Last edit at 02/10/2011 01:28PM by ingmar_k.

ingmar_k Wrote:
-------------------------------------------------------
...
> The downside is that I don't think that there's
> the Silicon Image Steelvine Manager for Linux. I
> don't know yet how to manage your Sil5744 Raid
> under Linux.
...

Just a little follow up to my own posting:
ATM I am only running a Linux System (Linux Mint Debian 64-Bit), but I still wanted to get my Raid enclosure initialized with 2 new disks (2x WD Scorpio 320GB 2.5"). So I had a look at the SteelVine Software available from Silicon Image and just tried the "latest" (2008) version, for FC6.
As I'm typing this message the rebuild steadily works, initializing my RAID1 setup with these two disks. After the rebuild is finished, I will then try the eSATA connection, too, which I didn't do yet.

It seems that one can fully use those Sil5744 based enclosures under Linux, today. And considering I'm planning on using 2 of these enclosures paired with my Dockstar (2x 320GB Raid1 and 2x 2TB Raid1), I hope they won't give me much trouble in the future.

Edit:
E-Sata seems to work just fine. Pretty quick and without problems so far.



Edited 1 time(s). Last edit at 05/07/2011 05:14AM by ingmar_k.

Hi ya,

Some of the better Sil5744 enclosures allow you to configure the RAID Mode via DIP Switches - avoiding the need for the SteelVine software altogether.

StarTech offer a couple of eSata/USB2 models that I've been lucky enough to test out recently - which support the selection of RAID 0 / RAID 1 / JBOD and a couple of hybrid RAID 0 / 1 modes as well. There's also a DIP setting to use 'GUI' mode - which in effect relies upon SteelVine to then perform all config.

I tested setting the drives up in RAID 1 mode - dropped some files onto the array - then converted the drive to 'GUI' mode and apart from a one-time 'rebuild' the drive's contents were all present and correct.

I like this approach, as for those who want a software-free install, the DIP-switch based RAID settings are a pretty good way to go.

Manufacturers other than StarTech may offer these features as well - but these are the only models I've so far had a play with.