Welcome! Log In Create A New Profile

Advanced

USB's lifetime

Posted by enki 
USB's lifetime
September 30, 2013 07:37PM
Hi All,

@DonCharisma has mentioned that it's wise to minimize activity on USB Debian stick used to boot GoFlexNet. Sure things this is true. An ordinary USB stick will have limited write/read cycles and will fail one day, ussually before you expect. I had this problem before and probably many of us have it.

Kindly search for your oppinion how is the best to save USB's lifetime by minimizing write/read actvity on USB. @DonCharisma suggested to disable as much as possible logs activity. My question is; what is the most efficient way to do this in Debian wheezy we have?

It was another suggestion not to place Swap partition on the USB. What is the best way to do it and how?

ES.
Re: USB's lifetime
October 01, 2013 01:39AM
To prolong the life of your rootfs USB thumb drive:

1. Use busybox-syslogd to log to RAM.
2. For each package you installed, either specify the log file in /tmp. Or overmount the log files to /tmp (mount -o bind).
3. Use a swap file so you can create it on an attached HDD. And create one on flash drive in case the HDD is not attached. Use a script to detect if the HDD is present during boot. If it is not then switch to the USB thumb swap (this should not happen, so it is just a precaution).
4. Watch the swap if it is on USB thumb. There should be no swap needed. If seeing swap actvities then always use the HDD to handle swap.
5. Use iotop command to watch frequent disk IOs (to figure out which log files needed to be moved or overmounted).
iotop -o -b -qqq
6. Use zswap if the kernel supports it.

With all these remedies, you can use Ext3 for the USB rootfs on a thumbdrive to prevent corruption during power outage.

-bodhi
===========================
Wiki
latest Kirkwood kernel builds and rootfs
latest u-boot-kirkwood builds
latest Oxnas kernel builds and rootfs
latest u-boot-oxnas builds
latest MVEBU Armada kernel builds and rootfs
U-Boot & Kernel Booting process
bodhi's u-boot GitHub
bodhi's corner



Edited 3 time(s). Last edit at 07/13/2015 06:01PM by bodhi.
Re: USB's lifetime
October 04, 2013 05:10PM
enki,

bodhi I know is an "expert" in these parts, so I reckon he knows what he's talking about.

I've tried to install Ubuntu on SD memory card before and that didn't work out too well. I thought I'd broken the memory card, but when I formatted it back on FAT/FAT32 it was fine. I read somewhere that SD cards are designed to work FAT formatted and don't like other file systems. I've tried different SD cards and results were the same/similar. The only common factor was the reader/writer USB unit, it's possible that this is the problem and a different unit would work OK.

For USB sticks, I only have had Sandisk Cruzer and Cruzer fit, and they seem to work OK, although I'd err to the side of the standard Cruzer (not the Fit). There may well be other brands and models, but I just don't have any experience.

If you're going to use Flash stick persistently, then my suggestion would be get the install how you want it, then pull the stick and duplicate it on a Desktop machine (double sure take an image of the rootfs). Keep the backup stick handy for when it gives up. Obviously if you're making major changes then you need to do the copy procedure again.

Another option would be to go the Embeded Debian on your Pogoplug/Goflex, I think that built in flash is faster than the USB and it's more likely designed for heavier usage. There are posts on the forum about how to do this.

One last option I know there is a company in UK called linitx, they sell industrial flash drives, presumably those bad-boys will last a lot longer.

Cheers

Don Charisma
http://DonCharisma.com
Re: USB's lifetime
October 04, 2013 05:56PM
Sandisk Fit is a real good choice! I've been using 4G Sandisk Fit ext3 rootfs for my 2 media NAS for about a year. Swap file is on an attached Ext3 HDD. So far it has survived about 5 power outages. Booted right backup as if nothing has happened.
shv
Re: USB's lifetime
October 05, 2013 03:47AM
Maybe a flash stick with SLC flash cells is recommended even it is more expensive. Some additional hints need to be regarded. It is more or less the same as for SSD usage. To prevent wear out of certain flash areas only one partition over the complete Flash memory shall be used. The partition shall be aligned to the Flash sectors. I don't know how to find out the right alignment for Flash sticks. Moreover the Flash controller in the stick must have a good wear leveling function to prevent wear out of certain flash regions. Because just one partition is available a swap file must be used.
I am not sure if TRIM is supported with USB sticks. If not invalid flash pages are deleted with a garbage collection function of the flash stick.

Many problems with TRIM, garbage collection and wear leveling occured with certain SSDs in the past. Thay were sometimes resolved by firmware updates. Therefore we can expect that there such problems also occure with USB sticks. But I didn't see any firmware update for USB sticks.

- shv
Re: USB's lifetime
October 07, 2013 10:43AM
Any personal experience on how long a life a USB stick lasts? I think mine is over a year or two already.
eas
Re: USB's lifetime
October 10, 2013 10:23PM
I have a couple Mushkin Mullholland 8GB USB sticks I've been using for two years without fail. I haven't babied them. I haven't bothered with having syslog write to RAM. I have SWAP on the stick. I don't even think I've totally disabled atime. What I did do though, is only use 50% of the storage space.
Re: USB's lifetime
October 15, 2013 10:17AM
Is there a practical way to watch for failing drives? Like SMART for flash? The wear leveling is entirely internal to the flash drive, but doesn't it report "bad" or "unusable" sectors to the OS? Is there a good tool for Linux to spot the count going up to preemptively change the drive? One thing that comes to mind in terms of partitioning swap and maybe all of /var/log (if a HDD isn't an option) is to just put a second small cheap flash drive in (we all have a dozen or so of those 1gb or less with no use anymore, amiright) and let it fail; when it fails just toss a new one in. But, if there is no good way to tell when the drive is failing it might be just as troubling to have the unit hang even if the main data is safe.
shv
Re: USB's lifetime
October 20, 2013 03:06AM
There is an interessting discussion about flash memory wear out of SD cards with RaspberryPi: Link.. They recommend to use the fstrim command so that unused blocks become available for wear leveling.

It seams that there might be USB3.0 sticks available which support trim: Link. With the tool TRIMcheck it can be tested if a drive supports trim: Link. If you have a Windows PC where you know that trim is working e.g. with SSD you could try to test USB stick as well.

The problem is not just wearing out of USB flash memory drives but also slowing down write operations if each write needs an erase before.



Edited 1 time(s). Last edit at 10/20/2013 03:09AM by shv.
Termo
Re: USB's lifetime
May 01, 2014 08:42AM
I to have thought a bit about this topic, and recently I fell over this small script:

http://www.nohl.eu/tech-resources/notes-to-linux/log-files-in-tmpfs-without-breaking-logging/

The last post is for debian. Maybe Bodhi could implement this in future rootfs builds?

Regards
TEN
Re: USB's lifetime
January 04, 2017 03:18PM
joebob2000 Wrote:
-------------------------------------------------------
> Is there a practical way to watch for failing drives?
> Is there a good tool for Linux to spot the count going up to preemptively change the drive?

Not exactly, but unlike SMART,
to test the (remaining) free space both initially and occasionally,
there is http://oss.digirati.com.br/f3/
as a native (unixoid) remake
of Heise.de's H2test(w) for (Win)DOS.

Over the past decade at least, I have committed no USB or SD to "production" here without a full test cycle under one of the above, and (besides forgeries) caught many that were faulty fresh-from-factory.



Edited 3 time(s). Last edit at 01/04/2017 03:23PM by TEN.
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: