Back-up USB question

Hello,

Got debian installed on a USB drive and working fine, thanks Jeff!

My next step is to back it up to another USB drive. This almost worked :-).

I did (from the command line, on my macintosh):

dd of=/dev/disk2s1 if=disk.img bs=1024

to get the image off. Then I switched USB drives and did:

dd of=/dev/disk2s1 if=disk.img bs=1024

This went fine too. I then put the second USB drive into my pogoplug, plugged it in and... nothing. That is, I can't ssh into it (and yes, I checked the IP address at my router).

So I pulled the USB drive, rebooted to 'recovery', plugged in the drive and mounted it. It LOOKS fine... so I turned on 'netconsole' and watched the boot. Looks just like it does with the working USB drive. OH YEAH, I did try booting the original drive, and it went fine, and I can ssh in just fine.

So what did I do wrong, if anything? I 'fdisk'd the new USB drive on the pogoplug, and set it up the same.

Where would I look on the mounted USB drive to get a clue to what's happening after netconsole quits reporting? I'll include the netconsole output below.

THANKS!
--yachris

U-Boot 2010.06-00695-gbd23130-dirty (Aug 25 2010 - 01:20:45)
Marvell-Dockstar/Pogoplug by Jeff Doozan
Hit any key to stop autoboot:  0 
(Re)start USB...
USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 4 USB Device(s) found
       scanning bus for storage devices... 1 Storage Device(s) found
Loading file "/boot/uImage" from usb device 0:1 (usbda1)
1430932 bytes read
Loading file "/boot/uInitrd" from usb device 0:1 (usbda1)
4704909 bytes read
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   Linux-2.6.32-5
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1430868 Bytes = 1.4 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   initramfs
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    4704845 Bytes = 4.5 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

I use this command to 'copy' installs (this assumes that you've booted into Debian and that your 'new' drive is mounted in '/tmp/backup':

tar \
  --exclude=/lost+found  \
  --exclude=/tmp  \
  --exclude=/sys  \
  --exclude=/proc \
  -cf - / | ( cd /tmp/backup ; tar -xpvf - )

Your bootlog looks fine. With netconsole, the last think you'll see is

Starting kernel ...

because that's when the kernel takes over and stops sending netconsole messages.

If you're switching between multiple dockstars, watch out for the udev init scripts. It will notice that the network adapters have different MAC addresses and rename eth0 to eth1. By default, eth1 is not configured in /etc/network/interfaces so you'll end up without ssh access.

If you're just switching USB devices on the same Dockstar, I'd recommend trying the above command instead of dd. If none of the above options work, and you're really curious, you may just want to spend the $4 and get a serial cable.

-- Jeff

Jeff Wrote:
-------------------------------------------------------
........................
> If you're switching between multiple dockstars,
> watch out for the udev init scripts. It will
> notice that the network adapters have different
> MAC addresses and rename eth0 to eth1. By
> default, eth1 is not configured in
> /etc/network/interfaces so you'll end up without
> ssh access.
>
......................
>
> -- Jeff

Jeff:
I think this is exactly what happening to me. I installed Squeeze on one Dockstar then cannot move to another one. It is not a problem for Lenny. Is there any way I can fix the problem?

twinclouds Wrote:
-------------------------------------------------------
> Jeff Wrote:
> --------------------------------------------------
> -----
> ........................
> > If you're switching between multiple dockstars,
> > watch out for the udev init scripts. It will
> > notice that the network adapters have different
> > MAC addresses and rename eth0 to eth1. By
> > default, eth1 is not configured in
> > /etc/network/interfaces so you'll end up
> without
> > ssh access.
> >
> ......................
> >
> > -- Jeff
>
> Jeff:
> I think this is exactly what happening to me. I
> installed Squeeze on one Dockstar then cannot move
> to another one. It is not a problem for Lenny.
> Is there any way I can fix the problem?

I've been bitten by this too. The solution is to remove these two files from the rootfs:

rm /etc/udev/rules.d/70-persistent-net.rules
rm /lib/udev/rules.d/75-persistent-net-generator.rules

This is what I read from Google. I will try that out.

Will removing these two files cause any problem?

I renamed these two files last night. Now, there's no issues for an Squeeze installation on other DockStars. It works great now!
Another advantage of Squeeze I found is that if you editor the resolv.conf file, it will not get over written in dhcp mode like Lenny. (There's a way to stop Lenny doing that but some other changes are needed.) One problem I found for Debian on Dockstar, for both Lenny and Squeeze, is that given the default installation, the DNS search is very slow. (You can verify that by do a ping google.com. The turn around is not bad but there is a large gap between the pings.) You can fix that problem by add one or two dns name server at the beginning of resolv.conf. In that case, you don't want the resolv.conf get overwritten.

ecc Wrote:
> I've been bitten by this too. The solution is to
> remove these two files from the rootfs:
>
> rm /etc/udev/rules.d/70-persistent-net.rules
> rm
> /lib/udev/rules.d/75-persistent-net-generator.rule
> s
>

Actually, you don't need to remove the files, you just need to delete the lines from them that have the MAC address of the earlier Dockstar in them.

Jeff Wrote:
-------------------------------------------------------
> I use this command to 'copy' installs (this
> assumes that you've booted into Debian and that
> your 'new' drive is mounted in '/tmp/backup':
>
>
> tar \
> --exclude=/lost+found \
> --exclude=/tmp \
> --exclude=/sys \
> --exclude=/proc \
> -cf - / | ( cd /tmp/backup ; tar -xpvf - )
>
>
> Your bootlog looks fine. With netconsole, the
> last think you'll see is
>
> Starting kernel ...
>
> because that's when the kernel takes over and
> stops sending netconsole messages.
>
> If you're switching between multiple dockstars,
> watch out for the udev init scripts. It will
> notice that the network adapters have different
> MAC addresses and rename eth0 to eth1. By
> default, eth1 is not configured in
> /etc/network/interfaces so you'll end up without
> ssh access.
>
> If you're just switching USB devices on the same
> Dockstar, I'd recommend trying the above command
> instead of dd. If none of the above options work,
> and you're really curious, you may just want to
> spend the $4 and get a serial cable.
>
> -- Jeff

Can I use this command to transfer the whole system from USB stick to a USB HDD? I tried it, does not work. My HDD is formatted to ext3.
It does work if I backup to another USB flash drive, but not for a USB HDD.
What might be missing?



Edited 1 time(s). Last edit at 09/26/2010 12:17AM by qingz.

qingz Wrote:
> Can I use this command to transfer the whole
> system from USB stick to a USB HDD? I tried it,
> does not work. My HDD is formatted to ext3.
> It does work if I backup to another USB flash
> drive, but not for a USB HDD.
> What might be missing?

what's not working? did you mount with type ext3 ?



Edited 2 time(s). Last edit at 09/26/2010 04:17PM by ano.

ano Wrote:
-------------------------------------------------------
> qingz Wrote:
> > Can I use this command to transfer the whole
> > system from USB stick to a USB HDD? I tried it,
> > does not work. My HDD is formatted to ext3.
> > It does work if I backup to another USB flash
> > drive, but not for a USB HDD.
> > What might be missing?
>
> what's not working? did you mount with type ext3 ?

I didn't do anything, but now it works.
Sometimes I see it get an IP address from my router and I can connect to it, but sometimes it does not get IP. The messages from netconsole are exactly the same. I can see it boots from the USB hard drive every time.

I used DD to copy my working Deb install to a larger usb stick (already partitioned). Of course because I used DD I ended up with the wrong sized partition. Here's a fix to get the size restored.

http://www.linuxquestions.org/questions/linux-newbie-8/df-reports-wrong-disk-size-681952/#post3339615

But skip step 8. That would turn on the journal and make your partition an ext3 rather than an ext2 (ext3 is considered a bad idea with flash drives).



Edited 1 time(s). Last edit at 09/30/2010 09:08PM by gzader.

I have a Clonezilla (which is Debian-based) liveusb stick that I can use to boot an Atom netbook. I then use that to do image backups (to a usb hard disk) of the Dockstar boot usb as well as the netbook's SSD, Sheevaplug boot SD cards, and other liveusb sticks.

All the images, when restored, are bootable. Clonezilla image restores create a device with exactly the same partition structure as the original device. A is a very nice tool. BTW, if you end up with partitions that are too small (like if you're restoring to a larger drive than the original image) you can use gparted to change the partition size.

would it be possible to take a backup/snapshot from a running system with dd to an external usb hdd?
i'd like to backup my entire system usb stick monthly. Backupping should run automatically.

would that be possible with dd?

I mount the file system and tar it. I get too nervous when I dd because one wrong key in the dd command and my data could be gone.

i talk about the file system debian runs on.
it's already mounted.

Mount your USB-HDD to directory /backup and then run the following command:

tar \
--exclude=/lost+found \
--exclude=/tmp \
--exclude=/sys \
--exclude=/proc \
--exclude=/backup \
-zcf /backup/backup1.tgz

It's much better then dd and saves also space on the HDD.

I used DD rather than TAR because I had a lot of mounted directories so the exclude list was getting pretty annoying. I agree though, TAR for most systems would likely be a better system.

Another option would be using rsync. You can use that to do incremental backups. I've built two NAS boxes and use RSYNC to copy the contents from one to the other on a regular basis. So far I've never had to do a restore from it. (knock on wood)

I use this script to back up my dockstar to a fileserver, but it should work fine to a USB disk too:

#!/bin/sh -e
mountpoint /mnt/backup
rsync -vax --delete / /mnt/backup