Welcome! Log In Create A New Profile

Advanced

Samba server project on POGO-V4-A3-01 + 2017.07 U-Boot Kirkwood + kernel linux-5.9.3-kirkwood-tld-1 + Debian 5.2.9 rootfs

Posted by chaos 
Samba server project on Pogo v4 using SATA harddisk instead of USB for rootfs

I write this post a little bit as a help for other newbies who may find same or similar issues to make a SATA HDD boot linux as I did at first.

It was not so easy to figure out how to set up a rootfs on a SATA harddisk, instead of on a regular USB-stick. There was no post for the Pogo v4 about this.

Some other posts in the forum indicated that the firmware bootflags need to be adjusted for SATA boot compared to USB boot. In my case, no altering of boot parameters was needed. My current settings work regardless of booting with USB or with SATA. This is how it looks like for Pogo v4 booting from SATA or USB:
root@debian:~# fw_printenv set_bootargs
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts $custom_params
root@debian:~# fw_printenv bootargs
bootargs=console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=orion_nand:2M(u-boot),3M(uImage),3M(uImage2),8M(failsafe),112M(root)
The label of the /dev/sda1 partition has to be rootfs for this to work. If both usb-stick with label rootfs is in the top slot (under the plastic hood there is a USB-connector and a SATA connector) and a SATA harddisk with label rootfs on first partition is detected, the Pogo v4 seems to prefer the usb-stick. You will have to mount the SATA first partition manually with something like:
mkdir /media/sataHDD
mount /dev/sdb1 /media/sataHDD
when preparing your SATA rootfs directly from Pogo v4. I tried to set up rootfs both from Linux Mint 20 (extracting the Debian-5.2.9-kirkwood-tld-1-rootfs-bodhi.tar.bz2) as well as extracting a file directly on the Pogo v4 to the first partition on SATA HDD. Linux Mint 20 was way quicker... but both ways work.

A note on backing up rootfs partition while it is operating:

  • I haven't figured out how to back up a running rootfs directly from Pogo v4. Sometimes I ended up with disk read errors, so I assume that the rootfs must be backed up when it is not running the linux system. I am sure there are ways to backup a running rootfs, but I don't know how to do this.

It took me a while to figure out the command how to backup the rootfs (note the red dot at the end!), so I share it here. This works in Linux Mint 20:
rootfsMountPoint="/media/$USER/rootfs"
StoreInDir="/home/$USER/"
identifier="SATA-samba-server"
dateOfArchiving="20201111"
osRelease="Debian10"
kernelVersion="5.9.3"
StoreFileWithName="$osRelease-$kernelVersion-kirkwood-tld-1-$identifier-$dateOfArchiving-rootfs.tar.bz2"
tar -cpjvf $StoreFileWithName -C $rootfsMountPoint .
Question about if it is useful to use the tar option --sort=inode

  • Does the order of the files in a linux system impact the perfomance when reading data from the disk? If files are close to each other it is said it optimizes reading speed.
  • Does the linux system shuffle files in an EXT4 to optimize reading speed?

What seemed for me to be an issue 3 times out of 5 was that if /etc/fstab says the rootfs is ext3 but in fact it is EXT4 formatted, the kernel will not always figure it out by itself and switch to EXT4 automatically ... it seems, but I could be wrong on this one, since I also had issues with the GPT. Changing /etc/fstab rootfs entry to ext4 solved the issue that it wouldn't boot always. But then it did boot with ext3 a couple of times as well, so I am not sure why it works sometimes and sometimes not. I write it as a hint that this could be an issue for you, as well as that you forgot to name it properly.

Another issue was that I used fdisk to partition the drive. Apparently, one should use gdisk instead for this. Two times the GPT got corrupt for some reason and I had to start all over. Maybe that was the reason why it would not boot an EXT4 with a setting ext3 in /etc/fstab.

I also tried to partition with gparted, ending up with wrong order in the partitions, where /dev/sda2 and /dev/sda4 had swapped partition numbers with each other. Weird. Even though I tried to fix the partition order, it would not work properly.

Another time I ended up with df -H showing that the disk was full but in fact it was almost empty. I tried to remedy it and have the kernel release the files that it thought was there, but in fact they were not existing, but in vain. I did all the tricks that there are to have df -H compose itself and show the same as du, but not. Had to reinstall in the end again.

Since I encoutered so many problems, I realized maybe someone else is also puzzeled about this so I decided to write this post and share my shortcomings.

Trial to deploy a Samba server on SATA disk

Previous thread suggested that it would be wise(r) to deploy a rootfs partition on a SATA disk instead of a USB-stick. I fount an old(er) SATA disk to test this with. I installed Linux Mint 20 as base system for all my testing.

Of course I forgot to edit
/etc/rc.local
residing on rootfs partion Debian-5.2.9-kirkwood-tld-1-rootfs-bodhi.tar.bz2 sitting physically on
/dev/sda1
which made me realize that /etc/rc.local is probably not run more than once on first boot. I got the same error on Linux Mint 20 as with Ubuntu 20.04 when it comes to the bug when openssh updates and the old keys suddenly are considered wrong. The result is that SSH port 22 never comes up after reboot, thus rendering the machine unusable if the keys cannot be re-generated. See these posts about that issue.

Hence the first question:

  • How can you force this Debian system on Pogo v4 to run a script on every boot, so that you can fix the issue with the SSH keys if you forget to fix the issue before you boot the first time?

Let's assume the script lines are:
#!/bin/bash
LOGFILE="/var/log/renewSSHkeys.log"

/bin/echo $(/bin/date) "Renewing SSH keys" >> "$LOGFILE"

# List the contents of the folder
/bin/ls -al /etc/ssh >> "$LOGFILE"
/bin/echo "$ /bin/ls -al /etc/ssh" >> "$LOGFILE"

# Delete all files in pattern /etc/ssh/ssh_host_*
/bin/rm /etc/ssh/ssh_host_*
/bin/echo "$ rm ssh_host_*/etc/ssh" >> "$LOGFILE"

# List the contents of the folder
/bin/ls -al /etc/ssh >> "$LOGFILE"
/bin/echo "$ /bin/ls -al /etc/ssh" >> "$LOGFILE"

# Re-generate SSH-keys
/usr/bin/ssh-keygen -A
/bin/echo "$ /usr/bin/ssh-keygen -A" >> "$LOGFILE"

/bin/echo "Tried to renew SSH keys. Check the following if new keys were created:" >> "$LOGFILE"

# List the contents of the folder
/bin/echo "$ /bin/ls -al /etc/ssh" >> "$LOGFILE"
/bin/ls -al /etc/ssh >>  "$LOGFILE"

/bin/echo "Done." >> "$LOGFILE"

unset LOGFILE

exit 0

I could't figure it out why /etc/rc.local is read only once, but similar discussions on internet says that Debian removed the functionality in release 9/10 to run it at every boot. Suggestions were made on internet to use the if-up functionality to make scripts run when an internet connection is detected. I haven't been able to figure out how to do this on Pogo v4. I tried also by editing /etc/init.d/rc.local but no new SSH keys were made.

How can you make:

  • if-up run script when internet is detected on this Debian system?
  • /etc/rc.local run a script at every boot? Or
  • /etc/init.d/rc.local run a script at every boot?

I tried to run systemd instead of initd, but systemd eats more RAM memory so I reverted to initd.

To test the server performance on SATA, I chose between setting up a DNS server, Samba server, Pi-hole (requires 512MB RAM according to site so I didn't bother to try to see if it runs on 128MB), nginx Web server, and a HSQL Database server.

I went for a Samba server capable of handling encrypted connections for a public share, a secure share and an LUKS encrypted drive and an LUKS encrypted container file with different levels of access control.

Further, SSH was restricted, root login disabled, SSH port was changed from default, long passwords were set on the linux system as well as on the Samba server.

The encrypted drive and container was setup on a more powerful system, because LUKS behaves differently depending on the system performance. Maybe it is "smarter" to create LUKS encryption directly from Pogo v4, since the encryption will probably be less heavy to deal with the performance issue.

I used the program gdisk to create GPT table, based on information from other posts.
EDIT: According to post below, GPT (made with gdisk) is not neccessary. MBR (made with fdisk) should be just fine.
Megabytes (MB) 	Mebibytes (MiB)
1 MB	        0.95367431640625 MiB
2 MB	        1.9073486328125 MiB
3 MB	        2.8610229492188 MiB
4 MB	        3.814697265625 MiB
5 MB	        4.7683715820312 MiB
6 MB	        5.7220458984375 MiB
7 MB	        6.6757202148438 MiB
8 MB	        7.62939453125 MiB
9 MB	        8.5830688476562 MiB
Gigabytes (GB)  Mebibytes (MiB)
1 GB            953.67431640625 MiB
2 GB            1907.3486328125 MiB
3 GB            2861.0229492188 MiB
4 GB            3814.697265625 MiB
5 GB            4768.3715820312 MiB
6 GB            5722.0458984375 MiB
7 GB            6675.7202148438 MiB
8 GB            7629.39453125 MiB
9 GB            8583.0688476562 MiB
Gigabytes (GB) 	Gibibytes (GiB)
1 GB	        0.93132257461548 GiB
2 GB	        1.862645149231 GiB
3 GB	        2.7939677238464 GiB
4 GB	        3.7252902984619 GiB
5 GB	        4.6566128730774 GiB
6 GB	        5.5879354476929 GiB
7 GB	        6.5192580223083 GiB
8 GB	        7.4505805969238 GiB
9 GB	        8.3819031715393 GiB
In the following example, the hard drive sits on /dev/sda. That may not be suitable in your case. On your system rootfs could sit on /dev/sdb, /dev/sdc or even /dev/sdd depending on how many things you have connected.
In such case, query replace all occurrences of /dev/sda with /dev/sdb for instance, if that is what you need.

This is how the partions are distributed:
Device         Start       End   Sectors   Size Type
/dev/sda1       2048   7815167   7813120   3.7G Linux filesystem
/dev/sda2    7815168 468490239 460675072 219.7G Linux filesystem
/dev/sda3  468490240 488022015  19531776   9.3G Linux filesystem
/dev/sda4  488022016 488397133    375118 183.2M Linux swap
EDIT: With 512MB swap partition it could look like this
Device Start End Sectors Size Type
/dev/sdd1       2048   7815167   7813120   3,7G Linux filesystem
/dev/sdd2    7815168 468490239 460675072 219,7G Linux filesystem
/dev/sdd3  468490240 487397375  18907136     9G Linux filesystem
/dev/sdd4  487397376 488396799    999424   488M Linux swap
EDIT: As indicated in a post below, we can use a swapfile instead of a swap partition.
A swapfile 512MB = 488 MiB would be made like this. source
 cd /
 dd if=/dev/zero of=/swapfile bs=1024 count=488
 #mkswap /swapfile
 #chmod 600 /swapfile
 #swapon /swapfile
 # Add /swapfile to /etc/fstab
 #echo "/swapfile    none    swap    sw    0    0" >> /etc/fstab
A rootfs partition on
/dev/sda1
was set to 4GB = 3.7 GiB.

A swap partition on
/dev/sda4
was set 1.5 times the RAM = 192MB = 183MiB allocated at the very end of the disk.
EDIT: From the post below it is recommended to use 4 times ram. 4 * 128 = 512 MB = 488 MiB
If you instead choose to use the file /swapfile that you created above, make the /swapfile available with:
 cd /
 #dd if=/dev/zero of=/swapfile bs=1024 count=488
 mkswap /swapfile
 chmod 600 /swapfile
 swapon /swapfile
 # Add /swapfile to /etc/fstab
 echo "/swapfile    none    swap    sw    0    0" >> /etc/fstab
For swap partition:
# Make linux swap partition work
 swapPartition='sda4'
 umount /dev/$swapPartition
 mkswap /dev/$swapPartition
 swapon /dev/$swapPartition
 swapon --show
 #Expected result:
 #mkswap: /dev/sda4: warning: wiping old swap signature.
 #Setting up swapspace version 1, size = 183.2 MiB (192053248 bytes)
 #no label, UUID=aaaaaaaa-0000-1111-abcd-123456789abc

 function get-UUID-of-disk ()
 {
   #echo "${@:1}"
   for v in /dev/disk/by-uuid/* ; do echo "`readlink $v`: $v" | grep ../"${@:1}" | cut -d\: -f2 | cut -d/ -f5 ; done
 }

 echo "UUID=$(get-UUID-of-disk $swapPartition)    none   swap  defaults   0   0" >> /etc/fstab

 #nano /etc/fstab
 cat /etc/fstab
The public share on
/dev/sda3
was set to 10GB = 9.3GiB and
the secure share on
/dev/sda2
was set to the remainder of the drive.

Partitions 1-3 were formatted as EXT4. gdisk uses code 8300 for Linux filesystem.
Partition 4 was formatted as Linux swap. gdisk uses code 8200 for Linux swap.

 mkfs.ext4 /dev/sda1
 mkfs.ext4 /dev/sda2
 mkfs.ext4 /dev/sda3

 tune2fs -L rootfs /dev/sda1 
 tune2fs -L secure /dev/sda2 
 tune2fs -L public /dev/sda3 
 # Swap partition cannot be labeled
 # tune2fs -L swap /dev/sda4

 e2label /dev/sda1 rootfs
 e2label /dev/sda2 secure
 e2label /dev/sda3 public
 e2label /dev/sda4 swap

Disabling bash history

Along the path I found out that it could be advisable to avoid logging bash history when typing passwords and make sure that the ssh logon is encrypted so that passwords are not leaked. Also the samba server has to be told to encrypt the traffic, otherwise the file names are fully visible in the packets (used program Wireshark for this with filter " ip.addr==192.168.xxx.yyy and (smb or smb2)").
# Example of how to clear bash history and avoid logging commands on linux
# If you add "[ \t]*" to your HISTIGNORE variable you can make any command be ignored by starting it with a space.
HISTCONTROL=ignoreboth
HISTIGNORE="&:ls:cp:mv:[bf]g:exit:pwd:clear:mount:umount:sshpass:ssh:rm:rmdir:[ \t]*"
 history -c
 unset HISTFILE
# That disables writing the history file, while it still allows to cycle 
# through the last commands using up/down keys.
# Disable history
 rm ~/.bash_history
 history | tail -10
Login can be made with sshpass on the default port 22 or other port specified with -p #### option
 ipAddress='xxx.yyy.zzz.aaa'
 myPassWord='Hack-Me-Now-Please!'
 sshpass -p "$myPassWord" ssh -o StrictHostKeyChecking=no root@$ipAddress -p 22
 unset myPassWord

swapiness

Change swapiness to somewhere between 5-35. This number may prove to be wrong, but it saves the disk a bit so it lasts longer than with default setting 60.
EDIT: According to post below, it is OK to keep it at default 60 on a SATA setup.
 # For SSD: vm.swappiness=1
 # For old SATA harddisk 5 -> 35 for new

 cat /proc/sys/vm/swappiness
 # Expected result: 60
 # Edit or add line vm.swappiness:
 # vm.swappiness=5
 nano /etc/sysctl.conf
 # or if this line is not in /etc/sysctl.conf
 # add it at the bottom of the file
 echo "vm.swappiness=5" >> /etc/sysctl.conf

Fix the locale error

 # Check what is in the file first
 cat /etc/default/locale

 # Fix
 echo LANG=C > /etc/default/locale
 echo LANGUAGE=C >> /etc/default/locale
 echo LC_CTYPE=C >> /etc/default/locale
 echo LC_NUMERIC=C >> /etc/default/locale
 echo LC_TIME=C >> /etc/default/locale
 echo LC_COLLATE=C >> /etc/default/locale
 echo LC_MONETARY=C >> /etc/default/locale
 echo LC_MESSAGES=C >> /etc/default/locale
 echo LC_PAPER=C >> /etc/default/locale
 echo LC_NAME=C >> /etc/default/locale
 echo LC_ADDRESS=C >> /etc/default/locale
 echo LC_TELEPHONE=C >> /etc/default/locale
 echo LC_MEASUREMENT=C >> /etc/default/locale
 echo LC_IDENTIFICATION=C >> /etc/default/locale
 echo LC_ALL=C >> /etc/default/locale

Add support for NTFS disks

Install package fuse before package ntfs-3g.
apt-get install fuse

Install samba

Install packages samba, cryptsetup, console-setup, openssl, ntfs-3g.
apt-get install samba cryptsetup console-setup openssl ntfs-3g
Create some new test groups for access levels: users, sambasecurity
 addgroup users 
 addgroup sambasecurity
 # Show groups on system
 getent group | awk -F: '{ print $1}'
Create some test users: writepublic writesecure writeencrypted
 function randompw ()
 {
    LC_ALL=C tr -dc 'A-Za-z0-9!"#$%&'\''()*+,-./:;<=>?@[\]^_`{|}~' </dev/urandom | head -c 32 ; echo
 }

 userName=writepublic

 # Add user without home directory
 # useradd -M -g users -d /home/$userName -s /bin/bash -p $(randompw | openssl passwd -1 -stdin) $userName
 # Add user with home directory
 useradd -m -g users -d /home/$userName -s /bin/bash -p $(randompw | openssl passwd -1 -stdin) $userName

 # Add samba user plus password
 sambaPassword='bird'
 (echo "$sambaPassword"; echo "$sambaPassword") | smbpasswd -s -a $userName

 userName=writesecure
 useradd -m -g users -G sambashare -d /home/$userName -s /bin/bash -p $(randompw | openssl passwd -1 -stdin) $userName
 sambaPassword='cat'
 (echo "$sambaPassword"; echo "$sambaPassword") | smbpasswd -s -a $userName

 userName=writeencrypted
 useradd -m -g users -G sambashare,sambasecurity -d /home/$userName -s /bin/bash -p $(randompw | openssl passwd -1 -stdin) $userName
 sambaPassword='dog'
 (echo "$sambaPassword"; echo "$sambaPassword") | smbpasswd -s -a $userName

 unset sambaPassword
You can change the password of these users with lines like this example:
 userName=writepublic
 passwd $userName
 # or
 password='newPassword'
 usermod --password $(openssl passwd -1 $password) $userName
 unset password
Mount the public and secure partitions:
 mkdir /mnt/secure
 mkdir /mnt/public
 echo "LABEL=public    /mnt/public    ext4    defaults    0    0" >> /etc/fstab
 echo "LABEL=secure    /mnt/secure    ext4    defaults    0    0" >> /etc/fstab
 mount -a
 # Create Samba directories for testing
 mkdir -p /media/diskNTFS
 mkdir /media/diskNTFS-2
 mkdir /media/diskNTFS-3
 mkdir /media/diskEXT4
 mkdir /media/diskEXT4-2
 mkdir /media/diskEXT4-3
 mkdir /media/sdcard
 mkdir /media/usbstick
 mkdir /media/usbstick-2
 mkdir /media/usbstick-3
 mkdir -p /mnt/luks-media-crypt/luksEXT4
 mkdir /mnt/luks-media-crypt/luksEXT4-2
 mkdir /mnt/luks-media-crypt/luksEXT4-3
 mkdir /mnt/luks-container-crypt
 # Change the permissions of the folders with the commands:
 chmod -R 0770 /mnt/secure
 chmod -R 0770 /mnt/luks-container-crypt
 chmod -R 0770 /mnt/luks-media-crypt
 chmod -R 0777 /mnt/public
 chown -R root:root /mnt/public
 chown -R root:sambashare /mnt/secure
 chown -R root:sambasecurity /mnt/luks-container-crypt
 chown -R root:sambasecurity /mnt/luks-media-crypt
 # Update /etc/fstab with your external disk labels mounting preferences.
 # This is an example of how to do it.
 echo "#LABEL=sdcard    /media/sdcard    ext4    ro,relatime    0    2" >> /etc/fstab
 echo "LABEL=sdcard    /media/sdcard    ext4    rw,relatime    0    2" >> /etc/fstab
 echo "#LABEL=NTFSdisk1    /media/diskNTFS    ntfs-3g    ro,relatime    0    0" >> /etc/fstab
 echo "LABEL=NTFSdisk1    /media/diskNTFS    ntfs-3g    rw,relatime    0    0" >> /etc/fstab
 echo "#LABEL=NTFSdisk2    /media/diskNTFS-2    ntfs-3g    rw,relatime    0    0" >> /etc/fstab
 echo "LABEL=NTFSdisk2    /media/diskNTFS-2    ntfs-3g    rw,relatime    0    0" >> /etc/fstab
 echo "#LABEL=EXTdisk1    /media/diskEXT4   ext4     ro,relatime    0    0" >> /etc/fstab
 echo "LABEL=EXTdisk1    /media/diskEXT4   ext4     rw,relatime    0    0" >> /etc/fstab
 echo "#LABEL=EXTdisk2    /media/diskEXT4   ext4     ro,relatime    0    0" >> /etc/fstab
 echo "LABEL=EXTdisk2    /media/diskEXT4-2   ext4     rw,relatime    0    0" >> /etc/fstab
 echo "# You can use the escape sequence \040 to escape spaces:" >> /etc/fstab
 echo "LABEL=USB\040STICK   /media/usbstick    vfat    ro,umask=0000    0    0" >> /etc/fstab
 echo "LABEL=USB\040STICK   /media/usbstick    vfat    rw,umask=0000    0    0" >> /etc/fstab
 echo "#UUID=AAAA-1234   /media/usbstick    vfat    rw,umask=0000    0       0" >> /etc/fstab

Add LUKS encrypted drive and LUKS encrypted container

 cd /
 # Create a small container.
 dd if=/dev/urandom of=luks-container.img bs=1M count=4

 cryptsetup luksFormat --type=luks1 --hash=sha512 --key-size=512 --cipher=aes-xts-plain64 luks-container.img

 chmod 0600 luks-container.img
 chown root:root luks-container.img
 
 cryptsetup luksDump luks-container.img

 # Make a key this way
 dd if=/dev/urandom of=container-key bs=512 count=8

 chmod 0600 luks-container.img
 chown root:root container-key

 cryptsetup luksAddKey luks-container.img container-key

 cryptsetup luksDump luks-container.img

 cryptsetup luksOpen luks-container.img luks-container-crypt --key-file=/container-key

 # Format the drive as EXT4
 mkfs.ext4 /dev/mapper/luks-container-crypt

 mount /dev/mapper/luks-container-crypt /mnt/luks-container-crypt
For testing purpose, add the same key to an external LUKS-encrypted harddrive.
First locate which device it is with command blkid. Here we assume it is /dev/sdb1.
 cryptsetup luksAddKey /dev/sdb1 container-key
 cryptsetup luksOpen /dev/sdb1 external-luks-drive --key-file=/container-key
 ls /dev/mapper
You should see external-luks-drive there.

To automatically decrypte luks encrypted containers and drives, we need to add the information to /etc/crypttab
Here comes the security concern, that the key is unencrypted in the root folder.
All that is done is to only allow root to read the key, but that doesn't stop anyone from using it if they gain access to the hard drive. Is there a better solution?
 # Add the luks encrypted drives to /etc/crypttab
 echo "luks-container-crypt    /luks-container.img    /container-key    luks" >> /etc/crypttab
 # We can reuse the function we defined earlier
 function get-UUID-of-disk ()
 {
   #echo "${@:1}"
   for v in /dev/disk/by-uuid/* ; do echo "`readlink $v`: $v" | grep ../"${@:1}" | cut -d\: -f2 | cut -d/ -f5 ; done
 }
 luksPartition=sdb1
 echo "external-luks-drive    UUID=$(get-UUID-of-disk $luksPartition)    /container-key    luks" >> /etc/crypttab
Now we need to add information to /etc/fstab to allow automounting of LUKS-ecrypted drives.
 echo "/dev/mapper/luks-container-crypt    /mnt/luks-container-crypt    ext4    defaults    0    0" >> /etc/fstab
 echo "/dev/mapper/external-luks-drive    /mnt/luks-media-crypt/luksEXT4    ext4    rw,relatime    0    0" >> /etc/fstab

 # Try to mount all connected drives with the information in /etc/fstab.
 # LUKS-encrypted drives are already manually decrepted, so re-reading /etc/fstab should mount them.
 mount -a
That should automount the encrypted drives at boot. The only way to be sure is to test it after reboot.

Now we need to update /etc/samba/smb.conf to reflect how we want it to work.
 cat <<EOT >> /etc/samba/smb.conf
[media]
  comment = Attached USB drives
  path = /media
  guest ok = yes
  browsable = yes
  read only = no
  read list = guest nobody write
  write list = @sambashare
  force user = nobody
  force group = nogroup
  create mask = 0660
  directory mask = 0771

[secure]
  comment = Access controlled share
  path = /mnt/secure
  smb encrypt = required
  valid users = @sambashare
  guest ok = no
  browsable = yes
  writable = yes
  read only = no
  force user = nobody
  force group = sambashare
  create mask = 0660
  directory mask = 0770

[luks-container-crypt]
  comment = Luks decrypted share
  path = /mnt/luks-container-crypt
  smb encrypt = required
  valid users = @sambasecurity
  guest ok = no
  browsable = yes
  writable = yes
  read only = no
  force user = nobody
  force group = sambasecurity
  create mask = 0660
  directory mask = 0770

[luks-media-crypt]
  comment = Luks decrypted share
  path = /mnt/luks-media-crypt
  smb encrypt = required
  valid users = @sambasecurity
  guest ok = no
  browsable = yes
  writable = yes
  read only = no
  force user = nobody
  force group = sambasecurity
  create mask = 0660
  directory mask = 0770

[public]
  comment = Public share
  path = /mnt/public
  guest ok = yes
  browsable = yes
  read list = guest nobody
  write list = @users
  read only = no
  force user = nobody
  force group = nogroup
  create mask = 0660
  directory mask = 0771

EOT

 service smbd restart

EDIT: According to suggestion in posts below, it is recommended to modify the [global] section manually in /etc/samba/smb.conf to set min protocol to SMBv2.

 # Add or modify the [global] section in /etc/samba/smb.conf
 # Look for the [global] section and add the lines indicated below.
 
 # Open the file on line 22
 nano +22 /etc/samba/smb.conf

[global]

## Browsing/Identification ###

# Change this to the workgroup/NT-domain name your Samba server will part of
   workgroup = WORKGROUP

#### Samba protocol ####
#  default: min protocol = NT1 # that is protocol SMB1 vith security vulnerabil$
#  default: server min protocol = NT1
#  default: client min protocol = NT1
   server min protocol = SMB2
   client min protocol = SMB2
#  It can be good to also set the server max protocol accorting to this article.
#  max protocol     This parameter is a synonym for server max protocol.
   server max protocol = SMB3
   client max protocol = SMB3

#### Encryption ####
#  Offer samba encryption but don't enforce it
   smb encrypt = auto
#  Require samba encryption
#  smb encrypt = required
Save /etc/samba/smb.conf and restart smbd.
 service smbd restart

Update intramfs

It happened to me a couple of time that intramfs update couldn't finish. Maybe it was due to memory being full and no swap file. It took me a while to figure out how to manually trigger an update of intramfs after such a fail, so I share it here.
 update-initramfs -u

Maybe you have seen some messages about update intramfs.
We also want to update to the kernel 5.9.3.
Now that we have a working samba server, we can easily drop the upgrade files on one of the available shares.
Then we transfer the new kernel files to /boot and update according to the instruction that comes with it.

First update the whole linux system:
 apt-get update && apt-get upgrade
Re-generate the uInitrd (the kernel files vmlinuz-5.2.9-kirkwood-tld-1 and initramfs-5.2.9-kirkwood-tld-1 are already generated by dpkg before). uImage does not have to be re-generated, but it doesn't hurt to re-generate it either if you by accident copied the whole three lines in one go.
 cd /boot
 # mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-kirkwood-tld-1 -d vmlinuz-5.2.9-kirkwood-tld-1 uImage
 mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs-5.2.9-kirkwood-tld-1 -d initrd.img-5.2.9-kirkwood-tld-1 uInitrd
Transfer the new 5.9.3 kernel file (*.tar.bz2) extract the contents to /boot and install it according to the instructions that follow with the upgrade.

I find it useful to create some utilities to upgrade the kernel, clean memory and such.
Add some new utilities for future use:
 cat <<EOF > /usr/local/bin/clean-memory
#!/bin/bash
echo "before cleaning"
free -m
sync
# su
echo 3 > /proc/sys/vm/drop_caches
echo "after cleaning"
free -m
EOF
 chmod +x /usr/local/bin/clean-memory

 cat <<EOF > /usr/local/bin/justcleanmemory
#!/bin/bash
echo 3 > /proc/sys/vm/drop_caches
EOF

 chmod +x /usr/local/bin/justcleanmemory

 cat <<EOF > /usr/local/bin/upgrade-system-5_9_3
#!/bin/bash
echo "Curent kernel version"
uname -r
kernelVersion="5.9.3"
echo Using: 
echo Linux-${kernelVersion}-kirkwood-tld-1
echo vmlinuz-${kernelVersion}-kirkwood-tld-1
echo initramfs-${kernelVersion}-kirkwood-tld-1
echo initrd.img-${kernelVersion}-kirkwood-tld-1
apt-get update && apt-get -y upgrade
cd /boot
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-${kernelVersion}-kirkwood-tld-1 -d vmlinuz-${kernelVersion}-kirkwood-tld-1 uImage
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs-${kernelVersion}-kirkwood-tld-1 -d initrd.img-${kernelVersion}-kirkwood-tld-1 uInitrd
EOF
 chmod +x /usr/local/bin/upgrade-system-5_9_3

Add a cron job that helps to clean the memory every minute if you notice that your RAM is getting clogged and the system is swapping a lot more than it should.

Add another cronjob that periodically looks if you have added a new drive to a usb after you started it. Kind of handy.
 # This must be set when you are root
 apt-get install cron
 su -
 crontab -e

 # Add these lines:
 # Try a value between 80-93. This means the memory is 80-93% full. Free space is 7-20%.
 * * * * * /usr/bin/test 90 -le $(/usr/bin/awk '$1=="MemTotal:"{t=$2} $1=="MemFree:"{f=$2} END{printf "%d", (t-f)/(t/100)}' /proc/meminfo) && justcleanmemory
 * * * * * /bin/mount -a
Save.
Restart cron.
 service cron restart
Adjusting the usage of ssh is encouraged by some. Here is what to change:
 nano /etc/ssh/sshd_config
Modify these lines:
# Port 22: It's good to change this port, since a lot of brute force attacks target this port.
# Port 3311: Any available port could do, it does not have to end in **22.
# For this exampe, we will use Port 2222.
Port 2222
# Disabling Root login is as well a safer meassure.
# Use another user on the system, and then go to root with:
# su -
PermitRootLogin no

# Turn this option to 'no' to deny password based login for public
PasswordAuthentication no

# Add below content to password based login for all users part of group 'sambasecurity'
Match Group sambasecurity
        PasswordAuthentication yes
Restart ssh
 service ssh restart
Before you restart your server, make sure one of your users can use ssh!!!

The default installation of samba on Debian 10 seems to allow connections from clients that only support SMB v1 protocol. Not sure if this is "good" or "bad" considering that SMB v1 has been a security issue (samba Wannacry), but for testing with from instance a mobile operating system which often has file explorers that lack the support for SMB v2 and SMB v3, it might actually be "conventient" (=dangerous?) to allow SMB v1. One idea is to hook up Pogo v4 to it's own router without access to internet to avoid the SMB v1 risks, but when Debian needs to be updated, you may immediately face this possible issue with SMB v1 allowed (is that correct?). It seems to be neccessary to allow min protocol SMB v2 for connectivity somethimes, while others suggest min protocol and max protocol should be SMB v3 only. I have no clue at the moment about what is best for a Pogo v4. Comments on this are appreciated if anyone knows what is good for Pogo v4 and its lacking system resources.

SMB v1 question:

  • Is it advisable to set min protocol to SMB v2 on a samba server on Pogo v4?
  • Is it ok to leave the default min protocol SMB v1 as it is?


Along the testing, I also came up with difficulty to decide if I wanted to use firewall ufw or not. I opted out due to suspicion that it would slow down performance. On the other hand, firewall is kind of good to have on, allowing only ssh and samba traffic. I assume that ufw will have a negative impact on performance, eat away some valuable memory and slow things down. Am I correct?

ufw question:

  • Is it advisable or just overkill from performance point of view to use ufw with samba server on Pogo v4?

Testing a LUKS encryption, I couldn't figure out how to store the password key securely which is used to open encrypted files automatically at server boot, without having to decrypt every device manually at server start. I guess this is a difficult one to crack. It is somehow necessary to hide the key somewhere on the server itself, but it is not so difficult to figure out where the key is stored. The issue seems to be that the key has to be fetched from some secure place when needed, a place where only root has access and nobody else. Mounting the SATA on another linux system would require any local key to be stored encrypted, but then we just move the problem ... where do we store the new key we use to access the LUKS encryption key which in turn decrypt an LUKS encrypted volume? The issue is that the first encryption key that decrypt all other keys has to be supplied to the server somehow... in a secure way. How do you resolve this normally with automatic decryption? Do you have to have another server that supplies the encryption key somehow? Puzzled. The other server will have the same issue of storing their encryption keys, but at least the key is on another machine and maybe in another location.

  • How do you store the encryption key on a server in a safe way?

Testing the Samba server shows a speed of about 2.7 MBps speed. With a newer SATA drive the speed is about 8.7 MBps. Not sure why this happens, but maybe different SATA drives have different speeds. It is quicker via USB to the same drive.

vnstat question:

  • Is it smart from a performance point of view to have vnstat on a Pogo v4 to measure the traffic volume, or will it just slow things down even more?
  • Is it smart from a performance point of view to have samba with encrypted traffic, ufw and vnstat on a Pogo v4 to measure the traffic volume, or will it just slow things down beyond acceptable?


Test the server

#!/bin/bash
# -*- coding: utf-8 -*-

# Clear history
# If you add "[ \t]*" to your HISTIGNORE variable you can make any command be ignored by starting it with a space.
HISTCONTROL=ignoreboth
HISTIGNORE="&:ls:cp:mv:[bf]g:exit:pwd:clear:mount:umount:sshpass:ssh:rm:rmdir:[ \t]*"
 history -c
 unset HISTFILE
 # That disables writing the history file, while it still allows to cycle 
 # through the last commands using up/down keys.
 # Disable history
 rm ~/.bash_history
 history | tail -10

 # Dowload test pictures
 wget -c https://www.vets4pets.com/siteassets/species/cat/close-up-of-cat-looking-up.jpg -O $(xdg-user-dir PICTURES)/cat.jpg
 wget -c https://blog.healthypawspetinsurance.com/wp-content/uploads/2019/05/dog-zoomies.jpg -O $(xdg-user-dir PICTURES)/dog.jpg
 wget -c https://www.flyer.co.uk/wp-content/uploads/2019/08/human-power-flight-1-1000x657.jpg -O $(xdg-user-dir PICTURES)/airplane.jpg
 wget -c https://www.lolwot.com/wp-content/uploads/2015/06/20-amazing-adventure-toursim-destinations-you-should-consider-visiting-2.jpg -O $(xdg-user-dir PICTURES)/hangglider.jpg

 # Set the IP address of the samba server
 ipAddress='192.168.xxx.yyy'

 # Show server shares
 smbclient -L //$ipAddress -U 'guest%anonymous'

 # Test login
 userName=writesecure
 password='cat'
 share=secure
 smbclient //$ipAddress/$share -U "$userName%$password"

 userName=writesecure
 password='cat'
 share=secure
 domain='WORKGROUP'
 sendFile='cat.jpg'
 storeInDir="$userName"
 command="prompt OFF; recurse ON; mkdir $storeInDir; cd $storeInDir; lcd $(xdg-user-dir PICTURES); put $sendFile; ls; cd /; ls"
 echo "Command to run:" $command
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"
 share=public
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"

 userName=writeencrypted
 password='dog'
 share=secure
 domain='WORKGROUP'
 sendFile='dog.jpg'
 storeInDir="$userName"
 command="prompt OFF; recurse ON; mkdir $storeInDir; cd $storeInDir; lcd $(xdg-user-dir PICTURES); put $sendFile; ls; cd /; ls"
 echo "Command to run:" $command
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"
 share=public
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"

 userName=writeencrypted
 password='dog'
 share='luks-media-crypt'
 domain='WORKGROUP'
 sendFile='dog.jpg'
 storeInDir="$userName"
 command="prompt OFF; recurse ON; ls"
 echo "Command to run:" $command
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"
 share=public
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"

 userName=writepublic
 password='bird'
 share=public
 domain='WORKGROUP'
 sendFile='hangglider.jpg hangglider.jpg'
 storeInDir="$userName"
 command="prompt OFF; recurse ON; mkdir $storeInDir; cd $storeInDir; lcd $(xdg-user-dir PICTURES); put $sendFile; ls; cd /; ls"
 echo "Command to run:" $command
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"

 # Test no access
 userName=writepublic
 password='bird'
 share=secure
 domain='WORKGROUP'
 storeInDir="$userName"
 command="prompt OFF; recurse ON; ls"
 echo "Command to run:" $command
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"

 # Test no access
 userName=guest
 password='anonymous'
 share=public
 domain='WORKGROUP'
 sendFile='hangglider.jpg'
 storeInDir="$userName"
 command="prompt OFF; recurse ON; mkdir $storeInDir; cd $storeInDir; lcd $(xdg-user-dir PICTURES); put $sendFile; ls; cd /; ls"
 echo "Command to run:" $command
 smbclient //$ipAddress/$share -W $domain -U "$userName%$password" \
 -c "$command"

 # Delete test files
 rm -r /mnt/secure/writesecure
 rm -r /mnt/secure/writeencrypted

 rm -r /mnt/public/writepublic
 rm -r /mnt/public/writesecure
 rm -r /mnt/public/writeencrypted

 rm $(xdg-user-dir PICTURES)/cat.jpg
 rm $(xdg-user-dir PICTURES)/dog.jpg
 rm $(xdg-user-dir PICTURES)/airplane.jpg
 rm $(xdg-user-dir PICTURES)/hangglider.jpg

 history -c



Extra thoughts

The samba server performance has been tested with media players such as Celluloid, vlc, Kodi to stream media content of varying quality and it seems to work well without lags or issues. Not running firewall ufw. Only samba encryption of samba traffic. Wireshark reports the usage of SMBv3.

For fun I tried to install a samba printer. Samba works with CUPS, but I didn't manage to have it produce any PDF-files from spool files ... yet. However, that function is maybe obsolete these days, but I thought it could be handy for mobile operating systems that doesn't have a printer to actually create a PDF when they print and fetch it from the public share. I didn't evaulate the performance impact, but it does seem to work. It is a bit intriguing to try to make it work for testing purposes and educational purposes.



Edited 7 time(s). Last edit at 03/22/2022 02:59PM by chaos.
chaos,

That was quite a long post packed with info! I will attempt to answer your questions one by one.

Quote

I haven't figured out how to back up a running rootfs directly from Pogo v4. Sometimes I ended up with disk read errors, so I assume that the rootfs must be backed up when it is not running the linux system. I am sure there are ways to backup a running rootfs, but I don't know how to do this.

You can use rsync to do this. I have a daily cronjob to run rsync and back up my main development box (where I build kernels and u-boot that I released here) rootfs to another box as a backup server for everything important.

Once in a while, I do a test run that I restore this rootfs to a USB drive and boot this box with it.

I could create a tutorial post for this rsync backup job if requested. There are many tutorials on the Web about this subject so I did not tthink it is needed here.


Quote

What seemed for me to be an issue 3 times out of 5 was that if /etc/fstab says the rootfs is ext3 but in fact it is EXT4 formatted, the kernel will not always figure it out by itself and switch to EXT4 automatically.

That's not true. The modern kernel can figure out the file system on disk and mount it as such. I don't think /etc/fstab will change the FS type.

Quote

Does the order of the files in a linux system impact the perfomance when reading data from the disk? If files are close to each other it is said it optimizes reading speed.
Does the linux system shuffle files in an EXT4 to optimize reading speed?

Fragmentation is not an issue that I know of.

Quote

Another issue was that I used fdisk to partition the drive. Apparently, one should use gdisk instead for this. Two times the GPT got corrupt for some reason and I had to start all over. Maybe that was the reason why it would not boot an EXT4 with a setting ext3 in /etc/fstab.

If the HDD partition is smaller than 2TB, then one should use fdisk. Don't use GPT, there is no need to. fdisk should work always (providing that you wiple out the partition table first).

Quote

How can you force this Debian system on Pogo v4 to run a script on every boot, so that you can fix the issue with the SSH keys if you forget to fix the issue before you boot the first time?

This is what I planned to do but have not get to it! I will add a new step to the rootf instruciton to fix this possible issue with SSH key.

Quote

/etc/rc.local run a script at every boot? Or

Yes. /etc/rc.local always run once at boot. This is the last init script in the boot sequence. You would just created an executable script and add the execution in /etc/rc.local. You need to specify the full path. For example, I have a movelog script that will move all the logging to /tmp (bindmount /var/log to /tmp,...) to save the USB from wear and tear.

# move frequently updated log files to /tmp
/root/movelog.sh


OK, this post is long enough, I will address the rest of the questions in the next post.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Quote

A swap partition on
/dev/sda4
was set 1.5 times the RAM = 192MB = 183MiB allocated at the very end of the disk.

You don't need to use swap partition, you can use swap file. It is more flexible and the performance is pratically the same.

And use at least 4x the RAM. The old convention is 2x, but that will get you to experience OOM killer when the memory is stressed enough. With the latest kernels (that I released here) we have the BFQ IO scheduler running as default (CONFIG_IOSCHED_BFQ). So you will not see OOM, and can log in to the box to fix problems that caused the memory stress.

Quote

Change swapiness to somewhere between 5-35. This number may prove to be wrong, but it saves the disk a bit so it lasts longer than with default setting 60.

It is OK to keep it at 60. For the reason mentioned above about BFQ IO scheduler. And swap file should be on HDD/SSD (If you have no HDD in the system then you'll have no choice but using swap on USB should be avoided).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 11/11/2020 10:39PM by bodhi.
Quote

Is it advisable to set min protocol to SMB v2 on a samba server on Pogo v4?

Yes.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 11/11/2020 10:55PM by bodhi.
bodhi Wrote:
-------------------------------------------------------
>
Quote

Is it advisable to set min protocol to SMB
> v2 on a samba server on Pogo v4?
>
> Yes.


Wow, this is the whole reason I put debian onto my nsa320. The stock nas os was stuck on smb v1,

I had hooked the old nas up after having it mothballed for about 7 years because HD has failed.

Both windows and linux would not allow share mapping to my nas because of smb v1, so I searched for options and found this group.

So yes set samba min v2. I agree
Thank you for your feedback. I am editing the top post with your comments as they come in.

After testing a bit with
min server protocol = SMB2
I discovered in other post about older versions of Kodi (prior to v.18) which cannot connect to a server using SMB v2/v3. So if you happen to have an old box running Kodi v1-17 that you cannot upgrade to v18 for some technical reason (hardware limitation), you are most probably stuck and have to use SMB v1 if you think the box still does its job well enough.

Some article on internet says it could be useful to also set
[global]
...
   server max protocol = SMB3
   client max protocol = SMB3
...
in /etc/samba/smb.conf

What is your opinion on that?



Edited 1 time(s). Last edit at 11/12/2020 01:32PM by chaos.
Quote
bodhi
I could create a tutorial post for this rsync backup job if requested. There are many tutorials on the Web about this subject so I did not tthink it is needed here.

For my part (I take the liberty to assume that others might benefit from such a guide as well), I personally would greatly appreciate a guide (a script.sh file maybe to avoid errors by typing it in manually) that automates the process of rsync backup of a running rootfs partition to another location of choice, be it
  • another samba share, example //serverIPaddress/share/backup-rootfs-date.tar.bz2 or
  • local file, example /path/to/local/directory/backup-rootfs-date.tar.bz2.
chaos,

Quote

[global]
...
server max protocol = SMB3
client max protocol = SMB3
...
in /etc/samba/smb.conf

I don't use Window much anymore other than for HTPC, so my opinion is not going to be informed.

My HTPC is Win7. And the media server boxes are GoFlex Home plugs. I purposely kept these at an older Debian version (8.x) for a while so I can do some tests. I also have some boxes running Debian 9.x, for the same purpose.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
I like this thread :)
Gravelrash,

> I like this thread :)

Indeed, likewise! but I cannot yet decide which category this thread belong to in the Wiki :) there are too many different very useful info that chaos packed into a single post.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
For now it is in the Wiki thread as:



Quote
https://forum.doozan.com/read.php?2,23630,23630#msg-23630

Samba

Samba smb.conf for a simple set up
HowTo setup Samba/CIFS shares
Samba server project on POGO-V4

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
The default for new configurations of samba is to be at the 2.x branch - so this would eliminate any cleints ability to connect prior to windows2000 and to force connections of version 3.* would limit you to Windows8 and above. however - the added benefit of forcing 3.* forces all clients to communicate over an encrypted stream. so the choice as they say is yours, depending on use case


See table here for version and connectivity options supported by MS
SAMBA/CIFS version simplified

chaos Wrote:
-------------------------------------------------------
> Thank you for your feedback. I am editing the top
> post with your comments as they come in.
>
> After testing a bit with
>
> min server protocol = SMB2
>
> I discovered in other
> post
> about older versions of Kodi (prior to v.18) which
> cannot connect to a server using SMB v2/v3. So if
> you happen to have an old box running Kodi v1-17
> that you cannot upgrade to v18 for some technical
> reason (hardware limitation), you are most
> probably stuck and have to use SMB v1 if you think
> the box still does its job well enough.
>
> Some
> article
> on internet says it could be useful to also set
>
> [global]
> ...
>    server max protocol =
> SMB3
>    client max protocol =
> SMB3
> ...
>
> in /etc/samba/smb.conf
>
> What is your opinion on that?
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: