Netgear Stora MS2110 smbd keeps crashing

Greetings. I have an old Netgear Stora MS2110 that I've been (mostly experimentally) been trying to get SMB2 support reliably working on, but so far with no luck.

My first attempt involved installing samba36 on the stock firmware, but it had an issue where the memory usage kept increasing as the number of files copied across grew, gradually slowing down as swap got hammered progressively harder, until eventually OOM killer would stop the service. So, I gave up on that.

My next attempt involved flashing the newest uBoot, then installing Debian Stretch and OMV4 as detailed in this thread. Unfortunately the web server for OMV seems to be too much for the Stora to properly handle, and smbd often became the very first victim of OOM killer as the web server sucked up the system resources. I tried disabling the web server, but then I kept running into an issue where the tmpfs at /run would fill up and stop smbd from working. The previously-linked thread includes a fix for this problem, but it didn't seem to work for me. It seems systemd (which OMV requires) isn't suited for the tiny memory on the Stora. So, I moved on again.

My current attempt involves the Debian 5.2.9 rootfs with samba installed via apt. This time, the smbd service doesn't appear as if it is being terminated by OOM killer. Rather, it just...stops working. It seems to still be running, but no longer accepts any data from clients (Windows reports "a device attached to the system is not functioning"). I'm assuming, though, that this is still an OOM issue. Restarting the service gets it working again, but only temporarily. No tmpfs full issues this time, it's just...boom, stops working.

I know the fact that the Stora has only 128MB of memory is a major bottleneck for reliable operation beyond bone stock functionality. Like I said, even on the severely cut-down stock firmware with a smaller memory footprint, I couldn't get samba36 to work without hitting OOM issues. But, I'm wondering if anyone here has any ideas as to how I might be able to reliably get SMB2 support working on this thing long-term. It's been retired as my primary NAS, but if I can find a way to give it a second life as a backup NAS or such, I'd greatly appreciate it.

Thank you.



Edited 1 time(s). Last edit at 03/07/2021 02:26PM by torindkflt.

Hi,

What does the syslog and smbd logs say? And what version of smbd are you using?

The default smbd that comes with debian works reliably and supports smb2.

Most kirkwoods have 128mb ram and have no issues with samba.

Forgive me, I'm still relatively inexperienced with Linux, so admittedly it is possible I'm looking in the wrong place. But, I cannot locate the syslog to post it here.

The only smbd log I can find that contains any contents is at /var/log/samba/log.smbd, and this is the only thing it currently says:

[2021/03/07 01:17:50.646169,  0] ../lib/util/become_daemon.c:138(daemon_ready)
  daemon_ready: STATUS=daemon 'smbd' finished starting up and ready to serve connections

I did notice the following when running dmesg | tail:

[ 4795.197140] [<c0766050>] (mv643xx_eth_poll) from [<c08a7758>] (net_rx_action+0x15c/0x3c4)
[ 4795.206066] [<c08a7758>] (net_rx_action) from [<c0102a24>] (__do_softirq+0x23c/0x334)
[ 4795.214642] [<c0102a24>] (__do_softirq) from [<c011e3e8>] (run_ksoftirqd+0x2c/0x44)
[ 4795.223040] [<c011e3e8>] (run_ksoftirqd) from [<c013b304>] (smpboot_thread_fn+0x254/0x268)
[ 4795.232049] [<c013b304>] (smpboot_thread_fn) from [<c013736c>] (kthread+0x118/0x12c)
[ 4795.240531] [<c013736c>] (kthread) from [<c01010e0>] (ret_from_fork+0x14/0x34)
[ 4795.248483] Exception stack(0xc789ffb0 to 0xc789fff8)
[ 4795.254260] ffa0:                                     00000000 00000000 00000000 00000000
[ 4795.263183] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 4795.272101] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000

I have been using Windows ROBOCOPY to test this by running a script that automatically backs up the contents of my main drive to the Stora. It'll run for a while, but then stop, displaying either "a device attached to the system is not functioning" or "the specified network name is no longer valid".

I installed samba from apt because it did not appear to be included with the 5.2.9 rootfs (smb.conf didn't exist, so I assumed this meant it wasn't installed). But, this could have been my inexperience with Linux again not knowing how to properly enable it. As for what version I'm running, doing smbd -V returns "Version 4.9.5-Debian".

torindkflt,

> I installed samba from apt because it did not
> appear to be included with the 5.2.9 rootfs
> (smb.conf didn't exist, so I assumed this meant it
> wasn't installed).

Correct. The rootfs is a basic rootfs so any thing you need you have to install it yourself.

Quote

Basic Debian buster Kirkwood rootfs for most Kirwood plugs:

- tarball size: 209M
- install size: 536M
- The init system used in this rootfs is sysvinit . To boot with systemd, see note 2 below.
- Installed packages: nano, avahi, ntp, busybox-syslogd (log to RAM), htop, isc-dhcp-client, dialog, bzip2, nfs server/client, iperf, ethtool, sysvinit-core, sysvinit, sysvinit-utils, u-boot-tools, and mtd-utils.
- see LED controls in /etc/rc.local, and /etc/init.d/halt
- see some useful aliases in /root/.profile
- root password: root

For Samba, see the Wiki thread:

https://forum.doozan.com/read.php?2,23630

Quote

Samba

Samba smb.conf for a simple set up
HowTo setup Samba/CIFS shares

For syslog, the rootfs has busybox-syslog logging to RAM, to read the log

logread

And Samba runs fine on all 128MB RAM NAS boxes. There should be no problem or limitation. But you will need to set up a swap file or swap partition to get the best performance.

See the Wiki thread for some tips about swap and memory:

https://forum.doozan.com/read.php?2,23630

Quote

Memory & Swap Settings

Tuning for low RAM boxes

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 03/08/2021 12:18AM by bodhi.

I got the samba share configured, my Windows 10 laptop can access it just fine, read and write to it, and so forth. So, at least as far as I can tell, that part is set up correctly. I also already set up a 512MB swap file and verified it is working (at least according to the top command), so I should be good there. Google was real helpful on figuring that part out.

Oh, and I just discovered...one of the problems I had when I was trying OMV4 on Debian Stretch has returned. After copying files to the share for a while, tmpfs /run gets full and stops smbd from working, killing the connection. It's currently set to 12MB, but when I was trying OMV4 it was set to 20MB and still would fill up. I've already commented out the 'SYSLOG_OPTS="-C128' setting of /etc/default/busybox-syslogd as suggested by the OMV4 HowTo thread, but that doesn't seem to be working, /run is still getting full. Here's what df -h says:

Filesystem      Size  Used Avail Use% Mounted on
udev             52M     0   52M   0% /dev
tmpfs            12M   12M     0 100% /run
/dev/sda1       7.9G  1.4G  6.1G  19% /
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            23M     0   23M   0% /dev/shm
tmpfs            57M     0   57M   0% /tmp
/dev/sdb1       932G   70G  862G   8% /mnt/hdd2

The logread command returns the following message:

logread: can't find syslogd buffer: No such file or directory

I'm assuming this means it's disabled.



Edited 1 time(s). Last edit at 03/08/2021 01:46AM by torindkflt.

torindkflt,


>

> logread: can't find syslogd buffer: No such file
> or directory
>

>
> I'm assuming this means it's disabled.

Yes. You have disbaled it by commenting out "SYSLOG_OPTS="-C128". The log will go to the normal location: /var/log/message. logread does not read this log. You would use cat or nano to see the content.

> tmpfs 57M 0 57M 0% /tmp

This is too much for this box to run. Look at the files on /tmp and see what's going on

ls -lart /tmp

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

/var/log/message doesn't exist either:

root@stora:/tmp# cat /var/log/message
cat: /var/log/message: No such file or directory

I can resize /tmp, what would you recommend? At the moment though, it appears to be empty, even though I have not yet rebooted the Stora since the last time /run filled up and smbd stopped working:

root@stora:~# ls -lart /tmp
total 4
drwxr-xr-x 21 root root 4096 Mar  7 02:12 ..
drwxrwxrwt  2 root root   40 Mar  7 23:14 .
root@stora:~# 

The problem is this:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs            12M   12M     0 100% /run

I'm no samba expert, but what's in /run/samba? I'm guessing there's a file in there growing without bound.

renojim Wrote:
-------------------------------------------------------
> The problem is this:

> Filesystem      Size  Used Avail Use% Mounted on
> tmpfs            12M   12M     0 100% /run
>

> I'm no samba expert, but what's in /run/samba?
> I'm guessing there's a file in there growing
> without bound.

root@stora:/run/samba# ls -al
total 11368
drwxr-xr-x  5 root root     360 Mar  7 23:59 .
drwxr-xr-x 14 root root     600 Mar  7 23:15 ..
-rw-r--r--  1 root root  281864 Mar  7 23:14 brlock.tdb
-rw-r--r--  1 root root  294912 Mar  7 23:14 gencache_notrans.tdb
-rw-r--r--  1 root root   61440 Mar  7 23:21 leases.tdb
-rw-r--r--  1 root root 4943872 Mar  7 23:59 locking.tdb
drwxr-xr-x  2 root root     140 Mar  8 18:19 msg.lock
-rw-r-----  1 root root   12288 Mar  7 23:14 names.tdb
drwxr-xr-x  3 root root      60 Mar  7 23:14 ncalrpc
drwxr-xr-x  2 root root      60 Mar  7 23:14 nmbd
-rw-r--r--  1 root root       5 Mar  7 23:14 nmbd.pid
-rw-------  1 root root    4792 Mar  7 23:14 smbXsrv_client_global.tdb
-rw-------  1 root root   32768 Mar  7 23:15 smbXsrv_open_global.tdb
-rw-------  1 root root 5955584 Mar  7 23:59 smbXsrv_session_global.tdb
-rw-------  1 root root   20480 Mar  7 23:49 smbXsrv_tcon_global.tdb
-rw-------  1 root root   20480 Mar  7 23:14 smbXsrv_version_global.tdb
-rw-r--r--  1 root root       5 Mar  7 23:14 smbd.pid
-rw-r--r--  1 root root       0 Mar  8 18:19 smbd_cleanupd.tdb
root@stora:/run/samba#

Deleting the files in here doesn't help though, df -h still shows /run as 100% full.

Do you have a log.smbd in /var/log/samba?

renojim Wrote:
-------------------------------------------------------
> Do you have a log.smbd in /var/log/samba?


Yes, this is all it currently contains:

root@stora:/var/log/samba# cat log.smbd
[2021/03/07 01:17:50.646169,  0] ../lib/util/become_daemon.c:138(daemon_ready)
  daemon_ready: STATUS=daemon 'smbd' finished starting up and ready to serve connections
root@stora:/var/log/samba#

You can see that the tdb files in /run/samba are using up all of /run. You can try to increase the size of /run using fstab, but I have no idea why your tdb files are so large.

Like I said though, deleting everything in /run/samba doesn't free up any tmpfs space, df -h still says it is full. Also, when I was trying OMV, I had it set to 20MB, which was the recommended maximum size for the Stora, and it still would fill up.

Is this a new rootfs? Or the one you had OMV on?

Either way it seems broken, have you tried creating a new one?

Brand new, in fact I even tried re-creating it from scratch since I originally started this thread, but the problem still exists.