Welcome! Log In Create A New Profile

Advanced

apt-get upgrade, systemd refuses to reload daemons

Posted by Nematocyst 
apt-get upgrade, systemd refuses to reload daemons
May 24, 2017 11:43PM
My dockstar server running debian stretch complains when doing updates. I got this a couple months ago, didn't get any useful hits on the problem, and avoided it since the server appeared to still do everything fine.

I updated just now and got several errors like this:
Warning: smbd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
Failed to reload daemon: Refusing to reload, not enough space available on /run/systemd. Currently, 10.0M are free, but a safety buffer of 16.0M is enforced.

Still, I get relatively few hits on the topic. I don't know what to do about it... it seems I need to have a bigger /run mount, but I'm not sure. Maybe these errors don't matter? They look horrible, and I even get them multiple times for the same daemons as it continues to try to restart them during apt-get upgrade.
Re: apt-get upgrade, systemd refuses to reload daemons
May 25, 2017 03:55PM
Do you have everything in a single file system? If so you are dangerously short of available disk space. You could try running:

apt-get autoremove
apt-get clean

The first command will auto remove packages that were installed to satisfy dependencies for other packages but are now no longer required. The second command deletes all .deb files from /var/cache/apt/archives and can free-up significant volume of disk space. You should also look into what is consuming the available disk space. The 'du' command can be helpful in determining what is using your disk space.



Edited 1 time(s). Last edit at 05/25/2017 03:57PM by rayknight.
Re: apt-get upgrade, systemd refuses to reload daemons
May 25, 2017 04:20PM
No, not short on disk space. I rebooted and all services appear to be working fine.

The bolded item is my rootfs. The red item is where the problem lies, I believe.

eric@deimos:~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev             59M     0   59M   0% /dev
tmpfs            13M  2.2M   10M  18% /run
/dev/sda5       585G  2.9G  553G   1% /
tmpfs            61M     0   61M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            61M     0   61M   0% /sys/fs/cgroup
ubi0:rootfs     232M  159M   73M  69% /srv/nand
/dev/sdb1       296G  252G   29G  90% /samba/server
/dev/sdb3       785G   21G  724G   3% /samba/backups
/dev/sdc1       932G  501G  432G  54% /samba/media
tmpfs            13M     0   13M   0% /run/user/1000



Edited 1 time(s). Last edit at 05/25/2017 04:22PM by Nematocyst.
Re: apt-get upgrade, systemd refuses to reload daemons
May 26, 2017 11:04AM
Yes, it is the /run mount. The reason rebooting works fine is that systemd isn't trying to reload anything, simply start the services. I can work around the problem by stopping each service and then starting it as such: [edit] 'restart' works too. the issue is 'systemctl daemon-reload' which isn't called during a reboot
[● root@deimos /home/eric] systemctl stop smbd
[● root@deimos /home/eric] systemctl start smbd
[● root@deimos /home/eric] systemctl stop nmbd
[● root@deimos /home/eric] systemctl start nmbd

That, however is a pain to do for each service after an upgrade. A better solution is by issuing the following before running upgrades:
[● root@deimos /home/eric] mount -t tmpfs tmpfs /run -o remount,size=32M,nosuid,noexec,relatime,mode=755

and then restoring the original size afterwards with:
[● root@deimos /home/eric] mount -t tmpfs tmpfs /run -o remount,size=13M,nosuid,noexec,relatime,mode=755



Edited 1 time(s). Last edit at 05/26/2017 11:26AM by Nematocyst.
Re: apt-get upgrade, systemd refuses to reload daemons
July 17, 2017 07:03AM
I got this also during dist-upgrade Jessie -> Stretch on GoFlexNet.

There's a thread about it on Debian bug forums - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=862013

One of the responders ("Michael Biebl") indicates that minimum memory for Debian now is 256MB, which I think is incorrect, the link he points to says actually a minimum of 128MB (and I checked for ARM, as this was a AMD64 link) ... Anyhow ...

A semi-permanent fix, as suggested later on in the same thread above, is to add this to /etc/fstab :
... bla bla bla ...
tmpfs /run tmpfs nosuid,noexec,size=18M,nr_inodes=4096 0 0
... bla bla bla ...

I tested below 18M and it didn't fix the problem, 18M seems to be the lowest for "systemctl daemon-reload" command to still work without complaint (and therefore presumably apt-get install, update, upgrade, remove etc)

Can anyone confirm I have all the correct options - "nosuid,noexec,size=18M,nr_inodes=4096" as that was just a copy and paste, and differs from what @Nematocyst suggests in this thread above :) ... also happy to know if there's "a better" solution ...

BR

Don

Don Charisma ... because anything is possible with Charisma

My blog - http://DonCharisma.org
Our commercial site - http://DonCharisma.com
Re: apt-get upgrade, systemd refuses to reload daemons
July 17, 2017 02:41PM
Don,

There is no hard minimum memory for Debian. It depends what you run on the system. 64MB is fine for a lot systems that run a few dedicated aps.

-bodhi
===========================
Wiki
latest Kirkwood kernel builds and rootfs
latest u-boot-kirkwood builds
latest Oxnas kernel builds and rootfs
latest u-boot-oxnas builds
latest MVEBU Armada kernel builds and rootfs
U-Boot & Kernel Booting process
bodhi's u-boot GitHub
bodhi's corner
Re: apt-get upgrade, systemd refuses to reload daemons
July 17, 2017 06:12PM
Hey bodhi ... yes, I understand, sorry I didn't mean to mislead anyone ! ... the link cited in the debian bugs site ( https://www.debian.org/releases/wheezy/amd64/ch03s04.html.en ) suggests that Debian may run in as little as 20MB in some configs ... the same page for Stretch indicates 60MB ... 128MB is their "recommended bare minimum" for both wheezy and stretch, although they'd generally prefer that we had more RAM.

BR

Don

Don Charisma ... because anything is possible with Charisma

My blog - http://DonCharisma.org
Our commercial site - http://DonCharisma.com
Re: apt-get upgrade, systemd refuses to reload daemons
July 17, 2017 07:05PM
Re: apt-get upgrade, systemd refuses to reload daemons
July 18, 2017 06:20AM
YW @bodhi :)

I think this is going to be a common problem for those upgrading Jessie -> Stretch on lower memory devices, such as some of the devices we hack/modify/improve here ...

A preferable solution would be a mod to the daemon-reload process so it doesn't have a hard limit of 16M, but I don't know how to hack that.

BR

Don Charisma ... because anything is possible with Charisma

My blog - http://DonCharisma.org
Our commercial site - http://DonCharisma.com
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: