Hi!

My LG N1T1 is slow, and normally apt upgrade takes a few minutes. But with 6.17.7 "Building dependency tree" took about 10 hours to 100% but then never continued. I tried:

apt autoremove
rm -f /var/cache/apt/*.bin
rm -rf /var/lib/apt/lists/*
apt clean
apt update

But it did not help.

I also have a backup script that did not work, it stopped at cat /media/data/backup/log/Backup-mail.log | ssmtp -vvv $THEEMAIL when it was supposed to email what it has done

So I tried to go back to 6.16.5 and now it is back to normal. Info about the system (from when I log in):

Model: LG N1T1
Linux version 6.16.5-kirkwood-tld-1 (root@tldDebian) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 PREEMPT Thu Sep  4 15:49:06 PDT 2025
Debian 13.1

What could be the issue?

raffe,

> My LG N1T1 is slow, and normally apt
> upgrade takes a few minutes. But with 6.17.7
> "Building dependency tree" took about 10 hours to
> 100% but then never continued.

It might be a coincidence, but this kernel runs differently from the 6.16.5. The 6.16.5 kernel was built with preemptive scheduler (good for response time), this 6.17.7 kernel was built with non-preemptive scheduler (good for server).

Let me try apt-get upgrade on my lowest memroy and CPU box (Pogo V4) and see if I can repeat your problem.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

BTW, the reason I configured this kernel to server mode was mentioned in the release notes.

Quote
https://forum.doozan.com/read.php?2,12096

- General kernel upgrade.
- This kernel is now running with the non-preempt scheduler. This is to avoid a kernel bug that was introduced recently in mainline Linux. Also, the performance should improve a little bit for Kirkwood boxes running as servers.
- Update the DTS for Promwad Thin Client
- Update the DTS for Synology DS112v10j
- See Important Note in Step 5 about uInitrd size

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 11/11/2025 02:45PM by bodhi.

GoFlex Net

1.2GHz CPU, 128MB RAM, 1Gbs Ethernet
Linux version 6.17.7-kirkwood-tld-1 (root@tldDebian) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 Mon Nov  3 16:38:22 PST 2025
USB 2.0 rootfs

Took 41 minutes to upgrade from Debian 12.8 to 12.12. This is normal for a USB rootfs.

Pogo V4

800 MHz CPU, 128 MB RAM,  1Gbs Ethernet
Linux version 6.17.7-kirkwood-tld-1 (root@tldDebian) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 Mon Nov  3 16:38:22 PST 2025
USB 3.0 rootfs

Taking forever, but still running OK, not stuck. It seems it was spending too much time in usb_storage process. I'm not sure what's the cause.

I've aborted the upgrade and will try again with a new USB 2.0 drive.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 11/12/2025 03:02AM by bodhi.

raffe,

Pogo V4

800 MHz CPU, 128 MB RAM,  1Gbs Ethernet
Linux version 6.17.7-kirkwood-tld-1 (root@tldDebian) (gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 Mon Nov  3 16:38:22 PST 2025
USB 2.0 rootfs
2GB swapfile on HDD

Took 42 minutes to upgrade from Debian 12.8 to 12.12. This is normal for a USB rootfs.

It really does not matter about USB 2.0 or 3.0 rootfs. Just that the Pogo V4 draw a lot of power for USB 3.0 so I don't want that to be a factor. But swap space is important for these low power boxes.

The upgrade massively reads and writes to the rootfs, and with the 128MB RAM, the swap process is running all the time. So the swap file must be on the HDD/SSD for this to run smoothly.

Your LG N1T1 has the same specs as the Pogo V4. You need to have your swap file on the HDD. Do you?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi, and thanks for testing!

Yes, all is on the HDD, I do not use USB.

Is there a way to during boot change from server mode?

Or maybe some other tinkering?

raffe,

> Yes, all is on the HDD, I do not use USB.

Then there is something not optimum in your rootfs. The Pogo V4 has the same specs as your LG N1T1, and I had no problem upgrading.

> Is there a way to during boot change from server
> mode?

You don't need to run the latest kernel always. If you think one does not work well for whatever reason, then don't upgrade. Wait until the next one.

> Or maybe some other tinkering?

Check the Wiki thread to see if you have done the recommended tuning.

Quote

Memory & Swap Settings

Tuning for low RAM boxes - System and HDD performance. Also read several posts after for setups for logging to RAM.
logrotate examples
How to create and use a Swap file

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

As you see above, on my old ARM Kirkwood NAS (armel, ~106 MiB RAM) apt-get was painfully slow and sometimes got killed near the end of “Reading package lists…”. In my case it was the OOM-killer (dmesg showed “Out of memory: Killed process … apt-get” first around 50%, after some tweaking 95–97% until after even more tweaking it finally worked).

Below is what finally made apt-get usable and stable (so far)

I lowered I/O and CPU scheduling priority and disabled translation downloads and pdiffs.
This alone changed apt update from “forever” to “acceptable”.

ionice -c3 nice -n 19
or
ionice -c2 -n7 nice -n 19
or just
ionice nice -n 19

This runs the command with the lowest practical I/O and CPU scheduling priority so it doesn’t compete aggressively with other work on the machine. See https://man7.org/linux/man-pages/man1/ionice.1.html (or https://www.geeksforgeeks.org/linux-unix/ionice-command-in-linux-with-examples/ for examples)

ionice -c3: puts the process in the “idle” I/O scheduling class. Disk I/O from this process is only issued when the system is otherwise idle (i.e., it won’t push aside other disk activity). This is especially useful on slow disks/flash and on small systems where background I/O (logs, daemons, etc.) can make the machine feel frozen or if you still get OOM.

ionice -c2 -n7: runs the process in the “best-effort” I/O scheduling class, with a low priority within that class (0 = highest, 7 = lowest). In other words, the process still gets disk I/O service, but it will generally yield to other best-effort I/O (apps/daemons doing reads/writes) and won’t hog the disk as aggressively. This is often a good middle ground on slow disks: it’s usually much faster than -c3 (idle) when there’s any background I/O at all, but it’s still “polite” compared to the default best-effort priority. I use this now.

nice -n 19: sets the process to the lowest CPU priority (highest “niceness”). The kernel will prefer running other processes first if there is CPU contention. It doesn’t cap CPU usage, it just makes the process “polite” when others need CPU. Should alone be enough to stop OOM.

Together they don’t reduce apt’s total work, but they can improve system responsiveness and reduce the chance that apt triggers worst-case thrashing when other services are active.

I also tweaked apt-get command with -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0. One-time command example:

ionice -c2 -n7 nice -n 19 apt-get -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 update

Acquire::Languages "none";
Disables downloading Translation-* files (including Translation-en) that contain human-readable package descriptions. On low-RAM/slow-I/O machines this can save several MB of downloads plus lots of disk writes and parsing, making apt update much faster and less likely to thrash/OOM. Trade-off: fewer/none package descriptions in some tools, but installs work normally. For more info, see https://manpages.debian.org/testing/apt/apt.conf.5.en.html

Acquire::PDiffs "0";
Disables incremental “pdiff” updates for package indexes and forces downloading full current index files instead. This often reduces CPU/disk churn (patching/rebuilding indexes) on tiny systems, at the cost of slightly larger downloads. For more info, see https://debian-handbook.info/browse/stable/sect.apt-get.html

Dpkg::Use-Pty "0";
Prevents apt/dpkg from using a pseudo-terminal for fancy progress output. This makes output/logs (tee/redirects) cleaner and sometimes slightly reduces overhead/buffering weirdness. Trade-off: less “pretty” interactive progress. I found some info here https://askubuntu.com/questions/258219/how-do-i-make-apt-get-install-less-noisy and https://unix.stackexchange.com/questions/747344/apt-get-y-qq-install-isnt-silent-output-incoming-like-processing-triggers

If you want permanent config (so you can just run apt-get update normally):
/etc/apt/apt.conf (or /etc/apt/apt.conf.d/99lean):

APT::Install-Recommends "0";
APT::Install-Suggests "0";
Acquire::Languages "none";
Acquire::PDiffs "0";
Dpkg::Use-Pty "0";

Add/expand zram swap (really helps with ~100 MiB RAM)
I already had disk swap, but zram made the big difference. For info, see https://docs.kernel.org/admin-guide/blockdev/zram.html

First I installed it with

ionice -c2 -n7 nice -n 19 apt-get -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 update
ionice -c2 -n7 nice -n 19 apt-get -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 install -y zram-tools

I increased zram to 768 MiB and used before lz4 compression, but today lzo-rle. Important: If present, you must write 1 to /sys/block/zram0/reset before changing disksize, otherwise you get “Device or resource busy”. For info about zramctl, see https://man7.org/linux/man-pages/man8/zramctl.8.html (or https://manpages.debian.org/testing/util-linux/zramctl.8.en.html )

Example manual sequence (where /sys/block/zram0/reset is present):

swapoff /dev/zram0
echo 1 > /sys/block/zram0/reset
echo lzo-rle > /sys/block/zram0/comp_algorithm
echo $((768*1024*1024)) > /sys/block/zram0/disksize
mkswap /dev/zram0
swapon -p 100 /dev/zram0

Check:

swapon --show
zramctl
free -h

If you want it at boot (/etc/rc.local style):

modprobe zram
echo 1 > /sys/block/zram0/reset
echo lzo-rle > /sys/block/zram0/comp_algorithm
echo $((768*1024*1024)) > /sys/block/zram0/disksize
mkswap /dev/zram0
swapon -p 100 /dev/zram0

Tune swappiness (optional but helpful).
See https://wiki.debian.org/swappiness or https://docs.kernel.org/admin-guide/sysctl/vm.html for more info.
Default swappiness was too aggressive for my use case. I set it lower:

sysctl vm.swappiness=10

To make permanent, add to /etc/sysctl.conf or /etc/sysctl.d/99-sysctl.conf:

vm.swappiness=10

Keep repositories minimal
If you don’t need contrib/non-free/non-free-firmware, removing them reduces the size of Packages lists that apt must parse (less RAM + less I/O).

I also commented out all deb-src entries in /etc/apt/sources.list.

Check current sources:

grep -R --line-number -E '^(deb|deb-src)' /etc/apt/sources.list /etc/apt/sources.list.d/*.list

Just in case, check and remove foreign architectures
Remove foreign architectures you don’t need (e.g. armhf on an armel system).
First check your architecture

dpkg --print-architecture

Then check which foreign architectures you have

dpkg --print-foreign-architectures

Remove those that are not yours

dpkg --remove-architecture armhf

Clean up apt lists if you suspect corrupted/old metadata
When experimenting, I often did:

rm -rf /var/lib/apt/lists/*
apt-get clean

Stop memory-hungry services during apt (and start them again after)
On tiny RAM, background services can easily push apt over the edge. I stop them before update/upgrade and start them afterwards. I’m not using systemd, so I used “service … stop/start”. You can check what is running and using most RAM with

ps aux --sort=-%mem | head -n 15

Services that mattered for me:
glances (python, monitoring)
fail2ban (python, intrusion prevention software framework)
nscd (name service cache)
nmbd (NetBIOS name daemon)
atop / atopacct (monitoring/logging)
smbd (samba, file sharing and printing services to Windows)
ntpd (Network Time Protocol daemon)

Example sequence:

service glances stop
service fail2ban stop
service nscd stop
service nmbd stop
service atop stop
service atopacct stop
service smbd stop
service ntp stop

Run apt…

# Then:
service ntp start
service smbd start
service atopacct start
service atop start
service nmbd start
service nscd start
service fail2ban start
service glances start

If your init script complains about missing /var/lock/subsys (old sysv style), you can create it once:

mkdir -p /var/lock/subsys

So, now run apt with lower scheduling priority (nice/ionice), fewer services etc.
This didn’t solve OOM by itself, but it helped system responsiveness and reduced the chance that background I/O spikes would tip it over:

service glances stop
service fail2ban stop
service nscd stop
service nmbd stop
service atop stop
service atopacct stop
service smbd stop
service ntp stop

ionice -c2 -n7 nice -n 19 apt-get -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 update
ionice -c2 -n7 nice -n 19 apt-get -y -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 upgrade

service ntp start
service smbd start
service atopacct start
service atop start
service nmbd start
service nscd start
service fail2ban start
service glances start

Or if you have edited /etc/apt/apt.conf (or /etc/apt/apt.conf.d/99lean) as above

# Stop services

ionice -c2 -n7 nice -n 19 apt-get update
ionice -c2 -n7 nice -n 19 apt-get -y upgrade

# Start services

An apt-get update script.

Simple manual script I use (update + upgrade + cleanup)

#!/bin/sh
set -eu

echo .
echo === Stopping services ===
service glances stop || true
service fail2ban stop || true
service nscd stop || true
service nmbd stop || true
# These two do not say anything, so I force some info
if service atop stop; then echo "Stopping atop = OK"; else echo "Stopping atop = FAILED"; fi
if service atopacct stop; then echo "Stopping atopacct = OK"; else echo "Stopping atopacct = FAILED"; fi
service smbd stop || true
service ntp stop || true

echo .
echo === Running apt-get update ===
ionice -c2 -n7 nice -n 19 apt-get -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 update

echo .
echo === Running apt-get upgrade ===
ionice -c2 -n7 nice -n 19 apt-get -y -o Acquire::Languages=none -o Acquire::PDiffs=0 -o Dpkg::Use-Pty=0 upgrade | tee /root/apt-upgrade.log

echo .
echo === Running apt-get autoremove and clean ===
apt-get -y autoremove --purge | tee -a /root/apt-upgrade.log
apt-get clean

# Just in case, create folder with -p = no error if existing
mkdir -p /var/lock/subsys
echo .
echo === Starting services ===
service ntp start || true
service smbd start || true
# These two do not say anything, so I force some info
if service atopacct start; then echo "Starting atopacct = OK"; else echo "Starting atopacct = FAILED"; fi
if service atop start; then echo "Starting atop = OK"; else echo "Starting atop = FAILED"; fi
service nmbd start || true
service nscd start || true
service fail2ban start || true
service glances start || true

date >> /root/apt-upgrade.log

echo '****  CHECKING FOR UPDATE-INITRAMFS!  **********'
echo '****  This  will look for things like this:'
echo 'update-initramfs: Generating /boot/initrd.img-6.18.10-kirkwood-tld-1'
echo ' '
echo '**** IF FOUND, you will have to do this (otherwies, do nothing):'
echo ' '
echo 'cd /boot'
echo 'mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs-6.18.10-kirkwood-tld-1 -d initrd.img-6.18.10-kirkwood-tld-1 uInitrd '
echo '(check what version you have!)'
echo ' '
echo '**** Checking (if you see nothing here under, nothing is found)....'
grep update-initramfs /root/apt-upgrade.log

When you run the script, it looks like this and as you see it now takes about 5 min (before 45 min, and before that 24 hours)

root@debian:~# time ./run-apt-upgrade.sh
.
=== Stopping services ===
Stopping Glances server: glances .
Stopping Authentication failure monitor: fail2ban.
Stopping Name Service Cache Daemon: nscd.
Stopping NetBIOS name server: nmbd.
Stopping atop = OK
Stopping atopacct = OK
Stopping Samba SMB/CIFS daemon: smbd.
Stopping NTP server: ntpd.
.
=== Running apt-get update ===
Hit:1 http://deb.debian.org/debian stable InRelease
Hit:2 http://deb.debian.org/debian-security stable-security InRelease
Hit:3 http://deb.debian.org/debian stable-updates InRelease
Reading package lists... Done
.
=== Running apt-get upgrade ===
Reading package lists...
Building dependency tree...
Reading state information...
Calculating upgrade...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
.
=== Running apt-get autoremove and clean ===
Reading package lists...
Building dependency tree...
Reading state information...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
.
=== Starting services ===
Starting NTP server: ntpd2026-03-02T10:49:09 ntpd[2367]: INIT: ntpd ntpsec-1.2.3: Starting
2026-03-02T10:49:09 ntpd[2367]: INIT: Command line: /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 104:106
.
Starting Samba SMB/CIFS daemon: smbd.
Starting atopacct = OK
Starting atop = OK
Starting NetBIOS name server: nmbd.
Starting Name Service Cache Daemon: nscd.
Starting Authentication failure monitor: fail2ban.
Starting Glances server: glances .

****  CHECKING FOR UPDATE-INITRAMFS!  **********
****  This  will look for things like this:
update-initramfs: Generating /boot/initrd.img-6.18.10-kirkwood-tld-1

**** IF FOUND, you will have to do this (otherwies, do nothing):

cd /boot
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs-6.18.10-kirkwood-tld-1 -d initrd.img-6.18.10-kirkwood-tld-1 uInitrd
(check what version you have!)

**** Checking (if you see nothing here under, nothing is found)....

real    4m21.498s
user    0m52.053s
sys     0m41.530s
root@debian:~#

Summary of results

Before: apt-get update frequently got OOM-killed near 95–97% (“Reading package lists…”) and overall apt was “unusable”.

After: with bigger zram + lower swappiness + disabling translations/pdiffs + stopping a few services during apt-get update/upgrade completes reliably.

Hope this helps anyone trying to keep very old low-RAM ARM NAS/routers running apt-get on modern Debian.



Edited 6 time(s). Last edit at 03/02/2026 03:51AM by raffe.

raffe,

Swap is the most important aspect for this low RAM low CPU box. Everything else is secondary.

Like I said above,
1. If you follow the recommended tuning tutorial, and
2. if you have a swap file on the HDD,

you would have no problem running apt/dpkg to upgrade Debian and/or kernel.

The Pogo V4 plug is also a low RAM (128MB) and low CPU (800Mhz). With a swap file on the HDD, everything is running normally, no OOM.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi,

Thanks for the follow-up! I did read through the tuning tutorial and the swap-file advice you linked earlier. It definitely helped, but on my box it wasn’t quite sufficient on its own to prevent apt-get from crawling or getting OOM-killed during “Reading package lists…”.

So my long post wasn’t meant as “better than swap”, but as a collection of additional, practical ways to reduce RAM/I/O pressure around apt on extremely constrained systems, based on what I observed in dmesg/vmstat.

Concretely, the extra things that made a real difference for me were:

Reducing APT metadata work:
* Acquire::Languages "none" (skip Translation-* files)
* Acquire::PDiffs "0" (avoid pdiff patch-apply churn)
* Dpkg::Use-Pty "0" (cleaner output when logging, slightly less overhead)

Adding compressed swap on top of disk swap:
* zram (768 MiB) so short-lived memory spikes don’t push the system into OOM as easily

Temporarily freeing RAM before running apt:
* stopping a few services (glances, fail2ban, nscd, nmbd, atop/atopacct) and starting them again afterwards

Making apt “polite” about I/O/CPU while the system is under pressure:
* ionice -c3 nice -n 19, mainly to keep the box responsive and reduce worst-case thrashing when other daemons are active

With these combined (plus swap), apt-get update/upgrade is now completing reliably on my ~106 MiB system, whereas before it was frequently killed late in the run. With all above, now a apt-get update takes less than 3 minutes :-)

I absolutely agree swap is the #1 requirement, I just wanted to document the additional knobs that, in my case, were needed to make apt stable and usable on this specific setup.

Thanks again for the guidance and for maintaining these kernels/tools for old Kirkwood boxes.