Welcome! Log In Create A New Profile

Advanced

OOM Killer in action against squeezelite on Seagate Dockstar

Posted by psychokiller 
OOM Killer in action against squeezelite on Seagate Dockstar
August 23, 2022 03:23PM
Hi all,
I have lately migrated a few Dockstar(s) I used with Arch Linux ARM, most of them worked fine for the purpose they were intended, but one I wasn't able to migrate.
The incriminated Dockstar was just installed with Arch Linux ARM and squeezelite, few utilities (lsusb/tmux/htop) but no other relevant services running, since it was used connected to a USB DAC and as a Logitech Media Server client.
On Debian (latest 5.19.2 kernel as well as 5.13.6) squeezelite is unable to stay running.
It actually starts fine at boot with systemd, but after a few seconds OOM Killer kicks in and kill it.
I modified swappiness and min_free_kbytes as described in the Wiki, but nothing changed.
Of course the Dockstar has swap:
root@debian:~# free
               total        used        free      shared  buff/cache   available
Mem:          110484       18928        3176          52       88380       86476
Swap:        2068476        6984     2061492
and these are the details of OOM killing squeezelite from dmesg:
[  204.468376] systemd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[  204.480203] CPU: 0 PID: 1 Comm: systemd Not tainted 5.18.6-kirkwood-tld-1 #1.0 480b168af3f03711133a59c80899744b6fc1b46b
[  204.492654] Hardware name: Marvell Kirkwood (Flattened Device Tree)
[  204.500998]  unwind_backtrace from show_stack+0x10/0x14
[  204.507477]  show_stack from dump_header+0x50/0x230
[  204.513115]  dump_header from oom_kill_process+0x74/0x26c
[  204.520152]  oom_kill_process from out_of_memory+0x320/0x378
[  204.527174]  out_of_memory from __alloc_pages+0x918/0xab0
[  204.533328]  __alloc_pages from wp_page_copy+0x234/0x4b0
[  204.540498]  wp_page_copy from handle_mm_fault+0x718/0x8a0
[  204.547318]  handle_mm_fault from do_page_fault+0x2b8/0x3c4
[  204.553646]  do_page_fault from do_DataAbort+0x38/0xb4
[  204.560630]  do_DataAbort from __dabt_usr+0x48/0x60
[  204.567537] Exception stack(0x88819fb0 to 0x88819ff8)
[  204.573330] 9fa0:                                     0075f318 0075f318 0075f318 76bcbad0
[  204.584266] 9fc0: 0075f318 00002028 76bcb74c 76bcb780 00764b28 76bcb760 00761340 000003e8
[  204.593549] 9fe0: 76bcbad8 7e90c950 76aedb04 76aed98c 20000010 ffffffff
[  204.601194] Mem-Info:
[  204.605450] active_anon:0 inactive_anon:0 isolated_anon:4
                active_file:22 inactive_file:36 isolated_file:0
                unevictable:20814 dirty:0 writeback:1
                slab_reclaimable:2117 slab_unreclaimable:1482
                mapped:15865 shmem:0 pagetables:222 bounce:0
                kernel_misc_reclaimable:0
                free:1237 free_pcp:27 free_cma:0
[  204.645668] Node 0 active_anon:0kB inactive_anon:0kB active_file:88kB inactive_file:144kB unevictable:83256kB isolated(anon):16kB isolated(file):0kB mapped:63460kB dirty:0kB writeback:4kB shmem:0kB writeback_tmp:0kB kernel_stack:736kB pagetables:888kB all_unreclaimable? no
[  204.671134] Normal free:4948kB boost:0kB min:1260kB low:1572kB high:1884kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:88kB inactive_file:144kB unevictable:83256kB writepending:4kB present:131072kB managed:110484kB mlocked:83256kB bounce:0kB free_pcp:100kB local_pcp:100kB free_cma:0kB
[  204.699588] lowmem_reserve[]: 0 0
[  204.703708] Normal: 1*4kB (M) 144*8kB (UME) 99*16kB (UE) 41*32kB (UE) 14*64kB (UE) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4948kB
[  204.718854] 15938 total pagecache pages
[  204.723814] 4 pages in swap cache
[  204.727964] Swap cache stats: add 2721, delete 2717, find 100/345
[  204.736761] Free swap  = 2059444kB
[  204.740966] Total swap = 2068476kB
[  204.745225] 32768 pages RAM
[  204.748829] 0 pages HighMem/MovableOnly
[  204.753462] 5147 pages reserved
[  204.760567] Tasks state (memory values in pages):
[  204.766189] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[  204.775807] [    218]     0   218     9390      586    32768      151          -250 systemd-journal
[  204.789223] [    231]     0   231     1079      292    12288       35             0 blkmapd
[  204.799319] [    239]     0   239     4727      507    14336      145         -1000 systemd-udevd
[  204.809105] [    298]     0   298      753      290    10240       42             0 rpc.idmapd
[  204.818598] [    305]   111   305     1893      506    14336       87             0 rpcbind
[  204.828040] [    314]     0   314     8396      679    22528      231             0 dhclient
[  204.837470] [    321]   103   321     1723      456    14336       76             0 avahi-daemon
[  204.847214] [    322]   102   322     1928      504    14336       93          -900 dbus-daemon
[  204.860449] [    324]   103   324     1684      160    14336       62             0 avahi-daemon
[  204.870551] [    325]     0   325     3245      564    18432      127             0 systemd-logind
[  204.880476] [    395]     0   395     1228      320    12288       94             0 rpc.mountd
[  204.889984] [    407]     0   407     3116      952    18432      167         -1000 sshd
[  204.901589] [    430]     0   430     1103      348    12288       23             0 agetty
[  204.910918] [    433]     0   433     2978      483    20480      124             0 login
[  204.920059] [    434]     0   434      722      299    10240       19             0 syslogd
[  204.929283] [    454]     0   454     2019      341    16384       87             0 bash
[  204.939655] [    517]   104   517     2004      545    14336      122             0 ntpd
[  204.949269] [    569]     0   569    35726    20816   126976        0             0 squeezelite
[  204.958944] [    573]     0   573     2781      386    16384       49             0 systemctl
[  204.968405] [    574]     0   574     2782      230    14336       49             0 (pager)
[  204.978363] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),task=squeezelite,pid=569,uid=0
[  204.987948] Out of memory: Killed process 569 (squeezelite) total-vm:142904kB, anon-rss:19804kB, file-rss:63460kB, shmem-rss:0kB, UID:0 pgtables:124kB oom_score_adj:0

I still have no idea why squeezelite gets killed with such a large swap file. The behaviour is the same regardless of squeezelite configuration, with USB DAC attached or not.

One thing I have noticed different from Arch Linux ARM is that ALSA cannot open the mixer when alsamixer is run (sorry for the italian locale):
root@debian:~# alsamixer
cannot open mixer: File o directory non esistente

Does anybody from the community has any idea how to fix this problem and keeping squeelite running?
Seems so strange to me squeezelite requires so much memory to get killed on a Dockstar with nothing else running (please find htop output attached).

Thanx for any help anyone might offer!

Paolo.
Attachments:
open | download - Istantanea_2022-08-23_22-20-13.png (143.9 KB)
Re: OOM Killer in action against squeezelite on Seagate Dockstar
August 23, 2022 04:03PM
Paolo,

Have you tried running with sysvinit? ie. temporarily remove init=systemd parameter from the bootargs.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Re: OOM Killer in action against squeezelite on Seagate Dockstar
August 23, 2022 04:31PM
Hi bodhi!
Yes, just tried removing systemd from set_bootargs and rebooted a couple of times, behaviour is the same.

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 5.18.6-kirkwood-tld-1 (root@tldDebian) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1.0 PREEMPT Fri Jun 24 15:26:02 PDT 2022
[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=0005397f
[    0.000000] CPU: VIVT data cache, VIVT instruction cache
[    0.000000] OF: fdt: Machine model: Seagate FreeAgent Dockstar
[    0.000000] Memory policy: Data cache writeback
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x0000000007ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000007ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000007ffffff]
[    0.000000] pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 32512
[    0.000000] Kernel command line: console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
[    0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
[    0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:on, heap free:off
[    0.000000] Memory: 99524K/131072K available (11264K kernel code, 1633K rwdata, 4936K rodata, 1024K init, 300K bss, 31548K reserved, 0K cma-reserved)
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] clocksource: orion_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 9556302233 ns
[    0.000002] sched_clock: 32 bits at 200MHz, resolution 5ns, wraps every 10737418237ns
[    0.000059] Switching to timer-based delay loop, resolution 5ns
[    0.001595] Console: colour dummy device 80x30
[    0.001672] Calibrating delay loop (skipped), value calculated using timer frequency.. 400.00 BogoMIPS (lpj=2000000)
[    0.001716] pid_max: default: 32768 minimum: 301
[    0.002392] LSM: Security Framework initializing
[    0.002817] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.002868] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.004742] CPU: Testing write buffer coherency: ok
[    0.007800] cblist_init_generic: Setting adjustable number of callback queues.
[    0.007838] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.008243] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.008728] Setting up static identity map for 0x100000 - 0x100058
[    0.009056] mvebu-soc-id: MVEBU SoC ID=0x6281, Rev=0x2
[    0.009477] rcu: Hierarchical SRCU implementation.
[    0.011318] devtmpfs: initialized
[    0.018045] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.018101] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.056456] prandom32: self test passed (less than 6 bits correlated)
[    0.056503] prandom: seed boundary self test passed
[    0.060810] prandom: 100 self tests passed
[    0.060841] pinctrl core: initialized pinctrl subsystem
[    0.063419] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.064357] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.065697] audit: initializing netlink subsys (disabled)
[    0.067612] thermal_sys: Registered thermal governor 'step_wise'
[    0.068097] audit: type=2000 audit(0.060:1): state=initialized audit_enabled=0 res=1
[    0.068263] cpuidle: using governor ladder
[    0.068361] cpuidle: using governor menu
[    0.068909] Feroceon L2: Enabling L2
[    0.068965] Feroceon L2: Cache support initialised.
[    0.078752] No ATAGs?
[    0.260126] raid6: int32x8  gen()   108 MB/s
[    0.430336] raid6: int32x4  gen()    95 MB/s
[    0.600147] raid6: int32x2  gen()   138 MB/s
[    0.770513] raid6: int32x1  gen()    99 MB/s
[    0.770537] raid6: using algorithm int32x2 gen() 138 MB/s
[    0.940089] raid6: .... xor() 81 MB/s, rmw enabled
[    0.940109] raid6: using intx1 recovery algorithm
[    0.940939] SCSI subsystem initialized
[    0.941273] libata version 3.00 loaded.
[    0.941641] usbcore: registered new interface driver usbfs
[    0.941724] usbcore: registered new interface driver hub
[    0.941787] usbcore: registered new device driver usb
[    0.942079] pps_core: LinuxPPS API ver. 1 registered
[    0.942097] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.942134] PTP clock support registered
[    0.943979] vgaarb: loaded
[    0.944755] clocksource: Switched to clocksource orion_clocksource
[    1.082635] VFS: Disk quotas dquot_6.6.0
[    1.082757] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    1.115512] NET: Registered PF_INET protocol family
[    1.115828] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    1.116938] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear)
[    1.117193] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    1.117226] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    1.117261] TCP bind hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    1.117291] TCP: Hash tables configured (established 1024 bind 1024)
[    1.117607] MPTCP token hash table entries: 512 (order: 0, 6144 bytes, linear)
[    1.117710] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    1.117752] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    1.118051] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    1.119100] RPC: Registered named UNIX socket transport module.
[    1.119127] RPC: Registered udp transport module.
[    1.119140] RPC: Registered tcp transport module.
[    1.119153] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.119173] PCI: CLS 0 bytes, default 32
[    1.119360] NetWinder Floating Point Emulator V0.97 (double precision)
[    1.120872] Trying to unpack rootfs image as initramfs...
[    2.959183] Freeing initrd memory: 9936K
[    3.784113] Initialise system trusted keyrings
[    3.784286] Key type blacklist registered
[    3.784675] workingset: timestamp_bits=30 max_order=15 bucket_order=0
[    3.784902] zbud: loaded
[    3.787294] NFS: Registering the id_resolver key type
[    3.787355] Key type id_resolver registered
[    3.787373] Key type id_legacy registered
[    3.787441] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    3.787469] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    3.787486] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[    3.787923] jffs2: version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
[    3.788476] fuse: init (API version 7.36)
[    3.788990] orangefs_debugfs_init: called with debug mask: :none: :0:
[    3.789250] orangefs_init: module version upstream loaded
[    3.789272] SGI XFS with ACLs, security attributes, realtime, scrub, quota, no debug enabled
[    3.888594] xor: measuring software checksum speed
[    3.897532]    arm4regs        :  1112 MB/sec
[    3.912301]    8regs           :   667 MB/sec
[    3.922721]    32regs          :   947 MB/sec
[    3.922739] xor: using function: arm4regs (1112 MB/sec)
[    3.922761] async_tx: api initialized (async)
[    3.922782] Key type asymmetric registered
[    3.922796] Asymmetric key parser 'x509' registered
[    3.960828] alg: self-tests for CTR-KDF (hmac(sha256)) passed
[    3.961088] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[    3.961258] io scheduler bfq registered
[    4.150871] kirkwood-pinctrl f1010000.pin-controller: registered pinctrl driver
[    4.153579] mv_xor f1060800.xor: Marvell shared XOR driver
[    4.215876] mv_xor f1060800.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[    4.219873] mv_xor f1060900.xor: Marvell shared XOR driver
[    4.275957] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[    4.280227] Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
[    4.281855] printk: console [ttyS0] disabled
[    4.281964] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 25, base_baud = 12500000) is a 16550A
[    5.126757] printk: console [ttyS0] enabled
[    5.142080] loop: module loaded
[    5.147248] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0xda
[    5.154349] nand: Micron MT29F2G08AAD
[    5.158763] nand: 256 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
[    5.167126] Scanning device for bad blocks
[    5.329391] 4 cmdlinepart partitions found on MTD device orion_nand
[    5.336417] Creating 4 MTD partitions on "orion_nand":
[    5.342287] 0x000000000000-0x000000100000 : "u-boot"
[    5.348653] 0x000000100000-0x000000500000 : "uImage"
[    5.355003] 0x000000500000-0x000002500000 : "rootfs"
[    5.361380] 0x000002500000-0x000010000000 : "data"
[    5.369040] wireguard: WireGuard 1.0.0 loaded. See www.wireguard.com for information.
[    5.377650] wireguard: Copyright (C) 2015-2019 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
[    5.391433] [Firmware Warn]: /ocp@f1000000/mdio-bus@72004/ethernet-phy@0: Whitelisted compatible string. Please remove
[    5.418473] mv643xx_eth: MV-643xx 10/100/1000 ethernet driver version 1.4
[    6.186002] mv643xx_eth_port mv643xx_eth_port.0 eth0: port 0 with MAC address 00:10:75:1a:20:50
[    6.195706] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    6.203003] ehci-pci: EHCI PCI platform driver
[    6.208261] ehci-orion: EHCI orion driver
[    6.213208] orion-ehci f1050000.ehci: EHCI Host Controller
[    6.219527] orion-ehci f1050000.ehci: new USB bus registered, assigned bus number 1
[    6.228190] orion-ehci f1050000.ehci: irq 29, io mem 0xf1050000
[    6.264824] orion-ehci f1050000.ehci: USB 2.0 started, EHCI 1.00
[    6.271893] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.18
[    6.280942] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    6.288929] usb usb1: Product: EHCI Host Controller
[    6.294534] usb usb1: Manufacturer: Linux 5.18.6-kirkwood-tld-1 ehci_hcd
[    6.301997] usb usb1: SerialNumber: f1050000.ehci
[    6.308375] hub 1-0:1.0: USB hub found
[    6.312931] hub 1-0:1.0: 1 port detected
[    6.318501] usbcore: registered new interface driver usb-storage
[    6.325805] mousedev: PS/2 mouse device common for all mice
[    6.604793] usb 1-1: new high-speed USB device number 2 using orion-ehci
[    6.806572] usb 1-1: New USB device found, idVendor=05e3, idProduct=0608, bcdDevice= 6.03
[    6.815540] usb 1-1: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[    6.823443] usb 1-1: Product: USB2.0 Hub
[    6.829080] hub 1-1:1.0: USB hub found
[    6.833952] hub 1-1:1.0: 4 ports detected
[    7.154916] usb 1-1.2: new high-speed USB device number 3 using orion-ehci
[    7.306571] usb 1-1.2: New USB device found, idVendor=18a5, idProduct=0250, bcdDevice=11.00
[    7.315719] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    7.323798] usb 1-1.2: Product: STORE N GO
[    7.328640] usb 1-1.2: Manufacturer: Verbatim
[    7.333722] usb 1-1.2: SerialNumber: 21061556630118
[    7.340284] usb-storage 1-1.2:1.0: USB Mass Storage device detected
[    7.348051] scsi host0: usb-storage 1-1.2:1.0
[    7.364786] rtc-mv f1010300.rtc: internal RTC not ticking
[    7.371140] i2c_dev: i2c /dev entries driver
[    7.377011] device-mapper: uevent: version 1.0.3
[    7.382715] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com
[    7.392789] device-mapper: multipath round-robin: version 1.2.0 loaded
[    7.400142] device-mapper: multipath queue-length: version 0.2.0 loaded
[    7.407544] device-mapper: multipath service-time: version 0.3.0 loaded
[    7.415039] device-mapper: dm-log-userspace: version 1.3.0 loaded
[    7.421880] device-mapper: raid: Loading target version 1.15.1
[    7.430301] hid: raw HID events driver (C) Jiri Kosina
[    7.436801] drop_monitor: Initializing network drop monitor service
[    7.444441] NET: Registered PF_INET6 protocol family
[    7.452141] Segment Routing with IPv6
[    7.456637] usb 1-1.3: new high-speed USB device number 4 using orion-ehci
[    7.464321] RPL Segment Routing with IPv6
[    7.469244] In-situ OAM (IOAM) with IPv6
[    7.474036] NET: Registered PF_PACKET protocol family
[    7.480240] Key type dns_resolver registered
[    7.485947] registered taskstats version 1
[    7.490823] Loading compiled-in X.509 certificates
[    7.497798] zswap: loaded using pool lzo/zbud
[    7.503651] Key type ._fscrypt registered
[    7.508520] Key type .fscrypt registered
[    7.513170] Key type fscrypt-provisioning registered
[    7.525140] Key type big_key registered
[    7.561929] Key type encrypted registered
[    7.616082] usb 1-1.3: New USB device found, idVendor=1e3d, idProduct=2092, bcdDevice= 1.00
[    7.633534] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    7.646880] usb 1-1.3: Product: USB 2.0         
[    7.652257] usb 1-1.3: Manufacturer: CHIPSBNK
[    7.665883] usb 1-1.3: SerialNumber: 100919000F9B3D06
[    7.674957] usb-storage 1-1.3:1.0: USB Mass Storage device detected
[    7.685827] scsi host1: usb-storage 1-1.3:1.0
[    7.710166] "cryptomgr_test" (107) uses obsolete ecb(arc4) skcipher
[    8.583411] scsi 0:0:0:0: Direct-Access     Verbatim STORE N GO       1100 PQ: 0 ANSI: 6
[    8.606612] sd 0:0:0:0: [sda] 61440000 512-byte logical blocks: (31.5 GB/29.3 GiB)
[    8.625232] sd 0:0:0:0: [sda] Write Protect is off
[    8.630784] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[    8.634809] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    8.660612]  sda: sda1
[    8.677348] sd 0:0:0:0: [sda] Attached SCSI removable disk
[    8.725766] scsi 1:0:0:0: Direct-Access     CHIPSBNK USB 2.0          5.00 PQ: 0 ANSI: 2
[    8.746363] sd 1:0:0:0: [sdb] 4139008 512-byte logical blocks: (2.12 GB/1.97 GiB)
[    8.764837] sd 1:0:0:0: [sdb] Write Protect is off
[    8.770395] sd 1:0:0:0: [sdb] Mode Sense: 0b 00 00 08
[    8.774806] sd 1:0:0:0: [sdb] No Caching mode page found
[    8.780878] sd 1:0:0:0: [sdb] Assuming drive cache: write through
[    8.809211]  sdb: sdb1
[    8.827105] sd 1:0:0:0: [sdb] Attached SCSI removable disk
[   12.222699] Freeing unused kernel image (initmem) memory: 1024K
[   12.245607] Checked W+X mappings: passed, no W+X pages found
[   12.252016] Run /init as init process
[   12.256485]   with arguments:
[   12.256499]     /init
[   12.256507]   with environment:
[   12.256515]     HOME=/
[   12.256523]     TERM=linux
[   13.341397] usbcore: registered new interface driver uas
[   23.999454] PM: Image not found (code -22)
[   24.262399] EXT4-fs (sda1): mounted filesystem with ordered data mode. Quota mode: none.
[   27.349405] orion_wdt: Initial timeout 21 sec
[   27.907277] marvell-cesa f1030000.crypto: CESA device successfully registered
[   28.003804] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   28.069229] sd 1:0:0:0: Attached scsi generic sg1 type 0
[   28.213624] random: crng init done
[   29.226984] Adding 2068476k swap on /dev/sdb1.  Priority:-2 extents:1 across:2068476k FS
[   29.558314] EXT4-fs (sda1): re-mounted. Quota mode: none.
[   36.503168] mv643xx_eth_port mv643xx_eth_port.0 eth0: link up, 1000 Mb/s, full duplex, flow control disabled
[   36.513832] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   51.742974] bash invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
[   51.753381] CPU: 0 PID: 1259 Comm: bash Not tainted 5.18.6-kirkwood-tld-1 #1.0 480b168af3f03711133a59c80899744b6fc1b46b
[   51.769116] Hardware name: Marvell Kirkwood (Flattened Device Tree)
[   51.776313]  unwind_backtrace from show_stack+0x10/0x14
[   51.784634]  show_stack from dump_header+0x50/0x230
[   51.791097]  dump_header from oom_kill_process+0x74/0x26c
[   51.797420]  oom_kill_process from out_of_memory+0x320/0x378
[   51.805187]  out_of_memory from __alloc_pages+0x918/0xab0
[   51.811473]  __alloc_pages from __read_swap_cache_async+0xd4/0x1bc
[   51.818674]  __read_swap_cache_async from read_swap_cache_async+0x20/0x64
[   51.826321]  read_swap_cache_async from swap_cluster_readahead+0x120/0x1d4
[   51.834018]  swap_cluster_readahead from do_swap_page+0x464/0x594
[   51.843443]  do_swap_page from handle_mm_fault+0x21c/0x8a0
[   51.850047]  handle_mm_fault from do_page_fault+0x2b8/0x3c4
[   51.856555]  do_page_fault from do_DataAbort+0x38/0xb4
[   51.862553]  do_DataAbort from __dabt_usr+0x48/0x60
[   51.875493] Exception stack(0x8966dfb0 to 0x8966dff8)
[   51.881466] dfa0:                                     76dd5e70 004687cc 00000000 00000000
[   51.890509] dfc0: 76f968f0 76dd5e70 0d696914 00000002 00455b90 00000001 76f968f0 00000001
[   51.904820] dfe0: 00000002 7ec19a18 76f77090 76f77004 60000010 ffffffff
[   51.929580] Mem-Info:
[   51.943720] active_anon:0 inactive_anon:2 isolated_anon:0
                active_file:29 inactive_file:36 isolated_file:0
                unevictable:22271 dirty:0 writeback:0
                slab_reclaimable:1931 slab_unreclaimable:1177
                mapped:15946 shmem:0 pagetables:189 bounce:0
                kernel_misc_reclaimable:0
                free:430 free_pcp:27 free_cma:0
[   51.985624] Node 0 active_anon:0kB inactive_anon:12kB active_file:112kB inactive_file:116kB unevictable:89084kB isolated(anon):12kB isolated(file):0kB mapped:63840kB dirty:0kB writeback:4kB shmem:0kB writeback_tmp:0kB kernel_stack:664kB pagetables:756kB all_unreclaimable? no
[   52.010994] Normal free:1748kB boost:4096kB min:5356kB low:5668kB high:5980kB reserved_highatomic:0KB active_anon:0kB inactive_anon:12kB active_file:112kB inactive_file:116kB unevictable:89084kB writepending:4kB present:131072kB managed:110484kB mlocked:89084kB bounce:0kB free_pcp:96kB local_pcp:96kB free_cma:0kB
[   52.039963] lowmem_reserve[]: 0 0
[   52.044098] Normal: 31*4kB (UME) 29*8kB (UE) 33*16kB (UE) 13*32kB (U) 7*64kB (U) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1748kB
[   52.057652] 16006 total pagecache pages
[   52.064814] 4 pages in swap cache
[   52.068909] Swap cache stats: add 1909, delete 1905, find 175/366
[   52.075933] Free swap  = 2062284kB
[   52.080115] Total swap = 2068476kB
[   52.094970] 32768 pages RAM
[   52.106848] 0 pages HighMem/MovableOnly
[   52.127731] 5147 pages reserved
[   52.143123] Tasks state (memory values in pages):
[   52.165607] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[   52.200737] [    353]     0   353     4727      478    16384      143             0 systemd-udevd
[   52.210498] [    937]     0   937     8396      710    20480      229             0 dhclient
[   52.220121] [   1015]   111  1015     1893      322    14336       87             0 rpcbind
[   52.229361] [   1029]   105  1029     1143      324    12288       71             0 rpc.statd
[   52.238753] [   1041]     0  1041      753      288    10240       42             0 rpc.idmapd
[   52.254549] [   1098]     0  1098      722      296    10240       19             0 syslogd
[   52.263764] [   1139]     0  1139      690      276    10240       24             0 klogd
[   52.272922] [   1147]   102  1147     1928      329    14336       90             0 dbus-daemon
[   52.282474] [   1158]   103  1158     1710      432    14336       77             0 avahi-daemon
[   52.292443] [   1159]   103  1159     1684      186    14336       63             0 avahi-daemon
[   52.306106] [   1183]   104  1183     2004      540    14336      122             0 ntpd
[   52.315198] [   1204]     0  1204    35693    22344   131072        0             0 squeezelite
[   52.324745] [   1214]     0  1214     3116      410    18432      167         -1000 sshd
[   52.333797] [   1239]     0  1239     1816      474    14336      123             0 login
[   52.342822] [   1241]     0  1241      480      285     8192       25             0 dhclient-script
[   52.352706] [   1246]     0  1246      480      290    10240       21             0 invoke-rc.d
[   52.362928] [   1259]     0  1259     1953      243    14336       26             0 bash
[   52.371907] [   1261]     0  1261      480      117    10240       21             0 invoke-rc.d
[   52.386947] [   1262]     0  1262      761      325    10240       27             0 ls
[   52.395781] [   1263]     0  1263      532      245     8192       18             0 xargs
[   52.404993] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),task=squeezelite,pid=1204,uid=0
[   52.414609] Out of memory: Killed process 1204 (squeezelite) total-vm:142772kB, anon-rss:25304kB, file-rss:64072kB, shmem-rss:0kB, UID:0 pgtables:128kB oom_score_adj:0

By observing with htop I see on debian squeezelite allocates up to 80MB reserved memory (20MB on Arch Linux ARM), then gets killed. The virtual memory amount is just a bit higher (139 MB compared to 108MB on ARCH Linux ARM), while shared memory rise up to more than 60MB (13MB on Arch Linux ARM).
Arch Linux ARM is using kernel 4.4:
[root@squeezedock ~]# uname -a
Linux squeezedock 4.4.271-1-ARCH #1 PREEMPT Sun Jun 6 01:22:10 UTC 2021 armv5tel GNU/Linux
while Debian uses 5.18.6 (not 5.19.2 as stated above):
root@debian:~# uname -a
Linux debian 5.18.6-kirkwood-tld-1 #1.0 PREEMPT Fri Jun 24 15:26:02 PDT 2022 armv5tel GNU/Linux

Any other help will be welcome.

Thanx, Paolo.
Re: OOM Killer in action against squeezelite on Seagate Dockstar
August 23, 2022 05:00PM
Additional information...
Arch Linux ARM squeezelite version is:
Squeezelite v1.9.8-1344, Copyright 2012-2015 Adrian Smith, 2015-2021 Ralph Irving. See -t for license terms
while on Debian it is:
Squeezelite v1.9.8-1317, Copyright 2012-2015 Adrian Smith, 2015-2021 Ralph Irving. See -t for license terms
Re: OOM Killer in action against squeezelite on Seagate Dockstar
August 23, 2022 05:16PM
Hi Paolo,

Unfortunately I have not used or looked at squeezelite for many years, so don't have better suggestion other than some observation.

The current Debian kernel 5.18.6-kirkwood-tld-1 is configured with

CONFIG_IOSCHED_BFQ=y
CONFIG_BFQ_GROUP_IOSCHED=y

BFQ is a remedy for the OOM killer stepping in when there are large swap space. So if squeezelite is not triggering OOM by itself, then I would think you'll see a lot of disk thrashing (but still be able to log in and look at the system).

Perhaps later version of Debian squeezelite had some changes in behavior and memory footprint (80MB is too much for 128MB RAM). That Arch Linux ARM is using kernel 4.4, and took only 20MB. And users here in this forum haved used squeezelite since Debian lenny, stretch in the Dockstar without problem. Lately nobody has reported how it behaves in jessie, buster or bullseye.

I'm wondering how much memory it is using now with the latest ALARM rootfs for ARMV5? or if that Arch version disabled SQL (IIRC known to cost a lot of memory in squeezelite)?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Re: OOM Killer in action against squeezelite on Seagate Dockstar
August 23, 2022 05:49PM
Paolo,

Look like we were posting at the same time, please see above!

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Re: OOM Killer in action against squeezelite on Seagate Dockstar
August 24, 2022 12:35PM
Hi bodhi,
I am not sure I am following you.

Quote
bodhi
BFQ is a remedy for the OOM killer stepping in when there are large swap space. So if squeezelite is not triggering OOM by itself, then I would think you'll see a lot of disk thrashing (but still be able to log in and look at the system).
BFQ should avoid OOM kiking in if I understand properly, and you suggest to monitor swap page usage: no swap space is used while squeezelite is started. neither htop or vmstat show any increase in swap use and swap is mostly unused.

Squeezelite starts at boot, either by SysV init or by systemd: in both cases I can easily log on (system is not hang or frozen) but I see squeezelite has been killed. If I restart it manually the system freezes for a really brief moment (one or two seconds) then it gets killed by OOM.

Quote
bodhi
Perhaps later version of Debian squeezelite had some changes in behavior and memory footprint (80MB is too much for 128MB RAM). That Arch Linux ARM is using kernel 4.4, and took only 20MB. And users here in this forum haved used squeezelite since Debian lenny, stretch in the Dockstar without problem. Lately nobody has reported how it behaves in jessie, buster or bullseye.
It actually seems the squeezelite version in the Debian repository is older than the one I installed in Arch Linux ARM right before support for armv5t was removed (in february or so), so maybe it was the Arch Linux that was optimized. Will try to investigate this...

Quote
bodhi
I'm wondering how much memory it is using now with the latest ALARM rootfs for ARMV5? or if that Arch version disabled SQL (IIRC known to cost a lot of memory in squeezelite)?
Hard to know how much memory squeezelite would use with latest ALARM, since it is impossible to get an image now that support for armv5t is gone. My other Dockstar was updated at the end of January, a few weeks before support was removed, so I think it has the latest ALARM kernel and squeezelite version.
Not sure I understand with SQL, but if you mean a builtin squeezelite option, these is what both version reports, and none mention SQL support in build options.

squeezelite -? from Arch Linux ARM
Squeezelite v1.9.8-1344, Copyright 2012-2015 Adrian Smith, 2015-2021 Ralph Irving. See -t for license terms
Usage: squeezelite [options]
  -s <server>[:<port>]  Connect to specified server, otherwise uses autodiscovery to find server
  -o <output device>    Specify output device, default "default", - = output to stdout
  -l                    List output devices
  -a <b>:<p>:<f>:<m>    Specify ALSA params to open output device, b = buffer time in ms or size in bytes, p = period count or size in bytes, f sample format (16|24|24_3|32), m = use mmap (0|1)
  -a <f>                Specify sample format (16|24|32) of output file when using -o - to output samples to stdout (interleaved little endian only)
  -b <stream>:<output>  Specify internal Stream and Output buffer sizes in Kbytes
  -c <codec1>,<codec2>  Restrict codecs to those specified, otherwise load all available codecs; known codecs: flac,pcm,mp3,ogg,aac,wma,alac,dsd (mad,mpg for specific mp3 codec)
                        Codecs reported to LMS in order listed, allowing codec priority refinement.
  -C <timeout>          Close output device when idle after timeout seconds, default is to keep it open while player is 'on'
  -d <log>=<level>      Set logging level, logs: all|slimproto|stream|decode|output, level: info|debug|sdebug
  -e <codec1>,<codec2>  Explicitly exclude native support of one or more codecs; known codecs: flac,pcm,mp3,ogg,aac,wma,alac,dsd (mad,mpg for specific mp3 codec)
  -f <logfile>          Write debug to logfile
  -m <mac addr>         Set mac address, format: ab:cd:ef:12:34:56
  -M <modelname>        Set the squeezelite player model name sent to the server (default: SqueezeLite)
  -n <name>             Set the player name
  -N <filename>         Store player name in filename to allow server defined name changes to be shared between servers (not supported with -n)
  -W                    Read wave and aiff format from header, ignore server parameters
  -p <priority>         Set real time priority of output thread (1-99)
  -P <filename>         Store the process id (PID) in filename
  -r <rates>[:<delay>]  Sample rates supported, allows output to be off when squeezelite is started; rates = <maxrate>|<minrate>-<maxrate>|<rate1>,<rate2>,<rate3>; delay = optional delay switching rates in ms
  -R -u [params]        Resample, params = <recipe>:<flags>:<attenuation>:<precision>:<passband_end>:<stopband_start>:<phase_response>,
                         recipe = (v|h|m|l|q)(L|I|M)(s) [E|X], E = exception - resample only if native rate not supported, X = async - resample to max rate for device, otherwise to max sync rate
                         flags = num in hex,
                         attenuation = attenuation in dB to apply (default is -1db if not explicitly set),
                         precision = number of bits precision (NB. HQ = 20. VHQ = 28),
                         passband_end = number in percent (0dB pt. bandwidth to preserve. nyquist = 100%),
                         stopband_start = number in percent (Aliasing/imaging control. > passband_end),
                         phase_response = 0-100 (0 = minimum / 50 = linear / 100 = maximum)
  -D [delay][:format]   Output device supports DSD, delay = optional delay switching between PCM and DSD in ms
                         format = dop (default if not specified), u8, u16le, u16be, u32le or u32be.
  -v                    Visualizer support
  -O <mixer device>     Specify mixer device, defaults to 'output device'
  -L                    List volume controls for output device
  -U <control>          Unmute ALSA control and set to full volume (not supported with -V)
  -V <control>          Use ALSA control for volume adjustment, otherwise use software volume adjustment
  -X                    Use linear volume adjustments instead of in terms of dB (only for hardware volume control)
  -z                    Daemonize
  -Z <rate>             Report rate to server in helo as the maximum sample rate we can support
  -t                    License terms
  -?                    Display this help text

Build options: LINUX ALSA EVENTFD RESAMPLE FFMPEG VISEXPORT DSD LINKALL

squeezelite -? from Debian
Squeezelite v1.9.8-1317, Copyright 2012-2015 Adrian Smith, 2015-2021 Ralph Irving. See -t for license terms
Usage: squeezelite [options]
  -s <server>[:<port>]  Connect to specified server, otherwise uses autodiscovery to find server
  -o <output device>    Specify output device, default "default", - = output to stdout
  -l                    List output devices
  -a <b>:<p>:<f>:<m>    Specify ALSA params to open output device, b = buffer time in ms or size in bytes, p = period count or size in bytes, f sample format (16|24|24_3|32), m = use mmap (0|1)
  -a <f>                Specify sample format (16|24|32) of output file when using -o - to output samples to stdout (interleaved little endian only)
  -b <stream>:<output>  Specify internal Stream and Output buffer sizes in Kbytes
  -c <codec1>,<codec2>  Restrict codecs to those specified, otherwise load all available codecs; known codecs: flac,pcm,mp3,ogg,aac,wma,alac,dsd (mad,mpg for specific mp3 codec)
                        Codecs reported to LMS in order listed, allowing codec priority refinement.
  -C <timeout>          Close output device when idle after timeout seconds, default is to keep it open while player is 'on'
  -d <log>=<level>      Set logging level, logs: all|slimproto|stream|decode|output|ir, level: info|debug|sdebug
  -e <codec1>,<codec2>  Explicitly exclude native support of one or more codecs; known codecs: flac,pcm,mp3,ogg,aac,wma,alac,dsd (mad,mpg for specific mp3 codec)
  -f <logfile>          Write debug to logfile
  -i [<filename>]       Enable lirc remote control support (lirc config file ~/.lircrc used if filename not specified)
  -m <mac addr>         Set mac address, format: ab:cd:ef:12:34:56
  -M <modelname>        Set the squeezelite player model name sent to the server (default: SqueezeLite)
  -n <name>             Set the player name
  -N <filename>         Store player name in filename to allow server defined name changes to be shared between servers (not supported with -n)
  -W                    Read wave and aiff format from header, ignore server parameters
  -p <priority>         Set real time priority of output thread (1-99)
  -P <filename>         Store the process id (PID) in filename
  -r <rates>[:<delay>]  Sample rates supported, allows output to be off when squeezelite is started; rates = <maxrate>|<minrate>-<maxrate>|<rate1>,<rate2>,<rate3>; delay = optional delay switching rates in ms
  -R -u [params]        Resample, params = <recipe>:<flags>:<attenuation>:<precision>:<passband_end>:<stopband_start>:<phase_response>,
                         recipe = (v|h|m|l|q)(L|I|M)(s) [E|X], E = exception - resample only if native rate not supported, X = async - resample to max rate for device, otherwise to max sync rate
                         flags = num in hex,
                         attenuation = attenuation in dB to apply (default is -1db if not explicitly set),
                         precision = number of bits precision (NB. HQ = 20. VHQ = 28),
                         passband_end = number in percent (0dB pt. bandwidth to preserve. nyquist = 100%),
                         stopband_start = number in percent (Aliasing/imaging control. > passband_end),
                         phase_response = 0-100 (0 = minimum / 50 = linear / 100 = maximum)
  -D [delay][:format]   Output device supports DSD, delay = optional delay switching between PCM and DSD in ms
                         format = dop (default if not specified), u8, u16le, u16be, u32le or u32be.
  -v                    Visualizer support
  -O <mixer device>     Specify mixer device, defaults to 'output device'
  -L                    List volume controls for output device
  -U <control>          Unmute ALSA control and set to full volume (not supported with -V)
  -V <control>          Use ALSA control for volume adjustment, otherwise use software volume adjustment
  -X                    Use linear volume adjustments instead of in terms of dB (only for hardware volume control)
  -z                    Daemonize
  -Z <rate>             Report rate to server in helo as the maximum sample rate we can support
  -t                    License terms
  -?                    Display this help text

Build options: LINUX ALSA EVENTFD RESAMPLE FFMPEG VISEXPORT IR DSD SSL LINKALL


Seems Debian squeezelite has build options enables for IR and SSL: no idea those would increase memory footprint so much...

My investigation continues.

Again, if anyone has any suggestion or has succeeded in running squeezelite with latest Debian on a kirkwood device with 128MB, please report.

TIA, Paolo.
Re: OOM Killer in action against squeezelite on Seagate Dockstar
September 04, 2022 05:19PM
Just an update from myself to those who might read this thread in the future...

I tried to recompile squeezelite from sources with
apt source
and removing for support for IR and SSL, but nothing changed.

I then cloned from Github the latest version (newer than the Arch Linux ARM one) and compiled it with the same options of the ALARM version: memory footprint is completely different.

I even tried to modify the stream and output buffer size reducing them to a ridiculosly small amount, but nothing changed...

I am totally puzzled. Squeezelite log does not show anything strange before the process gets killed...

I think I will stop trying to run Squeezelite on Debian Bullseye on the Dockstar.

Paolo
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: