Linux Kernel 5.15.158 Oxnas (OX820) Pogo V3 package and Debian rootfs

> However, I've concluded that this rootfs is no
> longer good to use (we can't say that if those
> invalid inodes will not cause problem in the
> future).

Sounds like rootfs will be corrupted several times more before file system drivers are OK.

My investigation about pincontrol has led here. First again a short cut from the original driver:

/* Dedicate the GPIO over to i2c.
     * NOTE: This may be confusing, but we are not using the i2c core here we
     * are using bit-bashed GPIO, so we must disable the primary, secondary and
     * tertiary functions of the relevant GPIO pins
     */
    spin_lock_irqsave(&oxnas_gpio_spinlock, flags);
#if defined(CONFIG_ARCH_OXNAS)
    writel(readl(SYS_CTRL_GPIO_PRIMSEL_CTRL_0) & ~mask, SYS_CTRL_GPIO_PRIMSEL_CTRL_0);
    writel(readl(SYS_CTRL_GPIO_SECSEL_CTRL_0)  & ~mask, SYS_CTRL_GPIO_SECSEL_CTRL_0);
    writel(readl(SYS_CTRL_GPIO_TERTSEL_CTRL_0) & ~mask, SYS_CTRL_GPIO_TERTSEL_CTRL_0);
#elif defined(CONFIG_ARCH_OX820)
#if (CONFIG_OXNAS_I2C_SDA < SYS_CTRL_NUM_PINS)
    writel(readl(SYS_CTRL_SECONDARY_SEL)   & ~mask, SYS_CTRL_SECONDARY_SEL);
    writel(readl(SYS_CTRL_TERTIARY_SEL)    & ~mask, SYS_CTRL_TERTIARY_SEL);
    writel(readl(SYS_CTRL_QUATERNARY_SEL)  & ~mask, SYS_CTRL_QUATERNARY_SEL);
    writel(readl(SYS_CTRL_DEBUG_SEL)       & ~mask, SYS_CTRL_DEBUG_SEL);
    writel(readl(SYS_CTRL_ALTERNATIVE_SEL) & ~mask, SYS_CTRL_ALTERNATIVE_SEL);
    writel(readl(GPIO_A_INPUT_DEBOUNCE_ENABLE) & ~mask, GPIO_A_INPUT_DEBOUNCE_ENABLE);
#else
    writel(readl(SEC_CTRL_SECONDARY_SEL)   & ~mask, SEC_CTRL_SECONDARY_SEL);
    writel(readl(SEC_CTRL_TERTIARY_SEL)    & ~mask, SEC_CTRL_TERTIARY_SEL);
    writel(readl(SEC_CTRL_QUATERNARY_SEL)  & ~mask, SEC_CTRL_QUATERNARY_SEL);
    writel(readl(SEC_CTRL_DEBUG_SEL)       & ~mask, SEC_CTRL_DEBUG_SEL);
    writel(readl(SEC_CTRL_ALTERNATIVE_SEL) & ~mask, SEC_CTRL_ALTERNATIVE_SEL);
    writel(readl(GPIO_B_INPUT_DEBOUNCE_ENABLE) & ~mask, GPIO_B_INPUT_DEBOUNCE_ENABLE);
#endif

Variables (or pointers) SYS_CTRL_GPIO_PRIMSEL_CTRL_0, SYS_CTRL_GPIO_SECSEL_CTRL_0 and SYS_CTRL_GPIO_TERTSEL_CTRL_0 are all not declared anywhere in this kernel source and if I remember correcty they are not used anywhere else but this pincontrol. I have faint memory traces seeing something like that in powerdown function too, but I'm not 100% sure. I suppose they are simply prowided by u-boot, because they seem to be nowhere else.

Second thing is, I did not find anything in 4.1.16 kernel source, that would make same GPIO disabling first three writel functions do in this old kernel. Might it be here, why GPIO does not function right if there is not warmboot after booting the old kernel first?



Edited 1 time(s). Last edit at 02/19/2016 03:19AM by tilator.

tilator,

> Sounds like rootfs will be corrupted several times
> more before file system drivers are OK.

I think we are safer to stay with kernel 4.1 Oxnas and not try to test using real rootfs until we figure out the reason. The puzzle is why Kirkwood 4.4 rootfs is working fine with Ext4 but not the Oxnas 4.4 rootfs? something in the Oxnas-specific code must have triggered this bug.

> SYS_CTRL_GPIO_PRIMSEL_CTRL_0,
> SYS_CTRL_GPIO_SECSEL_CTRL_0 and
> SYS_CTRL_GPIO_TERTSEL_CTRL_0 are all not declared
> anywhere in this kernel source and if I remember
> correcty they are not used anywhere else but this
> pincontrol.

It's rare, but sometime the GPL source does not match what is running.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

> > SYS_CTRL_GPIO_PRIMSEL_CTRL_0,
> > SYS_CTRL_GPIO_SECSEL_CTRL_0 and
> > SYS_CTRL_GPIO_TERTSEL_CTRL_0 are all not
> declared
> > anywhere in this kernel source and if I
> remember
> > correcty they are not used anywhere else but
> this
> > pincontrol.
>
> It's rare, but sometime the GPL source does not
> match what is running.

Yes I know, but this 2.6 kernel source code seems to work just fine. I have compiled it succesfully before.

The trouble seems to be I2C bitbanging driver. I suppose there is unfortunalety not any better I2C driver to try.

tilator,

I've just remember this. Which u-boot are you running? the version I've released has SNTP, but the RTC clock driver was commented out, since the Pogo V3 does not have it.

You could take a look at the source code in my GitHub. And see if you can put it back, and use SNTP to query the datetime, and that hopefully will update RTC during boot. However, I am not sure if our current u-boot has the correct driver for the Atikio RTC. Just a thought that could be a potential work around.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> tilator,
>
> I've just remember this. Which u-boot are you
> running? the version I've released has SNTP, but
> the RTC clock driver was commented out, since the
> Pogo V3 does not have it.

I have OpenWrt version 2014.10.

B.T.W there is one typo in this version Oxnas 300* patch. There is "endef" and obviously it should be "endif".

> You could take a look at the source code in my
> GitHub. And see if you can put it back, and use
> SNTP to query the datetime, and that hopefully
> will update RTC during boot. However, I am not
> sure if our current u-boot has the correct driver
> for the Atikio RTC. Just a thought that could be a
> potential work around.

Sounds like something to try. Even though it must be I2C driver making the problem. I suppose this I2C is integreted in same chip with RTC. Maybe there is other means than the "Oxnas way" to use it.

This kernel 4.1.6 pincontrol needs (or at least it seems like) some kind on pullup. How can I do it in userspace?



Edited 1 time(s). Last edit at 02/23/2016 08:31AM by tilator.

tilator Wrote:
-------------------------------------------------------
> This kernel 4.1.6 pincontrol needs (or at least it
> seems like) some kind on pullup. How can I do it
> in userspace?

There is a tool to set GPIO in userspace (posted somewhere in the forum). If you can't find it I'll look in my archive.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I've splitted sebr issue with headers to a new thread.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi again,

Some progress in this, because - thanks to Daniel Golle - RTC works now fine. Next step here would be making the device power down poperly. This DTS bellow produces the working RTC, but it does not allow the device boot up. It triggers power off while it loads. There is some discussion abot the same behavior with Raspberry PI here:

https://www.raspberrypi.org/forums/viewtopic.php?f=107&t=113789

(DTS with working RTC and "too well" working power off)

/*
* Copyright (C) 2014 Daniel Golle <daniel@makrotopia.org>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/

/dts-v1/;
#include "ox820.dtsi"

/ {
	model = "Akitio";

	chosen {
		bootargs = "root=/dev/sda1 rw console=ttyS0,115200";
	};

	pcie-controller@47C00000 {
		status = "disabled";
	};

	uart@44200000 {
		status = "okay";
	};

	sata@45900000 {
		nr-ports = <2>;
		status = "okay";
	};

	nand@41000000 {
		status = "okay";
			partition@0 {
				label = "_boot_";
				reg = <0x0 0x26c0000>;
			};

			partition@26c0000 {
				label = "ubi";
				reg = <0x26c0000 0xd940000>;
			};
	};

	ethernet@40400000 {
		status = "okay";
	};

	ehci@40200100 {
		status = "okay";
	};

	i2c-gpio {
		compatible = "i2c-gpio";
		gpios = <&GPIOB 9 0 &GPIOB 10 0>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_i2c>;
		i2c-gpio,delay-us = <10>;
		#address-cells = <1>;
		#size-cells = <0>;
		ds1307: rtc@68 {
			compatible = "dallas,ds1307";
			reg = <0x68>;
		};
	};

	gpio-keys-polled {
		compatible = "gpio-keys-polled";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_buttons>;
		#address-cells = <1>;
		#size-cells = <0>;
		poll-interval = <100>;
		power {
			label = "power";
			gpios = <&GPIOA 11 1>;
			linux,code = <116>;
		};
	};

	gpio-leds {
		compatible = "gpio-leds";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_leds>;
		status {
			label = "akitio:red:status";
			gpios = <&GPIOA 29 0>;
		};
	};

	gpio-poweroff {
		compatible = "gpio-poweroff";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_poweroff>;
		gpios = <&GPIOB 13 0>;
	};

	pinctrl {
		i2c {
			pinctrl_i2c: i2c-0 {
				plxtech,pins =
					<1 9 0 4 /* MF_B9 GPIO debounce */
					 1 10 0 4>; /* MF_B10 GPIO debounce */
			};

		};

		buttons {
			pinctrl_buttons: buttons-0 {
				plxtech,pins =
				<0 11 0 0>; /* MF_A11 GPIO */
			};
		};

		leds {
			pinctrl_leds: leds-0 {
				plxtech,pins =
			<0 29 0 0>; /* MF_A29 GPIO */
			};
		};

		poweroff {
			pinctrl_poweroff: poweroff-0 {
				plxtech,pins =
				<1 13 0 0>; /* MF_B13 GPIO */
			};
		};
	};
};

BTW - I got kernel 4.4.3 up and going woth it too.



Edited 1 time(s). Last edit at 03/20/2016 03:57AM by tilator.

tilator,

> BTW - I got kernel 4.4.3 up and going woth it too.

Good news! which patch set did you use? is that from OpenWRT? The ext4 problem was resolved in this version?

I'm going to work on this Oxnas kernel in a week or so. For the GPIO power off, I'll see if I can find out.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> tilator,
>
> > BTW - I got kernel 4.4.3 up and going woth it
> too.
>
> Good news! which patch set did you use? is that
> from OpenWRT? The ext4 problem was resolved in
> this version?
>
> I'm going to work on this Oxnas kernel in a week
> or so. For the GPIO power off, I'll see if I can
> find out.

Bodhi,

I used OpenWRT source and all default patches for it. There must be some differences in what Oxnas device to use as a base. If I remember it right, Pogoplug V3 was what made it work.

Power down works with DTS bellow (command given from console powers it completely down). Power button does not trigger it yet:

/*
* Copyright (C) 2014 Daniel Golle <daniel@makrotopia.org>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/

/dts-v1/;
#include "ox820.dtsi"

/ {
	model = "Akitio";

	chosen {
		bootargs = "root=/dev/sda1 rw console=ttyS0,115200";
	};

	pcie-controller@47C00000 {
		status = "disabled";
	};

	uart@44200000 {
		status = "okay";
	};

	sata@45900000 {
		nr-ports = <2>;
		status = "okay";
	};

	nand@41000000 {
		status = "okay";
			partition@0 {
				label = "_boot_";
				reg = <0x0 0x26c0000>;
			};

			partition@26c0000 {
				label = "ubi";
				reg = <0x26c0000 0xd940000>;
			};
	};

	ethernet@40400000 {
		status = "okay";
	};

	ehci@40200100 {
		status = "okay";
	};

	i2c-gpio {
		compatible = "i2c-gpio";
		gpios = <&GPIOB 9 0 &GPIOB 10 0>;
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_i2c>;
		i2c-gpio,delay-us = <10>;
		#address-cells = <1>;
		#size-cells = <0>;
		ds1307: rtc@68 {
			compatible = "dallas,ds1307";
			reg = <0x68>;
		};
	};

	gpio-keys-polled {
		compatible = "gpio-keys-polled";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_buttons>;
		#address-cells = <1>;
		#size-cells = <0>;
		poll-interval = <100>;
		power {
			label = "power";
			gpios = <&GPIOA 11 1>;
			linux,code = <116>;
		};
	};

	gpio-leds {
		compatible = "gpio-leds";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_leds>;
		status {
			label = "akitio:red:status";
			gpios = <&GPIOA 29 0>;
		};
	};

	gpio-poweroff {
		compatible = "gpio-poweroff";
		pinctrl-names = "default";
		pinctrl-0 = <&pinctrl_poweroff>;
		gpios = <&GPIOB 13 2>;
	};

	pinctrl {
		i2c {
			pinctrl_i2c: i2c-0 {
				plxtech,pins =
					<1 9 0 4 /* MF_B9 GPIO debounce */
					 1 10 0 4>; /* MF_B10 GPIO debounce */
			};
		};

		buttons {
			pinctrl_buttons: buttons-0 {
				plxtech,pins =
				<0 11 0 0>; /* MF_A11 GPIO */
			};
		};

		leds {
			pinctrl_leds: leds-0 {
				plxtech,pins =
			<0 29 0 0>; /* MF_A29 GPIO */
			};
		};

		poweroff {
			pinctrl_poweroff: poweroff-0 {
				plxtech,pins =
				<1 13 0 0>; /* MF_B13 GPIO */
			};
		};
	};
};

Edited 3 time(s). Last edit at 03/20/2016 12:44PM by tilator.

Here is attached latest working kernel config I used, if you want to see it.

Thanks tilator. Definitely it will help if I can't run it with my config.

I've briefly looked at the kernel GPIOs, and also u-boot GPIO. Oxnas specific code is just so out of mainline convention adn design that it won't be easy to track down these problem. When I have time, I'll build a test u-boot that hopefully will let us do better testing.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Quote

gpio-poweroff {
compatible = "gpio-poweroff";
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_poweroff>;
gpios = <&GPIOB 13 2>;
};

Good work :)

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

tilator,

Is there a way to download the whole patch (files and patches) from the OpenWrt site? they are using SVN, and I'm not familiar with it. It's more convenient just getting everything at once in an archive.

They've done good works on this Oxnas plug so I think I'll try to use their patch, and then patch my changes on top of it.

UPDATE:

NVM. I'm cloning their Git.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 03/21/2016 12:17AM by bodhi.

> NVM. I'm cloning their Git.

That's right way to go.

Patch kernel only with Oxnas spesific files and patches. Don't use OpenWRT patches at all, because they make some weird things like putting rootfs always to UBI if it is available.

If you use the config I put here, you must also turn GPIO button related settings on. They are in input devices under keyboard settings.

I have not had yet time to test how to turn buttons to work.

It will be a while before I can revisit this.

Did you patch smp_tlb.c ? I noticed that patch is not in OpenWrt Oxnas patch set.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> Did you patch smp_tlb.c ? I noticed that patch is
> not in OpenWrt Oxnas patch set.

I did not put any other patches but those in OpenWRT. Where is this SMP patch and how does it effect?

This? https://dev.openwrt.org/browser/trunk/target/linux/generic/patches-4.4/100-ARM-work-around-CONFIG_CPU_SW_DOMAIN_PAN-breakage-on.patch?rev=48335



Edited 1 time(s). Last edit at 03/23/2016 03:57AM by tilator.

This patch solves the kernel panic that we've seen before about v6wbi_flush_user_tlb_range.
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-October/376368.html

You meant with the OpenWrt patches you did not see this any more?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Yes that was it. So you applied /generic/patches-4.4/ too?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> Yes that was it. So you applied
> /generic/patches-4.4/ too?

I did not patch it with generic patches, but I did not (yet) run into any kernel panic either.

Is the fault so severe, it prevents booting totally or does it apply only once and while randomly? If faulty is severe enough it must have been taken care of already. Otherwise it would not boot at all without the patch. If it's random, patching might still be good idea.

tilator,

Try include something memory intensive during boot, such as samba. If your rootfs is lean, you might have not seen it even though the bug is there.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

tilator,

Running 4.4.3 with OpenWrt patch and the smp_tlb.c.patch.

I'm seeing this samba segfault (not a kernel panic). Everything else seems to work OK.

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.4.3-oxnas-tld-1 (root@tldDebian) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 SMP PREEMPT Fri Mar 25 21:05:00 PDT 2016
[    0.000000] CPU: ARMv6-compatible processor [410fb025] revision 5 (ARMv7), cr=00c5787d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Pogoplug Pro
[    0.000000] bootconsole [earlycon0] enabled


[   34.646968] Unhandled fault: page domain fault (0x81b) at 0xb6fc7000
[   34.653332] pgd = c6d14000
[   34.656030] [b6fc7000] *pgd=66d02831, *pte=6316b55f, *ppte=6316bc7e
[   34.662315] Internal error: : 81b [#1] PREEMPT SMP ARM
[   34.667432] Modules linked in: autofs4 netconsole configfs uas
[   34.673310] CPU: 0 PID: 1474 Comm: samba-tool Not tainted 4.4.3-oxnas-tld-1 #1
[   34.680501] Hardware name: PLXTECH NAS782X SoC (Flattened Device Tree)
[   34.687004] task: c75e0800 ti: c69fa000 task.ti: c69fa000
[   34.692392] PC is at v6_coherent_kern_range+0x4/0x2c
[   34.697349] LR is at arm_syscall+0x140/0x2d4
[   34.701606] pc : [<c00155c4>]    lr : [<c000dd7c>]    psr: 40000013
[   34.701606] sp : c69fbf08  ip : c75e0800  fp : 000002f4
[   34.713034] r10: 00000000  r9 : c69fa000  r8 : c000a684
[   34.718236] r7 : b6fc701b  r6 : c69fa000  r5 : b6fc701b  r4 : 00000002
[   34.724738] r3 : 00000000  r2 : b6fc7008  r1 : b6fc701b  r0 : b6fc7000
[   34.731240] Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   34.738344] Control: 00c5787d  Table: 66d1400a  DAC: 00000051
[   34.744071] Process samba-tool (pid: 1474, stack limit = 0xc69fa190)
[   34.750400] Stack: (0xc69fbf08 to 0xc69fc000)
[   34.754746] bf00:                   00000101 c000934c 00000000 00001000 00000077 00000007
[   34.762900] bf20: 00000000 c6416000 b6fc7000 00000001 c085080c c00ba578 000b6fc7 c02f2928
[   34.771051] bf40: ffffffff 00000000 c6416000 c6416038 00000000 00000022 00000007 00001000
[   34.779202] bf60: 00000000 00000000 b56bfecc c00a964c 00000022 00000000 000b6fc7 c69fbf84
[   34.787356] bf80: c69fa000 00000000 ffffffff 00000000 0033b498 b6fc7008 b5381434 b525dd50
[   34.795507] bfa0: 000f0002 c000a4c0 b6fc7008 b5381434 b6fc7008 b6fc701b 00000000 b6fc7008
[   34.803660] bfc0: b6fc7008 b5381434 b525dd50 000f0002 b56becc4 b5391eb0 0000012c 000002f4
[   34.811816] bfe0: 00000014 bee22d1c b56bfc04 b56bfd80 40000010 b6fc7008 e31300ff 1a000026
[   34.819975] [<c00155c4>] (v6_coherent_kern_range) from [<c000dd7c>] (arm_syscall+0x140/0x2d4)
[   34.828474] [<c000dd7c>] (arm_syscall) from [<c000a4c0>] (ret_fast_syscall+0x0/0x3c)
[   34.836198] Code: ee070f15 e12fff1e e12fff1e e3c0001f (ee070f3a) 
[   34.842273] ---[ end trace f014d8d894956d58 ]---
[   34.847572] ------------[ cut here ]------------
[   34.852203] WARNING: CPU: 0 PID: 1474 at kernel/smp.c:417 smp_call_function_many+0x29c/0x2cc()
[   34.860776] Modules linked in: autofs4 netconsole configfs uas
[   34.866659] CPU: 0 PID: 1474 Comm: samba-tool Tainted: G      D         4.4.3-oxnas-tld-1 #1
[   34.875059] Hardware name: PLXTECH NAS782X SoC (Flattened Device Tree)
[   34.881579] [<c00114e8>] (unwind_backtrace) from [<c000d6ec>] (show_stack+0x10/0x14)
[   34.889308] [<c000d6ec>] (show_stack) from [<c02ce660>] (dump_stack+0x84/0xa0)
[   34.896519] [<c02ce660>] (dump_stack) from [<c0019200>] (warn_slowpath_common+0x78/0xb4)
[   34.904588] [<c0019200>] (warn_slowpath_common) from [<c00192d8>] (warn_slowpath_null+0x1c/0x24)
[   34.913349] [<c00192d8>] (warn_slowpath_null) from [<c0079b64>] (smp_call_function_many+0x29c/0x2cc)
[   34.922455] [<c0079b64>] (smp_call_function_many) from [<c0079cd4>] (on_each_cpu_mask+0x38/0xa8)
[   34.931220] [<c0079cd4>] (on_each_cpu_mask) from [<c00b3c9c>] (unmap_single_vma+0x474/0x500)
[   34.939637] [<c00b3c9c>] (unmap_single_vma) from [<c00b4ac0>] (unmap_vmas+0x54/0x68)
[   34.947363] [<c00b4ac0>] (unmap_vmas) from [<c00ba954>] (exit_mmap+0xd8/0x1dc)
[   34.954572] [<c00ba954>] (exit_mmap) from [<c0016fac>] (mmput+0x44/0xe4)
[   34.961559] [<c0016fac>] (mmput) from [<c001a70c>] (do_exit+0x230/0x978)
[   34.968250] [<c001a70c>] (do_exit) from [<c000d900>] (die+0x210/0x2e8)
[   34.974766] [<c000d900>] (die) from [<c00093bc>] (do_DataAbort+0xa4/0xb4)
[   34.981542] [<c00093bc>] (do_DataAbort) from [<c000e120>] (__dabt_svc+0x40/0x60)
[   34.988912] Exception stack(0xc69fbeb8 to 0xc69fbf00)
[   34.993950] bea0:                                                       b6fc7000 b6fc701b
[   35.002106] bec0: b6fc7008 00000000 00000002 b6fc701b c69fa000 b6fc701b c000a684 c69fa000
[   35.010260] bee0: 00000000 000002f4 c75e0800 c69fbf08 c000dd7c c00155c4 40000013 ffffffff
[   35.018419] [<c000e120>] (__dabt_svc) from [<c00155c4>] (v6_coherent_kern_range+0x4/0x2c)
[   35.026575] [<c00155c4>] (v6_coherent_kern_range) from [<c000dd7c>] (arm_syscall+0x140/0x2d4)
[   35.035074] [<c000dd7c>] (arm_syscall) from [<c000a4c0>] (ret_fast_syscall+0x0/0x3c)
[   35.042790] ---[ end trace f014d8d894956d59 ]---
[   35.047390] ------------[ cut here ]------------
[   35.052001] WARNING: CPU: 0 PID: 1474 at kernel/smp.c:292 smp_call_function_single+0x114/0x1b8()
[   35.060748] Modules linked in: autofs4 netconsole configfs uas
[   35.066631] CPU: 0 PID: 1474 Comm: samba-tool Tainted: G      D W       4.4.3-oxnas-tld-1 #1
[   35.075036] Hardware name: PLXTECH NAS782X SoC (Flattened Device Tree)
[   35.081546] [<c00114e8>] (unwind_backtrace) from [<c000d6ec>] (show_stack+0x10/0x14)
[   35.089274] [<c000d6ec>] (show_stack) from [<c02ce660>] (dump_stack+0x84/0xa0)
[   35.096486] [<c02ce660>] (dump_stack) from [<c0019200>] (warn_slowpath_common+0x78/0xb4)
[   35.104558] [<c0019200>] (warn_slowpath_common) from [<c00192d8>] (warn_slowpath_null+0x1c/0x24)
[   35.113320] [<c00192d8>] (warn_slowpath_null) from [<c00796e8>] (smp_call_function_single+0x114/0x1b8)
[   35.122600] [<c00796e8>] (smp_call_function_single) from [<c0079cd4>] (on_each_cpu_mask+0x38/0xa8)
[   35.131537] [<c0079cd4>] (on_each_cpu_mask) from [<c00b3c9c>] (unmap_single_vma+0x474/0x500)
[   35.139955] [<c00b3c9c>] (unmap_single_vma) from [<c00b4ac0>] (unmap_vmas+0x54/0x68)
[   35.147678] [<c00b4ac0>] (unmap_vmas) from [<c00ba954>] (exit_mmap+0xd8/0x1dc)
[   35.154885] [<c00ba954>] (exit_mmap) from [<c0016fac>] (mmput+0x44/0xe4)
[   35.161571] [<c0016fac>] (mmput) from [<c001a70c>] (do_exit+0x230/0x978)
[   35.168256] [<c001a70c>] (do_exit) from [<c000d900>] (die+0x210/0x2e8)
[   35.174771] [<c000d900>] (die) from [<c00093bc>] (do_DataAbort+0xa4/0xb4)
[   35.181539] [<c00093bc>] (do_DataAbort) from [<c000e120>] (__dabt_svc+0x40/0x60)
[   35.188904] Exception stack(0xc69fbeb8 to 0xc69fbf00)
[   35.193940] bea0:                                                       b6fc7000 b6fc701b
[   35.202094] bec0: b6fc7008 00000000 00000002 b6fc701b c69fa000 b6fc701b c000a684 c69fa000
[   35.210245] bee0: 00000000 000002f4 c75e0800 c69fbf08 c000dd7c c00155c4 40000013 ffffffff
[   35.218401] [<c000e120>] (__dabt_svc) from [<c00155c4>] (v6_coherent_kern_range+0x4/0x2c)
[   35.226557] [<c00155c4>] (v6_coherent_kern_range) from [<c000dd7c>] (arm_syscall+0x140/0x2d4)
[   35.235056] [<c000dd7c>] (arm_syscall) from [<c000a4c0>] (ret_fast_syscall+0x0/0x3c)
[   35.242772] ---[ end trace f014d8d894956d5a ]---
[   35.291678] Unhandled fault: page domain fault (0x81b) at 0xb6f12000
[   35.298063] pgd = c6d30000
[   35.300766] [b6f12000] *pgd=66cf6831, *pte=64c1955f, *ppte=64c19c7e
[   35.307069] Internal error: : 81b [#2] PREEMPT SMP ARM
[   35.312192] Modules linked in: autofs4 netconsole configfs uas
[   35.318095] CPU: 1 PID: 1481 Comm: samba-tool Tainted: G      D W       4.4.3-oxnas-tld-1 #1
[   35.326504] Hardware name: PLXTECH NAS782X SoC (Flattened Device Tree)
[   35.333015] task: c6ce1800 ti: c69fe000 task.ti: c69fe000
[   35.338419] PC is at v6_coherent_kern_range+0x4/0x2c
[   35.343383] LR is at arm_syscall+0x140/0x2d4
[   35.347648] pc : [<c00155c4>]    lr : [<c000dd7c>]    psr: 40000013
[   35.347648] sp : c69fff08  ip : c6ce1800  fp : 000002f4
[   35.359080] r10: 00000000  r9 : c69fe000  r8 : c000a684
[   35.364293] r7 : b6f1201b  r6 : c69fe000  r5 : b6f1201b  r4 : 00000002
[   35.370803] r3 : 00000000  r2 : b6f12008  r1 : b6f1201b  r0 : b6f12000
[   35.377310] Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   35.384430] Control: 00c5787d  Table: 66d3000a  DAC: 00000051
[   35.390161] Process samba-tool (pid: 1481, stack limit = 0xc69fe190)
[   35.396492] Stack: (0xc69fff08 to 0xc6a00000)
[   35.400847] ff00:                   00000101 c000934c 00000000 00001000 00000077 00000007
[   35.409006] ff20: 00000000 c7680680 b6f12000 00000001 c085080c c00ba578 000b6f12 c02f2928
[   35.417167] ff40: ffffffff 00000000 c7680680 c76806b8 00000000 00000022 00000007 00001000
[   35.425326] ff60: 00000000 00000000 b560aecc c00a964c 00000022 00000000 000b6f12 c69fff84
[   35.433484] ff80: c69fe000 00000000 ffffffff 00000000 0033b498 b6f12008 b52cc434 b51a8d50
[   35.441640] ffa0: 000f0002 c000a4c0 b6f12008 b52cc434 b6f12008 b6f1201b 00000000 b6f12008
[   35.449799] ffc0: b6f12008 b52cc434 b51a8d50 000f0002 b5609cc4 b52dceb0 0000012c 000002f4
[   35.457955] ffe0: 00000014 bedb8d1c b560ac04 b560ad80 40000010 b6f12008 67ffd861 67ffdc61
[   35.466132] [<c00155c4>] (v6_coherent_kern_range) from [<c000dd7c>] (arm_syscall+0x140/0x2d4)
[   35.474636] [<c000dd7c>] (arm_syscall) from [<c000a4c0>] (ret_fast_syscall+0x0/0x3c)
[   35.482365] Code: ee070f15 e12fff1e e12fff1e e3c0001f (ee070f3a) 
[   35.488445] ---[ end trace f014d8d894956d5b ]---
Segmentation fault
[ ok ] Starting periodic command scheduler: cron.
[ ok ] Starting system message bus: dbus.
Segmentation fault
[ ok ] Starting NetBIOS name server: nmbd.
[ ok ] Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon.
[ ok ] Starting OpenBSD Secure Shell server: sshd.
[ ok ] Starting NTP server: ntpd.
[   38.602726] Unhandled fault: page domain fault (0x81b) at 0xb6f8f000
[   38.609085] pgd = c6e14000
[   38.611788] [b6f8f000] *pgd=66c51831, *pte=631da55f, *ppte=631dac7e
[   38.618076] Internal error: : 81b [#3] PREEMPT SMP ARM
[   38.623193] Modules linked in: autofs4 netconsole configfs uas
[   38.629089] CPU: 1 PID: 1810 Comm: samba-tool Tainted: G      D W       4.4.3-oxnas-tld-1 #1
[   38.637491] Hardware name: PLXTECH NAS782X SoC (Flattened Device Tree)
[   38.643996] task: c650fc00 ti: c6e50000 task.ti: c6e50000
[   38.649390] PC is at v6_coherent_kern_range+0x4/0x2c
[   38.654346] LR is at arm_syscall+0x140/0x2d4
[   38.658603] pc : [<c00155c4>]    lr : [<c000dd7c>]    psr: 40000013
[   38.658603] sp : c6e51f08  ip : c650fc00  fp : 000002f4
[   38.670033] r10: 00000000  r9 : c6e50000  r8 : c000a684
[   38.675240] r7 : b6f8f01b  r6 : c6e50000  r5 : b6f8f01b  r4 : 00000002
[   38.681743] r3 : 00000000  r2 : b6f8f008  r1 : b6f8f01b  r0 : b6f8f000
[   38.688245] Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   38.695349] Control: 00c5787d  Table: 66e1400a  DAC: 00000051
[   38.701073] Process samba-tool (pid: 1810, stack limit = 0xc6e50190)
[   38.707404] Stack: (0xc6e51f08 to 0xc6e52000)
[   38.711759] 1f00:                   00000101 c000934c 00000000 00001000 00000077 00000007
[   38.719915] 1f20: 00000000 c7681a00 b6f8f000 00000001 c085080c c00ba578 000b6f8f c02f2928
[   38.728069] 1f40: ffffffff 00000000 c7681a00 c7681a38 00000000 00000022 00000007 00001000
[   38.736222] 1f60: 00000000 00000000 b5687ecc c00a964c 00000022 00000000 000b6f8f c6e51f84
[   38.744373] 1f80: c6e50000 00000000 ffffffff 00000000 0033b498 b6f8f008 b5349434 b5225d50
[   38.752524] 1fa0: 000f0002 c000a4c0 b6f8f008 b5349434 b6f8f008 b6f8f01b 00000000 b6f8f008
[   38.760678] 1fc0: b6f8f008 b5349434 b5225d50 000f0002 b5686cc4 b5359eb0 0000012c 000002f4
[   38.768834] 1fe0: 00000014 be90ad1c b5687c04 b5687d80 40000010 b6f8f008 00000000 00000000
[   38.776998] [<c00155c4>] (v6_coherent_kern_range) from [<c000dd7c>] (arm_syscall+0x140/0x2d4)
[   38.785502] [<c000dd7c>] (arm_syscall) from [<c000a4c0>] (ret_fast_syscall+0x0/0x3c)
[   38.793223] Code: ee070f15 e12fff1e e12fff1e e3c0001f (ee070f3a) 
[   38.799298] ---[ end trace f014d8d894956d5c ]---
Segmentation fault
[ ok ] Starting SMB/CIFS daemon: smbd.

Samba is still running, but when restarting I still get a segfault.

Could you install samba, if it is not installed, and see if you get the same error.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

> Samba is still running, but when restarting I
> still get a segfault.
>
> Could you install samba, if it is not installed,
> and see if you get the same error.


I have Samba running, but I don't have that error.

Might it have something to do with Linux version? I have still years old Fedora 12 because that's what original software is based on and I have not changed it.

B.T.W led is working fine now. Still buttons to go ...



Edited 1 time(s). Last edit at 03/26/2016 08:12AM by tilator.

>
> I have Samba running, but I don't have that
> error.
>
> Might it have something to do with Linux version?
> I have still years old Fedora 12 because that's
> what original software is based on and I have not
> changed it.
>

I see. I think the error might point to a flaw in the memory management setting in the kernel. It used to be a problem in arm11mpcore. Samba is pretty big, loading it tends to result in task migration, cache invalidation,... Or it might be just samba :)

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

The good news: I am able to run w/o seeing any kernel panic in memory, or seeing segfault in samba. Apparently the problem was in the old kernel config that new kernel code does not like. Just what I've guessed before, but we got sidetracked into other directions :) ie. samba segfault is just a symptom.

Now the next step is to reconcile OpenWrt-style kernel config with our own config (there are a lot that OepnWrt don't use while we use them in Debian).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> The good news: I am able to run w/o seeing any
> kernel panic in memory, or seeing segfault in
> samba. Apparently the problem was in the old
> kernel config that new kernel code does not like.
> Just what I've guessed before, but we got
> sidetracked into other directions :) ie. samba
> segfault is just a symptom.

What patches did you apply? OpenWRT Oxnas spesific or something else?

tilator,

I used OpenWrt patches. And the smp_tld.c patch, and start with your (OpenWrt style) config.

I think finally understood the real problem after hours of researching. I'll do a few more tests to confirm my conclusion and let everybody know.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi, I am trying to install the latest kernel. I created the uImage using the command given. But when try the second command to create uImage, it doesn't run because initrd.img is not there.

The command I type is this:
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x60000000 -e 0x60000000 -n initramfs-4.1.0-oxnas-tld-1 -d initrd.img-4.1.0-oxnas-tld-1 uInitrd

I am currently on 3.17. So initrd.img-3.17.0-oxnas-tld-1 is there. How to get the initrd.img for 4.1?

> I am currently on 3.17. So
> initrd.img-3.17.0-oxnas-tld-1 is there. How to get
> the initrd.img for 4.1?

initrd.img-3.17.0-oxnas-tld-1 is automatically generated with dpkg command.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)