Hello everybody.
I know it is Debian mostly forum, but got great help in the past here, so I'll try again here.
The problem is, that my wired network don't works since the new 4.20 kernel - either on my default r8169 drivers (built in kernel) or the installed r8168.
Everything works fine in my 4.19.9 kernel.
When on r8169 I get:

dmesg | grep r8169 
[ 1.839771] r8169 0000:01:00.0: enabling device (0140 -> 0143) 
[ 1.846456] r8169 0000:01:00.0 (unnamed net_device) (uninitialized): rtl_ph 0 (loop: 20, delay: 25). 
[ 1.856836] r8160000:01:00.0 failed with error -5"

Tried with the r8168 driver, and it looked promising sinve I managed (I guess) to bring this network up

dmesg | grep r8168
[   18.133709] r8168: loading out-of-tree module taints kernel.
[   18.154774] r8168 Gigabit Ethernet driver 8.046.00-NAPI loaded
[   18.201727] r8168 0000:01:00.0: no MSI. Back to INTx.
[   18.336800] r8168 0000:01:00.0 (unnamed net_device) (uninitialized): Invalid ether addr 00:00:00:00:00:00
[   18.363031] r8168 0000:01:00.0 (unnamed net_device) (uninitialized): Random ether addr 12:52:e3:82:44:54
[   18.423468] r8168: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[   18.489400] r8168  Copyright (C) 2018  Realtek NIC software team <nicfae@realtek.com>

ifconfig -a
eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 192.168.1.4  netmask 255.255.255.0  broadcast 192.168.1.255
        ether cc:5d:4e:c9:f5:4a  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 35  base 0xc000

But the network is dead - I can ping only to myself.
The hardware:

01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 03)
	Subsystem: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller [10ec:8168]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 35
	Region 0: I/O ports at 10000 [size=256]
	Region 2: Memory at e0014000 (64-bit, prefetchable) [size=4K]
	Region 4: Memory at e0010000 (64-bit, prefetchable) [size=16K]
	[virtual] Expansion ROM at e0000000 [disabled] [size=64K]
	Capabilities: <access denied>
	Kernel driver in use: r8168
	Kernel modules: r8168

What can I try yet? I'm out of ideas - I presented my problems also on the archlinux forum here
https://archlinuxarm.org/forum/viewtopic.php?t=13394&p=60424
but still out of luck...

-----
Mod edit: changed thread title



Edited 1 time(s). Last edit at 02/10/2019 02:13PM by bodhi.

What is the build date of your 2 kernel versions?

Arch 4.19 kernel
Arch 4.20 kernel

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hello !
I didn't noticed your message - I need to set up some email notifications - anyway, I'm happy to hear from you.
My current - working environment:

uname -rv
4.19.9-1-ARCH #1 PREEMPT Fri Dec 14 03:34:38 UTC 2018

The recent kernel (and tried all previous 4.20 versions)

uname -rv
4.20.7-1-ARCH #1 PREEMPT Thu Feb 7 01:09:27 UTC 2019

Edit - maybe more useful will be to show the first revision where my network stopped working:

uname -rv
4.20.0-1-ARCH #1 PREEMPT Fri Dec 28 08:19:47 UTC 2018

Edited 2 time(s). Last edit at 02/10/2019 01:32PM by arti74.

arti74,

There were too many changes in the module rt8169 in between those dates in mainline. So it's hard to track this down.

I have moved this here (to Debian subforum) to ask other Debian users who have this box to verify.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 02/10/2019 02:15PM by bodhi.

Yes, thank you.

Yes, I'm getting this on my box as well.

[    9.575337] systemd[1]: Detected architecture arm.
[    9.625431] systemd[1]: Set hostname to <nsa310a>.
[    9.697020] random: systemd: uninitialized urandom read (16 bytes read)
[    9.836602] random: systemd-gpt-aut: uninitialized urandom read (16 bytes rea                                                                                       d)
[   10.408590] systemd[1]: Reached target Remote File Systems.
[   10.449463] systemd[1]: Started Forward Password Requests to Wall Directory W                                                                                       atch.
[   10.489295] systemd[1]: Listening on /dev/initctl Compatibility Named Pipe.
[   10.529372] systemd[1]: Listening on udev Control Socket.
[   10.569316] systemd[1]: Listening on Journal Socket (/dev/log).
[   10.609846] systemd[1]: Created slice System Slice.
[   10.642515] systemd[1]: Mounting POSIX Message Queue File System...
[   12.699856] EXT4-fs (sda3): re-mounted. Opts: errors=remount-ro
[   12.899587] systemd-journald[153]: Received request to flush runtime journal                                                                                        from PID 1
[   13.723256] input: gpio_keys as /devices/platform/gpio_keys/input/input0
[   14.137668] hwmon_vid: Unknown VRM version of your CPU
[   14.159472] lm85 0-002e: Starting monitoring
[   14.266694] r8169 0000:01:00.0: assign IRQ: got 38
[   14.266739] pcieport 0000:00:01.0: enabling bus mastering
[   14.266762] r8169 0000:01:00.0: enabling Mem-Wr-Inval
[   14.267601] r8169 0000:01:00.0 (unnamed net_device) (uninitialized): rtl_phya                                                                                       r_cond == 0 (loop: 20, delay: 25).
[   14.307694] orion_wdt: Initial timeout 21 sec
[   14.321694] sd 0:0:0:0: Attached scsi generic sg0 type 0
[   14.333567] marvell-cesa f1030000.crypto: CESA device successfully registered
[   14.439009] r8169: probe of 0000:01:00.0 failed with error -5
[   15.134921] Adding 975868k swap on /dev/sda2.  Priority:-2 extents:1 across:9                                                                                       75868k FS
[   16.101166] random: crng init done
[   16.104596] random: 7 urandom warning(s) missed due to ratelimiting

This is stock uboot

[  329.326325] reboot: Restarting system
▒
         __  __                      _ _
        |  \/  | __ _ _ ____   _____| | |
        | |\/| |/ _` | '__\ \ / / _ \ | |
        | |  | | (_| | |   \ V /  __/ | |
        |_|  |_|\__,_|_|    \_/ \___|_|_|
 _   _     ____              _
| | | |   | __ )  ___   ___ | |_
| | | |___|  _ \ / _ \ / _ \| __|
| |_| |___| |_) | (_) | (_) | |_
 \___/    |____/ \___/ \___/ \__|
 ** MARVELL BOARD: RD-88F6281A LE

U-Boot 1.1.4 (Feb 22 2011 - 10:31:35) Marvell version: 3.4.19

U-Boot code: 00600000 -> 0067FFF0  BSS: -> 006CFEE0

Soc: 88F6281 A1 (DDR2)
CPU running @ 1200Mhz L2 running @ 400Mhz
SysClock = 400Mhz , TClock = 200Mhz

DRAM CAS Latency = 5 tRP = 5 tRAS = 18 tRCD=6
DRAM CS[0] base 0x00000000   size 256MB
DRAM Total size 256MB  16bit width
Addresses 10M - 0M are saved for the U-Boot usage.
Mem malloc Initialization (10M - 7M): Done
NAND:128 MB
Flash:  0 kB

CPU : Marvell Feroceon (Rev 1)
Kernel address is 0x4640000.

Streaming disabled
Write allocate disabled

Module 0 is RGMII
Module 1 is TDM

USB 0: host mode
PEX 0: PCI Express Root Complex Interface
PEX interface detected Link X1
Net:   egiga0, egiga1 [PRIME]
Hit any key to stop autoboot:  0

I'm planning on upgrading to the newer uboot like on my nsa310s which boots perfectly on 4.20!

bodhi, I'm willing to test any fixes you have :) I'll leave the serial hooked up!

---
NSA310s, NSA310, C.H.I.P, Banana Pi, Orange Pi PC & Zero, RPI 1,2Bx8



Edited 1 time(s). Last edit at 02/12/2019 12:41PM by mpmc.

mpmc,

> Yes, I'm getting this on my box as well.

Thanks for letting me know!

Which kernel that worked for you the last time? And is it 4.20.6-kirkwood-tld-1 when you see the problem above?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I hadn't updated the box for a bit I was still on 4.16 I think. I can try with 4.20.6 if you'd like.

EDIT: Sorry I am on 4.20.6!

Edit 2: Looking at kernel.org there are SO many changes to the driver it could be any one of them >.<

---
NSA310s, NSA310, C.H.I.P, Banana Pi, Orange Pi PC & Zero, RPI 1,2Bx8



Edited 3 time(s). Last edit at 02/12/2019 02:33PM by mpmc.

So you have the same problem as mine - it looks, that the newer kernel isn't perfect. Nobody noticed that besides us yet? I couldn't find noone with this problem in the whole google. Weird...

It's just a driver issue :)

---
NSA310s, NSA310, C.H.I.P, Banana Pi, Orange Pi PC & Zero, RPI 1,2Bx8

arti74 Wrote:
-------------------------------------------------------
> So you have the same problem as mine - it looks,
> that the newer kernel isn't perfect. Nobody
> noticed that besides us yet? I couldn't find noone
> with this problem in the whole google. Weird...

This problem might be specific to the NSA310. The new driver version apparently had been tested on diferent box before it got to mainline.

I could try to reverse the changes and let mpmc test it?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Quote

So you have the same problem as mine - it looks, that the newer kernel isn't perfect. Nobody noticed that besides us yet? I couldn't find noone with this problem in the whole google. Weird...

4.20.x is right at the bleeding edge, it hasn't worked its way into much yet. Even debian testing only moved to 4.19 about a month ago though I beleive there is a 4.20 experimental package now. Sadly there seem to be a lot of changes to large groups of devices (like all marvell socs) that don't get tested against all of them and introduce bugs against some, there are also security type fixes that break certain things and don't get noticed right away. I've run into all of the above with with the armada-370 just between 4.14 and 4.19.

If I was going to make an argument for staying with a "stable" version running an LTS kernel this would be it.

So maybe there's a chance that somebody from the developers will notice this problem yet...
I'm still waiting for next versions to try then.

Yeah, usually by the time I can narrow down the issue to the specific patch/commit that caused the problem I find that some has reported it. By then you usually understand the problem well enough to search for it effectively.

@mpmc
I wonder if you have the same motherboard as me maybe? You have the zyxel 310 or the newer 310s?
Could you please show the output of this command?

lspci -nn
00:01.0 PCI bridge [0604]: Marvell Technology Group Ltd. 88F6281 [Kirkwood] ARM SoC [11ab:6281] (rev 03)
01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 03)

Interesting.. it seems that I have very similar ethernet controller in my laptop board?!

01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 06)

And it of course works flawlessly on the newest kernel (r8169 driver)
So the case is the different revision? 03 vs 06.



Edited 1 time(s). Last edit at 02/13/2019 03:06PM by arti74.

@arti74 I have both! The S with 4.20 is fine as it uses a different Ethernet chip

---
NSA310s, NSA310, C.H.I.P, Banana Pi, Orange Pi PC & Zero, RPI 1,2Bx8

And you have an identical chip on your NSA310? (lspci -knn output if possible - also from the NSA310S)



Edited 2 time(s). Last edit at 02/13/2019 04:02PM by arti74.

Arti74,

The NSA310S use the Marvell SoC built in chip (same as the NSA325 and NSA320). The NSA310 use the Realtek chip. These 2 are totally different.

I don't have a lot of time right now so not sure I can get to sit down and reverse the driver change for kernel 4.20.x. That old version from 4.19.x would be a good starting point.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 02/13/2019 04:37PM by bodhi.

Yes, the 4.19.9 works without problems for me as I mentioned before. And I can understand you are a busy man, just thanks for watching this thread.

The new kernel:

uname -rv
4.20.10-1-ARCH #1 PREEMPT Sat Feb 16 00:54:41 UTC 2019

the old behaviour:

dmesg | grep r816
[    2.019182] r8169 0000:01:00.0: enabling device (0140 -> 0143)
[    2.025865] r8169 0000:01:00.0 (unnamed net_device) (uninitialized): rtl_phyar_cond == 0 (loop: 20, delay: 25).
[    2.036246] r8169: probe of 0000:01:00.0 failed with error -5

mpmc,

I've built kernel linux-4.20.6-kirkwood-tld-2, with 4.19.8 version of the r8169 driver. Would you like to give it a shot to see if it will work?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Absolutely when I get out of bed (6am), or maybe arti74 will get a chance before I do :)

I tried to build Linux 5.0.rc7 to test as well but make-kpkg kept hanging on Debian unstable (yes cross compile).

---
NSA310s, NSA310, C.H.I.P, Banana Pi, Orange Pi PC & Zero, RPI 1,2Bx8

Sorry Arti, this will be Debian kernel :) but it would give you a reference test (hopefully if it works) to take back to ALARM forum.

I'll upload the kernel package. Since this is an in-kernel module, I'll have to upload the linux-image to Dropbox (larger than I can attach here).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 02/25/2019 12:14AM by bodhi.

I've removed the upload link in this post, since the kernel version linux-4.20.6-kirkwood-tld-2 is not usable.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 02/26/2019 03:02PM by bodhi.

Yes, I also hope it will work on Debian, then as a reverence I could present it somewhere on Archlinux forum.

Thanks for your work Bodhi.

No luck! It doesn't even detect it anymore, not even a hint of an error, I suppose that's something.

---
NSA310s, NSA310, C.H.I.P, Banana Pi, Orange Pi PC & Zero, RPI 1,2Bx8

OK. I'll take another look.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

mpmc,

1. Go back to kernel linux-4.20.6-kirkwood-tld-1. Reboot and do these following.

Unplug the ethernet cable, and replug. What are dmesg logs when you do that?

2. Go back to kernel linux-4.19.0-kirkwood-tld-1. Reboot. In Debian,

ethtool eth0

Does it show WOL is set, like this?

Quote

Supports Wake-on: g
Wake-on: g

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 02/25/2019 05:25PM by bodhi.

@Arti,

The above tests also applicable to Arch. Basically you would boot the latest (1). And then boot with the know working kernel (2).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

uname -rv
4.20.12-1-ARCH #1 PREEMPT Tue Feb 26 01:20:37 UTC 2019

And of course still no network (with the same error message).
Tried unplug and plug again the cable but either dmesg not showing anything nor the LAN led is not reacting at all (like turned off completely)
I did the same on the "working" kernel, and got this:

[   97.014963] r8169 0000:01:00.0 eth0: Link is Down
[  100.948638] r8169 0000:01:00.0 eth0: Link is Up - 1Gbps/Full - flow control rx/tx

But also the diode is not lit. However network works - I am sure that the leds worked previously - I just don't know now if it was on the 4.19.9 kernel or earlier...

[root@zyxel rtorrent]# ethtool eth0
Settings for eth0:
	Supported ports: [ TP MII ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Half 1000baseT/Full 
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	                        1000baseT/Half 1000baseT/Full 
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  10baseT/Half 10baseT/Full 
	                                     100baseT/Half 100baseT/Full 
	                                     1000baseT/Full 
	Link partner advertised pause frame use: Symmetric Receive-only
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 1000Mb/s
	Duplex: Full
	Port: MII
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	Supports Wake-on: pumbg
	Wake-on: g
	Current message level: 0x00000033 (51)
			       drv probe ifdown ifup
	Link detected: yes

Edited 2 time(s). Last edit at 02/26/2019 03:27AM by arti74.