Linux >= 4.2.1 / < 4.3 - ext4 dir_index / scheduler and other related issues on ARM device(s)

I'm using kernel 4.2.1 on all my devices without problems.
I decided to test newer kernels (4.2.2 / 4.2.3 ) but now the trouble starts.

First I noticed that I couldn't get files read from my samba share via mobile phone (es file manager)
no error message, just no files or directories are visible.
Same on my pc.
Accessing the files via ssh on the box(es) sometimes work, sometimes not.

http://pastebin.com/JSpTBKXW

This is happening sometimes when accessing the mounted root directory of the drives and mostly
The directory which raises this up contains 3366 directories, 31574 files (small audio files)

The filesystem is intact, it's working with kernel 4.2.1 and earlier and again with 4.3.0-rc1 and later
(4.3.0-rc6 atm on this box)

Because of this, I cannot update my other kirkwood devices

Any hints?

I could wait till kernel 4.3 is ready, shure.



Edited 2 time(s). Last edit at 12/10/2015 03:59PM by pengu.

I see that someone has similar problems with kernel 4.2.1 when using nfs + rsync
(two ext4 formatted disk, raid 0).

did someone has a similar setup?
I can't reproduce this because I have to (again) backup my HDDs.
After that I can perhaps check if this is the cause on my NSA325v2

Perhaps NFS is faster then SAMBA?



Edited 1 time(s). Last edit at 12/10/2015 03:58PM by pengu.

pengu,

Doesn't hurt to try NFS to see if you got the same error. It is faster.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

thanks bodhi,

I got the kernel log from him and voila .. I had the same issue on some devices

(DNS-325 and ix2-200)

ec 10 21:23:27 debianix2 kernel: blk_queue_start_tag: request c12c70f0 for device [sda] already tagged 27
Dec 10 21:23:27 debianix2 kernel: ------------[ cut here ]------------
Dec 10 21:23:27 debianix2 kernel: kernel BUG at block/blk-tag.c:324!
Dec 10 21:23:27 debianix2 kernel: Internal error: Oops - BUG: 0 [#1] PREEMPT ARM
Dec 10 21:23:27 debianix2 kernel: Modules linked in: ipv6 adt7475 hwmon_vid evdev marvell_cesa des_generic mv_cesa orion_wdt lm63 uio_pdrv_genirq uio gpio_keys raid0 md_mod sg sd_mod uas usb_storage sata_mv libata scsi_mod
Dec 10 21:23:27 debianix2 kernel: CPU: 0 PID: 945 Comm: jbd2/md0-8 Not tainted 4.2.1-kirkwood-rm-2 #3
Dec 10 21:23:27 debianix2 kernel: Hardware name: Marvell Kirkwood (Flattened Device Tree)
Dec 10 21:23:27 debianix2 kernel: task: ce948b00 ti: cfa76000 task.ti: cfa76000
Dec 10 21:23:27 debianix2 kernel: PC is at blk_queue_start_tag+0x4c/0x1a0
Dec 10 21:23:27 debianix2 kernel: LR is at blk_queue_start_tag+0x4c/0x1a0
Dec 10 21:23:27 debianix2 kernel: pc : [<c030bba0>]    lr : [<c030bba0>]    psr: 20000013\x0asp : cfa77d20  ip : 00000002 fp : 00000001
Dec 10 21:23:27 debianix2 kernel: r10: cfa76000  r9 : c07803e0  r8 : c12c70f0
Dec 10 21:23:27 debianix2 kernel: r7 : c12f7b38  r6 : c12f7b38  r5 : c120f0e0  r4 : c12c70f0
Dec 10 21:23:27 debianix2 kernel: r3 : cfa76000  r2 : 00000001  r1 : 00000000  r0 : 00000048
Dec 10 21:23:27 debianix2 kernel: Flags: nzCv  IRQs on  FIQs on Mode SVC_32  ISA ARM  Segment kernel
Dec 10 21:23:27 debianix2 kernel: Control: 0005397f  Table: 0fa0c000  DAC: 00000017
Dec 10 21:23:27 debianix2 kernel: Process jbd2/md0-8 (pid: 945, stack limit = 0xcfa76190)
Dec 10 21:23:27 debianix2 kernel: Stack: (0xcfa77d20 to 0xcfa78000)
Dec 10 21:23:27 debianix2 kernel: 7d20: 0000001b c12c70f0 c106b000 c1251800 0000000e c12f7b38 c12c70f0 c07803e0
Dec 10 21:23:27 debianix2 kernel: 7d40: cfa76000 bf009b48 c12f7b38 c106b488 00000001 c12f7b38 c11a8e00 c12c70f0
Dec 10 21:23:27 debianix2 kernel: 7d60: c1302578 00000002 c07803e0 24410411 00000001 c03071ac c1302ad0 c0327a88
Dec 10 21:23:27 debianix2 kernel: 7d80: 00000002 c12c70f0 cfa77ed4 c10f3c00 c12c002f c030a8f0 c12c70f0 cfa77ed4
Dec 10 21:23:27 debianix2 kernel: 7da0: cfa77e38 00000000 c03050c4 c033c4f8 00000000 00000000 c10f3c00 c12c70f0
Dec 10 21:23:27 debianix2 kernel: 7dc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Dec 10 21:23:27 debianix2 kernel: 7de0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 c12c70f0
Dec 10 21:23:27 debianix2 kernel: 7e00: c12f7b38 00000000 c12c70f0 002f0000 60000013 c12c70f0 c12f7b38 00000000
Dec 10 21:23:27 debianix2 kernel: 7e20: cfa77e38 00000000 60000013 00000000 00000000 c030b3dc cfa77e38 cfa77e38
Dec 10 21:23:27 debianix2 kernel: 7e40: cfa77e40 cfa77e40 00000000 c119a000 cee2df20 00000000 cee2df4c 00000000
Dec 10 21:23:27 debianix2 kernel: 7e60: 00200200 c030b7c0 c1329000 c01f0a98 0e88cd17 00000000 c07803e0 00000fdc
Dec 10 21:23:27 debianix2 kernel: 7e80: ce973000 ffffffff 00000008 00000000 377b344f 00000696 00000004 00000000
Dec 10 21:23:27 debianix2 kernel: 7ea0: 377b344f 00000696 ce948b30 00000000 00003f07 000140f4 0e88cd17 00000000
Dec 10 21:23:27 debianix2 kernel: 7ec0: 00000002 c83d0f2c c83d0f2c c5cf55ac c5cf55ac cfa77ed4 cfa77ed4 cfa77edc
Dec 10 21:23:27 debianix2 kernel: 7ee0: cfa77edc cfa77ee4 cfa77ee4 0001446b 00000001 00000000 0000000c 0000000c
Dec 10 21:23:27 debianix2 kernel: 7f00: 00000000 00000000 001a7b89 c119a210 00000001 00000002 00000000 c119a000
Dec 10 21:23:27 debianix2 kernel: 7f20: cfa76000 c119a038 c119a1fc c119a040 00000000 00000000 00000000 c01f5dc4
Dec 10 21:23:27 debianix2 kernel: 7f40: cfa76000 00000000 ce948b00 c00484a0 cfa77f50 cfa77f50 c10396e0 00000000
Dec 10 21:23:27 debianix2 kernel: 7f60: c119a000 c01f5cf0 00000000 c003a07c 00000000 00000000 000000f8 c119a000
Dec 10 21:23:27 debianix2 kernel: 7f80: 00000000 cfa77f84 cfa77f84 00000000 cfa77f90 cfa77f90 cfa77fac c10396e0
Dec 10 21:23:27 debianix2 kernel: 7fa0: c0039fb8 00000000 00000000 c000ef20 00000000 00000000 00000000 00000000
Dec 10 21:23:27 debianix2 kernel: 7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
Dec 10 21:23:27 debianix2 kernel: 7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
Dec 10 21:23:27 debianix2 kernel: [<c030bba0>] (blk_queue_start_tag) from [<bf009b48>] (scsi_request_fn+0x204/0x7b8 [scsi_mod])
Dec 10 21:23:27 debianix2 kernel: [<bf009b48>] (scsi_request_fn [scsi_mod]) from [<c03071ac>] (__blk_run_queue+0x34/0x44)
Dec 10 21:23:27 debianix2 kernel: [<c03071ac>] (__blk_run_queue) from [<c0327a88>] (cfq_insert_request+0x3d8/0x4a0)
Dec 10 21:23:27 debianix2 kernel: [<c0327a88>] (cfq_insert_request) from [<c030b3dc>] (blk_flush_plug_list+0x198/0x1d0)
Dec 10 21:23:28 debianix2 kernel: [<c030b3dc>] (blk_flush_plug_list) from [<c030b7c0>] (blk_finish_plug+0x28/0x44)
Dec 10 21:23:28 debianix2 kernel: [<c030b7c0>] (blk_finish_plug) from [<c01f0a98>] (jbd2_journal_commit_transaction+0xdf0/0x1978)

On ix2-200 .. it's happening with two HDDs (2 TB WD green, 500 GB WD blue) .. and now the third (an 1 TB WD green) .. ;-(
The HDDs were ok.. no bad SMART values .. no other errors when using them on my main system.

I cannot test newer kernels on DNS-325 because it recognizes only one hdd with kernel 4.3
doesn't matter which HDD I install
I've got a second DNS-325, let's see ...


In my NSA325v2 .. all fine but I'm using kernel 4.3 for weeks now and now kernel 4.4-rc2 ...
I hope that this all is only kernel 4.2 and not hardware related ;-(



Edited 1 time(s). Last edit at 12/10/2015 04:01PM by pengu.

pengu,

> In my NSA325v2 .. all fine but I'm using kernel
> 4.3 for weeks now and now kernel 4.4-rc2 ...
> I hope that this all is only kernel 4.2 and not
> hardware related ;-(

Then the quickest test is to go back to kernel 4.2 on NSA325v2! You still have this kernel installed? then switch uImage and uInitrd to that version and reboot.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

yeah I still have this kernel installed, I only need to rename the uImage / uInitrd.

but atm .. the 4.3.1 is building .. let's see when it's finished


I'll try your and my first one (4.2.1)
perhaps something is gone wrong ...

what I can say .. regarding my first kernel (4.2.1)
it's working on all my boxes .. the only thing is the second one but as I know, it's also there with 4.2.0

Edit

News from the ix2-200 (nfs, rsync)

Now let's see what's going on on the NSA325v2 with 4.2.0 and 4.2.1 kernels.

Edit 2

perhaps it's similar to this


Edit 3

reading / writing small and big files is working fine with 4.2.1
The same kernel is used by another one on his NSA325(v2) and also no troubles



Edited 4 time(s). Last edit at 12/11/2015 03:43AM by pengu.

The mentioned HDD is now available, so let's see.

On both mentioned devices (ix2-200 and DNS-325) .. also on other (Netgear Stora) I see some flash timing problems.
Pehaps this could be the same here?
some wrong timings in the DTB?


One thing that makes me beleve that there could be something wrong in the DTB is that on all ZyXEL devices I don't see anything like this.

I've used a NSA320 for months .. no problem
after that .. Seagate GoFlex Net (with two 2,5 '' HDDs) .. no problems ..
Now, no issues with the NSA325v2 (except the ext4 problem when using kernels > 4.2.1 and < 4.3.0.
Perhaps this is gone now, because one of the used WD Red is gone defective so I had to replace both.
I'm using the two 2,5'' Toshiba vor now ...


the mentioned HDD is installed (ix2-200) and while writhing zeroes to the drive, the above mentioned problem rises again as expected.

So I'm booting my built 4.3 kernel and now it seems to work

[    0.000000] Linux version 4.3.0-kirkwood-rm-1 (root@debianNSA325v2) (gcc version 4.9.2 (Debian 4.9.2-10) ) #1 PREEMPT Mon Nov 2 12:11:21 CET 2015
[    0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=0005397f
[    0.000000] CPU: VIVT data cache, VIVT instruction cache
[    0.000000] Machine model: Iomega StorCenter ix2-200

lsscsi
[0:0:0:0]    disk    ATA      WDC WD20EARS-00J 0A80  /dev/sda 
[2:0:0:0]    disk    General  USB Flash Disk   1100  /dev/sdb

Tomorrow I'll see if the device is still up ;-)



Edited 1 time(s). Last edit at 12/14/2015 03:12PM by pengu.