Fuloong and 2GB of RAM

[Roman Mamedov]

Hello,

I am continuing my experiments with pushing the limits of installed RAM size on
Fuloong/Yeeloong :)

Now I got a 2GB DDR2-800 stick from Hynix, it is a dual-side dual-rank module:
http://www.nix.ru/include/view-photo.html?good_id=87993&pid=2258

With it installed, the Fuloong does not boot with a very strange symptom:
it looks like the kernel can't read anything from the HDD(SSD). The error is
"sda: unknown partition table", so no "/dev/sda2" is found, and the root can't
be mounted: http://romanrm.ru/pics/2011/2011-04-24-fuloong-2gb-boot.jpg

The only way I can boot up with this 2GB RAM stick, is if I first go to the
PMON, enter "set -t highmemsize 768", then load and boot the kernel. That way
it boots up and then works perfectly fine, but of course only 1 GB of RAM is
visible to the kernel.

My guess is that RAM areas are not being set up properly by either PMON or the
kernel with more than 1GB of RAM, and specifically here the DMA reads from the
IDE controller can not be performed correctly.

The PMON version is LM6004-1.3.6, kernels tried are 2.6.38.2 and 2.6.38.3.

--
With respect,
Roman

[晏华]

the  kernel
1.  Does not set the memory map properly.  or
2.  The address window not set up properly.

Xiangyu:
       Could you help to look into this issue.

2011/4/24 Roman Mamedov <r...@romanrm.ru>

--
晏华
龙芯梦兰 软件部

[wu zhangjin]

2011/4/25 晏华 <ya...@lemote.com>:

> the kernel
> 1. Does not set the memory map properly. or
> 2. The address window not set up properly.

I think so.

>
> Xiangyu:
> Could you help to look into this issue.

These two files may be related: arch/mips/loongson/common/mem.c and
arch/mips/loongson/common/pci.c

For the DMA part, we may also need to touch
arch/mips/include/asm/mach-loongson/dma-coherence.h and need to
revisit the address mapping from PCIDMA to DDR(perhaps need to
increase the mapping range, currently, seems it is only the low 256M
memory can be used to do DMA operation, then, if the page cache is
allocated from higher physical space, then, DMA operation will fail).

To RM: for the DMA problem, does libata.force=1:pio4 make the IDE work
in pio mode?

BTW, the whole address mapping is a little confused, we may need to
re-organize it carefully.

Regards,
Wu Zhangjin

> --
> You received this message because you are subscribed to the Google Groups
> "loongson-dev" group.
> To post to this group, send email to loongs...@googlegroups.com.
> To unsubscribe from this group, send email to
> loongson-dev...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/loongson-dev?hl=en.
>

[项宇]

I test 4 2GB ddr2 memory sticks with diffrent brands(sumsung, Adata, MT,
SCS) right now��they all work fine on my Fuloong6003
with pmon version 1.3.6 and kernel version 2.6.27.1 ��

But the 2.6.38.4 kernel not work, I will give the root cause and try to
fix that later��

�� 2011��04��25�� 11:23, wu zhangjin �:
> 2011/4/25 �̻� <ya...@lemote.com>:

>> �̻�
>> ����� �����

>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "loongson-dev" group.
>> To post to this group, send email to loongs...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> loongson-dev...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/loongson-dev?hl=en.
>>

--
Xiang Yu (Software Department)
Tel : 0512-52308631
Jiangsu Lemote Technology Corporation Limited
Menglan, Yushan, Changshu City, Jiangsu Province, China

[项宇]

Hello, RM:
    I do some experiments with 2GB RAM on Fuloong Box。
    The preliminary conclusions is:
    1. Fuloong can't support 2GB RAM with 2 Ranks(up to now, only one special model works fine)
    2. Fuloong can support some 2GB RAM with 1 Rank.(but no warranty about the stablility)

 The 2.6.38 kernel may need little change to support 2GB RAM, which is:

--- a/arch/mips/loongson/common/pci.c
+++ b/arch/mips/loongson/common/pci.c
@@ -54,7 +54,7 @@ static void __init setup_pcimap(void)
         */
        LOONGSON_PCIBASE0 = 0x80000000ul;   /* base: 2G -> mmap: 0M */
        /* size: 256M, burst transmission, pre-fetch enable, 64bit */
-       LOONGSON_PCI_HIT0_SEL_L = 0xc000000cul;
+      LOONGSON_PCI_HIT0_SEL_L = 0x8000000cul;
        LOONGSON_PCI_HIT0_SEL_H = 0xfffffffful;
        LOONGSON_PCI_HIT1_SEL_L = 0x00000006ul; /* set this BAR as invalid */
        LOONGSON_PCI_HIT1_SEL_H = 0x00000000ul;

This new code expand DMA range from 1GB to 2GB.

After communication with our layout engineer and some experts about this issue,
I 'm sure the root cause has relation to layout design and ddr2 controller configuration.
They have fix these problem since 2009, but Fuloong&Yeeloong is designed earlier.
Only adjust DDR2 parameters can't fix this :-[ .

Sorry for this limit.

-- 
Xiang Yu  (Software Department)
Tel	: 0512-52308631
Jiangsu Lemote Technology Corporation Limited
Menglan, Yushan, Changshu City, Jiangsu Province, China

[Roman Mamedov]

On Mon, 25 Apr 2011 16:56:58 +0800
项宇 <xia...@lemote.com> wrote:

> The preliminary conclusions is:
> 1. Fuloong can't support 2GB RAM with 2 Ranks(up to now, only one
> special model works fine)
> 2. Fuloong can support some 2GB RAM with 1 Rank.(but no warranty
> about the stablility)
>
> The 2.6.38 kernel may need little change to support 2GB RAM

Thank you very much!

My Fuloong now boots successfully with your patch on 2.6.38.3 and a dual-rank
2GB DDR2-800 module from Hynix (8+8=16 chips). It seems to work ok, I am
running "memtester" now to check stability, no errors so far.

My guess is that dual-rank module can work here because FL6004 has a different
board design than your FL6003: the DDR2 slot here is turned 180 degrees, so the
row of contacts of DDR2 is much closer to the Loongson2F chip:

* FL6003: http://romanrm.ru/mirrors/lemote-unbox/0438.jpg

* FL6004: http://media.romanrm.ru/loongson/pics/fuloong2f-6004/sideB-take1.jpg
(sorry I don't have a proper photo camera :D)

Perhaps that change was enough to allow dual rank sticks to work properly?

> They have fix these problem since 2009, but Fuloong&Yeeloong is designed
> earlier.

Right, I also tested on the Yeeloong, but it does not even turn-on with a dual
rank 2GB or 4GB stick.

--
With respect,
Roman

[Roman Mamedov]

On Mon, 25 Apr 2011 11:23:40 +0800
wu zhangjin <wuzha...@gmail.com> wrote:

> To RM: for the DMA problem, does libata.force=1:pio4 make the IDE work
> in pio mode?

Sorry I did not get to try that :)
Because this patch fixed the problem and it now works with DMA too.

On Mon, 25 Apr 2011 16:56:58 +0800
项宇 <xia...@lemote.com> wrote:

> --- a/arch/mips/loongson/common/pci.c
> +++ b/arch/mips/loongson/common/pci.c
> @@ -54,7 +54,7 @@ static void __init setup_pcimap(void)
> */
> LOONGSON_PCIBASE0 = 0x80000000ul; /* base: 2G -> mmap: 0M */
> /* size: 256M, burst transmission, pre-fetch enable, 64bit */

> *- LOONGSON_PCI_HIT0_SEL_L = 0xc000000cul;
> + LOONGSON_PCI_HIT0_SEL_L = 0x8000000cul;*

> LOONGSON_PCI_HIT0_SEL_H = 0xfffffffful;
> LOONGSON_PCI_HIT1_SEL_L = 0x00000006ul; /* set this BAR as
> invalid */
> LOONGSON_PCI_HIT1_SEL_H = 0x00000000ul;

--
With respect,
Roman

[wu zhangjin]

On Mon, Apr 25, 2011 at 9:14 PM, Roman Mamedov <r...@romanrm.ru> wrote:
> On Mon, 25 Apr 2011 11:23:40 +0800
> wu zhangjin <wuzha...@gmail.com> wrote:
>
>> To RM: for the DMA problem, does libata.force=1:pio4 make the IDE work
>> in pio mode?
>
> Sorry I did not get to try that :)
> Because this patch fixed the problem and it now works with DMA too.

Ok, If DMA works, why we need the slow pio mode ;-)

Will apply the patch from xiangyu asap.

Thanks & Regards,
Wu Zhangjin

[wu zhangjin]

Does 4GB work on FL6004 with that patch? just want to get more
information about that patch and then determine if apply it or revisit
and apply it.

>
> --
> With respect,
> Roman
>

[Roman Mamedov]

On Mon, 25 Apr 2011 23:47:56 +0800
wu zhangjin <wuzha...@gmail.com> wrote:

> Does 4GB work on FL6004 with that patch? just want to get more
> information about that patch and then determine if apply it or revisit
> and apply it.

Unfortunately I do not have the 4GB stick anymore, as I returned it.

The problems with that stick started even before the kernel (EXTREME slowness
including during the period before PMON menu even shows up), the problems while
trying to boot the kernel were more serious ("IRQ14 nobody cared", repeating
"unable to handle kernel paging request" panics), and I couldn't get it to
boot even by limiting RAM in PMON.

So I guess that even if a 4GB stick can be made to work, it would require not
only this patch, but also changes in PMON.

However as I understood the patch expands the possible DMA area from 1GB to 2GB
-- so is there be anything harmful in making that 4GB right away, just in case
(maybe someone will also want to test a 4GB stick in future)? I think I will
try that out right now.

--
With respect,
Roman

[Roman Mamedov]

On Mon, 25 Apr 2011 22:02:13 +0600
Roman Mamedov <r...@romanrm.ru> wrote:

> However as I understood the patch expands the possible DMA area from 1GB to
> 2GB -- so is there be anything harmful in making that 4GB right away, just
> in case (maybe someone will also want to test a 4GB stick in future)?

> I think I will try that out right now.

Actually no, since it uses 32-bit addresses to specify both "low" and "high"
address of an area, it is not clear to me if it's even possible specify a
non-zero-size area with "low" at 4GB there. :)

--
With respect,
Roman

[项宇]

It is almost impossible to support 4GB RAM on Fuloong&Yeeloong.:-[
Now only Loongson3A can support that.

[Roman Mamedov]

On Mon, 25 Apr 2011 19:10:07 +0600
Roman Mamedov <r...@romanrm.ru> wrote:

> My Fuloong now boots successfully with your patch on 2.6.38.3 and a dual-rank
> 2GB DDR2-800 module from Hynix (8+8=16 chips). It seems to work ok, I am
> running "memtester" now to check stability, no errors so far.

Bad news, after a prolonged test there are some errors detected by memtester.
So it looks like this dual rank module is not completely stable on my system.
I wonder if it is possible to tweak the RAM controller from a running OS, e.g.
change the CAS latency from 5 to 6.

--
With respect,
Roman

[晏华]

There drive strength of loongson2F is not good enough to support big memory in high freequency. That is limited by the chip. So even tune the DDR2 controller setting, perhaps it can not run stable with 2G/4G memories.

2011/4/26 Roman Mamedov <r...@romanrm.ru>

--
晏华
龙芯梦兰 软件部

[Roman Mamedov]

Also here is how memtester sometimes crashes the kernel with this 2GB stick
(even when RAM is limited to 1 GB in PMON). So the dual rank module is really
not usable.

Apr 28 19:57:13 [ 1399.179476] Unhandled kernel unaligned access[#1]:
Apr 28 19:57:13 [ 1399.179954] Cpu 0
Apr 28 19:57:13 [ 1399.180175] $ 0 :
Apr 28 19:57:13 0000000000000000
Apr 28 19:57:13 ffffffffcfffffff
Apr 28 19:57:13 0000000000000000
Apr 28 19:57:13 0000000000000000
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.180812] $ 4 :
Apr 28 19:57:13 98000000bf4b3c10
Apr 28 19:57:13 7bfffe01009a8080
Apr 28 19:57:13 ffffffff804fb534
Apr 28 19:57:13 0000000000000000
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.181457] $ 8 :
Apr 28 19:57:13 7bfffe01009a8080
Apr 28 19:57:13 0000000000000000
Apr 28 19:57:13 0000000000000001
Apr 28 19:57:13 ffffffff8076cfa5
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.182084] $12 :
Apr 28 19:57:13 00000000100044e0
Apr 28 19:57:13 000000001000001e
Apr 28 19:57:13 ffffffff8076cfa8
Apr 28 19:57:13 0000000000000005
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.182707] $16 :
Apr 28 19:57:13 98000000bf4b3c10
Apr 28 19:57:13 7bfffe01009a8080
Apr 28 19:57:13 ffffffffdc840018
Apr 28 19:57:13 0000000000000001
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.183332] $20 :
Apr 28 19:57:13 ffffffff80280f88
Apr 28 19:57:13 ffffffff808b5520
Apr 28 19:57:13 0000000000000003
Apr 28 19:57:13 ffffffff808b5520
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.183953] $24 :
Apr 28 19:57:13 ffffffff808b0000
Apr 28 19:57:13 0000000000000001
Apr 28 19:57:13
Apr 28 19:57:13
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.184583] $28 :
Apr 28 19:57:13 98000000bf4b0000
Apr 28 19:57:13 98000000bf4b3950
Apr 28 19:57:13 7fffffffffffffff
Apr 28 19:57:13 ffffffff8020bae4
Apr 28 19:57:13
Apr 28 19:57:13 [ 1399.185208] Hi : 000000000003e810
Apr 28 19:57:13 [ 1399.185508] Lo : 0000000000036b0e
Apr 28 19:57:13 [ 1399.185835] epc : ffffffff8020b8f0 emulate_load_store_insn+0x370/0x490
Apr 28 19:57:13 [ 1399.186303] Not tainted
Apr 28 19:57:13 [ 1399.186572] ra : ffffffff8020bae4 do_ade+0xd4/0x180
Apr 28 19:57:13 [ 1399.186954] Status: 100044e2
Apr 28 19:57:13 KX
Apr 28 19:57:13 SX
Apr 28 19:57:13 UX
Apr 28 19:57:13 KERNEL
Apr 28 19:57:13 EXL
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.187581] Cause : 10000010
Apr 28 19:57:14 [ 1399.187844] BadVA : 7bfffe01009a8087
Apr 28 19:57:14 [ 1399.188145] PrId : 00006303 (ICT Loongson-2)
Apr 28 19:57:14 [ 1399.188487] Modules linked in:
Apr 28 19:57:14 netconsole
Apr 28 19:57:14 configfs
Apr 28 19:57:14 ipv6
Apr 28 19:57:14 evdev
Apr 28 19:57:14 usbhid
Apr 28 19:57:14 hid
Apr 28 19:57:14 option
Apr 28 19:57:14 usb_wwan
Apr 28 19:57:14 ohci_hcd
Apr 28 19:57:14 usbserial
Apr 28 19:57:14 snd_cs5535audio
Apr 28 19:57:14 snd_ac97_codec
Apr 28 19:57:14 ac97_bus
Apr 28 19:57:14 snd_pcm_oss
Apr 28 19:57:14 snd_mixer_oss
Apr 28 19:57:14 ehci_hcd
Apr 28 19:57:14 snd_pcm
Apr 28 19:57:14 usbcore
Apr 28 19:57:14 snd_timer
Apr 28 19:57:14 snd
Apr 28 19:57:14 r8169
Apr 28 19:57:14 loongson2_cpufreq
Apr 28 19:57:14 soundcore
Apr 28 19:57:14 cs5535_mfd
Apr 28 19:57:14 nls_base
Apr 28 19:57:14 mfd_core
Apr 28 19:57:14 snd_page_alloc
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.190692] Process memtester (pid: 1160, threadinfo=98000000bf4b0000, task=98000000bf5b4920, tls=000000002ae64ae0)
Apr 28 19:57:14 [ 1399.191453] Stack :
Apr 28 19:57:14 ffffffffb73e39bc
Apr 28 19:57:14 ffffffff806ba58c
Apr 28 19:57:14 98000000bf4b3a30
Apr 28 19:57:14 98000000bf4b3a88
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.191959]
Apr 28 19:57:14 98000000bfc27218
Apr 28 19:57:14 0000000000000002
Apr 28 19:57:14 0000000000000000
Apr 28 19:57:14 0000000000000000
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.192585]
Apr 28 19:57:14 ffffffffb73e399c
Apr 28 19:57:14 0000000000000000
Apr 28 19:57:14 0000000000000001
Apr 28 19:57:14 0000000000000000
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.193203]
Apr 28 19:57:14 0000000000045138
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 ffffffff8064bb90
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.193935]
Apr 28 19:57:14 0000000000000000
Apr 28 19:57:14 98000000bfc14700
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 0000000000000000
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.194558]
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 ffffffff806497d4
Apr 28 19:57:14 98000000bfc14700
Apr 28 19:57:14 98000000bfc14700
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.195176]
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 98000000bfbd3658
Apr 28 19:57:14 0000000000000020
Apr 28 19:57:14 98000000bfd04b28
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.195795]
Apr 28 19:57:14 98000000bfc14700
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 98000000bfbd3658
Apr 28 19:57:14 ffffffff806be5b4
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.196416]
Apr 28 19:57:14 98000000bb16cb00
Apr 28 19:57:14 98000000bf086c00
Apr 28 19:57:14 0000000000000020
Apr 28 19:57:14 98000000bfd07b00
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.197034]
Apr 28 19:57:14 98000000bfd04b00
Apr 28 19:57:14 98000000bfc14700
Apr 28 19:57:14 98000000bfc14d28
Apr 28 19:57:14 98000000bf4b3c10
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.197651]
Apr 28 19:57:14 ...
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.198042] Call Trace:
Apr 28 19:57:14 [ 1399.198292] [<ffffffff8020b8f0>] emulate_load_store_insn+0x370/0x490
Apr 28 19:57:14 [ 1399.198751] [<ffffffff8020bae4>] do_ade+0xd4/0x180
Apr 28 19:57:14 [ 1399.199121] [<ffffffff80204480>] ret_from_exception+0x0/0x1c
Apr 28 19:57:14 [ 1399.199545] [<ffffffff804fb534>] timerqueue_add+0x44/0xf0
Apr 28 19:57:14 [ 1399.199955] [<ffffffff80271f90>] __run_hrtimer+0x130/0x2b0
Apr 28 19:57:14 [ 1399.200363] [<ffffffff802723a4>] hrtimer_interrupt+0xe4/0x238
Apr 28 19:57:14 [ 1399.200785] [<ffffffff8020bf68>] c0_compare_interrupt+0x40/0x90
Apr 28 19:57:14 [ 1399.201218] [<ffffffff802adbe8>] handle_IRQ_event+0x80/0x240
Apr 28 19:57:14 [ 1399.201636] [<ffffffff802b0d30>] handle_percpu_irq+0x68/0xf0
Apr 28 19:57:14 [ 1399.202051] [<ffffffff80206318>] do_IRQ+0x38/0x48
Apr 28 19:57:14 [ 1399.202415] [<ffffffff8020449c>] ret_from_irq+0x0/0x4
Apr 28 19:57:14 [ 1399.202789]
Apr 28 19:57:14 [ 1399.202978]
Apr 28 19:57:14 [ 1399.202981] Code:
Apr 28 19:57:14 00431024
Apr 28 19:57:14 1440ff57
Apr 28 19:57:14 00200825
Apr 28 19:57:14 a330007>
Apr 28 19:57:14 6e330000
Apr 28 19:57:14 24110000
Apr 28 19:57:14 1620ff90
Apr 28 19:57:14 00200825
Apr 28 19:57:14 08082daa
Apr 28 19:57:14
Apr 28 19:57:14 [ 1399.204195] Disabling lock debugging due to kernel taint
Apr 28 19:57:14 [ 1399.204599] Kernel panic - not syncing: Fatal exception in interrupt

--
With respect,
Roman