RAM not working on larger chip

[Oliver Seitz]

Hi all!

I seem to have found a severe bug somewhere... No idea where. Could be device file or compiler most likely.

PIC18F27K42 has a great lot of RAM, 8kb. But addresses beyond 0x7ff don't work properly. For a first quick "fix" the device file could be modified to only use 2kb to stop unreliable programs from compiling, but that is of course no cure.

I did test two chips, one of them never been programmed before.

Attached is a small test program with asm and hex files, and also the output.

Greets,

Kiste

[Oliver Seitz]

There's an errata:

When the BSR points to the last bank of the SFR region (BSR=0x3F) and the low byte of the source or destination address of a MOVFF/MOVSF instruction equals the low byte of an indirect addressing operation register address(INDFx, POSTINCx, POSTDECx, PREINCx,PLUSWx), the operation will not be completed as expected. Either, one or more of the destination, FSR value, or location pointed to by the FSR will be corrupted, or the move will simply not occur.

This should however not be the cause of the problem, as the actual data is transferred using MOVWF and MOVF instructions, not a MOVFF.

Greets,

Kiste

[Rob CJ]

Hi Kiste,

Interesting problem. If I have the time I will give it a try this weekend but also test your program on another PIC to see if that makes a difference.

Kind regards,

Rob

---

Van: 'Oliver Seitz' via jallib <jal...@googlegroups.com>
Verzonden: woensdag 11 november 2020 17:00
Aan: 'Oliver Seitz' via jallib <jal...@googlegroups.com>
Onderwerp: Re: [jallib] RAM not working on larger chip

 

--
You received this message because you are subscribed to the Google Groups "jallib" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jallib+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/877417096.4281292.1605110442111%40mail.yahoo.com.

[Oliver Seitz]

Hi Rob!

It does make a difference: On a pic18f26k22, all the RAM seems to be working properly. To my knowledge, the 18f27k42 was the first 8-bit PIC controller with more than 4k of RAM. Yet to test would be 18f[2|4|5]7q43 with 8k and 18f[2|4|5]7q84 with 13kb of RAM. Both again with a different layout, Access bank on these is comprised of parts of bank 5 and 4 of 64 banks in total.

Greets,

Kiste

Am Mittwoch, 11. November 2020, 19:57:33 MEZ hat Rob CJ <rob...@hotmail.com> Folgendes geschrieben:

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/AM0PR07MB62413F3F5E98A409EF58CCADE6E80%40AM0PR07MB6241.eurprd07.prod.outlook.com.

[Rob Hamerling]

Hi Kiste,

On 12/11/2020 12.57, 'Oliver Seitz' via jallib wrote:

Hi Rob!

It does make a difference: On a pic18f26k22, all the RAM seems to be working properly. To my knowledge, the 18f27k42 was the first 8-bit PIC controller with more than 4k of RAM. Yet to test would be 18f[2|4|5]7q43 with 8k and 18f[2|4|5]7q84 with 13kb of RAM. Both again with a different layout, Access bank on these is comprised of parts of bank 5 and 4 of 64 banks in total.

I have a 18F27Q43 here and tried to use your program. That didn't go straight forward: fusedef 'debug' was refused, TX and RX on my test board were at different pins, and the serial lib gave errors (PIR3 and PIE3 had to be changed to PIR4 and PIE4). After the required (provisional) changed the program finally compiled fine! The output is quite different from yours, see attachment, but also not as expected I think!  I doubt if this helps to find the cause of the problem, but at least it proves that it is not unique to the 18f27k42.

Regards, the other Rob!

--
Rob Hamerling, Vianen, NL

[Oliver Seitz]

Hi, the other Rob ;-)

Thanks, looks very interesting. My suspicion so far is that, from a certain address on, the variables are falsely mapped to unimplemented and SFR areas. This is probable for both the k42 and the q43 types, as their SFR pattern is different: q43 has all SFRs densely packed in five banks with few "reserved" spaces in between. k42 has the SFRs scattered over eight banks with lots of "unimplemented, reads as zero" spaces in between. Those patterns resemble parts of the patterns of the wrong variable readings.

Sorry for not making the test program portable, I had to suffer myself: The 18f26k22, which was the only other controller I had which would fit on the board at all, has no pps and no open-drain function. After finally choosing the keyboard over the soldering iron, I made a copy of serial_software.jal and modified it to mimic open-drain behaviour.

Am Donnerstag, 12. November 2020, 16:38:45 MEZ hat Rob Hamerling <robham...@gmail.com> Folgendes geschrieben:

--

You received this message because you are subscribed to the Google Groups "jallib" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jallib+un...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/jallib/0c59a330-f251-c28e-6bd9-eec1e9a759ea%40gmail.com

.

[Rob CJ]

Hi Kiste,

I am playing with your program but do not have a 18f27k42 but I do have a 18f26k42 which has 4k of memory. 

While experimenting I found something strange. The results are different when I use serial_hardware instead of serial_hw_int_cts. The results are still wrong but completely different, see attached file where the two results are listed.

Not sure yet what is happening here. I added some explicit typecasting in large_array_4 but it did not make a difference.

To be continued.

Kind regards,

Rob

---

Van: 'Oliver Seitz' via jallib <jal...@googlegroups.com>

Verzonden: donderdag 12 november 2020 21:14
Aan: jal...@googlegroups.com <jal...@googlegroups.com>

Onderwerp: Re: [jallib] RAM not working on larger chip

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/1955662830.4818762.1605212057510%40mail.yahoo.com.

[Oliver Seitz]

Hi Rob,

serial_hw_int_cts uses a lot of RAM for buffers, so it is likely that the large_array variables are placed differently depending on which serial lib is used.

In the asm file, you can see where the variables are placed:

v__large_array_1_byte_1h       EQU 0x0100  ; _large_array_1_byte_1h

v__large_array_1_byte_2h       EQU 0x0200  ; _large_array_1_byte_2h

v__large_array_1_byte_3h       EQU 0x0300  ; _large_array_1_byte_3h

v__large_array_1_byte_4h       EQU 0x0400  ; _large_array_1_byte_4h

v__large_array_1_byte_5h       EQU 0x0500  ; _large_array_1_byte_5h

I've seen that variables with starting address from 0x0800 on will not work. The obvious difference in your files is at the end of array_4, I presume that last array has different locations depending on which serial lib you used.

Greets,

Kiste

(En nog extra groetjes naar de ander Rob. Heb ik vergeten de laatste keer ;-) )

Am Donnerstag, 12. November 2020, 21:35:59 MEZ hat Rob CJ <rob...@hotmail.com> Folgendes geschrieben:

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/AM0PR07MB62412E4C90417CF19C32BC81E6E70%40AM0PR07MB6241.eurprd07.prod.outlook.com.

[Rob Hamerling]

Hello Kiste,

On 12/11/2020 21.57, 'Oliver Seitz' via jallib wrote:

serial_hw_int_cts uses a lot of RAM for buffers, so it is likely that the large_array variables are placed differently depending on which serial lib is used.

This library reserves (by default) 32 + 64 bytes for output and input buffering. Not much compared to the large arrays, and if you want they can be reduced.
Furthermore: the serial buffers are located in the first data bank among other variables, while the large arrays are located in completely free banks.
So I don't think (the locations of) the serial buffers are not of influence.

Regards, Rob

[Oliver Seitz]

Hi again :-)

In this case I think it is: Rob CJ modified the program to run on a chip with only 4kb of RAM, three large_arrays of 1024 bytes each and one large_array of 448 bytes. This makes 13 base arrays of 256 bytes each (filling banks 1 to 14) and one array whith the remaining 192 bytes. This last array would fit to bank 0 if there are no serial buffers there. With the serial buffers, this last array would go to bank 15.

By increasing the size of the fourth large array to 512 bytes, thus forcing the last base array with 256 bytes into its own bank, Rob probably would get a result similar to the first run with serial_hw_int_cts. 

Greets,

Kiste

Am Freitag, 13. November 2020, 12:09:13 MEZ hat Rob Hamerling <robham...@gmail.com> Folgendes geschrieben:

--

You received this message because you are subscribed to the Google Groups "jallib" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jallib+un...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/jallib/11de9a37-949e-00c3-f457-e72681011fa5%40gmail.com

.

[Oliver Seitz]

Hi!

This could be the origin of the problem. Here's the bit codes of the machine instructions which load a literal value to a file select register:

18f26k22: LFSR 1110 1110 00ff kkkk - 1111 0000 kkkk kkkk

18f27k42: LFSR 1110 1110 00ff kkkk - 1111 00kk kkkk kkkk

18f27q43: LFSR 1110 1110 00ff kkkk - 1111 00kk kkkk kkkk

 

k22 has only 12 bit deep FSR, while the ones with the larger RAM have 14 bit. The low byte is the same on all chips, but the high bits are arranged differently. While on k22 bits 11,10,9 and 8 are in the lowest nibble of the first word, that nibble on the larger chips hold bits 13,12,11 and 10. Bits 9 and 8 are in locations which were defined as zero on k22. 

This means, if the compiler only generates the k22 form of the command, banks are skipped. Only banks 0,4,8,12... are addressed. 

Unpopulated RAM locations on the k42 start at 0x2000. Divided by four due to the lost bits that region would be adressed from 0x0800 on, which is exactly what the test program showed.

Greets,

Kiste

[Rob Hamerling]

I haven't gone into these details, but your analysis sounds plausible. I'm afraid I cannot help with compiler issues.

Regards, Rob.

[Rob CJ]

Hi Kiste, Rob,

I had a short look and found this code in the compiler sources:

    case PIC_OPCODE_LFSR:

      {

        unsigned base;

        value_t  bval;

        bval = pic_code_value_get(code);

        base = value_base_get(bval)

              + value_const_get(value_baseofs_get(bval))

              + pic_code_ofs_get(code);

        pcode_lo = 0xee00 | (pic_code_fsr_n_get(code) << 4) | (base >> 8);

        pcode_hi = 0xf000 | (base & 0x00ff);

        ct = 2;

      }

If I know a reliable way to detect the other PIC types I could change this code generation part for that specific situation to generate the right LFSR for those PICs. Not sure if this is all I need to do, I only had a short look.

But what I need to know for which PIC types they changed this LFSR. Is that always when the data space is >4k or is it with PICs that have 64 banks? 

Kind regards,

Rob

---

Van: jal...@googlegroups.com <jal...@googlegroups.com> namens Rob Hamerling <robham...@gmail.com>
Verzonden: vrijdag 13 november 2020 14:18

Aan: jal...@googlegroups.com <jal...@googlegroups.com>
Onderwerp: Re: [jallib] RAM not working on larger chip

--

You received this message because you are subscribed to the Google Groups "jallib" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jallib+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/1a4940ae-ade2-0ceb-7fbe-59a90993f287%40gmail.com.

[Oliver Seitz]

Hi Rob,

I've found a probably reliable hint in the device files:

var volatile bit*6  FSR0H_FSR0H               at FSR0H : 0

or

var volatile bit*4  FSR0H_FSR0H               at FSR0H : 0

As far as I have seen, all 18f*-devices have such a definition in the device file.

It is only 57 of 486 18f*-device files which declare bit*6.

Greets,

Kiste

Am Freitag, 13. November 2020, 18:21:52 MEZ hat Rob CJ <rob...@hotmail.com> Folgendes geschrieben:

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/AM0PR07MB62410D9CB95A613C3657C28DE6E60%40AM0PR07MB6241.eurprd07.prod.outlook.com.

[Oliver Seitz]

Sorry, it would have been too easy... But there's something else, 

FSR0H_SHAD_FSR0H

seems to be defined only for devices and for all devices with 14 bit wide FSRs.

Greets,

Kiste

Am Freitag, 13. November 2020, 19:10:27 MEZ hat 'Oliver Seitz' via jallib <jal...@googlegroups.com> Folgendes geschrieben:

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/1636633074.5199862.1605290913916%40mail.yahoo.com.

[Rob CJ]

Hi Kiste,

Thanks for the info. I still wonder if it also valid for PICs that have 64 banks (did not check that yet). If so I could update the device files for the 18F series with the pragma numbanks like I did for the PIC14H device files. I do already have the check in the compiler for this pragma so that would be a nice solution.

I made a quick hack to the compiler source to see if it works but since I only have a PIC18F47Q43 I need to get your program running on it. Unfortuntely I ran into compatibility issues with all serial libraries that do not compile for this PIC. The easiest thing for me to do was to 'hack' the serial software library but nothing comes out yet.

So if this issue can be fixed in the compiler, all serial libraries also need some rework too.

I will keep you posted. Weekend is there ....

Kind regards,

Rob

---

Van: 'Oliver Seitz' via jallib <jal...@googlegroups.com>
Verzonden: vrijdag 13 november 2020 19:35
Aan: 'Oliver Seitz' via jallib <jal...@googlegroups.com>

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/1877556776.5199977.1605292524536%40mail.yahoo.com.

[Rob Hamerling]

Hi Rob,

On 13/11/2020 19.55, Rob CJ wrote:

Thanks for the info. I still wonder if it also valid for PICs that have 64 banks (did not check that yet). If so I could update the device files for the 18F series with the pragma numbanks like I did for the PIC14H device files. I do already have the check in the compiler for this pragma so that would be a nice solution.

I made a quick hack to the compiler source to see if it works but since I only have a PIC18F47Q43 I need to get your program running on it. Unfortuntely I ran into compatibility issues with all serial libraries that do not compile for this PIC. The easiest thing for me to do was to 'hack' the serial software library but nothing comes out yet.

So if this issue can be fixed in the compiler, all serial libraries also need some rework too.

I'll send you my provisionally modified serial-hw-int-cts library, which works with my 18f27q43. And indeed the serial_hw2_int_cts library needs an update too because of different PIEx en PIRx locations for TXIE/IF and RXIE/IF. If I can do something else, let me know.

[Oliver Seitz]

Hi Rob,

I'd really like to help... but I would prefer to have a look into the .pic-files the device files are generated from. I only have MPLAB IDE on my old little laptop I'm using for pic programming. No .pic-files there. I tried installing the linux version of the XC compiler, but even though it is titled "Linux 32-Bit and Linux 64-Bit", it won't install on my 32bit system.

So I dug up a small machine with Windows 10 on it. However, when I double-click the installer it tells me "This software cannot run on this machine. Please ask the maufacturer for a compatible version." No more information, I can only guess that again the problem is, that it is a 32bit system.

I think I'll have to power up my big old intel x64 linux machine... Will take a while.

In the jallib device files there are entries like "(banks=64)", but e.g. on the q43 devices only the populated banks are counted. So, for the 18f25q43, which definitely has 14 bit FSRs, "(banks=14)" would be of no help. 

Greets,

Kiste

Am Freitag, 13. November 2020, 19:55:56 MEZ hat Rob CJ <rob...@hotmail.com> Folgendes geschrieben:

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/AM0PR07MB624135B540E97D9C116B2679E6E60%40AM0PR07MB6241.eurprd07.prod.outlook.com.

[Rob Hamerling]

Hi Kiste, Rob,

On 14/11/2020 08.52, 'Oliver Seitz' via jallib wrote:

I'd really like to help... but I would prefer to have a look into the .pic-files the device files are generated from.

@Kiste: Have a look in the jallib/tools directory. The script mplabxtract.py contains the locations where you can find the .PIC files (win, linux, mac). You might run this script to get more human-readable xml files.

As far as I have seen all PICs of a family with a member with more than 16 memory banks have the following properties:
1.   ArchDef -> MemTraits -> bankcount="64"
2.   InstructionSet -> instructionsetid="cpu_pic18f_v6"
So also for example also the 18f05q40 with 'only' 14 memory banks has these properties.
I counted 51 PICs with these properties, and probably evenly important: no other PICs have one of these properties.

[Rob CJ]

Hi Rob, Oliver,

So the safest way to detect which instruction to use is maybe to incorporate the instructionset in the device file as pragma.

@Kiste. If  you use a Windows computer the .PIC files can be found at:

C:\Program Files\Microchip\MPLABX\v5.45\packs\Microchip

Since MPLABX version 5.45 the Windows application is 64-bit so will not run on a 32-bit Windows machine anymore. 

I got the program working on a PIC18F47Q43 with the help of the other Rob and did some slide modifications to the output.

I ran it with the current compiler version (run 1) and a version with the modification (run 2) but that did not make a difference. I may have made the wrong modification in the compiler so need to check that first. See attached runs.

Kind regards,

Rob

---

Van: jal...@googlegroups.com <jal...@googlegroups.com> namens Rob Hamerling <robham...@gmail.com>

Verzonden: zaterdag 14 november 2020 10:44

Aan: jal...@googlegroups.com <jal...@googlegroups.com>
Onderwerp: Re: [jallib] RAM not working on larger chip

--

You received this message because you are subscribed to the Google Groups "jallib" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jallib+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/4df78b60-d781-d359-140f-b898d358a19c%40gmail.com.

[Rob CJ]

Hi Kiste, Rob,

I fixed it, I forgot to shift some bits so it seems to work, see the new run 2 file.

The only I need to do now is to detect the right instructionset. 

@Rob. I think you made an overview of instructionsets some time ago. Could you send it to me?

Thanks.

Kind regards,

Rob

---

Van: jal...@googlegroups.com <jal...@googlegroups.com> namens Rob CJ <rob...@hotmail.com>
Verzonden: zaterdag 14 november 2020 12:37

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/VI1PR07MB6256A3143D8B0B9CDE8B3B34E6E50%40VI1PR07MB6256.eurprd07.prod.outlook.com.

[Oliver Seitz]

Hey Rob,

looks great!

Greets,

Kiste

Am Samstag, 14. November 2020, 12:52:04 MEZ hat Rob CJ <rob...@hotmail.com> Folgendes geschrieben:

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/VI1PR07MB6256EBDE47219BFA62D05DF7E6E50%40VI1PR07MB6256.eurprd07.prod.outlook.com.

[Rob Hamerling]

Hi Rob,

On 14/11/2020 12.52, Rob CJ wrote:

Hi Kiste, Rob,

I fixed it, I forgot to shift some bits so it seems to work, see the new run 2 file.

The only I need to do now is to detect the right instructionset. 

@Rob. I think you made an overview of instructionsets some time ago. Could you send it to me?

Your results are promising! Hopefully it is backward compatible, such that code for other PICs is untouched.
I'll send you per PM the script and its output to determine the instruction set of all PICs.

[Oliver Seitz]

Hi!

I'm checkin on another base of data. I've downloaded 244 datasheets, 66 of which contain more than 1* occurrence of "LFSR". All of those explain one of two possible bitfield patterns, which are the patterns for 12 and 14 bit we discussed.

Manually reading the results, I can confirm that only these families do have 14 bit:

*k42

*k83

*q40

*q41

*q43

*q84

That is, if I did catch all of the relevant data sheets.

Greets,

Kiste

* 1 occurrence is used as abbreviation for "Linear Feedback Shift Register" 

[Rob CJ]

Hi Kiste,

Thanks for the info. Must have been some work to find that out 🙂.

I made changes to the Python script that generate the device files and I added a new pragma for the compiler.

In onder to determine for which PIC I need to change the instruction set I used the instruction set information provided by Microchip. The other Rob has sent me a list of all PICs and which instruction set they use and based on that I could also confirm that this is the way forward.

I am running some tests and it seems that it works OK. If I have sufficient time I will finish it this weekend and maybe be on time to upload it all for the next bee package.

As said I will also update all device files with the new pragma and a new build of the compiler since only the new built compiler can handle it.

Kind regards,

Rob

---

Van: 'Oliver Seitz' via jallib <jal...@googlegroups.com>

Verzonden: dinsdag 17 november 2020 16:40

Aan: jal...@googlegroups.com <jal...@googlegroups.com>
Onderwerp: Re: [jallib] RAM not working on larger chip

--
You received this message because you are subscribed to the Google Groups "jallib" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jallib+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jallib/1306993254.6549348.1605627636131%40mail.yahoo.com.

[rob...@hotmail.com]

Hi Kiste,

I fixed the issue. It will be available in the next bee-package. Note that both the compiler and many device files where changed for this fix so make sure to install the whole bee-package.

If you cannot wait for that you could download the new compiler and your specific device file from GitHub.

Kind regards,

Rob

Op woensdag 11 november 2020 om 16:37:38 UTC+1 schreef Kiste: