How would we handle multicore MCUs with a mix of core types?

[Nathan Hartman]

Previously there was a discussion about thinking 10 years ahead and

what kinds of microcontrollers might exist.

It seems to me that we should expect a mix of multiple cores to become

commonplace. This has already begun.

For example, STM32H747AI has an Arm Cortex-M7 and an Arm Cortex-M4.

How would a scenario like this be handled? Would it require two builds

and two operating system images?

[patacongo]

It seems to me that we should expect a mix of multiple cores to become
commonplace. This has already begun.

That has been going on for years.  The LPC43 was the first I heard of and that was several years ago, I am guessing around 2014.

For example, STM32H747AI has an Arm Cortex-M7 and an Arm Cortex-M4.

How would a scenario like this be handled? Would it require two builds
and two operating system images?

In most cases yes.  That most cases where there are two ISA incompatible architectures like and M4 and an M0.  I don't know about "almost" compatible architectures like an M7 and an M4.

I am really only aware to two multi-core approaches:

1) AMP where you would use to separate builds with some inter-CPU communication framework like OpenAMP.

2) And SMP which is pretty well supported by NuttX but requires a command shared address space and completely symmetric CPU architectures.

I don't know what you would do with an M7 and an M4.  I suspect that there would be many risks in trying an SMP solution so AMP would probably be more appropriate.

I have heard of people doing multiple CPU implementations using OpenAMP.  That is something outside of the scope of the OS since in that configuration, the OS is a component of the configuration, not the controller of the configuration.

Greg

[patacongo]

On Wednesday, October 30, 2019 at 10:29:55 AM UTC-6, patacongo wrote:

It seems to me that we should expect a mix of multiple cores to become
commonplace. This has already begun.

That has been going on for years.  The LPC43 was the first I heard of and that was several years ago, I am guessing around 2014.

Even older than I though:  "December 2011, NXP announced the LPC4300 series, the first dual-core chip with an ARM Cortex-M4F and ARM Cortex-M0" https://en.wikipedia.org/wiki/NXP_LPC

[patacongo]

It seems that there should be some OS involved in the OpenAMP inter-CPU communications (MCAPI).  That is probably something that should be considered.

It should be possible to bring in Mentor Graphics OpenMCAPI implementation:  https://bitbucket.org/hollisb/openmcapi/wiki/Home, https://bitbucket.org/hollisb/openmcapi/wiki/Home.  That Wiki says, however:  "The MCAPI specification defines an application-layer messaging API, rather than a kernel networking protocol (e.g. it does not involve sockets)."  But this it goes on to talk about Linux kernel drivers for the Linux port?

I don't really know enough to speak about OpenAMP or MCAPI.

Greg

[patacongo]

I don't know what you would do with an M7 and an M4.  I suspect that there would be many risks in trying an SMP solution so AMP would probably be more appropriate.

Cache coherency would certainly be an issue.

[Jerome Kasper]

Hello Greg, Nathan,

I think i answered to Nathan only because of my mail client. I asked an alike question to Greg some times ago . I am still very interested about working toward the OpenAMP implementation. i have the specs and some hardware around to work with. Maybe we could set up something like a trello to organize multi peoples developpment ?

Anyway i'd like to take part to that implementation by taking small steps, hence my ask for an organization structure like trello (it's free) and also it would make easier to share the work.

Feel Free to contact me privately if necessary :)

Best Regards,
Jerome KASPER

--
You received this message because you are subscribed to the Google Groups "NuttX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/e93eca5a-31ef-4e31-b895-640118cbc02e%40googlegroups.com.

[Nathan Hartman]

This all makes sense.

I didn't even know the term AMP (asymmetric multi processing) although

I probably should have guessed that, because it's the asymmetric

counterpart to SMP. I learned something today. Now I can go home. :-p

Previously we built a board that contained several MCUs that

communicated with each other through serial communications (a very

complicated beast of a board). It looks like AMP takes this idea,

shrinks it down to a single chip, and gives it a much faster

communication mechanism, but so far it looks like the same idea.

Jerome Kasper replied to me privately earlier but I see now a message

to the group. Jerome gave me a link to OpenAMP

(https://www.openampproject.org/) and I have been looking at that for a

little while. I will take a look at the Mentor Graphics OpenMCAPI

project you mentioned as well. Knowing the terms AMP and MCAPI has just

opened the door to a whole new rabbit hole.

I agree that you can't run one OS image on non-identical cores. This is

definitely a multi-RTOS image situation -- or alternately one core

running RTOS, the other core running some sort of bare metal code.

On Wednesday, October 30, 2019 at 12:52:56 PM UTC-4, patacongo wrote:

> It seems that there should be some OS involved in the OpenAMP inter-

> CPU communications (MCAPI).  That is probably something that should

> be considered.

Yes. It looks like each core needs some sort of driver that acts as a

doorway to the other core, and then applications on each core can use

that driver to communicate with each other.

The reason I asked (other than being curious about the future direction

after the last conversation) is that I need to select a MCU for the

next version of my product and I'm trying to think ahead and

"overspecify" the MCU so there will be room to grow. (That's also in

line with what we discussed previously, giving NuttX room to grow,

too.) The current generation uses a 120 MHz Arm Cortex M4. For the next

generation, I'm considering a 480 MHz Arm Cortex M7 from the STM32H7

family. This family is interesting for a variety of reasons. Some parts

in the family have a 2nd Arm Cortex M4 core; others have just the

single M7 core.

Obviously I have to weigh the potential benefits of this second core

against the up-front investment in time, headache, $$$, etc.

Right now, I don't know if this is the direction... Just trying to get

a feel for how deep the rabbit hole goes.

Thanks for all your thoughts and input.

Nathan

[Gregory Nutt]

> Previously we built a board that contained several MCUs that
> communicated with each other through serial communications (a very
> complicated beast of a board). It looks like AMP takes this idea,
> shrinks it down to a single chip, and gives it a much faster
> communication mechanism, but so far it looks like the same idea.

Yes, the OS is not deeply involved with AMP designs because there
multiple copies of the OS in the architecture, each running independently.

> > It seems that there should be some OS involved in the OpenAMP inter-
> > CPU communications (MCAPI).  That is probably something that should
> > be considered.
>
> Yes. It looks like each core needs some sort of driver that acts as a
> doorway to the other core, and then applications on each core can use
> that driver to communicate with each other.

On the surface, I think that the OpenMCAPI port would simply amount to
re-implementing the OpenMCAPI Linux drivers to run in NuttX.  Those
Linux Drivers are dual license GPL/BSD so probably could not come into
the OS directly; they would need to be re-implemented to avoid any
possibility of GPL taint.  I don't even want those three letters
together in any file in the repository.

> Obviously I have to weigh the potential benefits of this second core
> against the up-front investment in time, headache, $$$, etc.

I would avoid the complexity unless it is necessary to the solution.

> Right now, I don't know if this is the direction... Just trying to get
> a feel for how deep the rabbit hole goes.

No really so deep from the NuttX point of view.  Other than some
innocuous drivers, the solution is outside of the OS.

[Gregory Nutt]

>> Obviously I have to weigh the potential benefits of this second core
>> against the up-front investment in time, headache, $$$, etc.
> I would avoid the complexity unless it is necessary to the solution.

In other asymmetric multi-core solutions that I have looked at, one core
is specialize to have very low power consumption (hence, usually an
M0+).  So in very low power modes you just shut down the big pig CPU
during IDLE times and let the low power CPU take charge.

The M4 in the H7 powers doesn't seem like such a good choice for lower
power.  Perhaps it has some other intended usage.

[Nathan Hartman]

On Wednesday, October 30, 2019 at 3:24:54 PM UTC-4, patacongo wrote:

On the surface, I think that the OpenMCAPI port would simply amount to
re-implementing the OpenMCAPI Linux drivers to run in NuttX.  Those
Linux Drivers are dual license GPL/BSD so probably could not come into
the OS directly; they would need to be re-implemented to avoid any
possibility of GPL taint.  I don't even want those three letters
together in any file in the repository.

I agree! Thank you for not putting those letters in!!!

> Obviously I have to weigh the potential benefits of this second core
> against the up-front investment in time, headache, $$$, etc.  

I would avoid the complexity unless it is necessary to the solution.

Keep it simple... Thanks for the reminder :-)

[Nathan Hartman]

On Wednesday, October 30, 2019 at 3:28:21 PM UTC-4, patacongo wrote:

In other asymmetric multi-core solutions that I have looked at, one core
is specialize to have very low power consumption (hence, usually an
M0+).  So in very low power modes you just shut down the big pig CPU
during IDLE times and let the low power CPU take charge.

The M4 in the H7 powers doesn't seem like such a good choice for lower
power.  Perhaps it has some other intended usage.

I don't know what the manufacturer has in mind, but I think it's along

the lines of running a high-level OS on the big core and realtime code

on the M4. This seems to be one of the strategies for doing real-time

with Linux; that is, instead of patching the kernel with realtime

extensions, put the real time code on a separate processor. Not

coincidentally, I think, STmicro have been advertising their own Open

ST Linux distro all over the place. I just saw it a moment ago on the

Digikey front page. But this H7, for all its strengths, looks vastly

under-provisioned to run such a heavy OS: up to 2 MB flash 1 MB RAM

for both cores; if you can cram Linux in there, you're out of room.

So, again, I'm not sure what the angle is for this MCU design.

[spudaneco]

Sent from Samsung tablet.

> So, again, I'm not sure what the angle is for this MCU design.

It is hard to imagine that the product could be successful if its potential customers don't know what it is for. 

[Xiang Xiao]

Hi all,

We have done the major OpenAMP work on NuttX:

1.Port OpenAMP to NuttX:

   https://github.com/FishsemiCode/open-amp

   https://github.com/FishsemiCode/libmetal

   and upstream to OpenAMP community: 

   https://github.com/OpenAMP/libmetal/tree/master/lib/system/nuttx

2.Follow NuttX driver model style to wrap OpenAMP:

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/mbox/mbox.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/rptun/rptun.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/rptun/openamp.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/rptun/rptun.c

3.Implement rpmsg syslog to redirect log from other core:

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/syslog/syslog_rpmsg.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/syslog/syslog_rpmsg.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/syslog/syslog_rpmsg.c

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/syslog/syslog_rpmsg_server.c

4.Implement rpmsg serial driver so we can use cu talk with nsh on other core:

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/serial/uart_rpmsg.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/serial/uart_rpmsg.c

5.Implement rpmsg rtc driver to get the wall clock from main core:

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/timers/rpmsg_rtc.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/timers/rpmsg_rtc.c

6.Implemnet rpmsg hostfs so other core could access file system on main core:

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/fs/hostfs_rpmsg.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/fs/hostfs/hostfs_rpmsg.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/fs/hostfs/hostfs_rpmsg.c

   https://github.com/FishsemiCode/nuttx/blob/song-u1/fs/hostfs/hostfs_rpmsg_server.c

7.Implement rpmsg net driver to redirect TCP/IP package to communicaton core(MAC and PHY process):

   https://github.com/FishsemiCode/nuttx/blob/song-u1/include/nuttx/net/rpmsgdrv.h

   https://github.com/FishsemiCode/nuttx/blob/song-u1/drivers/net/rpmsgdrv.c

8.Implement rpmsg usrsock client/service to redirect socket API to main core:

   https://github.com/FishsemiCode/usrsock

And more, you can search rpmsg to find other drivers.

We also implement the similar drivers on Linux side, so the same code could not only talk NuttX with NuttX but and also talk NuttX with Linux.

The attached slides has the more detailed information for each drivers, maybe we can work together to make AMP support part of the NuttX official release.

Thanks

Xiang

[Xiang Xiao]

BTW, you can sync the whole code base by:

  repo init -u g...@github.com:FishsemiCode/manifest.git -b u1 -m u1.xml

 repo sync

[Gregory Nutt]

> We have done the major OpenAMP work on NuttX:

Perhaps some of the OS changes should be brought upstream?

Has the NuttX port of the OpenAMP user space been accepted?  For Linux,
OpenAMP distributes the Linux drivers as part of OpenAMP. I presume that
the an RTOS that would be to difficult. So perhaps it would be best to
bring the drivers and syslog support into NuttX mainstream?

Greg

[Alin Jerpelea]

Please push your changes to mainline

Alin

--
You received this message because you are subscribed to the Google Groups "NuttX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/94461294-79d2-65e8-8a45-d61ac441b0be%40gmail.com.

[Nathan Hartman]

On Wednesday, October 30, 2019 at 11:12:07 PM UTC-4, Xiang Xiao wrote:

> Hi all,

> We have done the major OpenAMP work on NuttX:

Wow that is impressive and exciting! Thank you, Xiang!

On Thursday, October 31, 2019 at 9:24:30 AM UTC-4, Alin Jerpelea wrote:

> Please push your changes to mainline

I agree!

[Gregory Nutt]

> > Please push your changes to mainline
>
> I agree!

Since the NuttX OpenAMP changes have been submitted upstream, we do need
to let the OpenAMP team decide how best to deliver their solution.  With
Linux, for example, the Linux drivers are provided by OpenAMP.  I
believe that an embedded solution would have to handled differently. 
Providing those matching drivers within NuttX would be the most straight
forward way to do that.

But hosting the driver-related components in NuttX also raises issues of
versioning and how OpenAMP can make improvements in their solution if
they are so coupled to NuttX internals.

If Xiaomi and OpenAMP are comfortable, I would welcome the changes.

Greg

[Xiang Xiao]

On Thursday, October 31, 2019 at 10:33:17 PM UTC+8, patacongo wrote:

> > Please push your changes to mainline
>
> I agree!

Since the NuttX OpenAMP changes have been submitted upstream, we do need
to let the OpenAMP team decide how best to deliver their solution.  With
Linux, for example, the Linux drivers are provided by OpenAMP.  I

believe that an embedded solution would have to handled differently. 
Providing those matching drivers within NuttX would be the most straight
forward way to do that.

Actually OpenAMP is justed used in RTOS(e.g. NuttX, FreeRTOS, Zephyr...), Linux kernel has it's owned implementation, but both share the same memory layout and transfer flow.

The whole sulotion contain two parts:

1.The low layer is called remoteproc which is responsible to parse and load firmware into the memoy, control the remote cpu start, stop and reset.

   https://www.kernel.org/doc/Documentation/remoteproc.txt

   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/remoteproc

   https://github.com/OpenAMP/open-amp/tree/master/lib/remoteproc

2.The high layer is called rpmsg which is responsible to name<->port mapping(like DNS) and package transfer(like UDP but reliable).

   https://www.kernel.org/doc/Documentation/rpmsg.txt

   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/rpmsg

   https://github.com/OpenAMP/open-amp/tree/master/lib/rpmsg

Other protocol implementation which build on top of rpmsg(like syslog_rpmsg, uart_rpmsg) tightly couple with the particular OS driver model isn't part of OpenAMP.

 

But hosting the driver-related components in NuttX also raises issues of
versioning and how OpenAMP can make improvements in their solution if
they are so coupled to NuttX internals.

The interaction betwwen OpenAMP and RTOS has two sides:

1.RTOS need implement some callback defined by remoteproc layer to:

   a.Where is the share memory location and size

   b.How to power on/off and sart/stop/reset the remote core

   c.How to send the interrupt to the remote core(like SMP IPI)

2.The subsystem IPC driver call rpmsg API to register/connect service, send/recv package

The 1st part is changed frequently in each release, but the 2nd part is very stable.

[Gregory Nutt]

This all sounds good.  Who wants to take the responsibility of getting
the NuttX portions into the repository?  Xiao Xiang has already made the
code available in the GitHub repository that he referenced, perhaps
someone else should work up the NuttX patch?

I peeked at a few of the files:  They have BSD licensing and appear to
follow the NuttX coding standard (but running tools/nxstyle is always
recommended).

> 1.RTOS need implement some callback defined by remoteproc layer ...

Of course, we can't really do callbacks from the OS.  In PROTECTED modes
that would be a security hole.  In kernel mode, it is very difficult to
setup.

I hope you are talking about an IOCTL call, such as with the framebuffer
driver.  You call the ioctl() and it returns the address of the
framebuffer (mapped into user space as appropriate).

Otherwise, it seems like a straight forward protocol with the OS
provided only the necessary memory management and low-level operations.

Greg

[Jerome KASPER]

You did a great job Xiang. I'm very happy and thankful that it has been done. Hope this could be merged in NuttX Mainline in a near future with Gregory approval of course. I didn't expected this to be implemented such quickly.
My best greetings and thanks for sharing the associated code. I hope to test this out soon. Will give feedback for sure.

Kind Regards,
Jerome Kasper

--
You received this message because you are subscribed to the Google Groups "NuttX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/d34a4915-bde6-449e-9b32-019f28a0cfbd%40googlegroups.com.

[Jerome KASPER]

On 31/10/2019 18:39, Gregory Nutt wrote:
> This all sounds good.  Who wants to take the responsibility of getting
> the NuttX portions into the repository?  Xiao Xiang has already made
> the code available in the GitHub repository that he referenced,
> perhaps someone else should work up the NuttX patch?
>
> I peeked at a few of the files:  They have BSD licensing and appear to
> follow the NuttX coding standard (but running tools/nxstyle is always
> recommended).

I'm surely not not the most experimented to do that but i can give it a
try as I wanted to fiddle with OpenAMP. It would be a great exercise for
me . Anyone wanting to join the effort is welcome to make him known :)
I'm all open to team work !  Xiang , if you were already on the way,
feel free to share me some work.

>> 1.RTOS need implement some callback defined by remoteproc layer ...
>
> Of course, we can't really do callbacks from the OS.  In PROTECTED
> modes that would be a security hole.  In kernel mode, it is very
> difficult to setup.
>
> I hope you are talking about an IOCTL call, such as with the
> framebuffer driver.  You call the ioctl() and it returns the address
> of the framebuffer (mapped into user space as appropriate).

Greg, any particular convention you would like to be followed about the
regarding IOCTL call?

>
> Otherwise, it seems like a straight forward protocol with the OS
> provided only the necessary memory management and low-level operations.

Yes , OpenAMP design is interesting and would help real world
scalability. Hence my interest !

Kind regards,
Jerome

[Xiang Xiao]

On Friday, November 1, 2019 at 1:39:49 AM UTC+8, patacongo wrote:

This all sounds good.  Who wants to take the responsibility of getting
the NuttX portions into the repository?  Xiao Xiang has already made the
code available in the GitHub repository that he referenced, perhaps
someone else should work up the NuttX patch?

I peeked at a few of the files:  They have BSD licensing and appear to
follow the NuttX coding standard (but running tools/nxstyle is always
recommended).

> 1.RTOS need implement some callback defined by remoteproc layer ...

Of course, we can't really do callbacks from the OS.  In PROTECTED modes
that would be a security hole.  In kernel mode, it is very difficult to
setup.

I hope you are talking about an IOCTL call, such as with the framebuffer
driver.  You call the ioctl() and it returns the address of the
framebuffer (mapped into user space as appropriate).

But, many user of rpmsg is the subsystem driver(e.g. syslog/serial/fs/net and more), so it's nature to make OpenAMP as a kernel subsystem like Linux kernel.

Of course, the userspace also need IPC capacity(e.g. usrsock),  but I think the socket API is more suitable than IOCTL interface here.

How about we create a new family call PF_RPMSG for userspace IPC? Then we can implement usrsock, MCAPI event grpc on the new socket family.

Since many userspace RPC library already desgin for socket API, this approach could extend these library from cross process boundary to cross processor boundary quickly.

Something like Unix Domain Socket but between the processor.

[Xiang Xiao]

On Friday, November 1, 2019 at 8:54:41 AM UTC+8, Jerome KASPER wrote:

On 31/10/2019 18:39, Gregory Nutt wrote:
> This all sounds good.  Who wants to take the responsibility of getting
> the NuttX portions into the repository?  Xiao Xiang has already made
> the code available in the GitHub repository that he referenced,
> perhaps someone else should work up the NuttX patch?
>
> I peeked at a few of the files:  They have BSD licensing and appear to
> follow the NuttX coding standard (but running tools/nxstyle is always
> recommended).

I'm surely not not the most experimented to do that but i can give it a
try as I wanted to fiddle with OpenAMP. It would be a great exercise for
me . Anyone wanting to join the effort is welcome to make him known :)
I'm all open to team work !  Xiang , if you were already on the way,
feel free to share me some work.

Now our code is very stable, but we aren't starting to upstream the code to NuttX.

It's great if you can port the code to your platform and verify our solution is general enough to support the different hardware.

We can work together to upstream the code.

[Gregory Nutt]

> Now our code is very stable, but we aren't starting to upstream the
> code to NuttX.
> It's great if you can port the code to your platform and verify our
> solution is general enough to support the different hardware.
> We can work together to upstream the code.

I will create a branch and bring the files that Xiang mentioned in a
previous post.  Can I list you (Xiao Xiang) as the author of the files?

I will let you know when the branch is in place.

[Gregory Nutt]

The branch is in place and initially populated with the list of files
provided by Xiang in a previous email (including the additional
*rpmsg*.* files that he mentions.  The initial commit is just those
files alone and cannot even be built until the corresponding changes to
the Kconfig and Make.defs files in the affected directories.

See https://bitbucket.org/nuttx/nuttx/branch/OpenAMP

Looking at the code, I am wondering if this is too specific to Xiaomi's
application.  This includes a lot of specific logic to remote things
like time, RTC, file system access, and networking to different CPUs. 
Would that be a general capability needed by all OpenAMP applications? 
I think that answer is that that RPC infrastructure would be used by
many, but not all OpenAMP architectures.  I hope that the individual RPC
servers can be controlled with Kconfig options so that it can be
tailored to any application.

Greg

[Nathan Hartman]

On Fri, Nov 1, 2019 at 10:28 AM Gregory Nutt <spud...@gmail.com> wrote:

Looking at the code, I am wondering if this is too specific to Xiaomi's
application.  This includes a lot of specific logic to remote things
like time, RTC, file system access, and networking to different CPUs. 
Would that be a general capability needed by all OpenAMP applications? 
I think that answer is that that RPC infrastructure would be used by
many, but not all OpenAMP architectures.  I hope that the individual RPC
servers can be controlled with Kconfig options so that it can be
tailored to any application.

While I haven't yet worked on an AMP system, I have worked on quite a

few systems that involved multiple separate MCUs that have to

communicate and coordinate with each other.

Things like synchronizing events and communicating between the different

CPUs are always the biggest hassle.

Debugging when multiple CPUs are in play becomes orders of magnitude

more complicated.

For these reasons, I think RPC will be important. Also, making one core

responsible for logging and channeling the logging from other cores to

it, seems pretty important too.

[Xiang Xiao]

On Friday, November 1, 2019 at 10:28:44 PM UTC+8, patacongo wrote:

>> Now our code is very stable, but we aren't starting to upstream the
>> code to NuttX.
>> It's great if you can port the code to your platform and verify our
>> solution is general enough to support the different hardware.
>> We can work together to upstream the code.
>
> I will create a branch and bring the files that Xiang mentioned in a
> previous post.  Can I list you (Xiao Xiang) as the author of the files?
>
> I will let you know when the branch is in place.

The branch is in place and initially populated with the list of files
provided by Xiang in a previous email (including the additional
*rpmsg*.* files that he mentions.  The initial commit is just those
files alone and cannot even be built until the corresponding changes to
the Kconfig and Make.defs files in the affected directories.

See https://bitbucket.org/nuttx/nuttx/branch/OpenAMP

Looking at the code, I am wondering if this is too specific to Xiaomi's
application.  This includes a lot of specific logic to remote things
like time, RTC, file system access, and networking to different CPUs. 

There are three different categories:

1.Xiaomi specfic driver, we don't need port:

   include/nuttx/misc/misc_rpmsg.h

   drivers/misc/misc_rpmsg.c

2.General, but it's a new subsystem for NuttX:

   drivers/clk/clk-rpmsg.c

   drivers/power/rpmsg_regulator.c

   We need upstream the subsystem before rpmsg driver.

   clk/regulator is model from Linux, we need Greg check whether the idea is good for NuttX

3.General rpmsg driver and fit into the exist NuttX subsystem, the rest driver could be reused without any modification.

   Actually, we have made many different combination in real product:

   a.CEVA DSP<->Linux Kernel

   b.Cortex M4 <->Linux Kernel

   c.Rocketchip<->Linux Kernel

   d.CEVA DSP<->Cortex M4

   e.Cortex M4 <->Cortex M4

   f.RISCY       <->RISCV RISCY

Would that be a general capability needed by all OpenAMP applications? 

OpenAMP only need:

1.Shared memory which could be accessed by both CPU

2.One side could trigger a software interrupt to another side

Almost multi-core SoC design support this two requirement.

 

I think that answer is that that RPC infrastructure would be used by
many, but not all OpenAMP architectures.  I hope that the individual RPC
servers can be controlled with Kconfig options so that it can be
tailored to any application.

Yes, each rpmsg driver is controlled by a indivual Kconfig option in our code base.

Actually, we turn on different option in each core, like this:

1.AP(application processor)

   a.syslog server: receive the syslog from other core

   b.uart driver: cu to other core for nsh

   c.hostfs client: redirect FS API to SP

   d.usrsock server: redirect Socket API to SP

2.SP(service processor)

   a.syslog client: send the syslog to AP

   b.uart driver: nsh receive the command from AP/send result to AP

   c.hostfs server: forward FS API to the real filesystem

   d.usrsock client: forward AP Socket API to the real TCP/IP stack

   e.netdev driver: forward IP package to CP

3.CP(comunication processor)

   a.syslog client: send the syslog to AP

   b.uart driver: nsh receive the command from AP/send result to AP

   c.hostfs client: redirect FS API to SP

   d.adhoc driver: receive IP package from SP's netdev and forwardd to MAC/PHY layer

You can see the different CPU enable the different rpmsg driver or role(client v.s. server) as needed.

ST make a good slides to introduce this scenario in this year ELC:

https://elinux.org/images/6/63/ELC_EU19_rpmsg.pdf

  

Greg

[Gregory Nutt]

Hi, Xiang,

Thanks for input this it is always helpful.

> There are three different categories:
> 1.Xiaomi specfic driver, we don't need port:
>    include/nuttx/misc/misc_rpmsg.h
>    drivers/misc/misc_rpmsg.c

I have removed these from the branch

> 2.General, but it's a new subsystem for NuttX:
>    drivers/clk/clk-rpmsg.c
>    drivers/power/rpmsg_regulator.c
>    We need upstream the subsystem before rpmsg driver.
>    clk/regulator is model from Linux, we need Greg check whether the
> idea is good for NuttX

I removed these from the branch too.  Let's do one thing at a time.  We
can talk about the clk/regulator model later and bring these back in
if/when appropriate.

As I mentioned, I will complete the Make.defs and Kconfig changes.  Then
it will be up to someone else (Jerome?) to integrate and test the
feature before it comes into the master.

Greg

[Gregory Nutt]

> 1.Xiaomi specfic driver, we don't need port:
>    include/nuttx/misc/misc_rpmsg.h
>    drivers/misc/misc_rpmsg.c

I also see a file drivers/rptun/song_rptun.c.  That appears to be a chip
specific file.  That was not included in the original file list and, it
appears, contains chip-specific logic which really belongs under arch/. 
We would probably need some architectural support to support chip- and
board-specific RPCs.

That extensibility seems necessary to be able to support a general
purpose RPC functionality.

[Gregory Nutt]

Hi, Xiang,

Thanks for input this it is always helpful.

There are three different categories:

1.Xiaomi specfic driver, we don't need port:

   include/nuttx/misc/misc_rpmsg.h

   drivers/misc/misc_rpmsg.c

I have removed these from the branch

[patacongo]

The branch is in place and initially populated with the list of files
provided by Xiang in a previous email (including the additional
*rpmsg*.* files that he mentions.  The initial commit is just those
files alone and cannot even be built until the corresponding changes to
the Kconfig and Make.defs files in the affected directories.

See https://bitbucket.org/nuttx/nuttx/branch/OpenAMP

Okay... everything should be there now.  I think I have all of Kconfig and Make.defs update and I have brought in the OpenAMP configuration options.  So everything is in place ready to for the next steps.  The next steps are:

1. Verify the OpenAMP NuttX drivers, and

2. Run all of the modified files through tools/nxstyle.  The coding style looks perfect to me now, but lets check.

This the job that you signed up to do Jerome, correct.  So let me know when you are ready and I will merge the changes to master.

Greg

[Gregory Nutt]

I tried a test build and I get this error many times:

nuttx/include/nuttx/rptun/openamp.h:44:30: fatal error:
openamp/open_amp.h: No such file or directory

My guess is that there must be some missing installation of OpenAMP
header files at include/openamp/.  Is that right?  I don't see any
reference to openamp/ in any .gitignore files.  I expected to see that
in include/.gitignore.  Is there documentation about that?

Getting the clean build build isn't really critical to me;  I didn't
volunteer to do this integration.  But I thought I would try to get as
much in place for Jerome as possible.

Greg

[Gregory Nutt]

Hi, Xiang

Have you considered a simulator-based test base.  I think you might be
able to do that with two copies of the NuttX simulator running with
shared memory and perhaps pthreads running to service simulated
inter-CPU events.

Greg

[Jerome Kasper]

Hi Xiang, Greg,

I'm all in for the integration and tests . I will try several OpenAMP
tests between various boards (Discovery STM32F4+STM32F7/ Flip and click
PIC32) and will report. Does using the CAN bus for rpmsg communications
as a start sounds like a good idea ?

Best regards,
Jerome

[spud...@gmail.com]

 

 

I'm all in for the integration and tests . I will try several OpenAMP

tests between various boards (Discovery STM32F4+STM32F7/ Flip and click

PIC32) and will report. Does using the CAN bus for rpmsg communications

as a start sounds like a good idea ?

 

I think I recall Xiang saying that is required a shared memory interface.  A multicore i.MX6 or two simulator processes would be a better bet if that is the correct understanding.

 

[Jerome Kasper]

As mentionned in the previous email i will do that . Will work on those 2 tasks this weekend. Real life tests might take a bit longer.
Thanks for giving me that opportunity :)
Regards,
Jerome

Greg

--
You received this message because you are subscribed to the Google Groups "NuttX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/702eca7f-3fbd-4764-a93b-20552bd1874c%40googlegroups.com.

[Jerome Kasper]

I have some memory chips hanging around and adapters to solder them, i could try to set a configuration with "shared memory" setup. That said , it's obvious that a two simulator process would be a better start. I will do that first.
Greg could you advice me a "low cost" I.MX6 board ? I'll be able to afford one next month . ( I saw that one https://fr.aliexpress.com/item/32807969666.html , can probably contact manufacturer for schematics)
Regards,
Jerome

 

--
You received this message because you are subscribed to the Google Groups "NuttX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/5dbcc9bf.1c69fb81.8225e.1b6a%40mx.google.com.

[spud...@gmail.com]

 

I have some memory chips hanging around and adapters to solder them, i could try to set a configuration with "shared memory" setup. That said , it's obvious that a two simulator process would be a better start. I will do that first.
Greg could you advice me a "low cost" I.MX6 board ? I'll be able to afford one next month . ( I saw that one https://fr.aliexpress.com/item/32807969666.html , can probably contact manufacturer for schematics)

 

There are over lower cost architectures you could consider.  Such as the LPC43.

 

We should wait to see if Xiang can verify the minimal interface requirements.  I might be mistaken since I know nothing about OpenAMP.

 

I suspect that you may be grossly under-estimating the level of effort required to do any of the options that you proposing. Possibly man months of effort and of very high complexity.  Are you sure you are up for this?

 

[Jerome Kasper]

Le 02/11/2019 à 02:43, spud...@gmail.com a écrit :

 

I have some memory chips hanging around and adapters to solder them, i could try to set a configuration with "shared memory" setup. That said , it's obvious that a two simulator process would be a better start. I will do that first.
Greg could you advice me a "low cost" I.MX6 board ? I'll be able to afford one next month . ( I saw that one https://fr.aliexpress.com/item/32807969666.html , can probably contact manufacturer for schematics)

 

There are over lower cost architectures you could consider.  Such as the LPC43.

Didn't knew much about LPC43 . Going low cost ARM multi-core , there is also the STM32H7 (https://fr.aliexpress.com/item/4000015743692.html) that i found interesting. Might be an interesting bet.

 

We should wait to see if Xiang can verify the minimal interface requirements.  I might be mistaken since I know nothing about OpenAMP.

True. I will take a closer look at the code and wait for Xiang "go"

 

I suspect that you may be grossly under-estimating the level of effort required to do any of the options that you proposing. Possibly man months of effort and of very high complexity.  Are you sure you are up for this?

I was wrong regarding the architecture choice (I.mx6), but i'm really interested in OpenAMP .
So not that afraid that would take some time , i did expected this not to be solved into this weekend :)
Plan is first , deep code read . Then will try simulator. We'll see after that :)

Regards,
Jerome

--
You received this message because you are subscribed to the Google Groups "NuttX" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/5dbcdf49.1c69fb81.57a87.4ba1%40mx.google.com.

[Jerome Kasper]

Le 02/11/2019 à 02:43, spud...@gmail.com a écrit :

 

I have some memory chips hanging around and adapters to solder them, i could try to set a configuration with "shared memory" setup. That said , it's obvious that a two simulator process would be a better start. I will do that first.
Greg could you advice me a "low cost" I.MX6 board ? I'll be able to afford one next month . ( I saw that one https://fr.aliexpress.com/item/32807969666.html , can probably contact manufacturer for schematics)

 

There are over lower cost architectures you could consider.  Such as the LPC43.

I read too fast and only some series of STM32H7 are dual core (and specially not the board i linked)

 

We should wait to see if Xiang can verify the minimal interface requirements.  I might be mistaken since I know nothing about OpenAMP.

 

I suspect that you may be grossly under-estimating the level of effort required to do any of the options that you proposing. Possibly man months of effort and of very high complexity.  Are you sure you are up for this?

And yes , i do expect some months of work, i have a buch of PIC32 samples that i wanted to get synced with a STM32F7, this is the right time i think.

Regards, jerome

[Xiang Xiao]

Yes, this file is specific to our chipset.

We put it here because all Pinecone chipset even with different arch(ceva, arm, riscv...) share the same rptun implementation.

 

 

[Xiang Xiao]

On Saturday, November 2, 2019 at 5:17:34 AM UTC+8, patacongo wrote:

I tried a test build and I get this error many times:

nuttx/include/nuttx/rptun/openamp.h:44:30: fatal error:
openamp/open_amp.h: No such file or directory

My guess is that there must be some missing installation of OpenAMP
header files at include/openamp/.  Is that right?  I don't see any
reference to openamp/ in any .gitignore files.  I expected to see that
in include/.gitignore.  Is there documentation about that?

Yes, Makefile will install openamp/libmetal header file to include folder, but we put Makefile into openamp/libmetal git.

Since multiple git get invovled here, it's better to use repo to get all source code:

  repo init -u g...@github.com:FishsemiCode/manifest.git -b u1 -m u1.xml

  repo sync

Actually after sync, you can try build with one command and compare the result side by side:

  ./build.sh u1/ap

  ./build.sh u1/sp

[Xiang Xiao]

Yes, I have talked the same idea with Anchao, but we never find the time to implement it.

 

Greg

[Xiang Xiao]

Yes, OpenAMP implementation is heavy base on share memory/software interrupt. CAN bus isn't a good candidate for this task.

Here is an introduction slide from OpenAMP maintainer cover the virtio/remoteproc/rpmsg:

https://www.slideshare.net/linaroorg/hkg18411-introduction-to-openamp-which-is-an-open-source-solution-for-heterogeneous-system-orchestration-and-communication

Actually, OpenAMP reuse VirtioIO(the core part of Linux virtualization) to manage the share memory and software interrupt, from this paper you can see the tight coupling:

https://www.ozlabs.org/~rusty/virtio-spec/virtio-paper.pdf

[xiang xiao]

Just talk with AnChao who do the major work on NuttX/OpenAMP, he can
work on the porting in the next week to:
1.Fix the build break
2.Remove Pinecone dependence
3.Make OpenAMP work on two sim instance
Jerome, you could focus on porting OpenAMP to the hardware you choice.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/58426cac-251a-5aea-c368-4d8dd1f28f1f%40gmail.com.

[xiang xiao]

Greg, the attached patch is the minor fix I found during review OpenAMP branch.
Anchao will take a more close review in the next week.

> --
> You received this message because you are subscribed to the Google Groups "NuttX" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/548adbc6-8da1-88fe-38b4-56bf244a2f68%40gmail.com.

[Gregory Nutt]

With the reorganization of the boards/ directory there is no a place for architecture specific, common code.  Look how Alin did that in boards/arm/cxd56xx directory:   There are directories there to hold common code and chip-specific drivers.  Perhaps we could use that structure here too?

[Gregory Nutt]

I tried a test build and I get this error many times:

nuttx/include/nuttx/rptun/openamp.h:44:30: fatal error:
openamp/open_amp.h: No such file or directory

My guess is that there must be some missing installation of OpenAMP
header files at include/openamp/.  Is that right?  I don't see any
reference to openamp/ in any .gitignore files.  I expected to see that
in include/.gitignore.  Is there documentation about that?

Yes, Makefile will install openamp/libmetal header file to include folder, but we put Makefile into openamp/libmetal git.

Since multiple git get invovled here, it's better to use repo to get all source code:

  repo init -u g...@github.com:FishsemiCode/manifest.git -b u1 -m u1.xml

  repo sync

I am making some progress, but still no clean build.  Here are the steps that I did:

Get repositories
  git clone https://github.com/FishsemiCode/open-amp.git open-amp
  git clone https://github.com/FishsemiCode/libmetal.git libmetal

Copy header files:
  719  cp -a  libmetal/lib nuttx/include/metal
  724  cp -a open-amp/lib/include/openamp/ nuttx/include/openamp

Copy source directories
  738  cp -a open-amp nuttx/openamp/open-amp
  738  cp -a libmetal nuttx/openamp/libmetal

When I try to build, I get this error:

make[1]: Entering directory '/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/openamp'
gcc: warning: ./Makefile: linker input file unused because linking not done
In file included from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/metal/condition.h:15:0,
                 from ./libmetal/lib/system/nuttx/condition.c:12:
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/metal/mutex.h:22:49: fatal error: metal/system/@PROJECT_SYSTEM@/mutex.h: No such file or directory
compilation terminated.
ERROR: gcc failed: 1
       command: gcc -M -Wall -Wstrict-prototypes -Wshadow -Wundef -g -fno-builtin -I. -isystem /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include -D__KERNEL__ -pipe -DMETAL_INTERNAL ./libmetal/lib/system/nuttx/condition.c
make[1]: *** [Makefile:70: .depend] Error 1
make[1]: Leaving directory '/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/openamp'

Yech.. Cmake.

[Gregory Nutt]

I am making some progress, but still no clean build.  Here are the steps that I did:

Get repositories
  git clone https://github.com/FishsemiCode/open-amp.git open-amp
  git clone https://github.com/FishsemiCode/libmetal.git libmetal

Copy header files:

  cp -a  libmetal/lib nuttx/include/metal

  cp -a open-amp/lib/include/openamp/ nuttx/include/openamp

Copy source directories

  cp -a open-amp nuttx/openamp/open-amp
  cp -a libmetal nuttx/openamp/libmetal

I worked around the build problems with this kludge:

find nuttx/include/metal -name "*.h" | xargs sed -i -e "s/@PROJECT_SYSTEM@/nuttx/g"
find nuttx/include/metal -name "*.h" | xargs sed -i -e "s/@PROJECT_PROCESSOR@/x86_64/g"

I know that is not correct, but it lets me continue testing.  I am first trying to get the drivers/ to build.  Most things build now but here are the things that still need to be cleaned up:

CC:  rptun/rptun.c
In file included from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/open_amp.h:13:0,
                 from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/nuttx/rptun/openamp.h:44,
                 from rptun/rptun.c:45:
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:163:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:203:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:228:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:356:10: error: variably modified ‘algo’ at file scope
  uint8_t algo[B2C(16)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:357:10: error: variably modified ‘chksum’ at file scope
  uint8_t chksum[B2C(64)];
          ^
In file included from rptun/rptun.c:46:0:
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/nuttx/rptun/rptun.h:264:1: error: requested alignment is not an integer constant
 };
 ^

The above errors are all due to the B2C macro that does not exist in NuttX (see below).

rptun/rptun.c: In function ‘rptun_thread’:
rptun/rptun.c:179:10: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
   priv = (FAR struct rptun_priv_s *)atoi(argv[1]);
          ^
rptun/rptun.c: In function ‘rptun_initialize’:
rptun/rptun.c:807:8: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
   itoa((int)priv, str, 10);

These need to be fixed too.  The assumption is that a pointer will fit in a type int.  That is not correct.  On 64-bit CPUs (such as the host NuttX simulation), sizof(int) is 32-bits but a pointer is 64-bits.  The above code will NOT work on the simulator and needs to be fixed

and

CC:  serial/uart_rpmsg.c
In file included from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/open_amp.h:13:0,
                 from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/nuttx/rptun/openamp.h:44,
                 from serial/uart_rpmsg.c:49:
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:163:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:203:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:228:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:356:10: error: variably modified ‘algo’ at file scope
  uint8_t algo[B2C(16)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:357:10: error: variably modified ‘chksum’ at file scope
  uint8_t chksum[B2C(64)];
          ^
serial/uart_rpmsg.c: In function ‘uart_rpmsg_dmasend’:
serial/uart_rpmsg.c:263:11: warning: implicit declaration of function ‘C2B’ [-Wimplicit-function-declaration]
   space = C2B(space - sizeof(*msg));
           ^
serial/uart_rpmsg.c:272:7: warning: implicit declaration of function ‘cmem2bmem’ [-Wimplicit-function-declaration]
       cmem2bmem(msg->data, 0, xfer->buffer, xfer->length);
       ^
serial/uart_rpmsg.c:273:29: warning: implicit declaration of function ‘B2C_OFF’ [-Wimplicit-function-declaration]
       cmem2bmem(msg->data + B2C_OFF(xfer->length), B2C_REM(xfer->length),
                             ^
serial/uart_rpmsg.c:273:52: warning: implicit declaration of function ‘B2C_REM’ [-Wimplicit-function-declaration]
       cmem2bmem(msg->data + B2C_OFF(xfer->length), B2C_REM(xfer->length),
                                                    ^
serial/uart_rpmsg.c: In function ‘uart_rpmsg_dmareceive’:
serial/uart_rpmsg.c:304:7: warning: implicit declaration of function ‘bmem2cmem’ [-Wimplicit-function-declaration]
       bmem2cmem(xfer->buffer, msg->data, 0, xfer->length);
       ^
make[1]: [Makefile:122: uart_rpmsg.o] Error 1 (ignored)

The above errors are again due to the B2C macro that does not exist in NuttX (see below).  And

In file included from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/open_amp.h:13:0,
                 from /cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/nuttx/rptun/openamp.h:44,
                 from syslog/syslog_rpmsg_server.c:44:
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:163:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:203:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:228:10: error: variably modified ‘name’ at file scope
  uint8_t name[B2C(32)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:356:10: error: variably modified ‘algo’ at file scope
  uint8_t algo[B2C(16)];
          ^
/cygdrive/d/Spuda/Documents/projects/nuttx/master/nuttx/include/openamp/remoteproc.h:357:10: error: variably modified ‘chksum’ at file scope
  uint8_t chksum[B2C(64)];
          ^
make[1]: [Makefile:122: syslog_rpmsg_server.o] Error 1 (ignored)

The above errors are all due to the B2C macro that does not exist in NuttX

The biggest issue preventing a clean build now the the various macros added to included/strings.h and to include/nuttx/mm/mm.h, all with the naming B2C or C2B.  These are apparently added to handle cases where is character is larger than one byte.  My gut feeling is that strings.h is not the correct place to put these since they are non-standard.  Perhaps they should go in include/nuttx/b2c.h and then be included in strings.h?

What do you think?

Greg

[Gregory Nutt]

The last bit about OpenAMP for the day:

...  Most things build now but here are the things that still need to be cleaned up: ...

rptun/rptun.c: In function ‘rptun_thread’:

rptun/rptun.c:179:10: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
   priv = (FAR struct rptun_priv_s *)atoi(argv[1]);
          ^
rptun/rptun.c: In function ‘rptun_initialize’:
rptun/rptun.c:807:8: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
   itoa((int)priv, str, 10);

These need to be fixed too.  The assumption is that a pointer will fit in a type int.  That is not correct.  On 64-bit CPUs (such as the host NuttX simulation), sizof(int) is 32-bits but a pointer is 64-bits.  The above code will NOT work on the simulator and needs to be fixed

This is still a problem and needs to be fixed.  So does this one:

hostfs/hostfs_rpmsg.c: In function ‘host_opendir’:
hostfs/hostfs_rpmsg.c:557:27: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
   return ret < 0 ? NULL : (FAR void *)ret;
                           ^
I did not try to fix these because there may be design changes involved.  These fixes will be needed for the 64-bit simulation.

The biggest issue preventing a clean build now the the various macros added to included/strings.h and to include/nuttx/mm/mm.h, all with the naming B2C or C2B.  These are apparently added to handle cases where is character is larger than one byte.  My gut feeling is that strings.h is not the correct place to put these since they are non-standard.  Perhaps they should go in include/nuttx/b2c.h and then be included in strings.h?

I did that.  I created include/nuttx/b2c.h and included that in strings.h.  I copied the functions for the case of CHAR_BIT != 1 into libs/libc/string/.  I did not add the changes to mm.h that you have for related B2C definitions.  Those were needed yet.

Now everything compiles cleanly (except for openamp/libmetal which still has some configuration/setup that is not right).  So I went ahead and merged the branch into master.  It does not add any value yet with no testing and with no build instructions, but it now has been completely reviewed and its presence appears to be harmless to the overall health of the system.

We do need to have a simple configuration setup script for OpenAMP and libmetal.  The build.sh script in your build repository is too focused on your BSP to be usable in the most general NuttX context.  Is that something that you could produce?  I would greatly appreciate that contribution.

I have removed the OpenAMP branch.

Greg

[Gregory Nutt]

> Just talk with AnChao who do the major work on NuttX/OpenAMP, he can
> work on the porting in the next week to:
> 1.Fix the build break

I think the build is clean now EXCEPT that we need a script or some way
to configure and setup openamp/libmetal and openamp/open-amp.
> 2.Remove Pinecone dependence
You should verify this, but I think I have all of that removed.  We need
to revisit the clk/regulator logic which I did not bring in and also the
chip-specific support.  As I mentioned before, I would think that the
chip-specific should go in the boards/ directory in the new
infrastructure that Alin Jerpelea creates for chip common drivers and logic.

> 3.Make OpenAMP work on two sim instance

I think that this would require fixing the casts from type int to
pointers that I mentioned (or forcing a 32-bit build).

I would be happy to help out AnChao with this part.  Since I have
brought the code into the master branch, I am completely committed to
making this work smoothly.

My understanding is that the communications is performed in application
space via the OpenAMP shared memory.  The interface to the OS is via
drivers/rptun.  Is that right?  I don't know enough enough about this to
do the job myself (at least not without some guidance), but I offer
myself as a helper.

Greg

[Xiang Xiao]

On Sunday, November 3, 2019 at 12:38:13 AM UTC+8, patacongo wrote:

I am making some progress, but still no clean build.  Here are the steps that I did:

Get repositories
  git clone https://github.com/FishsemiCode/open-amp.git open-amp
  git clone https://github.com/FishsemiCode/libmetal.git libmetal

Copy header files:
  cp -a  libmetal/lib nuttx/include/metal
  cp -a open-amp/lib/include/openamp/ nuttx/include/openamp

Copy source directories
  cp -a open-amp nuttx/openamp/open-amp
  cp -a libmetal nuttx/openamp/libmetal

I worked around the build problems with this kludge:

find nuttx/include/metal -name "*.h" | xargs sed -i -e "s/@PROJECT_SYSTEM@/nuttx/g"
find nuttx/include/metal -name "*.h" | xargs sed -i -e "s/@PROJECT_PROCESSOR@/x86_64/g"

I know that is not correct, but it lets me continue testing.  I am first trying to get the drivers/ to build.  Most things build now but here are the things that still need to be cleaned up:

Yes, your change is very similar to what we have done in this Makefile:

https://github.com/FishsemiCode/libmetal/blob/song-u1/Make.defs

Anchao will integrate this next week into:

https://bitbucket.org/nuttx/nuttx/src/master/openamp/Makefile

Yes, we need change all to strtoul/(uintptr_t). Again, Anchao will review all code carefully to fix this issue.

We have a very special(stupid?) 32bit audio DSP IP which use the two bytes as the basic unit and then many strange things happen:

1.CHAR_BIT report 16 not 8

2.sizeof(char) report 1, but actually occupy two bytes

3.sizeof(int32_t) report 2, but actually occupy four bytes

4."abc" memory layout like this: 00 61 00 62 00 63 00 00

5.(void *)0x100 really point to 0x200 bytes

we have to use B2C macro to handle these nasty things in a portable way. 

included in strings.h?

What do you think?

b2c.h is a very good place.

 

Greg

[xiang xiao]

On Sun, Nov 3, 2019 at 1:45 AM Gregory Nutt <spud...@gmail.com> wrote:
>
> The last bit about OpenAMP for the day:
>
> ... Most things build now but here are the things that still need to be cleaned up: ...
>
> rptun/rptun.c: In function ‘rptun_thread’:
>
> rptun/rptun.c:179:10: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
> priv = (FAR struct rptun_priv_s *)atoi(argv[1]);
> ^
> rptun/rptun.c: In function ‘rptun_initialize’:
> rptun/rptun.c:807:8: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
> itoa((int)priv, str, 10);
>
> These need to be fixed too. The assumption is that a pointer will fit in a type int. That is not correct. On 64-bit CPUs (such as the host NuttX simulation), sizof(int) is 32-bits but a pointer is 64-bits. The above code will NOT work on the simulator and needs to be fixed
>

Fix in the attached patch.

> This is still a problem and needs to be fixed. So does this one:
>
> hostfs/hostfs_rpmsg.c: In function ‘host_opendir’:
> hostfs/hostfs_rpmsg.c:557:27: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
> return ret < 0 ? NULL : (FAR void *)ret;
> ^
> I did not try to fix these because there may be design changes involved. These fixes will be needed for the 64-bit simulation.
>

It isn't a problem since the handler passed between processor is an
index, not a real pointer.
The code convert int to void * just because hostfs_opendir require this type.
But the warning need be fixed.

> The biggest issue preventing a clean build now the the various macros added to included/strings.h and to include/nuttx/mm/mm.h, all with the naming B2C or C2B. These are apparently added to handle cases where is character is larger than one byte. My gut feeling is that strings.h is not the correct place to put these since they are non-standard. Perhaps they should go in include/nuttx/b2c.h and then be included in strings.h?
>
> I did that. I created include/nuttx/b2c.h and included that in strings.h. I copied the functions for the case of CHAR_BIT != 1 into libs/libc/string/. I did not add the changes to mm.h that you have for related B2C definitions. Those were needed yet.
>
> Now everything compiles cleanly (except for openamp/libmetal which still has some configuration/setup that is not right). So I went ahead and merged the branch into master. It does not add any value yet with no testing and with no build instructions, but it now has been completely reviewed and its presence appears to be harmless to the overall health of the system.
>

Greet, Ancho will take the left work and finish in the next week.

> We do need to have a simple configuration setup script for OpenAMP and libmetal. The build.sh script in your build repository is too focused on your BSP to be usable in the most general NuttX context. Is that something that you could produce? I would greatly appreciate that contribution.
>

Ok, we will upstream the build.sh after clean up. This script do three things:
1.Mount a unionfs to support the out of tree build
2.Call configure.sh with -i to update the config automatically
3.Issue make with the rest command line

> I have removed the OpenAMP branch.
>
> Greg
>
>

> --
> You received this message because you are subscribed to the Google Groups "NuttX" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/2a85a1d5-c2fe-df90-a0d5-93fda41c8cb1%40gmail.com.

[xiang xiao]

On Sun, Nov 3, 2019 at 3:41 AM Gregory Nutt <spud...@gmail.com> wrote:
>
>
> > Just talk with AnChao who do the major work on NuttX/OpenAMP, he can
> > work on the porting in the next week to:
> > 1.Fix the build break
> I think the build is clean now EXCEPT that we need a script or some way
> to configure and setup openamp/libmetal and openamp/open-amp.

Anchao will finish this work soon.

> > 2.Remove Pinecone dependence
> You should verify this, but I think I have all of that removed. We need
> to revisit the clk/regulator logic which I did not bring in and also the
> chip-specific support. As I mentioned before, I would think that the
> chip-specific should go in the boards/ directory in the new
> infrastructure that Alin Jerpelea creates for chip common drivers and logic.

Sure, we can upstream the hardware unrelated portion first, then move
Pinecone specific to arch/board folder.
We still have a common rpmsg code that isn't upstream yet:
https://github.com/FishsemiCode/usrsock
It could redirect Socket API from one cpu to another through usrsock
interface, I think apps/system is a good place.

> > 3.Make OpenAMP work on two sim instance
>
> I think that this would require fixing the casts from type int to
> pointers that I mentioned (or forcing a 32-bit build).
>

The fix attached in the previous email.

> I would be happy to help out AnChao with this part. Since I have
> brought the code into the master branch, I am completely committed to
> making this work smoothly.
>
> My understanding is that the communications is performed in application
> space via the OpenAMP shared memory. The interface to the OS is via
> drivers/rptun. Is that right? I don't know enough enough about this to
> do the job myself (at least not without some guidance), but I offer
> myself as a helper.
>

rptun(rpmsg tunnel) is an adapter layer to simplify arch implement:
1.Load firmware from file system
2.Manage the share memory and resource table
3.Trigger and receive the interrupt from remote
3.Control the remote cpu start, stop and reset
rptun also expose START/STOP control to userspace, so application
could control the remote subsystem as needed.
As you can see, rptun just do the life cycle management. The real
transfer is handled by rpmsg layer.
rpmsg API have two type of user:
1.kernel user like(syslog, uart, fs, netdrv, rtc)
2.userspace user(usrsock, grpc...)
The first user could call rpmsg API directly, the second user need be
handled specially:
1.Expose rpmsg API through the conventional IOCTL interface or
2.Expose rpmsg API trhough a new socket family(e.g. PF_RPMSG)
Both method is fine, but I think the second one may better because:
1.Rpmsg channel is very match Unix Domain Socket concept
2.Many RPC library already support the socket API.
BTW, the attched patch ensure rptun just call file_* API.


> Greg

>
> --
> You received this message because you are subscribed to the Google Groups "NuttX" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/nuttx/edfee466-6987-8fc1-9a06-7050c874f5ef%40gmail.com.

[Xiang Xiao]

One more patch fix a minor error by 0001-Fix-64bit-build-issue-in-rpmsg-driver.patch

> To unsubscribe from this group and stop receiving emails from it, send an email to nuttx+unsubscribe@googlegroups.com.

[Gregory Nutt]

>> The last bit about OpenAMP for the day:
>>
>> ... Most things build now but here are the things that still need to be cleaned up: ...
>>
>> rptun/rptun.c: In function ‘rptun_thread’:
>>
>> rptun/rptun.c:179:10: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
>> priv = (FAR struct rptun_priv_s *)atoi(argv[1]);
>> ^
>> rptun/rptun.c: In function ‘rptun_initialize’:
>> rptun/rptun.c:807:8: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
>> itoa((int)priv, str, 10);
>>
>> These need to be fixed too. The assumption is that a pointer will fit in a type int. That is not correct. On 64-bit CPUs (such as the host NuttX simulation), sizof(int) is 32-bits but a pointer is 64-bits. The above code will NOT work on the simulator and needs to be fixed
>>
> Fix in the attached patch.

Committed.  Thanks!  The solves the problem.

Greg

[Gregory Nutt]

>>
>>> Just talk with AnChao who do the major work on NuttX/OpenAMP, he can
>>> work on the porting in the next week to:
>>> 1.Fix the build break
>> I think the build is clean now EXCEPT that we need a script or some way
>> to configure and setup openamp/libmetal and openamp/open-amp.
> Anchao will finish this work soon.

Could the substitutions from the chip-specific Make.defs be added to
openamp/Makefile?

Perhaps some of the values cound be configuration options selected in
openamp/Kconfig?

> BTW, the attched patch ensure rptun just call file_* API.

I noticed that when I was reviewing the code.  I am glad that you
addressed it.  I have commited the patch.

Greg

[Gregory Nutt]

One more patch fix a minor error by 0001-Fix-64bit-build-issue-in-rpmsg-driver.patch

Thanks!  But I already fixed that one with the original commit.

Greg

[Xiang Xiao]

The new patch add rptun command to start/stop the remote subsystem I mention before.

Please review them , Greg.

Thanks

Xiang

On Sunday, November 3, 2019 at 9:37:00 PM UTC+8, patacongo wrote:

>>
>>> Just talk with AnChao who do the major work on NuttX/OpenAMP, he can
>>> work on the porting in the next week to:
>>> 1.Fix the build break
>> I think the build is clean now EXCEPT that we need a script or some way
>> to configure and setup openamp/libmetal and openamp/open-amp.
> Anchao will finish this work soon.

Could the substitutions from the chip-specific Make.defs be added to
openamp/Makefile?

Perhaps some of the values cound be configuration options selected in
openamp/Kconfig?

Yes, that is what we plan to do: move NuttX related stuff from the sub folder's Make.defs to openamp/Makefile.

[Gregory Nutt]

On 11/2/2019 2:22 AM, xiang xiao wrote:

Just talk with AnChao who do the major work on NuttX/OpenAMP, he can

work on the porting in the next week to:

1.Fix the build break

I think the build is clean now EXCEPT that we need a script or some way to configure and setup openamp/libmetal and openamp/open-amp.

2.Remove Pinecone dependence

You should verify this, but I think I have all of that removed.  We need to revisit the clk/regulator logic which I did not bring in and also the chip-specific support.  As I mentioned before, I would think that this should go in the boards/ directory in the new infracture that Alin Jerpelea creates for chip common dirvers and logic.

3.Make OpenAMP work on two sim instance

I think that this would require fixing the casts from type int to pointers that I mentioned (or forcing a 32-bit build).

I would be happy to help out AnChao with this part.  Since I have brought the code into the master branch, I am completely committed to making this work smoothly.

My understanding is that the communications is performed in application space via the OpenAMP shared memory.  The interface to the OS is vie

 

[Xiang Xiao]

Greg, how about https://github.com/FishsemiCode/usrsock?

It is a general usrsock implementation which could redirect socket API from one cpu to another through OpenAMP.

I think apps/system is a good place. 

Thanks

Xiang

[Gregory Nutt]

Greg, how about https://github.com/FishsemiCode/usrsock?

I haven't studied it in detail, but it seems to be a good addition.

It is a general usrsock implementation which could redirect socket API from one cpu to another through OpenAMP.

I think apps/system is a good place.

Yes, I think apps/system is a good place.  The only other usrsock implementations are under apps/wireless and apps/examples.

Greg