[Boost-users] Boost.MSM cc1plus.exe:: error: out of memory allocating 65532

[RaRi]

Hi all,

I'm using boost.msm from the boost 1.53.0 library.

I have together 8 submachines and almost 50 states distributed between the
differente submachines. The number of events is 44.

Unfortunately I got the following compilation error:
*
cc1plus.exe:: error: out of memory allocating 65532*

First of all I have observed the memory allocation while the compilation
phase: It exceeded 2Gm memory and than I've received that error.

So, my solution was to switch to an 64 bit system with 8 GB memory.

But to hell, I still got this error.

My system environment is configured as follows:

Operating system : Windows 7 64-Bit
MinGW32,
gcc v.4.72

Is this an gcc bug or do I have to extend to 16 GB? But I don't think so, as
the failue is always reported after almost 2 Gb memory allocation while the
compilation phase...

Thank you for your help in advance,

BR,

Rafael





--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468.html
Sent from the Boost - Users mailing list archive at Nabble.com.
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[niXman]

2013/2/26 RaRi:

> My system environment is configured as follows:
>
> Operating system : Windows 7 64-Bit
> MinGW32,
> gcc v.4.72

You are using 32-bit mingw or 64-bit?
Please show the output of 'g++ -v'


--
Regards,
niXman
___________________________________________________
Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows:
http://sourceforge.net/projects/mingwbuilds/
___________________________________________________
Another online IDE: http://liveworkspace.org/

[degoah]

Hi,

I'm User using the 32bit version of gcc. The compiled binary has to be used on a 32 bit system. I know, i can pass to gcc the 32 bit flag "-m32"....


Regards,

R

niXman <i.ni...@gmail.com> schrieb:

2013/2/26 RaRi:

> My system environment is configured as follows:
>
> Operating system : Windows 7 64-Bit
> MinGW32,
> gcc v.4.72

You are using 32-bit mingw or 64-bit?
Please show the output of 'g++ -v'


-- 
Regards,
niXman

Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows:
http://sourceforge.net/projects/mingwbuilds/

Another online IDE: http://liveworkspace.org/

[RaRi]

Hi,

does anyone have a tip, how this problem could be solved?

Maybe the boost.MSM developer/maintainer (Christophe Henry) has some
suggestions....


Thanks so far.




--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643502.html

Sent from the Boost - Users mailing list archive at Nabble.com.
_______________________________________________

[niXman]

2013/2/27 RaRi:

> does anyone have a tip, how this problem could be solved?

Please show the output of 'g++ -v'



--
Regards,
niXman

___________________________________________________

Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows:
http://sourceforge.net/projects/mingwbuilds/

___________________________________________________

Another online IDE: http://liveworkspace.org/

[RaRi]

Hi Nixam!

Please find the output below:

C:\Users\g++ -v
Es werden eingebaute Spezifikationen verwendet.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/4.7.2/lto-wrapper.exe
Ziel: mingw32
Konfiguriert mit: ../gcc-4.7.2/configure
--enable-languages=c,c++,ada,fortran,ob
jc,obj-c++ --disable-sjlj-exceptions --with-dwarf2 --enable-shared
--enable-libg
omp --disable-win32-registry --enable-libstdcxx-debug
--disable-build-poststage1
-with-cxx --enable-version-specific-runtime-libs --build=mingw32
--prefix=/mingw

Thread-Modell: win32
gcc-Version 4.7.2 (GCC)

I'm looking forward for your response.

Br,

RR



--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643504.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[niXman]

2013/2/27 RaRi

> Hi Nixam!
Hi,

> Please find the output below:
>
> C:\Users\g++ -v
> Es werden eingebaute Spezifikationen verwendet.
> COLLECT_GCC=g++
>
> COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/4.7.2/lto-wrapper.exe
> Ziel: mingw32
> Konfiguriert mit: ../gcc-4.7.2/configure
> --enable-languages=c,c++,ada,fortran,ob
> jc,obj-c++ --disable-sjlj-exceptions --with-dwarf2 --enable-shared
> --enable-libg
> omp --disable-win32-registry --enable-libstdcxx-debug
> --disable-build-poststage1
> -with-cxx --enable-version-specific-runtime-libs --build=mingw32
> --prefix=/mingw
>
> Thread-Modell: win32
> gcc-Version 4.7.2 (GCC)

You can increase stack size when build the MinGW.
You need to add '-Wl,--stack=0x2000000' option to LDFLAGS when
cc1plus.exe is linking to get this.

Or you can choose another MinGW build.

--
Regards,
niXman
___________________________________________________
Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows:
http://sourceforge.net/projects/mingwbuilds/
___________________________________________________
Another online IDE: http://liveworkspace.org/

[RaRi]

Hi,

thanks for your fast response.
I've added this flags and first of all it seemed to work properly.
Unfortunately I get now the following
Assembler messages:

Assembler messages:
C:\Users\**\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't write
src\TransportLayer\Statemachine\main.o: File too big
c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/as.exe:
src\TransportLayer\Statemachine\main.o: too many sections (40428)
C:\Users\***\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't close
src\TransportLayer\Statemachine\main.o: File too bi

Do you what is going on here?

BTW: I have tried to compile my boost.MSM code under linux: Under linux I
receive no "...out of memory message" from the compiler and everything works
fine. But this cannot be the solution, as I have to release my boost.MSM
based code as a binary for windows and linux systems. Maybe I could you a
cross-compiler on linux to produce a binary for windows....!?

Any further comments?

Thanks in advanced!

Rafael



--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643517.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[niXman]

2013/2/27 RaRi:

> Assembler messages:
> C:\Users\**\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't write
> src\TransportLayer\Statemachine\main.o: File too big
> c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/as.exe:
> src\TransportLayer\Statemachine\main.o: too many sections (40428)
> C:\Users\***\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't close
> src\TransportLayer\Statemachine\main.o: File too bi

About this you need to write to binutils maling list:
binu...@sourceware.org

--
Regards,
niXman
___________________________________________________
Dual-target(32 & 64-bit) MinGW compilers for 32 and 64-bit Windows:
http://sourceforge.net/projects/mingwbuilds/
___________________________________________________
Another online IDE: http://liveworkspace.org/

[Mathias Gaunard]

On 27/02/13 12:09, RaRi wrote:
> Hi,
>
> thanks for your fast response.
> I've added this flags and first of all it seemed to work properly.
> Unfortunately I get now the following
> Assembler messages:
>
> Assembler messages:
> C:\Users\**\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't write
> src\TransportLayer\Statemachine\main.o: File too big
> c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/as.exe:
> src\TransportLayer\Statemachine\main.o: too many sections (40428)
> C:\Users\***\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't close
> src\TransportLayer\Statemachine\main.o: File too bi
>
> Do you what is going on here?

Looks like you're generating files that are too big for your filesystem.
You're probably using FAT32 or such which is limited to files of 2GB.
You could also look into what could be done to reduce the size of those
binaries. Maybe you're building with debug information enabled?

> Maybe I could you a
> cross-compiler on linux to produce a binary for windows....!?

You can indeed build for Windows from Linux.
On a Debian/Ubuntu system, just install the g++-mingw-w64-i686 package.

[Christophe Henry]

> Hi,
>
> thanks for your fast response.
> I've added this flags and first of all it seemed to work properly.
> Unfortunately I get now the following
> Assembler messages:
>
> Assembler messages:
> C:\Users\**\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't write
> src\TransportLayer\Statemachine\main.o: File too big
> c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/as.exe:
> src\TransportLayer\Statemachine\main.o: too many sections (40428)
> C:\Users\***\AppData\Local\Temp\ccxOY2Lj.s: Fatal error: can't close
> src\TransportLayer\Statemachine\main.o: File too bi
>
> Do you what is going on here?
>
> BTW: I have tried to compile my boost.MSM code under linux: Under linux I
> receive no "...out of memory message" from the compiler and everything
> works
> fine. But this cannot be the solution, as I have to release my boost.MSM
> based code as a binary for windows and linux systems. Maybe I could you a
> cross-compiler on linux to produce a binary for windows....!?
>
> Any further comments?
>
> Thanks in advanced!
>
> Rafael

Hi,
I got this once. I think it's a Windows limitation. I was building in debug
with -O0, I switched to -O2 and it compiled. Though then debugging becomes
less easy... :(

HTH,
Christophe

[Christophe Henry]

> Hi,
>
> does anyone have a tip, how this problem could be solved?
>
> Maybe the boost.MSM developer/maintainer (Christophe Henry) has some
> suggestions....
>
>
> Thanks so far.

You're at the max of what gcc on Windows offers ;-)
Seriously, you have to simplify your machine. 8 submachines are really hard,
plus 44 events, it's already a wonder the compiler survives. You might be
building the God pattern as a fsm.
What I suggest:
- replace submachines by orthogonal regions where you can
- reduce your number of events. Events are where the biggest costs come
from. Use some event hierarchy + guards to differentiate or maybe some
events for serveral similar tasks and differentiate with guards. Sometimes,
some data passed in the event suffices.
- you can try favor_compile_time. It has limitations but it will compile
faster, and on more TUs, which will help
- if none of this help, there is still the last ultimate trick ;-) : force
instantiation of process_event calls for some events on different TUs by
defining the fsm in a header but making the process_event calls in different
.cpp. Ok, this is really as a last resort...
- maybe, simply breaking your fsm into different classes behind interfaces
with smaller fsms would be advisable. It'll also make it easier to
understand.

HTH,
Christophe

[RaRi]

Thank you all so far,

I will investigate, how I can simplify my state machine and reduce the
number of events.

Damn, I spent 1 month on that machine and now I have to change it...

On linux there is no problem with that. Okay, the memory exceeds 1 GB while
compiling my boost.msm based library. But it works fine!

The problem what remains is to compile the stuff for windows.

I have already just test one thing: Cross-compilation of my boost.Msm state
machine on linux for windows with mingw32msvc-g++. But already doing this
produced some errors:

*../src/TransportLayer/Statemachine/Master/CStateMasterHandshaking.hpp:33:
error: class ‘CStateMasterHandshaking_’ does not have any field named
‘BaseStateMachineFrontEnd’
../src/TransportLayer/Statemachine/Master/CStateMasterHandshaking.hpp:33:
error: no matching function for call to
‘BaseStateMachineFrontEnd<CStateMasterHandshaking_>::BaseStateMachineFrontEnd()’
../src/TransportLayer/Statemachine/Master/../Common/BaseStateMachineFrontEnd.hpp:48:
note: candidates are:
BaseStateMachineFrontEnd<TYPE>::BaseStateMachineFrontEnd(EventQueue&) [with
TYPE = CStateMasterHandshaking_]
../src/TransportLayer/Statemachine/Master/../Common/BaseStateMachineFrontEnd.hpp:26:
note:
BaseStateMachineFrontEnd<CStateMasterHandshaking_>::BaseStateMachineFrontEnd(const
BaseStateMachineFrontEnd<CStateMasterHandshaking_>&)*

I think, the MSVC-specific cross compiler is responsible for this error.

I don't now, how to continue with this problem. At the worst I have to
change the state machine completely.....

Thanks so far,

Rafael

--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643543.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[Jeff Flinn]

IIRC, you also discovered turning off /Gm (Enable Minimal Rebuild)
alleviates build issues. :-)

Jeff

[Christophe Henry]

>> Hi,
>> I got this once. I think it's a Windows limitation. I was building in
>> debug with -O0, I switched to -O2 and it compiled. Though then debugging
>> becomes less easy... :(
>
> IIRC, you also discovered turning off /Gm (Enable Minimal Rebuild)
> alleviates build issues. :-)
>
> Jeff

Oh yeah, I remember. Let's see if we can help with g++ too ;-)

Cheers,
Christophe

[Christophe Henry]

Hi,

not sure how to help you there without code, but from what I see, I'd say
that BaseStateMachineFrontEnd has a ctor which hides the default-ctor. Try
to provide one and see if that helps.

I don't think you have to change it completely. Probably replacing a bunch
of events by a single one containing an id as attribute, then checking it in
a guard to see what event you really posted would probably be enough.

Cheers,
Christophe

[RaRi]

Hi Christophe,
I will check the thing with the cstor.

FYI: my library that uses the boost.msm-based state machine mechanism shall
be released as shared library. It is used to realize Remote Procedure Calls.
So my library consists of one Master state machine (which consists of 5
submachines and together 21 states) and one Slave state machine (which
consists of 4 submachines and 19 states). The user application linking
against my library has the possibility to choose if the slave state machine
or the master state machine part shall be used. Now the problem could be,
that I use a boost::variant type to store the 44 different application event
types. This variant type I pass to the process_event method of each state
machine (master and slave) although the slave state machine doesn't support
the same event types like the master and vice-versa. This produces
unnecessary template invocations and memory allocation.

So the solution could be, to split my single shared library into two shared
libraries: one for the master and one for the slave logic and each of it has
a reduced boost::variant type( just with the event types of the concrete
state machine).

Or I keep the solution with the single shared library and I split the
single boost::variant type into two different ones: one for the master and
one for the slave state machine. And each boost::variant type will just be
forwarded to the appropriate process_event method of the corresponding state
machine. Doing so, will also prevent unnecessary template invocations for
each event type on the corresponding state machine. Maybe this will also
reduce the memory consumption...

What is your opinion about this two putative solutions?

Best regards (viele grüsse), 

Rafael

--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643554.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[Christophe Henry]

Hi,

> FYI: my library that uses the boost.msm-based state machine mechanism
> shall
> be released as shared library. It is used to realize Remote Procedure
> Calls.
> So my library consists of one Master state machine (which consists of 5
> submachines and together 21 states) and one Slave state machine (which
> consists of 4 submachines and 19 states). The user application linking
> against my library has the possibility to choose if the slave state
> machine
> or the master state machine part shall be used. Now the problem could be,
> that I use a boost::variant type to store the 44 different application
> event
> types. This variant type I pass to the process_event method of each state
> machine (master and slave) although the slave state machine doesn't
> support
> the same event types like the master and vice-versa. This produces
> unnecessary template invocations and memory allocation.

Yes, every process_event with a different event type costs template
instantiations, and process_event is the most expensive part.

> So the solution could be, to split my single shared library into two
> shared
> libraries: one for the master and one for the slave logic and each of it
> has
> a reduced boost::variant type( just with the event types of the concrete
> state machine).

Sounds logical. Though I don't get it with the variant. Are you doing type
switching with boost::get?
You could also do 1 library with 2 different classes, each implemented with
one different fsm. Maybe some states or actions can be reused, though it
doesn't improve your compile-time.
In any case, 2 TUs are nicer to the compiler than 1 big one, yes.

I will also take the chance to advertise type_erasure (see a previous post
on this list) as a good replacement of boost::variant.

> Or I keep the solution with the single shared library and I split the
> single boost::variant type into two different ones: one for the master and
> one for the slave state machine. And each boost::variant type will just be
> forwarded to the appropriate process_event method of the corresponding
> state
> machine. Doing so, will also prevent unnecessary template invocations for
> each event type on the corresponding state machine. Maybe this will also
> reduce the memory consumption...

If your variant "knows" less types and thus provokes less different
process_event instantiations within one TU, yes.

> What is your opinion about this two putative solutions?

I suppose it depends on your use case. I personally tend to build several
smaller state machines. I find them easier to understand and more likely to
be reused.
Sometimes I go as far as hiding each fsm in the cpp of a different class.
Then I make sure each class follows a given, small interface.
This way, they are easier to exchange with others or to combine. Sure I lose
the tight coupling of submachines, but I don't think I make a bad deal.
But again, this is a matter of taste and of your personal use case.

> Best regards (viele grüsse),
>
> Rafael

Viele Grüße,
Christophe

[Larry Evans]

On 02/27/13 16:14, Christophe Henry wrote:
[snip]
> Hi,

> I will also take the chance to advertise type_erasure (see a previous
> post on this list) as a good replacement of boost::variant.
>

This list has *many* posts. Could you please be more specific about
where the previous post is?
[snip]

[Christophe Henry]

> On 02/27/13 16:14, Christophe Henry wrote:
> [snip]
>> Hi,
>> I will also take the chance to advertise type_erasure (see a previous
>> post on this list) as a good replacement of boost::variant.
>>
> This list has *many* posts. Could you please be more specific about
> where the previous post is?
> [snip]

See thread: [MSM] How to access event-attributes through a "kleene"-event?

[RaRi]

Hi,

Im going to test tomorrow the proposed solutions with fewer event types in
my boost::variant type (which holds currently 44 different event types) and
the additional separation of the library into two TU's. Maybe I'll also test
the erasure type you've mentioned.

Regarding my boost::variant: It is used to compose 44 different event types.
Additionally this event type is passed as template parameter to the
stl::queue <> container , which is used as global event queue for all my
state machines (submachines). So each state of a submachine can put in a
corresponding event to that special/global event queue. Afterwards my "event
dispatcher" gets from the global event queue the latest event and forwards
it to the process_event method. I use the approach with the global event
queue due to the fact, that the submachines in boost.msm are not able to put
events into the event queue of the parent state machine (enclosing state
machine). (We already talk about this almost two month ago if you
remember...).

I will inform you tomorrow about my proceeding.

Bye,

Rafael



--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643569.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[Christophe Henry]

> Regarding my boost::variant: It is used to compose 44 different event
> types.
> Additionally this event type is passed as template parameter to the
> stl::queue <> container , which is used as global event queue for all my
> state machines (submachines). So each state of a submachine can put in a
> corresponding event to that special/global event queue. Afterwards my
> "event
> dispatcher" gets from the global event queue the latest event and forwards
> it to the process_event method. I use the approach with the global event
> queue due to the fact, that the submachines in boost.msm are not able to
> put
> events into the event queue of the parent state machine (enclosing state
> machine). (We already talk about this almost two month ago if you
> remember...).

Hi,

wait, I never said anything like "the submachines in boost.msm are not able
to put events into the event queue of the parent state machine"!!! All I
said is that they don't know their parents. Actually they can and there are
2 ways:
- the UML conform. Use a pseudo exit, this will move the flow of control to
the parent fsm, in the out transition you can call process_event, which will
post... in the parent fsm queue.
- C++ tricks: after creating the parent fsm, which also creates all
submachines, you can pass to the submachine a pointer to the parent on which
you can call process_event. You will have to forward-declare the parent fsm,
but hell, it's still doable. Alternatively, you can use an event and pass
the pointer in the event, or more cleanly, to avoid cycles, which are a bad
design, pass instead a boost::function or type_erasure, which calls map to a
process_event on the parent.

I don't see a reason for a global queue (global is such a bad word...).

Cheers,
Christophe

[Christophe Henry]

>Hi Christophe!
>
>...yes, you told me that the submachines have no direct access to its parent state machine. Sorry for the confusion...

No problem. I just want to make sure others will not misunderstand.

 

>The reason why we are using a "global" event queue (each submachine has access to it) is because we don't think that it is not a

>good design to call process_event from within a submachine: doing so, we will lose the advantages of a hierachical

>statemachine design.

 

I admit I am not completely decided on this. Ususally I prefer to give the submachine from outside a boost::function or something equivalent or I exit the submachine with a pseudo exit, even if I have to re-enter it. This preserves my encapsulation and avoids a global queue, but I admit it is more work.

In any case, I need to provide a better solution. I had no luck so far but I don't despair ;-)

 

>Further, how you will stop a submachine in case it calls permantely its process_event method:

>doing so would cut the possibility to stop the complete statemachine from the outside world!

I'm not sure I understand this. Permanently would mean stack overflow, in any case, outside or not it is dead ;-)

 

>Our approach is to start an event collector, which is gathering events from different sources and puts it afterwards into the "global"

>event queue. In the next step an event dispatcher pops the events from our "global" queue of type std::queue <boost::variant <eventtype1, eventtype2, ..., eventtype44>>

>and forwards it to to the process_event method of the hierarchical Master or Slave state machines, which perform the appropriate transistions and actions.

>Note: within one state an event can be put to the "global" event queue, but it is in our opinion a bad design to call the process_event

>method of the current sub machine. We used the book" Practical UML Statecharts in C/C++: Event-Driven Programming for

>Embedded Systems - Miro Samek" as basis for such kind of design( not calling process_event event from within submachines).
>
>I would very appreciate your opinion regarding my comment above.

I don't see why it would be bad design to call process_event on either the outer or the submachine. But I didn't read this book.

As long as the library makes sure that an event is completed before the next one starts (which msm does), I see no problem.

Furthermore, if calling process_event on the submachine is enough for some events, you have 1) a more localized code (good) and 2) a better performance (you avoid processing the event from the outer to the submachine, so one process_event less).

But really, if you like the global queue, I don't want to talk you out of it ;-)

There is no such thing as the Unique Perfect Solution and I surely don't have it.

 

>Regarding the " out of memory problem": Today I've splitted my library into two parts: one slave and one master shared library. So the slave state machine does only now 20 events and the master knows 24 events.

>We have now a better compilation performance, but the memory allocation is still big for master state machine: at the highest almost 900Mb memory is being allocated while the compilation on windows.

>In comparison to the approach with a slave and master state machine in one shared library we saved almost 400Mb memory allocation while compile time.

>This all is already compiled without debug information and it is still a massive memory consumption.

Sorry, I know too little about compilers to provide an answer, sounds strange that it makes a difference. OTOH 900Mb is not unreasonable for heavily templatized code.

 

>Tomorrow I will have a look if I can reduce the number of events or maybe even the number of submachines...
>
>Question: does the use of guards and actions in a transition table also produce compilation time (template invocations)

>in boost.msm like with events? Please say no,

 

No :)

Seriously, no, I never found a measurable impact.

 

 

>cause I still have to define actions and some guards into the state machines.

>If yes, so we have ultimatively to decide if it would be better to switch to another state machine library (state chart, etc.).

>Actually, I don't want this:)

Me neither ;-)

But clearly, statechart would compile faster, yes.

Or you can give a try to favor_compile_time. With so many submachines, it could help much.

>Br,
>
>Rafael

HTH,

Christophe

 

[RaRi]

Hi Christophe!

...yes, you told me that the submachines have no direct access to its parent state machine. Sorry for the confusion...

The reason why we are using a "global" event queue (each submachine has access to it) is because we don't think that it is not a good design to call process_event from within a submachine: doing so, we will lose the advantages of a hierachical statemachine design. Further, how you will stop a submachine in case it calls permantely its process_event method: doing so would cut the possibility to stop the complete statemachine from the outside world!

Our approach is to start an event collector, which is gathering events from different sources and puts it afterwards into the "global" event queue. In the next step an event dispatcher pops the events from our "global" queue of type std::queue <boost::variant <eventtype1, eventtype2, ..., eventtype44>> and forwards it to to the process_event method of the hierarchical Master or Slave state machines, which perform the appropriate transistions and actions. Note: within one state an event can be put to the "global" event queue, but it is in our opinion a bad design to call the process_event method of the current sub machine. We used the book" Practical UML Statecharts in C/C++: Event-Driven Programming for Embedded Systems - Miro Samek" as basis for such kind of design( not calling process_event event from within submachines).

I would very appreciate your opinion regarding my comment above.

Regarding the " out of memory problem": Today I've splitted my library into two parts: one slave and one master shared library. So the slave state machine does only now 20 events and the master knows 24 events. We have now a better compilation performance, but the memory allocation is still big for master state machine: at the highest almost 900Mb memory is being allocated while the compilation on windows. In comparison to the approach with a slave and master state machine in one shared library we saved almost 400Mb memory allocation while compile time. This all is already compiled without debug information and it is still a massive memory consumption.

Tomorrow I will have a look if I can reduce the number of events or maybe even the number of submachines...

Question: does the use of guards and actions in a transition table also produce compilation time (template invocations) in boost.msm like with events? Please say no, cause I still have to define actions and some guards into the state machines. If yes, so we have ultimatively to decide if it would be better to switch to another state machine library (state chart, etc.). Actually, I don't want this:)

Br,

Rafael

"Christophe Henry-3 [via Boost]" <[hidden email]> schrieb:

[hidden email]
http://lists.boost.org/mailman/listinfo.cgi/boost-users

---

If you reply to this email, your message will be added to the discussion below:

http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643611.html

To unsubscribe from Boost.MSM cc1plus.exe:: error: out of memory allocating 65532, click here.
NAML

---

View this message in context: Re: Re: Boost.MSM cc1plus.exe:: error: out of memory allocating 65532

[RaRi]

Hey!

Thanks for your fast and helpful response!

One point regarding the "favor_compile_time" approach: I've alrady used it
for the outermost and all enclosed submachines already. Unfortunately I
couldn't detemine such a deep performance advantage. But one comment makes
me a bit sceptical: two days avo you have suggested another user to also use
favor_compile_time in context with the eUML front-end and kleene events, but
according to your comment it has a disadvantage: the statemachines will lose
the hierarchical support. Will this also happen for the functor-frontend?
Maybe I've misunderstood something...

Ciao,

Rafael



--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643622.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[Christophe Henry]

> Hey!
>
> Thanks for your fast and helpful response!
>
> One point regarding the "favor_compile_time" approach: I've alrady used it
> for the outermost and all enclosed submachines already. Unfortunately I
> couldn't detemine such a deep performance advantage. But one comment
> makes
> me a bit sceptical: two days avo you have suggested another user to also
> use
> favor_compile_time in context with the eUML front-end and kleene events,
> but
> according to your comment it has a disadvantage: the statemachines will
> lose
> the hierarchical support. Will this also happen for the functor-frontend?
> Maybe I've misunderstood something...
>
> Ciao,
>
> Rafael

There are limitations but it usually compiles a bit faster, plus as the fsm
is divided between different TUs, with -j9 (8 submachines + outer) you
should see some difference ;-) (supposing you really use several cpp files)

The limitation is not the fsm hierarchy, which of course still works. What
you lose is the event hierarchy, the capability to use in a transition an
event, which is the base class of the event you actually process. Little
known feature but useful, unfortunately lost in this mode.
And yes, it will happen in all front-ends :(

Cheers,
Christophe

[RaRi]

Servus!

I have investigated the memory allocation while compiling my master state
machine with the favor_compile_time policy (5 submachine -> 5 *.hpp-files +
5 *.cpp files + each submachine back-end has the favor_compile_time policy
as 2nd parameter) and without the favor_compile_time policy (5 submachines
-> 5 *.hpp-files + 1 main.cpp with the instantiation of the master machine)

My results:
*Without policy:*
Memory allocation: 600 Mb; Compile time: 01:10min; dll size: 1 Mb;
Conclusion -> with the "-g3" option we run out of memory and we will receive
the curious "out of memory allocation" problem from the cc1plus.exe again.

*With policy:*
Memory allocation: 400 Mb; Compile time: 01:50min; dll size: 1.5 Mb;
Conclusion -> we can now also use the "-g3" without running into the
"cc1plus.exe out of memory problem".

But why is the compilation time with "favor_compile_time" bigger than
without the policy? And why to hell is the dll file size about 50 % bigger
(okay, we have know 5 object files and with additional symbols, maybe this
is the reason... I'm not an expert regarding this..)

Christophe, I'm curious about your opinion:)...

BR,
Rafael






--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643644.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[Christophe Henry]

> Servus!
>
> I have investigated the memory allocation while compiling my master state
> machine with the favor_compile_time policy (5 submachine -> 5 *.hpp-files
> +
> 5 *.cpp files + each submachine back-end has the favor_compile_time policy
> as 2nd parameter) and without the favor_compile_time policy (5 submachines
> -> 5 *.hpp-files + 1 main.cpp with the instantiation of the master
> machine)
>
> My results:
> *Without policy:*
> Memory allocation: 600 Mb; Compile time: 01:10min; dll size: 1 Mb;
> Conclusion -> with the "-g3" option we run out of memory and we will
> receive
> the curious "out of memory allocation" problem from the cc1plus.exe again.
>
> *With policy:*
> Memory allocation: 400 Mb; Compile time: 01:50min; dll size: 1.5 Mb;
> Conclusion -> we can now also use the "-g3" without running into the
> "cc1plus.exe out of memory problem".

Are you talking about the added compile time of 5 modules or does the
parallel build take 01:50min?
If latest, I'd be very suprised. The biggest advantage of this mode is the
ability to build faster because of parallel building.
That the added time is higher is possible yes. For example, the outer fsm
still need to instantiate a some templates for its submachines.
The gain comes from the process_event instantiations, which are divided
between several modules.

> But why is the compilation time with "favor_compile_time" bigger than
> without the policy? And why to hell is the dll file size about 50 % bigger
> (okay, we have know 5 object files and with additional symbols, maybe this
> is the reason... I'm not an expert regarding this..)

As I said, with the policy, there is a bit more work for the compiler. How
much depends mostly on the number of states of the submachines.
The 50% bigger probably come from the symbols yes. gcc is known for this,
it's about the only point where VC is better.
That said, I'm not a compiler writer ;-)

> Christophe, I'm curious about your opinion:)...
>
> BR,
> Rafael

HTH,
Christophe

[RaRi]

Hi,

>Are you talking about the added compile time of 5 modules or does the
>parallel build take 01:50min?

The complete compilation time with two parallel processes takes without
policy 01:10min and with policy it takes 01:50min.

Okay Christophe, I just think, there is no workaround for me to keep the
slave machine and the master machine in one dll: The memory problem is gone
( in case I use the favor_compile_time policy), but now I receive "bad reloc
address in section xyz" messages from the linker regarding the object files
of the master and slave state machines (which are compiled in their own
TU's.)

Here is the failure message:

*ld.exe:
src\TransportLayer\Statemachine\EventDispatcher\SlaveEventDispatcher.o: bad
reloc address 0x45 in section
`.text$_ZN5boost3msm4back14dispatch_tableINS1_13state_machineI18SlaveStateMachine_NS1_18favor_compile_timeENS_9parameter5void_ES7_S7_EENS_3mpl6v_itemINS8_5_row_INS0_5front3RowINS3_I23CStateSlaveHandshaking_S5_S7_S7_S7_E7exit_ptINSE_29ExitPointHandshakingSucceededEEENSC_4noneENS3_I23CStateSlaveOperational_S5_S7_S7_S7_EESJ_SJ_EEEENSA_INSB_INSD_INSL_7exit_ptINSK_28ExitPointMasterHeartbeatLostEEESJ_SF_SJ_SJ_EEEENSA_INS8_6a_row_INSD_INS4_12StateNoErrorE15EventFatalErrorNS4_14StateTerminateEN11ActionError11ReportErrorESJ_EEEENSA_INST_INSD_ISU_17EventNoTransitionSU_NSX_18ReportNoTransitionESJ_EEEENS9_7vector0IN4mpl_2naEEELi1EEELi1EEELi1EEELi1EEE15EventInitFailedS5_E15call_submachineISL_EENS1_11HandledEnumERS8_iiRKS1D_[__ZN5boost3msm4back14dispatch_tableINS1_13state_machineI18SlaveStateMachine_NS1_18favor_compile_timeENS_9parameter5void_ES7_S7_EENS_3mpl6v_itemINS8_5_row_INS0_5front3Ro_?I'*

The *SlaveEventDispatcher.o* contains the instantiation of the Slave state
machine and the process_event call for the boost::variant type (containing
the 20 event types). I use the boost::static_visitor to travers over all
event types and to pass the concrete event afterwards to the process_event
method:

*template<class TStateMachine>
class EventTypeVisitor : public boost::static_visitor<>{

private:
// Reference to the concrete state machine
TStateMachine& m_concreteStateMachine;

public:
EventTypeVisitor(TStateMachine& stateMachine) :
m_concreteStateMachine(stateMachine) {};

// Generic-visitor operator function.
template<class TEventType >
void operator()(TEventType & event) {
// Forward the event to the concrete state machine.
m_concreteStateMachine.process_event(event);
};
};*



...splitting the master and slave state machines into two different dlls
produces no problems.

Hmm, I was thinking, if you would be interested to take a look at my code. I
can send it to you via personal mail etc.. Maybe there is a bug in the use
of your library or what ever...

BR,

Rafael





--
View this message in context: http://boost.2283326.n4.nabble.com/Boost-MSM-cc1plus-exe-error-out-of-memory-allocating-65532-tp4643468p4643656.html

Sent from the Boost - Users mailing list archive at Nabble.com.

[Christophe Henry]

>>Are you talking about the added compile time of 5 modules or does the
>>parallel build take 01:50min?
>
> The complete compilation time with two parallel processes takes without
> policy 01:10min and with policy it takes 01:50min.

Ok, with 2 processes, yes, it might be.

> Okay Christophe, I just think, there is no workaround for me to keep the
> slave machine and the master machine in one dll: The memory problem is
> gone
> ( in case I use the favor_compile_time policy),

I also think it's better to separate. When the compiler comes close to the
breaking point, the next problem is not far anyway.

>but now I receive "bad reloc
> address in section xyz" messages from the linker regarding the object
> files
> of the master and slave state machines (which are compiled in their own
> TU's.)

> Here is the failure message:
>
> *ld.exe:
> src\TransportLayer\Statemachine\EventDispatcher\SlaveEventDispatcher.o:
> bad
> reloc address 0x45 in section
> `.text$_ZN5boost3msm4back14dispatch_tableINS1_13state_machineI18SlaveStateMachine_NS1_18favor_compile_timeENS_9parameter5void_ES7_S7_EENS_3mpl6v_itemINS8_5_row_INS0_5front3RowINS3_I23CStateSlaveHandshaking_S5_S7_S7_S7_E7exit_ptINSE_29ExitPointHandshakingSucceededEEENSC_4noneENS3_I23CStateSlaveOperational_S5_S7_S7_S7_EESJ_SJ_EEEENSA_INSB_INSD_INSL_7exit_ptINSK_28ExitPointMasterHeartbeatLostEEESJ_SF_SJ_SJ_EEEENSA_INS8_6a_row_INSD_INS4_12StateNoErrorE15EventFatalErrorNS4_14StateTerminateEN11ActionError11ReportErrorESJ_EEEENSA_INST_INSD_ISU_17EventNoTransitionSU_NSX_18ReportNoTransitionESJ_EEEENS9_7vector0IN4mpl_2naEEELi1EEELi1EEELi1EEELi1EEE15EventInitFailedS5_E15call_submachineISL_EENS1_11HandledEnumERS8_iiRKS1D_[__ZN5boost3msm4back14dispatch_tableINS1_13state_machineI18SlaveStateMachine_NS1
> _18favor_compile_timeENS_9parameter5void_ES7_S7_EENS_3mpl6v_itemINS8_5_row_INS0_5front3Ro_?I'*

Hmmm, never seen this. I'd be surprised if it was a msm bug. Usually, msm
bugs are at the compiler level. But I have seen many strange linker problems
when coming to the limit, yes.

> ...splitting the master and slave state machines into two different dlls
> produces no problems.

This confirms what I thought.

> Hmm, I was thinking, if you would be interested to take a look at my code.
> I
> can send it to you via personal mail etc.. Maybe there is a bug in the use
> of your library or what ever...

I can have a look. It could take a few days before I can look thoughly
though, but I might be able to give you a few tips.
You know my email address ;-)

> BR,
>
> Rafael

Cheers,
Christophe