Crestron performance Problems with Modul 4.3 and 4.4

[Jens]

Hi all,

i´ve updated my Crestron Modul from 4.1 to Version 4.4 today and
detected some very bad performance problems of my cp2e prozessor. The
ReaLIdle threat is 0.0% and the SMn0006 thread has over 86%. When I
changed back to 4.1 i have a ReaLIdle threat of 78% an no SMn0006
thead is shown.

Does anyone have the same problem ?

thanks for answer and sorry for my english ;-)

jens

[Jarrod Bell]

Try 4.3

Jarrod

[Jens]

Hi Jarrod,

I have the same Problem with 4.3, I don´t have 4.2 so I can only say
that 4.1 is OK and 4.3 and 4.4 have the Problem.

thx Jens

[Jason Melvin]

The high processor usage in 4.4 is due to the while(1) loop in the main loop.  Adding a delay(15) fixes that by slowing it down a bit without any noticeable lag.

Jarrod: care to comment why you are not using Gather()?

--
You received this message because you are subscribed to the Google Groups "CommandFusion" group.
To post to this group, send email to comman...@googlegroups.com.
To unsubscribe from this group, send email to commandfusio...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/commandfusion?hl=en.

[Jens]

Hi Jason,

thx for your help, with the delay the ReaLIdle thread is back with 79%

cu Jens

On 10 Dez., 18:00, Jason Melvin <jwmel...@gmail.com> wrote:
> The high processor usage in 4.4 is due to the while(1) loop in the main
> loop.  Adding a delay(15) fixes that by slowing it down a bit without any
> noticeable lag.
>
> Jarrod: care to comment why you are not using Gather()?
>

> On Fri, Dec 10, 2010 at 7:46 AM, Jens <jensvierkot...@googlemail.com> wrote:
> > Hi Jarrod,
>
> > I have the same Problem with 4.3, I don´t have 4.2 so I can only say
> > that 4.1 is OK and 4.3 and 4.4 have the Problem.
>
> > thx Jens
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "CommandFusion" group.
> > To post to this group, send email to comman...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > commandfusio...@googlegroups.com<commandfusion%2Bunsu...@googlegroups.com>
> > .

[Jarrod Bell]

I guess we should be using Gather now that all the processing is done in the main function loop.
Originally we werent using Gather because it allows for a task switch which would result in other incoming data being parsed before the current set of data if a long list of commands came in at once.

Jarrod

[Jason Melvin]

Gather should probably be in a SocketReceive task, and the point of it is to continue to gather incoming data until a particular delimiter character is reached. In my experience you still need a semaphore to prevent reentrancy (which is inconsistent with the Crestron help file instructions for Gather), but that done, the gather() statement will continue to read the incoming data into a variable.  Once the gather has executed, you can release the data to the processing routine and clear the semaphore for another gather to trigger.  I will do some tests if I get some time to see how it works out. 

-jason

[Jarrod Bell]

One thing I noticed if doing the processing in the SocketReceive event is that when multiple commands come in at once, and hence require multiple gathers, other data can come in and be processed before the first messages are all done processing, which makes them end up being processed out of order. Not a good idea when a button release is processed before a button press, and hence the button remains pressed (think volume ramping).

Jarrod

[EtienneTerblanche]

Jason,

Could you tell me where you added the delay(15)? Within the While(1)
loop, just after the first IF statement? As below...


WHILE (1) {
// DIRTY WAY TO ENSURE DATA IS PROCESSED AS IT COMES -IN ORDER-,
THANKS TO CRESTRON WANTING TO TASK SWITCH WHENEVER IT FEELS LIKE IT
IF (LEN(incomingBuffer) > 0) {
temp = Remove("\x03", incomingBuffer);
temp = LEFT(temp, LEN(temp) - 1);
if (LEN(temp) > 0) {
ProcessCommands(temp);
}
}
ProcessLogic();
Delay(1)
}



Thank you,
Et

On Dec 10, 7:00 pm, Jason Melvin <jwmel...@gmail.com> wrote:
> The high processor usage in 4.4 is due to the while(1) loop in the main
> loop.  Adding a delay(15) fixes that by slowing it down a bit without any
> noticeable lag.
>
> Jarrod: care to comment why you are not using Gather()?
>

> On Fri, Dec 10, 2010 at 7:46 AM, Jens <jensvierkot...@googlemail.com> wrote:
> > Hi Jarrod,
>
> > I have the same Problem with 4.3, I don´t have 4.2 so I can only say
> > that 4.1 is OK and 4.3 and 4.4 have the Problem.
>
> > thx Jens
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "CommandFusion" group.
> > To post to this group, send email to comman...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > commandfusio...@googlegroups.com<commandfusion%2Bunsu...@googlegroups.com>
> > .

[Simon Cope]

Etienne,

I changed the Main() function in 4.4 to this yesterday (so far so good).  Do you think this is robust enough, or do we need to be including a semaphore somewhere as well?

Function Main()

 {

 STRING temp[255];

 AccessGranted = 0;

 Offline = 1;

 

 WHILE (1)

  {

  temp = GATHER ("\x03", incomingBuffer);

  temp = LEFT (temp, LEN(temp) - 1);

  if (LEN(temp) > 0)

   {

   ProcessCommands(temp);

            }                                                                                          

  DELAY (5);

  }

 }

Simon

To unsubscribe from this group, send email to commandfusio...@googlegroups.com.

[Jason Melvin]

Yes, that is how I modified it also:  with the delay() statement inside the while() loop but outside the if() statement.  I noticed that I am actually using delay(10).

[Jason Melvin]

Sorry, I missed your modification using Gather(). While that is generally the approach I would take, the point of using Gather() is to avoid the while(1) loop.  So try moving the Gather() into the SocketRecieve event and comment out the while(1) loop.  You will have to test to see if you need a semaphore. 

[Simon Cope]

Jason,

The problem with using Gather() on an event is that it is not suited to dealing with high volumes of data.  Crestron processors have an overhead each time they have to create a Simpl+ thread (I believe each instance takes 20ms), so if you are triggering lots of Simpl+ thread creation (and subsequent destruction) due to your SocketReceive event being repeatedly triggered, you're going to run into problems.

Using While(1) is an accepted method (by the cleverer people at Crestron TrueBlue!) of removing the thread creation overhead by ensuring that your Simpl+ thread is created only once, and then stays open.  I removed Jarrod's IF (or was it a 2nd WHILE?) in v4.4 and replaced it with a GATHER, as GATHER performs the equivalent of a PROCESSLOGIC() as it is working, thereby allowing other logic to run while it is looking for the delimiter.

Hope that makes sense!

Regards,

Simon Cope

Sensory International

[Jason Melvin]

I understand your point but even though you are persisting the while(1) loop, you will still have multiple SocketReceive events triggered, right? And right now there is no semaphore-gather used to force the initial SocketReceive to wait until the delimiter is received. So your multiple SocketReceive events will be feeding pieces of the data into incomingBuffer until the delimiter is reached and Remove (or Gather in your case) can successfully execute.  

Please explain if I'm missing something. It seems like using Gather rather than Remove in the while(1) loop will just halt execution of the loop until the delimiter is received. It may be more efficient than repeatedly attempting to execute Remove, but but I'm not sure why it would result in fewer threads.  I have no problem with the while(1) loop, but I don't see what it offers compared to staying within a semaphore-protected SocketReceive.

Just being able to discuss this is very helpful, thank you,

Jason

[Simon Cope]

Jason,

Yes, the multiple SocketReceive events are still being triggered, but because of the WHILE(1) in Main they are no longer creating/destroying Simpl+ threads, which is the main reason behind using a WHILE(1) loop in the first place.  You have one Simpl+ thread, triggered by WHILE(1), that stays open indefinitely.

Having GATHER in the WHILE(1) loop does not halt execution of the loop, as GATHER effectively performs the equivalent of a PROCESSLOGIC() while it is looking for the specified delimiter.

I think that's how it works....

:-)

Simon

[Jason Melvin]

Thanks Simon. I started a discussion on the Crestron group in the hope that we can draw from a larger pool of knowledge:

http://tech.groups.yahoo.com/group/Crestron/message/130460

[Jason Melvin]

After a pretty thorough discussion on the Crestron group, I modified the module to eliminate the SocketReceive event and put the while() loop in the SocketConnect event, using a gather() to look for new data.  That way, the gather loop is only active when the socket is connected and will simply look at the socket's incoming buffer and take off each command in order. It should reduce the number of events triggered.

So far it seems to be working for me with pretty low processor usage. I'd be interested to hear how it works out for anyone else. Simon, this is almost the same as what you had done, but eliminates the SocketReceive completely.  I didn't preserve the ReceivedData pulse but you could add that within the while() loop just before the retimewait() statement.

Available at: http://pastebin.com/0wzxFCkL

-jason

[Simon Cope]

Jason,

Thanks for that - been trying to do the same myself today, but failing miserably.  My brain just isn't working properly at all at the moment!

I'll give your revised module a go now.

Thanks again,

Simon

[Jason Melvin]

Well, after more discussion in the Crestron group regarding the operation of Gather(), I realized that leaving the while(1) loop in main is no problem. SO, I'm basically back to where you were, without the SocketReceive event.  

I did, however, change how the ProcessCommands function operates, because I don't like RemoveByLength.  It seem unnecessary so I rewrote using mid(). The theory is that it may be faster to avoid rewriting the Request string, and just use whatever piece of it is required at any given point. I think that all possible efforts should go into efficiency, since the Crestron sometimes struggles.

My current version:

http://pastebin.com/rCXaQLuD 

Best,

Jason

[EtienneTerblanche]

Guys,

Any update on this and if you're recommended changes are an
improvement on the current version 4.4 module?


Thanks!

On Dec 17 2010, 4:41 am, Jason Melvin <jwmel...@gmail.com> wrote:
> Well, after more discussion in the Crestron group regarding the operation of
> Gather(), I realized that leaving the while(1) loop in main is no problem.
> SO, I'm basically back to where you were, without the SocketReceive event.
>
> I did, however, change how the ProcessCommands function operates, because I
> don't like RemoveByLength.  It seem unnecessary so I rewrote using mid().
> The theory is that it may be faster to avoid rewriting the Request string,
> and just use whatever piece of it is required at any given point. I think
> that all possible efforts should go into efficiency, since the Crestron
> sometimes struggles.
>
> My current version:http://pastebin.com/rCXaQLuD
>
> Best,
> Jason
>
> On Thu, Dec 16, 2010 at 10:32 AM, Simon Cope <sjc...@gmail.com> wrote:
> > Jason,
>
> > Thanks for that - been trying to do the same myself today, but failing
> > miserably.  My brain just isn't working properly at all at the moment!
>
> > I'll give your revised module a go now.
>
> > Thanks again,
>
> > Simon
>

> > On 16 December 2010 15:30, Jason Melvin <jwmel...@gmail.com> wrote:
>
> >> After a pretty thorough discussion on the Crestron group, I modified the
> >> module to eliminate the SocketReceive event and put the while() loop in the
> >> SocketConnect event, using a gather() to look for new data.  That way, the
> >> gather loop is only active when the socket is connected and will simply look
> >> at the socket's incoming buffer and take off each command in order. It
> >> should reduce the number of events triggered.
>
> >> So far it seems to be working for me with pretty low processor usage. I'd
> >> be interested to hear how it works out for anyone else. Simon, this is
> >> almost the same as what you had done, but eliminates the SocketReceive
> >> completely.  I didn't preserve the ReceivedData pulse but you could add that
> >> within the while() loop just before the retimewait() statement.
>
> >> Available at:http://pastebin.com/0wzxFCkL
>
> >> -jason
>

[Jason Melvin]

It's working really well for me. No problems yet and it definitely reduced the number of SIMPL+ threads. Simon reported that it didn't work for him but I'm not sure he ever figured out why. We are both using the latest Crestron firmware and latest iViewer.

[Greg S.]

The modified module Jason posted is working great for me as well. I
was having issues with buttons not releasing (volume ramps) and the
changes he made seemed to have fixed that problem completely.

On Jan 4, 9:46 am, Jason Melvin <jwmel...@gmail.com> wrote:
> It's working really well for me. No problems yet and it definitely reduced
> the number of SIMPL+ threads. Simon reported that it didn't work for him but
> I'm not sure he ever figured out why. We are both using the latest Crestron
> firmware and latest iViewer.
>
> On Tue, Jan 4, 2011 at 4:35 AM, EtienneTerblanche

> <etie...@4ward-design.com>wrote:

[Simon Cope]

Hi Jason,

Yep, it's a new year and I'm still tearing my hair out!

I've spent this morning setting up a basic test system in the office - stand-alone router/AP/CP2E/iPad - and created a stripped down smw file with only your v4.4.3 module in it, and a stripped down CF gui file as well.  I've attached them all in a zip file for anybody else who is interested in testing it out for me.

I really can't spot what I'm doing wrong.  When you trigger Enable_Connections, the CF iViewer app connects fine, but then nothing appears to be getting received by the Crestron processor - the "new command gathered" debug line in the main while(1) loop is never triggered.  After CF iViewer on the iPad not receiving any response to its heartbeat ping, it times out, closes and re-opens the connection from its end.  Repeat ad infinitum.

Aarrrgghhh!

Simon

--

[Karl Addrison]

Just tried this and connects – can display / hide system bar from simpl debugger (just changed the system ip address)

[Simon Cope]

Thanks for trying it Karl.  Out of interest, did you re-compile the Simpl Windows program, or just upload the .spz that was in the .zip file?  I'm starting to clutch at straws now, and am thinking I have a problem with some part of my Crestron software installation!

Simon

[Karl Addrison]

Compiled it – worked

 

Just tried your spz and does not look like it works – (very quick test thou)

 

Karl

[Karl Addrison]

Simon

 

Forget that yours also works

 

Karl

[Karl Addrison]

More feedback – your spz disconnects very quickly – but will work (show hide system bar) while connected

 

My SPZ attached

[Simon Cope]

OK, thanks.  Can you confirm that your spz holds its connection OK - i.e. more than 3 seconds (the interval between the heartbeat messages that the iViewer app sends out)?

I'm guessing we need to compare and contrast Simpl Windows installations as well.  I've currently got:

SIMPL Windows:  2.12.39

SIMPL+:  3.03.12

        Path:  C:\Program Files\Crestron\Simpl

Data Files:

        Path: C:\Program Files\Crestron\Cresdb\Programming

        symlib.tio:  690

        iodev.tio:  690

        Include3.dat 1.69

Other Crestron SW:

        Device Database: 31.05.005.00

        Crestron Database:  23.05.010.01

        Crestron Toolbox: 1.23.056.25

Legacy Device Tools:  Not Available

Windows System Information:   

        Windows OS: Windows XP Professional Edition Service Pack 3

        Windows User Name: simonc

        Administrator Privileges: Yes

[Karl Addrison]

About 16 sec it stays on line

 

 

19:35:37 - 01-06-2011: [UI_05]_[iViewer]_Socket_Status~ -> 2d

19:35:37 - 01-06-2011: [UI_05]_[iViewer]_Offline -> 0

19:35:53 - 01-06-2011: [SIMPL+ Trace] :

19:35:53 - 01-06-2011: [SIMPL+ Trace] :

19:35:53 - 01-06-2011: [UI_05]_[iViewer]_Socket_Status~ -> 4d

19:35:53 - 01-06-2011: [UI_05]_[iViewer]_Offline -> 1

19:35:53 - 01-06-2011: [UI_05]_[iViewer]_Socket_Status~ -> 2d

19:35:53 - 01-06-2011: [UI_05]_[iViewer]_Offline -> 0

19:36:09 - 01-06-2011: [SIMPL+ Trace] :

19:36:09 - 01-06-2011: [SIMPL+ Trace] :

19:36:09 - 01-06-2011: [UI_05]_[iViewer]_Socket_Status~ -> 4d

19:36:09 - 01-06-2011: [UI_05]_[iViewer]_Offline -> 1

19:36:09 - 01-06-2011: [UI_05]_[iViewer]_Socket_Status~ -> 2d

19:36:09 - 01-06-2011: [UI_05]_[iViewer]_Offline -> 0

19:36:32 - 01-06-2011: [SIMPL+ Trace] :

My Simpl looks newer

 

SIMPL Windows:  2.12.44

SIMPL+:  3.03.13

        Path:  C:\Program Files (x86)\Crestron\Simpl

 

Data Files:

        Path: C:\Program Files (x86)\Crestron\Cresdb\Programming

 

        symlib.tio:  690

        iodev.tio:  690

        Include3.dat 1.69

 

Other Crestron SW:

 

        Device Database: 31.05.005.00

        Crestron Database:  23.05.010.01

 

        Crestron Toolbox: 1.23.056.25

 

      Legacy Device Tools:  Not Available

 

 

Windows System Information:  

        Windows OS: Windows 7 Professional Edition

        Windows User Name: Karl

        Administrator Privileges: Yes

[Simon Cope]

Hey, we've done it!!!

I've updated my Simpl Windows to v2.12.44 (no idea why I never got round to updating it!), and everything is working as it should.

Phew...

[Jason Melvin]

Excellent! I would love to know _why_ that caused a problem, but at least it's a problem no more.