command complete '^' not received from motor controller

[VelmexControls]

     I have been working with velmex motor controllers to control 2 stepper motors in an experimental setup. I have tested two separate controllers here at Los Alamos:

     -model 2

     -dual model 1 (1-1?)

     A colleague, who is developing control software for our experiment is testing within the ORCA framework

     ORCA is running a simple script that sequentially sends the following commands to the motor controller, here is a copy of the input/output:

111912 14:11:42 write: E,X

111912 14:11:42 received: X+0000000

111912 14:11:42 write: E,Y

111912 14:11:42 received: Y+0000000

111912 14:11:42 write: F,C,S2M200,I2M-100,R

111912 14:11:42 received: CS2M200,I2M-100,

111912 14:11:42 received: R

111912 14:11:43 write: E,X

111912 14:11:43 received: +0000000

111912 14:11:43 write: E,Y

111912 14:11:43 received: -0000024

111912 14:11:43 received: ^

111912 14:11:43 write: E,X

111912 14:11:43 received: X+0000000

111912 14:11:43 write: E,Y

111912 14:11:43 received: Y-0000100

111912 14:11:44 write: F,C,S1M50,I1M20,R

111912 14:11:44 received: CS1M50,I1M20,R

111912 14:11:44 write: E,X

111912 14:11:44 received: +0000006

111912 14:11:44 write: E,Y

111912 14:11:44 received: -0000100

111912 14:11:44 write: F,C,S2M200,I2M-100,R

111912 14:11:44 received: ^

     As you may guess, ORCA relies on a receipt of the '^' character to continue with execution of the script. What I am seeing is that, at random, either the motor does not send the '^' command, or the software does not receive it, so that the script hangs:

111912 14:11:55 write: F,C,S2M200,I2M-100,R

111912 14:11:55 received: ^

111912 14:11:55 write: E,X

111912 14:11:55 received: X+0000140

111912 14:11:55 write: E,Y

111912 14:11:55 received: Y-0000100

….

111912 14:11:56 Script stopped

  The developer is not seeing this behavior, and suspects it is due to a firmware difference between our devices. 

   I should add that I am running on a macbook pro through a keyspan model usa49wg 4 port usb-serial adapter.

   Any expertise you could offer would be much appreciated.

[ny.rs232]

Just looking at the script logs

 

1.) you do not have to send the "E" all the time (unless you are just ensuring that it is online)

2.)  an F after an E has no effect (as you can see by the log where it shows)

111912 14:11:44 write: F,C,S1M50,I1M20,R

111912 14:11:44 received: CS1M50,I1M20,R

3.) It could cascade from there but from the looks of it (without seeing source code) I would hazzard to guess that the problem is in the code when it looks and when it clears what to look for

 

5.) I do not know orca, but all languages follow a pattern so it is worth looking into

 

Charlie

[hbnj...@gmail.com]

I encounter the exact problem. I use C# to send the commands. The fact that the issue is random and inconsistent makes me think there is something wrong with the Controller. If the issue was due to the code and command strings I would assume the issue would happen consistently and at the same line or time every run.
I know this was posted a year and a half ago but I would appreciate a reply if you have found the cause and the solution.
THX

[Velmex Suppt]

Q.) The fact that the issue is random and inconsistent makes me think there is something wrong with the Controller.
A.) Incorrect, the VXM always sends the caret ("^") when it has completed running its program. The idea that it
	only APPEARS to be random is due to either a timing issue or other software issue in your code. 
	The shorter your program execution time and the more rapidly your checks for the ^ the more likely that
	it will miss one either by a read that actually had 2 ^^ in the buffer, satisfying the 1st check, but 
	then the next check has none because it was already read. (or a check was made before, or at the moment
	that the ^ is arriving at the port)

Serial port timing is not a constant, it is an average.
If you can post compilable code then it can be shown where in the code that the "random loss of a ^" can be shown.
	

Best Regards,
Velmex Support
Velmex, Inc.

"Positioning Systems for Science and Industry"
Tel: 585-657-6151 / Toll Free: 800-642-6446 / Fax: 585-657-6153
www.velmex.com
www.velmexcontrols.com

On 8/20/2014 8:10 AM, Cliff wrote:

-------- Original Message --------

Subject:	Re: command complete '^' not received from motor controller
Date:	Tue, 19 Aug 2014 09:35:49 -0700 (PDT)
From:	hbnj...@gmail.com
Reply-To:	velmex-...@googlegroups.com
To:	velmex-...@googlegroups.com

I encounter the exact problem. I use C# to send the commands. The fact that the issue is random and inconsistent makes me think there is something wrong with the Controller. If the issue was due to the code and command strings I would assume the issue would happen consistently and at the same line or time every run.
I know this was posted a year and a half ago but I would appreciate a reply if you have found the cause and the solution. 
THX

-- 
You received this message because you are subscribed to the Google Groups "Velmex Controls" group.
To unsubscribe from this group and stop receiving emails from it, send an email to velmex-contro...@googlegroups.com.
To post to this group, send email to velmex-...@googlegroups.com.
Visit this group at http://groups.google.com/group/velmex-controls.
For more options, visit https://groups.google.com/d/optout.

[hbnj...@gmail.com]

Thanks for the reply.
Here is my move method:
______________________________________________________________________________
public int move(int motorNumber, char direction, int distance, bool pulse) // pulse Boolean value is true when distance is sent in pulse counts, if false the distance is sent in degrees.
{
try
{

Vxm.DriverTerminalShowState(1, 0);
if (Vxm.PortIsOpen() != 1)
Vxm.PortOpen(Settings.velmexComPort, Settings.velmexBaudRate);
int _distance = 0;
if (!pulse)
_distance = GetDistanceInDegrees(motorNumber, distance);
else _distance = distance;

Vxm.PortSendCommands("F,C,S");
Vxm.PortSendCommands(motorNumber.ToString());

Vxm.PortSendCommands("M");

if (motorNumber == 1)
Vxm.PortSendCommands(Settings.M1Speed.ToString());
else
Vxm.PortSendCommands(Settings.M2Speed.ToString());

Vxm.PortSendCommands(",I");

Vxm.PortSendCommands(motorNumber.ToString());

Vxm.PortSendCommands("M");

if (direction == '-')
Vxm.PortSendCommands(direction.ToString());

Vxm.PortSendCommands(_distance.ToString());

Vxm.PortSendCommands(",R");

Vxm.PortWaitForChar("^", 0);

Vxm.DriverTerminalShowState(0, 0);
if (motorNumber == 1)
Settings.M1Position = Convert.ToInt32(Vxm.MotorPosition(1)) / Settings.M1PulsePerDegree;
if (motorNumber == 2)
Settings.M2Position = Convert.ToInt32(Vxm.MotorPosition(2)) / Settings.M2PulsePerDegree;
Vxm.PortClose();
distance = updatePosition(motorNumber, direction, distance);
return 0;
}
catch (Exception except)
{
System.Windows.Forms.MessageBox.Show(except.Message, "Motor Control: Move()");
return 0;
}

}
_______________________________________________________________________________
After you memntioned there might be some timing issues I added several delays to give the controller enough time.
Here is the Code with delays:
_______________________________________________________________________________
public int move(int motorNumber, char direction, int distance, bool pulse) // pulse Boolean value is true when distance is sent in pulse counts, if false the distance is sent in degrees.
{
try
{

Thread.Sleep(50);
Vxm.DriverTerminalShowState(1, 0);
if (Vxm.PortIsOpen() != 1)
Vxm.PortOpen(Settings.velmexComPort, Settings.velmexBaudRate);
int _distance = 0;
if (!pulse)
_distance = GetDistanceInDegrees(motorNumber, distance);
else _distance = distance;

Vxm.PortSendCommands("F,C,S");

Vxm.PortSendCommands(motorNumber.ToString());
Thread.Sleep(200);
Vxm.PortSendCommands("M");

if (motorNumber == 1)
Vxm.PortSendCommands(Settings.M1Speed.ToString());
else
Vxm.PortSendCommands(Settings.M2Speed.ToString());

Vxm.PortSendCommands(",I");

Vxm.PortSendCommands(motorNumber.ToString());

Vxm.PortSendCommands("M");
Thread.Sleep(300);
if (direction == '-')
Vxm.PortSendCommands(direction.ToString());

Vxm.PortSendCommands(_distance.ToString());
Thread.Sleep(300);
Vxm.PortSendCommands(",R");
Thread.Sleep(900);
Vxm.PortWaitForChar("^", 0);
Thread.Sleep(300);
//Vxm.DriverTerminalShowState(1, 0);
if (motorNumber == 1)
Settings.M1Position = Convert.ToInt32(Vxm.MotorPosition(1)) / Settings.M1PulsePerDegree;
if (motorNumber == 2)
Settings.M2Position = Convert.ToInt32(Vxm.MotorPosition(2)) / Settings.M2PulsePerDegree;
Vxm.PortClose();
distance = updatePosition(motorNumber, direction, distance);
return 0;
}
catch (Exception except)
{
System.Windows.Forms.MessageBox.Show(except.Message, "Motor Control: Move()");
return 0;
}

}
___________________________________________________________________________
Here are other functions that are called by the above (move()) function:
___________________________________________________________________________
private static int GetDistanceInDegrees(int motorNumber, int distance)
{
int _distance = distance;
if (motorNumber == 1)
_distance = _distance * Settings.M1PulsePerDegree;
if (motorNumber == 2)
_distance = _distance * Settings.M2PulsePerDegree;
return _distance;
}
___________________________________________________________________________
private int updatePosition(int motorNumber, char direction, int distance)
{
if (direction == '-')
distance = -1 * distance;
if (motorNumber == 1)
M1Position += distance;
if (motorNumber == 2)
M2Position += distance;
return distance;
}
____________________________________________________________________________

[hbnj...@gmail.com]

The problem is still randomly shows up, even with the delays added.
Please advise how I can solve this issue.
Regards,
hbnjm

[VelmexControls]

What kind of distance moves are you using?
Also I see threaded sleeps (both could be a cause for something missed)

[rob....@thermofisher.com]

You need to solve it by properly asynchronously capturing the caret character.  Don't just splice in delays and "cross your fingers" that your debugger will get lucky.  Create a function which does nothing else but trigger on that one character.

Think about the ramifications of blindly using stacked delays: say that in 1 week / 1 month / 1 year's time, someone else changes your script and adds or subtracts instructions.  Now your timing delay is broken.

[hbnj...@gmail.com]

I'm using two rotary positioners. The move distance usually is no smaller than 1 degree. Sometimes I do use pulse counts for higher resolution and to align the apparatus that the positioners move to the home position.
The initial move() function did not have the thread sleep lines. I added them to see if this will improve the issue. It did not.
I use multiple threads however I have a class that manages the threads to make sure there aren't any cross threading. Also, The position controller has a dedicated class and i'ts being accessed through another class that manages all the commands that need to be sent to the VXM controller. Somehow the "^" is either not returned, or missed. I leave the com Terminal open so I can see the sent and received commands. The last thing I see is "R" and system waiting for the "^". Since I don't use a timeout delay the positioners just sit there and wait for the return.
If another thread is grabbing the "^" I should be able to see the commands that are sent by that thread to the controller in the terminal window as well, right? But I don't.
The question is what happens to the "^" and how I can assure that the move is completed?
Thanks,
hbnjm

[hbnj...@gmail.com]

I totally agree with you delays are not the way to go. I hardly use delays unless I am required. I was trying to see if I can see any improvement by slowing down the code and giving the VXM controller more time, which I did not.
You mentioned that I should have a designated function that triggers on the caret character. Would you elaborate on that a little more.
Thanks,
hbnjm

[VelmexControls]

I'm using two rotary positioners. The move distance usually is no smaller than 1 degree.

Q1.) 1 degree of what? the rotary table or the motor driving it?
Q2.) Which rotary table model? (the model will indicate how many steps per degree of the table)

Sometimes I do use pulse counts for higher resolution and to align the apparatus that the positioners move to the home position.

Q1.) Could you elaborate? (Personally I would just home to the home switch)

The initial move() function did not have the thread sleep lines. I added them to see if this will improve the issue. It did not.

A1.) No it wouldn't given the following:

I use multiple threads however I have a class that manages the threads to make sure there aren't any cross threading. Also, The position controller has a dedicated class and i'ts being accessed through another class that manages all the commands that need to be sent to the VXM controller. Somehow the "^" is either not returned, or missed. I leave the com Terminal open so I can see the sent and received commands. The last thing I see is "R" and system waiting for the "^". Since I don't use a timeout delay the positioners just sit there and wait for the return.
If another thread is grabbing the "^" I should be able to see the commands that are sent by that thread to the controller in the terminal window as well, right?

A1.) Incorrect. The VXM driver was never built to handle threading. Anything somehow caught in another thread would never be seen in the terminal window.
A2.) I would not have a clue what sorts of weird things could happen when other threads are doing things outside the scope of the intended driver purpose

The question is what happens to the "^" and how I can assure that the move is completed?

A1.) Do not use threads
A2.) Use threads and write routines for controlling the serial port instead of using the driver.
A3.) If you decide to go with A2 then I am sure we would all appreciate to see how threading can be used with a single object (AKA: Serial Port)

You mentioned that I should have a designated function that triggers on the caret character. Would you elaborate on that a little more.

I may be wrong, but I think what Rob meant was more like in the Windows API where you can wait on an event, check what that event was, and if it met your criteria then trigger an event of your own.
But I digress because that gets into much more difficult programming (at least for me)
I will let Rob answer for himself in the case I misunderstand as well

[hbnj...@gmail.com]

A1: Rotary table, 1 degree out of 360 degrees.
A2: Motor 1: B4800TS, 80 pulse/degree
Motor 2: B5990TS, 100 pulse/degree
A3: The home position needs to be at a specific location in respect to a measurement detector therefor after finding the reference sensor the rotary table needs to rotate to a specific position.
A4: I doubt that multiple threads send commands to the VXM controller simultaneously. However, I do not have a proof of that. Also, We have seen the same issue with this specific controller even with a single thread code. That being said is it possible that this specific controller would have a problem? I do not have another dual connection to swap them and see if the issue follows or not.
Could you tell me how does the controller know that the rotary table has completed the requested move and then sends the "^" character without an encoder (no closed loop)? is it just counting the steps based on the pulses it sends to the motors? Basically I want to know how does it know the motion is completed? Could it be that rotary table does not complete the motion, maybe even one less pulse therefore controller does not send the Caret character?
Thanks,
hbnjm

[rob....@thermofisher.com]

You are correct.  I developed our VXM application with a framework called LabWindows (made by National Instruments, makers of LabView).  It's a C-based visual programming platform, kinda like Visual C++ but with better, more direct hardware hooks.

Anyway, correct, I have a dedicated function (LabWindows refers to them as 'callbacks') which only triggers when a single caret character is received into the serial buffer.  When that event occurs, I then process it.  So to translate that concept into your language, you need to develop an equivalent interrupt-like function.

But I tend to agree that it is probably dangerous to use threads to interact with the VXM driver.  The problem with relying on threading is priority and OS lag.

[Velmex Suppt]

On what may be an unrelated issue (maybe not though) try changing

Vxm.PortWaitForChar("^", 0);
to
Vxm.PortWaitForChar("^", 5000);

and see if the program only locks up for 5 seconds? or if it continues to be locked up for 30 seconds or more?
That may give me an idea where to look if I can locate the original code.

Best Regards,
Velmex Support
Velmex, Inc.

"Positioning Systems for Science and Industry"
Tel: 585-657-6151 / Toll Free: 800-642-6446 / Fax: 585-657-6153
www.velmex.com
www.velmexcontrols.com