Vector Logging Converter WORK Download

[Septimio Sallah]

Further information:

• Manual for the CAN Logger Utility in PDF format (includes a step-by-step introduction, printable)
• Description of the CAN Snooper Mode (a simple CAN-bus monitor, implemented as an add-on in the logger firmware)
• Manual for the CAN Terminal programming tool (link only works if you installed the tool in the default directory).
• How to invoke the System Menu in your terminal (and how to enter the unlock-code if necessary)
• Homepage of MKT Systemtechnik (Contact us if you need an unlock-code to enable the logger, or to check if an updated firmware is available)
• Direct link to the download on the MKT website

Hint: If you use a browser like Firefox, and open the help system from theutility, this document will automtically be scrolled to the right position.The same was possible with IE6. But with IE7, this didn't work anymore. Nowit's your decision which browser to use...Note: If you received this document as a Word- or Adobe Actobat file (pdf)on a CD-ROM, you will possibly read an outdated 'copy' in this moment. Theguaranteed latest revision is only the documentation in HTML format, whichcan be found in the subdirectory \help in the CAN Logger Configuration Utility! To make sure you are studying the latest version of this document, downloadthe CAN Logger Utility from the MKT website, install it, and read the manual in HTML-form with your favourite browser. (Hint: With a good HTML browser like Firefox, these pages can alsobe printed nicely, with properly formatted page borders ) 1. IntroductionThe CAN-Logger/Snooper is an extra software feature in certain terminalsby MKT Systemtechnik or their resellers. A special firmware is needed touse the features described in this document, and the terminal must have aCF- or SD- memory card drive, because the storage medium for the logged datais a Flash memory card with DOS-compatible file system (FAT16, for MKT-View III/IV also FAT32).At the time of this writing, the CAN-Logger/Snooper was supported by thefollowing devices (there may be more than listed below) :

• MKT-VIEW "plus" with 40 MHz clock, CompactFlash card interface (CF), and a special firmware (art11089.hex) .
  Note: In this firmware, the CAN Snooper and CAN Logger functions must be enabled by entering a special unlock code. If you want to use one of these functions but haven't unlocked it yet, please contact the manufacturer. Only if you ordered the terminal with this functions already unlocked, you don't need to go through the unlock procedure.
• MKT-VIEW "II" with ARM-CPU, 72 MHz clock, 4.32 TFT, SD card, firmware #11314.
• MKT-VIEW "III" with ARM-Cortex M3, 96 MHz clock, 4.3" TFT, SD card, firmware #11392.
• MKT-VIEW "IV" with ARM-Cortex M4, 200 MHz clock, 7.0" TFT, SD card, firmware #11532.
  In all devices listed above, the CAN logger feature must be unlocked before use.
  Devices not listed here can be found in MKT's Device Feature Matrix (external link).

Some definitions of terms which are frequently used throughout this document,to make sure we know what we are talking about.. The CAN-Logger is a piece of hard- and firmware inside the programmable terminal. It writes the received CAN traffic to a disk file, message-by-message. To reduce the file size, the CAN identifiers of the logged messages can be selected (from a CANdb file or "manually").

• normal "terminal" function remains active in CAN Logger mode
  (with reduced performance = lower page update rate if bus load is high)
• high-resolution timestamp for received messages
• no display from the CAN logger, it works silently in the background
• logfile must be analyzed on a windows PC later
• limited message rate due to memory card hardware restrictions (see chapter"Limitations")

The CAN-Snooper is a subroutine in the CAN-logger-firmware. When activated, it works like a simple CAN analyzer utility to monitor "unknown" CAN traffic. Received CAN messages are displayed on the terminal's screen. You don't need a PC to use the CAN-Snooper. The CAN-Snooper is a part of the firmware in all display terminals which also have the CAN-Logger functionality. There is no CAN-Snooper without CAN-Logger, and there is no CAN-Logger without CAN-Snooper - except for the possibility that one of these functions is unlocked while the other is not ! A more detailed description of the CAN-Snooper-mode can be found in the description of the programmable terminal, but not in this document (this link only works if you installed the tool in the default directory). This document only describes the CAN-Logger.


The CAN-Logger-Configuration / File-Conversion Tool (abbrev'd "CAN logger utility") is a PC program which helps you to configure the CAN logger, and to convert the logged data into a format which can be loaded in your analysis program. The CAN logger utility is *not* a CAN-Logger itself, but it...

• can read CANdb-files so you can select thesignals which you want to log,
• lets you modify the logger settings, liketrigger mode, FIFO settings, file size limits, recording interval and more,
• converts the logfile which is producedby the CAN logger into some format which can be post-processed with otherprograms, like spreadsheets, data visualization tools, statistic evaluationprograms, etc etc.

In principle you can use the CAN logger without the CAN logger utility.You can instead write the CAN logger configuration file yourself (with atext editor) and write your own import routine for the logfiles (with aprogramming language of your choice). If you really consider this, look intothe "file formats" section in the appendix ofthis document.The next chapters provide detailed information about the operation of theCAN-Logger Configuration and File Conversion Tool. More information aboutDisplay Terminals with CAN-Logger can be found in the manual of theterminals itself or in the help system of the terminal programming tool.The description of the programmable terminal is beyond the scope ofthis document ! The basic usage of the CANlogger is briefly explained in a later chapter.If you have installed the terminal programming tool in its defaultdirectory, you can follow this link to thedescription of some interpreter commands, which give you "extended" controlover the CAN logger. For normal applications, you don't need to control thelogger via interpreter commands. The only exceptions are the commandspower=1 (to inform the terminal that the powersupply is safe now, so files may be written) andpower=0 (to indicate that the power supply isnot safe anymore, and that ALL files with write-access must be closedIMMEDIATELY). More information is in the help system of the terminal programmingtool.back to top 2. Usage of the CAN Logger Configuration UtilityThis chapter explains how to use the CAN Logger Configuration Utility. Producinga logger configuration file is usually the first step. Theusage of the CAN logger itself is explainedin a later chapter, also how to convertlogfiles will be described later.2.1 Starting the CAN Logger UtilityThe simplest but most ineffective way to start the CAN Logger Configuration/ File Conversion Utility is to double-click its icon on the desktop, orin the task bar. It is easier (for every-day use) to associate the fileextensions "CLC" (CAN Logger Configuration) and "CLD" (CAN Logger Data) withthe utility. A detailed how-to can be found in theappendix.If you double-click on a CAN logger configuration file to start the utility,it automatically switches to the Logger Configuration tab.If you double-click on a CAN logfile (*.cld), the utility automatically switchesto the File Converter tab.To convert logfiles automatically, the program can be invoked with acommand line (use the \bm switch to enter"batch mode").
2.1.1 Selecting the target hardware
When launching the utility for the first time (after installation), the programwill ask for the target hardware. This is necessary (since February 2008),because some features and array sizes depend on the target. For example,the "MKT-View I" (terminal with C167 CPU) has less trigger signals than alogger built inside the newer "MKT-View II" (terminals with ARM-7 CPU) .The following dialog allows you to select the target hardware (its appearancemay have changed since the time this document was written) :This screen will also appear when you create a new logger configuration,with the option to restore everything to the default values.The properties of the used hardware can be checked in the tabsheet "Setup",on the "Options" panel. Some of the hardware/firmware properties of the selecteddevice are displayed on this panel, for example:When changing the logger device family, beware that they are not 100% compatible.For example, the CAN logger inside the "MKT-View I" can filter out a maximumof 48 different CAN messages from the stream, while the "MKT-View II" supportsa maximum of 128 (or even more). So when switching the device family from"MKT-View II" to "MKT-View I", some message- and/or signal definitions mayget lost.2.2 The CANdb Import / Signal List TabBecause the CAN logger was primarily intended to logCAN-SIGNALS(*), you must selectthese signals on a special tab of the utility:(screenshot CANdb tab)On the CANdb tab, you can

• Select messages or signals from CANdb files (which can be loaded via the File menu)
• Add or remove signals from the list of "logged" signals
• Rearrange the sequenze of logged signals (the buttons with blue arrows mean "move up" / "move down")
• See some information about a logged signal, including the CAN-ID and some mapping parameters

To import ("load") a CANdb file, use the file menu. If you have used a certainCANdb file earlier, you will find it in the "Most recent CANdb file" list- which is quite handy because this way you don't have to remember the fullpath name, and it may save a dozen mouseclicks in the file selector box.To select signals from a loaded CANdb file, you can use..

• a tree view (as shown in the screenshot)
• a table view (which contains ALL information about the loaded signals)
• a group of three lists (NODES, MESSAGES, SIGNALS)

To copy a signal from the treeview, you can drag it with the mouse and drop it into the list oflogged signals. The same is possible with entire messages (containing multiplesignals), nodes (containing multiple messages), or CAN buses (containingmultiple nodes). You may get a warning if duplicate signal names are alreadypresent in the list of logged signals (in the right part of the window).Avoid duplicate signal names in your CAN databases whereever possible.Alternatively, you can select an item in the table view and thenclick the "Add signal(s) to list" button.The list of logged signals will be saved as part of the logger configurationfile. The CANdb file will never be modified because the CAN logger configurationutility is not a CANdb-editor !You can use up to four of the logged signals astrigger signals. (*)footnote: Apart from selecting logged signals from a CANdb file, it is possible to..

• define own messages for logging
• log "other data" which are not CAN-messages

2.2.1 Searching for signal namesTo simplify the selection of signals from large databases, there is asearch function for signal names. You can enter a search patternin the edit field close to the "Search"-button, and press ENTER after that(or click the search button). The search pattern may contain the followingwildcard characters: ? Placeholder for "any single character (letter or digit) * Placeholder for a whole group of characters. Must not be used at the 1st position in the string !Search pattern examples:

• Thermo searches all signals named "Thermo", in all messages, on all CAN buses, sent by any node in the network.
• A* searches all signals beginning with 'A'
• ?bc* searches all signals with 2nd letter = 'b', 3rd letter = 'c', and an unknown number of letters afterwards
  

2.2.2 The "Signal Info" displayThe properties of any logged signal can be displayed in the "Signal Info"box. It is in the lower right corner of the signal definition tab. The contentsof this editor-like window may look like this:

Display (example) Explanation PrintedChars Signal name id=0x0000007B CAN id (+ explanation of the most significant bits 31..29 if set) node="Terminal" name of the sending node msg="MuxedMsg" name of message containing this signal mapping=(8,8,I,U) mapping info: position of LSBit, number of data bits,
byte order: Intel/Motorola,
data type: Signed/Unsigned/Float/Double MuxVal=#0, MuxMap=(0,8,I) Only if multiplexed : multiplexor value, multiplexor mapping (see above) To see the properties of the logged signal, select it in the list of loggedsignals.
Additionally, you can check the CAN message layout in graphical form as explainedin the next chapter .2.2.3 CAN-Message-Layout displayThe layout of a logged CAN message can be displayed in graphical form. Thismay be handy when problems with the interpretation of different bit-countingschemes occurr (for example, "Intel-format", "Motorola forward", "Motorolabackwards" and other oddities).The graphic shown on the left shows an example of a CAN-message which carriesvarious signals. The currently selected signal is marked with a bold arrow,and its name is displayed at the bottom of the window. The arrow always pointsfrom the least significant bit (LSB at the end) to the most significant bit(MSB at the arrow head).The bit numbers within a data byte are listed at the top of the display.Please note that -as usually for binary numbers- the most significant databitis on the left side; the numbers for 8 bits within a byte always runs from0 (=LSB, right) to 7 (=MSB, left).Within the 64 bit of a CAN data field, the bit count runs as shown in thediagram: The first byte [0] containts bits 7 to 0; the last byte [7] containsbits 63 to 56. It doesn't matter if the CAN message contains signals in Motorola-oder/and Intel-Format ! (Caution, some CANdb-editors count bits in a differentfashion, but of course this must not change the physical position of a bitwithin the CAN frame).If the selected signal is multiplexed (as shown here in the example), notonly the signal's data bits, but also the multiplexer forthis signal will be shown (using the same colour, but hatched). The valuesof the multiplexer-bits will also be displayed (since the multiplexer valueis a constant which identifies this signal). Note: The multiplexer/multiplexor is also called "mode signal" in other documents, but this name is rather meaningless so we avoid it. To select other signals in the displayed message, click into the diagramwith the left mouse button.
Contents 2.3 The Logged Messages TabThis tab of the Logger Configuration Tool contains a list of the CAN messageswhich shall be logged (and some other specialities too).The lower part of this tab contains a table for "other" logged values, whichwill be explained in one of the next chapters.But first look at the upper table. It contains a list of allmessages which shall be logged.There is one row in this table for every CAN-message. Thecolumns in this table are:

• Nr
  This index references the message. It runs from zero to the maximum number of logged messages.
• Bus
  1 = "Message shall be received from Bus 1 (the first CAN interface)"
  2 = "Message shall be received from Bus 2 (the second CAN interface, if available)"
• CAN-ID
  CAN message identifier in hexadecimal notation. "0x" is the prefix for a hex number. 11-bit ID's contain three hex digits, 29-bit-ID's eight digits.
• Message Name
  Usually copied from the CANdb-file, in the moment when you added CAN signal(s) as explained in the previous chapter. For self-defined CAN messages, you may type short descriptive names, without space characters and other special characters (only letters and underscores).
• Sender (Node)
  Identifies the sender (the sending "node") of this message. Remember, one node can send more than one message. Like the message name, this name originates from the CANdb-file.
• Intv.-Limit (write-interval limit)
  In this column you can define a message-specific limitation for the write interval in the logger. For example, a message may be transmitted on the CAN bus every millisecond, but you only want the logger to write this message to the memory card every 200 milliseconds. In that case, type "200" into the Interval-Limit column for this particular message. Zero means "there is no individual writing interval limit for this message".
  Note: There is also a common write-interval limit for all messages (and for logging 'other' data) on the logger configuration tab.

Usually this table will be filled automatically when youselect signals for logging. To add entriesmanually (because you have no CANdb file to select signals from), lookhere.
Contents 2.4 The Logger Configuration TabSwitch to the "Logger Configuration" tab of the utility. You will see thecurrent settings in several edit fields, check boxes, combo lists etc. Modifythem for your needs and save the modified configuration (the tool will askyou upon exit if you want to save the changes, if any).
(screenshot "logger configuration" tab)The controls on this tab are:

• Info string
  A string of up to 40 characters which can be displayed on the terminal screen, using the interpreter function "logger.info" .

The upper part of the panel titled "What shall be logged ?" contains three radio buttons (only ONE option canbe selected at a time) :

• Only the selected messages/signals
  The logger only writes the signals into the logfile, which you selected on the CANdb (or "No CANdb") tab.
• All signals received by the application
  The logger only writes those signals into the logfile, which are received by the terminal application. The advantage is, you don't have to produce an extra configuration for every new application of the terminal. Instead, you log all messages which are handled (received) by the terminal.
• Receive and log ALL CAN messages
  Useful to analyze the complete CAN bus traffic. This only makes sense if you export a message list later, not a signal list !
  Caution #1 : This option cannot be used together with the option "Limit write interval for messages" ! Explanation: see appendix.
  Caution #2 : As a matter of course, CAN logfiles can only be converted (decoded) into physical signals, if the required signal decoding- and scaling information is available to the logfile converter ! Setting the option "Receive and log ALL CAN messages" alone (without adding the required signal definitions from the CAN database) is not sufficient to export decoded signals (not messages) later. Also see the notes on the logfile converter, option "Ignore Signal Definitions from Logfile, use currently loaded signal definitions instead" .

The lower part of the "What shall be logged ?" panel allows to select some other data(which are not related with normal CAN messages):

• script variables declared with the 'logged:' attribute (details in chapter 'Logging other data (no CAN messages)'.)
• Data from an external GPS receiver
• CAN error frames (rarely used, not supported by older devices like MKT-View "Plus")

Group "Options":

• Use logfile as circular FIFO, overwrite old data
  If this option is set, the opened logfile will work like a FIFO buffer, which wraps to the start of the file when the maximum file- or disk-space is exceeded. In other words, if the logfile is "full", the writing-position will jump back to the begin of the file, and the oldest entries are overwritten. When the logfile is closed, only the "newest" data are in it - how many, depends on the maximum file- or disk size.
  If this option is not set, old data in the logfile are not overwritten. Instead, when the file size is reached, a NEW logfile will be created as explained below (under "maximum size of a single logfile").
  Note: This has not got anything to do with the pretrigger . The pretrigger memory is always a RAM buffer, to avoid unnecessary wear of the FLASH memory card, as long as no "interesting" data are received !
• Use a new logfile on startup
  If this option is set, the logger will use a new filename (with incremented index) on system start. In other words, a new file for each time you turn the power on.
  Otherwise the logger firmware will try to re-open an already existing file on system start, and append new data to that file. This option is usually combined with the option "use file as FIFO". Please note that high-resolution time channels in the logfile may be not monotonously rising without this option, as explained here .
• Use a new filename for each trigger event
  Similar as above. If this option is selected, the logger will use a new filename whenever the trigger switches from "OFF" to "ON".
  Without this option, multiple trigger cycles (trigger ON -> trigger OFF -> trigger ON) will be recorded in a single file.
• Keep logfile open
  With this option, a logfile will be kept open even if the trigger stops a recording. This is a bit faster, especially when combined with the pretrigger option.
  Without this option, the logfile will be closed immediately if the trigger stops a recording, and re-opened when restarted. Use this option if the logger will be mostly "not triggered", because it reduces the risk of data loss if the power breaks down by accident (see notes on the power supply ).
• Use password and encryption
  You may encrypt the logger configuration file and the header section of the logfile to protect the CAN configuration data. Some user's were not happy about the ASCII files. If a password is used, the letters and digits in these files will be scrambled.

Other options..

• Maximum size of a single logfile
  A value of zero means "unlimited file size" (only limited by the capacity of the memory card). Non-zero means the maximum size of a single logfile shall be limited - usually to a few Megabytes. If the filesize is reached, the behaviour of the logger depends on the option "Use logfile as circular FIFO" (explained above). Without this option, the current logfile is closed and a new file with a different name (increased sequence number in the last two letters of the filename, for example canlog00.cld, canlog01.cld, canlog02.cld). This is helpful when you only need to copy a part of the logged data to your PC. A 500-MByte-monster takes a some time for copying, even with a fast memory card reader on the USB port - and even longer if you want to send it to someone via email for examination.
  
  
• Limit write interval for messages (and other data)
  If you already know that you don't need millisecond-resolution for the logged data before starting a logging session, limit the write interval for messages here (in the logger configuration). This will reduce the size on the flash memory card, and reduce the CPU load of the CAN logger itself, and will make the conversion of the logfiles into some other formats faster. See also: write interval of the CAN logfile converter.
  The same parameter is also used as the recording interval (inverse of the sampling rate) for 'other data', for example calculated numeric expressions or logged script variables.
  Notes:
  • This option has no effect on CAN messages when used together with the option "Receive and log ALL CAN message identifiers" ! Reason / explanation: see appendix.
    
  • There is also an individual write-interval-limiter for certain CAN messages on the "Messages" tab.

Note: The trigger conditions andtrigger options are configured on othertabsheets ! 2.5 Trigger2.5.1 The Trigger State MachineThe CAN logger is controlled by a state machine which can be checked withthe function logger.state (if the CAN logger is part of aprogrammable terminal):The logger statesare:Stopped(or "off")

• The logger (!) is passive and does not consume significant CPU power
• Messages are not placed in the log buffer
• The logfiles on the flash memory card are closed
• The trigger condition are not monitored (because messages are not handled in this state). This saves a large amount of CPU power !

Engaged

• The logger function is active and analyses all received CAN messages. The CPU load is already higher than in the stopped state.
• Logfiles on the flash memory card are already opened, and -if the file is used as a giant FIFO- new entries may already be written to the file.
• The trigger conditions are permanently monitored to detect a possible "start" condition. If a START condition is detected, the logger switches from enaged to triggered.

Triggered

• The logger function is active and analyses all received CAN messages. The CPU load is even higher than in the engaged state.
• Received messages are written from a RAM buffer to the memory card.
• The trigger conditions are still monitored to detect a possible "stop" condition. If a STOP condition is detected while the START condition if false, the logger switches from triggered to engaged state (but not to stopped !).
• Note: the post-trigger phase is almost the same as the 'triggered'. The only difference is the value returned by the logger.state function of the interpreter.

The initial logger state after power-on can is defined on theTrigger Option tab ("triggered by default"etc).The transitions between the logger states are either initiatedby the logger itself, depending on the trigger conditions, or with an interpretercommand. A complete list of logger control commands is in themanual of the terminal programmingtool. Here just a few of those commands :
logger.run Activates the CAN logger, but does not necessarily trigger it (depending on the configuration). Switches from stopped to engaged, but not from triggered to engaged. If the firmware refuses to switch into the engaged state, there may be something wrong with the memory card.
logger.stop Stops the CAN logger. The logfile on the memory card is closed, so you can safely remove the card. No popup window appears (in contrast to the 'card.remove' command. In the stopped state, the trigger condition is not permanently checked in every received CAN message, which saves a large amount of CPU power !
logger.trigger(X) Triggers a recording (X=1) or clears the trigger flag (X=0). Does not close the logfile. Intention: Manual trigger or trigger by the user-programmable terminal application. Examples:
logger.trigger(1) : rem Start recording to the CF card
logger.trigger=1 : rem Same effect as above, only different syntax
logger.trigger=0 : rem Stop recording, but keep buffering received frames
If the firmware refuses to write data to the memory card, the reason may be power=0 (power-good flag not set).
Note about "logger.run" vs "logger.trigger":
Setting the trigger (=begin writing to the logfile) with this command automatically includes activating the logger (logger.run). But, if the logger was not running when executing the trigger-command, there will be nothing in the pretrigger-buffer.
So, if you need pretrigger-data, activate the logger before triggering it !
logger.state Returns the current state of the CAN logger as a numeric value. Possible results :
0 = "stopped" (file not open, or buffer not filled in interrupt, possibly a problem with the memory card)
1 = "engaged" (file open, ISR active, but not triggered yet)
2 = "triggered" (file open, ISR active, AND "triggered" )
3 = "post-trigger" (after "triggered"). Only occurrs if the post-trigger time in the logger configuration is non-zero.

Notes:

• You don't have to use these interpreter commands to controll the logger's trigger ! Usually, the logger works silently in the background, and you don't have to care for it when programming your terminal application. The logger control commands have only been implemented for 'very special cases', when the built-in trigger possibilities are not sufficient.
• There are some other interpreter commands to control the logger, and to check the memory card, explained in the manual of the terminal programming tool. Such commands include: card.insert, card.remove, card.dspace (retrieve free disk space in kByte)

2.5.2 Pre- and PosttriggerPretriggerYou may be interested in the history of the logged signals a short while*before* the trigger event. This is possible, because the logger recordsthe most recent message in a RAM buffer even if the logger is not in the'triggered' state. By setting the pretrigger-field to a value greater thanzero (seconds), a part of the messages in the RAM buffer will be flushedto the logfile when the trigger start condition becomes TRUE.
Note: Depending on the size of the RAM buffer, and the average message rateon the CAN bus, the logger firmware may not be able to save as much 'secondsof pretrigger data' as you want. The buffer capacity which is available forpretriggering is roughly 3000 messages.
The pretrigger interval is defined on the'trigger options' tab.PosttriggerTo record a few seconds *after* the trigger condition became FALSE (and/orthe stop condition became TRUE), set the posttrigger-field to a value greaterthan zero (0 means "no post-trigger"). Usually, recording to the logfilestops 0...10 milliseconds after the trigger is stopped. With the posttriggeroption, you can record some additional seconds. The posttrigger's internalfunction is very different from the pretrigger, there is no limitation dueto the RAM buffer size for the postt