Methods Of Testing And Debugging Hardware And Software Plc
Penélope Lichlyter
In software development, the debugging process begins when a developer locates a code error in a computer program and is able to reproduce it. Debugging is part of the software testing process and is an integral part of the entire software development lifecycle.
In hardware development, the debugging process looks for hardware components that are not installed or configured correctly. For example, an engineer might run a JTAG connection test to debug connections on an integrated circuit.
Debugging and testing are complementary processes. The purpose of testing is to identify what happens when there is a mistake in a program's source code. The purpose of debugging is to locate and fix the mistake.
The testing process does not help the developer figure out what the coding mistake is -- it simply reveals what effects the coding error has on the program. Once the mistake has been error identified, debugging helps the developer determine the cause of the error so it can be fixed.
Embedded systems design combines the highly technical disciplines of hardware design and firmware and application software development. Embedded systems engineers face significant challenges throughout the design process, especially when it comes to the integration and debugging of hardware and software systems.
To help bring relief to developers in debug purgatory, we're offering up some of our preferred debugging techniques that actually work. We'll review some of the traditional run-time debugging techniques, describe the benefits of integration testing, and explain how developers can implement real-time trace debugging to discover and rectify software bugs, bringing products to market faster and with fewer errors.
Our exploration of debugging techniques for embedded systems begins with traditional debugging techniques. These techniques can all be used to identify and isolate coding errors for removal from your firmware or software application, but the efficacy of each method depends on the unique circumstances of your project. Some of these methods include automation while several others are heavily manual processes. More complex embedded systems projects will typically benefit from more sophisticated debugging methodologies.
The print debugging method is really just the simplest version of a run-time debugging technique. Run-time debugging methods share a common characteristic: they monitor the live execution of a process or a piece of code while the developer uses either manual techniques or debugging software to debug the process. In addition to the print method, run-time debugging methods include remote debugging and communication-based debugging.
Without the right tools, debugging can prove an expensive and time-consuming process. Techniques like integration testing and real-time trace can help embedded systems engineers troubleshoot code errors more quickly than traditional, manual debugging methods.
Integration testing, sometimes referred to as System Integration Testing (SIT), describes a fundamentally different approach to testing and debugging embedded systems throughout the development process. This method borrows wisdom from new working methods of software development such as Agile and DevOps that promote periodic testing throughout the development process instead of a single, lengthy debugging process in the late stages of product development.
The final debugging technique that we want to mention here, and the most powerful, is known as a real-time trace debugging. With real-time trace, developers implement a hardware device such as a protocol analyzer that records information about the execution of processes in the code. This information can include sequences of program counter values, register reads and writes, and changes in device memory and associated data values.
Real-time trace debugging can be implemented at any stage of development, as a complement to integration testing or when the software and hardware for the embedded system has been fully integrated. Trace information can also be captured non-intrusively, meaning that the system's timing and performance will not be affected by the trace. Real-time trace also enables developers to more easily capture, record and collect test results for future analysis of bugs.
Tests regarding functional safety play an important role in the development of vehicle ECUs. Tests at various stages help the developer to ensure conformity with ASIL standards. What role does hardware debugging play here? An overview.
The assumption here is that separation between partitions is not achieved by hardware methods, i.e., methods such as a memory protection unit (MPU) or a hypervisor. Either such methods are not planned or available, or they are not yet enabled. In the latter case, the tool helps troubleshoot or prepare the software for an MPU. Instead of debugging one MPU exception after another once the MPU is enabled, the software can be prepared in advance. And that without running the software on any real hardware.
After unit testing, we move on to system-level testing, for example within a hardware-in-the- loop configuration. In this case, a debugger can be very useful to inject faults into the system to test the impact on the FFI in terms of memory corruption, execution, and information sharing. The debugger is used to perform on-the-fly manipulations on on-chip resources such as CPU core registers and memory and then examine the effects (Figure 4). But faults can also be injected at external interfaces: Add-on modules for CAN/LIN and analog/digital signals connected directly to the debugger hardware and target system can be used to inject faulty data into an ADC or from external sensors connected via CAN or SPI.
But on-chip traces not only enable the behavior of the software to be monitored. Since hardware traces are non-intrusive, i.e., have no influence on the runtime, they are also ideal for timing analysis. Timing tests should be performed as part of system integration testing. The increasing load on the operating system tasks naturally affects the entire timing schedule, and later critical timing requirements can no longer be met.
Design for testing or design for testability (DFT) consists of IC design techniques that add testability features to a hardware product design. The added features make it easier to develop and apply manufacturing tests to the designed hardware. The purpose of manufacturing tests is to validate that the product hardware contains no manufacturing defects that could adversely affect the product's correct functioning.
The third state-of-the-art approach is to conduct a formal power analysis of the compiled software for the microcontroller. For example, in [9] the power consumption for each compiler instruction has been determined for the overall power consumption. Said approach also considers the effects of caches, cache misses, or stalls. In [10], a formal analysis for an 8-bit microcontroller has been proposed which also considers the power consumption of peripherals and not only the microcontroller itself. Both methods can reveal particularly power intensive software parts and estimate the rough overall power consumption. However, the analysis is not data dependent making these approaches unsuitable for systems where the state depends on input data. A formal approach including data dependency has been presented in [11] which estimates the worst case power consumption at instruction level. However, often the average power consumption is more relevant than the worst case power consumption. Furthermore, the authors in [11] stated, that a formal analysis of whole complex programs can be very time-consuming. None of the above-mentioned formal methods is able to also analyze the power consumption of connected hardware such as inertial sensors.
Perhaps you might remember the module of interest on theFPGA: it was thesame debugging buscore we built hereon the blog before I discovered formalmethods. Sinceit had never beenformally verified, Isuspected a fault within it.
The best way to decrease the time spent debugging and keeping a project on track is to use a combination of hardware debugging tools and freely available software to gain insights into how a system is performing and where the issues are occurring. As such, for both professionals and hobbyists alike, having the right tools for the job dramatically helps in getting it done quickly and correctly.
Next up is Ozone. Ozone is a debugger interface and performance analyzer. Developers can load their elf file into the tool and perform source-level debugging. They can set breakpoints and update their code. An especially useful feature for developers is that they can also perform instruction tracing (if their hardware supports it) and identify what assembly and C code statements have been executed. This is particularly useful for verifying code coverage of hardware-in-loop (HiL) testing.
A key part of having a robust testing process is the devices on which the test is being run. A thorough testing procedure can also generate a variance in its results when performed on different devices. Thus, the significance of the testing device is paramount. The significance is even greater for testing on mobile devices, given the variety of devices, with different operating systems, browsers, and hardware configurations.
The prime reason for this kind of testing is to ensure consistent behavior of the website and app on different mobile devices. Since these devices differ by hardware configuration, operating system, and screen resolution, what might work in a certain way on one of the devices could vary for the other.
An Emulator is a software that mimics the hardware and software of the target device on your computer. They do this by translating the ISA (Instruction Set Architecture) of the target device to the one used by the computer you are using to conduct testing using binary translation.
Once you flash firmware onto the board or boot from an image, you might find other problems that were not evident in other sets of tests. Even if the code appeared to be written perfectly, functional errors could indicate a hardware design problem or a logical problem. If you're doing anything with embedded, I would start by flashing the board after doing an initial power-on test and check that the main functional blocks receive power in standby mode. Then I would flash the code and start doing functional testing and debugging. Throughout the process, if you see functional errors during testing and you've eliminated any hardware problems, it's time to go back to your code and verify it was written correctly.
dd2b598166