Library Trace

[Holly Coffell]

Tracing is a technique used to understand what goes on in a running softwaresystem. The software used for tracing is called a tracer, which isconceptually similar to a tape recorder.When recording, specific instrumentation points placed in the software sourcecode generate events that are saved on a giant tape: a trace file.The trace file then later can be opened in trace viewers to visualize andanalyze the trace events with timestamps and multi-core views.Such a mechanism will be useful for resolving a wide range of problems such asmulti-core synchronization issues, latency measurements, finding out thepost analysis information like CPU idle time, etc that would otherwise beextremely challenging to get.

Tracing is often compared to logging. However, tracers and loggers are twodifferent tools, serving two different purposes.Tracers are designed to record much lower-level events that occur much morefrequently than log messages, often in the range of thousands per second, withvery little execution overhead.Logging is more appropriate for a very high-level analysis of less frequentevents: user accesses, exceptional conditions (errors and warnings, forexample), database transactions, instant messaging communications, and such.Simply put, logging is one of the many use cases that can be satisfied withtracing.

The above macro creates app_trace_string tracepoint.The user can choose any name for the tracepoint.However, when adding a tracepoint in the DPDK library, therte__trace_[_] naming convention must befollowed.The examples are rte_eal_trace_generic_str, rte_mempool_trace_create.

The above code snippet registers the app_trace_string tracepoint totrace library. Here, the my_tracepoint.h is the header filethat the user created in the first step Create the tracepoint header file.

The second argument for the RTE_TRACE_POINT_REGISTER is the name for thetracepoint. This string will be used for tracepoint lookup or regularexpression and/or glob based tracepoint operations.There is no requirement for the tracepoint function and its name to be similar.However, it is recommended to have a similar name for a better namingconvention.

In order to avoid performance impact in fast path code, the library introducedRTE_TRACE_POINT_FP. When adding the tracepoint in fast path code,the user must use RTE_TRACE_POINT_FP instead of RTE_TRACE_POINT.

Tracecompass is another tool to view/analyze the DPDK traces which givesa graphical view of events. Like babeltrace, tracecompass also providesan interface to search for a particular event.To use tracecompass, following are the minimum required steps:

For lcore threads, if trace points are enabled through a EAL option, the tracememory is allocated when the threads are known of DPDK(rte_eal_init for EAL lcores, rte_thread_register for non-EAL lcores).Otherwise, when trace points are enabled later in the life of the application,the behavior is the same as non lcore threads and the trace memory is allocatedon the first trace emission.

When trace data isn't automatically captured, you need to instrumentyour application to collect this data.You can instrument your application so that it collectsapplication-specific information. Several open-sourceinstrumentation frameworks let you collect metrics, logs, andtraces from your application and send that data to any vendor,including Google Cloud. To instrument your application, we recommend that you use avendor-neutral instrumentation framework that is open source, such asOpenTelemetry, instead ofvendor- and product-specific APIs or client libraries.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

The trace module allows you to trace program execution, generateannotated statement coverage listings, print caller/callee relationships andlist functions executed during a program run. It can be used in another programor from the command line.

At least one of the following options must be specified when invokingtrace. The --listfuncs option is mutually exclusive withthe --trace and --count options. When--listfuncs is provided, neither --count nor--trace are accepted, and vice versa.

Create an object to trace execution of a single statement or expression. Allparameters are optional. count enables counting of line numbers. traceenables line execution tracing. countfuncs enables listing of thefunctions called during the run. countcallers enables call relationshiptracking. ignoremods is a list of modules or packages to ignore.ignoredirs is a list of directories whose modules or packages should beignored. infile is the name of the file from which to read stored countinformation. outfile is the name of the file in which to write updatedcount information. timing enables a timestamp relative to when tracing wasstarted to be displayed.

Execute the command and gather statistics from the execution with thecurrent tracing parameters, in the defined global and localenvironments. If not defined, globals and locals default to emptydictionaries.

Write coverage results. Set show_missing to show lines that had nohits. Set summary to include in the output the coverage summary permodule. coverdir specifies the directory into which the coverageresult files will be output. If None, the results for each sourcefile are placed in its directory.

To trace most memory blocks allocated by Python, the module should be startedas early as possible by setting the PYTHONTRACEMALLOC environmentvariable to 1, or by using -X tracemalloc command lineoption. The tracemalloc.start() function can be called at runtime tostart tracing Python memory allocations.

By default, a trace of an allocated memory block only stores the most recentframe (1 frame). To store 25 frames at startup: set thePYTHONTRACEMALLOC environment variable to 25, or use the-X tracemalloc=25 command line option.

We can see that Python has loaded 8173 KiB of module data (bytecode andconstants), and that this is 4428 KiB more than had been loaded before thetests, when the previous snapshot was taken. Similarly, the linecachemodule has cached 940 KiB of Python source code to format tracebacks, allof it since the previous snapshot.

We can see that the most memory was allocated in the importlib module toload data (bytecode and constants) from modules: 870.1 KiB. The traceback iswhere the importlib loaded data most recently: on the import pdbline of the doctest module. The traceback may change if a new module isloaded.

The following code computes two sums like 0 + 1 + 2 + ... inefficiently, bycreating a list of those numbers. This list consumes a lot of memorytemporarily. We can use get_traced_memory() and reset_peak() toobserve the small memory usage after the sum is computed as well as the peakmemory usage during the computations:

Using reset_peak() ensured we could accurately record the peak during thecomputation of small_sum, even though it is much smaller than the overallpeak size of memory blocks since the start() call. Without the call toreset_peak(), second_peak would still be the peak from thecomputation large_sum (that is, equal to first_peak). In this case,both peaks are much higher than the final memory usage, and which suggests wecould optimise (by removing the unnecessary call to list, and writingsum(range(...))).

Get the traceback where the Python object obj was allocated.Return a Traceback instance, or None if the tracemallocmodule is not tracing memory allocations or did not trace the allocation ofthe object.

This function only modifies the recorded peak size, and does not modify orclear any traces, unlike clear_traces(). Snapshots taken withtake_snapshot() before a call to reset_peak() can bemeaningfully compared to snapshots taken after the call.

Start tracing Python memory allocations: install hooks on Python memoryallocators. Collected tracebacks of traces will be limited to nframeframes. By default, a trace of a memory block only stores the most recentframe: the limit is 1. nframe must be greater or equal to 1.

The result is sorted from the biggest to the smallest by: absolute valueof StatisticDiff.size_diff, StatisticDiff.size, absolutevalue of StatisticDiff.count_diff, Statistic.count andthen by StatisticDiff.traceback.

Create a new Snapshot instance with a filtered tracessequence, filters is a list of DomainFilter andFilter instances. If filters is an empty list, return a newSnapshot instance with a copy of the traces.

If cumulative is True, cumulate size and count of memory blocks ofall frames of the traceback of a trace, not only the most recent frame.The cumulative mode can only be used with key_type equals to'filename' and 'lineno'.

When a snapshot is taken, tracebacks of traces are limited toget_traceback_limit() frames. See the take_snapshot() function.The original number of frames of the traceback is stored in theTraceback.total_nframe attribute. That allows to know if a tracebackhas been truncated by the traceback limit.

Format the traceback as a list of lines. Use the linecache module toretrieve lines from the source code. If limit is set, format the limitmost recent frames if limit is positive. Otherwise, format theabs(limit) oldest frames. If most_recent_first is True, the orderof the formatted frames is reversed, returning the most recent frame firstinstead of last.

A collector is a backend that is typically registered at program startup. Only applications should ever specify a backend. There is a simple trace-tef library that provides a backend that writes into a .json file in the Catapult/TEF format (compatible with chrome://tracing in chrome, and with perfetto).

Warning: In asynchronous code that uses async/await syntax,Span::enter may produce incorrect traces if the returned dropguard is held across an await point. Seethe method documentation for details.

An Event represents a moment in time. It signifies something thathappened while a trace was being recorded. Events are comparable to the logrecords emitted by unstructured logging code, but unlike a typical log line,an Event may occur within the context of a span.

c80f0f1006