Resguardo inicial

This commit is contained in:
lastdragon
2026-09-06 12:53:21 -06:00
parent 662efe3c2a
commit ad2088d178
94853 changed files with 43200404 additions and 0 deletions
@@ -0,0 +1,294 @@
.. SPDX-License-Identifier: GPL-2.0
======================================
CoreSight System Configuration Manager
======================================
:Author: Mike Leach <mike.leach@linaro.org>
:Date: October 2020
Introduction
============
The CoreSight System Configuration manager is an API that allows the
programming of the CoreSight system with pre-defined configurations that
can then be easily enabled from sysfs or perf.
Many CoreSight components can be programmed in complex ways - especially ETMs.
In addition, components can interact across the CoreSight system, often via
the cross trigger components such as CTI and CTM. These system settings can
be defined and enabled as named configurations.
Basic Concepts
==============
This section introduces the basic concepts of a CoreSight system configuration.
Features
--------
A feature is a named set of programming for a CoreSight device. The programming
is device dependent, and can be defined in terms of absolute register values,
resource usage and parameter values.
The feature is defined using a descriptor. This descriptor is used to load onto
a matching device, either when the feature is loaded into the system, or when the
CoreSight device is registered with the configuration manager.
The load process involves interpreting the descriptor into a set of register
accesses in the driver - the resource usage and parameter descriptions
translated into appropriate register accesses. This interpretation makes it easy
and efficient for the feature to be programmed onto the device when required.
The feature will not be active on the device until the feature is enabled, and
the device itself is enabled. When the device is enabled then enabled features
will be programmed into the device hardware.
A feature is enabled as part of a configuration being enabled on the system.
Parameter Value
~~~~~~~~~~~~~~~
A parameter value is a named value that may be set by the user prior to the
feature being enabled that can adjust the behaviour of the operation programmed
by the feature.
For example, this could be a count value in a programmed operation that repeats
at a given rate. When the feature is enabled then the current value of the
parameter is used in programming the device.
The feature descriptor defines a default value for a parameter, which is used
if the user does not supply a new value.
Users can update parameter values using the configfs API for the CoreSight
system - which is described below.
The current value of the parameter is loaded into the device when the feature
is enabled on that device.
Configurations
--------------
A configuration defines a set of features that are to be used in a trace
session where the configuration is selected. For any trace session only one
configuration may be selected.
The features defined may be on any type of device that is registered
to support system configuration. A configuration may select features to be
enabled on a class of devices - i.e. any ETMv4, or specific devices, e.g. a
specific CTI on the system.
As with the feature, a descriptor is used to define the configuration.
This will define the features that must be enabled as part of the configuration
as well as any preset values that can be used to override default parameter
values.
Preset Values
~~~~~~~~~~~~~
Preset values are easily selectable sets of parameter values for the features
that the configuration uses. The number of values in a single preset set, equals
the sum of parameter values in the features used by the configuration.
e.g. a configuration consists of 3 features, one has 2 parameters, one has
a single parameter, and another has no parameters. A single preset set will
therefore have 3 values.
Presets are optionally defined by the configuration, up to 15 can be defined.
If no preset is selected, then the parameter values defined in the feature
are used as normal.
Operation
~~~~~~~~~
The following steps take place in the operation of a configuration.
1) In this example, the configuration is 'autofdo', which has an
associated feature 'strobing' that works on ETMv4 CoreSight Devices.
2) The configuration is enabled. For example 'perf' may select the
configuration as part of its command line::
perf record -e cs_etm/autofdo/ myapp
which will enable the 'autofdo' configuration.
3) perf starts tracing on the system. As each ETMv4 that perf uses for
trace is enabled, the configuration manager will check if the ETMv4
has a feature that relates to the currently active configuration.
In this case 'strobing' is enabled & programmed into the ETMv4.
4) When the ETMv4 is disabled, any registers marked as needing to be
saved will be read back.
5) At the end of the perf session, the configuration will be disabled.
Viewing Configurations and Features
===================================
The set of configurations and features that are currently loaded into the
system can be viewed using the configfs API.
Mount configfs as normal and the 'cs-syscfg' subsystem will appear::
$ ls /config
cs-syscfg stp-policy
This has two sub-directories::
$ cd cs-syscfg/
$ ls
configurations features
The system has the configuration 'autofdo' built in. It may be examined as
follows::
$ cd configurations/
$ ls
autofdo
$ cd autofdo/
$ ls
description feature_refs preset1 preset3 preset5 preset7 preset9
enable preset preset2 preset4 preset6 preset8
$ cat description
Setup ETMs with strobing for autofdo
$ cat feature_refs
strobing
Each preset declared has a 'preset<n>' subdirectory declared. The values for
the preset can be examined::
$ cat preset1/values
strobing.window = 0x1388 strobing.period = 0x2
$ cat preset2/values
strobing.window = 0x1388 strobing.period = 0x4
The 'enable' and 'preset' files allow the control of a configuration when
using CoreSight with sysfs.
The features referenced by the configuration can be examined in the features
directory::
$ cd ../../features/strobing/
$ ls
description matches nr_params params
$ cat description
Generate periodic trace capture windows.
parameter 'window': a number of CPU cycles (W)
parameter 'period': trace enabled for W cycles every period x W cycles
$ cat matches
SRC_ETMV4
$ cat nr_params
2
Move to the params directory to examine and adjust parameters::
cd params
$ ls
period window
$ cd period
$ ls
value
$ cat value
0x2710
# echo 15000 > value
# cat value
0x3a98
Parameters adjusted in this way are reflected in all device instances that have
loaded the feature.
Using Configurations in perf
============================
The configurations loaded into the CoreSight configuration management are
also declared in the perf 'cs_etm' event infrastructure so that they can
be selected when running trace under perf::
$ ls /sys/devices/cs_etm
cpu0 cpu2 events nr_addr_filters power subsystem uevent
cpu1 cpu3 format perf_event_mux_interval_ms sinks type
The key directory here is 'events' - a generic perf directory which allows
selection on the perf command line. As with the sinks entries, this provides
a hash of the configuration name.
The entry in the 'events' directory uses perfs built in syntax generator
to substitute the syntax for the name when evaluating the command::
$ ls events/
autofdo
$ cat events/autofdo
configid=0xa7c3dddd
The 'autofdo' configuration may be selected on the perf command line::
$ perf record -e cs_etm/autofdo/u --per-thread <application>
A preset to override the current parameter values can also be selected::
$ perf record -e cs_etm/autofdo,preset=1/u --per-thread <application>
When configurations are selected in this way, then the trace sink used is
automatically selected.
Using Configurations in sysfs
=============================
Coresight can be controlled using sysfs. When this is in use then a configuration
can be made active for the devices that are used in the sysfs session.
In a configuration there are 'enable' and 'preset' files.
To enable a configuration for use with sysfs::
$ cd configurations/autofdo
$ echo 1 > enable
This will then use any default parameter values in the features - which can be
adjusted as described above.
To use a preset<n> set of parameter values::
$ echo 3 > preset
This will select preset3 for the configuration.
The valid values for preset are 0 - to deselect presets, and any value of
<n> where a preset<n> sub-directory is present.
Note that the active sysfs configuration is a global parameter, therefore
only a single configuration can be active for sysfs at any one time.
Attempting to enable a second configuration will result in an error.
Additionally, attempting to disable the configuration while in use will
also result in an error.
The use of the active configuration by sysfs is independent of the configuration
used in perf.
Creating and Loading Custom Configurations
==========================================
Custom configurations and / or features can be dynamically loaded into the
system by using a loadable module.
An example of a custom configuration is found in ./samples/coresight.
This creates a new configuration that uses the existing built in
strobing feature, but provides a different set of presets.
When the module is loaded, then the configuration appears in the configfs
file system and is selectable in the same way as the built in configuration
described above.
Configurations can use previously loaded features. The system will ensure
that it is not possible to unload a feature that is currently in use, by
enforcing the unload order as the strict reverse of the load order.
@@ -0,0 +1,193 @@
==========================
Coresight CPU Debug Module
==========================
:Author: Leo Yan <leo.yan@linaro.org>
:Date: April 5th, 2017
Introduction
------------
Coresight CPU debug module is defined in ARMv8-a architecture reference manual
(ARM DDI 0487A.k) Chapter 'Part H: External debug', the CPU can integrate
debug module and it is mainly used for two modes: self-hosted debug and
external debug. Usually the external debug mode is well known as the external
debugger connects with SoC from JTAG port; on the other hand the program can
explore debugging method which rely on self-hosted debug mode, this document
is to focus on this part.
The debug module provides sample-based profiling extension, which can be used
to sample CPU program counter, secure state and exception level, etc; usually
every CPU has one dedicated debug module to be connected. Based on self-hosted
debug mechanism, Linux kernel can access these related registers from mmio
region when the kernel panic happens. The callback notifier for kernel panic
will dump related registers for every CPU; finally this is good for assistant
analysis for panic.
Implementation
--------------
- During driver registration, it uses EDDEVID and EDDEVID1 - two device ID
registers to decide if sample-based profiling is implemented or not. On some
platforms this hardware feature is fully or partially implemented; and if
this feature is not supported then registration will fail.
- At the time this documentation was written, the debug driver mainly relies on
information gathered by the kernel panic callback notifier from three
sampling registers: EDPCSR, EDVIDSR and EDCIDSR: from EDPCSR we can get
program counter; EDVIDSR has information for secure state, exception level,
bit width, etc; EDCIDSR is context ID value which contains the sampled value
of CONTEXTIDR_EL1.
- The driver supports a CPU running in either AArch64 or AArch32 mode. The
registers naming convention is a bit different between them, AArch64 uses
'ED' for register prefix (ARM DDI 0487A.k, chapter H9.1) and AArch32 uses
'DBG' as prefix (ARM DDI 0487A.k, chapter G5.1). The driver is unified to
use AArch64 naming convention.
- ARMv8-a (ARM DDI 0487A.k) and ARMv7-a (ARM DDI 0406C.b) have different
register bits definition. So the driver consolidates two difference:
If PCSROffset=0b0000, on ARMv8-a the feature of EDPCSR is not implemented;
but ARMv7-a defines "PCSR samples are offset by a value that depends on the
instruction set state". For ARMv7-a, the driver checks furthermore if CPU
runs with ARM or thumb instruction set and calibrate PCSR value, the
detailed description for offset is in ARMv7-a ARM (ARM DDI 0406C.b) chapter
C11.11.34 "DBGPCSR, Program Counter Sampling Register".
If PCSROffset=0b0010, ARMv8-a defines "EDPCSR implemented, and samples have
no offset applied and do not sample the instruction set state in AArch32
state". So on ARMv8 if EDDEVID1.PCSROffset is 0b0010 and the CPU operates
in AArch32 state, EDPCSR is not sampled; when the CPU operates in AArch64
state EDPCSR is sampled and no offset are applied.
Clock and power domain
----------------------
Before accessing debug registers, we should ensure the clock and power domain
have been enabled properly. In ARMv8-a ARM (ARM DDI 0487A.k) chapter 'H9.1
Debug registers', the debug registers are spread into two domains: the debug
domain and the CPU domain.
::
+---------------+
| |
| |
+----------+--+ |
dbg_clock -->| |**| |<-- cpu_clock
| Debug |**| CPU |
dbg_power_domain -->| |**| |<-- cpu_power_domain
+----------+--+ |
| |
| |
+---------------+
For debug domain, the user uses DT binding "clocks" and "power-domains" to
specify the corresponding clock source and power supply for the debug logic.
The driver calls the pm_runtime_{put|get} operations as needed to handle the
debug power domain.
For CPU domain, the different SoC designs have different power management
schemes and finally this heavily impacts external debug module. So we can
divide into below cases:
- On systems with a sane power controller which can behave correctly with
respect to CPU power domain, the CPU power domain can be controlled by
register EDPRCR in driver. The driver firstly writes bit EDPRCR.COREPURQ
to power up the CPU, and then writes bit EDPRCR.CORENPDRQ for emulation
of CPU power down. As result, this can ensure the CPU power domain is
powered on properly during the period when access debug related registers;
- Some designs will power down an entire cluster if all CPUs on the cluster
are powered down - including the parts of the debug registers that should
remain powered in the debug power domain. The bits in EDPRCR are not
respected in these cases, so these designs do not support debug over
power down in the way that the CoreSight / Debug designers anticipated.
This means that even checking EDPRSR has the potential to cause a bus hang
if the target register is unpowered.
In this case, accessing to the debug registers while they are not powered
is a recipe for disaster; so we need preventing CPU low power states at boot
time or when user enable module at the run time. Please see chapter
"How to use the module" for detailed usage info for this.
Device Tree Bindings
--------------------
See Documentation/devicetree/bindings/arm/arm,coresight-cpu-debug.yaml for
details.
How to use the module
---------------------
If you want to enable debugging functionality at boot time, you can add
"coresight_cpu_debug.enable=1" to the kernel command line parameter.
The driver also can work as module, so can enable the debugging when insmod
module::
# insmod coresight_cpu_debug.ko debug=1
When boot time or insmod module you have not enabled the debugging, the driver
uses the debugfs file system to provide a knob to dynamically enable or disable
debugging:
To enable it, write a '1' into /sys/kernel/debug/coresight_cpu_debug/enable::
# echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable
To disable it, write a '0' into /sys/kernel/debug/coresight_cpu_debug/enable::
# echo 0 > /sys/kernel/debug/coresight_cpu_debug/enable
As explained in chapter "Clock and power domain", if you are working on one
platform which has idle states to power off debug logic and the power
controller cannot work well for the request from EDPRCR, then you should
firstly constraint CPU idle states before enable CPU debugging feature; so can
ensure the accessing to debug logic.
If you want to limit idle states at boot time, you can use "nohlt" or
"cpuidle.off=1" in the kernel command line.
At the runtime you can disable idle states with below methods:
It is possible to disable CPU idle states by way of the PM QoS
subsystem, more specifically by using the "/dev/cpu_dma_latency"
interface (see Documentation/power/pm_qos_interface.rst for more
details). As specified in the PM QoS documentation the requested
parameter will stay in effect until the file descriptor is released.
For example::
# exec 3<> /dev/cpu_dma_latency; echo 0 >&3
...
Do some work...
...
# exec 3<>-
The same can also be done from an application program.
Disable specific CPU's specific idle state from cpuidle sysfs (see
Documentation/admin-guide/pm/cpuidle.rst)::
# echo 1 > /sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
Output format
-------------
Here is an example of the debugging output format::
ARM external debug module:
coresight-cpu-debug 850000.debug: CPU[0]:
coresight-cpu-debug 850000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 850000.debug: EDPCSR: handle_IPI+0x174/0x1d8
coresight-cpu-debug 850000.debug: EDCIDSR: 00000000
coresight-cpu-debug 850000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
coresight-cpu-debug 852000.debug: CPU[1]:
coresight-cpu-debug 852000.debug: EDPRSR: 00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 852000.debug: EDPCSR: debug_notifier_call+0x23c/0x358
coresight-cpu-debug 852000.debug: EDCIDSR: 00000000
coresight-cpu-debug 852000.debug: EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
@@ -0,0 +1,32 @@
.. SPDX-License-Identifier: GPL-2.0
=============================
Coresight Dummy Trace Module
=============================
:Author: Hao Zhang <quic_hazha@quicinc.com>
:Date: June 2023
Introduction
------------
The Coresight dummy trace module is for the specific devices that kernel don't
have permission to access or configure, e.g., CoreSight TPDMs on Qualcomm
platforms. For these devices, a dummy driver is needed to register them as
Coresight devices. The module may also be used to define components that may
not have any programming interfaces, so that paths can be created in the driver.
It provides Coresight API for operations on dummy devices, such as enabling and
disabling them. It also provides the Coresight dummy sink/source paths for
debugging.
Config details
--------------
There are two types of nodes, dummy sink and dummy source. These nodes
are available at ``/sys/bus/coresight/devices``.
Example output::
$ ls -l /sys/bus/coresight/devices | grep dummy
dummy_sink0 -> ../../../devices/platform/soc@0/soc@0:sink/dummy_sink0
dummy_source0 -> ../../../devices/platform/soc@0/soc@0:source/dummy_source0
@@ -0,0 +1,226 @@
.. SPDX-License-Identifier: GPL-2.0
=============================================
CoreSight Embedded Cross Trigger (CTI & CTM).
=============================================
:Author: Mike Leach <mike.leach@linaro.org>
:Date: November 2019
Hardware Description
--------------------
The CoreSight Cross Trigger Interface (CTI) is a hardware device that takes
individual input and output hardware signals known as triggers to and from
devices and interconnects them via the Cross Trigger Matrix (CTM) to other
devices via numbered channels, in order to propagate events between devices.
e.g.::
0000000 in_trigs :::::::
0 C 0----------->: : +======>(other CTI channel IO)
0 P 0<-----------: : v
0 U 0 out_trigs : : Channels ***** :::::::
0000000 : CTI :<=========>*CTM*<====>: CTI :---+
####### in_trigs : : (id 0-3) ***** ::::::: v
# ETM #----------->: : ^ #######
# #<-----------: : +---# ETR #
####### out_trigs ::::::: #######
The CTI driver enables the programming of the CTI to attach triggers to
channels. When an input trigger becomes active, the attached channel will
become active. Any output trigger attached to that channel will also
become active. The active channel is propagated to other CTIs via the CTM,
activating connected output triggers there, unless filtered by the CTI
channel gate.
It is also possible to activate a channel using system software directly
programming registers in the CTI.
The CTIs are registered by the system to be associated with CPUs and/or other
CoreSight devices on the trace data path. When these devices are enabled the
attached CTIs will also be enabled. By default/on power up the CTIs have
no programmed trigger/channel attachments, so will not affect the system
until explicitly programmed.
The hardware trigger connections between CTIs and devices is implementation
defined, unless the CPU/ETM combination is a v8 architecture, in which case
the connections have an architecturally defined standard layout.
The hardware trigger signals can also be connected to non-CoreSight devices
(e.g. UART), or be propagated off chip as hardware IO lines.
All the CTI devices are associated with a CTM. On many systems there will be a
single effective CTM (one CTM, or multiple CTMs all interconnected), but it is
possible that systems can have nets of CTIs+CTM that are not interconnected by
a CTM to each other. On these systems a CTM index is declared to associate
CTI devices that are interconnected via a given CTM.
Sysfs files and directories
---------------------------
The CTI devices appear on the existing CoreSight bus alongside the other
CoreSight devices::
>$ ls /sys/bus/coresight/devices
cti_cpu0 cti_cpu2 cti_sys0 etm0 etm2 funnel0 replicator0 tmc_etr0
cti_cpu1 cti_cpu3 cti_sys1 etm1 etm3 funnel1 tmc_etf0 tpiu0
The ``cti_cpu<N>`` named CTIs are associated with a CPU, and any ETM used by
that core. The ``cti_sys<N>`` CTIs are general system infrastructure CTIs that
can be associated with other CoreSight devices, or other system hardware
capable of generating or using trigger signals.::
>$ ls /sys/bus/coresight/devices/etm0/cti_cpu0
channels ctmid enable nr_trigger_cons mgmt power powered regs
connections subsystem triggers0 triggers1 uevent
*Key file items are:-*
* ``enable``: enables/disables the CTI. Read to determine current state.
If this shows as enabled (1), but ``powered`` shows unpowered (0), then
the enable indicates a request to enabled when the device is powered.
* ``ctmid`` : associated CTM - only relevant if system has multiple CTI+CTM
clusters that are not interconnected.
* ``nr_trigger_cons`` : total connections - triggers<N> directories.
* ``powered`` : Read to determine if the CTI is currently powered.
*Sub-directories:-*
* ``triggers<N>``: contains list of triggers for an individual connection.
* ``channels``: Contains the channel API - CTI main programming interface.
* ``regs``: Gives access to the raw programmable CTI regs.
* ``mgmt``: the standard CoreSight management registers.
* ``connections``: Links to connected *CoreSight* devices. The number of
links can be 0 to ``nr_trigger_cons``. Actual number given by ``nr_links``
in this directory.
triggers<N> directories
~~~~~~~~~~~~~~~~~~~~~~~
Individual trigger connection information. This describes trigger signals for
CoreSight and non-CoreSight connections.
Each triggers directory has a set of parameters describing the triggers for
the connection.
* ``name`` : name of connection
* ``in_signals`` : input trigger signal indexes used in this connection.
* ``in_types`` : functional types for in signals.
* ``out_signals`` : output trigger signals for this connection.
* ``out_types`` : functional types for out signals.
e.g::
>$ ls ./cti_cpu0/triggers0/
in_signals in_types name out_signals out_types
>$ cat ./cti_cpu0/triggers0/name
cpu0
>$ cat ./cti_cpu0/triggers0/out_signals
0-2
>$ cat ./cti_cpu0/triggers0/out_types
pe_edbgreq pe_dbgrestart pe_ctiirq
>$ cat ./cti_cpu0/triggers0/in_signals
0-1
>$ cat ./cti_cpu0/triggers0/in_types
pe_dbgtrigger pe_pmuirq
If a connection has zero signals in either the 'in' or 'out' triggers then
those parameters will be omitted.
Channels API Directory
~~~~~~~~~~~~~~~~~~~~~~
This provides an easy way to attach triggers to channels, without needing
the multiple register operations that are required if manipulating the
'regs' sub-directory elements directly.
A number of files provide this API::
>$ ls ./cti_sys0/channels/
chan_clear chan_inuse chan_xtrigs_out trigin_attach
chan_free chan_pulse chan_xtrigs_reset trigin_detach
chan_gate_disable chan_set chan_xtrigs_sel trigout_attach
chan_gate_enable chan_xtrigs_in trig_filter_enable trigout_detach
trigout_filtered
Most access to these elements take the form::
echo <chan> [<trigger>] > /<device_path>/<operation>
where the optional <trigger> is only needed for trigXX_attach | detach
operations.
e.g.::
>$ echo 0 1 > ./cti_sys0/channels/trigout_attach
>$ echo 0 > ./cti_sys0/channels/chan_set
Attaches trigout(1) to channel(0), then activates channel(0) generating a
set state on cti_sys0.trigout(1)
*API operations*
* ``trigin_attach, trigout_attach``: Attach a channel to a trigger signal.
* ``trigin_detach, trigout_detach``: Detach a channel from a trigger signal.
* ``chan_set``: Set the channel - the set state will be propagated around
the CTM to other connected devices.
* ``chan_clear``: Clear the channel.
* ``chan_pulse``: Set the channel for a single CoreSight clock cycle.
* ``chan_gate_enable``: Write operation sets the CTI gate to propagate
(enable) the channel to other devices. This operation takes a channel
number. CTI gate is enabled for all channels by default at power up. Read
to list the currently enabled channels on the gate.
* ``chan_gate_disable``: Write channel number to disable gate for that
channel.
* ``chan_inuse``: Show the current channels attached to any signal
* ``chan_free``: Show channels with no attached signals.
* ``chan_xtrigs_sel``: write a channel number to select a channel to view,
read to show the selected channel number.
* ``chan_xtrigs_in``: Read to show the input triggers attached to
the selected view channel.
* ``chan_xtrigs_out``:Read to show the output triggers attached to
the selected view channel.
* ``trig_filter_enable``: Defaults to enabled, disable to allow potentially
dangerous output signals to be set.
* ``trigout_filtered``: Trigger out signals that are prevented from being
set if filtering ``trig_filter_enable`` is enabled. One use is to prevent
accidental ``EDBGREQ`` signals stopping a core.
* ``chan_xtrigs_reset``: Write 1 to clear all channel / trigger programming.
Resets device hardware to default state.
The example below attaches input trigger index 1 to channel 2, and output
trigger index 6 to the same channel. It then examines the state of the
channel / trigger connections using the appropriate sysfs attributes.
The settings mean that if either input trigger 1, or channel 2 go active then
trigger out 6 will go active. We then enable the CTI, and use the software
channel control to activate channel 2. We see the active channel on the
``choutstatus`` register and the active signal on the ``trigoutstatus``
register. Finally clearing the channel removes this.
e.g.::
.../cti_sys0/channels# echo 2 1 > trigin_attach
.../cti_sys0/channels# echo 2 6 > trigout_attach
.../cti_sys0/channels# cat chan_free
0-1,3
.../cti_sys0/channels# cat chan_inuse
2
.../cti_sys0/channels# echo 2 > chan_xtrigs_sel
.../cti_sys0/channels# cat chan_xtrigs_trigin
1
.../cti_sys0/channels# cat chan_xtrigs_trigout
6
.../cti_sys0/# echo 1 > enable
.../cti_sys0/channels# echo 2 > chan_set
.../cti_sys0/channels# cat ../regs/choutstatus
0x4
.../cti_sys0/channels# cat ../regs/trigoutstatus
0x40
.../cti_sys0/channels# echo 2 > chan_clear
.../cti_sys0/channels# cat ../regs/trigoutstatus
0x0
.../cti_sys0/channels# cat ../regs/choutstatus
0x0
@@ -0,0 +1,827 @@
===============================================
ETMv4 sysfs linux driver programming reference.
===============================================
:Author: Mike Leach <mike.leach@linaro.org>
:Date: October 11th, 2019
Supplement to existing ETMv4 driver documentation.
Sysfs files and directories
---------------------------
Root: ``/sys/bus/coresight/devices/etm<N>``
The following paragraphs explain the association between sysfs files and the
ETMv4 registers that they effect. Note the register names are given without
the ‘TRC’ prefix.
----
:File: ``mode`` (rw)
:Trace Registers: {CONFIGR + others}
:Notes:
Bit select trace features. See ‘mode’ section below. Bits
in this will cause equivalent programming of trace config and
other registers to enable the features requested.
:Syntax & eg:
``echo bitfield > mode``
bitfield up to 32 bits setting trace features.
:Example:
``$> echo 0x012 > mode``
----
:File: ``reset`` (wo)
:Trace Registers: All
:Notes:
Reset all programming to trace nothing / no logic programmed.
:Syntax:
``echo 1 > reset``
----
:File: ``enable_source`` (wo)
:Trace Registers: PRGCTLR, All hardware regs.
:Notes:
- > 0 : Programs up the hardware with the current values held in the driver
and enables trace.
- = 0 : disable trace hardware.
:Syntax:
``echo 1 > enable_source``
----
:File: ``cpu`` (ro)
:Trace Registers: None.
:Notes:
CPU ID that this ETM is attached to.
:Example:
``$> cat cpu``
``$> 0``
----
:File: ``ts_source`` (ro)
:Trace Registers: None.
:Notes:
When FEAT_TRF is implemented, value of TRFCR_ELx.TS used for trace session. Otherwise -1
indicates an unknown time source. Check trcidr0.tssize to see if a global timestamp is
available.
:Example:
``$> cat ts_source``
``$> 1``
----
:File: ``addr_idx`` (rw)
:Trace Registers: None.
:Notes:
Virtual register to index address comparator and range
features. Set index for first of the pair in a range.
:Syntax:
``echo idx > addr_idx``
Where idx < nr_addr_cmp x 2
----
:File: ``addr_range`` (rw)
:Trace Registers: ACVR[idx, idx+1], VIIECTLR
:Notes:
Pair of addresses for a range selected by addr_idx. Include
/ exclude according to the optional parameter, or if omitted
uses the current ‘mode’ setting. Select comparator range in
control register. Error if index is odd value.
:Depends: ``mode, addr_idx``
:Syntax:
``echo addr1 addr2 [exclude] > addr_range``
Where addr1 and addr2 define the range and addr1 < addr2.
Optional exclude value:-
- 0 for include
- 1 for exclude.
:Example:
``$> echo 0x0000 0x2000 0 > addr_range``
----
:File: ``addr_single`` (rw)
:Trace Registers: ACVR[idx]
:Notes:
Set a single address comparator according to addr_idx. This
is used if the address comparator is used as part of event
generation logic etc.
:Depends: ``addr_idx``
:Syntax:
``echo addr1 > addr_single``
----
:File: ``addr_start`` (rw)
:Trace Registers: ACVR[idx], VISSCTLR
:Notes:
Set a trace start address comparator according to addr_idx.
Select comparator in control register.
:Depends: ``addr_idx``
:Syntax:
``echo addr1 > addr_start``
----
:File: ``addr_stop`` (rw)
:Trace Registers: ACVR[idx], VISSCTLR
:Notes:
Set a trace stop address comparator according to addr_idx.
Select comparator in control register.
:Depends: ``addr_idx``
:Syntax:
``echo addr1 > addr_stop``
----
:File: ``addr_context`` (rw)
:Trace Registers: ACATR[idx,{6:4}]
:Notes:
Link context ID comparator to address comparator addr_idx
:Depends: ``addr_idx``
:Syntax:
``echo ctxt_idx > addr_context``
Where ctxt_idx is the index of the linked context id / vmid
comparator.
----
:File: ``addr_ctxtype`` (rw)
:Trace Registers: ACATR[idx,{3:2}]
:Notes:
Input value string. Set type for linked context ID comparator
:Depends: ``addr_idx``
:Syntax:
``echo type > addr_ctxtype``
Type one of {all, vmid, ctxid, none}
:Example:
``$> echo ctxid > addr_ctxtype``
----
:File: ``addr_exlevel_s_ns`` (rw)
:Trace Registers: ACATR[idx,{14:8}]
:Notes:
Set the ELx secure and non-secure matching bits for the
selected address comparator
:Depends: ``addr_idx``
:Syntax:
``echo val > addr_exlevel_s_ns``
val is a 7 bit value for exception levels to exclude. Input
value shifted to correct bits in register.
:Example:
``$> echo 0x4F > addr_exlevel_s_ns``
----
:File: ``addr_instdatatype`` (rw)
:Trace Registers: ACATR[idx,{1:0}]
:Notes:
Set the comparator address type for matching. Driver only
supports setting instruction address type.
:Depends: ``addr_idx``
----
:File: ``addr_cmp_view`` (ro)
:Trace Registers: ACVR[idx, idx+1], ACATR[idx], VIIECTLR
:Notes:
Read the currently selected address comparator. If part of
address range then display both addresses.
:Depends: ``addr_idx``
:Syntax:
``cat addr_cmp_view``
:Example:
``$> cat addr_cmp_view``
``addr_cmp[0] range 0x0 0xffffffffffffffff include ctrl(0x4b00)``
----
:File: ``nr_addr_cmp`` (ro)
:Trace Registers: From IDR4
:Notes:
Number of address comparator pairs
----
:File: ``sshot_idx`` (rw)
:Trace Registers: None
:Notes:
Select single shot register set.
----
:File: ``sshot_ctrl`` (rw)
:Trace Registers: SSCCR[idx]
:Notes:
Access a single shot comparator control register.
:Depends: ``sshot_idx``
:Syntax:
``echo val > sshot_ctrl``
Writes val into the selected control register.
----
:File: ``sshot_status`` (ro)
:Trace Registers: SSCSR[idx]
:Notes:
Read a single shot comparator status register
:Depends: ``sshot_idx``
:Syntax:
``cat sshot_status``
Read status.
:Example:
``$> cat sshot_status``
``0x1``
----
:File: ``sshot_pe_ctrl`` (rw)
:Trace Registers: SSPCICR[idx]
:Notes:
Access a single shot PE comparator input control register.
:Depends: ``sshot_idx``
:Syntax:
``echo val > sshot_pe_ctrl``
Writes val into the selected control register.
----
:File: ``ns_exlevel_vinst`` (rw)
:Trace Registers: VICTLR{23:20}
:Notes:
Program non-secure exception level filters. Set / clear NS
exception filter bits. Setting ‘1’ excludes trace from the
exception level.
:Syntax:
``echo bitfield > ns_exlevel_viinst``
Where bitfield contains bits to set clear for EL0 to EL2
:Example:
``%> echo 0x4 > ns_exlevel_viinst``
Excludes EL2 NS trace.
----
:File: ``vinst_pe_cmp_start_stop`` (rw)
:Trace Registers: VIPCSSCTLR
:Notes:
Access PE start stop comparator input control registers
----
:File: ``bb_ctrl`` (rw)
:Trace Registers: BBCTLR
:Notes:
Define ranges that Branch Broadcast will operate in.
Default (0x0) is all addresses.
:Depends: BB enabled.
----
:File: ``cyc_threshold`` (rw)
:Trace Registers: CCCTLR
:Notes:
Set the threshold for which cycle counts will be emitted.
Error if attempt to set below minimum defined in IDR3, masked
to width of valid bits.
:Depends: CC enabled.
----
:File: ``syncfreq`` (rw)
:Trace Registers: SYNCPR
:Notes:
Set trace synchronisation period. Power of 2 value, 0 (off)
or 8-20. Driver defaults to 12 (every 4096 bytes).
----
:File: ``cntr_idx`` (rw)
:Trace Registers: none
:Notes:
Select the counter to access
:Syntax:
``echo idx > cntr_idx``
Where idx < nr_cntr
----
:File: ``cntr_ctrl`` (rw)
:Trace Registers: CNTCTLR[idx]
:Notes:
Set counter control value.
:Depends: ``cntr_idx``
:Syntax:
``echo val > cntr_ctrl``
Where val is per ETMv4 spec.
----
:File: ``cntrldvr`` (rw)
:Trace Registers: CNTRLDVR[idx]
:Notes:
Set counter reload value.
:Depends: ``cntr_idx``
:Syntax:
``echo val > cntrldvr``
Where val is per ETMv4 spec.
----
:File: ``nr_cntr`` (ro)
:Trace Registers: From IDR5
:Notes:
Number of counters implemented.
----
:File: ``ctxid_idx`` (rw)
:Trace Registers: None
:Notes:
Select the context ID comparator to access
:Syntax:
``echo idx > ctxid_idx``
Where idx < numcidc
----
:File: ``ctxid_pid`` (rw)
:Trace Registers: CIDCVR[idx]
:Notes:
Set the context ID comparator value
:Depends: ``ctxid_idx``
----
:File: ``ctxid_masks`` (rw)
:Trace Registers: CIDCCTLR0, CIDCCTLR1, CIDCVR<0-7>
:Notes:
Pair of values to set the byte masks for 1-8 context ID
comparators. Automatically clears masked bytes to 0 in CID
value registers.
:Syntax:
``echo m3m2m1m0 [m7m6m5m4] > ctxid_masks``
32 bit values made up of mask bytes, where mN represents a
byte mask value for Context ID comparator N.
Second value not required on systems that have fewer than 4
context ID comparators
----
:File: ``numcidc`` (ro)
:Trace Registers: From IDR4
:Notes:
Number of Context ID comparators
----
:File: ``vmid_idx`` (rw)
:Trace Registers: None
:Notes:
Select the VM ID comparator to access.
:Syntax:
``echo idx > vmid_idx``
Where idx < numvmidc
----
:File: ``vmid_val`` (rw)
:Trace Registers: VMIDCVR[idx]
:Notes:
Set the VM ID comparator value
:Depends: ``vmid_idx``
----
:File: ``vmid_masks`` (rw)
:Trace Registers: VMIDCCTLR0, VMIDCCTLR1, VMIDCVR<0-7>
:Notes:
Pair of values to set the byte masks for 1-8 VM ID comparators.
Automatically clears masked bytes to 0 in VMID value registers.
:Syntax:
``echo m3m2m1m0 [m7m6m5m4] > vmid_masks``
Where mN represents a byte mask value for VMID comparator N.
Second value not required on systems that have fewer than 4
VMID comparators.
----
:File: ``numvmidc`` (ro)
:Trace Registers: From IDR4
:Notes:
Number of VMID comparators
----
:File: ``res_idx`` (rw)
:Trace Registers: None.
:Notes:
Select the resource selector control to access. Must be 2 or
higher as selectors 0 and 1 are hardwired.
:Syntax:
``echo idx > res_idx``
Where 2 <= idx < nr_resource x 2
----
:File: ``res_ctrl`` (rw)
:Trace Registers: RSCTLR[idx]
:Notes:
Set resource selector control value. Value per ETMv4 spec.
:Depends: ``res_idx``
:Syntax:
``echo val > res_cntr``
Where val is per ETMv4 spec.
----
:File: ``nr_resource`` (ro)
:Trace Registers: From IDR4
:Notes:
Number of resource selector pairs
----
:File: ``event`` (rw)
:Trace Registers: EVENTCTRL0R
:Notes:
Set up to 4 implemented event fields.
:Syntax:
``echo ev3ev2ev1ev0 > event``
Where evN is an 8 bit event field. Up to 4 event fields make up the
32-bit input value. Number of valid fields is implementation dependent,
defined in IDR0.
----
:File: ``event_instren`` (rw)
:Trace Registers: EVENTCTRL1R
:Notes:
Choose events which insert event packets into trace stream.
:Depends: EVENTCTRL0R
:Syntax:
``echo bitfield > event_instren``
Where bitfield is up to 4 bits according to number of event fields.
----
:File: ``event_ts`` (rw)
:Trace Registers: TSCTLR
:Notes:
Set the event that will generate timestamp requests.
:Depends: ``TS activated``
:Syntax:
``echo evfield > event_ts``
Where evfield is an 8 bit event selector.
----
:File: ``seq_idx`` (rw)
:Trace Registers: None
:Notes:
Sequencer event register select - 0 to 2
----
:File: ``seq_state`` (rw)
:Trace Registers: SEQSTR
:Notes:
Sequencer current state - 0 to 3.
----
:File: ``seq_event`` (rw)
:Trace Registers: SEQEVR[idx]
:Notes:
State transition event registers
:Depends: ``seq_idx``
:Syntax:
``echo evBevF > seq_event``
Where evBevF is a 16 bit value made up of two event selectors,
- evB : back
- evF : forwards.
----
:File: ``seq_reset_event`` (rw)
:Trace Registers: SEQRSTEVR
:Notes:
Sequencer reset event
:Syntax:
``echo evfield > seq_reset_event``
Where evfield is an 8 bit event selector.
----
:File: ``nrseqstate`` (ro)
:Trace Registers: From IDR5
:Notes:
Number of sequencer states (0 or 4)
----
:File: ``nr_pe_cmp`` (ro)
:Trace Registers: From IDR4
:Notes:
Number of PE comparator inputs
----
:File: ``nr_ext_inp`` (ro)
:Trace Registers: From IDR5
:Notes:
Number of external inputs
----
:File: ``nr_ss_cmp`` (ro)
:Trace Registers: From IDR4
:Notes:
Number of Single Shot control registers
----
*Note:* When programming any address comparator the driver will tag the
comparator with a type used - i.e. RANGE, SINGLE, START, STOP. Once this tag
is set, then only the values can be changed using the same sysfs file / type
used to program it.
Thus::
% echo 0 > addr_idx ; select address comparator 0
% echo 0x1000 0x5000 0 > addr_range ; set address range on comparators 0, 1.
% echo 0x2000 > addr_start ; error as comparator 0 is a range comparator
% echo 2 > addr_idx ; select address comparator 2
% echo 0x2000 > addr_start ; this is OK as comparator 2 is unused.
% echo 0x3000 > addr_stop ; error as comparator 2 set as start address.
% echo 2 > addr_idx ; select address comparator 3
% echo 0x3000 > addr_stop ; this is OK
To remove programming on all the comparators (and all the other hardware) use
the reset parameter::
% echo 1 > reset
The ‘mode’ sysfs parameter.
---------------------------
This is a bitfield selection parameter that sets the overall trace mode for the
ETM. The table below describes the bits, using the defines from the driver
source file, along with a description of the feature these represent. Many
features are optional and therefore dependent on implementation in the
hardware.
Bit assignments shown below:-
----
**bit (0):**
ETM_MODE_EXCLUDE
**description:**
This is the default value for the include / exclude function when
setting address ranges. Set 1 for exclude range. When the mode
parameter is set this value is applied to the currently indexed
address range.
.. _coresight-branch-broadcast:
**bit (4):**
ETM_MODE_BB
**description:**
Set to enable branch broadcast if supported in hardware [IDR0]. The primary use for this feature
is when code is patched dynamically at run time and the full program flow may not be able to be
reconstructed using only conditional branches.
There is currently no support in Perf for supplying modified binaries to the decoder, so this
feature is only intended to be used for debugging purposes or with a 3rd party tool.
Choosing this option will result in a significant increase in the amount of trace generated -
possible danger of overflows, or fewer instructions covered. Note, that this option also
overrides any setting of :ref:`ETM_MODE_RETURNSTACK <coresight-return-stack>`, so where a branch
broadcast range overlaps a return stack range, return stacks will not be available for that
range.
.. _coresight-cycle-accurate:
**bit (5):**
ETMv4_MODE_CYCACC
**description:**
Set to enable cycle accurate trace if supported [IDR0].
**bit (6):**
ETMv4_MODE_CTXID
**description:**
Set to enable context ID tracing if supported in hardware [IDR2].
**bit (7):**
ETM_MODE_VMID
**description:**
Set to enable virtual machine ID tracing if supported [IDR2].
.. _coresight-timestamp:
**bit (11):**
ETMv4_MODE_TIMESTAMP
**description:**
Set to enable timestamp generation if supported [IDR0].
.. _coresight-return-stack:
**bit (12):**
ETM_MODE_RETURNSTACK
**description:**
Set to enable trace return stack use if supported [IDR0].
**bit (13-14):**
ETM_MODE_QELEM(val)
**description:**
‘val’ determines level of Q element support enabled if
implemented by the ETM [IDR0]
**bit (19):**
ETM_MODE_ATB_TRIGGER
**description:**
Set to enable the ATBTRIGGER bit in the event control register
[EVENTCTLR1] if supported [IDR5].
**bit (20):**
ETM_MODE_LPOVERRIDE
**description:**
Set to enable the LPOVERRIDE bit in the event control register
[EVENTCTLR1], if supported [IDR5].
**bit (21):**
ETM_MODE_ISTALL_EN
**description:**
Set to enable the ISTALL bit in the stall control register
[STALLCTLR]
**bit (23):**
ETM_MODE_INSTPRIO
**description:**
Set to enable the INSTPRIORITY bit in the stall control register
[STALLCTLR] , if supported [IDR0].
**bit (24):**
ETM_MODE_NOOVERFLOW
**description:**
Set to enable the NOOVERFLOW bit in the stall control register
[STALLCTLR], if supported [IDR3].
**bit (25):**
ETM_MODE_TRACE_RESET
**description:**
Set to enable the TRCRESET bit in the viewinst control register
[VICTLR] , if supported [IDR3].
**bit (26):**
ETM_MODE_TRACE_ERR
**description:**
Set to enable the TRCCTRL bit in the viewinst control register
[VICTLR].
**bit (27):**
ETM_MODE_VIEWINST_STARTSTOP
**description:**
Set the initial state value of the ViewInst start / stop logic
in the viewinst control register [VICTLR]
**bit (30):**
ETM_MODE_EXCL_KERN
**description:**
Set default trace setup to exclude kernel mode trace (see note a)
**bit (31):**
ETM_MODE_EXCL_USER
**description:**
Set default trace setup to exclude user space trace (see note a)
----
*Note a)* On startup the ETM is programmed to trace the complete address space
using address range comparator 0. ‘mode’ bits 30 / 31 modify this setting to
set EL exclude bits for NS state in either user space (EL0) or kernel space
(EL1) in the address range comparator. (the default setting excludes all
secure EL, and NS EL2)
Once the reset parameter has been used, and/or custom programming has been
implemented - using these bits will result in the EL bits for address
comparator 0 being set in the same way.
*Note b)* Bits 2-3, 8-10, 15-16, 18, 22, control features that only work with
data trace. As A-profile data trace is architecturally prohibited in ETMv4,
these have been omitted here. Possible uses could be where a kernel has
support for control of R or M profile infrastructure as part of a heterogeneous
system.
Bits 17, 28-29 are unused.
@@ -0,0 +1,117 @@
.. SPDX-License-Identifier: GPL-2.0
================
CoreSight - Perf
================
:Author: Carsten Haitzler <carsten.haitzler@arm.com>
:Date: June 29th, 2022
Perf is able to locally access CoreSight trace data and store it to the
output perf data files. This data can then be later decoded to give the
instructions that were traced for debugging or profiling purposes. You
can log such data with a perf record command like::
perf record -e cs_etm//u testbinary
This would run some test binary (testbinary) until it exits and record
a perf.data trace file. That file would have AUX sections if CoreSight
is working correctly. You can dump the content of this file as
readable text with a command like::
perf report --stdio --dump -i perf.data
You should find some sections of this file have AUX data blocks like::
0x1e78 [0x30]: PERF_RECORD_AUXTRACE size: 0x11dd0 offset: 0 ref: 0x1b614fc1061b0ad1 idx: 0 tid: 531230 cpu: -1
. ... CoreSight ETM Trace data: size 73168 bytes
Idx:0; ID:10; I_ASYNC : Alignment Synchronisation.
Idx:12; ID:10; I_TRACE_INFO : Trace Info.; INFO=0x0 { CC.0 }
Idx:17; ID:10; I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000000000000000;
Idx:26; ID:10; I_TRACE_ON : Trace On.
Idx:27; ID:10; I_ADDR_CTXT_L_64IS0 : Address & Context, Long, 64 bit, IS0.; Addr=0x0000FFFFB6069140; Ctxt: AArch64,EL0, NS;
Idx:38; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
Idx:39; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
Idx:40; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEEEEEEEEEEEEEE
Idx:41; ID:10; I_ATOM_F6 : Atom format 6.; EEEEEEEEEEEN
...
If you see these above, then your system is tracing CoreSight data
correctly.
To compile perf with CoreSight support in the tools/perf directory do::
make CORESIGHT=1
This requires OpenCSD to build. You may install distribution packages
for the support such as libopencsd and libopencsd-dev or download it
and build yourself. Upstream OpenCSD is located at:
https://github.com/Linaro/OpenCSD
For complete information on building perf with CoreSight support and
more extensive usage look at:
https://github.com/Linaro/OpenCSD/blob/master/HOWTO.md
Kernel CoreSight Support
------------------------
You will also want CoreSight support enabled in your kernel config.
Ensure it is enabled with::
CONFIG_CORESIGHT=y
There are various other CoreSight options you probably also want
enabled like::
CONFIG_CORESIGHT_LINKS_AND_SINKS=y
CONFIG_CORESIGHT_LINK_AND_SINK_TMC=y
CONFIG_CORESIGHT_CATU=y
CONFIG_CORESIGHT_SINK_TPIU=y
CONFIG_CORESIGHT_SINK_ETBV10=y
CONFIG_CORESIGHT_SOURCE_ETM4X=y
CONFIG_CORESIGHT_CTI=y
CONFIG_CORESIGHT_CTI_INTEGRATION_REGS=y
Please refer to the kernel configuration help for more information.
Fine-grained tracing with AUX pause and resume
----------------------------------------------
Arm CoreSight may generate a large amount of hardware trace data, which
will lead to overhead in recording and distract users when reviewing
profiling result. To mitigate the issue of excessive trace data, Perf
provides AUX pause and resume functionality for fine-grained tracing.
The AUX pause and resume can be triggered by associated events. These
events can be ftrace tracepoints (including static and dynamic
tracepoints) or PMU events (e.g. CPU PMU cycle event). To create a perf
session with AUX pause / resume, three configuration terms are
introduced:
- "aux-action=start-paused": it is specified for the cs_etm PMU event to
launch in a paused state.
- "aux-action=pause": an associated event is specified with this term
to pause AUX trace.
- "aux-action=resume": an associated event is specified with this term
to resume AUX trace.
Example for triggering AUX pause and resume with ftrace tracepoints::
perf record -e cs_etm/aux-action=start-paused/k,syscalls:sys_enter_openat/aux-action=resume/,syscalls:sys_exit_openat/aux-action=pause/ ls
Example for triggering AUX pause and resume with PMU event::
perf record -a -e cs_etm/aux-action=start-paused/k \
-e cycles/aux-action=pause,period=10000000/ \
-e cycles/aux-action=resume,period=1050000/ -- sleep 1
Perf test - Verify kernel and userspace perf CoreSight work
-----------------------------------------------------------
There are a set of Perf tests for CoreSight which can be run with::
sudo perf test coresight
@@ -0,0 +1,52 @@
.. SPDX-License-Identifier: GPL-2.0
=================================================================
The trace performance monitoring and diagnostics aggregator(TPDA)
=================================================================
:Author: Jinlong Mao <quic_jinlmao@quicinc.com>
:Date: January 2023
Hardware Description
--------------------
TPDA - The trace performance monitoring and diagnostics aggregator or
TPDA in short serves as an arbitration and packetization engine for the
performance monitoring and diagnostics network specification.
The primary use case of the TPDA is to provide packetization, funneling
and timestamping of Monitor data.
Sysfs files and directories
---------------------------
Root: ``/sys/bus/coresight/devices/tpda<N>``
Config details
---------------------------
The tpdm and tpda nodes should be observed at the coresight path
"/sys/bus/coresight/devices".
e.g.
/sys/bus/coresight/devices # ls -l | grep tpd
tpda0 -> ../../../devices/platform/soc@0/6004000.tpda/tpda0
tpdm0 -> ../../../devices/platform/soc@0/6c08000.mm.tpdm/tpdm0
We can use the commands are similar to the below to validate TPDMs.
Enable coresight sink first. The port of tpda which is connected to
the tpdm will be enabled after commands below.
echo 1 > /sys/bus/coresight/devices/tmc_etf0/enable_sink
echo 1 > /sys/bus/coresight/devices/tpdm0/enable_source
echo 1 > /sys/bus/coresight/devices/tpdm0/integration_test
echo 2 > /sys/bus/coresight/devices/tpdm0/integration_test
The test data will be collected in the coresight sink which is enabled.
If rwp register of the sink is keeping updating when do
integration_test (by cat tmc_etf0/mgmt/rwp), it means there is data
generated from TPDM to sink.
There must be a tpda between tpdm and the sink. When there are some
other trace event hw components in the same HW block with tpdm, tpdm
and these hw components will connect to the coresight funnel. When
there is only tpdm trace hw in the HW block, tpdm will connect to
tpda directly.
@@ -0,0 +1,45 @@
.. SPDX-License-Identifier: GPL-2.0
==========================================================
Trace performance monitoring and diagnostics monitor(TPDM)
==========================================================
:Author: Jinlong Mao <quic_jinlmao@quicinc.com>
:Date: January 2023
Hardware Description
--------------------
TPDM - The trace performance monitoring and diagnostics monitor or TPDM in
short serves as data collection component for various dataset types.
The primary use case of the TPDM is to collect data from different data
sources and send it to a TPDA for packetization, timestamping and funneling.
Sysfs files and directories
---------------------------
Root: ``/sys/bus/coresight/devices/tpdm<N>``
----
:File: ``enable_source`` (RW)
:Notes:
- > 0 : enable the datasets of TPDM.
- = 0 : disable the datasets of TPDM.
:Syntax:
``echo 1 > enable_source``
----
:File: ``integration_test`` (wo)
:Notes:
Integration test will generate test data for tpdm.
:Syntax:
``echo value > integration_test``
value - 1 or 2.
----
.. This text is intentionally added to make Sphinx happy.
@@ -0,0 +1,38 @@
.. SPDX-License-Identifier: GPL-2.0
==============================
Trace Buffer Extension (TRBE).
==============================
:Author: Anshuman Khandual <anshuman.khandual@arm.com>
:Date: November 2020
Hardware Description
--------------------
Trace Buffer Extension (TRBE) is a percpu hardware which captures in system
memory, CPU traces generated from a corresponding percpu tracing unit. This
gets plugged in as a coresight sink device because the corresponding trace
generators (ETE), are plugged in as source device.
The TRBE is not compliant to CoreSight architecture specifications, but is
driven via the CoreSight driver framework to support the ETE (which is
CoreSight compliant) integration.
Sysfs files and directories
---------------------------
The TRBE devices appear on the existing coresight bus alongside the other
coresight devices::
>$ ls /sys/bus/coresight/devices
trbe0 trbe1 trbe2 trbe3
The ``trbe<N>`` named TRBEs are associated with a CPU.::
>$ ls /sys/bus/coresight/devices/trbe0/
align flag
*Key file items are:-*
* ``align``: TRBE write pointer alignment
* ``flag``: TRBE updates memory with access and dirty flags
@@ -0,0 +1,698 @@
======================================
Coresight - HW Assisted Tracing on ARM
======================================
:Author: Mathieu Poirier <mathieu.poirier@linaro.org>
:Date: September 11th, 2014
Introduction
------------
Coresight is an umbrella of technologies allowing for the debugging of ARM
based SoC. It includes solutions for JTAG and HW assisted tracing. This
document is concerned with the latter.
HW assisted tracing is becoming increasingly useful when dealing with systems
that have many SoCs and other components like GPU and DMA engines. ARM has
developed a HW assisted tracing solution by means of different components, each
being added to a design at synthesis time to cater to specific tracing needs.
Components are generally categorised as source, link and sinks and are
(usually) discovered using the AMBA bus.
"Sources" generate a compressed stream representing the processor instruction
path based on tracing scenarios as configured by users. From there the stream
flows through the coresight system (via ATB bus) using links that are connecting
the emanating source to a sink(s). Sinks serve as endpoints to the coresight
implementation, either storing the compressed stream in a memory buffer or
creating an interface to the outside world where data can be transferred to a
host without fear of filling up the onboard coresight memory buffer.
At typical coresight system would look like this::
*****************************************************************
**************************** AMBA AXI ****************************===||
***************************************************************** ||
^ ^ | ||
| | * **
0000000 ::::: 0000000 ::::: ::::: @@@@@@@ ||||||||||||
0 CPU 0<-->: C : 0 CPU 0<-->: C : : C : @ STM @ || System ||
|->0000000 : T : |->0000000 : T : : T :<--->@@@@@ || Memory ||
| #######<-->: I : | #######<-->: I : : I : @@@<-| ||||||||||||
| # ETM # ::::: | # PTM # ::::: ::::: @ |
| ##### ^ ^ | ##### ^ ! ^ ! . | |||||||||
| |->### | ! | |->### | ! | ! . | || DAP ||
| | # | ! | | # | ! | ! . | |||||||||
| | . | ! | | . | ! | ! . | | |
| | . | ! | | . | ! | ! . | | *
| | . | ! | | . | ! | ! . | | SWD/
| | . | ! | | . | ! | ! . | | JTAG
*****************************************************************<-|
*************************** AMBA Debug APB ************************
*****************************************************************
| . ! . ! ! . |
| . * . * * . |
*****************************************************************
******************** Cross Trigger Matrix (CTM) *******************
*****************************************************************
| . ^ . . |
| * ! * * |
*****************************************************************
****************** AMBA Advanced Trace Bus (ATB) ******************
*****************************************************************
| ! =============== |
| * ===== F =====<---------|
| ::::::::: ==== U ====
|-->:: CTI ::<!! === N ===
| ::::::::: ! == N ==
| ^ * == E ==
| ! &&&&&&&&& IIIIIII == L ==
|------>&& ETB &&<......II I =======
| ! &&&&&&&&& II I .
| ! I I .
| ! I REP I<..........
| ! I I
| !!>&&&&&&&&& II I *Source: ARM ltd.
|------>& TPIU &<......II I DAP = Debug Access Port
&&&&&&&&& IIIIIII ETM = Embedded Trace Macrocell
; PTM = Program Trace Macrocell
; CTI = Cross Trigger Interface
* ETB = Embedded Trace Buffer
To trace port TPIU= Trace Port Interface Unit
SWD = Serial Wire Debug
While on target configuration of the components is done via the APB bus,
all trace data are carried out-of-band on the ATB bus. The CTM provides
a way to aggregate and distribute signals between CoreSight components.
The coresight framework provides a central point to represent, configure and
manage coresight devices on a platform. This first implementation centers on
the basic tracing functionality, enabling components such ETM/PTM, funnel,
replicator, TMC, TPIU and ETB. Future work will enable more
intricate IP blocks such as STM and CTI.
Acronyms and Classification
---------------------------
Acronyms:
PTM:
Program Trace Macrocell
ETM:
Embedded Trace Macrocell
STM:
System trace Macrocell
ETB:
Embedded Trace Buffer
ITM:
Instrumentation Trace Macrocell
TPIU:
Trace Port Interface Unit
TMC-ETR:
Trace Memory Controller, configured as Embedded Trace Router
TMC-ETF:
Trace Memory Controller, configured as Embedded Trace FIFO
CTI:
Cross Trigger Interface
Classification:
Source:
ETMv3.x ETMv4, PTMv1.0, PTMv1.1, STM, STM500, ITM
Link:
Funnel, replicator (intelligent or not), TMC-ETR
Sinks:
ETBv1.0, ETB1.1, TPIU, TMC-ETF
Misc:
CTI
Device Tree Bindings
--------------------
See ``Documentation/devicetree/bindings/arm/arm,coresight-*.yaml`` for details.
As of this writing drivers for ITM, STMs and CTIs are not provided but are
expected to be added as the solution matures.
Framework and implementation
----------------------------
The coresight framework provides a central point to represent, configure and
manage coresight devices on a platform. Any coresight compliant device can
register with the framework for as long as they use the right APIs:
.. c:function:: struct coresight_device *coresight_register(struct coresight_desc *desc);
.. c:function:: void coresight_unregister(struct coresight_device *csdev);
The registering function is taking a ``struct coresight_desc *desc`` and
register the device with the core framework. The unregister function takes
a reference to a ``struct coresight_device *csdev`` obtained at registration time.
If everything goes well during the registration process the new devices will
show up under /sys/bus/coresight/devices, as showns here for a TC2 platform::
root:~# ls /sys/bus/coresight/devices/
replicator 20030000.tpiu 2201c000.ptm 2203c000.etm 2203e000.etm
20010000.etb 20040000.funnel 2201d000.ptm 2203d000.etm
root:~#
The functions take a ``struct coresight_device``, which looks like this::
struct coresight_desc {
enum coresight_dev_type type;
struct coresight_dev_subtype subtype;
const struct coresight_ops *ops;
struct coresight_platform_data *pdata;
struct device *dev;
const struct attribute_group **groups;
};
The "coresight_dev_type" identifies what the device is, i.e, source link or
sink while the "coresight_dev_subtype" will characterise that type further.
The ``struct coresight_ops`` is mandatory and will tell the framework how to
perform base operations related to the components, each component having
a different set of requirement. For that ``struct coresight_ops_sink``,
``struct coresight_ops_link`` and ``struct coresight_ops_source`` have been
provided.
The next field ``struct coresight_platform_data *pdata`` is acquired by calling
``of_get_coresight_platform_data()``, as part of the driver's _probe routine and
``struct device *dev`` gets the device reference embedded in the ``amba_device``::
static int etm_probe(struct amba_device *adev, const struct amba_id *id)
{
...
...
drvdata->dev = &adev->dev;
...
}
Specific class of device (source, link, or sink) have generic operations
that can be performed on them (see ``struct coresight_ops``). The ``**groups``
is a list of sysfs entries pertaining to operations
specific to that component only. "Implementation defined" customisations are
expected to be accessed and controlled using those entries.
Device Naming scheme
--------------------
The devices that appear on the "coresight" bus were named the same as their
parent devices, i.e, the real devices that appears on AMBA bus or the platform bus.
Thus the names were based on the Linux Open Firmware layer naming convention,
which follows the base physical address of the device followed by the device
type. e.g::
root:~# ls /sys/bus/coresight/devices/
20010000.etf 20040000.funnel 20100000.stm 22040000.etm
22140000.etm 230c0000.funnel 23240000.etm 20030000.tpiu
20070000.etr 20120000.replicator 220c0000.funnel
23040000.etm 23140000.etm 23340000.etm
However, with the introduction of ACPI support, the names of the real
devices are a bit cryptic and non-obvious. Thus, a new naming scheme was
introduced to use more generic names based on the type of the device. The
following rules apply::
1) Devices that are bound to CPUs, are named based on the CPU logical
number.
e.g, ETM bound to CPU0 is named "etm0"
2) All other devices follow a pattern, "<device_type_prefix>N", where :
<device_type_prefix> - A prefix specific to the type of the device
N - a sequential number assigned based on the order
of probing.
e.g, tmc_etf0, tmc_etr0, funnel0, funnel1
Thus, with the new scheme the devices could appear as ::
root:~# ls /sys/bus/coresight/devices/
etm0 etm1 etm2 etm3 etm4 etm5 funnel0
funnel1 funnel2 replicator0 stm0 tmc_etf0 tmc_etr0 tpiu0
Some of the examples below might refer to old naming scheme and some
to the newer scheme, to give a confirmation that what you see on your
system is not unexpected. One must use the "names" as they appear on
the system under specified locations.
Topology Representation
-----------------------
Each CoreSight component has a ``connections`` directory which will contain
links to other CoreSight components. This allows the user to explore the trace
topology and for larger systems, determine the most appropriate sink for a
given source. The connection information can also be used to establish
which CTI devices are connected to a given component. This directory contains a
``nr_links`` attribute detailing the number of links in the directory.
For an ETM source, in this case ``etm0`` on a Juno platform, a typical
arrangement will be::
linaro-developer:~# ls - l /sys/bus/coresight/devices/etm0/connections
<file details> cti_cpu0 -> ../../../23020000.cti/cti_cpu0
<file details> nr_links
<file details> out:0 -> ../../../230c0000.funnel/funnel2
Following the out port to ``funnel2``::
linaro-developer:~# ls -l /sys/bus/coresight/devices/funnel2/connections
<file details> in:0 -> ../../../23040000.etm/etm0
<file details> in:1 -> ../../../23140000.etm/etm3
<file details> in:2 -> ../../../23240000.etm/etm4
<file details> in:3 -> ../../../23340000.etm/etm5
<file details> nr_links
<file details> out:0 -> ../../../20040000.funnel/funnel0
And again to ``funnel0``::
linaro-developer:~# ls -l /sys/bus/coresight/devices/funnel0/connections
<file details> in:0 -> ../../../220c0000.funnel/funnel1
<file details> in:1 -> ../../../230c0000.funnel/funnel2
<file details> nr_links
<file details> out:0 -> ../../../20010000.etf/tmc_etf0
Finding the first sink ``tmc_etf0``. This can be used to collect data
as a sink, or as a link to propagate further along the chain::
linaro-developer:~# ls -l /sys/bus/coresight/devices/tmc_etf0/connections
<file details> cti_sys0 -> ../../../20020000.cti/cti_sys0
<file details> in:0 -> ../../../20040000.funnel/funnel0
<file details> nr_links
<file details> out:0 -> ../../../20150000.funnel/funnel4
via ``funnel4``::
linaro-developer:~# ls -l /sys/bus/coresight/devices/funnel4/connections
<file details> in:0 -> ../../../20010000.etf/tmc_etf0
<file details> in:1 -> ../../../20140000.etf/tmc_etf1
<file details> nr_links
<file details> out:0 -> ../../../20120000.replicator/replicator0
and a ``replicator0``::
linaro-developer:~# ls -l /sys/bus/coresight/devices/replicator0/connections
<file details> in:0 -> ../../../20150000.funnel/funnel4
<file details> nr_links
<file details> out:0 -> ../../../20030000.tpiu/tpiu0
<file details> out:1 -> ../../../20070000.etr/tmc_etr0
Arriving at the final sink in the chain, ``tmc_etr0``::
linaro-developer:~# ls -l /sys/bus/coresight/devices/tmc_etr0/connections
<file details> cti_sys0 -> ../../../20020000.cti/cti_sys0
<file details> in:0 -> ../../../20120000.replicator/replicator0
<file details> nr_links
As described below, when using sysfs it is sufficient to enable a sink and
a source for successful trace. The framework will correctly enable all
intermediate links as required.
Note: ``cti_sys0`` appears in two of the connections lists above.
CTIs can connect to multiple devices and are arranged in a star topology
via the CTM. See (Documentation/trace/coresight/coresight-ect.rst)
[#fourth]_ for further details.
Looking at this device we see 4 connections::
linaro-developer:~# ls -l /sys/bus/coresight/devices/cti_sys0/connections
<file details> nr_links
<file details> stm0 -> ../../../20100000.stm/stm0
<file details> tmc_etf0 -> ../../../20010000.etf/tmc_etf0
<file details> tmc_etr0 -> ../../../20070000.etr/tmc_etr0
<file details> tpiu0 -> ../../../20030000.tpiu/tpiu0
How to use the tracer modules
-----------------------------
There are two ways to use the Coresight framework:
1. using the perf cmd line tools.
2. interacting directly with the Coresight devices using the sysFS interface.
Preference is given to the former as using the sysFS interface
requires a deep understanding of the Coresight HW. The following sections
provide details on using both methods.
Using the sysFS interface
~~~~~~~~~~~~~~~~~~~~~~~~~
Before trace collection can start, a coresight sink needs to be identified.
There is no limit on the amount of sinks (nor sources) that can be enabled at
any given moment. As a generic operation, all device pertaining to the sink
class will have an "active" entry in sysfs::
root:/sys/bus/coresight/devices# ls
replicator 20030000.tpiu 2201c000.ptm 2203c000.etm 2203e000.etm
20010000.etb 20040000.funnel 2201d000.ptm 2203d000.etm
root:/sys/bus/coresight/devices# ls 20010000.etb
enable_sink status trigger_cntr
root:/sys/bus/coresight/devices# echo 1 > 20010000.etb/enable_sink
root:/sys/bus/coresight/devices# cat 20010000.etb/enable_sink
1
root:/sys/bus/coresight/devices#
At boot time the current etm3x driver will configure the first address
comparator with "_stext" and "_etext", essentially tracing any instruction
that falls within that range. As such "enabling" a source will immediately
trigger a trace capture::
root:/sys/bus/coresight/devices# echo 1 > 2201c000.ptm/enable_source
root:/sys/bus/coresight/devices# cat 2201c000.ptm/enable_source
1
root:/sys/bus/coresight/devices# cat 20010000.etb/status
Depth: 0x2000
Status: 0x1
RAM read ptr: 0x0
RAM wrt ptr: 0x19d3 <----- The write pointer is moving
Trigger cnt: 0x0
Control: 0x1
Flush status: 0x0
Flush ctrl: 0x2001
root:/sys/bus/coresight/devices#
Trace collection is stopped the same way::
root:/sys/bus/coresight/devices# echo 0 > 2201c000.ptm/enable_source
root:/sys/bus/coresight/devices#
The content of the ETB buffer can be harvested directly from /dev::
root:/sys/bus/coresight/devices# dd if=/dev/20010000.etb \
of=~/cstrace.bin
64+0 records in
64+0 records out
32768 bytes (33 kB) copied, 0.00125258 s, 26.2 MB/s
root:/sys/bus/coresight/devices#
The file cstrace.bin can be decompressed using "ptm2human", DS-5 or Trace32.
Following is a DS-5 output of an experimental loop that increments a variable up
to a certain value. The example is simple and yet provides a glimpse of the
wealth of possibilities that coresight provides.
::
Info Tracing enabled
Instruction 106378866 0x8026B53C E52DE004 false PUSH {lr}
Instruction 0 0x8026B540 E24DD00C false SUB sp,sp,#0xc
Instruction 0 0x8026B544 E3A03000 false MOV r3,#0
Instruction 0 0x8026B548 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B54C E59D3004 false LDR r3,[sp,#4]
Instruction 0 0x8026B550 E3530004 false CMP r3,#4
Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1
Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c
Timestamp Timestamp: 17106715833
Instruction 319 0x8026B54C E59D3004 false LDR r3,[sp,#4]
Instruction 0 0x8026B550 E3530004 false CMP r3,#4
Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1
Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c
Instruction 9 0x8026B54C E59D3004 false LDR r3,[sp,#4]
Instruction 0 0x8026B550 E3530004 false CMP r3,#4
Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1
Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c
Instruction 7 0x8026B54C E59D3004 false LDR r3,[sp,#4]
Instruction 0 0x8026B550 E3530004 false CMP r3,#4
Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1
Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c
Instruction 7 0x8026B54C E59D3004 false LDR r3,[sp,#4]
Instruction 0 0x8026B550 E3530004 false CMP r3,#4
Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1
Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c
Instruction 10 0x8026B54C E59D3004 false LDR r3,[sp,#4]
Instruction 0 0x8026B550 E3530004 false CMP r3,#4
Instruction 0 0x8026B554 E2833001 false ADD r3,r3,#1
Instruction 0 0x8026B558 E58D3004 false STR r3,[sp,#4]
Instruction 0 0x8026B55C DAFFFFFA true BLE {pc}-0x10 ; 0x8026b54c
Instruction 6 0x8026B560 EE1D3F30 false MRC p15,#0x0,r3,c13,c0,#1
Instruction 0 0x8026B564 E1A0100D false MOV r1,sp
Instruction 0 0x8026B568 E3C12D7F false BIC r2,r1,#0x1fc0
Instruction 0 0x8026B56C E3C2203F false BIC r2,r2,#0x3f
Instruction 0 0x8026B570 E59D1004 false LDR r1,[sp,#4]
Instruction 0 0x8026B574 E59F0010 false LDR r0,[pc,#16] ; [0x8026B58C] = 0x80550368
Instruction 0 0x8026B578 E592200C false LDR r2,[r2,#0xc]
Instruction 0 0x8026B57C E59221D0 false LDR r2,[r2,#0x1d0]
Instruction 0 0x8026B580 EB07A4CF true BL {pc}+0x1e9344 ; 0x804548c4
Info Tracing enabled
Instruction 13570831 0x8026B584 E28DD00C false ADD sp,sp,#0xc
Instruction 0 0x8026B588 E8BD8000 true LDM sp!,{pc}
Timestamp Timestamp: 17107041535
Using perf framework
~~~~~~~~~~~~~~~~~~~~
Coresight tracers are represented using the Perf framework's Performance
Monitoring Unit (PMU) abstraction. As such the perf framework takes charge of
controlling when tracing gets enabled based on when the process of interest is
scheduled. When configured in a system, Coresight PMUs will be listed when
queried by the perf command line tool:
linaro@linaro-nano:~$ ./perf list pmu
List of pre-defined events (to be used in -e):
cs_etm// [Kernel PMU event]
Regardless of the number of tracers available in a system (usually equal to the
amount of processor cores), the "cs_etm" PMU will be listed only once.
A Coresight PMU works the same way as any other PMU, i.e the name of the PMU is
provided along with configuration options within forward slashes '/' (see
`Config option formats`_).
Advanced Perf framework usage
-----------------------------
Sink selection
~~~~~~~~~~~~~~
An appropriate sink will be selected automatically for use with Perf, but since
there will typically be more than one sink, the name of the sink to use may be
specified as a special config option prefixed with '@'.
The available sinks are listed in sysFS under
($SYSFS)/bus/event_source/devices/cs_etm/sinks/::
root@localhost:/sys/bus/event_source/devices/cs_etm/sinks# ls
tmc_etf0 tmc_etr0 tpiu0
root@linaro-nano:~# perf record -e cs_etm/@tmc_etr0/u --per-thread program
More information on the above and other example on how to use Coresight with
the perf tools can be found in the "HOWTO.md" file of the openCSD gitHub
repository [#third]_.
AutoFDO analysis using the perf tools
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
perf can be used to record and analyze trace of programs.
Execution can be recorded using 'perf record' with the cs_etm event,
specifying the name of the sink to record to, e.g::
perf record -e cs_etm//u --per-thread
The 'perf report' and 'perf script' commands can be used to analyze execution,
synthesizing instruction and branch events from the instruction trace.
'perf inject' can be used to replace the trace data with the synthesized events.
The --itrace option controls the type and frequency of synthesized events
(see perf documentation).
Note that only 64-bit programs are currently supported - further work is
required to support instruction decode of 32-bit Arm programs.
Tracing PID
~~~~~~~~~~~
The kernel can be built to write the PID value into the PE ContextID registers.
For a kernel running at EL1, the PID is stored in CONTEXTIDR_EL1. A PE may
implement Arm Virtualization Host Extensions (VHE), which the kernel can
run at EL2 as a virtualisation host; in this case, the PID value is stored in
CONTEXTIDR_EL2.
perf provides PMU formats that program the ETM to insert these values into the
trace data; the PMU formats are defined as below:
"contextid1": Available on both EL1 kernel and EL2 kernel. When the
kernel is running at EL1, "contextid1" enables the PID
tracing; when the kernel is running at EL2, this enables
tracing the PID of guest applications.
"contextid2": Only usable when the kernel is running at EL2. When
selected, enables PID tracing on EL2 kernel.
"contextid": Will be an alias for the option that enables PID
tracing. I.e,
contextid == contextid1, on EL1 kernel.
contextid == contextid2, on EL2 kernel.
perf will always enable PID tracing at the relevant EL, this is accomplished by
automatically enable the "contextid" config - but for EL2 it is possible to make
specific adjustments using configs "contextid1" and "contextid2", E.g. if a user
wants to trace PIDs for both host and guest, the two configs "contextid1" and
"contextid2" can be set at the same time:
perf record -e cs_etm/contextid1,contextid2/u -- vm
Generating coverage files for Feedback Directed Optimization: AutoFDO
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
'perf inject' accepts the --itrace option in which case tracing data is
removed and replaced with the synthesized events. e.g.
::
perf inject --itrace --strip -i perf.data -o perf.data.new
Below is an example of using ARM ETM for autoFDO. It requires autofdo
(https://github.com/google/autofdo) and gcc version 5. The bubble
sort example is from the AutoFDO tutorial (https://gcc.gnu.org/wiki/AutoFDO/Tutorial).
::
$ gcc-5 -O3 sort.c -o sort
$ taskset -c 2 ./sort
Bubble sorting array of 30000 elements
5910 ms
$ perf record -e cs_etm//u --per-thread taskset -c 2 ./sort
Bubble sorting array of 30000 elements
12543 ms
[ perf record: Woken up 35 times to write data ]
[ perf record: Captured and wrote 69.640 MB perf.data ]
$ perf inject -i perf.data -o inj.data --itrace=il64 --strip
$ create_gcov --binary=./sort --profile=inj.data --gcov=sort.gcov -gcov_version=1
$ gcc-5 -O3 -fauto-profile=sort.gcov sort.c -o sort_autofdo
$ taskset -c 2 ./sort_autofdo
Bubble sorting array of 30000 elements
5806 ms
Config option formats
~~~~~~~~~~~~~~~~~~~~~
The following strings can be provided between // on the perf command line to enable various options.
They are also listed in the folder /sys/bus/event_source/devices/cs_etm/format/
.. list-table::
:header-rows: 1
* - Option
- Description
* - branch_broadcast
- Session local version of the system wide setting:
:ref:`ETM_MODE_BB <coresight-branch-broadcast>`
* - contextid
- See `Tracing PID`_
* - contextid1
- See `Tracing PID`_
* - contextid2
- See `Tracing PID`_
* - configid
- Selection for a custom configuration. This is an implementation detail and not used directly,
see :ref:`trace/coresight/coresight-config:Using Configurations in perf`
* - preset
- Override for parameters in a custom configuration, see
:ref:`trace/coresight/coresight-config:Using Configurations in perf`
* - sinkid
- Hashed version of the string to select a sink, automatically set when using the @ notation.
This is an internal implementation detail and is not used directly, see `Using perf
framework`_.
* - cycacc
- Session local version of the system wide setting: :ref:`ETMv4_MODE_CYCACC
<coresight-cycle-accurate>`
* - retstack
- Session local version of the system wide setting: :ref:`ETM_MODE_RETURNSTACK
<coresight-return-stack>`
* - timestamp
- Controls generation and interval of timestamps.
0 = off, 1 = minimum interval .. 15 = maximum interval.
Values 1 - 14 use a counter that decrements every cycle to generate a
timestamp on underflow. The reload value for the counter is 2 ^ (interval
- 1). If the value is 1 then the reload value is 1, if the value is 11
then the reload value is 1024 etc.
Setting the maximum interval (15) will disable the counter generated
timestamps, freeing the counter resource, leaving only ones emitted when
a SYNC packet is generated. The sync interval is controlled with
TRCSYNCPR.PERIOD which is every 4096 bytes of trace by default.
* - cc_threshold
- Cycle count threshold value. If nothing is provided here or the provided value is 0, then the
default value i.e 0x100 will be used. If provided value is less than minimum cycles threshold
value, as indicated via TRCIDR3.CCITMIN, then the minimum value will be used instead.
How to use the STM module
-------------------------
Using the System Trace Macrocell module is the same as the tracers - the only
difference is that clients are driving the trace capture rather
than the program flow through the code.
As with any other CoreSight component, specifics about the STM tracer can be
found in sysfs with more information on each entry being found in [#first]_::
root@genericarmv8:~# ls /sys/bus/coresight/devices/stm0
enable_source hwevent_select port_enable subsystem uevent
hwevent_enable mgmt port_select traceid
root@genericarmv8:~#
Like any other source a sink needs to be identified and the STM enabled before
being used::
root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/tmc_etf0/enable_sink
root@genericarmv8:~# echo 1 > /sys/bus/coresight/devices/stm0/enable_source
From there user space applications can request and use channels using the devfs
interface provided for that purpose by the generic STM API::
root@genericarmv8:~# ls -l /dev/stm0
crw------- 1 root root 10, 61 Jan 3 18:11 /dev/stm0
root@genericarmv8:~#
Details on how to use the generic STM API can be found here:
- Documentation/trace/stm.rst [#second]_.
The CTI & CTM Modules
---------------------
The CTI (Cross Trigger Interface) provides a set of trigger signals between
individual CTIs and components, and can propagate these between all CTIs via
channels on the CTM (Cross Trigger Matrix).
A separate documentation file is provided to explain the use of these devices.
(Documentation/trace/coresight/coresight-ect.rst) [#fourth]_.
CoreSight System Configuration
------------------------------
CoreSight components can be complex devices with many programming options.
Furthermore, components can be programmed to interact with each other across the
complete system.
A CoreSight System Configuration manager is provided to allow these complex programming
configurations to be selected and used easily from perf and sysfs.
See the separate document for further information.
(Documentation/trace/coresight/coresight-config.rst) [#fifth]_.
.. [#first] Documentation/ABI/testing/sysfs-bus-coresight-devices-stm
.. [#second] Documentation/trace/stm.rst
.. [#third] https://github.com/Linaro/perf-opencsd
.. [#fourth] Documentation/trace/coresight/coresight-ect.rst
.. [#fifth] Documentation/trace/coresight/coresight-config.rst
@@ -0,0 +1,9 @@
==============================
CoreSight - ARM Hardware Trace
==============================
.. toctree::
:maxdepth: 2
:glob:
*
@@ -0,0 +1,362 @@
===================================================
Using Coresight for Kernel panic and Watchdog reset
===================================================
Introduction
------------
This documentation is about using Linux coresight trace support to
debug kernel panic and watchdog reset scenarios.
Coresight trace during Kernel panic
-----------------------------------
From the coresight driver point of view, addressing the kernel panic
situation has four main requirements.
a. Support for allocation of trace buffer pages from reserved memory area.
Platform can advertise this using a new device tree property added to
relevant coresight nodes.
b. Support for stopping coresight blocks at the time of panic
c. Saving required metadata in the specified format
d. Support for reading trace data captured at the time of panic
Allocation of trace buffer pages from reserved RAM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A new optional device tree property "memory-region" is added to the
Coresight TMC device nodes, that would give the base address and size of trace
buffer.
Static allocation of trace buffers would ensure that both IOMMU enabled
and disabled cases are handled. Also, platforms that support persistent
RAM will allow users to read trace data in the subsequent boot without
booting the crashdump kernel.
Note:
For ETR sink devices, this reserved region will be used for both trace
capture and trace data retrieval.
For ETF sink devices, internal SRAM would be used for trace capture,
and they would be synced to reserved region for retrieval.
Disabling coresight blocks at the time of panic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In order to avoid the situation of losing relevant trace data after a
kernel panic, it would be desirable to stop the coresight blocks at the
time of panic.
This can be achieved by configuring the comparator, CTI and sink
devices as below::
Trigger on panic
Comparator --->External out --->CTI -->External In---->ETR/ETF stop
Saving metadata at the time of kernel panic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Coresight metadata involves all additional data that are required for a
successful trace decode in addition to the trace data. This involves
ETR/ETF/ETB register snapshot etc.
A new optional device property "memory-region" is added to
the ETR/ETF/ETB device nodes for this.
Reading trace data captured at the time of panic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Trace data captured at the time of panic, can be read from rebooted kernel
or from crashdump kernel using a special device file /dev/crash_tmc_xxx.
This device file is created only when there is a valid crashdata available.
General flow of trace capture and decode in case of kernel panic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. Enable source and sink on all the cores using the sysfs interface.
ETR sinks should have trace buffers allocated from reserved memory,
by selecting "resrv" buffer mode from sysfs.
2. Run relevant tests.
3. On a kernel panic, all coresight blocks are disabled, necessary
metadata is synced by kernel panic handler.
System would eventually reboot or boot a crashdump kernel.
4. For platforms that supports crashdump kernel, raw trace data can be
dumped using the coresight sysfs interface from the crashdump kernel
itself. Persistent RAM is not a requirement in this case.
5. For platforms that supports persistent RAM, trace data can be dumped
using the coresight sysfs interface in the subsequent Linux boot.
Crashdump kernel is not a requirement in this case. Persistent RAM
ensures that trace data is intact across reboot.
Coresight trace during Watchdog reset
-------------------------------------
The main difference between addressing the watchdog reset and kernel panic
case are below,
a. Saving coresight metadata need to be taken care by the
SCP(system control processor) firmware in the specified format,
instead of kernel.
b. Reserved memory region given by firmware for trace buffer and metadata
has to be in persistent RAM.
Note: This is a requirement for watchdog reset case but optional
in kernel panic case.
Watchdog reset can be supported only on platforms that meet the above
two requirements.
Sample commands for testing a Kernel panic case with ETR sink
-------------------------------------------------------------
1. Boot Linux kernel with "crash_kexec_post_notifiers" added to the kernel
bootargs. This is mandatory if the user would like to read the tracedata
from the crashdump kernel.
2. Enable the preloaded ETM configuration::
#echo 1 > /sys/kernel/config/cs-syscfg/configurations/panicstop/enable
3. Configure CTI using sysfs interface::
#./cti_setup.sh
#cat cti_setup.sh
cd /sys/bus/coresight/devices/
ap_cti_config () {
#ETM trig out[0] trigger to Channel 0
echo 0 4 > channels/trigin_attach
}
etf_cti_config () {
#ETF Flush in trigger from Channel 0
echo 0 1 > channels/trigout_attach
echo 1 > channels/trig_filter_enable
}
etr_cti_config () {
#ETR Flush in from Channel 0
echo 0 1 > channels/trigout_attach
echo 1 > channels/trig_filter_enable
}
ctidevs=`find . -name "cti*"`
for i in $ctidevs
do
cd $i
connection=`find . -name "ete*"`
if [ ! -z "$connection" ]
then
echo "AP CTI config for $i"
ap_cti_config
fi
connection=`find . -name "tmc_etf*"`
if [ ! -z "$connection" ]
then
echo "ETF CTI config for $i"
etf_cti_config
fi
connection=`find . -name "tmc_etr*"`
if [ ! -z "$connection" ]
then
echo "ETR CTI config for $i"
etr_cti_config
fi
cd ..
done
Note: CTI connections are SOC specific and hence the above script is
added just for reference.
4. Choose reserved buffer mode for ETR buffer::
#echo "resrv" > /sys/bus/coresight/devices/tmc_etr0/buf_mode_preferred
5. Enable stop on flush trigger configuration::
#echo 1 > /sys/bus/coresight/devices/tmc_etr0/stop_on_flush
6. Start Coresight tracing on cores 1 and 2 using sysfs interface
7. Run some application on core 1::
#taskset -c 1 dd if=/dev/urandom of=/dev/null &
8. Invoke kernel panic on core 2::
#echo 1 > /proc/sys/kernel/panic
#taskset -c 2 echo c > /proc/sysrq-trigger
9. From rebooted kernel or crashdump kernel, read crashdata::
#dd if=/dev/crash_tmc_etr0 of=/trace/cstrace.bin
10. Run opencsd decoder tools/scripts to generate the instruction trace.
Sample instruction trace dump
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Core1 dump::
A etm4_enable_hw: ffff800008ae1dd4
CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4
I etm4_enable_hw: ffff800008ae1dd4:
d503201f nop
I etm4_enable_hw: ffff800008ae1dd8:
d503201f nop
I etm4_enable_hw: ffff800008ae1ddc:
d503201f nop
I etm4_enable_hw: ffff800008ae1de0:
d503201f nop
I etm4_enable_hw: ffff800008ae1de4:
d503201f nop
I etm4_enable_hw: ffff800008ae1de8:
d503233f paciasp
I etm4_enable_hw: ffff800008ae1dec:
a9be7bfd stp x29, x30, [sp, #-32]!
I etm4_enable_hw: ffff800008ae1df0:
910003fd mov x29, sp
I etm4_enable_hw: ffff800008ae1df4:
a90153f3 stp x19, x20, [sp, #16]
I etm4_enable_hw: ffff800008ae1df8:
2a0003f4 mov w20, w0
I etm4_enable_hw: ffff800008ae1dfc:
900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48>
I etm4_enable_hw: ffff800008ae1e00:
910f4273 add x19, x19, #0x3d0
I etm4_enable_hw: ffff800008ae1e04:
f8747a60 ldr x0, [x19, x20, lsl #3]
E etm4_enable_hw: ffff800008ae1e08:
b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50>
I 149.039572921 etm4_enable_hw: ffff800008ae1e30:
a94153f3 ldp x19, x20, [sp, #16]
I 149.039572921 etm4_enable_hw: ffff800008ae1e34:
52800000 mov w0, #0x0 // #0
I 149.039572921 etm4_enable_hw: ffff800008ae1e38:
a8c27bfd ldp x29, x30, [sp], #32
..snip
149.052324811 chacha_block_generic: ffff800008642d80:
9100a3e0 add x0,
I 149.052324811 chacha_block_generic: ffff800008642d84:
b86178a2 ldr w2, [x5, x1, lsl #2]
I 149.052324811 chacha_block_generic: ffff800008642d88:
8b010803 add x3, x0, x1, lsl #2
I 149.052324811 chacha_block_generic: ffff800008642d8c:
b85fc063 ldur w3, [x3, #-4]
I 149.052324811 chacha_block_generic: ffff800008642d90:
0b030042 add w2, w2, w3
I 149.052324811 chacha_block_generic: ffff800008642d94:
b8217882 str w2, [x4, x1, lsl #2]
I 149.052324811 chacha_block_generic: ffff800008642d98:
91000421 add x1, x1, #0x1
I 149.052324811 chacha_block_generic: ffff800008642d9c:
f100443f cmp x1, #0x11
Core 2 dump::
A etm4_enable_hw: ffff800008ae1dd4
CONTEXT EL2 etm4_enable_hw: ffff800008ae1dd4
I etm4_enable_hw: ffff800008ae1dd4:
d503201f nop
I etm4_enable_hw: ffff800008ae1dd8:
d503201f nop
I etm4_enable_hw: ffff800008ae1ddc:
d503201f nop
I etm4_enable_hw: ffff800008ae1de0:
d503201f nop
I etm4_enable_hw: ffff800008ae1de4:
d503201f nop
I etm4_enable_hw: ffff800008ae1de8:
d503233f paciasp
I etm4_enable_hw: ffff800008ae1dec:
a9be7bfd stp x29, x30, [sp, #-32]!
I etm4_enable_hw: ffff800008ae1df0:
910003fd mov x29, sp
I etm4_enable_hw: ffff800008ae1df4:
a90153f3 stp x19, x20, [sp, #16]
I etm4_enable_hw: ffff800008ae1df8:
2a0003f4 mov w20, w0
I etm4_enable_hw: ffff800008ae1dfc:
900085b3 adrp x19, ffff800009b95000 <reserved_mem+0xc48>
I etm4_enable_hw: ffff800008ae1e00:
910f4273 add x19, x19, #0x3d0
I etm4_enable_hw: ffff800008ae1e04:
f8747a60 ldr x0, [x19, x20, lsl #3]
E etm4_enable_hw: ffff800008ae1e08:
b4000140 cbz x0, ffff800008ae1e30 <etm4_starting_cpu+0x50>
I 149.046243445 etm4_enable_hw: ffff800008ae1e30:
a94153f3 ldp x19, x20, [sp, #16]
I 149.046243445 etm4_enable_hw: ffff800008ae1e34:
52800000 mov w0, #0x0 // #0
I 149.046243445 etm4_enable_hw: ffff800008ae1e38:
a8c27bfd ldp x29, x30, [sp], #32
I 149.046243445 etm4_enable_hw: ffff800008ae1e3c:
d50323bf autiasp
E 149.046243445 etm4_enable_hw: ffff800008ae1e40:
d65f03c0 ret
A ete_sysreg_write: ffff800008adfa18
..snip
I 149.05422547 panic: ffff800008096300:
a90363f7 stp x23, x24, [sp, #48]
I 149.05422547 panic: ffff800008096304:
6b00003f cmp w1, w0
I 149.05422547 panic: ffff800008096308:
3a411804 ccmn w0, #0x1, #0x4, ne // ne = any
N 149.05422547 panic: ffff80000809630c:
540001e0 b.eq ffff800008096348 <panic+0xe0> // b.none
I 149.05422547 panic: ffff800008096310:
f90023f9 str x25, [sp, #64]
E 149.05422547 panic: ffff800008096314:
97fe44ef bl ffff8000080276d0 <panic_smp_self_stop>
A panic: ffff80000809634c
I 149.05422547 panic: ffff80000809634c:
910102d5 add x21, x22, #0x40
I 149.05422547 panic: ffff800008096350:
52800020 mov w0, #0x1 // #1
E 149.05422547 panic: ffff800008096354:
94166b8b bl ffff800008631180 <bust_spinlocks>
N 149.054225518 bust_spinlocks: ffff800008631180:
340000c0 cbz w0, ffff800008631198 <bust_spinlocks+0x18>
I 149.054225518 bust_spinlocks: ffff800008631184:
f000a321 adrp x1, ffff800009a98000 <pbufs.0+0xbb8>
I 149.054225518 bust_spinlocks: ffff800008631188:
b9405c20 ldr w0, [x1, #92]
I 149.054225518 bust_spinlocks: ffff80000863118c:
11000400 add w0, w0, #0x1
I 149.054225518 bust_spinlocks: ffff800008631190:
b9005c20 str w0, [x1, #92]
E 149.054225518 bust_spinlocks: ffff800008631194:
d65f03c0 ret
A panic: ffff800008096358
Perf based testing
------------------
Starting perf session
~~~~~~~~~~~~~~~~~~~~~
ETF::
perf record -e cs_etm/panicstop,@tmc_etf1/ -C 1
perf record -e cs_etm/panicstop,@tmc_etf2/ -C 2
ETR::
perf record -e cs_etm/panicstop,@tmc_etr0/ -C 1,2
Reading trace data after panic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Same sysfs based method explained above can be used to retrieve and
decode the trace data after the reboot on kernel panic.
@@ -0,0 +1,83 @@
.. SPDX-License-Identifier: GPL-2.0
======================================
UltraSoc - HW Assisted Tracing on SoC
======================================
:Author: Qi Liu <liuqi115@huawei.com>
:Date: January 2023
Introduction
------------
UltraSoc SMB is a per SCCL (Super CPU Cluster) hardware. It provides a
way to buffer and store CPU trace messages in a region of shared system
memory. The device acts as a coresight sink device and the
corresponding trace generators (ETM) are attached as source devices.
Sysfs files and directories
---------------------------
The SMB devices appear on the existing coresight bus alongside other
devices::
$# ls /sys/bus/coresight/devices/
ultra_smb0 ultra_smb1 ultra_smb2 ultra_smb3
The ``ultra_smb<N>`` names SMB device associated with SCCL.::
$# ls /sys/bus/coresight/devices/ultra_smb0
enable_sink mgmt
$# ls /sys/bus/coresight/devices/ultra_smb0/mgmt
buf_size buf_status read_pos write_pos
Key file items are:
* ``read_pos``: Shows the value on the read pointer register.
* ``write_pos``: Shows the value on the write pointer register.
* ``buf_status``: Shows the value on the status register.
BIT(0) is zero value which means the buffer is empty.
* ``buf_size``: Shows the buffer size of each device.
Firmware Bindings
-----------------
The device is only supported with ACPI. Its binding describes device
identifier, resource information and graph structure.
The device is identified as ACPI HID "HISI03A1". Device resources are allocated
using the _CRS method. Each device must present two base address; the first one
is the configuration base address of the device, the second one is the 32-bit
base address of shared system memory.
Example::
Device(USMB) { \
Name(_HID, "HISI03A1") \
Name(_CRS, ResourceTemplate() { \
QWordMemory (ResourceConsumer, , MinFixed, MaxFixed, NonCacheable, \
ReadWrite, 0x0, 0x95100000, 0x951FFFFF, 0x0, 0x100000) \
QWordMemory (ResourceConsumer, , MinFixed, MaxFixed, Cacheable, \
ReadWrite, 0x0, 0x50000000, 0x53FFFFFF, 0x0, 0x4000000) \
}) \
Name(_DSD, Package() { \
ToUUID("ab02a46b-74c7-45a2-bd68-f7d344ef2153"), \
/* Use CoreSight Graph ACPI bindings to describe connections topology */
Package() { \
0, \
1, \
Package() { \
1, \
ToUUID("3ecbc8b6-1d0e-4fb3-8107-e627f805c6cd"), \
8, \
Package() {0x8, 0, \_SB.S00.SL11.CL28.F008, 0}, \
Package() {0x9, 0, \_SB.S00.SL11.CL29.F009, 0}, \
Package() {0xa, 0, \_SB.S00.SL11.CL2A.F010, 0}, \
Package() {0xb, 0, \_SB.S00.SL11.CL2B.F011, 0}, \
Package() {0xc, 0, \_SB.S00.SL11.CL2C.F012, 0}, \
Package() {0xd, 0, \_SB.S00.SL11.CL2D.F013, 0}, \
Package() {0xe, 0, \_SB.S00.SL11.CL2E.F014, 0}, \
Package() {0xf, 0, \_SB.S00.SL11.CL2F.F015, 0}, \
} \
} \
}) \
}