Resguardo inicial

This commit is contained in:
lastdragon
2026-09-06 12:53:21 -06:00
parent 662efe3c2a
commit ad2088d178
94853 changed files with 43200404 additions and 0 deletions
@@ -0,0 +1,26 @@
.. SPDX-License-Identifier: GPL-2.0
==============================
am65-cpsw-nuss devlink support
==============================
This document describes the devlink features implemented by the ``am65-cpsw-nuss``
device driver.
Parameters
==========
The ``am65-cpsw-nuss`` driver implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``switch_mode``
- Boolean
- runtime
- Enable switch mode
@@ -0,0 +1,84 @@
.. SPDX-License-Identifier: GPL-2.0
====================
bnxt devlink support
====================
This document describes the devlink features implemented by the ``bnxt``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
* - Name
- Mode
* - ``enable_sriov``
- Permanent
* - ``ignore_ari``
- Permanent
* - ``msix_vec_per_pf_max``
- Permanent
* - ``msix_vec_per_pf_min``
- Permanent
* - ``enable_remote_dev_reset``
- Runtime
* - ``enable_roce``
- Permanent
The ``bnxt`` driver also implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``gre_ver_check``
- Boolean
- Permanent
- Generic Routing Encapsulation (GRE) version check will be enabled in
the device. If disabled, the device will skip the version check for
incoming packets.
Info versions
=============
The ``bnxt_en`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``board.id``
- fixed
- Part number identifying the board design
* - ``asic.id``
- fixed
- ASIC design identifier
* - ``asic.rev``
- fixed
- ASIC design revision
* - ``fw.psid``
- stored, running
- Firmware parameter set version of the board
* - ``fw``
- stored, running
- Overall board firmware version
* - ``fw.mgmt``
- stored, running
- NIC hardware resource management firmware version
* - ``fw.mgmt.api``
- running
- Minimum firmware interface spec version supported between driver and firmware
* - ``fw.nsci``
- stored, running
- General platform management firmware version
* - ``fw.roce``
- stored, running
- RoCE management firmware version
@@ -0,0 +1,252 @@
.. SPDX-License-Identifier: GPL-2.0
=============
Devlink DPIPE
=============
Background
==========
While performing the hardware offloading process, much of the hardware
specifics cannot be presented. These details are useful for debugging, and
``devlink-dpipe`` provides a standardized way to provide visibility into the
offloading process.
For example, the routing longest prefix match (LPM) algorithm used by the
Linux kernel may differ from the hardware implementation. The pipeline debug
API (DPIPE) is aimed at providing the user visibility into the ASIC's
pipeline in a generic way.
The hardware offload process is expected to be done in a way that the user
should not be able to distinguish between the hardware vs. software
implementation. In this process, hardware specifics are neglected. In
reality those details can have lots of meaning and should be exposed in some
standard way.
This problem is made even more complex when one wishes to offload the
control path of the whole networking stack to a switch ASIC. Due to
differences in the hardware and software models some processes cannot be
represented correctly.
One example is the kernel's LPM algorithm which in many cases differs
greatly to the hardware implementation. The configuration API is the same,
but one cannot rely on the Forward Information Base (FIB) to look like the
Level Path Compression trie (LPC-trie) in hardware.
In many situations trying to analyze systems failure solely based on the
kernel's dump may not be enough. By combining this data with complementary
information about the underlying hardware, this debugging can be made
easier; additionally, the information can be useful when debugging
performance issues.
Overview
========
The ``devlink-dpipe`` interface closes this gap. The hardware's pipeline is
modeled as a graph of match/action tables. Each table represents a specific
hardware block. This model is not new, first being used by the P4 language.
Traditionally it has been used as an alternative model for hardware
configuration, but the ``devlink-dpipe`` interface uses it for visibility
purposes as a standard complementary tool. The system's view from
``devlink-dpipe`` should change according to the changes done by the
standard configuration tools.
For example, it’s quite common to implement Access Control Lists (ACL)
using Ternary Content Addressable Memory (TCAM). The TCAM memory can be
divided into TCAM regions. Complex TC filters can have multiple rules with
different priorities and different lookup keys. On the other hand hardware
TCAM regions have a predefined lookup key. Offloading the TC filter rules
using TCAM engine can result in multiple TCAM regions being interconnected
in a chain (which may affect the data path latency). In response to a new TC
filter new tables should be created describing those regions.
Model
=====
The ``DPIPE`` model introduces several objects:
* headers
* tables
* entries
A ``header`` describes packet formats and provides names for fields within
the packet. A ``table`` describes hardware blocks. An ``entry`` describes
the actual content of a specific table.
The hardware pipeline is not port specific, but rather describes the whole
ASIC. Thus it is tied to the top of the ``devlink`` infrastructure.
Drivers can register and unregister tables at run time, in order to support
dynamic behavior. This dynamic behavior is mandatory for describing hardware
blocks like TCAM regions which can be allocated and freed dynamically.
``devlink-dpipe`` generally is not intended for configuration. The exception
is hardware counting for a specific table.
The following commands are used to obtain the ``dpipe`` objects from
userspace:
* ``table_get``: Receive a table's description.
* ``headers_get``: Receive a device's supported headers.
* ``entries_get``: Receive a table's current entries.
* ``counters_set``: Enable or disable counters on a table.
Table
-----
The driver should implement the following operations for each table:
* ``matches_dump``: Dump the supported matches.
* ``actions_dump``: Dump the supported actions.
* ``entries_dump``: Dump the actual content of the table.
* ``counters_set_update``: Synchronize hardware with counters enabled or
disabled.
Header/Field
------------
In a similar way to P4 headers and fields are used to describe a table's
behavior. There is a slight difference between the standard protocol headers
and specific ASIC metadata. The protocol headers should be declared in the
``devlink`` core API. On the other hand ASIC meta data is driver specific
and should be defined in the driver. Additionally, each driver-specific
devlink documentation file should document the driver-specific ``dpipe``
headers it implements. The headers and fields are identified by enumeration.
In order to provide further visibility some ASIC metadata fields could be
mapped to kernel objects. For example, internal router interface indexes can
be directly mapped to the net device ifindex. FIB table indexes used by
different Virtual Routing and Forwarding (VRF) tables can be mapped to
internal routing table indexes.
Match
-----
Matches are kept primitive and close to hardware operation. Match types like
LPM are not supported due to the fact that this is exactly a process we wish
to describe in full detail. Example of matches:
* ``field_exact``: Exact match on a specific field.
* ``field_exact_mask``: Exact match on a specific field after masking.
* ``field_range``: Match on a specific range.
The id's of the header and the field should be specified in order to
identify the specific field. Furthermore, the header index should be
specified in order to distinguish multiple headers of the same type in a
packet (tunneling).
Action
------
Similar to match, the actions are kept primitive and close to hardware
operation. For example:
* ``field_modify``: Modify the field value.
* ``field_inc``: Increment the field value.
* ``push_header``: Add a header.
* ``pop_header``: Remove a header.
Entry
-----
Entries of a specific table can be dumped on demand. Each eentry is
identified with an index and its properties are described by a list of
match/action values and specific counter. By dumping the tables content the
interactions between tables can be resolved.
Abstraction Example
===================
The following is an example of the abstraction model of the L3 part of
Mellanox Spectrum ASIC. The blocks are described in the order they appear in
the pipeline. The table sizes in the following examples are not real
hardware sizes and are provided for demonstration purposes.
LPM
---
The LPM algorithm can be implemented as a list of hash tables. Each hash
table contains routes with the same prefix length. The root of the list is
/32, and in case of a miss the hardware will continue to the next hash
table. The depth of the search will affect the data path latency.
In case of a hit the entry contains information about the next stage of the
pipeline which resolves the MAC address. The next stage can be either local
host table for directly connected routes, or adjacency table for next-hops.
The ``meta.lpm_prefix`` field is used to connect two LPM tables.
.. code::
table lpm_prefix_16 {
size: 4096,
counters_enabled: true,
match: { meta.vr_id: exact,
ipv4.dst_addr: exact_mask,
ipv6.dst_addr: exact_mask,
meta.lpm_prefix: exact },
action: { meta.adj_index: set,
meta.adj_group_size: set,
meta.rif_port: set,
meta.lpm_prefix: set },
}
Local Host
----------
In the case of local routes the LPM lookup already resolves the egress
router interface (RIF), yet the exact MAC address is not known. The local
host table is a hash table combining the output interface id with
destination IP address as a key. The result is the MAC address.
.. code::
table local_host {
size: 4096,
counters_enabled: true,
match: { meta.rif_port: exact,
ipv4.dst_addr: exact},
action: { ethernet.daddr: set }
}
Adjacency
---------
In case of remote routes this table does the ECMP. The LPM lookup results in
ECMP group size and index that serves as a global offset into this table.
Concurrently a hash of the packet is generated. Based on the ECMP group size
and the packet's hash a local offset is generated. Multiple LPM entries can
point to the same adjacency group.
.. code::
table adjacency {
size: 4096,
counters_enabled: true,
match: { meta.adj_index: exact,
meta.adj_group_size: exact,
meta.packet_hash_index: exact },
action: { ethernet.daddr: set,
meta.erif: set }
}
ERIF
----
In case the egress RIF and destination MAC have been resolved by previous
tables this table does multiple operations like TTL decrease and MTU check.
Then the decision of forward/drop is taken and the port L3 statistics are
updated based on the packet's type (broadcast, unicast, multicast).
.. code::
table erif {
size: 800,
counters_enabled: true,
match: { meta.rif_port: exact,
meta.is_l3_unicast: exact,
meta.is_l3_broadcast: exact,
meta.is_l3_multicast, exact },
action: { meta.l3_drop: set,
meta.l3_forward: set }
}
@@ -0,0 +1,89 @@
.. SPDX-License-Identifier: GPL-2.0
==========================
Devlink E-Switch Attribute
==========================
Devlink E-Switch supports two modes of operation: legacy and switchdev.
Legacy mode operates based on traditional MAC/VLAN steering rules. Switching
decisions are made based on MAC addresses, VLANs, etc. There is limited ability
to offload switching rules to hardware.
On the other hand, switchdev mode allows for more advanced offloading
capabilities of the E-Switch to hardware. In switchdev mode, more switching
rules and logic can be offloaded to the hardware switch ASIC. It enables
representor netdevices that represent the slow path of virtual functions (VFs)
or scalable-functions (SFs) of the device. See more information about
:ref:`Documentation/networking/switchdev.rst <switchdev>` and
:ref:`Documentation/networking/representors.rst <representors>`.
In addition, the devlink E-Switch also comes with other attributes listed
in the following section.
Attributes Description
======================
The following is a list of E-Switch attributes.
.. list-table:: E-Switch attributes
:widths: 8 5 45
* - Name
- Type
- Description
* - ``mode``
- enum
- The mode of the device. The mode can be one of the following:
* ``legacy`` operates based on traditional MAC/VLAN steering
rules.
* ``switchdev`` allows for more advanced offloading capabilities of
the E-Switch to hardware.
* ``switchdev_inactive`` switchdev mode but starts inactive, doesn't allow traffic
until explicitly activated. This mode is useful for orchestrators that
want to prepare the device in switchdev mode but only activate it when
all configurations are done.
* - ``inline-mode``
- enum
- Some HWs need the VF driver to put part of the packet
headers on the TX descriptor so the e-switch can do proper
matching and steering. Support for both switchdev mode and legacy mode.
* ``none`` none.
* ``link`` L2 mode.
* ``network`` L3 mode.
* ``transport`` L4 mode.
* - ``encap-mode``
- enum
- The encapsulation mode of the device. Support for both switchdev mode
and legacy mode. The mode can be one of the following:
* ``none`` Disable encapsulation support.
* ``basic`` Enable encapsulation support.
Example Usage
=============
.. code:: shell
# enable switchdev mode
$ devlink dev eswitch set pci/0000:08:00.0 mode switchdev
# set inline-mode and encap-mode
$ devlink dev eswitch set pci/0000:08:00.0 inline-mode none encap-mode basic
# display devlink device eswitch attributes
$ devlink dev eswitch show pci/0000:08:00.0
pci/0000:08:00.0: mode switchdev inline-mode none encap-mode basic
# enable encap-mode with legacy mode
$ devlink dev eswitch set pci/0000:08:00.0 mode legacy inline-mode none encap-mode basic
# start switchdev mode in inactive state
$ devlink dev eswitch set pci/0000:08:00.0 mode switchdev_inactive
# setup switchdev configurations, representors, FDB entries, etc..
...
# activate switchdev mode to allow traffic
$ devlink dev eswitch set pci/0000:08:00.0 mode switchdev
@@ -0,0 +1,121 @@
.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
.. _devlink_flash:
=============
Devlink Flash
=============
The ``devlink-flash`` API allows updating device firmware. It replaces the
older ``ethtool-flash`` mechanism, and doesn't require taking any
networking locks in the kernel to perform the flash update. Example use::
$ devlink dev flash pci/0000:05:00.0 file flash-boot.bin
Note that the file name is a path relative to the firmware loading path
(usually ``/lib/firmware/``). Drivers may send status updates to inform
user space about the progress of the update operation.
Overwrite Mask
==============
The ``devlink-flash`` command allows optionally specifying a mask indicating
how the device should handle subsections of flash components when updating.
This mask indicates the set of sections which are allowed to be overwritten.
.. list-table:: List of overwrite mask bits
:widths: 5 95
* - Name
- Description
* - ``DEVLINK_FLASH_OVERWRITE_SETTINGS``
- Indicates that the device should overwrite settings in the components
being updated with the settings found in the provided image.
* - ``DEVLINK_FLASH_OVERWRITE_IDENTIFIERS``
- Indicates that the device should overwrite identifiers in the
components being updated with the identifiers found in the provided
image. This includes MAC addresses, serial IDs, and similar device
identifiers.
Multiple overwrite bits may be combined and requested together. If no bits
are provided, it is expected that the device only update firmware binaries
in the components being updated. Settings and identifiers are expected to be
preserved across the update. A device may not support every combination and
the driver for such a device must reject any combination which cannot be
faithfully implemented.
Firmware Loading
================
Devices which require firmware to operate usually store it in non-volatile
memory on the board, e.g. flash. Some devices store only basic firmware on
the board, and the driver loads the rest from disk during probing.
``devlink-info`` allows users to query firmware information (loaded
components and versions).
In other cases the device can both store the image on the board, load from
disk, or automatically flash a new image from disk. The ``fw_load_policy``
devlink parameter can be used to control this behavior
(:ref:`Documentation/networking/devlink/devlink-params.rst <devlink_params_generic>`).
On-disk firmware files are usually stored in ``/lib/firmware/``.
Firmware Version Management
===========================
Drivers are expected to implement ``devlink-flash`` and ``devlink-info``
functionality, which together allow for implementing vendor-independent
automated firmware update facilities.
``devlink-info`` exposes the ``driver`` name and three version groups
(``fixed``, ``running``, ``stored``).
The ``driver`` attribute and ``fixed`` group identify the specific device
design, e.g. for looking up applicable firmware updates. This is why
``serial_number`` is not part of the ``fixed`` versions (even though it
is fixed) - ``fixed`` versions should identify the design, not a single
device.
``running`` and ``stored`` firmware versions identify the firmware running
on the device, and firmware which will be activated after reboot or device
reset.
The firmware update agent is supposed to be able to follow this simple
algorithm to update firmware contents, regardless of the device vendor:
.. code-block:: sh
# Get unique HW design identifier
$hw_id = devlink-dev-info['fixed']
# Find out which FW flash we want to use for this NIC
$want_flash_vers = some-db-backed.lookup($hw_id, 'flash')
# Update flash if necessary
if $want_flash_vers != devlink-dev-info['stored']:
$file = some-db-backed.download($hw_id, 'flash')
devlink-dev-flash($file)
# Find out the expected overall firmware versions
$want_fw_vers = some-db-backed.lookup($hw_id, 'all')
# Update on-disk file if necessary
if $want_fw_vers != devlink-dev-info['running']:
$file = some-db-backed.download($hw_id, 'disk')
write($file, '/lib/firmware/')
# Try device reset, if available
if $want_fw_vers != devlink-dev-info['running']:
devlink-reset()
# Reboot, if reset wasn't enough
if $want_fw_vers != devlink-dev-info['running']:
reboot()
Note that each reference to ``devlink-dev-info`` in this pseudo-code
is expected to fetch up-to-date information from the kernel.
For the convenience of identifying firmware files some vendors add
``bundle_id`` information to the firmware versions. This meta-version covers
multiple per-component versions and can be used e.g. in firmware file names
(all component versions could get rather long.)
@@ -0,0 +1,142 @@
.. SPDX-License-Identifier: GPL-2.0
==============
Devlink Health
==============
Background
==========
The ``devlink`` health mechanism is targeted for Real Time Alerting, in
order to know when something bad happened to a PCI device.
* Provide alert debug information.
* Self healing.
* If problem needs vendor support, provide a way to gather all needed
debugging information.
Overview
========
The main idea is to unify and centralize driver health reports in the
generic ``devlink`` instance and allow the user to set different
attributes of the health reporting and recovery procedures.
The ``devlink`` health reporter:
Device driver creates a "health reporter" per each error/health type.
Error/Health type can be a known/generic (e.g. PCI error, fw error, rx/tx error)
or unknown (driver specific).
For each registered health reporter a driver can issue error/health reports
asynchronously. All health reports handling is done by ``devlink``.
Device driver can provide specific callbacks for each "health reporter", e.g.:
* Recovery procedures
* Diagnostics procedures
* Object dump procedures
Drivers also provide default values for generic reporter parameters when
creating a health reporter.
Different parts of the driver can register different types of health reporters
with different handlers.
Actions
=======
Once an error is reported, devlink health will perform the following actions:
* A log is being send to the kernel trace events buffer
* Health status and statistics are being updated for the reporter instance
* Object dump is being taken and saved at the reporter instance. This is
best effort and skipped when recovery is aborted, auto-dump is disabled,
no dump callback is registered, or a dump is already stored.
* Auto recovery attempt is being done. Depends on:
- Auto-recovery configuration
- Grace period (and burst period) vs. time passed since last recover
Devlink formatted message
=========================
To handle devlink health diagnose and health dump requests, devlink creates a
formatted message structure ``devlink_fmsg`` and send it to the driver's callback
to fill the data in using the devlink fmsg API.
Devlink fmsg is a mechanism to pass descriptors between drivers and devlink, in
json-like format. The API allows the driver to add nested attributes such as
object, object pair and value array, in addition to attributes such as name and
value.
Driver should use this API to fill the fmsg context in a format which will be
translated by the devlink to the netlink message later. When it needs to send
the data using SKBs to the netlink layer, it fragments the data between
different SKBs. In order to do this fragmentation, it uses virtual nests
attributes, to avoid actual nesting use which cannot be divided between
different SKBs.
User Interface
==============
User can access/change each reporter's parameters and driver specific callbacks
via ``devlink``, e.g. per error type (per health reporter). Reporters may be
registered for the whole devlink instance or for a specific devlink port.
* Configure reporter's generic parameters (like: disable/enable auto recovery)
* Invoke recovery procedure
* Run diagnostics
* Object dump
.. list-table:: List of devlink health interfaces
:widths: 10 90
* - Name
- Description
* - ``DEVLINK_CMD_HEALTH_REPORTER_GET``
- Retrieves status and configuration info per DEV and reporter.
* - ``DEVLINK_CMD_HEALTH_REPORTER_SET``
- Allows reporter-related configuration setting.
* - ``DEVLINK_CMD_HEALTH_REPORTER_RECOVER``
- Triggers reporter's recovery procedure.
* - ``DEVLINK_CMD_HEALTH_REPORTER_TEST``
- Triggers a fake health event on the reporter. The effects of the test
event in terms of recovery flow should follow closely that of a real
event.
* - ``DEVLINK_CMD_HEALTH_REPORTER_DIAGNOSE``
- Retrieves current device state related to the reporter.
* - ``DEVLINK_CMD_HEALTH_REPORTER_DUMP_GET``
- Retrieves the last stored dump. Devlink health
saves a single dump. If an dump is not already stored by devlink
for this reporter, devlink generates a new dump.
Dump output is defined by the reporter.
* - ``DEVLINK_CMD_HEALTH_REPORTER_DUMP_CLEAR``
- Clears the last saved dump file for the specified reporter.
The following diagram provides a general overview of ``devlink-health``::
netlink
+--------------------------+
| |
| + |
| | |
+--------------------------+
|request for ops
|(diagnose,
driver devlink |recover,
|dump)
+--------+ +--------------------------+
| | | reporter| |
| | | +---------v----------+ |
| | ops execution | | | |
| <----------------------------------+ | |
| | | | | |
| | | + ^------------------+ |
| | | | request for ops |
| | | | (recover, dump) |
| | | | |
| | | +-+------------------+ |
| | health report | | health handler | |
| +-------------------------------> | |
| | | +--------------------+ |
| | health reporter create | |
| +----------------------------> |
+--------+ +--------------------------+
@@ -0,0 +1,224 @@
.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
============
Devlink Info
============
The ``devlink-info`` mechanism enables device drivers to report device
(hardware and firmware) information in a standard, extensible fashion.
The original motivation for the ``devlink-info`` API was twofold:
- making it possible to automate device and firmware management in a fleet
of machines in a vendor-independent fashion (see also
:ref:`Documentation/networking/devlink/devlink-flash.rst <devlink_flash>`);
- name the per component FW versions (as opposed to the crowded ethtool
version string).
``devlink-info`` supports reporting multiple types of objects. Reporting driver
versions is generally discouraged - here, and via any other Linux API.
.. list-table:: List of top level info objects
:widths: 5 95
* - Name
- Description
* - ``driver``
- Name of the currently used device driver, also available through sysfs.
* - ``serial_number``
- Serial number of the device.
This is usually the serial number of the ASIC, also often available
in PCI config space of the device in the *Device Serial Number*
capability.
The serial number should be unique per physical device.
Sometimes the serial number of the device is only 48 bits long (the
length of the Ethernet MAC address), and since PCI DSN is 64 bits long
devices pad or encode additional information into the serial number.
One example is adding port ID or PCI interface ID in the extra two bytes.
Drivers should make sure to strip or normalize any such padding
or interface ID, and report only the part of the serial number
which uniquely identifies the hardware. In other words serial number
reported for two ports of the same device or on two hosts of
a multi-host device should be identical.
* - ``board.serial_number``
- Board serial number of the device.
This is usually the serial number of the board, often available in
PCI *Vital Product Data*.
* - ``fixed``
- Group for hardware identifiers, and versions of components
which are not field-updatable.
Versions in this section identify the device design. For example,
component identifiers or the board version reported in the PCI VPD.
Data in ``devlink-info`` should be broken into the smallest logical
components, e.g. PCI VPD may concatenate various information
to form the Part Number string, while in ``devlink-info`` all parts
should be reported as separate items.
This group must not contain any frequently changing identifiers,
such as serial numbers. See
:ref:`Documentation/networking/devlink/devlink-flash.rst <devlink_flash>`
to understand why.
* - ``running``
- Group for information about currently running software/firmware.
These versions often only update after a reboot, sometimes device reset.
* - ``stored``
- Group for software/firmware versions in device flash.
Stored values must update to reflect changes in the flash even
if reboot has not yet occurred. If device is not capable of updating
``stored`` versions when new software is flashed, it must not report
them.
Each version can be reported at most once in each version group. Firmware
components stored on the flash should feature in both the ``running`` and
``stored`` sections, if device is capable of reporting ``stored`` versions
(see :ref:`Documentation/networking/devlink/devlink-flash.rst <devlink_flash>`).
In case software/firmware components are loaded from the disk (e.g.
``/lib/firmware``) only the running version should be reported via
the kernel API.
Please note that any security versions reported via devlink are purely
informational. Devlink does not use a secure channel to communicate with
the device.
Generic Versions
================
It is expected that drivers use the following generic names for exporting
version information. If a generic name for a given component doesn't exist yet,
driver authors should consult existing driver-specific versions and attempt
reuse. As last resort, if a component is truly unique, using driver-specific
names is allowed, but these should be documented in the driver-specific file.
All versions should try to use the following terminology:
.. list-table:: List of common version suffixes
:widths: 10 90
* - Name
- Description
* - ``id``, ``revision``
- Identifiers of designs and revision, mostly used for hardware versions.
* - ``api``
- Version of API between components. API items are usually of limited
value to the user, and can be inferred from other versions by the vendor,
so adding API versions is generally discouraged as noise.
* - ``bundle_id``
- Identifier of a distribution package which was flashed onto the device.
This is an attribute of a firmware package which covers multiple versions
for ease of managing firmware images (see
:ref:`Documentation/networking/devlink/devlink-flash.rst <devlink_flash>`).
``bundle_id`` can appear in both ``running`` and ``stored`` versions,
but it must not be reported if any of the components covered by the
``bundle_id`` was changed and no longer matches the version from
the bundle.
board.id
--------
Unique identifier of the board design.
board.rev
---------
Board design revision.
asic.id
-------
ASIC design identifier.
asic.rev
--------
ASIC design revision/stepping.
board.manufacture
-----------------
An identifier of the company or the facility which produced the part.
board.part_number
-----------------
Part number of the board and its components.
fw
--
Overall firmware version, often representing the collection of
fw.mgmt, fw.app, etc.
fw.mgmt
-------
Control unit firmware version. This firmware is responsible for house
keeping tasks, PHY control etc. but not the packet-by-packet data path
operation.
fw.mgmt.api
-----------
Firmware interface specification version of the software interfaces between
driver and firmware.
fw.app
------
Data path microcode controlling high-speed packet processing.
fw.undi
-------
UNDI software, may include the UEFI driver, firmware or both.
fw.ncsi
-------
Version of the software responsible for supporting/handling the
Network Controller Sideband Interface.
fw.psid
-------
Unique identifier of the firmware parameter set. These are usually
parameters of a particular board, defined at manufacturing time.
fw.roce
-------
RoCE firmware version which is responsible for handling roce
management.
fw.bundle_id
------------
Unique identifier of the entire firmware bundle.
fw.bootloader
-------------
Version of the bootloader.
Future work
===========
The following extensions could be useful:
- on-disk firmware file names - drivers list the file names of firmware they
may need to load onto devices via the ``MODULE_FIRMWARE()`` macro. These,
however, are per module, rather than per device. It'd be useful to list
the names of firmware files the driver will try to load for a given device,
in order of priority.
@@ -0,0 +1,122 @@
.. SPDX-License-Identifier: GPL-2.0
=================
Devlink Line card
=================
Background
==========
The ``devlink-linecard`` mechanism is targeted for manipulation of
line cards that serve as a detachable PHY modules for modular switch
system. Following operations are provided:
* Get a list of supported line card types.
* Provision of a slot with specific line card type.
* Get and monitor of line card state and its change.
Line card according to the type may contain one or more gearboxes
to mux the lanes with certain speed to multiple ports with lanes
of different speed. Line card ensures N:M mapping between
the switch ASIC modules and physical front panel ports.
Overview
========
Each line card devlink object is created by device driver,
according to the physical line card slots available on the device.
Similar to splitter cable, where the device might have no way
of detection of the splitter cable geometry, the device
might not have a way to detect line card type. For that devices,
concept of provisioning is introduced. It allows the user to:
* Provision a line card slot with certain line card type
- Device driver would instruct the ASIC to prepare all
resources accordingly. The device driver would
create all instances, namely devlink port and netdevices
that reside on the line card, according to the line card type
* Manipulate of line card entities even without line card
being physically connected or powered-up
* Setup splitter cable on line card ports
- As on the ordinary ports, user may provision a splitter
cable of a certain type, without the need to
be physically connected to the port
* Configure devlink ports and netdevices
Netdevice carrier is decided as follows:
* Line card is not inserted or powered-down
- The carrier is always down
* Line card is inserted and powered up
- The carrier is decided as for ordinary port netdevice
Line card state
===============
The ``devlink-linecard`` mechanism supports the following line card states:
* ``unprovisioned``: Line card is not provisioned on the slot.
* ``unprovisioning``: Line card slot is currently being unprovisioned.
* ``provisioning``: Line card slot is currently in a process of being provisioned
with a line card type.
* ``provisioning_failed``: Provisioning was not successful.
* ``provisioned``: Line card slot is provisioned with a type.
* ``active``: Line card is powered-up and active.
The following diagram provides a general overview of ``devlink-linecard``
state transitions::
+-------------------------+
| |
+----------------------------------> unprovisioned |
| | |
| +--------|-------^--------+
| | |
| | |
| +--------v-------|--------+
| | |
| | provisioning |
| | |
| +------------|------------+
| |
| +-----------------------------+
| | |
| +------------v------------+ +------------v------------+ +-------------------------+
| | | | ----> |
+----- provisioning_failed | | provisioned | | active |
| | | | <---- |
| +------------^------------+ +------------|------------+ +-------------------------+
| | |
| | |
| | +------------v------------+
| | | |
| | | unprovisioning |
| | | |
| | +------------|------------+
| | |
| +-----------------------------+
| |
+-----------------------------------------------+
Example usage
=============
.. code:: shell
$ devlink lc show [ DEV [ lc LC_INDEX ] ]
$ devlink lc set DEV lc LC_INDEX [ { type LC_TYPE | notype } ]
# Show current line card configuration and status for all slots:
$ devlink lc
# Set slot 8 to be provisioned with type "16x100G":
$ devlink lc set pci/0000:01:00.0 lc 8 type 16x100G
# Set slot 8 to be unprovisioned:
$ devlink lc set pci/0000:01:00.0 lc 8 notype
@@ -0,0 +1,167 @@
.. SPDX-License-Identifier: GPL-2.0
==============
Devlink Params
==============
``devlink`` provides capability for a driver to expose device parameters for low
level device functionality. Since devlink can operate at the device-wide
level, it can be used to provide configuration that may affect multiple
ports on a single device.
This document describes a number of generic parameters that are supported
across multiple drivers. Each driver is also free to add their own
parameters. Each driver must document the specific parameters they support,
whether generic or not.
Configuration modes
===================
Parameters may be set in different configuration modes.
.. list-table:: Possible configuration modes
:widths: 5 90
* - Name
- Description
* - ``runtime``
- set while the driver is running, and takes effect immediately. No
reset is required.
* - ``driverinit``
- applied while the driver initializes. Requires the user to restart
the driver using the ``devlink`` reload command.
* - ``permanent``
- written to the device's non-volatile memory. A hard reset is required
for it to take effect.
Reloading
---------
In order for ``driverinit`` parameters to take effect, the driver must
support reloading via the ``devlink-reload`` command. This command will
request a reload of the device driver.
Default parameter values
=========================
Drivers may optionally export default values for parameters of cmode
``runtime`` and ``permanent``. For ``driverinit`` parameters, the last
value set by the driver will be used as the default value. Drivers can
also support resetting params with cmode ``runtime`` and ``permanent``
to their default values. Resetting ``driverinit`` params is supported
by devlink core without additional driver support needed.
.. _devlink_params_generic:
Generic configuration parameters
================================
The following is a list of generic configuration parameters that drivers may
add. Use of generic parameters is preferred over each driver creating their
own name.
.. list-table:: List of generic parameters
:widths: 5 5 90
* - Name
- Type
- Description
* - ``enable_sriov``
- Boolean
- Enable Single Root I/O Virtualization (SRIOV) in the device.
* - ``ignore_ari``
- Boolean
- Ignore Alternative Routing-ID Interpretation (ARI) capability. If
enabled, the adapter will ignore ARI capability even when the
platform has support enabled. The device will create the same number
of partitions as when the platform does not support ARI.
* - ``msix_vec_per_pf_max``
- u32
- Provides the maximum number of MSI-X interrupts that a device can
create. Value is the same across all physical functions (PFs) in the
device.
* - ``msix_vec_per_pf_min``
- u32
- Provides the minimum number of MSI-X interrupts required for the
device to initialize. Value is the same across all physical functions
(PFs) in the device.
* - ``fw_load_policy``
- u8
- Control the device's firmware loading policy.
- ``DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DRIVER`` (0)
Load firmware version preferred by the driver.
- ``DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_FLASH`` (1)
Load firmware currently stored in flash.
- ``DEVLINK_PARAM_FW_LOAD_POLICY_VALUE_DISK`` (2)
Load firmware currently available on host's disk.
* - ``reset_dev_on_drv_probe``
- u8
- Controls the device's reset policy on driver probe.
- ``DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_UNKNOWN`` (0)
Unknown or invalid value.
- ``DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_ALWAYS`` (1)
Always reset device on driver probe.
- ``DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_NEVER`` (2)
Never reset device on driver probe.
- ``DEVLINK_PARAM_RESET_DEV_ON_DRV_PROBE_VALUE_DISK`` (3)
Reset the device only if firmware can be found in the filesystem.
* - ``enable_roce``
- Boolean
- Enable handling of RoCE traffic in the device.
* - ``enable_eth``
- Boolean
- When enabled, the device driver will instantiate Ethernet specific
auxiliary device of the devlink device.
* - ``enable_rdma``
- Boolean
- When enabled, the device driver will instantiate RDMA specific
auxiliary device of the devlink device.
* - ``enable_vnet``
- Boolean
- When enabled, the device driver will instantiate VDPA networking
specific auxiliary device of the devlink device.
* - ``enable_iwarp``
- Boolean
- Enable handling of iWARP traffic in the device.
* - ``internal_error_reset``
- Boolean
- When enabled, the device driver will reset the device on internal
errors.
* - ``max_macs``
- u32
- Typically macvlan, vlan net devices mac are also programmed in their
parent netdevice's Function rx filter. This parameter limit the
maximum number of unicast mac address filters to receive traffic from
per ethernet port of this device.
* - ``region_snapshot_enable``
- Boolean
- Enable capture of ``devlink-region`` snapshots.
* - ``enable_remote_dev_reset``
- Boolean
- Enable device reset by remote host. When cleared, the device driver
will NACK any attempt of other host to reset the device. This parameter
is useful for setups where a device is shared by different hosts, such
as multi-host setup.
* - ``io_eq_size``
- u32
- Control the size of I/O completion EQs.
* - ``event_eq_size``
- u32
- Control the size of asynchronous control events EQ.
* - ``enable_phc``
- Boolean
- Enable PHC (PTP Hardware Clock) functionality in the device.
* - ``clock_id``
- u64
- Clock ID used by the device for registering DPLL devices and pins.
* - ``total_vfs``
- u32
- The max number of Virtual Functions (VFs) exposed by the PF.
after reboot/pci reset, 'sriov_totalvfs' entry under the device's sysfs
directory will report this value.
* - ``num_doorbells``
- u32
- Controls the number of doorbells used by the device.
* - ``max_mac_per_vf``
- u32
- Controls the maximum number of MAC address filters that can be assigned
to a Virtual Function (VF).
@@ -0,0 +1,487 @@
.. SPDX-License-Identifier: GPL-2.0
.. _devlink_port:
============
Devlink Port
============
``devlink-port`` is a port that exists on the device. It has a logically
separate ingress/egress point of the device. A devlink port can be any one
of many flavours. A devlink port flavour along with port attributes
describe what a port represents.
A device driver that intends to publish a devlink port sets the
devlink port attributes and registers the devlink port.
Devlink port flavours are described below.
.. list-table:: List of devlink port flavours
:widths: 33 90
* - Flavour
- Description
* - ``DEVLINK_PORT_FLAVOUR_PHYSICAL``
- Any kind of physical port. This can be an eswitch physical port or any
other physical port on the device.
* - ``DEVLINK_PORT_FLAVOUR_DSA``
- This indicates a DSA interconnect port.
* - ``DEVLINK_PORT_FLAVOUR_CPU``
- This indicates a CPU port applicable only to DSA.
* - ``DEVLINK_PORT_FLAVOUR_PCI_PF``
- This indicates an eswitch port representing a port of PCI
physical function (PF).
* - ``DEVLINK_PORT_FLAVOUR_PCI_VF``
- This indicates an eswitch port representing a port of PCI
virtual function (VF).
* - ``DEVLINK_PORT_FLAVOUR_PCI_SF``
- This indicates an eswitch port representing a port of PCI
subfunction (SF).
* - ``DEVLINK_PORT_FLAVOUR_VIRTUAL``
- Any virtual port facing the user.
Devlink port can have a different type based on the link layer described below.
.. list-table:: List of devlink port types
:widths: 23 90
* - Type
- Description
* - ``DEVLINK_PORT_TYPE_ETH``
- Driver should set this port type when a link layer of the port is
Ethernet.
* - ``DEVLINK_PORT_TYPE_IB``
- Driver should set this port type when a link layer of the port is
InfiniBand.
* - ``DEVLINK_PORT_TYPE_AUTO``
- This type is indicated by the user when driver should detect the port
type automatically.
PCI controllers
---------------
In most cases a PCI device has only one controller. A controller consists of
potentially multiple physical, virtual functions and subfunctions. A function
consists of one or more ports. This port is represented by the devlink eswitch
port.
A PCI device connected to multiple CPUs or multiple PCI root complexes or a
SmartNIC, however, may have multiple controllers. For a device with multiple
controllers, each controller is distinguished by a unique controller number.
An eswitch is on the PCI device which supports ports of multiple controllers.
An example view of a system with two controllers::
---------------------------------------------------------
| |
| --------- --------- ------- ------- |
----------- | | vf(s) | | sf(s) | |vf(s)| |sf(s)| |
| server | | ------- ----/---- ---/----- ------- ---/--- ---/--- |
| pci rc |=== | pf0 |______/________/ | pf1 |___/_______/ |
| connect | | ------- ------- |
----------- | | controller_num=1 (no eswitch) |
------|--------------------------------------------------
(internal wire)
|
---------------------------------------------------------
| devlink eswitch ports and reps |
| ----------------------------------------------------- |
| |ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 | ctrl-0 |ctrl-0 | |
| |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | |
| ----------------------------------------------------- |
| |ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 | ctrl-1 |ctrl-1 | |
| |pf0 | pf0vfN | pf0sfN | pf1 | pf1vfN |pf1sfN | |
| ----------------------------------------------------- |
| |
| |
----------- | --------- --------- ------- ------- |
| smartNIC| | | vf(s) | | sf(s) | |vf(s)| |sf(s)| |
| pci rc |==| ------- ----/---- ---/----- ------- ---/--- ---/--- |
| connect | | | pf0 |______/________/ | pf1 |___/_______/ |
----------- | ------- ------- |
| |
| local controller_num=0 (eswitch) |
---------------------------------------------------------
In the above example, the external controller (identified by controller number = 1)
doesn't have the eswitch. Local controller (identified by controller number = 0)
has the eswitch. The Devlink instance on the local controller has eswitch
devlink ports for both the controllers.
Function configuration
======================
Users can configure one or more function attributes before enumerating the PCI
function. Usually it means, user should configure function attribute
before a bus specific device for the function is created. However, when
SRIOV is enabled, virtual function devices are created on the PCI bus.
Hence, function attribute should be configured before binding virtual
function device to the driver. For subfunctions, this means user should
configure port function attribute before activating the port function.
A user may set the hardware address of the function using
`devlink port function set hw_addr` command. For Ethernet port function
this means a MAC address.
Users may also set the RoCE capability of the function using
`devlink port function set roce` command.
Users may also set the function as migratable using
`devlink port function set migratable` command.
Users may also set the IPsec crypto capability of the function using
`devlink port function set ipsec_crypto` command.
Users may also set the IPsec packet capability of the function using
`devlink port function set ipsec_packet` command.
The ``migratable`` attribute may be set only on ports with
``DEVLINK_PORT_FLAVOUR_PCI_VF``.
Users may also set the maximum IO event queues of the function
using `devlink port function set max_io_eqs` command.
Function attributes
===================
MAC address setup
-----------------
The configured MAC address of the PCI VF/SF will be used by netdevice and rdma
device created for the PCI VF/SF.
- Get the MAC address of the VF identified by its unique devlink port index::
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00
- Set the MAC address of the VF identified by its unique devlink port index::
$ devlink port function set pci/0000:06:00.0/2 hw_addr 00:11:22:33:44:55
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:11:22:33:44:55
- Get the MAC address of the SF identified by its unique devlink port index::
$ devlink port show pci/0000:06:00.0/32768
pci/0000:06:00.0/32768: type eth netdev enp6s0pf0sf88 flavour pcisf pfnum 0 sfnum 88
function:
hw_addr 00:00:00:00:00:00
- Set the MAC address of the SF identified by its unique devlink port index::
$ devlink port function set pci/0000:06:00.0/32768 hw_addr 00:00:00:00:88:88
$ devlink port show pci/0000:06:00.0/32768
pci/0000:06:00.0/32768: type eth netdev enp6s0pf0sf88 flavour pcisf pfnum 0 sfnum 88
function:
hw_addr 00:00:00:00:88:88
RoCE capability setup
---------------------
Not all PCI VFs/SFs require RoCE capability.
When RoCE capability is disabled, it saves system memory per PCI VF/SF.
When user disables RoCE capability for a VF/SF, user application cannot send or
receive any RoCE packets through this VF/SF and RoCE GID table for this PCI
will be empty.
When RoCE capability is disabled in the device using port function attribute,
VF/SF driver cannot override it.
- Get RoCE capability of the VF device::
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 roce enable
- Set RoCE capability of the VF device::
$ devlink port function set pci/0000:06:00.0/2 roce disable
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 roce disable
migratable capability setup
---------------------------
Live migration is the process of transferring a live virtual machine
from one physical host to another without disrupting its normal
operation.
User who want PCI VFs to be able to perform live migration need to
explicitly enable the VF migratable capability.
When user enables migratable capability for a VF, and the HV binds the VF to VFIO driver
with migration support, the user can migrate the VM with this VF from one HV to a
different one.
However, when migratable capability is enable, device will disable features which cannot
be migrated. Thus migratable cap can impose limitations on a VF so let the user decide.
Example of LM with migratable function configuration:
- Get migratable capability of the VF device::
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 migratable disable
- Set migratable capability of the VF device::
$ devlink port function set pci/0000:06:00.0/2 migratable enable
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 migratable enable
- Bind VF to VFIO driver with migration support::
$ echo <pci_id> > /sys/bus/pci/devices/0000:08:00.0/driver/unbind
$ echo mlx5_vfio_pci > /sys/bus/pci/devices/0000:08:00.0/driver_override
$ echo <pci_id> > /sys/bus/pci/devices/0000:08:00.0/driver/bind
Attach VF to the VM.
Start the VM.
Perform live migration.
IPsec crypto capability setup
-----------------------------
When user enables IPsec crypto capability for a VF, user application can offload
XFRM state crypto operation (Encrypt/Decrypt) to this VF.
When IPsec crypto capability is disabled (default) for a VF, the XFRM state is
processed in software by the kernel.
- Get IPsec crypto capability of the VF device::
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 ipsec_crypto disabled
- Set IPsec crypto capability of the VF device::
$ devlink port function set pci/0000:06:00.0/2 ipsec_crypto enable
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 ipsec_crypto enabled
IPsec packet capability setup
-----------------------------
When user enables IPsec packet capability for a VF, user application can offload
XFRM state and policy crypto operation (Encrypt/Decrypt) to this VF, as well as
IPsec encapsulation.
When IPsec packet capability is disabled (default) for a VF, the XFRM state and
policy is processed in software by the kernel.
- Get IPsec packet capability of the VF device::
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 ipsec_packet disabled
- Set IPsec packet capability of the VF device::
$ devlink port function set pci/0000:06:00.0/2 ipsec_packet enable
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 ipsec_packet enabled
Maximum IO events queues setup
------------------------------
When user sets maximum number of IO event queues for a SF or
a VF, such function driver is limited to consume only enforced
number of IO event queues.
IO event queues deliver events related to IO queues, including network
device transmit and receive queues (txq and rxq) and RDMA Queue Pairs (QPs).
For example, the number of netdevice channels and RDMA device completion
vectors are derived from the function's IO event queues. Usually, the number
of interrupt vectors consumed by the driver is limited by the number of IO
event queues per device, as each of the IO event queues is connected to an
interrupt vector.
- Get maximum IO event queues of the VF device::
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 ipsec_packet disabled max_io_eqs 10
- Set maximum IO event queues of the VF device::
$ devlink port function set pci/0000:06:00.0/2 max_io_eqs 32
$ devlink port show pci/0000:06:00.0/2
pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
function:
hw_addr 00:00:00:00:00:00 ipsec_packet disabled max_io_eqs 32
Subfunction
============
Subfunction is a lightweight function that has a parent PCI function on which
it is deployed. Subfunction is created and deployed in unit of 1. Unlike
SRIOV VFs, a subfunction doesn't require its own PCI virtual function.
A subfunction communicates with the hardware through the parent PCI function.
To use a subfunction, 3 steps setup sequence is followed:
1) create - create a subfunction;
2) configure - configure subfunction attributes;
3) deploy - deploy the subfunction;
Subfunction management is done using devlink port user interface.
User performs setup on the subfunction management device.
(1) Create
----------
A subfunction is created using a devlink port interface. A user adds the
subfunction by adding a devlink port of subfunction flavour. The devlink
kernel code calls down to subfunction management driver (devlink ops) and asks
it to create a subfunction devlink port. Driver then instantiates the
subfunction port and any associated objects such as health reporters and
representor netdevice.
(2) Configure
-------------
A subfunction devlink port is created but it is not active yet. That means the
entities are created on devlink side, the e-switch port representor is created,
but the subfunction device itself is not created. A user might use e-switch port
representor to do settings, putting it into bridge, adding TC rules, etc. A user
might as well configure the hardware address (such as MAC address) of the
subfunction while subfunction is inactive.
(3) Deploy
----------
Once a subfunction is configured, user must activate it to use it. Upon
activation, subfunction management driver asks the subfunction management
device to instantiate the subfunction device on particular PCI function.
A subfunction device is created on the :ref:`Documentation/driver-api/auxiliary_bus.rst <auxiliary_bus>`.
At this point a matching subfunction driver binds to the subfunction's auxiliary device.
Rate object management
======================
Devlink provides API to manage tx rates of single devlink port or a group.
This is done through rate objects, which can be one of the two types:
``leaf``
Represents a single devlink port; created/destroyed by the driver. Since leaf
have 1to1 mapping to its devlink port, in user space it is referred as
``pci/<bus_addr>/<port_index>``;
``node``
Represents a group of rate objects (leafs and/or nodes); created/deleted by
request from the userspace; initially empty (no rate objects added). In
userspace it is referred as ``pci/<bus_addr>/<node_name>``, where
``node_name`` can be any identifier, except decimal number, to avoid
collisions with leafs.
API allows to configure following rate object's parameters:
``tx_share``
Minimum TX rate value shared among all other rate objects, or rate objects
that parts of the parent group, if it is a part of the same group.
``tx_max``
Maximum TX rate value.
``tx_priority``
Allows for usage of strict priority arbiter among siblings. This
arbitration scheme attempts to schedule nodes based on their priority
as long as the nodes remain within their bandwidth limit. The higher the
priority the higher the probability that the node will get selected for
scheduling.
``tx_weight``
Allows for usage of Weighted Fair Queuing arbitration scheme among
siblings. This arbitration scheme can be used simultaneously with the
strict priority. As a node is configured with a higher rate it gets more
BW relative to its siblings. Values are relative like a percentage
points, they basically tell how much BW should node take relative to
its siblings.
``parent``
Parent node name. Parent node rate limits are considered as additional limits
to all node children limits. ``tx_max`` is an upper limit for children.
``tx_share`` is a total bandwidth distributed among children.
``tc_bw``
Allow users to set the bandwidth allocation per traffic class on rate
objects. This enables fine-grained QoS configurations by assigning a relative
share value to each traffic class. The bandwidth is distributed in proportion
to the share value for each class, relative to the sum of all shares.
When applied to a non-leaf node, tc_bw determines how bandwidth is shared
among its child elements.
``tx_priority`` and ``tx_weight`` can be used simultaneously. In that case
nodes with the same priority form a WFQ subgroup in the sibling group
and arbitration among them is based on assigned weights.
Arbitration flow from the high level:
#. Choose a node, or group of nodes with the highest priority that stays
within the BW limit and are not blocked. Use ``tx_priority`` as a
parameter for this arbitration.
#. If group of nodes have the same priority perform WFQ arbitration on
that subgroup. Use ``tx_weight`` as a parameter for this arbitration.
#. Select the winner node, and continue arbitration flow among its children,
until leaf node is reached, and the winner is established.
#. If all the nodes from the highest priority sub-group are satisfied, or
overused their assigned BW, move to the lower priority nodes.
Driver implementations are allowed to support both or either rate object types
and setting methods of their parameters. Additionally driver implementation
may export nodes/leafs and their child-parent relationships.
Terms and Definitions
=====================
.. list-table:: Terms and Definitions
:widths: 22 90
* - Term
- Definitions
* - ``PCI device``
- A physical PCI device having one or more PCI buses consists of one or
more PCI controllers.
* - ``PCI controller``
- A controller consists of potentially multiple physical functions,
virtual functions and subfunctions.
* - ``Port function``
- An object to manage the function of a port.
* - ``Subfunction``
- A lightweight function that has parent PCI function on which it is
deployed.
* - ``Subfunction device``
- A bus device of the subfunction, usually on a auxiliary bus.
* - ``Subfunction driver``
- A device driver for the subfunction auxiliary device.
* - ``Subfunction management device``
- A PCI physical function that supports subfunction management.
* - ``Subfunction management driver``
- A device driver for PCI physical function that supports
subfunction management using devlink port interface.
* - ``Subfunction host driver``
- A device driver for PCI physical function that hosts subfunction
devices. In most cases it is same as subfunction management driver. When
subfunction is used on external controller, subfunction management and
host drivers are different.
@@ -0,0 +1,83 @@
.. SPDX-License-Identifier: GPL-2.0
==============
Devlink Region
==============
``devlink`` regions enable access to driver defined address regions using
devlink.
Each device can create and register its own supported address regions. The
region can then be accessed via the devlink region interface.
Region snapshots are collected by the driver, and can be accessed via read
or dump commands. This allows future analysis on the created snapshots.
Regions may optionally support triggering snapshots on demand.
Snapshot identifiers are scoped to the devlink instance, not a region.
All snapshots with the same snapshot id within a devlink instance
correspond to the same event.
The major benefit to creating a region is to provide access to internal
address regions that are otherwise inaccessible to the user.
Regions may also be used to provide an additional way to debug complex error
states, but see also Documentation/networking/devlink/devlink-health.rst
Regions may optionally support capturing a snapshot on demand via the
``DEVLINK_CMD_REGION_NEW`` netlink message. A driver wishing to allow
requested snapshots must implement the ``.snapshot`` callback for the region
in its ``devlink_region_ops`` structure. If snapshot id is not set in
the ``DEVLINK_CMD_REGION_NEW`` request kernel will allocate one and send
the snapshot information to user space.
Regions may optionally allow directly reading from their contents without a
snapshot. Direct read requests are not atomic. In particular a read request
of size 256 bytes or larger will be split into multiple chunks. If atomic
access is required, use a snapshot. A driver wishing to enable this for a
region should implement the ``.read`` callback in the ``devlink_region_ops``
structure. User space can request a direct read by using the
``DEVLINK_ATTR_REGION_DIRECT`` attribute instead of specifying a snapshot
id.
example usage
-------------
.. code:: shell
$ devlink region help
$ devlink region show [ DEV/REGION ]
$ devlink region del DEV/REGION snapshot SNAPSHOT_ID
$ devlink region dump DEV/REGION [ snapshot SNAPSHOT_ID ]
$ devlink region read DEV/REGION [ snapshot SNAPSHOT_ID ] address ADDRESS length LENGTH
# Show all of the exposed regions with region sizes:
$ devlink region show
pci/0000:00:05.0/cr-space: size 1048576 snapshot [1 2] max 8
pci/0000:00:05.0/fw-health: size 64 snapshot [1 2] max 8
# Delete a snapshot using:
$ devlink region del pci/0000:00:05.0/cr-space snapshot 1
# Request an immediate snapshot, if supported by the region
$ devlink region new pci/0000:00:05.0/cr-space
pci/0000:00:05.0/cr-space: snapshot 5
# Dump a snapshot:
$ devlink region dump pci/0000:00:05.0/fw-health snapshot 1
0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
# Read a specific part of a snapshot:
$ devlink region read pci/0000:00:05.0/fw-health snapshot 1 address 0 length 16
0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
# Read from the region without a snapshot
$ devlink region read pci/0000:00:05.0/fw-health address 16 length 16
0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
As regions are likely very device or driver specific, no generic regions are
defined. See the driver-specific documentation files for information on the
specific regions a driver supports.
@@ -0,0 +1,90 @@
.. SPDX-License-Identifier: GPL-2.0
==============
Devlink Reload
==============
``devlink-reload`` provides mechanism to reinit driver entities, applying
``devlink-params`` and ``devlink-resources`` new values. It also provides
mechanism to activate firmware.
Reload Actions
==============
User may select a reload action.
By default ``driver_reinit`` action is selected.
.. list-table:: Possible reload actions
:widths: 5 90
* - Name
- Description
* - ``driver-reinit``
- Devlink driver entities re-initialization, including applying
new values to devlink entities which are used during driver
load which are:
* ``devlink-params`` in configuration mode ``driverinit``
* ``devlink-resources``
Other devlink entities may stay over the re-initialization:
* ``devlink-health-reporter``
* ``devlink-region``
The rest of the devlink entities have to be removed and readded.
* - ``fw_activate``
- Firmware activate. Activates new firmware if such image is stored and
pending activation. If no limitation specified this action may involve
firmware reset. If no new image pending this action will reload current
firmware image.
Note that even though user asks for a specific action, the driver
implementation might require to perform another action alongside with
it. For example, some driver do not support driver reinitialization
being performed without fw activation. Therefore, the devlink reload
command returns the list of actions which were actrually performed.
Reload Limits
=============
By default reload actions are not limited and driver implementation may
include reset or downtime as needed to perform the actions.
However, some drivers support action limits, which limit the action
implementation to specific constraints.
.. list-table:: Possible reload limits
:widths: 5 90
* - Name
- Description
* - ``no_reset``
- No reset allowed, no down time allowed, no link flap and no
configuration is lost.
Change Namespace
================
The netns option allows user to be able to move devlink instances into
namespaces during devlink reload operation.
By default all devlink instances are created in init_net and stay there.
example usage
-------------
.. code:: shell
$ devlink dev reload help
$ devlink dev reload DEV [ netns { PID | NAME | ID } ] [ action { driver_reinit | fw_activate } ] [ limit no_reset ]
# Run reload command for devlink driver entities re-initialization:
$ devlink dev reload pci/0000:82:00.0 action driver_reinit
reload_actions_performed:
driver_reinit
# Run reload command to activate firmware:
# Note that mlx5 driver reloads the driver while activating firmware
$ devlink dev reload pci/0000:82:00.0 action fw_activate
reload_actions_performed:
driver_reinit fw_activate
@@ -0,0 +1,146 @@
.. SPDX-License-Identifier: GPL-2.0
================
Devlink Resource
================
``devlink`` provides the ability for drivers to register resources, which
can allow administrators to see the device restrictions for a given
resource, as well as how much of the given resource is currently
in use. Additionally, these resources can optionally have configurable size.
This could enable the administrator to limit the number of resources that
are used.
For example, the ``netdevsim`` driver enables ``/IPv4/fib`` and
``/IPv4/fib-rules`` as resources to limit the number of IPv4 FIB entries and
rules for a given device.
Resource Ids
============
Each resource is represented by an id, and contains information about its
current size and related sub resources. To access a sub resource, you
specify the path of the resource. For example ``/IPv4/fib`` is the id for
the ``fib`` sub-resource under the ``IPv4`` resource.
Generic Resources
=================
Generic resources are used to describe resources that can be shared by multiple
device drivers and their description must be added to the following table:
.. list-table:: List of Generic Resources
:widths: 10 90
* - Name
- Description
* - ``physical_ports``
- A limited capacity of physical ports that the switch ASIC can support
example usage
-------------
The resources exposed by the driver can be observed, for example:
.. code:: shell
$devlink resource show pci/0000:03:00.0
pci/0000:03:00.0:
name kvd size 245760 unit entry
resources:
name linear size 98304 occ 0 unit entry size_min 0 size_max 147456 size_gran 128
name hash_double size 60416 unit entry size_min 32768 size_max 180224 size_gran 128
name hash_single size 87040 unit entry size_min 65536 size_max 212992 size_gran 128
Some resource's size can be changed. Examples:
.. code:: shell
$devlink resource set pci/0000:03:00.0 path /kvd/hash_single size 73088
$devlink resource set pci/0000:03:00.0 path /kvd/hash_double size 74368
The changes do not apply immediately, this can be validated by the 'size_new'
attribute, which represents the pending change in size. For example:
.. code:: shell
$devlink resource show pci/0000:03:00.0
pci/0000:03:00.0:
name kvd size 245760 unit entry size_valid false
resources:
name linear size 98304 size_new 147456 occ 0 unit entry size_min 0 size_max 147456 size_gran 128
name hash_double size 60416 unit entry size_min 32768 size_max 180224 size_gran 128
name hash_single size 87040 unit entry size_min 65536 size_max 212992 size_gran 128
Note that changes in resource size may require a device reload to properly
take effect.
Port-level Resources and Full Dump
==================================
In addition to device-level resources, ``devlink`` also supports port-level
resources. These resources are associated with a specific devlink port rather
than the device as a whole.
To list resources for all devlink devices and ports:
.. code:: shell
$ devlink resource show
pci/0000:03:00.0:
name max_local_SFs size 128 unit entry dpipe_tables none
name max_external_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.0/196608:
name max_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.0/196609:
name max_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.1:
name max_local_SFs size 128 unit entry dpipe_tables none
name max_external_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.1/196708:
name max_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.1/196709:
name max_SFs size 128 unit entry dpipe_tables none
To show resources for a specific port:
.. code:: shell
$ devlink resource show pci/0000:03:00.0/196608
pci/0000:03:00.0/196608:
name max_SFs size 128 unit entry dpipe_tables none
Resource Scope Filtering
========================
When dumping resources for all devices, ``devlink resource show`` accepts
an optional ``scope`` parameter to restrict the response to device-level
resources, port-level resources, or both (the default).
To dump only device-level resources across all devices:
.. code:: shell
$ devlink resource show scope dev
pci/0000:03:00.0:
name max_local_SFs size 128 unit entry dpipe_tables none
name max_external_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.1:
name max_local_SFs size 128 unit entry dpipe_tables none
name max_external_SFs size 128 unit entry dpipe_tables none
To dump only port-level resources across all devices:
.. code:: shell
$ devlink resource show scope port
pci/0000:03:00.0/196608:
name max_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.0/196609:
name max_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.1/196708:
name max_SFs size 128 unit entry dpipe_tables none
pci/0000:03:00.1/196709:
name max_SFs size 128 unit entry dpipe_tables none
Note that port-level resources are read-only.
@@ -0,0 +1,38 @@
.. SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
=================
Devlink Selftests
=================
The ``devlink-selftests`` API allows executing selftests on the device.
Tests Mask
==========
The ``devlink-selftests`` command should be run with a mask indicating
the tests to be executed.
Tests Description
=================
The following is a list of tests that drivers may execute.
.. list-table:: List of tests
:widths: 5 90
* - Name
- Description
* - ``DEVLINK_SELFTEST_FLASH``
- Devices may have the firmware on non-volatile memory on the board, e.g.
flash. This particular test helps to run a flash selftest on the device.
Implementation of the test is left to the driver/firmware.
example usage
-------------
.. code:: shell
# Query selftests supported on the devlink device
$ devlink dev selftests show DEV
# Query selftests supported on all devlink devices
$ devlink dev selftests show
# Executes selftests on the device
$ devlink dev selftests run DEV id flash
@@ -0,0 +1,97 @@
.. SPDX-License-Identifier: GPL-2.0
========================
Devlink Shared Instances
========================
Overview
========
Shared devlink instances allow multiple physical functions (PFs) on the same
chip to share a devlink instance for chip-wide operations.
Multiple PFs may reside on the same physical chip, running a single firmware.
Some of the resources and configurations may be shared among these PFs. The
shared devlink instance provides an object to pin configuration knobs on.
There are two possible usage models:
1. The shared devlink instance is used alongside individual PF devlink
instances, providing chip-wide configuration in addition to per-PF
configuration.
2. The shared devlink instance is the only devlink instance, without
per-PF instances.
It is up to the driver to decide which usage model to use.
The shared devlink instance is not backed by any struct *device*.
Implementation
==============
Architecture
------------
The implementation uses:
* **Chip identification**: PFs are grouped by chip using a driver-specific identifier
* **Shared instance management**: Global list of shared instances with reference counting
API Functions
-------------
The following functions are provided for managing shared devlink instances:
* ``devlink_shd_get()``: Get or create a shared devlink instance identified by a string ID
* ``devlink_shd_put()``: Release a reference on a shared devlink instance
* ``devlink_shd_get_priv()``: Get private data from shared devlink instance
Initialization Flow
-------------------
1. **PF calls shared devlink init** during driver probe
2. **Chip identification** using driver-specific method to determine device identity
3. **Get or create shared instance** using ``devlink_shd_get()``:
* The function looks up existing instance by identifier
* If none exists, creates new instance:
- Allocates and registers devlink instance
- Adds to global shared instances list
- Increments reference count
4. **Set nested devlink instance** for the PF devlink instance using
``devl_nested_devlink_set()`` before registering the PF devlink instance
Cleanup Flow
------------
1. **Cleanup** when PF is removed
2. **Call** ``devlink_shd_put()`` to release reference (decrements reference count)
3. **Shared instance is automatically destroyed** when the last PF removes (reference count reaches zero)
Chip Identification
-------------------
PFs belonging to the same chip are identified using a driver-specific method.
The driver is free to choose any identifier that is suitable for determining
whether two PFs are part of the same device. Examples include:
* **PCI VPD serial numbers**: Extract from PCI VPD
* **Device tree properties**: Read chip identifier from device tree
* **Other hardware-specific identifiers**: Any unique identifier that groups PFs by chip
Locking
-------
A global mutex (``shd_mutex``) protects the shared instances list during registration/deregistration.
Similarly to other nested devlink instance relationships, devlink lock of
the shared instance should be always taken after the devlink lock of PF.
Reference Counting
------------------
Each shared devlink instance maintains a reference count (``refcount_t refcount``).
The reference count is incremented when ``devlink_shd_get()`` is called and decremented
when ``devlink_shd_put()`` is called. When the reference count reaches zero, the shared
instance is automatically destroyed.
@@ -0,0 +1,642 @@
.. SPDX-License-Identifier: GPL-2.0
============
Devlink Trap
============
Background
==========
Devices capable of offloading the kernel's datapath and perform functions such
as bridging and routing must also be able to send specific packets to the
kernel (i.e., the CPU) for processing.
For example, a device acting as a multicast-aware bridge must be able to send
IGMP membership reports to the kernel for processing by the bridge module.
Without processing such packets, the bridge module could never populate its
MDB.
As another example, consider a device acting as router which has received an IP
packet with a TTL of 1. Upon routing the packet the device must send it to the
kernel so that it will route it as well and generate an ICMP Time Exceeded
error datagram. Without letting the kernel route such packets itself, utilities
such as ``traceroute`` could never work.
The fundamental ability of sending certain packets to the kernel for processing
is called "packet trapping".
Overview
========
The ``devlink-trap`` mechanism allows capable device drivers to register their
supported packet traps with ``devlink`` and report trapped packets to
``devlink`` for further analysis.
Upon receiving trapped packets, ``devlink`` will perform a per-trap packets and
bytes accounting and potentially report the packet to user space via a netlink
event along with all the provided metadata (e.g., trap reason, timestamp, input
port). This is especially useful for drop traps (see :ref:`Trap-Types`)
as it allows users to obtain further visibility into packet drops that would
otherwise be invisible.
The following diagram provides a general overview of ``devlink-trap``::
Netlink event: Packet w/ metadata
Or a summary of recent drops
^
|
Userspace |
+---------------------------------------------------+
Kernel |
|
+-------+--------+
| |
| drop_monitor |
| |
+-------^--------+
|
| Non-control traps
|
+----+----+
| | Kernel's Rx path
| devlink | (non-drop traps)
| |
+----^----+ ^
| |
+-----------+
|
+-------+-------+
| |
| Device driver |
| |
+-------^-------+
Kernel |
+---------------------------------------------------+
Hardware |
| Trapped packet
|
+--+---+
| |
| ASIC |
| |
+------+
.. _Trap-Types:
Trap Types
==========
The ``devlink-trap`` mechanism supports the following packet trap types:
* ``drop``: Trapped packets were dropped by the underlying device. Packets
are only processed by ``devlink`` and not injected to the kernel's Rx path.
The trap action (see :ref:`Trap-Actions`) can be changed.
* ``exception``: Trapped packets were not forwarded as intended by the
underlying device due to an exception (e.g., TTL error, missing neighbour
entry) and trapped to the control plane for resolution. Packets are
processed by ``devlink`` and injected to the kernel's Rx path. Changing the
action of such traps is not allowed, as it can easily break the control
plane.
* ``control``: Trapped packets were trapped by the device because these are
control packets required for the correct functioning of the control plane.
For example, ARP request and IGMP query packets. Packets are injected to
the kernel's Rx path, but not reported to the kernel's drop monitor.
Changing the action of such traps is not allowed, as it can easily break
the control plane.
.. _Trap-Actions:
Trap Actions
============
The ``devlink-trap`` mechanism supports the following packet trap actions:
* ``trap``: The sole copy of the packet is sent to the CPU.
* ``drop``: The packet is dropped by the underlying device and a copy is not
sent to the CPU.
* ``mirror``: The packet is forwarded by the underlying device and a copy is
sent to the CPU.
Generic Packet Traps
====================
Generic packet traps are used to describe traps that trap well-defined packets
or packets that are trapped due to well-defined conditions (e.g., TTL error).
Such traps can be shared by multiple device drivers and their description must
be added to the following table:
.. list-table:: List of Generic Packet Traps
:widths: 5 5 90
* - Name
- Type
- Description
* - ``source_mac_is_multicast``
- ``drop``
- Traps incoming packets that the device decided to drop because of a
multicast source MAC
* - ``vlan_tag_mismatch``
- ``drop``
- Traps incoming packets that the device decided to drop in case of VLAN
tag mismatch: The ingress bridge port is not configured with a PVID and
the packet is untagged or prio-tagged
* - ``ingress_vlan_filter``
- ``drop``
- Traps incoming packets that the device decided to drop in case they are
tagged with a VLAN that is not configured on the ingress bridge port
* - ``ingress_spanning_tree_filter``
- ``drop``
- Traps incoming packets that the device decided to drop in case the STP
state of the ingress bridge port is not "forwarding"
* - ``port_list_is_empty``
- ``drop``
- Traps packets that the device decided to drop in case they need to be
flooded (e.g., unknown unicast, unregistered multicast) and there are
no ports the packets should be flooded to
* - ``port_loopback_filter``
- ``drop``
- Traps packets that the device decided to drop in case after layer 2
forwarding the only port from which they should be transmitted through
is the port from which they were received
* - ``blackhole_route``
- ``drop``
- Traps packets that the device decided to drop in case they hit a
blackhole route
* - ``ttl_value_is_too_small``
- ``exception``
- Traps unicast packets that should be forwarded by the device whose TTL
was decremented to 0 or less
* - ``tail_drop``
- ``drop``
- Traps packets that the device decided to drop because they could not be
enqueued to a transmission queue which is full
* - ``non_ip``
- ``drop``
- Traps packets that the device decided to drop because they need to
undergo a layer 3 lookup, but are not IP or MPLS packets
* - ``uc_dip_over_mc_dmac``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed and they have a unicast destination IP and a multicast destination
MAC
* - ``dip_is_loopback_address``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed and their destination IP is the loopback address (i.e., 127.0.0.0/8
and ::1/128)
* - ``sip_is_mc``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed and their source IP is multicast (i.e., 224.0.0.0/8 and ff::/8)
* - ``sip_is_loopback_address``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed and their source IP is the loopback address (i.e., 127.0.0.0/8 and ::1/128)
* - ``ip_header_corrupted``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed and their IP header is corrupted: wrong checksum, wrong IP version
or too short Internet Header Length (IHL)
* - ``ipv4_sip_is_limited_bc``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed and their source IP is limited broadcast (i.e., 255.255.255.255/32)
* - ``ipv6_mc_dip_reserved_scope``
- ``drop``
- Traps IPv6 packets that the device decided to drop because they need to
be routed and their IPv6 multicast destination IP has a reserved scope
(i.e., ffx0::/16)
* - ``ipv6_mc_dip_interface_local_scope``
- ``drop``
- Traps IPv6 packets that the device decided to drop because they need to
be routed and their IPv6 multicast destination IP has an interface-local scope
(i.e., ffx1::/16)
* - ``mtu_value_is_too_small``
- ``exception``
- Traps packets that should have been routed by the device, but were bigger
than the MTU of the egress interface
* - ``unresolved_neigh``
- ``exception``
- Traps packets that did not have a matching IP neighbour after routing
* - ``mc_reverse_path_forwarding``
- ``exception``
- Traps multicast IP packets that failed reverse-path forwarding (RPF)
check during multicast routing
* - ``reject_route``
- ``exception``
- Traps packets that hit reject routes (i.e., "unreachable", "prohibit")
* - ``ipv4_lpm_miss``
- ``exception``
- Traps unicast IPv4 packets that did not match any route
* - ``ipv6_lpm_miss``
- ``exception``
- Traps unicast IPv6 packets that did not match any route
* - ``non_routable_packet``
- ``drop``
- Traps packets that the device decided to drop because they are not
supposed to be routed. For example, IGMP queries can be flooded by the
device in layer 2 and reach the router. Such packets should not be
routed and instead dropped
* - ``decap_error``
- ``exception``
- Traps NVE and IPinIP packets that the device decided to drop because of
failure during decapsulation (e.g., packet being too short, reserved
bits set in VXLAN header)
* - ``overlay_smac_is_mc``
- ``drop``
- Traps NVE packets that the device decided to drop because their overlay
source MAC is multicast
* - ``ingress_flow_action_drop``
- ``drop``
- Traps packets dropped during processing of ingress flow action drop
* - ``egress_flow_action_drop``
- ``drop``
- Traps packets dropped during processing of egress flow action drop
* - ``stp``
- ``control``
- Traps STP packets
* - ``lacp``
- ``control``
- Traps LACP packets
* - ``lldp``
- ``control``
- Traps LLDP packets
* - ``igmp_query``
- ``control``
- Traps IGMP Membership Query packets
* - ``igmp_v1_report``
- ``control``
- Traps IGMP Version 1 Membership Report packets
* - ``igmp_v2_report``
- ``control``
- Traps IGMP Version 2 Membership Report packets
* - ``igmp_v3_report``
- ``control``
- Traps IGMP Version 3 Membership Report packets
* - ``igmp_v2_leave``
- ``control``
- Traps IGMP Version 2 Leave Group packets
* - ``mld_query``
- ``control``
- Traps MLD Multicast Listener Query packets
* - ``mld_v1_report``
- ``control``
- Traps MLD Version 1 Multicast Listener Report packets
* - ``mld_v2_report``
- ``control``
- Traps MLD Version 2 Multicast Listener Report packets
* - ``mld_v1_done``
- ``control``
- Traps MLD Version 1 Multicast Listener Done packets
* - ``ipv4_dhcp``
- ``control``
- Traps IPv4 DHCP packets
* - ``ipv6_dhcp``
- ``control``
- Traps IPv6 DHCP packets
* - ``arp_request``
- ``control``
- Traps ARP request packets
* - ``arp_response``
- ``control``
- Traps ARP response packets
* - ``arp_overlay``
- ``control``
- Traps NVE-decapsulated ARP packets that reached the overlay network.
This is required, for example, when the address that needs to be
resolved is a local address
* - ``ipv6_neigh_solicit``
- ``control``
- Traps IPv6 Neighbour Solicitation packets
* - ``ipv6_neigh_advert``
- ``control``
- Traps IPv6 Neighbour Advertisement packets
* - ``ipv4_bfd``
- ``control``
- Traps IPv4 BFD packets
* - ``ipv6_bfd``
- ``control``
- Traps IPv6 BFD packets
* - ``ipv4_ospf``
- ``control``
- Traps IPv4 OSPF packets
* - ``ipv6_ospf``
- ``control``
- Traps IPv6 OSPF packets
* - ``ipv4_bgp``
- ``control``
- Traps IPv4 BGP packets
* - ``ipv6_bgp``
- ``control``
- Traps IPv6 BGP packets
* - ``ipv4_vrrp``
- ``control``
- Traps IPv4 VRRP packets
* - ``ipv6_vrrp``
- ``control``
- Traps IPv6 VRRP packets
* - ``ipv4_pim``
- ``control``
- Traps IPv4 PIM packets
* - ``ipv6_pim``
- ``control``
- Traps IPv6 PIM packets
* - ``uc_loopback``
- ``control``
- Traps unicast packets that need to be routed through the same layer 3
interface from which they were received. Such packets are routed by the
kernel, but also cause it to potentially generate ICMP redirect packets
* - ``local_route``
- ``control``
- Traps unicast packets that hit a local route and need to be locally
delivered
* - ``external_route``
- ``control``
- Traps packets that should be routed through an external interface (e.g.,
management interface) that does not belong to the same device (e.g.,
switch ASIC) as the ingress interface
* - ``ipv6_uc_dip_link_local_scope``
- ``control``
- Traps unicast IPv6 packets that need to be routed and have a destination
IP address with a link-local scope (i.e., fe80::/10). The trap allows
device drivers to avoid programming link-local routes, but still receive
packets for local delivery
* - ``ipv6_dip_all_nodes``
- ``control``
- Traps IPv6 packets that their destination IP address is the "All Nodes
Address" (i.e., ff02::1)
* - ``ipv6_dip_all_routers``
- ``control``
- Traps IPv6 packets that their destination IP address is the "All Routers
Address" (i.e., ff02::2)
* - ``ipv6_router_solicit``
- ``control``
- Traps IPv6 Router Solicitation packets
* - ``ipv6_router_advert``
- ``control``
- Traps IPv6 Router Advertisement packets
* - ``ipv6_redirect``
- ``control``
- Traps IPv6 Redirect Message packets
* - ``ipv4_router_alert``
- ``control``
- Traps IPv4 packets that need to be routed and include the Router Alert
option. Such packets need to be locally delivered to raw sockets that
have the IP_ROUTER_ALERT socket option set
* - ``ipv6_router_alert``
- ``control``
- Traps IPv6 packets that need to be routed and include the Router Alert
option in their Hop-by-Hop extension header. Such packets need to be
locally delivered to raw sockets that have the IPV6_ROUTER_ALERT socket
option set
* - ``ptp_event``
- ``control``
- Traps PTP time-critical event messages (Sync, Delay_req, Pdelay_Req and
Pdelay_Resp)
* - ``ptp_general``
- ``control``
- Traps PTP general messages (Announce, Follow_Up, Delay_Resp,
Pdelay_Resp_Follow_Up, management and signaling)
* - ``flow_action_sample``
- ``control``
- Traps packets sampled during processing of flow action sample (e.g., via
tc's sample action)
* - ``flow_action_trap``
- ``control``
- Traps packets logged during processing of flow action trap (e.g., via
tc's trap action)
* - ``early_drop``
- ``drop``
- Traps packets dropped due to the RED (Random Early Detection) algorithm
(i.e., early drops)
* - ``vxlan_parsing``
- ``drop``
- Traps packets dropped due to an error in the VXLAN header parsing which
might be because of packet truncation or the I flag is not set.
* - ``llc_snap_parsing``
- ``drop``
- Traps packets dropped due to an error in the LLC+SNAP header parsing
* - ``vlan_parsing``
- ``drop``
- Traps packets dropped due to an error in the VLAN header parsing. Could
include unexpected packet truncation.
* - ``pppoe_ppp_parsing``
- ``drop``
- Traps packets dropped due to an error in the PPPoE+PPP header parsing.
This could include finding a session ID of 0xFFFF (which is reserved and
not for use), a PPPoE length which is larger than the frame received or
any common error on this type of header
* - ``mpls_parsing``
- ``drop``
- Traps packets dropped due to an error in the MPLS header parsing which
could include unexpected header truncation
* - ``arp_parsing``
- ``drop``
- Traps packets dropped due to an error in the ARP header parsing
* - ``ip_1_parsing``
- ``drop``
- Traps packets dropped due to an error in the first IP header parsing.
This packet trap could include packets which do not pass an IP checksum
check, a header length check (a minimum of 20 bytes), which might suffer
from packet truncation thus the total length field exceeds the received
packet length etc
* - ``ip_n_parsing``
- ``drop``
- Traps packets dropped due to an error in the parsing of the last IP
header (the inner one in case of an IP over IP tunnel). The same common
error checking is performed here as for the ip_1_parsing trap
* - ``gre_parsing``
- ``drop``
- Traps packets dropped due to an error in the GRE header parsing
* - ``udp_parsing``
- ``drop``
- Traps packets dropped due to an error in the UDP header parsing.
This packet trap could include checksum errors, an improper UDP
length detected (smaller than 8 bytes) or detection of header
truncation.
* - ``tcp_parsing``
- ``drop``
- Traps packets dropped due to an error in the TCP header parsing.
This could include TCP checksum errors, improper combination of SYN, FIN
and/or RESET etc.
* - ``ipsec_parsing``
- ``drop``
- Traps packets dropped due to an error in the IPSEC header parsing
* - ``sctp_parsing``
- ``drop``
- Traps packets dropped due to an error in the SCTP header parsing.
This would mean that port number 0 was used or that the header is
truncated.
* - ``dccp_parsing``
- ``drop``
- Traps packets dropped due to an error in the DCCP header parsing
* - ``gtp_parsing``
- ``drop``
- Traps packets dropped due to an error in the GTP header parsing
* - ``esp_parsing``
- ``drop``
- Traps packets dropped due to an error in the ESP header parsing
* - ``blackhole_nexthop``
- ``drop``
- Traps packets that the device decided to drop in case they hit a
blackhole nexthop
* - ``dmac_filter``
- ``drop``
- Traps incoming packets that the device decided to drop because
the destination MAC is not configured in the MAC table and
the interface is not in promiscuous mode
* - ``eapol``
- ``control``
- Traps "Extensible Authentication Protocol over LAN" (EAPOL) packets
specified in IEEE 802.1X
* - ``locked_port``
- ``drop``
- Traps packets that the device decided to drop because they failed the
locked bridge port check. That is, packets that were received via a
locked port and whose {SMAC, VID} does not correspond to an FDB entry
pointing to the port
Driver-specific Packet Traps
============================
Device drivers can register driver-specific packet traps, but these must be
clearly documented. Such traps can correspond to device-specific exceptions and
help debug packet drops caused by these exceptions. The following list includes
links to the description of driver-specific traps registered by various device
drivers:
* Documentation/networking/devlink/netdevsim.rst
* Documentation/networking/devlink/mlxsw.rst
* Documentation/networking/devlink/prestera.rst
.. _Generic-Packet-Trap-Groups:
Generic Packet Trap Groups
==========================
Generic packet trap groups are used to aggregate logically related packet
traps. These groups allow the user to batch operations such as setting the trap
action of all member drop traps whose action may legally change. Exception and
control traps remain unchanged. In addition, ``devlink-trap`` can report
aggregated per-group packets and bytes statistics, in case per-trap statistics
are too narrow. The description of these groups must be added to the following
table:
.. list-table:: List of Generic Packet Trap Groups
:widths: 10 90
* - Name
- Description
* - ``l2_drops``
- Contains packet traps for packets that were dropped by the device during
layer 2 forwarding (i.e., bridge)
* - ``l3_drops``
- Contains packet traps for packets that were dropped by the device during
layer 3 forwarding
* - ``l3_exceptions``
- Contains packet traps for packets that hit an exception (e.g., TTL
error) during layer 3 forwarding
* - ``buffer_drops``
- Contains packet traps for packets that were dropped by the device due to
an enqueue decision
* - ``tunnel_drops``
- Contains packet traps for packets that were dropped by the device during
tunnel encapsulation / decapsulation
* - ``acl_drops``
- Contains packet traps for packets that were dropped by the device during
ACL processing
* - ``stp``
- Contains packet traps for STP packets
* - ``lacp``
- Contains packet traps for LACP packets
* - ``lldp``
- Contains packet traps for LLDP packets
* - ``mc_snooping``
- Contains packet traps for IGMP and MLD packets required for multicast
snooping
* - ``dhcp``
- Contains packet traps for DHCP packets
* - ``neigh_discovery``
- Contains packet traps for neighbour discovery packets (e.g., ARP, IPv6
ND)
* - ``bfd``
- Contains packet traps for BFD packets
* - ``ospf``
- Contains packet traps for OSPF packets
* - ``bgp``
- Contains packet traps for BGP packets
* - ``vrrp``
- Contains packet traps for VRRP packets
* - ``pim``
- Contains packet traps for PIM packets
* - ``uc_loopback``
- Contains a packet trap for unicast loopback packets (i.e.,
``uc_loopback``). This trap is singled-out because in cases such as
one-armed router it will be constantly triggered. To limit the impact on
the CPU usage, a packet trap policer with a low rate can be bound to the
group without affecting other traps
* - ``local_delivery``
- Contains packet traps for packets that should be locally delivered after
routing, but do not match more specific packet traps (e.g.,
``ipv4_bgp``)
* - ``external_delivery``
- Contains packet traps for packets that should be routed through an
external interface (e.g., management interface) that does not belong to
the same device (e.g., switch ASIC) as the ingress interface
* - ``ipv6``
- Contains packet traps for various IPv6 control packets (e.g., Router
Advertisements)
* - ``ptp_event``
- Contains packet traps for PTP time-critical event messages (Sync,
Delay_req, Pdelay_Req and Pdelay_Resp)
* - ``ptp_general``
- Contains packet traps for PTP general messages (Announce, Follow_Up,
Delay_Resp, Pdelay_Resp_Follow_Up, management and signaling)
* - ``acl_sample``
- Contains packet traps for packets that were sampled by the device during
ACL processing
* - ``acl_trap``
- Contains packet traps for packets that were trapped (logged) by the
device during ACL processing
* - ``parser_error_drops``
- Contains packet traps for packets that were marked by the device during
parsing as erroneous
* - ``eapol``
- Contains packet traps for "Extensible Authentication Protocol over LAN"
(EAPOL) packets specified in IEEE 802.1X
Packet Trap Policers
====================
As previously explained, the underlying device can trap certain packets to the
CPU for processing. In most cases, the underlying device is capable of handling
packet rates that are several orders of magnitude higher compared to those that
can be handled by the CPU.
Therefore, in order to prevent the underlying device from overwhelming the CPU,
devices usually include packet trap policers that are able to police the
trapped packets to rates that can be handled by the CPU.
The ``devlink-trap`` mechanism allows capable device drivers to register their
supported packet trap policers with ``devlink``. The device driver can choose
to associate these policers with supported packet trap groups (see
:ref:`Generic-Packet-Trap-Groups`) during its initialization, thereby exposing
its default control plane policy to user space.
Device drivers should allow user space to change the parameters of the policers
(e.g., rate, burst size) as well as the association between the policers and
trap groups by implementing the relevant callbacks.
If possible, device drivers should implement a callback that allows user space
to retrieve the number of packets that were dropped by the policer because its
configured policy was violated.
Testing
=======
See ``tools/testing/selftests/drivers/net/netdevsim/devlink_trap.sh`` for a
test covering the core infrastructure. Test cases should be added for any new
functionality.
Device drivers should focus their tests on device-specific functionality, such
as the triggering of supported packet traps.
@@ -0,0 +1,36 @@
.. SPDX-License-Identifier: GPL-2.0
==========================
etas_es58x devlink support
==========================
This document describes the devlink features implemented by the
``etas_es58x`` device driver.
Info versions
=============
The ``etas_es58x`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fw``
- running
- Version of the firmware running on the device. Also available
through ``ethtool -i`` as the first member of the
``firmware-version``.
* - ``fw.bootloader``
- running
- Version of the bootloader running on the device. Also available
through ``ethtool -i`` as the second member of the
``firmware-version``.
* - ``board.rev``
- fixed
- The hardware revision of the device.
* - ``serial_number``
- fixed
- The USB serial number. Also available through ``lsusb -v``.
@@ -0,0 +1,30 @@
.. SPDX-License-Identifier: GPL-2.0
====================
hns3 devlink support
====================
This document describes the devlink features implemented by the ``hns3``
device driver.
The ``hns3`` driver supports reloading via ``DEVLINK_CMD_RELOAD``.
Info versions
=============
The ``hns3`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 10 10 80
* - Name
- Type
- Description
* - ``fw``
- running
- Used to represent the firmware version.
* - ``fw.scc``
- running
- Used to represent the Soft Congestion Control (SSC) firmware version.
SCC is a firmware component which provides multiple RDMA congestion
control algorithms, including DCQCN.
@@ -0,0 +1,93 @@
.. SPDX-License-Identifier: GPL-2.0
====================
i40e devlink support
====================
This document describes the devlink features implemented by the ``i40e``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
:widths: 5 5 90
* - Name
- Mode
- Notes
* - ``max_mac_per_vf``
- runtime
- Controls the maximum number of MAC addresses a VF can use
on i40e devices.
By default (``0``), the driver enforces its internally calculated per-VF
MAC filter limit, which is based on the number of allocated VFS.
If set to a non-zero value, this parameter acts as a strict cap:
the driver will use the user-provided value instead of its internal
calculation.
**Important notes:**
- This value **must be set before enabling SR-IOV**.
Attempting to change it while SR-IOV is enabled will return an error.
- MAC filters are a **shared hardware resource** across all VFs.
Setting a high value may cause other VFs to be starved of filters.
- This value is a **Administrative policy**. The hardware may return
errors when its absolute limit is reached, regardless of the value
set here.
The default value is ``0`` (internal calculation is used).
Info versions
=============
The ``i40e`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 5 90
* - Name
- Type
- Example
- Description
* - ``board.id``
- fixed
- K15190-000
- The Product Board Assembly (PBA) identifier of the board.
* - ``fw.mgmt``
- running
- 9.130
- 2-digit version number of the management firmware that controls the
PHY, link, etc.
* - ``fw.mgmt.api``
- running
- 1.15
- 2-digit version number of the API exported over the AdminQ by the
management firmware. Used by the driver to identify what commands
are supported.
* - ``fw.mgmt.build``
- running
- 73618
- Build number of the source for the management firmware.
* - ``fw.undi``
- running
- 1.3429.0
- Version of the Option ROM containing the UEFI driver. The version is
reported in ``major.minor.patch`` format. The major version is
incremented whenever a major breaking change occurs, or when the
minor version would overflow. The minor version is incremented for
non-breaking changes and reset to 1 when the major version is
incremented. The patch version is normally 0 but is incremented when
a fix is delivered as a patch against an older base Option ROM.
* - ``fw.psid.api``
- running
- 9.30
- Version defining the format of the flash contents.
* - ``fw.bundle_id``
- running
- 0x8000e5f3
- Unique identifier of the firmware image file that was loaded onto
the device. Also referred to as the EETRACK identifier of the NVM.
@@ -0,0 +1,487 @@
.. SPDX-License-Identifier: GPL-2.0
===================
ice devlink support
===================
This document describes the devlink features implemented by the ``ice``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
:widths: 5 5 90
* - Name
- Mode
- Notes
* - ``enable_roce``
- runtime
- mutually exclusive with ``enable_iwarp``
* - ``enable_iwarp``
- runtime
- mutually exclusive with ``enable_roce``
* - ``tx_scheduling_layers``
- permanent
- The ice hardware uses hierarchical scheduling for Tx with a fixed
number of layers in the scheduling tree. Each of them are decision
points. Root node represents a port, while all the leaves represent
the queues. This way of configuring the Tx scheduler allows features
like DCB or devlink-rate (documented below) to configure how much
bandwidth is given to any given queue or group of queues, enabling
fine-grained control because scheduling parameters can be configured
at any given layer of the tree.
The default 9-layer tree topology was deemed best for most workloads,
as it gives an optimal ratio of performance to configurability. However,
for some specific cases, this 9-layer topology might not be desired.
One example would be sending traffic to queues that are not a multiple
of 8. Because the maximum radix is limited to 8 in 9-layer topology,
the 9th queue has a different parent than the rest, and it's given
more bandwidth credits. This causes a problem when the system is
sending traffic to 9 queues:
| tx_queue_0_packets: 24163396
| tx_queue_1_packets: 24164623
| tx_queue_2_packets: 24163188
| tx_queue_3_packets: 24163701
| tx_queue_4_packets: 24163683
| tx_queue_5_packets: 24164668
| tx_queue_6_packets: 23327200
| tx_queue_7_packets: 24163853
| tx_queue_8_packets: 91101417 < Too much traffic is sent from 9th
To address this need, you can switch to a 5-layer topology, which
changes the maximum topology radix to 512. With this enhancement,
the performance characteristic is equal as all queues can be assigned
to the same parent in the tree. The obvious drawback of this solution
is a lower configuration depth of the tree.
Use the ``tx_scheduling_layer`` parameter with the devlink command
to change the transmit scheduler topology. To use 5-layer topology,
use a value of 5. For example:
$ devlink dev param set pci/0000:16:00.0 name tx_scheduling_layers
value 5 cmode permanent
Use a value of 9 to set it back to the default value.
You must do PCI slot powercycle for the selected topology to take effect.
To verify that value has been set:
$ devlink dev param show pci/0000:16:00.0 name tx_scheduling_layers
* - ``msix_vec_per_pf_max``
- driverinit
- Set the max MSI-X that can be used by the PF, rest can be utilized for
SRIOV. The range is from min value set in msix_vec_per_pf_min to
2k/number of ports.
* - ``msix_vec_per_pf_min``
- driverinit
- Set the min MSI-X that will be used by the PF. This value inform how many
MSI-X will be allocated statically. The range is from 2 to value set
in msix_vec_per_pf_max.
.. list-table:: Driver specific parameters implemented
:widths: 5 5 90
* - Name
- Mode
- Description
* - ``local_forwarding``
- runtime
- Controls loopback behavior by tuning scheduler bandwidth.
It impacts all kinds of functions: physical, virtual and
subfunctions.
Supported values are:
``enabled`` - loopback traffic is allowed on port
``disabled`` - loopback traffic is not allowed on this port
``prioritized`` - loopback traffic is prioritized on this port
Default value of ``local_forwarding`` parameter is ``enabled``.
``prioritized`` provides ability to adjust loopback traffic rate to increase
one port capacity at cost of the another. User needs to disable
local forwarding on one of the ports in order have increased capacity
on the ``prioritized`` port.
Info versions
=============
The ``ice`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 5 90
* - Name
- Type
- Example
- Description
* - ``board.id``
- fixed
- K65390-000
- The Product Board Assembly (PBA) identifier of the board.
* - ``cgu.id``
- fixed
- 36
- The Clock Generation Unit (CGU) hardware revision identifier.
* - ``fw.mgmt``
- running
- 2.1.7
- 3-digit version number of the management firmware running on the
Embedded Management Processor of the device. It controls the PHY,
link, access to device resources, etc. Intel documentation refers to
this as the EMP firmware.
* - ``fw.mgmt.api``
- running
- 1.5.1
- 3-digit version number (major.minor.patch) of the API exported over
the AdminQ by the management firmware. Used by the driver to
identify what commands are supported. Historical versions of the
kernel only displayed a 2-digit version number (major.minor).
* - ``fw.mgmt.build``
- running
- 0x305d955f
- Unique identifier of the source for the management firmware.
* - ``fw.undi``
- running
- 1.2581.0
- Version of the Option ROM containing the UEFI driver. The version is
reported in ``major.minor.patch`` format. The major version is
incremented whenever a major breaking change occurs, or when the
minor version would overflow. The minor version is incremented for
non-breaking changes and reset to 1 when the major version is
incremented. The patch version is normally 0 but is incremented when
a fix is delivered as a patch against an older base Option ROM.
* - ``fw.psid.api``
- running
- 0.80
- Version defining the format of the flash contents.
* - ``fw.bundle_id``
- running
- 0x80002ec0
- Unique identifier of the firmware image file that was loaded onto
the device. Also referred to as the EETRACK identifier of the NVM.
* - ``fw.app.name``
- running
- ICE OS Default Package
- The name of the DDP package that is active in the device. The DDP
package is loaded by the driver during initialization. Each
variation of the DDP package has a unique name.
* - ``fw.app``
- running
- 1.3.1.0
- The version of the DDP package that is active in the device. Note
that both the name (as reported by ``fw.app.name``) and version are
required to uniquely identify the package.
* - ``fw.app.bundle_id``
- running
- 0xc0000001
- Unique identifier for the DDP package loaded in the device. Also
referred to as the DDP Track ID. Can be used to uniquely identify
the specific DDP package.
* - ``fw.netlist``
- running
- 1.1.2000-6.7.0
- The version of the netlist module. This module defines the device's
Ethernet capabilities and default settings, and is used by the
management firmware as part of managing link and device
connectivity.
* - ``fw.netlist.build``
- running
- 0xee16ced7
- The first 4 bytes of the hash of the netlist module contents.
* - ``fw.cgu``
- running
- 8032.16973825.6021
- The version of Clock Generation Unit (CGU). Format:
<CGU type>.<configuration version>.<firmware version>.
Flash Update
============
The ``ice`` driver implements support for flash update using the
``devlink-flash`` interface. It supports updating the device flash using a
combined flash image that contains the ``fw.mgmt``, ``fw.undi``, and
``fw.netlist`` components.
.. list-table:: List of supported overwrite modes
:widths: 5 95
* - Bits
- Behavior
* - ``DEVLINK_FLASH_OVERWRITE_SETTINGS``
- Do not preserve settings stored in the flash components being
updated. This includes overwriting the port configuration that
determines the number of physical functions the device will
initialize with.
* - ``DEVLINK_FLASH_OVERWRITE_SETTINGS`` and ``DEVLINK_FLASH_OVERWRITE_IDENTIFIERS``
- Do not preserve either settings or identifiers. Overwrite everything
in the flash with the contents from the provided image, without
performing any preservation. This includes overwriting device
identifying fields such as the MAC address, VPD area, and device
serial number. It is expected that this combination be used with an
image customized for the specific device.
The ice hardware does not support overwriting only identifiers while
preserving settings, and thus ``DEVLINK_FLASH_OVERWRITE_IDENTIFIERS`` on its
own will be rejected. If no overwrite mask is provided, the firmware will be
instructed to preserve all settings and identifying fields when updating.
Reload
======
The ``ice`` driver supports activating new firmware after a flash update
using ``DEVLINK_CMD_RELOAD`` with the ``DEVLINK_RELOAD_ACTION_FW_ACTIVATE``
action.
.. code:: shell
$ devlink dev reload pci/0000:01:00.0 reload action fw_activate
The new firmware is activated by issuing a device specific Embedded
Management Processor reset which requests the device to reset and reload the
EMP firmware image.
The driver does not currently support reloading the driver via
``DEVLINK_RELOAD_ACTION_DRIVER_REINIT``.
Port split
==========
The ``ice`` driver supports port splitting only for port 0, as the FW has
a predefined set of available port split options for the whole device.
A system reboot is required for port split to be applied.
The following command will select the port split option with 4 ports:
.. code:: shell
$ devlink port split pci/0000:16:00.0/0 count 4
The list of all available port options will be printed to dynamic debug after
each ``split`` and ``unsplit`` command. The first option is the default.
.. code:: shell
ice 0000:16:00.0: Available port split options and max port speeds (Gbps):
ice 0000:16:00.0: Status Split Quad 0 Quad 1
ice 0000:16:00.0: count L0 L1 L2 L3 L4 L5 L6 L7
ice 0000:16:00.0: Active 2 100 - - - 100 - - -
ice 0000:16:00.0: 2 50 - 50 - - - - -
ice 0000:16:00.0: Pending 4 25 25 25 25 - - - -
ice 0000:16:00.0: 4 25 25 - - 25 25 - -
ice 0000:16:00.0: 8 10 10 10 10 10 10 10 10
ice 0000:16:00.0: 1 100 - - - - - - -
There could be multiple FW port options with the same port split count. When
the same port split count request is issued again, the next FW port option with
the same port split count will be selected.
``devlink port unsplit`` will select the option with a split count of 1. If
there is no FW option available with split count 1, you will receive an error.
Regions
=======
The ``ice`` driver implements the following regions for accessing internal
device data.
.. list-table:: regions implemented
:widths: 15 85
* - Name
- Description
* - ``nvm-flash``
- The contents of the entire flash chip, sometimes referred to as
the device's Non Volatile Memory.
* - ``shadow-ram``
- The contents of the Shadow RAM, which is loaded from the beginning
of the flash. Although the contents are primarily from the flash,
this area also contains data generated during device boot which is
not stored in flash.
* - ``device-caps``
- The contents of the device firmware's capabilities buffer. Useful to
determine the current state and configuration of the device.
Both the ``nvm-flash`` and ``shadow-ram`` regions can be accessed without a
snapshot. The ``device-caps`` region requires a snapshot as the contents are
sent by firmware and can't be split into separate reads.
Users can request an immediate capture of a snapshot for all three regions
via the ``DEVLINK_CMD_REGION_NEW`` command.
.. code:: shell
$ devlink region show
pci/0000:01:00.0/nvm-flash: size 10485760 snapshot [] max 1
pci/0000:01:00.0/device-caps: size 4096 snapshot [] max 10
$ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
$ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
$ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
$ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 length 16
0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
$ devlink region delete pci/0000:01:00.0/nvm-flash snapshot 1
$ devlink region new pci/0000:01:00.0/device-caps snapshot 1
$ devlink region dump pci/0000:01:00.0/device-caps snapshot 1
0000000000000000 01 00 01 00 00 00 00 00 01 00 00 00 00 00 00 00
0000000000000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000020 02 00 02 01 32 03 00 00 0a 00 00 00 25 00 00 00
0000000000000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000040 04 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00
0000000000000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000060 05 00 01 00 03 00 00 00 00 00 00 00 00 00 00 00
0000000000000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000080 06 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00
0000000000000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000a0 08 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000c0 12 00 01 00 01 00 00 00 01 00 01 00 00 00 00 00
00000000000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000e0 13 00 01 00 00 01 00 00 00 00 00 00 00 00 00 00
00000000000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000100 14 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00
0000000000000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000120 15 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00
0000000000000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000140 16 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00
0000000000000150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000160 17 00 01 00 06 00 00 00 00 00 00 00 00 00 00 00
0000000000000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000180 18 00 01 00 01 00 00 00 01 00 00 00 08 00 00 00
0000000000000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000001a0 22 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00
00000000000001b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000001c0 40 00 01 00 00 08 00 00 08 00 00 00 00 00 00 00
00000000000001d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000001e0 41 00 01 00 00 08 00 00 00 00 00 00 00 00 00 00
00000000000001f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000200 42 00 01 00 00 08 00 00 00 00 00 00 00 00 00 00
0000000000000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
$ devlink region delete pci/0000:01:00.0/device-caps snapshot 1
Devlink Rate
============
The ``ice`` driver implements devlink-rate API. It allows for offload of
the Hierarchical QoS to the hardware. It enables user to group Virtual
Functions in a tree structure and assign supported parameters: tx_share,
tx_max, tx_priority and tx_weight to each node in a tree. So effectively
user gains an ability to control how much bandwidth is allocated for each
VF group. This is later enforced by the HW.
It is assumed that this feature is mutually exclusive with DCB performed
in FW and ADQ, or any driver feature that would trigger changes in QoS,
for example creation of the new traffic class. The driver will prevent DCB
or ADQ configuration if user started making any changes to the nodes using
devlink-rate API. To configure those features a driver reload is necessary.
Correspondingly if ADQ or DCB will get configured the driver won't export
hierarchy at all, or will remove the untouched hierarchy if those
features are enabled after the hierarchy is exported, but before any
changes are made.
This feature is also dependent on switchdev being enabled in the system.
It's required because devlink-rate requires devlink-port objects to be
present, and those objects are only created in switchdev mode.
If the driver is set to the switchdev mode, it will export internal
hierarchy the moment VF's are created. Root of the tree is always
represented by the node_0. This node can't be deleted by the user. Leaf
nodes and nodes with children also can't be deleted.
.. list-table:: Attributes supported
:widths: 15 85
* - Name
- Description
* - ``tx_max``
- maximum bandwidth to be consumed by the tree Node. Rate Limit is
an absolute number specifying a maximum amount of bytes a Node may
consume during the course of one second. Rate limit guarantees
that a link will not oversaturate the receiver on the remote end
and also enforces an SLA between the subscriber and network
provider.
* - ``tx_share``
- minimum bandwidth allocated to a tree node when it is not blocked.
It specifies an absolute BW. While tx_max defines the maximum
bandwidth the node may consume, the tx_share marks committed BW
for the Node.
* - ``tx_priority``
- allows for usage of strict priority arbiter among siblings. This
arbitration scheme attempts to schedule nodes based on their
priority as long as the nodes remain within their bandwidth limit.
Range 0-7. Nodes with priority 7 have the highest priority and are
selected first, while nodes with priority 0 have the lowest
priority. Nodes that have the same priority are treated equally.
* - ``tx_weight``
- allows for usage of Weighted Fair Queuing arbitration scheme among
siblings. This arbitration scheme can be used simultaneously with
the strict priority. Range 1-200. Only relative values matter for
arbitration.
``tx_priority`` and ``tx_weight`` can be used simultaneously. In that case
nodes with the same priority form a WFQ subgroup in the sibling group
and arbitration among them is based on assigned weights.
.. code:: shell
# enable switchdev
$ devlink dev eswitch set pci/0000:4b:00.0 mode switchdev
# at this point driver should export internal hierarchy
$ echo 2 > /sys/class/net/ens785np0/device/sriov_numvfs
$ devlink port function rate show
pci/0000:4b:00.0/node_25: type node parent node_24
pci/0000:4b:00.0/node_24: type node parent node_0
pci/0000:4b:00.0/node_32: type node parent node_31
pci/0000:4b:00.0/node_31: type node parent node_30
pci/0000:4b:00.0/node_30: type node parent node_16
pci/0000:4b:00.0/node_19: type node parent node_18
pci/0000:4b:00.0/node_18: type node parent node_17
pci/0000:4b:00.0/node_17: type node parent node_16
pci/0000:4b:00.0/node_14: type node parent node_5
pci/0000:4b:00.0/node_5: type node parent node_3
pci/0000:4b:00.0/node_13: type node parent node_4
pci/0000:4b:00.0/node_12: type node parent node_4
pci/0000:4b:00.0/node_11: type node parent node_4
pci/0000:4b:00.0/node_10: type node parent node_4
pci/0000:4b:00.0/node_9: type node parent node_4
pci/0000:4b:00.0/node_8: type node parent node_4
pci/0000:4b:00.0/node_7: type node parent node_4
pci/0000:4b:00.0/node_6: type node parent node_4
pci/0000:4b:00.0/node_4: type node parent node_3
pci/0000:4b:00.0/node_3: type node parent node_16
pci/0000:4b:00.0/node_16: type node parent node_15
pci/0000:4b:00.0/node_15: type node parent node_0
pci/0000:4b:00.0/node_2: type node parent node_1
pci/0000:4b:00.0/node_1: type node parent node_0
pci/0000:4b:00.0/node_0: type node
pci/0000:4b:00.0/1: type leaf parent node_25
pci/0000:4b:00.0/2: type leaf parent node_25
# let's create some custom node
$ devlink port function rate add pci/0000:4b:00.0/node_custom parent node_0
# second custom node
$ devlink port function rate add pci/0000:4b:00.0/node_custom_1 parent node_custom
# reassign second VF to newly created branch
$ devlink port function rate set pci/0000:4b:00.0/2 parent node_custom_1
# assign tx_weight to the VF
$ devlink port function rate set pci/0000:4b:00.0/2 tx_weight 5
# assign tx_share to the VF
$ devlink port function rate set pci/0000:4b:00.0/2 tx_share 500Mbps
@@ -0,0 +1,105 @@
Linux Devlink Documentation
===========================
devlink is an API to expose device information and resources not directly
related to any device class, such as chip-wide/switch-ASIC-wide configuration.
Locking
-------
Driver facing APIs are currently transitioning to allow more explicit
locking. Drivers can use the existing ``devlink_*`` set of APIs, or
new APIs prefixed by ``devl_*``. The older APIs handle all the locking
in devlink core, but don't allow registration of most sub-objects once
the main devlink object is itself registered. The newer ``devl_*`` APIs assume
the devlink instance lock is already held. Drivers can take the instance
lock by calling ``devl_lock()``. It is also held across all callbacks of
devlink netlink commands.
Drivers are encouraged to use the devlink instance lock for their own needs.
Drivers need to be cautious when taking devlink instance lock and
taking RTNL lock at the same time. Devlink instance lock needs to be taken
first, only after that RTNL lock could be taken.
Nested instances
----------------
Some objects, like linecards or port functions, could have another
devlink instances created underneath. In that case, drivers should make
sure to respect following rules:
- Lock ordering should be maintained. If driver needs to take instance
lock of both nested and parent instances at the same time, devlink
instance lock of the parent instance should be taken first, only then
instance lock of the nested instance could be taken.
- Driver should use object-specific helpers to setup the nested relationship
before registering the nested devlink instance:
- ``devl_nested_devlink_set()`` - called to setup devlink -> nested
devlink relationship (could be used for multiple nested instances).
- ``devl_port_fn_devlink_set()`` - called to setup port function ->
nested devlink relationship.
- ``devlink_linecard_nested_dl_set()`` - called to setup linecard ->
nested devlink relationship.
The nested devlink info is exposed to the userspace over object-specific
attributes of devlink netlink.
Interface documentation
-----------------------
The following pages describe various interfaces available through devlink in
general.
.. toctree::
:maxdepth: 1
devlink-dpipe
devlink-eswitch-attr
devlink-flash
devlink-health
devlink-info
devlink-linecard
devlink-params
devlink-port
devlink-region
devlink-reload
devlink-resource
devlink-selftests
devlink-trap
devlink-shared
Driver-specific documentation
-----------------------------
Each driver that implements ``devlink`` is expected to document what
parameters, info versions, and other features it supports.
.. toctree::
:maxdepth: 1
am65-nuss-cpsw-switch
bnxt
etas_es58x
hns3
i40e
ice
ionic
iosm
ixgbe
kvaser_pciefd
kvaser_usb
mlx4
mlx5
mlxsw
mv88e6xxx
netdevsim
nfp
octeontx2
prestera
qed
sfc
stmmac
ti-cpsw-switch
zl3073x
@@ -0,0 +1,29 @@
.. SPDX-License-Identifier: GPL-2.0
=====================
ionic devlink support
=====================
This document describes the devlink features implemented by the ``ionic``
device driver.
Info versions
=============
The ``ionic`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fw``
- running
- Version of firmware running on the device
* - ``asic.id``
- fixed
- The ASIC type for this device
* - ``asic.rev``
- fixed
- The revision of the ASIC for this device
@@ -0,0 +1,162 @@
.. SPDX-License-Identifier: GPL-2.0
====================
iosm devlink support
====================
This document describes the devlink features implemented by the ``iosm``
device driver.
Parameters
==========
The ``iosm`` driver implements the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``erase_full_flash``
- u8
- runtime
- erase_full_flash parameter is used to check if full erase is required for
the device during firmware flashing.
If set, Full nand erase command will be sent to the device. By default,
only conditional erase support is enabled.
Flash Update
============
The ``iosm`` driver implements support for flash update using the
``devlink-flash`` interface.
It supports updating the device flash using a combined flash image which contains
the Bootloader images and other modem software images.
The driver uses DEVLINK_SUPPORT_FLASH_UPDATE_COMPONENT to identify type of
firmware image that need to be flashed as requested by user space application.
Supported firmware image types.
.. list-table:: Firmware Image types
:widths: 15 85
* - Name
- Description
* - ``PSI RAM``
- Primary Signed Image
* - ``EBL``
- External Bootloader
* - ``FLS``
- Modem Software Image
PSI RAM and EBL are the RAM images which are injected to the device when the
device is in BOOT ROM stage. Once this is successful, the actual modem firmware
image is flashed to the device. The modem software image contains multiple files
each having one secure bin file and at least one Loadmap/Region file. For flashing
these files, appropriate commands are sent to the modem device along with the
data required for flashing. The data like region count and address of each region
has to be passed to the driver using the devlink param command.
If the device has to be fully erased before firmware flashing, user application
need to set the erase_full_flash parameter using devlink param command.
By default, conditional erase feature is supported.
Flash Commands:
===============
1) When modem is in Boot ROM stage, user can use below command to inject PSI RAM
image using devlink flash command.
$ devlink dev flash pci/0000:02:00.0 file <PSI_RAM_File_name>
2) If user want to do a full erase, below command need to be issued to set the
erase full flash param (To be set only if full erase required).
$ devlink dev param set pci/0000:02:00.0 name erase_full_flash value true cmode runtime
3) Inject EBL after the modem is in PSI stage.
$ devlink dev flash pci/0000:02:00.0 file <EBL_File_name>
4) Once EBL is injected successfully, then the actual firmware flashing takes
place. Below is the sequence of commands used for each of the firmware images.
a) Flash secure bin file.
$ devlink dev flash pci/0000:02:00.0 file <Secure_bin_file_name>
b) Flashing the Loadmap/Region file
$ devlink dev flash pci/0000:02:00.0 file <Load_map_file_name>
Regions
=======
The ``iosm`` driver supports dumping the coredump logs.
In case a firmware encounters an exception, a snapshot will be taken by the
driver. Following regions are accessed for device internal data.
.. list-table:: Regions implemented
:widths: 15 85
* - Name
- Description
* - ``report.json``
- The summary of exception details logged as part of this region.
* - ``coredump.fcd``
- This region contains the details related to the exception occurred in the
device (RAM dump).
* - ``cdd.log``
- This region contains the logs related to the modem CDD driver.
* - ``eeprom.bin``
- This region contains the eeprom logs.
* - ``bootcore_trace.bin``
- This region contains the current instance of bootloader logs.
* - ``bootcore_prev_trace.bin``
- This region contains the previous instance of bootloader logs.
Region commands
===============
$ devlink region show
$ devlink region new pci/0000:02:00.0/report.json
$ devlink region dump pci/0000:02:00.0/report.json snapshot 0
$ devlink region del pci/0000:02:00.0/report.json snapshot 0
$ devlink region new pci/0000:02:00.0/coredump.fcd
$ devlink region dump pci/0000:02:00.0/coredump.fcd snapshot 1
$ devlink region del pci/0000:02:00.0/coredump.fcd snapshot 1
$ devlink region new pci/0000:02:00.0/cdd.log
$ devlink region dump pci/0000:02:00.0/cdd.log snapshot 2
$ devlink region del pci/0000:02:00.0/cdd.log snapshot 2
$ devlink region new pci/0000:02:00.0/eeprom.bin
$ devlink region dump pci/0000:02:00.0/eeprom.bin snapshot 3
$ devlink region del pci/0000:02:00.0/eeprom.bin snapshot 3
$ devlink region new pci/0000:02:00.0/bootcore_trace.bin
$ devlink region dump pci/0000:02:00.0/bootcore_trace.bin snapshot 4
$ devlink region del pci/0000:02:00.0/bootcore_trace.bin snapshot 4
$ devlink region new pci/0000:02:00.0/bootcore_prev_trace.bin
$ devlink region dump pci/0000:02:00.0/bootcore_prev_trace.bin snapshot 5
$ devlink region del pci/0000:02:00.0/bootcore_prev_trace.bin snapshot 5
@@ -0,0 +1,171 @@
.. SPDX-License-Identifier: GPL-2.0
=====================
ixgbe devlink support
=====================
This document describes the devlink features implemented by the ``ixgbe``
device driver.
Info versions
=============
Any of the versions dealing with the security presented by ``devlink-info``
is purely informational. Devlink does not use a secure channel to communicate
with the device.
The ``ixgbe`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 5 90
* - Name
- Type
- Example
- Description
* - ``board.id``
- fixed
- H49289-000
- The Product Board Assembly (PBA) identifier of the board.
* - ``fw.undi``
- running
- 1.1937.0
- Version of the Option ROM containing the UEFI driver. The version is
reported in ``major.minor.patch`` format. The major version is
incremented whenever a major breaking change occurs, or when the
minor version would overflow. The minor version is incremented for
non-breaking changes and reset to 1 when the major version is
incremented. The patch version is normally 0 but is incremented when
a fix is delivered as a patch against an older base Option ROM.
* - ``fw.undi.srev``
- running
- 4
- Number indicating the security revision of the Option ROM.
* - ``fw.bundle_id``
- running
- 0x80000d0d
- Unique identifier of the firmware image file that was loaded onto
the device. Also referred to as the EETRACK identifier of the NVM.
* - ``fw.mgmt.api``
- running
- 1.5.1
- 3-digit version number (major.minor.patch) of the API exported over
the AdminQ by the management firmware. Used by the driver to
identify what commands are supported. Historical versions of the
kernel only displayed a 2-digit version number (major.minor).
* - ``fw.mgmt.build``
- running
- 0x305d955f
- Unique identifier of the source for the management firmware.
* - ``fw.mgmt.srev``
- running
- 3
- Number indicating the security revision of the firmware.
* - ``fw.psid.api``
- running
- 0.80
- Version defining the format of the flash contents.
* - ``fw.netlist``
- running
- 1.1.2000-6.7.0
- The version of the netlist module. This module defines the device's
Ethernet capabilities and default settings, and is used by the
management firmware as part of managing link and device
connectivity.
* - ``fw.netlist.build``
- running
- 0xee16ced7
- The first 4 bytes of the hash of the netlist module contents.
Flash Update
============
The ``ixgbe`` driver implements support for flash update using the
``devlink-flash`` interface. It supports updating the device flash using a
combined flash image that contains the ``fw.mgmt``, ``fw.undi``, and
``fw.netlist`` components.
.. list-table:: List of supported overwrite modes
:widths: 5 95
* - Bits
- Behavior
* - ``DEVLINK_FLASH_OVERWRITE_SETTINGS``
- Do not preserve settings stored in the flash components being
updated. This includes overwriting the port configuration that
determines the number of physical functions the device will
initialize with.
* - ``DEVLINK_FLASH_OVERWRITE_SETTINGS`` and ``DEVLINK_FLASH_OVERWRITE_IDENTIFIERS``
- Do not preserve either settings or identifiers. Overwrite everything
in the flash with the contents from the provided image, without
performing any preservation. This includes overwriting device
identifying fields such as the MAC address, Vital product Data (VPD) area,
and device serial number. It is expected that this combination be used with an
image customized for the specific device.
Reload
======
The ``ixgbe`` driver supports activating new firmware after a flash update
using ``DEVLINK_CMD_RELOAD`` with the ``DEVLINK_RELOAD_ACTION_FW_ACTIVATE``
action.
.. code:: shell
$ devlink dev reload pci/0000:01:00.0 reload action fw_activate
The new firmware is activated by issuing a device specific Embedded
Management Processor reset which requests the device to reset and reload the
EMP firmware image.
The driver does not currently support reloading the driver via
``DEVLINK_RELOAD_ACTION_DRIVER_REINIT``.
Regions
=======
The ``ixgbe`` driver implements the following regions for accessing internal
device data.
.. list-table:: regions implemented
:widths: 15 85
* - Name
- Description
* - ``nvm-flash``
- The contents of the entire flash chip, sometimes referred to as
the device's Non Volatile Memory.
* - ``shadow-ram``
- The contents of the Shadow RAM, which is loaded from the beginning
of the flash. Although the contents are primarily from the flash,
this area also contains data generated during device boot which is
not stored in flash.
* - ``device-caps``
- The contents of the device firmware's capabilities buffer. Useful to
determine the current state and configuration of the device.
Both the ``nvm-flash`` and ``shadow-ram`` regions can be accessed without a
snapshot. The ``device-caps`` region requires a snapshot as the contents are
sent by firmware and can't be split into separate reads.
Users can request an immediate capture of a snapshot for all three regions
via the ``DEVLINK_CMD_REGION_NEW`` command.
.. code:: shell
$ devlink region show
pci/0000:01:00.0/nvm-flash: size 10485760 snapshot [] max 1
pci/0000:01:00.0/device-caps: size 4096 snapshot [] max 10
$ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
$ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
$ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0 length 16
0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
$ devlink region delete pci/0000:01:00.0/device-caps snapshot 1
@@ -0,0 +1,24 @@
.. SPDX-License-Identifier: GPL-2.0
=============================
kvaser_pciefd devlink support
=============================
This document describes the devlink features implemented by the
``kvaser_pciefd`` device driver.
Info versions
=============
The ``kvaser_pciefd`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fw``
- running
- Version of the firmware running on the device. Also available
through ``ethtool -i`` as ``firmware-version``.
@@ -0,0 +1,33 @@
.. SPDX-License-Identifier: GPL-2.0
==========================
kvaser_usb devlink support
==========================
This document describes the devlink features implemented by the
``kvaser_usb`` device driver.
Info versions
=============
The ``kvaser_usb`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fw``
- running
- Version of the firmware running on the device. Also available
through ``ethtool -i`` as ``firmware-version``.
* - ``board.rev``
- fixed
- The device hardware revision.
* - ``board.id``
- fixed
- The device EAN (product number).
* - ``serial_number``
- fixed
- The device serial number.
@@ -0,0 +1,56 @@
.. SPDX-License-Identifier: GPL-2.0
====================
mlx4 devlink support
====================
This document describes the devlink features implemented by the ``mlx4``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
* - Name
- Mode
* - ``internal_err_reset``
- driverinit, runtime
* - ``max_macs``
- driverinit
* - ``region_snapshot_enable``
- driverinit, runtime
The ``mlx4`` driver also implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``enable_64b_cqe_eqe``
- Boolean
- driverinit
- Enable 64 byte CQEs/EQEs, if the FW supports it.
* - ``enable_4k_uar``
- Boolean
- driverinit
- Enable using the 4k UAR.
The ``mlx4`` driver supports reloading via ``DEVLINK_CMD_RELOAD``
Regions
=======
The ``mlx4`` driver supports dumping the firmware PCI crspace and health
buffer during a critical firmware issue.
In case a firmware command times out, firmware getting stuck, or a non zero
value on the catastrophic buffer, a snapshot will be taken by the driver.
The ``cr-space`` region will contain the firmware PCI crspace contents. The
``fw-health`` region will contain the device firmware's health buffer.
Snapshots for both of these regions are taken on the same event triggers.
@@ -0,0 +1,421 @@
.. SPDX-License-Identifier: GPL-2.0
====================
mlx5 devlink support
====================
This document describes the devlink features implemented by the ``mlx5``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
* - Name
- Mode
- Validation
- Notes
* - ``enable_roce``
- driverinit
- Boolean
- If the device supports RoCE disablement, RoCE enablement state controls
device support for RoCE capability. Otherwise, the control occurs in the
driver stack. When RoCE is disabled at the driver level, only raw
ethernet QPs are supported.
* - ``io_eq_size``
- driverinit
- The range is between 64 and 4096.
-
* - ``event_eq_size``
- driverinit
- The range is between 64 and 4096.
-
* - ``max_macs``
- driverinit
- The range is between 1 and 2^31. Only power of 2 values are supported.
-
* - ``enable_sriov``
- permanent
- Boolean
- Applies to each physical function (PF) independently, if the device
supports it. Otherwise, it applies symmetrically to all PFs.
* - ``total_vfs``
- permanent
- The range is between 1 and a device-specific max.
- Applies to each physical function (PF) independently, if the device
supports it. Otherwise, it applies symmetrically to all PFs.
Note: permanent parameters such as ``enable_sriov`` and ``total_vfs`` require FW reset to take effect
.. code-block:: bash
# setup parameters
devlink dev param set pci/0000:01:00.0 name enable_sriov value true cmode permanent
devlink dev param set pci/0000:01:00.0 name total_vfs value 8 cmode permanent
# Fw reset
devlink dev reload pci/0000:01:00.0 action fw_activate
# for PCI related config such as sriov PCI reset/rescan is required:
echo 1 >/sys/bus/pci/devices/0000:01:00.0/remove
echo 1 >/sys/bus/pci/rescan
grep ^ /sys/bus/pci/devices/0000:01:00.0/sriov_*
* - ``num_doorbells``
- driverinit
- This controls the number of channel doorbells used by the netdev. In all
cases, an additional doorbell is allocated and used for non-channel
communication (e.g. for PTP, HWS, etc.). Supported values are:
- 0: No channel-specific doorbells, use the global one for everything.
- [1, max_num_channels]: Spread netdev channels equally across these
doorbells.
The ``mlx5`` driver also implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``flow_steering_mode``
- string
- runtime
- Controls the flow steering mode of the driver
* ``dmfs`` Device managed flow steering. In DMFS mode, the HW
steering entities are created and managed through firmware.
* ``smfs`` Software managed flow steering. In SMFS mode, the HW
steering entities are created and manage through the driver without
firmware intervention.
* ``hmfs`` Hardware managed flow steering. In HMFS mode, the driver
is configuring steering rules directly to the HW using Work Queues with
a special new type of WQE (Work Queue Element).
SMFS mode is faster and provides better rule insertion rate compared to
default DMFS mode.
* - ``fdb_large_groups``
- u32
- driverinit
- Control the number of large groups (size > 1) in the FDB table.
* The default value is 15, and the range is between 1 and 1024.
* - ``esw_multiport``
- Boolean
- runtime
- Control MultiPort E-Switch shared fdb mode.
An experimental mode where a single E-Switch is used and all the vports
and physical ports on the NIC are connected to it.
An example is to send traffic from a VF that is created on PF0 to an
uplink that is natively associated with the uplink of PF1
Note: Future devices, ConnectX-8 and onward, will eventually have this
as the default to allow forwarding between all NIC ports in a single
E-switch environment and the dual E-switch mode will likely get
deprecated.
Default: disabled
* - ``esw_port_metadata``
- Boolean
- runtime
- When applicable, disabling eswitch metadata can increase packet rate up
to 20% depending on the use case and packet sizes.
Eswitch port metadata state controls whether to internally tag packets
with metadata. Metadata tagging must be enabled for multi-port RoCE,
failover between representors and stacked devices. By default metadata is
enabled on the supported devices in E-switch. Metadata is applicable only
for E-switch in switchdev mode and users may disable it when NONE of the
below use cases will be in use:
1. HCA is in Dual/multi-port RoCE mode.
2. VF/SF representor bonding (Usually used for Live migration)
3. Stacked devices
When metadata is disabled, the above use cases will fail to initialize if
users try to enable them.
Note: Setting this parameter does not take effect immediately. Setting
must happen in legacy mode and eswitch port metadata takes effect after
enabling switchdev mode.
* - ``hairpin_num_queues``
- u32
- driverinit
- We refer to a TC NIC rule that involves forwarding as "hairpin".
Hairpin queues are mlx5 hardware specific implementation for hardware
forwarding of such packets.
Control the number of hairpin queues.
* - ``hairpin_queue_size``
- u32
- driverinit
- Control the size (in packets) of the hairpin queues.
* - ``pcie_cong_inbound_high``
- u16
- driverinit
- High threshold configuration for PCIe congestion events. The firmware
will send an event once device side inbound PCIe traffic went
above the configured high threshold for a long enough period (at least
200ms).
See pci_bw_inbound_high ethtool stat.
Units are 0.01 %. Accepted values are in range [0, 10000].
pcie_cong_inbound_low < pcie_cong_inbound_high.
Default value: 9000 (Corresponds to 90%).
* - ``pcie_cong_inbound_low``
- u16
- driverinit
- Low threshold configuration for PCIe congestion events. The firmware
will send an event once device side inbound PCIe traffic went
below the configured low threshold, only after having been previously in
a congested state.
See pci_bw_inbound_low ethtool stat.
Units are 0.01 %. Accepted values are in range [0, 10000].
pcie_cong_inbound_low < pcie_cong_inbound_high.
Default value: 7500.
* - ``pcie_cong_outbound_high``
- u16
- driverinit
- High threshold configuration for PCIe congestion events. The firmware
will send an event once device side outbound PCIe traffic went
above the configured high threshold for a long enough period (at least
200ms).
See pci_bw_outbound_high ethtool stat.
Units are 0.01 %. Accepted values are in range [0, 10000].
pcie_cong_outbound_low < pcie_cong_outbound_high.
Default value: 9000 (Corresponds to 90%).
* - ``pcie_cong_outbound_low``
- u16
- driverinit
- Low threshold configuration for PCIe congestion events. The firmware
will send an event once device side outbound PCIe traffic went
below the configured low threshold, only after having been previously in
a congested state.
See pci_bw_outbound_low ethtool stat.
Units are 0.01 %. Accepted values are in range [0, 10000].
pcie_cong_outbound_low < pcie_cong_outbound_high.
Default value: 7500.
* - ``cqe_compress_type``
- string
- permanent
- Configure which mechanism/algorithm should be used by the NIC that will
affect the rate (aggressiveness) of compressed CQEs depending on PCIe bus
conditions and other internal NIC factors. This mode affects all queues
that enable compression.
* ``balanced`` : Merges fewer CQEs, resulting in a moderate compression ratio but maintaining a balance between bandwidth savings and performance
* ``aggressive`` : Merges more CQEs into a single entry, achieving a higher compression rate and maximizing performance, particularly under high traffic loads
* - ``swp_l4_csum_mode``
- string
- permanent
- Configure how the L4 checksum is calculated by the device when using
Software Parser (SWP) hints for header locations.
* ``default`` : Use the device's default checksum calculation
mode. The driver will discover during init whether or
full_csum or l4_only is in use. Setting this value explicitly
from userspace is not allowed, but some firmware versions may
return this value on param read.
* ``full_csum`` : Calculate full checksum including the pseudo-header
* ``l4_only`` : Calculate L4-only checksum, excluding the pseudo-header
The ``mlx5`` driver supports reloading via ``DEVLINK_CMD_RELOAD``
Info versions
=============
The ``mlx5`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fw.psid``
- fixed
- Used to represent the board id of the device.
* - ``fw.version``
- stored, running
- Three digit major.minor.subminor firmware version number.
Health reporters
================
tx reporter
-----------
The tx reporter is responsible for reporting and recovering of the following three error scenarios:
- tx timeout
Report on kernel tx timeout detection.
Recover by searching lost interrupts.
- tx error completion
Report on error tx completion.
Recover by flushing the tx queue and reset it.
- tx PTP port timestamping CQ unhealthy
Report too many CQEs never delivered on port ts CQ.
Recover by flushing and re-creating all PTP channels.
tx reporter also support on demand diagnose callback, on which it provides
real time information of its send queues status.
User commands examples:
- Diagnose send queues status::
$ devlink health diagnose pci/0000:82:00.0 reporter tx
.. note::
This command has valid output only when interface is up, otherwise the command has empty output.
- Show number of tx errors indicated, number of recover flows ended successfully,
is autorecover enabled and graceful period from last recover::
$ devlink health show pci/0000:82:00.0 reporter tx
rx reporter
-----------
The rx reporter is responsible for reporting and recovering of the following two error scenarios:
- rx queues' initialization (population) timeout
Population of rx queues' descriptors on ring initialization is done
in napi context via triggering an irq. In case of a failure to get
the minimum amount of descriptors, a timeout would occur, and
descriptors could be recovered by polling the EQ (Event Queue).
- rx completions with errors (reported by HW on interrupt context)
Report on rx completion error.
Recover (if needed) by flushing the related queue and reset it.
rx reporter also supports on demand diagnose callback, on which it
provides real time information of its receive queues' status.
- Diagnose rx queues' status and corresponding completion queue::
$ devlink health diagnose pci/0000:82:00.0 reporter rx
.. note::
This command has valid output only when interface is up. Otherwise, the command has empty output.
- Show number of rx errors indicated, number of recover flows ended successfully,
is autorecover enabled, and graceful period from last recover::
$ devlink health show pci/0000:82:00.0 reporter rx
fw reporter
-----------
The fw reporter implements `diagnose` and `dump` callbacks.
It follows symptoms of fw error such as fw syndrome by triggering
fw core dump and storing it into the dump buffer.
The fw reporter diagnose command can be triggered any time by the user to check
current fw status.
User commands examples:
- Check fw heath status::
$ devlink health diagnose pci/0000:82:00.0 reporter fw
- Read FW core dump if already stored or trigger new one::
$ devlink health dump show pci/0000:82:00.0 reporter fw
.. note::
This command can run only on the PF which has fw tracer ownership,
running it on other PF or any VF will return "Operation not permitted".
fw fatal reporter
-----------------
The fw fatal reporter implements `dump` and `recover` callbacks.
It follows fatal errors indications by CR-space dump and recover flow.
The CR-space dump uses vsc interface which is valid even if the FW command
interface is not functional, which is the case in most FW fatal errors.
The recover function runs recover flow which reloads the driver and triggers fw
reset if needed.
On firmware error, the health buffer is dumped into the dmesg. The log
level is derived from the error's severity (given in health buffer).
User commands examples:
- Run fw recover flow manually::
$ devlink health recover pci/0000:82:00.0 reporter fw_fatal
- Read FW CR-space dump if already stored or trigger new one::
$ devlink health dump show pci/0000:82:00.1 reporter fw_fatal
.. note::
This command can run only on PF.
vnic reporter
-------------
The vnic reporter implements only the `diagnose` callback.
It is responsible for querying the vnic diagnostic counters from fw and displaying
them in realtime.
Description of the vnic counters:
- total_error_queues
number of queues in an error state due to
an async error or errored command.
- send_queue_priority_update_flow
number of QP/SQ priority/SL update events.
- cq_overrun
number of times CQ entered an error state due to an overflow.
- async_eq_overrun
number of times an EQ mapped to async events was overrun.
- comp_eq_overrun
number of times an EQ mapped to completion events was
overrun.
- quota_exceeded_command
number of commands issued and failed due to quota exceeded.
- invalid_command
number of commands issued and failed dues to any reason other than quota
exceeded.
- nic_receive_steering_discard
number of packets that completed RX flow
steering but were discarded due to a mismatch in flow table.
- generated_pkt_steering_fail
number of packets generated by the VNIC experiencing unexpected steering
failure (at any point in steering flow).
- handled_pkt_steering_fail
number of packets handled by the VNIC experiencing unexpected steering
failure (at any point in steering flow owned by the VNIC, including the FDB
for the eswitch owner).
- icm_consumption
amount of Interconnect Host Memory (ICM) consumed by the vnic in
granularity of 4KB. ICM is host memory allocated by SW upon HCA request
and is used for storing data structures that control HCA operation.
- bar_uar_access
number of WRITE or READ access operations to the UAR on the PCIe BAR.
- odp_local_triggered_page_fault
number of locally-triggered page-faults due to ODP.
- odp_remote_triggered_page_fault
number of remotly-triggered page-faults due to ODP.
User commands examples:
- Diagnose PF/VF vnic counters::
$ devlink health diagnose pci/0000:82:00.1 reporter vnic
- Diagnose representor vnic counters (performed by supplying devlink port of the
representor, which can be obtained via devlink port command)::
$ devlink health diagnose pci/0000:82:00.1/65537 reporter vnic
.. note::
This command can run over all interfaces such as PF/VF and representor ports.
@@ -0,0 +1,105 @@
.. SPDX-License-Identifier: GPL-2.0
=====================
mlxsw devlink support
=====================
This document describes the devlink features implemented by the ``mlxsw``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
* - Name
- Mode
* - ``fw_load_policy``
- driverinit
The ``mlxsw`` driver also implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``acl_region_rehash_interval``
- u32
- runtime
- Sets an interval for periodic ACL region rehashes. The value is
specified in milliseconds, with a minimum of ``3000``. The value of
``0`` disables periodic work entirely. The first rehash will be run
immediately after the value is set.
The ``mlxsw`` driver supports reloading via ``DEVLINK_CMD_RELOAD``
Info versions
=============
The ``mlxsw`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``hw.revision``
- fixed
- The hardware revision for this board
* - ``fw.psid``
- fixed
- Firmware PSID
* - ``fw.version``
- running
- Three digit firmware version
Line card auxiliary device info versions
========================================
The ``mlxsw`` driver reports the following versions for line card auxiliary device
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``hw.revision``
- fixed
- The hardware revision for this line card
* - ``ini.version``
- running
- Version of line card INI loaded
* - ``fw.psid``
- fixed
- Line card device PSID
* - ``fw.version``
- running
- Three digit firmware version of line card device
Driver-specific Traps
=====================
.. list-table:: List of Driver-specific Traps Registered by ``mlxsw``
:widths: 5 5 90
* - Name
- Type
- Description
* - ``irif_disabled``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed from a disabled router interface (RIF). This can happen during
RIF dismantle, when the RIF is first disabled before being removed
completely
* - ``erif_disabled``
- ``drop``
- Traps packets that the device decided to drop because they need to be
routed through a disabled router interface (RIF). This can happen during
RIF dismantle, when the RIF is first disabled before being removed
completely
@@ -0,0 +1,28 @@
.. SPDX-License-Identifier: GPL-2.0
=========================
mv88e6xxx devlink support
=========================
This document describes the devlink features implemented by the ``mv88e6xxx``
device driver.
Parameters
==========
The ``mv88e6xxx`` driver implements the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``ATU_hash``
- u8
- runtime
- Select one of four possible hashing algorithms for MAC addresses in
the Address Translation Unit. A value of 3 may work better than the
default of 1 when many MAC addresses have the same OUI. Only the
values 0 to 3 are valid for this parameter.
@@ -0,0 +1,99 @@
.. SPDX-License-Identifier: GPL-2.0
=========================
netdevsim devlink support
=========================
This document describes the ``devlink`` features supported by the
``netdevsim`` device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
* - Name
- Mode
* - ``max_macs``
- driverinit
The ``netdevsim`` driver also implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``test1``
- Boolean
- driverinit
- Test parameter used to show how a driver-specific devlink parameter
can be implemented.
The ``netdevsim`` driver supports reloading via ``DEVLINK_CMD_RELOAD``
Regions
=======
The ``netdevsim`` driver exposes a ``dummy`` region as an example of how the
devlink-region interfaces work. A snapshot is taken whenever the
``take_snapshot`` debugfs file is written to.
Resources
=========
The ``netdevsim`` driver exposes resources to control the number of FIB
entries, FIB rule entries and nexthops that the driver will allow.
.. code:: shell
$ devlink resource set netdevsim/netdevsim0 path /IPv4/fib size 96
$ devlink resource set netdevsim/netdevsim0 path /IPv4/fib-rules size 16
$ devlink resource set netdevsim/netdevsim0 path /IPv6/fib size 64
$ devlink resource set netdevsim/netdevsim0 path /IPv6/fib-rules size 16
$ devlink resource set netdevsim/netdevsim0 path /nexthops size 16
$ devlink dev reload netdevsim/netdevsim0
Rate objects
============
The ``netdevsim`` driver supports rate objects management, which includes:
- registering/unregistering leaf rate objects per VF devlink port;
- creation/deletion node rate objects;
- setting tx_share and tx_max rate values for any rate object type;
- setting parent node for any rate object type.
Rate nodes and their parameters are exposed in ``netdevsim`` debugfs in RO mode.
For example created rate node with name ``some_group``:
.. code:: shell
$ ls /sys/kernel/debug/netdevsim/netdevsim0/rate_groups/some_group
rate_parent tx_max tx_share
Same parameters are exposed for leaf objects in corresponding ports directories.
For ex.:
.. code:: shell
$ ls /sys/kernel/debug/netdevsim/netdevsim0/ports/1
dev ethtool rate_parent tx_max tx_share
Driver-specific Traps
=====================
.. list-table:: List of Driver-specific Traps Registered by ``netdevsim``
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fid_miss``
- ``exception``
- When a packet enters the device it is classified to a filtering
identifier (FID) based on the ingress port and VLAN. This trap is used
to trap packets for which a FID could not be found
@@ -0,0 +1,68 @@
.. SPDX-License-Identifier: GPL-2.0
===================
nfp devlink support
===================
This document describes the devlink features implemented by the ``nfp``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
* - Name
- Mode
* - ``fw_load_policy``
- permanent
* - ``reset_dev_on_drv_probe``
- permanent
Info versions
=============
The ``nfp`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``board.id``
- fixed
- Identifier of the board design
* - ``board.rev``
- fixed
- Revision of the board design
* - ``board.manufacture``
- fixed
- Vendor of the board design
* - ``board.model``
- fixed
- Model name of the board design
* - ``board.part_number``
- fixed
- Part number of the board and its components
* - ``fw.bundle_id``
- stored, running
- Firmware bundle id
* - ``fw.mgmt``
- stored, running
- Version of the management firmware
* - ``fw.cpld``
- stored, running
- The CPLD firmware component version
* - ``fw.app``
- stored, running
- The APP firmware component version
* - ``fw.undi``
- stored, running
- The UNDI firmware component version
* - ``fw.ncsi``
- stored, running
- The NSCI firmware component version
* - ``chip.init``
- stored, running
- The CFGR firmware component version
@@ -0,0 +1,79 @@
.. SPDX-License-Identifier: GPL-2.0
=========================
octeontx2 devlink support
=========================
This document describes the devlink features implemented by the ``octeontx2 AF, PF and VF``
device drivers.
Parameters
==========
The ``octeontx2 PF and VF`` drivers implement the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``mcam_count``
- u16
- runtime
- Select number of match CAM entries to be allocated for an interface.
The same is used for ntuple filters of the interface. Supported by
PF and VF drivers.
The ``octeontx2 AF`` driver implements the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``dwrr_mtu``
- u32
- runtime
- Use to set the quantum which hardware uses for scheduling among transmit queues.
Hardware uses weighted DWRR algorithm to schedule among all transmit queues.
* - ``npc_mcam_high_zone_percent``
- u8
- runtime
- Use to set the number of high priority zone entries in NPC MCAM that can be allocated
by a user, out of the three priority zone categories high, mid and low.
* - ``npc_def_rule_cntr``
- bool
- runtime
- Use to enable or disable hit counters for the default rules in NPC MCAM.
Its not guaranteed that counters gets enabled and mapped to all the default rules,
since the counters are scarce and driver follows a best effort approach.
The default rule serves as the primary packet steering rule for a specific PF or VF,
based on its DMAC address which is installed by AF driver as part of its initialization.
Sample command to read hit counters for default rule from debugfs is as follows,
cat /sys/kernel/debug/cn10k/npc/mcam_rules
* - ``nix_maxlf``
- u16
- runtime
- Use to set the maximum number of LFs in NIX hardware block. This would be useful
to increase the availability of default resources allocated to enabled LFs like
MCAM entries for example.
The ``octeontx2 PF`` driver implements the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``unicast_filter_count``
- u8
- runtime
- Set the maximum number of unicast filters that can be programmed for
the device. This can be used to achieve better device resource
utilization, avoiding over consumption of unused MCAM table entries.
@@ -0,0 +1,141 @@
.. SPDX-License-Identifier: GPL-2.0
========================
prestera devlink support
========================
This document describes the devlink features implemented by the ``prestera``
device driver.
Driver-specific Traps
=====================
.. list-table:: List of Driver-specific Traps Registered by ``prestera``
:widths: 5 5 90
* - Name
- Type
- Description
.. list-table:: List of Driver-specific Traps Registered by ``prestera``
:widths: 5 5 90
* - Name
- Type
- Description
* - ``arp_bc``
- ``trap``
- Traps ARP broadcast packets (both requests/responses)
* - ``is_is``
- ``trap``
- Traps IS-IS packets
* - ``ospf``
- ``trap``
- Traps OSPF packets
* - ``ip_bc_mac``
- ``trap``
- Traps IPv4 packets with broadcast DA Mac address
* - ``stp``
- ``trap``
- Traps STP BPDU
* - ``lacp``
- ``trap``
- Traps LACP packets
* - ``lldp``
- ``trap``
- Traps LLDP packets
* - ``router_mc``
- ``trap``
- Traps multicast packets
* - ``vrrp``
- ``trap``
- Traps VRRP packets
* - ``dhcp``
- ``trap``
- Traps DHCP packets
* - ``mtu_error``
- ``trap``
- Traps (exception) packets that exceeded port's MTU
* - ``mac_to_me``
- ``trap``
- Traps packets with switch-port's DA Mac address
* - ``ttl_error``
- ``trap``
- Traps (exception) IPv4 packets whose TTL exceeded
* - ``ipv4_options``
- ``trap``
- Traps (exception) packets due to the malformed IPV4 header options
* - ``ip_default_route``
- ``trap``
- Traps packets that have no specific IP interface (IP to me) and no forwarding prefix
* - ``local_route``
- ``trap``
- Traps packets that have been send to one of switch IP interfaces addresses
* - ``ipv4_icmp_redirect``
- ``trap``
- Traps (exception) IPV4 ICMP redirect packets
* - ``arp_response``
- ``trap``
- Traps ARP replies packets that have switch-port's DA Mac address
* - ``acl_code_0``
- ``trap``
- Traps packets that have ACL priority set to 0 (tc pref 0)
* - ``acl_code_1``
- ``trap``
- Traps packets that have ACL priority set to 1 (tc pref 1)
* - ``acl_code_2``
- ``trap``
- Traps packets that have ACL priority set to 2 (tc pref 2)
* - ``acl_code_3``
- ``trap``
- Traps packets that have ACL priority set to 3 (tc pref 3)
* - ``acl_code_4``
- ``trap``
- Traps packets that have ACL priority set to 4 (tc pref 4)
* - ``acl_code_5``
- ``trap``
- Traps packets that have ACL priority set to 5 (tc pref 5)
* - ``acl_code_6``
- ``trap``
- Traps packets that have ACL priority set to 6 (tc pref 6)
* - ``acl_code_7``
- ``trap``
- Traps packets that have ACL priority set to 7 (tc pref 7)
* - ``ipv4_bgp``
- ``trap``
- Traps IPv4 BGP packets
* - ``ssh``
- ``trap``
- Traps SSH packets
* - ``telnet``
- ``trap``
- Traps Telnet packets
* - ``icmp``
- ``trap``
- Traps ICMP packets
* - ``rxdma_drop``
- ``drop``
- Drops packets (RxDMA) due to the lack of ingress buffers etc.
* - ``port_no_vlan``
- ``drop``
- Drops packets due to faulty-configured network or due to internal bug (config issue).
* - ``local_port``
- ``drop``
- Drops packets whose decision (FDB entry) is to bridge packet back to the incoming port/trunk.
* - ``invalid_sa``
- ``drop``
- Drops packets with multicast source MAC address.
* - ``illegal_ip_addr``
- ``drop``
- Drops packets with illegal SIP/DIP multicast/unicast addresses.
* - ``illegal_ipv4_hdr``
- ``drop``
- Drops packets with illegal IPV4 header.
* - ``ip_uc_dip_da_mismatch``
- ``drop``
- Drops packets with destination MAC being unicast, but destination IP address being multicast.
* - ``ip_sip_is_zero``
- ``drop``
- Drops packets with zero (0) IPV4 source address.
* - ``met_red``
- ``drop``
- Drops non-conforming packets (dropped by Ingress policer, metering drop), e.g. packet rate exceeded configured bandwidth.
@@ -0,0 +1,26 @@
.. SPDX-License-Identifier: GPL-2.0
===================
qed devlink support
===================
This document describes the devlink features implemented by the ``qed`` core
device driver.
Parameters
==========
The ``qed`` driver implements the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``iwarp_cmt``
- Boolean
- runtime
- Enable iWARP functionality for 100g devices. Note that this impacts
L2 performance, and is therefore not enabled by default.
@@ -0,0 +1,71 @@
.. SPDX-License-Identifier: GPL-2.0
===================
sfc devlink support
===================
This document describes the devlink features implemented by the ``sfc``
device driver for the ef10 and ef100 devices.
Info versions
=============
The ``sfc`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 90
* - Name
- Type
- Description
* - ``fw.bundle_id``
- stored
- Version of the firmware "bundle" image that was last used to update
multiple components.
* - ``fw.mgmt.suc``
- running
- For boards where the management function is split between multiple
control units, this is the SUC control unit's firmware version.
* - ``fw.mgmt.cmc``
- running
- For boards where the management function is split between multiple
control units, this is the CMC control unit's firmware version.
* - ``fpga.rev``
- running
- FPGA design revision.
* - ``fpga.app``
- running
- Datapath programmable logic version.
* - ``fw.app``
- running
- Datapath software/microcode/firmware version.
* - ``coproc.boot``
- running
- SmartNIC application co-processor (APU) first stage boot loader version.
* - ``coproc.uboot``
- running
- SmartNIC application co-processor (APU) co-operating system loader version.
* - ``coproc.main``
- running
- SmartNIC application co-processor (APU) main operating system version.
* - ``coproc.recovery``
- running
- SmartNIC application co-processor (APU) recovery operating system version.
* - ``fw.exprom``
- running
- Expansion ROM version. For boards where the expansion ROM is split between
multiple images (e.g. PXE and UEFI), this is the specifically the PXE boot
ROM version.
* - ``fw.uefi``
- running
- UEFI driver version (No UNDI support).
Flash Update
============
The ``sfc`` driver implements support for flash update using the
``devlink-flash`` interface. It supports updating the device flash using a
combined flash image ("bundle") that contains multiple components (on ef10,
typically ``fw.mgmt``, ``fw.app``, ``fw.exprom`` and ``fw.uefi``).
The driver does not support any overwrite mask flags.
@@ -0,0 +1,40 @@
.. SPDX-License-Identifier: GPL-2.0
=======================================
stmmac (synopsys dwmac) devlink support
=======================================
This document describes the devlink features implemented by the ``stmmac``
device driver.
Parameters
==========
The ``stmmac`` driver implements the following driver-specific parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``phc_coarse_adj``
- Boolean
- runtime
- Enable the Coarse timestamping mode, as defined in the DWMAC TRM.
A detailed explanation of this timestamping mode can be found in the
Socfpga Functional Description [1].
In Coarse mode, the ptp clock is expected to be fed by a high-precision
clock that is externally adjusted, and the subsecond increment used for
timestamping is set to 1/ptp_clock_rate.
In Fine mode (i.e. Coarse mode == false), the ptp clock frequency is
continuously adjusted, but the subsecond increment is set to
2/ptp_clock_rate.
Coarse mode is suitable for PTP Grand Master operation. If unsure, leave
the parameter to False.
[1] https://www.intel.com/content/www/us/en/docs/programmable/683126/21-2/functional-description-of-the-emac.html
@@ -0,0 +1,31 @@
.. SPDX-License-Identifier: GPL-2.0
==============================
ti-cpsw-switch devlink support
==============================
This document describes the devlink features implemented by the ``ti-cpsw-switch``
device driver.
Parameters
==========
The ``ti-cpsw-switch`` driver implements the following driver-specific
parameters.
.. list-table:: Driver-specific parameters implemented
:widths: 5 5 5 85
* - Name
- Type
- Mode
- Description
* - ``ale_bypass``
- Boolean
- runtime
- Enables ALE_CONTROL(4).BYPASS mode for debugging purposes. In this
mode, all packets will be sent to the host port only.
* - ``switch_mode``
- Boolean
- runtime
- Enable switch mode
@@ -0,0 +1,65 @@
.. SPDX-License-Identifier: GPL-2.0
=======================
zl3073x devlink support
=======================
This document describes the devlink features implemented by the ``zl3073x``
device driver.
Parameters
==========
.. list-table:: Generic parameters implemented
:widths: 5 5 90
* - Name
- Mode
- Notes
* - ``clock_id``
- driverinit
- Set the clock ID that is used by the driver for registering DPLL devices
and pins.
Info versions
=============
The ``zl3073x`` driver reports the following versions
.. list-table:: devlink info versions implemented
:widths: 5 5 5 90
* - Name
- Type
- Example
- Description
* - ``asic.id``
- fixed
- 1E94
- Chip identification number
* - ``asic.rev``
- fixed
- 300
- Chip revision number
* - ``fw``
- running
- 7006
- Firmware version number
* - ``custom_cfg``
- running
- 1.3.0.1
- Device configuration version customized by OEM
Flash Update
============
The ``zl3073x`` driver implements support for flash update using the
``devlink-flash`` interface. It supports updating the device flash using a
combined flash image ("bundle") that contains multiple components (firmware
parts and configurations).
During the flash procedure, the standard firmware interface is not available,
so the driver unregisters all DPLLs and associated pins, and re-registers them
once the flash procedure is complete.
The driver does not support any overwrite mask flags.