lastdragon/linux-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Resguardo inicial

Browse Source
This commit is contained in:
lastdragon
2026-09-06 12:53:21 -06:00
parent 662efe3c2a
commit ad2088d178
94853 changed files with 43200404 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
linux-7.2.3/Documentation/admin-guide/mm/damon/index.rst
+17
View File
@@ -0,0 +1,17 @@
.. SPDX-License-Identifier: GPL-2.0
================================================================
DAMON: Data Access MONitoring and Access-aware System Operations
================================================================
:doc:`DAMON </mm/damon/index>` is a Linux kernel subsystem for efficient data
access monitoring and access-aware system operations.
.. toctree::
:maxdepth: 2
start
usage
reclaim
lru_sort
stat
linux-7.2.3/Documentation/admin-guide/mm/damon/lru_sort.rst
+368
View File
@@ -0,0 +1,368 @@
.. SPDX-License-Identifier: GPL-2.0
=============================
DAMON-based LRU-lists Sorting
=============================
DAMON-based LRU-lists Sorting (DAMON_LRU_SORT) is a static kernel module that
aimed to be used for proactive and lightweight data access pattern based
(de)prioritization of pages on their LRU-lists for making LRU-lists a more
trusworthy data access pattern source.
Where Proactive LRU-lists Sorting is Required?
==============================================
As page-granularity access checking overhead could be significant on huge
systems, LRU lists are normally not proactively sorted but partially and
reactively sorted for special events including specific user requests, system
calls and memory pressure. As a result, LRU lists are sometimes not so
perfectly prepared to be used as a trustworthy access pattern source for some
situations including reclamation target pages selection under sudden memory
pressure.
Because DAMON can identify access patterns of best-effort accuracy while
inducing only user-specified range of overhead, proactively running
DAMON_LRU_SORT could be helpful for making LRU lists more trustworthy access
pattern source with low and controlled overhead.
How It Works?
=============
DAMON_LRU_SORT finds hot pages (pages of memory regions that showing access
rates that higher than a user-specified threshold) and cold pages (pages of
memory regions that showing no access for a time that longer than a
user-specified threshold) using DAMON, and prioritizes hot pages while
deprioritizing cold pages on their LRU-lists. To avoid it consuming too much
CPU for the prioritizations, a CPU time usage limit can be configured. Under
the limit, it prioritizes and deprioritizes more hot and cold pages first,
respectively. System administrators can also configure under what situation
this scheme should automatically activated and deactivated with three memory
pressure watermarks.
Its default parameters for hotness/coldness thresholds and CPU quota limit are
conservatively chosen. That is, the module under its default parameters could
be widely used without harm for common situations while providing a level of
benefits for systems having clear hot/cold access patterns under memory
pressure while consuming only a limited small portion of CPU time.
Interface: Module Parameters
============================
To use this feature, you should first ensure your system is running on a kernel
that is built with ``CONFIG_DAMON_LRU_SORT=y``.
To let sysadmins enable or disable it and tune for the given system,
DAMON_LRU_SORT utilizes module parameters. That is, you can put
``damon_lru_sort.<parameter>=<value>`` on the kernel boot command line or write
proper values to ``/sys/module/damon_lru_sort/parameters/<parameter>`` files.
Below are the description of each parameter.
enabled
-------
Enable or disable DAMON_LRU_SORT.
You can enable DAMON_LRU_SORT by setting the value of this parameter as ``Y``.
Setting it as ``N`` disables DAMON_LRU_SORT. Note that DAMON_LRU_SORT could do
no real monitoring and LRU-lists sorting due to the watermarks-based activation
condition. Refer to below descriptions for the watermarks parameter for this.
commit_inputs
-------------
Make DAMON_LRU_SORT reads the input parameters again, except ``enabled``.
Input parameters that updated while DAMON_LRU_SORT is running are not applied
by default. Once this parameter is set as ``Y``, DAMON_LRU_SORT reads values
of parameters except ``enabled`` again. Once the re-reading is done, this
parameter is set as ``N``. If invalid parameters are found while the
re-reading, DAMON_LRU_SORT will be disabled.
Once ``Y`` is written to this parameter, the user must not write to any
parameters until reading ``commit_inputs`` again returns ``N``. If users
violate this rule, the kernel may exhibit undefined behavior.
active_mem_bp
-------------
Desired active to [in]active memory ratio in bp (1/10,000).
While keeping the caps that set by other quotas, DAMON_LRU_SORT automatically
increases and decreases the effective level of the quota aiming the LRU
[de]prioritizations of the hot and cold memory resulting in this active to
[in]active memory ratio. Value zero means disabling this auto-tuning feature.
Disabled by default.
autotune_monitoring_intervals
-----------------------------
If this parameter is set as ``Y``, DAMON_LRU_SORT automatically tunes DAMON's
sampling and aggregation intervals. The auto-tuning aims to capture meaningful
amount of access events in each DAMON-snapshot, while keeping the sampling
interval 5 milliseconds in minimum, and 10 seconds in maximum. Setting this as
``N`` disables the auto-tuning.
Disabled by default.
filter_young_pages
------------------
Filter [non-]young pages accordingly for LRU [de]prioritizations.
If this is set, check page level access (youngness) once again before each
LRU [de]prioritization operation. LRU prioritization operation is skipped
if the page has not accessed since the last check (not young). LRU
deprioritization operation is skipped if the page has accessed since the
last check (young). The feature is enabled or disabled if this parameter is
set as ``Y`` or ``N``, respectively.
Disabled by default.
hot_thres_access_freq
---------------------
Access frequency threshold for hot memory regions identification in permil.
If a memory region is accessed in frequency of this or higher, DAMON_LRU_SORT
identifies the region as hot, and mark it as accessed on the LRU list, so that
it could not be reclaimed under memory pressure. 50% by default.
cold_min_age
------------
Time threshold for cold memory regions identification in microseconds.
If a memory region is not accessed for this or longer time, DAMON_LRU_SORT
identifies the region as cold, and mark it as unaccessed on the LRU list, so
that it could be reclaimed first under memory pressure. 120 seconds by
default.
quota_ms
--------
Limit of time for trying the LRU lists sorting in milliseconds.
DAMON_LRU_SORT tries to use only up to this time within a time window
(quota_reset_interval_ms) for trying LRU lists sorting. This can be used
for limiting CPU consumption of DAMON_LRU_SORT. If the value is zero, the
limit is disabled.
10 ms by default.
quota_reset_interval_ms
-----------------------
The time quota charge reset interval in milliseconds.
The charge reset interval for the quota of time (quota_ms). That is,
DAMON_LRU_SORT does not try LRU-lists sorting for more than quota_ms
milliseconds or quota_sz bytes within quota_reset_interval_ms milliseconds.
1 second by default.
wmarks_interval
---------------
The watermarks check time interval in microseconds.
Minimal time to wait before checking the watermarks, when DAMON_LRU_SORT is
enabled but inactive due to its watermarks rule. 5 seconds by default.
wmarks_high
-----------
Free memory rate (per thousand) for the high watermark.
If free memory of the system in bytes per thousand bytes is higher than this,
DAMON_LRU_SORT becomes inactive, so it does nothing but periodically checks the
watermarks. 200 (20%) by default.
wmarks_mid
----------
Free memory rate (per thousand) for the middle watermark.
If free memory of the system in bytes per thousand bytes is between this and
the low watermark, DAMON_LRU_SORT becomes active, so starts the monitoring and
the LRU-lists sorting. 150 (15%) by default.
wmarks_low
----------
Free memory rate (per thousand) for the low watermark.
If free memory of the system in bytes per thousand bytes is lower than this,
DAMON_LRU_SORT becomes inactive, so it does nothing but periodically checks the
watermarks. 50 (5%) by default.
sample_interval
---------------
Sampling interval for the monitoring in microseconds.
The sampling interval of DAMON for the cold memory monitoring. Please refer to
the DAMON documentation (:doc:`usage`) for more detail. 5ms by default.
aggr_interval
-------------
Aggregation interval for the monitoring in microseconds.
The aggregation interval of DAMON for the cold memory monitoring. Please
refer to the DAMON documentation (:doc:`usage`) for more detail. 100ms by
default.
min_nr_regions
--------------
Minimum number of monitoring regions.
The minimal number of monitoring regions of DAMON for the cold memory
monitoring. This can be used to set lower-bound of the monitoring quality.
But, setting this too high could result in increased monitoring overhead.
Please refer to the DAMON documentation (:doc:`usage`) for more detail. 10 by
default.
Note that this must be 3 or higher. Please refer to the :ref:`Monitoring
<damon_design_monitoring>` section of the design document for the rationale
behind this lower bound.
max_nr_regions
--------------
Maximum number of monitoring regions.
The maximum number of monitoring regions of DAMON for the cold memory
monitoring. This can be used to set upper-bound of the monitoring overhead.
However, setting this too low could result in bad monitoring quality. Please
refer to the DAMON documentation (:doc:`usage`) for more detail. 1000 by
defaults.
monitor_region_start
--------------------
Start of target memory region in physical address.
The start physical address of memory region that DAMON_LRU_SORT will do work
against. By default, the system's entire physical memory is used as the
region.
monitor_region_end
------------------
End of target memory region in physical address.
The end physical address of memory region that DAMON_LRU_SORT will do work
against. By default, the system's entire physical memory is used as the
region.
addr_unit
---------
A scale factor for memory addresses and bytes.
This parameter is for setting and getting the :ref:`address unit
<damon_design_addr_unit>` parameter of the DAMON instance for DAMON_RECLAIM.
``monitor_region_start`` and ``monitor_region_end`` should be provided in this
unit. For example, let's suppose ``addr_unit``, ``monitor_region_start`` and
``monitor_region_end`` are set as ``1024``, ``0`` and ``10``, respectively.
Then DAMON_LRU_SORT will work for 10 KiB length of physical address range that
starts from address zero (``[0 * 1024, 10 * 1024)`` in bytes).
Stat parameters having ``bytes_`` prefix are also in this unit. For example,
let's suppose values of ``addr_unit``, ``bytes_lru_sort_tried_hot_regions`` and
``bytes_lru_sorted_hot_regions`` are ``1024``, ``42``, and ``32``,
respectively. Then it means DAMON_LRU_SORT tried to LRU-sort 42 KiB of hot
memory and successfully LRU-sorted 32 KiB of the memory in total.
If unsure, use only the default value (``1``) and forget about this.
kdamond_pid
-----------
PID of the DAMON thread.
If DAMON_LRU_SORT is enabled, this becomes the PID of the worker thread. Else,
-1.
nr_lru_sort_tried_hot_regions
-----------------------------
Number of hot memory regions that tried to be LRU-sorted.
bytes_lru_sort_tried_hot_regions
--------------------------------
Total bytes of hot memory regions that tried to be LRU-sorted.
nr_lru_sorted_hot_regions
-------------------------
Number of hot memory regions that successfully be LRU-sorted.
bytes_lru_sorted_hot_regions
----------------------------
Total bytes of hot memory regions that successfully be LRU-sorted.
nr_hot_quota_exceeds
--------------------
Number of times that the time quota limit for hot regions have exceeded.
nr_lru_sort_tried_cold_regions
------------------------------
Number of cold memory regions that tried to be LRU-sorted.
bytes_lru_sort_tried_cold_regions
---------------------------------
Total bytes of cold memory regions that tried to be LRU-sorted.
nr_lru_sorted_cold_regions
--------------------------
Number of cold memory regions that successfully be LRU-sorted.
bytes_lru_sorted_cold_regions
-----------------------------
Total bytes of cold memory regions that successfully be LRU-sorted.
nr_cold_quota_exceeds
---------------------
Number of times that the time quota limit for cold regions have exceeded.
Example
=======
Below runtime example commands make DAMON_LRU_SORT to find memory regions
having >=50% access frequency and LRU-prioritize while LRU-deprioritizing
memory regions that not accessed for 120 seconds. The prioritization and
deprioritization is limited to be done using only up to 1% CPU time to avoid
DAMON_LRU_SORT consuming too much CPU time for the (de)prioritization. It also
asks DAMON_LRU_SORT to do nothing if the system's free memory rate is more than
50%, but start the real works if it becomes lower than 40%. If DAMON_RECLAIM
doesn't make progress and therefore the free memory rate becomes lower than
20%, it asks DAMON_LRU_SORT to do nothing again, so that we can fall back to
the LRU-list based page granularity reclamation. ::
# cd /sys/module/damon_lru_sort/parameters
# echo 500 > hot_thres_access_freq
# echo 120000000 > cold_min_age
# echo 10 > quota_ms
# echo 1000 > quota_reset_interval_ms
# echo 500 > wmarks_high
# echo 400 > wmarks_mid
# echo 200 > wmarks_low
# echo Y > enabled
Note that this module (damon_lru_sort) cannot run simultaneously with other
DAMON-based special-purpose modules. Refer to :ref:`DAMON design special
purpose modules exclusivity <damon_design_special_purpose_modules_exclusivity>`
for more details.
linux-7.2.3/Documentation/admin-guide/mm/damon/reclaim.rst
+349
View File
@@ -0,0 +1,349 @@
.. SPDX-License-Identifier: GPL-2.0
=======================
DAMON-based Reclamation
=======================
DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to
be used for proactive and lightweight reclamation under light memory pressure.
It doesn't aim to replace the LRU-list based page_granularity reclamation, but
to be selectively used for different level of memory pressure and requirements.
Where Proactive Reclamation is Required?
========================================
On general memory over-committed systems, proactively reclaiming cold pages
helps saving memory and reducing latency spikes that incurred by the direct
reclaim of the process or CPU consumption of kswapd, while incurring only
minimal performance degradation [1]_ [2]_ .
Free Pages Reporting [3]_ based memory over-commit virtualization systems are
good example of the cases. In such systems, the guest VMs reports their free
memory to host, and the host reallocates the reported memory to other guests.
As a result, the memory of the systems are fully utilized. However, the
guests could be not so memory-frugal, mainly because some kernel subsystems and
user-space applications are designed to use as much memory as available. Then,
guests could report only small amount of memory as free to host, results in
memory utilization drop of the systems. Running the proactive reclamation in
guests could mitigate this problem.
How It Works?
=============
DAMON_RECLAIM finds memory regions that didn't accessed for specific time
duration and page out. To avoid it consuming too much CPU for the paging out
operation, a speed limit can be configured. Under the speed limit, it pages
out memory regions that didn't accessed longer time first. System
administrators can also configure under what situation this scheme should
automatically activated and deactivated with three memory pressure watermarks.
Interface: Module Parameters
============================
To use this feature, you should first ensure your system is running on a kernel
that is built with ``CONFIG_DAMON_RECLAIM=y``.
To let sysadmins enable or disable it and tune for the given system,
DAMON_RECLAIM utilizes module parameters. That is, you can put
``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write
proper values to ``/sys/module/damon_reclaim/parameters/<parameter>`` files.
Below are the description of each parameter.
enabled
-------
Enable or disable DAMON_RECLAIM.
You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``.
Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do
no real monitoring and reclamation due to the watermarks-based activation
condition. Refer to below descriptions for the watermarks parameter for this.
commit_inputs
-------------
Make DAMON_RECLAIM reads the input parameters again, except ``enabled``.
Input parameters that updated while DAMON_RECLAIM is running are not applied
by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values
of parameters except ``enabled`` again. Once the re-reading is done, this
parameter is set as ``N``. If invalid parameters are found while the
re-reading, DAMON_RECLAIM will be disabled.
Once ``Y`` is written to this parameter, the user must not write to any
parameters until reading ``commit_inputs`` again returns ``N``. If users
violate this rule, the kernel may exhibit undefined behavior.
min_age
-------
Time threshold for cold memory regions identification in microseconds.
If a memory region is not accessed for this or longer time, DAMON_RECLAIM
identifies the region as cold, and reclaims it.
120 seconds by default.
autotune_monitoring_intervals
-----------------------------
If this parameter is set as ``Y``, DAMON_RECLAIM automatically tunes DAMON's
sampling and aggregation intervals. The auto-tuning aims to capture meaningful
amount of access events in each DAMON-snapshot, while keeping the sampling
interval 5 milliseconds in minimum, and 10 seconds in maximum. Setting this as
``N`` disables the auto-tuning.
Disabled by default.
quota_ms
--------
Limit of time for the reclamation in milliseconds.
DAMON_RECLAIM tries to use only up to this time within a time window
(quota_reset_interval_ms) for trying reclamation of cold pages. This can be
used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the
limit is disabled.
10 ms by default.
quota_sz
--------
Limit of size of memory for the reclamation in bytes.
DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time
window (quota_reset_interval_ms) and makes no more than this limit is tried.
This can be used for limiting consumption of CPU and IO. If this value is
zero, the limit is disabled.
128 MiB by default.
quota_reset_interval_ms
-----------------------
The time/size quota charge reset interval in milliseconds.
The charget reset interval for the quota of time (quota_ms) and size
(quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than
quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms
milliseconds.
1 second by default.
quota_mem_pressure_us
---------------------
Desired level of memory pressure-stall time in microseconds.
While keeping the caps that set by other quotas, DAMON_RECLAIM automatically
increases and decreases the effective level of the quota aiming this level of
memory pressure is incurred. System-wide ``some`` memory PSI in microseconds
per quota reset interval (``quota_reset_interval_ms``) is collected and
compared to this value to see if the aim is satisfied. Value zero means
disabling this auto-tuning feature.
Disabled by default.
quota_autotune_feedback
-----------------------
User-specifiable feedback for auto-tuning of the effective quota.
While keeping the caps that set by other quotas, DAMON_RECLAIM automatically
increases and decreases the effective level of the quota aiming receiving this
feedback of value ``10,000`` from the user. DAMON_RECLAIM assumes the feedback
value and the quota are positively proportional. Value zero means disabling
this auto-tuning feature.
Disabled by default.
wmarks_interval
---------------
Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is
enabled but inactive due to its watermarks rule.
wmarks_high
-----------
Free memory rate (per thousand) for the high watermark.
If free memory of the system in bytes per thousand bytes is higher than this,
DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks
the watermarks.
wmarks_mid
----------
Free memory rate (per thousand) for the middle watermark.
If free memory of the system in bytes per thousand bytes is between this and
the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and
the reclaiming.
wmarks_low
----------
Free memory rate (per thousand) for the low watermark.
If free memory of the system in bytes per thousand bytes is lower than this,
DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the
watermarks. In the case, the system falls back to the LRU-list based page
granularity reclamation logic.
sample_interval
---------------
Sampling interval for the monitoring in microseconds.
The sampling interval of DAMON for the cold memory monitoring. Please refer to
the DAMON documentation (:doc:`usage`) for more detail.
aggr_interval
-------------
Aggregation interval for the monitoring in microseconds.
The aggregation interval of DAMON for the cold memory monitoring. Please
refer to the DAMON documentation (:doc:`usage`) for more detail.
min_nr_regions
--------------
Minimum number of monitoring regions.
The minimal number of monitoring regions of DAMON for the cold memory
monitoring. This can be used to set lower-bound of the monitoring quality.
But, setting this too high could result in increased monitoring overhead.
Please refer to the DAMON documentation (:doc:`usage`) for more detail.
Note that this must be 3 or higher. Please refer to the :ref:`Monitoring
<damon_design_monitoring>` section of the design document for the rationale
behind this lower bound.
max_nr_regions
--------------
Maximum number of monitoring regions.
The maximum number of monitoring regions of DAMON for the cold memory
monitoring. This can be used to set upper-bound of the monitoring overhead.
However, setting this too low could result in bad monitoring quality. Please
refer to the DAMON documentation (:doc:`usage`) for more detail.
monitor_region_start
--------------------
Start of target memory region in physical address.
The start physical address of memory region that DAMON_RECLAIM will do work
against. That is, DAMON_RECLAIM will find cold memory regions in this region
and reclaims. By default, the system's entire physical memory is used as the
region.
monitor_region_end
------------------
End of target memory region in physical address.
The end physical address of memory region that DAMON_RECLAIM will do work
against. That is, DAMON_RECLAIM will find cold memory regions in this region
and reclaims. By default, the system's entire physical memory is used as the
region.
addr_unit
---------
A scale factor for memory addresses and bytes.
This parameter is for setting and getting the :ref:`address unit
<damon_design_addr_unit>` parameter of the DAMON instance for DAMON_RECLAIM.
``monitor_region_start`` and ``monitor_region_end`` should be provided in this
unit. For example, let's suppose ``addr_unit``, ``monitor_region_start`` and
``monitor_region_end`` are set as ``1024``, ``0`` and ``10``, respectively.
Then DAMON_RECLAIM will work for 10 KiB length of physical address range that
starts from address zero (``[0 * 1024, 10 * 1024)`` in bytes).
``bytes_reclaim_tried_regions`` and ``bytes_reclaimed_regions`` are also in
this unit. For example, let's suppose values of ``addr_unit``,
``bytes_reclaim_tried_regions`` and ``bytes_reclaimed_regions`` are ``1024``,
``42``, and ``32``, respectively. Then it means DAMON_RECLAIM tried to reclaim
42 KiB memory and successfully reclaimed 32 KiB memory in total.
If unsure, use only the default value (``1``) and forget about this.
skip_anon
---------
Skip anonymous pages reclamation.
If this parameter is set as ``Y``, DAMON_RECLAIM does not reclaim anonymous
pages. By default, ``N``.
kdamond_pid
-----------
PID of the DAMON thread.
If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else,
-1.
nr_reclaim_tried_regions
------------------------
Number of memory regions that tried to be reclaimed by DAMON_RECLAIM.
bytes_reclaim_tried_regions
---------------------------
Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM.
nr_reclaimed_regions
--------------------
Number of memory regions that successfully be reclaimed by DAMON_RECLAIM.
bytes_reclaimed_regions
-----------------------
Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM.
nr_quota_exceeds
----------------
Number of times that the time/space quota limits have exceeded.
Example
=======
Below runtime example commands make DAMON_RECLAIM to find memory regions that
not accessed for 30 seconds or more and pages out. The reclamation is limited
to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too
much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do
nothing if the system's free memory rate is more than 50%, but start the real
works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and
therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to
do nothing again, so that we can fall back to the LRU-list based page
granularity reclamation. ::
# cd /sys/module/damon_reclaim/parameters
# echo 30000000 > min_age
# echo $((1 * 1024 * 1024 * 1024)) > quota_sz
# echo 1000 > quota_reset_interval_ms
# echo 500 > wmarks_high
# echo 400 > wmarks_mid
# echo 200 > wmarks_low
# echo Y > enabled
Note that this module (damon_reclaim) cannot run simultaneously with other
DAMON-based special-purpose modules. Refer to :ref:`DAMON design special
purpose modules exclusivity <damon_design_special_purpose_modules_exclusivity>`
for more details.
.. [1] https://research.google/pubs/pub48551/
.. [2] https://lwn.net/Articles/787611/
.. [3] https://www.kernel.org/doc/html/latest/mm/free_page_reporting.html
linux-7.2.3/Documentation/admin-guide/mm/damon/start.rst
+178
View File
@@ -0,0 +1,178 @@
.. SPDX-License-Identifier: GPL-2.0
===============
Getting Started
===============
This document briefly describes how you can use DAMON by demonstrating its
default user space tool. Please note that this document describes only a part
of its features for brevity. Please refer to the usage `doc
<https://github.com/damonitor/damo/blob/next/USAGE.md>`_ of the tool for more
details.
Prerequisites
=============
Kernel
------
You should first ensure your system is running on a kernel built with
``CONFIG_DAMON_*=y``.
User Space Tool
---------------
For the demonstration, we will use the default user space tool for DAMON,
called DAMON Operator (DAMO). It is available at
https://github.com/damonitor/damo. The examples below assume that ``damo`` is on
your ``$PATH``. It's not mandatory, though.
Because DAMO is using the sysfs interface (refer to :doc:`usage` for the
detail) of DAMON, you should ensure :doc:`sysfs </filesystems/sysfs>` is
mounted.
Snapshot Data Access Patterns
=============================
The commands below show the memory access pattern of a program at the moment of
the execution. ::
$ git clone https://github.com/sjp38/masim; cd masim; make
$ sudo damo start "./masim ./configs/stairs.cfg --quiet"
$ sudo damo report access
heatmap: 641111111000000000000000000000000000000000000000000000[...]33333333333333335557984444[...]7
# min/max temperatures: -1,840,000,000, 370,010,000, column size: 3.925 MiB
0 addr 86.182 TiB size 8.000 KiB access 0 % age 14.900 s
1 addr 86.182 TiB size 8.000 KiB access 60 % age 0 ns
2 addr 86.182 TiB size 3.422 MiB access 0 % age 4.100 s
3 addr 86.182 TiB size 2.004 MiB access 95 % age 2.200 s
4 addr 86.182 TiB size 29.688 MiB access 0 % age 14.100 s
5 addr 86.182 TiB size 29.516 MiB access 0 % age 16.700 s
6 addr 86.182 TiB size 29.633 MiB access 0 % age 17.900 s
7 addr 86.182 TiB size 117.652 MiB access 0 % age 18.400 s
8 addr 126.990 TiB size 62.332 MiB access 0 % age 9.500 s
9 addr 126.990 TiB size 13.980 MiB access 0 % age 5.200 s
10 addr 126.990 TiB size 9.539 MiB access 100 % age 3.700 s
11 addr 126.990 TiB size 16.098 MiB access 0 % age 6.400 s
12 addr 127.987 TiB size 132.000 KiB access 0 % age 2.900 s
total size: 314.008 MiB
$ sudo damo stop
The first command of the above example downloads and builds an artificial
memory access generator program called ``masim``. The second command asks DAMO
to start the program via the given command and make DAMON monitors the newly
started process. The third command retrieves the current snapshot of the
monitored access pattern of the process from DAMON and shows the pattern in a
human readable format.
The first line of the output shows the relative access temperature (hotness) of
the regions in a single row hetmap format. Each column on the heatmap
represents regions of same size on the monitored virtual address space. The
position of the colun on the row and the number on the column represents the
relative location and access temperature of the region. ``[...]`` means
unmapped huge regions on the virtual address spaces. The second line shows
additional information for better understanding the heatmap.
Each line of the output from the third line shows which virtual address range
(``addr XX size XX``) of the process is how frequently (``access XX %``)
accessed for how long time (``age XX``). For example, the evelenth region of
~9.5 MiB size is being most frequently accessed for last 3.7 seconds. Finally,
the fourth command stops DAMON.
Note that DAMON can monitor not only virtual address spaces but multiple types
of address spaces including the physical address space.
Recording Data Access Patterns
==============================
The commands below record the memory access patterns of a program and save the
monitoring results to a file. ::
$ ./masim ./configs/zigzag.cfg &
$ sudo damo record -o damon.data $(pidof masim)
The line of the commands run the artificial memory access
generator program again. The generator will repeatedly
access two 100 MiB sized memory regions one by one. You can substitute this
with your real workload. The last line asks ``damo`` to record the access
pattern in the ``damon.data`` file.
Visualizing Recorded Patterns
=============================
You can visualize the pattern in a heatmap, showing which memory region
(x-axis) got accessed when (y-axis) and how frequently (number).::
$ sudo damo report heatmap
22222222222222222222222222222222222222211111111111111111111111111111111111111100
44444444444444444444444444444444444444434444444444444444444444444444444444443200
44444444444444444444444444444444444444433444444444444444444444444444444444444200
33333333333333333333333333333333333333344555555555555555555555555555555555555200
33333333333333333333333333333333333344444444444444444444444444444444444444444200
22222222222222222222222222222222222223355555555555555555555555555555555555555200
00000000000000000000000000000000000000288888888888888888888888888888888888888400
00000000000000000000000000000000000000288888888888888888888888888888888888888400
33333333333333333333333333333333333333355555555555555555555555555555555555555200
88888888888888888888888888888888888888600000000000000000000000000000000000000000
88888888888888888888888888888888888888600000000000000000000000000000000000000000
33333333333333333333333333333333333333444444444444444444444444444444444444443200
00000000000000000000000000000000000000288888888888888888888888888888888888888400
[...]
# access_frequency: 0 1 2 3 4 5 6 7 8 9
# x-axis: space (139728247021568-139728453431248: 196.848 MiB)
# y-axis: time (15256597248362-15326899978162: 1 m 10.303 s)
# resolution: 80x40 (2.461 MiB and 1.758 s for each character)
You can also visualize the distribution of the working set size, sorted by the
size.::
$ sudo damo report wss --range 0 101 10
# <percentile> <wss>
# target_id 18446632103789443072
# avr: 107.708 MiB
0 0 B | |
10 95.328 MiB |**************************** |
20 95.332 MiB |**************************** |
30 95.340 MiB |**************************** |
40 95.387 MiB |**************************** |
50 95.387 MiB |**************************** |
60 95.398 MiB |**************************** |
70 95.398 MiB |**************************** |
80 95.504 MiB |**************************** |
90 190.703 MiB |********************************************************* |
100 196.875 MiB |***********************************************************|
Using ``--sortby`` option with the above command, you can show how the working
set size has chronologically changed.::
$ sudo damo report wss --range 0 101 10 --sortby time
# <percentile> <wss>
# target_id 18446632103789443072
# avr: 107.708 MiB
0 3.051 MiB | |
10 190.703 MiB |***********************************************************|
20 95.336 MiB |***************************** |
30 95.328 MiB |***************************** |
40 95.387 MiB |***************************** |
50 95.332 MiB |***************************** |
60 95.320 MiB |***************************** |
70 95.398 MiB |***************************** |
80 95.398 MiB |***************************** |
90 95.340 MiB |***************************** |
100 95.398 MiB |***************************** |
Data Access Pattern Aware Memory Management
===========================================
Below command makes every memory region of size >=4K that has not accessed for
>=60 seconds in your workload to be swapped out. ::
$ sudo damo start --damos_access_rate 0 0 --damos_sz_region 4K max \
--damos_age 60s max --damos_action pageout \
--target_pid <pid of your workload>
linux-7.2.3/Documentation/admin-guide/mm/damon/stat.rst
+98
View File
@@ -0,0 +1,98 @@
.. SPDX-License-Identifier: GPL-2.0
===================================
Data Access Monitoring Results Stat
===================================
Data Access Monitoring Results Stat (DAMON_STAT) is a static kernel module that
is aimed to be used for simple access pattern monitoring. It monitors accesses
on the system's entire physical memory using DAMON, and provides simplified
access monitoring results statistics, namely idle time percentiles and
estimated memory bandwidth.
.. _damon_stat_monitoring_accuracy_overhead:
Monitoring Accuracy and Overhead
================================
DAMON_STAT uses monitoring intervals :ref:`auto-tuning
<damon_design_monitoring_intervals_autotuning>` to make its accuracy high and
overhead minimum. It auto-tunes the intervals aiming 4 % of observable access
events to be captured in each snapshot, while limiting the resulting sampling
interval to be 5 milliseconds in minimum and 10 seconds in maximum. On a few
production server systems, it resulted in consuming only 0.x % single CPU time,
while capturing reasonable quality of access patterns. The tuning-resulting
intervals can be retrieved via ``aggr_interval_us`` :ref:`parameter
<damon_stat_aggr_interval_us>`.
Interface: Module Parameters
============================
To use this feature, you should first ensure your system is running on a kernel
that is built with ``CONFIG_DAMON_STAT=y``. The feature can be enabled by
default at build time, by setting ``CONFIG_DAMON_STAT_ENABLED_DEFAULT`` true.
To let sysadmins enable or disable it at boot and/or runtime, and read the
monitoring results, DAMON_STAT provides module parameters. Following
sections are descriptions of the parameters.
enabled
-------
Enable or disable DAMON_STAT.
You can enable DAMON_STAT by setting the value of this parameter as ``Y``.
Setting it as ``N`` disables DAMON_STAT. The default value is set by
``CONFIG_DAMON_STAT_ENABLED_DEFAULT`` build config option.
Note that this module (damon_stat) cannot run simultaneously with other
DAMON-based special-purpose modules. Refer to :ref:`DAMON design special
purpose modules exclusivity <damon_design_special_purpose_modules_exclusivity>`
for more details.
.. _damon_stat_aggr_interval_us:
aggr_interval_us
----------------
Auto-tuned aggregation time interval in microseconds.
Users can read the aggregation interval of DAMON that is being used by the
DAMON instance for DAMON_STAT. It is :ref:`auto-tuned
<damon_stat_monitoring_accuracy_overhead>` and therefore the value is
dynamically changed.
estimated_memory_bandwidth
--------------------------
Estimated memory bandwidth consumption (bytes per second) of the system.
DAMON_STAT reads observed access events on the current DAMON results snapshot
and converts it to memory bandwidth consumption estimation in bytes per second.
The resulting metric is exposed to user via this read-only parameter. Because
DAMON uses sampling, this is only an estimation of the access intensity rather
than accurate memory bandwidth.
memory_idle_ms_percentiles
--------------------------
Per-byte idle time (milliseconds) percentiles of the system.
DAMON_STAT calculates how long each byte of the memory was not accessed until
now (idle time), based on the current DAMON results snapshot. For regions
having access frequency (nr_accesses) larger than zero, how long the current
access frequency level was kept multiplied by ``-1`` becomes the idlee time of
every byte of the region. If a region has zero access frequency (nr_accesses),
how long the region was keeping the zero access frequency (age) becomes the
idle time of every byte of the region. Then, DAMON_STAT exposes the
percentiles of the idle time values via this read-only parameter. Reading the
parameter returns 101 idle time values in milliseconds, separated by comma.
Each value represents 0-th, 1st, 2nd, 3rd, ..., 99th and 100th percentile idle
times.
kdamond_pid
-----------
PID of the DAMON thread.
If DAMON_STAT is enabled, this becomes the PID of the worker thread. Else, -1.
linux-7.2.3/Documentation/admin-guide/mm/damon/usage.rst
+757
View File
@@ -0,0 +1,757 @@
.. SPDX-License-Identifier: GPL-2.0
===============
Detailed Usages
===============
DAMON provides below interfaces for different users.
- *Special-purpose DAMON modules.*
:ref:`This <damon_modules_special_purpose>` is for people who are building,
distributing, and/or administrating the kernel with special-purpose DAMON
usages. Using this, users can use DAMON's major features for the given
purposes in build, boot, or runtime in simple ways.
- *DAMON user space tool.*
`This <https://github.com/damonitor/damo>`_ is for privileged people such as
system administrators who want a just-working human-friendly interface.
Using this, users can use the DAMON’s major features in a human-friendly way.
It may not be highly tuned for special cases, though. For more detail,
please refer to its `usage document
<https://github.com/damonitor/damo/blob/next/USAGE.md>`_.
- *sysfs interface.*
:ref:`This <sysfs_interface>` is for privileged user space programmers who
want more optimized use of DAMON. Using this, users can use DAMON’s major
features by reading from and writing to special sysfs files. Therefore,
you can write and use your personalized DAMON sysfs wrapper programs that
reads/writes the sysfs files instead of you. The `DAMON user space tool
<https://github.com/damonitor/damo>`_ is one example of such programs.
- *Kernel Space Programming Interface.*
:doc:`This </mm/damon/api>` is for kernel space programmers. Using this,
users can utilize every feature of DAMON most flexibly and efficiently by
writing kernel space DAMON application programs for you. You can even extend
DAMON for various address spaces. For detail, please refer to the interface
:doc:`document </mm/damon/api>`.
.. _sysfs_interface:
sysfs Interface
===============
DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined. It
creates multiple directories and files under its sysfs directory,
``<sysfs>/kernel/mm/damon/``. You can control DAMON by writing to and reading
from the files under the directory.
For a short example, users can monitor the virtual address space of a given
workload as below. ::
# cd /sys/kernel/mm/damon/admin/
# echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
# echo vaddr > kdamonds/0/contexts/0/operations
# echo 1 > kdamonds/0/contexts/0/targets/nr_targets
# echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
# echo on > kdamonds/0/state
Files Hierarchy
---------------
The files hierarchy of DAMON sysfs interface is shown below. In the below
figure, parents-children relations are represented with indentations, each
directory is having ``/`` suffix, and files in each directory are separated by
comma (",").
.. parsed-literal::
:ref:`/sys/kernel/mm/damon <sysfs_root>`/admin
│ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds
│ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms
│ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts
│ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit,
│ │ │ │ pause
│ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/
│ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
│ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us
│ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ :ref:`probes <damon_usage_sysfs_probes>`/nr_probes
│ │ │ │ │ │ │ 0/filters/nr_filters
│ │ │ │ │ │ │ │ 0/type,matching,allow,path
│ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets
│ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target
│ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions
│ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end
│ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes
│ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us
│ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/
│ │ │ │ │ │ │ │ sz/min,max
│ │ │ │ │ │ │ │ nr_accesses/min,max
│ │ │ │ │ │ │ │ age/min,max
│ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,
│ │ │ │ │ │ │ effective_bytes,goal_tuner,
│ │ │ │ │ │ │ fail_charge_num,fail_charge_denom
│ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
│ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals
│ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path
│ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
│ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters
│ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,damon_target_idx,min,max
│ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests
│ │ │ │ │ │ │ │ 0/id,weight
│ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots
│ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
│ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
│ │ │ │ │ │ │ │ │ probes
│ │ │ │ │ │ │ │ │ │ 0/hits
│ │ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ ...
│ │ ...
.. _sysfs_root:
Root
----
The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
has one directory named ``admin``. The directory contains the files for
privileged user space programs' control of DAMON. User space tools or daemons
having the root permission could use this directory.
.. _sysfs_kdamonds:
kdamonds/
---------
Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
controlling the kdamonds (refer to
:ref:`design <damon_design_execution_model_and_data_structures>` for more
details) exists. In the beginning, this directory has only one file,
``nr_kdamonds``. Writing a number (``N``) to the file creates the number of
child directories named ``0`` to ``N-1``. Each directory represents each
kdamond.
.. _sysfs_kdamond:
kdamonds/<N>/
-------------
In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``)
and one directory (``contexts``) exist.
Reading ``state`` returns ``on`` if the kdamond is currently running, or
``off`` if it is not running.
Users can write below commands for the kdamond to the ``state`` file.
- ``on``: Start running.
- ``off``: Stop running.
- ``commit``: Read the user inputs in the sysfs files except ``state`` file
again. Monitoring :ref:`target region <sysfs_regions>` inputs are also be
ignored if no target region is specified.
- ``update_tuned_intervals``: Update the contents of ``sample_us`` and
``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling
interval`` and ``aggregation interval`` for the files. Please refer to
:ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>`
for more details.
- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes'
:ref:`quota goals <sysfs_schemes_quota_goals>`.
- ``update_schemes_stats``: Update the contents of stats files for each
DAMON-based operation scheme of the kdamond. For details of the stats,
please refer to :ref:`stats section <sysfs_schemes_stats>`.
- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme
action tried regions directory for each DAMON-based operation scheme of the
kdamond. For details of the DAMON-based operation scheme action tried
regions directory, please refer to
:ref:`tried_regions section <sysfs_schemes_tried_regions>`.
- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes``
files.
- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme
action tried regions directory for each DAMON-based operation scheme of the
kdamond.
- ``update_schemes_effective_quotas``: Update the contents of
``effective_bytes`` files for each DAMON-based operation scheme of the
kdamond. For more details, refer to :ref:`quotas directory <sysfs_quotas>`.
If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
Users can ask the kernel to periodically update files showing auto-tuned
parameters and DAMOS stats instead of manually writing
``update_tuned_intervals`` like keywords to ``state`` file. For this, users
should write the desired update time interval in milliseconds to ``refresh_ms``
file. If the interval is zero, the periodic update is disabled. Reading the
file shows currently set time interval.
``contexts`` directory contains files for controlling the monitoring contexts
that this kdamond will execute.
.. _sysfs_contexts:
kdamonds/<N>/contexts/
----------------------
In the beginning, this directory has only one file, ``nr_contexts``. Writing a
number (``N``) to the file creates the number of child directories named as
``0`` to ``N-1``. Each directory represents each monitoring context (refer to
:ref:`design <damon_design_execution_model_and_data_structures>` for more
details). At the moment, only one context per kdamond is supported, so only
``0`` or ``1`` can be written to the file.
.. _sysfs_context:
contexts/<N>/
-------------
In each context directory, four files (``avail_operations``, ``operations``,
``addr_unit`` and ``pause``) and three directories (``monitoring_attrs``,
``targets``, and ``schemes``) exist.
DAMON supports multiple types of :ref:`monitoring operations
<damon_design_configurable_operations_set>`, including those for virtual address
space and the physical address space. You can get the list of available
monitoring operations set on the currently running kernel by reading
``avail_operations`` file. Based on the kernel configuration, the file will
list different available operation sets. Please refer to the :ref:`design
<damon_operations_set>` for the list of all available operation sets and their
brief explanations.
You can set and get what type of monitoring operations DAMON will use for the
context by writing one of the keywords listed in ``avail_operations`` file and
reading from the ``operations`` file.
``addr_unit`` file is for setting and getting the :ref:`address unit
<damon_design_addr_unit>` parameter of the operations set.
``pause`` file is for setting and getting the :ref:`pause request
<damon_design_execution_model_and_data_structures>` parameter of the context.
.. _sysfs_monitoring_attrs:
contexts/<N>/monitoring_attrs/
------------------------------
Files for specifying attributes of the monitoring including required quality
and efficiency of the monitoring are in ``monitoring_attrs`` directory.
Specifically, three directories, ``intervals``, ``nr_regions`` and ``probes``
exist in this directory.
Under ``intervals`` directory, three files for DAMON's sampling interval
(``sample_us``), aggregation interval (``aggr_us``), and update interval
(``update_us``) exist. You can set and get the values in micro-seconds by
writing to and reading from the files.
Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
of DAMON's monitoring regions (``min`` and ``max``, respectively), which
controls the monitoring overhead, exist. You can set and get the values by
writing to and rading from the files.
For more details about the intervals and monitoring regions range, please refer
to the Design document (:doc:`/mm/damon/design`).
.. _damon_usage_sysfs_monitoring_intervals_goal:
contexts/<N>/monitoring_attrs/intervals/intervals_goal/
-------------------------------------------------------
Under the ``intervals`` directory, one directory for automated tuning of
``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists.
Under the directory, four files for the auto-tuning control, namely
``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist.
Please refer to the :ref:`design document of the feature
<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning
mechanism. Reading and writing the four files under ``intervals_goal``
directory shows and updates the tuning parameters that described in the
:ref:design doc <damon_design_monitoring_intervals_autotuning>` with the same
names. The tuning starts with the user-set ``sample_us`` and ``aggr_us``. The
tuning-applied current values of the two intervals can be read from the
``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to
the ``state`` file.
.. _damon_usage_sysfs_probes:
contexts/<N>/monitoring_attrs/probes/
-------------------------------------
A directory for registering :ref:`data attributes monitoring
<damon_design_data_attrs_monitoring>` probes.
In the beginning, this directory has only one file, ``nr_probes``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each monitoring probe.
In each probe directory, one directory, ``filters`` exists. The directory
contains files for installing filters for the probe, that is used to determine
the data attribute for the probe.
In the beginning, ``filters`` directory has only one file, ``nr_filters``.
Writing a number (``N``) to the file creates the number of child directories
named ``0`` to ``N-1``. Each directory represents each filter and works in a
way similar to that for :ref:`DAMOS filter <sysfs_filters>`. When the filter
``type`` is ``memcg``, ``path`` file acts as ``memcg_path`` for :ref:`DAMOS
filter <sysfs_filters>`.
.. _sysfs_targets:
contexts/<N>/targets/
---------------------
In the beginning, this directory has only one file, ``nr_targets``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each monitoring target.
.. _sysfs_target:
targets/<N>/
------------
In each target directory, two files (``pid_target`` and ``obsolete_target``)
and one directory (``regions``) exist.
If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
be a process. You can specify the process to DAMON by writing the pid of the
process to the ``pid_target`` file.
Users can selectively remove targets in the middle of the targets array by
writing non-zero value to ``obsolete_target`` file and committing it (writing
``commit`` to ``state`` file). DAMON will remove the matching targets from its
internal targets array. Users are responsible to construct target directories
again, so that those correctly represent the changed internal targets array.
.. _sysfs_regions:
targets/<N>/regions
-------------------
In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are
required to set the monitoring target address ranges. In case of ``vaddr``
operations set, it is not mandatory, but users can optionally set the initial
monitoring region to specific address ranges. Please refer to the :ref:`design
<damon_design_vaddr_target_regions_construction>` for more details.
For such cases, users can explicitly set the initial monitoring target regions
as they want, by writing proper values to the files under this directory.
In the beginning, this directory has only one file, ``nr_regions``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each initial monitoring target region.
If ``nr_regions`` is zero when committing new DAMON parameters online (writing
``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit
logic ignores the target regions. In other words, the current monitoring
results for the target are preserved.
.. _sysfs_region:
regions/<N>/
------------
In each region directory, you will find two files (``start`` and ``end``). You
can set and get the start and end addresses of the initial monitoring target
region by writing to and reading from the files, respectively.
Each region should not overlap with others. ``end`` of directory ``N`` should
be equal or smaller than ``start`` of directory ``N+1``.
.. _sysfs_schemes:
contexts/<N>/schemes/
---------------------
The directory for DAMON-based Operation Schemes (:ref:`DAMOS
<damon_design_damos>`). Users can get and set the schemes by reading from and
writing to files under this directory.
In the beginning, this directory has only one file, ``nr_schemes``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each DAMON-based operation scheme.
.. _sysfs_scheme:
schemes/<N>/
------------
In each scheme directory, nine directories (``access_pattern``, ``quotas``,
``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``,
``stats``, and ``tried_regions``) and three files (``action``, ``target_nid``
and ``apply_interval``) exist.
The ``action`` file is for setting and getting the scheme's :ref:`action
<damon_design_damos_action>`. The keywords that can be written to and read
from the file and their meaning are same to those of the list on
:ref:`design doc <damon_design_damos_action>`.
The ``target_nid`` file is for setting the migration target node, which is
only meaningful when the ``action`` is either ``migrate_hot`` or
``migrate_cold``.
The ``apply_interval_us`` file is for setting and getting the scheme's
:ref:`apply_interval <damon_design_damos>` in microseconds.
.. _sysfs_access_pattern:
schemes/<N>/access_pattern/
---------------------------
The directory for the target access :ref:`pattern
<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.
Under the ``access_pattern`` directory, three directories (``sz``,
``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
exist. You can set and get the access pattern for the given scheme by writing
to and reading from the ``min`` and ``max`` files under ``sz``,
``nr_accesses``, and ``age`` directories, respectively. Note that the ``min``
and the ``max`` form a closed interval.
.. _sysfs_quotas:
schemes/<N>/quotas/
-------------------
The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
DAMON-based operation scheme.
Under ``quotas`` directory, seven files (``ms``, ``bytes``,
``reset_interval_ms``, ``effective_bytes``, ``goal_tuner``, ``fail_charge_num``
and ``fail_charge_denom``) and two directories (``weights`` and ``goals``)
exist.
You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
``reset interval`` in milliseconds by writing the values to the three files,
respectively. Then, DAMON tries to use only up to ``time quota`` milliseconds
for applying the ``action`` to memory regions of the ``access_pattern``, and to
apply the action to only up to ``bytes`` bytes of memory regions within the
``reset_interval_ms``. Setting both ``ms`` and ``bytes`` zero disables the
quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is
set.
You can set the goal-based effective quota auto-tuning algorithm to use, by
writing the algorithm name to ``goal_tuner`` file. Reading the file returns
the currently selected tuner algorithm. Refer to the design documentation of
:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for
the background design of the feature and the name of the selectable algorithms.
Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals
setup.
You can set the action-failed memory quota charging ratio by writing the
numerator and the denominator for the ratio to ``fail_charge_num`` and
``fail_charge_denom`` files, respectively. Reading those files will return the
current set values. Refer to :ref:`design
<damon_design_damos_quotas_failed_memory_charging_ratio>` for more details of
the ratio feature.
The time quota is internally transformed to a size quota. Between the
transformed size quota and user-specified size quota, smaller one is applied.
Based on the user-specified :ref:`goal <sysfs_schemes_quota_goals>`, the
effective size quota is further adjusted. Reading ``effective_bytes`` returns
the current effective size quota. The file is not updated in real time, so
users should ask DAMON sysfs interface to update the content of the file for
the stats by writing a special keyword, ``update_schemes_effective_quotas`` to
the relevant ``kdamonds/<N>/state`` file.
Under ``weights`` directory, three files (``sz_permil``,
``nr_accesses_permil``, and ``age_permil``) exist.
You can set the :ref:`prioritization weights
<damon_design_damos_quotas_prioritization>` for size, access frequency, and age
in per-thousand unit by writing the values to the three files under the
``weights`` directory.
.. _sysfs_schemes_quota_goals:
schemes/<N>/quotas/goals/
-------------------------
The directory for the :ref:`automatic quota tuning goals
<damon_design_damos_quotas_auto_tuning>` of the given DAMON-based operation
scheme.
In the beginning, this directory has only one file, ``nr_goals``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each goal and current achievement.
Among the multiple feedback, the best one is used.
Each goal directory contains five files, namely ``target_metric``,
``target_value``, ``current_value``, ``nid``, and ``path``. Users can set and
get the five parameters for the quota auto-tuning goals that specified on the
:ref:`design doc <damon_design_damos_quotas_auto_tuning>` by writing to and
reading from each of the files. Because the kernel does not update
``current_value``, reading it only makes sense when ``target_metric`` is
``user_input``. Note that users should further write
``commit_schemes_quota_goals`` to the ``state`` file of the :ref:`kdamond
directory <sysfs_kdamond>` to pass the feedback to DAMON.
.. _sysfs_watermarks:
schemes/<N>/watermarks/
-----------------------
The directory for the :ref:`watermarks <damon_design_damos_watermarks>` of the
given DAMON-based operation scheme.
Under the watermarks directory, five files (``metric``, ``interval_us``,
``high``, ``mid``, and ``low``) for setting the metric, the time interval
between check of the metric, and the three watermarks exist. You can set and
get the five values by writing to and reading from the files, respectively.
Keywords and meanings of those that can be written to the ``metric`` file are
as below.
- none: Ignore the watermarks
- free_mem_rate: System's free memory rate (per thousand)
The ``interval_us`` should be written in microseconds unit.
.. _sysfs_filters:
schemes/<N>/{core\_,ops\_,}filters/
-----------------------------------
Directories for :ref:`filters <damon_design_damos_filters>` of the given
DAMON-based operation scheme.
``core_filters`` and ``ops_filters`` directories are for the filters handled by
the DAMON core layer and operations set layer, respectively. ``filters``
directory can be used for installing filters regardless of their handled
layers. Filters that requested by ``core_filters`` and ``ops_filters`` will be
installed before those of ``filters``. All three directories have same files.
Use of ``filters`` directory can make filters evaluation orders confusing to
expect. For this reason, ``filters`` directory is deprecated. It is still
functioning, but is scheduled for removal in the near future. Users should use
``core_filters`` and ``ops_filters`` directories instead.
In the beginning, the directory has only one file, ``nr_filters``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each filter. The filters are evaluated
in the numeric order.
Each filter directory contains nine files, namely ``type``, ``matching``,
``allow``, ``memcg_path``, ``addr_start``, ``addr_end``, ``min``, ``max``
and ``damon_target_idx``. To ``type`` file, you can write the type of the
filter. Refer to :ref:`the design doc <damon_design_damos_filters>` for
available type names, their meaning and on what layer those are handled.
For ``memcg`` type, you can specify the memory cgroup of the interest by
writing the path of the memory cgroup from the cgroups mount point to
``memcg_path`` file. For ``addr`` type, you can specify the start and end
address of the range (open-ended interval) to ``addr_start`` and ``addr_end``
files, respectively. For ``hugepage_size`` type, you can specify the minimum
and maximum size of the range (closed interval) to ``min`` and ``max`` files,
respectively. For ``target`` type, you can specify the index of the target
between the list of the DAMON context's monitoring targets list to
``damon_target_idx`` file.
You can write ``Y`` or ``N`` to ``matching`` file to specify whether the filter
is for memory that matches the ``type``. You can write ``Y`` or ``N`` to
``allow`` file to specify if applying the action to the memory that satisfies
the ``type`` and ``matching`` should be allowed or not.
For example, below restricts a DAMOS action to be applied to only non-anonymous
pages of all memory cgroups except ``/having_care_already``.::
# cd ops_filters/0/
# echo 2 > nr_filters
# # disallow anonymous pages
echo anon > 0/type
echo Y > 0/matching
echo N > 0/allow
# # further filter out all cgroups except one at '/having_care_already'
echo memcg > 1/type
echo /having_care_already > 1/memcg_path
echo Y > 1/matching
echo N > 1/allow
Refer to the :ref:`DAMOS filters design documentation
<damon_design_damos_filters>` for more details including how multiple filters
of different ``allow`` works, when each of the filters are supported, and
differences on stats.
.. _damon_sysfs_dests:
schemes/<N>/dests/
------------------
Directory for specifying the destinations of given DAMON-based operation
scheme's action. This directory is ignored if the action of the given scheme
is not supporting multiple destinations. Only ``DAMOS_MIGRATE_{HOT,COLD}``
actions are supporting multiple destinations.
In the beginning, the directory has only one file, ``nr_dests``. Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``. Each directory represents each action destination.
Each destination directory contains two files, namely ``id`` and ``weight``.
Users can write and read the identifier of the destination to ``id`` file.
For ``DAMOS_MIGRATE_{HOT,COLD}`` actions, the migrate destination node's node
id should be written to ``id`` file. Users can write and read the weight of
the destination among the given destinations to the ``weight`` file. The
weight can be an arbitrary integer. When DAMOS apply the action to each entity
of the memory region, it will select the destination of the action based on the
relative weights of the destinations.
.. _sysfs_schemes_stats:
schemes/<N>/stats/
------------------
DAMON counts statistics for each scheme. This statistics can be used for
online analysis or tuning of the schemes. Refer to :ref:`design doc
<damon_design_damos_stat>` for more details about the stats.
The statistics can be retrieved by reading the files under ``stats`` directory
(``nr_tried``, ``sz_tried``, ``nr_applied``, ``sz_applied``,
``sz_ops_filter_passed``, ``qt_exceeds``, ``nr_snapshots`` and
``max_nr_snapshots``), respectively.
The files are not updated in real time by default. Users should ask DAMON
sysfs interface to periodically update those using ``refresh_ms``, or do a one
time update by writing a special keyword, ``update_schemes_stats`` to the
relevant ``kdamonds/<N>/state`` file. Refer to :ref:`kdamond directory
<sysfs_kdamond>` for more details.
.. _sysfs_schemes_tried_regions:
schemes/<N>/tried_regions/
--------------------------
This directory initially has one file, ``total_bytes``.
When a special keyword, ``update_schemes_tried_regions``, is written to the
relevant ``kdamonds/<N>/state`` file, DAMON updates the ``total_bytes`` file so
that reading it returns the total size of the scheme tried regions, and creates
directories named integer starting from ``0`` under this directory. Each
directory contains files exposing detailed information about each of the memory
region that the corresponding scheme's ``action`` has tried to be applied under
this directory, during next :ref:`apply interval <damon_design_damos>` of the
corresponding scheme. The information includes address range, ``nr_accesses``,
and ``age`` of the region.
Writing ``update_schemes_tried_bytes`` to the relevant ``kdamonds/<N>/state``
file will only update the ``total_bytes`` file, and will not create the
subdirectories.
The directories will be removed when another special keyword,
``clear_schemes_tried_regions``, is written to the relevant
``kdamonds/<N>/state`` file.
The expected usage of this directory is investigations of schemes' behaviors,
and query-like efficient data access monitoring results retrievals. For the
latter use case, in particular, users can set the ``action`` as ``stat`` and
set the ``access pattern`` as their interested pattern that they want to query.
.. _sysfs_schemes_tried_region:
tried_regions/<N>/
------------------
In each region directory, you will find five files (``start``, ``end``,
``nr_accesses``, ``age`` and ``sz_filter_passed``). Reading the files will
show the properties of the region that corresponding DAMON-based operation
scheme ``action`` has tried to be applied.
tried_regions/<N>/probes/
-------------------------
In each region directory, one directory (``probes``) also exists. In the
directory, subdirectories named ``0`` to ``N-1`` exists. ``N`` is the number
of installed probes. In each number-named directory, a file (``hits``) exist.
Reading the file shows the number of data attributes monitoring probe-hit
positive samples of the region.
Example
~~~~~~~
Below commands applies a scheme saying "If a memory region of size in [4KiB,
8KiB] is showing accesses per aggregate interval in [0, 5] for aggregate
interval in [10, 20], page out the region. For the paging out, use only up to
10ms per second, and also don't page out more than 1GiB per second. Under the
limitation, page out memory regions having longer age first. Also, check the
free memory rate of the system every 5 seconds, start the monitoring and paging
out when the free memory rate becomes lower than 50%, but stop it if the free
memory rate becomes larger than 60%, or lower than 30%". ::
# cd <sysfs>/kernel/mm/damon/admin
# # populate directories
# echo 1 > kdamonds/nr_kdamonds; echo 1 > kdamonds/0/contexts/nr_contexts;
# echo 1 > kdamonds/0/contexts/0/schemes/nr_schemes
# cd kdamonds/0/contexts/0/schemes/0
# # set the basic access pattern and the action
# echo 4096 > access_pattern/sz/min
# echo 8192 > access_pattern/sz/max
# echo 0 > access_pattern/nr_accesses/min
# echo 5 > access_pattern/nr_accesses/max
# echo 10 > access_pattern/age/min
# echo 20 > access_pattern/age/max
# echo pageout > action
# # set quotas
# echo 10 > quotas/ms
# echo $((1024*1024*1024)) > quotas/bytes
# echo 1000 > quotas/reset_interval_ms
# # set watermark
# echo free_mem_rate > watermarks/metric
# echo 5000000 > watermarks/interval_us
# echo 600 > watermarks/high
# echo 500 > watermarks/mid
# echo 300 > watermarks/low
Please note that it's highly recommended to use user space tools like `damo
<https://github.com/damonitor/damo>`_ rather than manually reading and writing
the files as above. Above is only for an example.
.. _tracepoint:
Tracepoints for Monitoring Results
==================================
Users can get the monitoring results via the :ref:`tried_regions
<sysfs_schemes_tried_regions>`. The interface is useful for getting a
snapshot, but it could be inefficient for fully recording all the monitoring
results. For the purpose, two trace points, namely ``damon:damon_aggregated``
and ``damon:damos_before_apply``, are provided. ``damon:damon_aggregated``
provides the whole monitoring results, while ``damon:damos_before_apply``
provides the monitoring results for regions that each DAMON-based Operation
Scheme (:ref:`DAMOS <damon_design_damos>`) is gonna be applied. Hence,
``damon:damos_before_apply`` is more useful for recording internal behavior of
DAMOS, or DAMOS target access
:ref:`pattern <damon_design_damos_access_pattern>` based query-like efficient
monitoring results recording.
While the monitoring is turned on, you could record the tracepoint events and
show results using tracepoint supporting tools like ``perf``. For example::
# echo on > kdamonds/0/state
# perf record -e damon:damon_aggregated &
# sleep 5
# kill 9 $(pidof perf)
# echo off > kdamonds/0/state
# perf script
kdamond.0 46568 [027] 79357.842179: damon:damon_aggregated: target_id=0 nr_regions=11 122509119488-135708762112: 0 864
[...]
Each line of the perf script output represents each monitoring region. The
first five fields are as usual other tracepoint outputs. The sixth field
(``target_id=X``) shows the id of the monitoring target of the region. The
seventh field (``nr_regions=X``) shows the total number of monitoring regions
for the target. The eighth field (``X-Y:``) shows the start (``X``) and end
(``Y``) addresses of the region in bytes. The ninth field (``X``) shows the
``nr_accesses`` of the region (refer to
:ref:`design <damon_design_region_based_sampling>` for more details of the
counter). Finally the tenth field (``X``) shows the ``age`` of the region
(refer to :ref:`design <damon_design_age_tracking>` for more details of the
counter).
If the event was ``damon:damos_beofre_apply``, the ``perf script`` output would
be somewhat like below::
kdamond.0 47293 [000] 80801.060214: damon:damos_before_apply: ctx_idx=0 scheme_idx=0 target_idx=0 nr_regions=11 121932607488-135128711168: 0 136
[...]
Each line of the output represents each monitoring region that each DAMON-based
Operation Scheme was about to be applied at the traced time. The first five
fields are as usual. It shows the index of the DAMON context (``ctx_idx=X``)
of the scheme in the list of the contexts of the context's kdamond, the index
of the scheme (``scheme_idx=X``) in the list of the schemes of the context, in
addition to the output of ``damon_aggregated`` tracepoint.