lastdragon/linux-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Resguardo inicial

Browse Source
This commit is contained in:
lastdragon
2026-09-06 12:53:21 -06:00
parent 662efe3c2a
commit ad2088d178
94853 changed files with 43200404 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
linux-7.2.3/Documentation/filesystems/xfs/index.rst
+14
View File
@@ -0,0 +1,14 @@
.. SPDX-License-Identifier: GPL-2.0
============================
XFS Filesystem Documentation
============================
.. toctree::
:maxdepth: 2
:numbered:
xfs-delayed-logging-design
xfs-maintainer-entry-profile
xfs-self-describing-metadata
xfs-online-fsck-design
linux-7.2.3/Documentation/filesystems/xfs/xfs-delayed-logging-design.rst
+1087
View File
File diff suppressed because it is too large Load Diff
linux-7.2.3/Documentation/filesystems/xfs/xfs-maintainer-entry-profile.rst
+194
View File
@@ -0,0 +1,194 @@
XFS Maintainer Entry Profile
============================
Overview
--------
XFS is a well known high-performance filesystem in the Linux kernel.
The aim of this project is to provide and maintain a robust and
performant filesystem.
Patches are generally merged to the for-next branch of the appropriate
git repository.
After a testing period, the for-next branch is merged to the master
branch.
Kernel code are merged to the xfs-linux tree[0].
Userspace code are merged to the xfsprogs tree[1].
Test cases are merged to the xfstests tree[2].
Ondisk format documentation are merged to the xfs-documentation tree[3].
All patchsets involving XFS *must* be cc'd in their entirety to the mailing
list linux-xfs@vger.kernel.org.
Roles
-----
There are eight key roles in the XFS project.
A person can take on multiple roles, and a role can be filled by
multiple people.
Anyone taking on a role is advised to check in with themselves and
others on a regular basis about burnout.
- **Outside Contributor**: Anyone who sends a patch but is not involved
in the XFS project on a regular basis.
These folks are usually people who work on other filesystems or
elsewhere in the kernel community.
- **Developer**: Someone who is familiar with the XFS codebase enough to
write new code, documentation, and tests.
Developers can often be found in the IRC channel mentioned by the ``C:``
entry in the kernel MAINTAINERS file.
- **Senior Developer**: A developer who is very familiar with at least
some part of the XFS codebase and/or other subsystems in the kernel.
These people collectively decide the long term goals of the project
and nudge the community in that direction.
They should help prioritize development and review work for each release
cycle.
Senior developers tend to be more active participants in the IRC channel.
- **Reviewer**: Someone (most likely also a developer) who reads code
submissions to decide:
0. Is the idea behind the contribution sound?
1. Does the idea fit the goals of the project?
2. Is the contribution designed correctly?
3. Is the contribution polished?
4. Can the contribution be tested effectively?
Reviewers should identify themselves with an ``R:`` entry in the kernel
and fstests MAINTAINERS files.
- **Testing Lead**: This person is responsible for setting the test
coverage goals of the project, negotiating with developers to decide
on new tests for new features, and making sure that developers and
release managers execute on the testing.
The testing lead should identify themselves with an ``M:`` entry in
the XFS section of the fstests MAINTAINERS file.
- **Bug Triager**: Someone who examines incoming bug reports in just
enough detail to identify the person to whom the report should be
forwarded.
The bug triagers should identify themselves with a ``B:`` entry in
the kernel MAINTAINERS file.
- **Release Manager**: This person merges reviewed patchsets into an
integration branch, tests the result locally, pushes the branch to a
public git repository, and sends pull requests further upstream.
The release manager is not expected to work on new feature patchsets.
If a developer and a reviewer fail to reach a resolution on some point,
the release manager must have the ability to intervene to try to drive a
resolution.
The release manager should identify themselves with an ``M:`` entry in
the kernel MAINTAINERS file.
- **Community Manager**: This person calls and moderates meetings of as many
XFS participants as they can get when mailing list discussions prove
insufficient for collective decisionmaking.
They may also serve as liaison between managers of the organizations
sponsoring work on any part of XFS.
- **LTS Maintainer**: Someone who backports and tests bug fixes from
upstream to the LTS kernels.
There tend to be six separate LTS trees at any given time.
The maintainer for a given LTS release should identify themselves with an
``M:`` entry in the MAINTAINERS file for that LTS tree.
Unmaintained LTS kernels should be marked with status ``S: Orphan`` in that
same file.
Submission Checklist Addendum
-----------------------------
Please follow these additional rules when submitting to XFS:
- Patches affecting only the filesystem itself should be based against
the latest -rc or the for-next branch.
These patches will be merged back to the for-next branch.
- Authors of patches touching other subsystems need to coordinate with
the maintainers of XFS and the relevant subsystems to decide how to
proceed with a merge.
- Any patchset changing XFS should be cc'd in its entirety to linux-xfs.
Do not send partial patchsets; that makes analysis of the broader
context of the changes unnecessarily difficult.
- Anyone making kernel changes that have corresponding changes to the
userspace utilities should send the userspace changes as separate
patchsets immediately after the kernel patchsets.
- Authors of bug fix patches are expected to use fstests[2] to perform
an A/B test of the patch to determine that there are no regressions.
When possible, a new regression test case should be written for
fstests.
- Authors of new feature patchsets must ensure that fstests will have
appropriate functional and input corner-case test cases for the new
feature.
- When implementing a new feature, it is strongly suggested that the
developers write a design document to answer the following questions:
* **What** problem is this trying to solve?
* **Who** will benefit from this solution, and **where** will they
access it?
* **How** will this new feature work? This should touch on major data
structures and algorithms supporting the solution at a higher level
than code comments.
* **What** userspace interfaces are necessary to build off of the new
features?
* **How** will this work be tested to ensure that it solves the
problems laid out in the design document without causing new
problems?
The design document should be committed in the kernel documentation
directory.
It may be omitted if the feature is already well known to the
community.
- Patchsets for the new tests should be submitted as separate patchsets
immediately after the kernel and userspace code patchsets.
- Changes to the on-disk format of XFS must be described in the ondisk
format document[3] and submitted as a patchset after the fstests
patchsets.
- Patchsets implementing bug fixes and further code cleanups should put
the bug fixes at the beginning of the series to ease backporting.
Key Release Cycle Dates
-----------------------
Bug fixes may be sent at any time, though the release manager may decide to
defer a patch when the next merge window is close.
Code submissions targeting the next merge window should be sent between
-rc1 and -rc6.
This gives the community time to review the changes, to suggest other changes,
and for the author to retest those changes.
Code submissions also requiring changes to fs/iomap and targeting the
next merge window should be sent between -rc1 and -rc4.
This allows the broader kernel community adequate time to test the
infrastructure changes.
Review Cadence
--------------
In general, please wait at least one week before pinging for feedback.
To find reviewers, either consult the MAINTAINERS file, or ask
developers that have Reviewed-by tags for XFS changes to take a look and
offer their opinion.
References
----------
| [0] https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/
| [1] https://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git/
| [2] https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/
| [3] https://git.kernel.org/pub/scm/fs/xfs/xfs-documentation.git/
linux-7.2.3/Documentation/filesystems/xfs/xfs-online-fsck-design.rst
+5279
View File
File diff suppressed because it is too large Load Diff
linux-7.2.3/Documentation/filesystems/xfs/xfs-self-describing-metadata.rst
+353
View File
@@ -0,0 +1,353 @@
.. SPDX-License-Identifier: GPL-2.0
.. _xfs_self_describing_metadata:
============================
XFS Self Describing Metadata
============================
Introduction
============
The largest scalability problem facing XFS is not one of algorithmic
scalability, but of verification of the filesystem structure. Scalabilty of the
structures and indexes on disk and the algorithms for iterating them are
adequate for supporting PB scale filesystems with billions of inodes, however it
is this very scalability that causes the verification problem.
Almost all metadata on XFS is dynamically allocated. The only fixed location
metadata is the allocation group headers (SB, AGF, AGFL and AGI), while all
other metadata structures need to be discovered by walking the filesystem
structure in different ways. While this is already done by userspace tools for
validating and repairing the structure, there are limits to what they can
verify, and this in turn limits the supportable size of an XFS filesystem.
For example, it is entirely possible to manually use xfs_db and a bit of
scripting to analyse the structure of a 100TB filesystem when trying to
determine the root cause of a corruption problem, but it is still mainly a
manual task of verifying that things like single bit errors or misplaced writes
weren't the ultimate cause of a corruption event. It may take a few hours to a
few days to perform such forensic analysis, so for at this scale root cause
analysis is entirely possible.
However, if we scale the filesystem up to 1PB, we now have 10x as much metadata
to analyse and so that analysis blows out towards weeks/months of forensic work.
Most of the analysis work is slow and tedious, so as the amount of analysis goes
up, the more likely that the cause will be lost in the noise. Hence the primary
concern for supporting PB scale filesystems is minimising the time and effort
required for basic forensic analysis of the filesystem structure.
Self Describing Metadata
========================
One of the problems with the current metadata format is that apart from the
magic number in the metadata block, we have no other way of identifying what it
is supposed to be. We can't even identify if it is the right place. Put simply,
you can't look at a single metadata block in isolation and say "yes, it is
supposed to be there and the contents are valid".
Hence most of the time spent on forensic analysis is spent doing basic
verification of metadata values, looking for values that are in range (and hence
not detected by automated verification checks) but are not correct. Finding and
understanding how things like cross linked block lists (e.g. sibling
pointers in a btree end up with loops in them) are the key to understanding what
went wrong, but it is impossible to tell what order the blocks were linked into
each other or written to disk after the fact.
Hence we need to record more information into the metadata to allow us to
quickly determine if the metadata is intact and can be ignored for the purpose
of analysis. We can't protect against every possible type of error, but we can
ensure that common types of errors are easily detectable. Hence the concept of
self describing metadata.
The first, fundamental requirement of self describing metadata is that the
metadata object contains some form of unique identifier in a well known
location. This allows us to identify the expected contents of the block and
hence parse and verify the metadata object. IF we can't independently identify
the type of metadata in the object, then the metadata doesn't describe itself
very well at all!
Luckily, almost all XFS metadata has magic numbers embedded already - only the
AGFL, remote symlinks and remote attribute blocks do not contain identifying
magic numbers. Hence we can change the on-disk format of all these objects to
add more identifying information and detect this simply by changing the magic
numbers in the metadata objects. That is, if it has the current magic number,
the metadata isn't self identifying. If it contains a new magic number, it is
self identifying and we can do much more expansive automated verification of the
metadata object at runtime, during forensic analysis or repair.
As a primary concern, self describing metadata needs some form of overall
integrity checking. We cannot trust the metadata if we cannot verify that it has
not been changed as a result of external influences. Hence we need some form of
integrity check, and this is done by adding CRC32c validation to the metadata
block. If we can verify the block contains the metadata it was intended to
contain, a large amount of the manual verification work can be skipped.
CRC32c was selected as metadata cannot be more than 64k in length in XFS and
hence a 32 bit CRC is more than sufficient to detect multi-bit errors in
metadata blocks. CRC32c is also now hardware accelerated on common CPUs so it is
fast. So while CRC32c is not the strongest of possible integrity checks that
could be used, it is more than sufficient for our needs and has relatively
little overhead. Adding support for larger integrity fields and/or algorithms
does really provide any extra value over CRC32c, but it does add a lot of
complexity and so there is no provision for changing the integrity checking
mechanism.
Self describing metadata needs to contain enough information so that the
metadata block can be verified as being in the correct place without needing to
look at any other metadata. This means it needs to contain location information.
Just adding a block number to the metadata is not sufficient to protect against
mis-directed writes - a write might be misdirected to the wrong LUN and so be
written to the "correct block" of the wrong filesystem. Hence location
information must contain a filesystem identifier as well as a block number.
Another key information point in forensic analysis is knowing who the metadata
block belongs to. We already know the type, the location, that it is valid
and/or corrupted, and how long ago that it was last modified. Knowing the owner
of the block is important as it allows us to find other related metadata to
determine the scope of the corruption. For example, if we have a extent btree
object, we don't know what inode it belongs to and hence have to walk the entire
filesystem to find the owner of the block. Worse, the corruption could mean that
no owner can be found (i.e. it's an orphan block), and so without an owner field
in the metadata we have no idea of the scope of the corruption. If we have an
owner field in the metadata object, we can immediately do top down validation to
determine the scope of the problem.
Different types of metadata have different owner identifiers. For example,
directory, attribute and extent tree blocks are all owned by an inode, while
freespace btree blocks are owned by an allocation group. Hence the size and
contents of the owner field are determined by the type of metadata object we are
looking at. The owner information can also identify misplaced writes (e.g.
freespace btree block written to the wrong AG).
Self describing metadata also needs to contain some indication of when it was
written to the filesystem. One of the key information points when doing forensic
analysis is how recently the block was modified. Correlation of set of corrupted
metadata blocks based on modification times is important as it can indicate
whether the corruptions are related, whether there's been multiple corruption
events that lead to the eventual failure, and even whether there are corruptions
present that the run-time verification is not detecting.
For example, we can determine whether a metadata object is supposed to be free
space or still allocated if it is still referenced by its owner by looking at
when the free space btree block that contains the block was last written
compared to when the metadata object itself was last written. If the free space
block is more recent than the object and the object's owner, then there is a
very good chance that the block should have been removed from the owner.
To provide this "written timestamp", each metadata block gets the Log Sequence
Number (LSN) of the most recent transaction it was modified on written into it.
This number will always increase over the life of the filesystem, and the only
thing that resets it is running xfs_repair on the filesystem. Further, by use of
the LSN we can tell if the corrupted metadata all belonged to the same log
checkpoint and hence have some idea of how much modification occurred between
the first and last instance of corrupt metadata on disk and, further, how much
modification occurred between the corruption being written and when it was
detected.
Runtime Validation
==================
Validation of self-describing metadata takes place at runtime in two places:
- immediately after a successful read from disk
- immediately prior to write IO submission
The verification is completely stateless - it is done independently of the
modification process, and seeks only to check that the metadata is what it says
it is and that the metadata fields are within bounds and internally consistent.
As such, we cannot catch all types of corruption that can occur within a block
as there may be certain limitations that operational state enforces of the
metadata, or there may be corruption of interblock relationships (e.g. corrupted
sibling pointer lists). Hence we still need stateful checking in the main code
body, but in general most of the per-field validation is handled by the
verifiers.
For read verification, the caller needs to specify the expected type of metadata
that it should see, and the IO completion process verifies that the metadata
object matches what was expected. If the verification process fails, then it
marks the object being read as EFSCORRUPTED. The caller needs to catch this
error (same as for IO errors), and if it needs to take special action due to a
verification error it can do so by catching the EFSCORRUPTED error value. If we
need more discrimination of error type at higher levels, we can define new
error numbers for different errors as necessary.
The first step in read verification is checking the magic number and determining
whether CRC validating is necessary. If it is, the CRC32c is calculated and
compared against the value stored in the object itself. Once this is validated,
further checks are made against the location information, followed by extensive
object specific metadata validation. If any of these checks fail, then the
buffer is considered corrupt and the EFSCORRUPTED error is set appropriately.
Write verification is the opposite of the read verification - first the object
is extensively verified and if it is OK we then update the LSN from the last
modification made to the object, After this, we calculate the CRC and insert it
into the object. Once this is done the write IO is allowed to continue. If any
error occurs during this process, the buffer is again marked with a EFSCORRUPTED
error for the higher layers to catch.
Structures
==========
A typical on-disk structure needs to contain the following information::
struct xfs_ondisk_hdr {
__be32 magic; /* magic number */
__be32 crc; /* CRC, not logged */
uuid_t uuid; /* filesystem identifier */
__be64 owner; /* parent object */
__be64 blkno; /* location on disk */
__be64 lsn; /* last modification in log, not logged */
};
Depending on the metadata, this information may be part of a header structure
separate to the metadata contents, or may be distributed through an existing
structure. The latter occurs with metadata that already contains some of this
information, such as the superblock and AG headers.
Other metadata may have different formats for the information, but the same
level of information is generally provided. For example:
- short btree blocks have a 32 bit owner (ag number) and a 32 bit block
number for location. The two of these combined provide the same
information as @owner and @blkno in eh above structure, but using 8
bytes less space on disk.
- directory/attribute node blocks have a 16 bit magic number, and the
header that contains the magic number has other information in it as
well. hence the additional metadata headers change the overall format
of the metadata.
A typical buffer read verifier is structured as follows::
#define XFS_FOO_CRC_OFF offsetof(struct xfs_ondisk_hdr, crc)
static void
xfs_foo_read_verify(
struct xfs_buf *bp)
{
struct xfs_mount *mp = bp->b_mount;
if ((xfs_sb_version_hascrc(&mp->m_sb) &&
!xfs_verify_cksum(bp->b_addr, BBTOB(bp->b_length),
XFS_FOO_CRC_OFF)) ||
!xfs_foo_verify(bp)) {
XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp, bp->b_addr);
xfs_buf_ioerror(bp, EFSCORRUPTED);
}
}
The code ensures that the CRC is only checked if the filesystem has CRCs enabled
by checking the superblock of the feature bit, and then if the CRC verifies OK
(or is not needed) it verifies the actual contents of the block.
The verifier function will take a couple of different forms, depending on
whether the magic number can be used to determine the format of the block. In
the case it can't, the code is structured as follows::
static bool
xfs_foo_verify(
struct xfs_buf *bp)
{
struct xfs_mount *mp = bp->b_mount;
struct xfs_ondisk_hdr *hdr = bp->b_addr;
if (hdr->magic != cpu_to_be32(XFS_FOO_MAGIC))
return false;
if (!xfs_sb_version_hascrc(&mp->m_sb)) {
if (!uuid_equal(&hdr->uuid, &mp->m_sb.sb_uuid))
return false;
if (bp->b_bn != be64_to_cpu(hdr->blkno))
return false;
if (hdr->owner == 0)
return false;
}
/* object specific verification checks here */
return true;
}
If there are different magic numbers for the different formats, the verifier
will look like::
static bool
xfs_foo_verify(
struct xfs_buf *bp)
{
struct xfs_mount *mp = bp->b_mount;
struct xfs_ondisk_hdr *hdr = bp->b_addr;
if (hdr->magic == cpu_to_be32(XFS_FOO_CRC_MAGIC)) {
if (!uuid_equal(&hdr->uuid, &mp->m_sb.sb_uuid))
return false;
if (bp->b_bn != be64_to_cpu(hdr->blkno))
return false;
if (hdr->owner == 0)
return false;
} else if (hdr->magic != cpu_to_be32(XFS_FOO_MAGIC))
return false;
/* object specific verification checks here */
return true;
}
Write verifiers are very similar to the read verifiers, they just do things in
the opposite order to the read verifiers. A typical write verifier::
static void
xfs_foo_write_verify(
struct xfs_buf *bp)
{
struct xfs_mount *mp = bp->b_mount;
struct xfs_buf_log_item *bip = bp->b_fspriv;
if (!xfs_foo_verify(bp)) {
XFS_CORRUPTION_ERROR(__func__, XFS_ERRLEVEL_LOW, mp, bp->b_addr);
xfs_buf_ioerror(bp, EFSCORRUPTED);
return;
}
if (!xfs_sb_version_hascrc(&mp->m_sb))
return;
if (bip) {
struct xfs_ondisk_hdr *hdr = bp->b_addr;
hdr->lsn = cpu_to_be64(bip->bli_item.li_lsn);
}
xfs_update_cksum(bp->b_addr, BBTOB(bp->b_length), XFS_FOO_CRC_OFF);
}
This will verify the internal structure of the metadata before we go any
further, detecting corruptions that have occurred as the metadata has been
modified in memory. If the metadata verifies OK, and CRCs are enabled, we then
update the LSN field (when it was last modified) and calculate the CRC on the
metadata. Once this is done, we can issue the IO.
Inodes and Dquots
=================
Inodes and dquots are special snowflakes. They have per-object CRC and
self-identifiers, but they are packed so that there are multiple objects per
buffer. Hence we do not use per-buffer verifiers to do the work of per-object
verification and CRC calculations. The per-buffer verifiers simply perform basic
identification of the buffer - that they contain inodes or dquots, and that
there are magic numbers in all the expected spots. All further CRC and
verification checks are done when each inode is read from or written back to the
buffer.
The structure of the verifiers and the identifiers checks is very similar to the
buffer code described above. The only difference is where they are called. For
example, inode read verification is done in xfs_inode_from_disk() when the inode
is first read out of the buffer and the struct xfs_inode is instantiated. The
inode is already extensively verified during writeback in xfs_iflush_int, so the
only addition here is to add the LSN and CRC to the inode as it is copied back
into the buffer.
XXX: inode unlinked list modification doesn't recalculate the inode CRC! None of
the unlinked list modifications check or update CRCs, neither during unlink nor
log recovery. So, it's gone unnoticed until now. This won't matter immediately -
repair will probably complain about it - but it needs to be fixed.