73 lines
2.8 KiB
ReStructuredText
73 lines
2.8 KiB
ReStructuredText
===================================
|
|
pNFS block layout server user guide
|
|
===================================
|
|
|
|
The Linux NFS server now supports the pNFS block layout extension. In this
|
|
case the NFS server acts as Metadata Server (MDS) for pNFS, which in addition
|
|
to handling all the metadata access to the NFS export also hands out layouts
|
|
to the clients to directly access the underlying block devices that are
|
|
shared with the client.
|
|
|
|
To use pNFS block layouts with the Linux NFS server the exported file
|
|
system needs to support the pNFS block layouts (currently just XFS), and the
|
|
file system must sit on shared storage (typically iSCSI) that is accessible
|
|
to the clients in addition to the MDS. As of now the file system needs to
|
|
sit directly on the exported volume, striping or concatenation of
|
|
volumes on the MDS and clients is not supported yet.
|
|
|
|
On the server, pNFS block volume support is automatically if the file system
|
|
support it. On the client make sure the kernel has the CONFIG_PNFS_BLOCK
|
|
option enabled, the blkmapd daemon from nfs-utils is running, and the
|
|
file system is mounted using the NFSv4.1 protocol version (mount -o vers=4.1).
|
|
|
|
If the nfsd server needs to fence a non-responding client it calls
|
|
/sbin/nfsd-recall-failed with the first argument set to the IP address of
|
|
the client, and the second argument set to the device node without the /dev
|
|
prefix for the file system to be fenced. Below is an example file that shows
|
|
how to translate the device into a serial number from SCSI EVPD 0x80::
|
|
|
|
cat > /sbin/nfsd-recall-failed << EOF
|
|
|
|
.. code-block:: sh
|
|
|
|
#!/bin/sh
|
|
|
|
CLIENT="$1"
|
|
DEV="/dev/$2"
|
|
EVPD=`sg_inq --page=0x80 ${DEV} | \
|
|
grep "Unit serial number:" | \
|
|
awk -F ': ' '{print $2}'`
|
|
|
|
echo "fencing client ${CLIENT} serial ${EVPD}" >> /var/log/pnfsd-fence.log
|
|
EOF
|
|
|
|
If the nfsd server needs to fence a non-responding client and the
|
|
fencing operation fails, the server logs a warning message in the
|
|
system log with the following format:
|
|
|
|
FENCE failed client[IP_address] clid[#n] device[dev_name]
|
|
|
|
where:
|
|
|
|
- IP_address: refers to the IP address of the affected client.
|
|
- #n: indicates the unique client identifier.
|
|
- dev_name: specifies the name of the block device related
|
|
to the fencing attempt.
|
|
|
|
The server will repeatedly retry the operation indefinitely. During
|
|
this time, access to the affected file is restricted for all other
|
|
clients. This is to prevent potential data corruption if multiple
|
|
clients access the same file simultaneously.
|
|
|
|
To restore access to the affected file for other clients, the admin
|
|
needs to take the following actions:
|
|
|
|
- shutdown or power off the client being fenced.
|
|
- manually expire the client to release all its state on the server::
|
|
|
|
echo 'expire' > /proc/fs/nfsd/clients/clid/ctl
|
|
|
|
where:
|
|
|
|
- clid: is the unique client identifier displayed in the system log.
|