What is XFS?
XFS is a high-performance 64-bit journaling file system created by Silicon Graphics, Inc (SGI) in 1993 [1]. It was designed for use on large storage systems like those used for video creation and scientific computing. Some of the main features and capabilities of XFS include:
– Very high scalability and performance, able to handle large files and file systems up to 8 exabytes [2].
– Journaling for quick crash recovery.
– Native high performance metadata transactions.
– Concurrent IO for parallel throughput.
– Sustained high throughput under heavy loads.
XFS has been ported to Linux and is well supported across major Linux distributions like Red Hat Enterprise Linux. It is optimized to deliver maximum performance on Linux systems utilizing very large storage capacities.
XFS Pros
XFS has several advantages that make it a popular choice for many use cases:
Scalability – XFS was designed for scalability and can efficiently manage very large filesystems and high volumes of data. It supports filesystems up to 8 exabytes and files up to 8 exabytes as well. This makes it a good choice for large storage servers and big data workloads. 1
Performance – Tests show XFS provides excellent performance, especially with very large files and high levels of parallel I/O. The extent-based allocation used by XFS also helps reduce fragmentation. This makes it well-suited for multimedia production and scientific computing workloads. 2
Reliability – XFS utilizes journaling to provide good reliability and quick crash recovery. Metadata integrity is protected with CRCs. Online defragmentation can be done while the filesystem is mounted and active. These features reduce the likelihood of data loss.3
Easy Administration – XFS provides good administrative utilities for maintenance tasks like growing or shrinking filesystems on the fly. Creating and managing XFS filesystems is straightforward. This makes administration simpler compared to more complex filesystems.
XFS Cons
XFS has some drawbacks compared to other filesystems like ext4 that are worth considering before deciding to use it:
Higher memory requirements – XFS needs more RAM than ext4, especially for filesystems over 1TB. This is because XFS uses B+ trees for allocation which take up more memory. Servers with limited RAM may want to avoid XFS or allocate sufficient memory if using it [1].
Less flexible than other filesystems – Unlike ext4, XFS cannot shrink filesystems, only grow them. This makes it less flexible for systems where available storage may change. XFS also does not support as many inode sizes as ext4 [2].
Limited tooling/utilities – There are fewer mature utilities for tasks like defragmentation and filesystem checking compared to ext4. The xfs_repair tool is not as robust as e2fsck for ext4. This can make maintenance and recovery more challenging [3].
XFS Use Cases
XFS is well-suited for certain use cases thanks to its performance advantages and robust feature set. Some of the best use cases for XFS include:
Media storage and editing: XFS handles large files very efficiently, making it ideal for storing and manipulating high-resolution audio, video, and image files. Its high throughput supports real-time, multi-stream video editing.
Big data analytics: The parallel I/O capabilities of XFS allow it to rapidly ingest and process large datasets for big data analytics. It scales smoothly to handle massive storage requirements.
High performance computing: XFS provides fast metadata operations and high throughput I/O that HPC applications demand. It is highly optimized for performance on Linux HPC clusters.
Comparing XFS to ext4
When it comes to performance, benchmarks have shown XFS to have better throughput for large files, while ext4 has lower latency for small files (XFS vs EXT4 Filesystem | Difference Between XFS and EXT4). XFS uses extent-based allocation rather than block-based like ext4, allowing it to work better with large files and large filesystems. However, ext4 may be more suitable for things like boot partitions or root filesystems where many small files are present.
In terms of reliability, ext4 is generally considered more mature and stable, while XFS has historically had issues with data corruption after crashes, especially on RAID arrays. However, XFS has improved greatly in this regard in recent years. One advantage XFS does offer for reliability is checksumming to detect silent data corruption (Linux filesystems EXT4 vs XFS, what to choose, what is better).
Some key feature differences: XFS supports metadata journaling while ext4 journals both data and metadata. XFS allows online defragmentation while ext4 does not. XFS has snapshot capabilities built-in, while ext4 would need an add-on tool like Snapper. Ext4 supports hole punching for sparse files, which XFS does not. In general, XFS has more enterprise-oriented features while ext4 aims for general purpose use.
Using XFS on Linux
To format a device with XFS in Linux, use the mkfs.xfs command. For example:
mkfs.xfs /dev/sdb1
This will create an XFS filesystem on the /dev/sdb1 device. You can specify additional mkfs.xfs options like block size, inode size, etc. according to your use case (1).
To mount an XFS filesystem on Linux, use the mount command. For example:
mount /dev/sdb1 /mnt
This will mount the XFS filesystem on /dev/sdb1 to the /mnt mount point. You can specify various mount options like nouuid, noatime, etc. (2).
Some useful commands for managing XFS filesystems include:
- xfs_info – show details about an XFS filesystem
- xfs_repair – repair corrupt XFS filesystems
- xfs_growfs – expand an XFS filesystem
- xfs_quota – manage disk quotas for XFS
Refer to the xfsprogs documentation for more details on these and other XFS administration commands (3).
(1) https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/ch-xfs
(2) https://www.linuxbuzz.com/create-xfs-file-system-in-linux/
(3) https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/managing_file_systems/assembly_managing-an-xfs-filesystem_managing-file-systems
XFS Performance Tuning
There are several ways to tune XFS performance for optimal speed on Linux systems. This includes adjusting filesystem options, mount options, and tuning specifically for SSDs vs HDDs.
Some key filesystem options to optimize performance include:1
- Increasing xfs_log_size – This allows more filesystem operations to be logged before syncing.
- Setting xfs_rotorstep to a higher value – This reduces how often XFS rotates the log.
- Increasing xfs_logbufs – This buffers more metadata log operations.
Mount options can also improve performance:2
- noatime – Prevents writing inode access times to the filesystem.
- nobarrier – Disables barriers for faster, but less safe, write caching.
- largeio – Improves performance doing large IO operations.
For SSDs, tuning XFS for reduced logging and barriers can help.3 For HDDs, increasing allocation group size with allocsize can boost throughput.
Data Recovery on XFS
Recovering lost or deleted data from XFS filesystems can be challenging compared to other filesystems like ext4 due to how XFS stores metadata. However, there are some built-in and third party tools that can help:
Built-in tools:
The xfs_repair utility can help fix some filesystem errors and recover data, but does not recover files marked as deleted.[1] The xfs_admin tool can restore deleted files still available in free space, but lacks a user friendly interface.[2]
Third party tools:
xfs_undelete is an open source tool specifically for recovering deleted XFS files even after a format. It searches the metadata btrees for file records.[3] Commercial tools like Reclaime and R-Studio also claim to recover XFS partitions and raid arrays.
Challenges:
XFS does not store as much metadata about each file compared to other filesystems. It also aggressively reuses space marked as deleted. This makes undeletion more difficult unless file records can be found in metadata.[1] Corruption in key metadata structures like the allocation btrees can make recovery impossible.
When to Use XFS
XFS is well-suited for certain use cases on Linux systems:
- High throughput, large file workloads like video editing or scientific computing. XFS excels at fast, concurrent read/write operations on large files (https://unix.stackexchange.com/questions/525613/xfs-vs-ext4-performance).
- Multi-terabyte partitions. XFS scales very well as partition sizes grow into the petabyte range (https://www.partitionwizard.com/partitionmanager/xfs-vs-ext4.html).
- Systems with high amounts of RAM. XFS caches file system metadata aggressively in RAM, so it performs best when ample memory is available (https://www.reddit.com/r/DataHoarder/comments/11ar65f/linux_filesystems_ext4_vs_xfs_what_to_choose_what/).
XFS may not be the best choice in these situations:
- Small files and directories. XFS can be slower than ext4 for workloads dominated by small files due to overhead from Extent-based allocation (https://unix.stackexchange.com/questions/525613/xfs-vs-ext4-performance).
- Limited memory systems. With less RAM for caching, XFS performance suffers compared to more memory efficient file systems.
- Boot drives. Some Linux distributions don’t recommend XFS for /boot due to potential startup issues.
The Future of XFS
XFS continues to be actively developed and improved by the open source community. The most recent XFS release in January 2023 included performance enhancements for multi-threaded workloads and improved scalability on large systems [1]. Developers are working on adding new features like reflinks and space deduplication in upcoming releases.
XFS is a mature and time-tested filesystem, but some newer filesystems like Btrfs and ZFS offer compelling features that XFS currently lacks. Btrfs supports snapshotting, checksums, and built-in RAID while ZFS offers pooled storage, snapshots, and data integrity checking. However, XFS still has advantages in its proven stability and high performance for certain workloads like large files and directories [2].
While adoption of XFS has slowed, it is still widely used, especially in high performance computing environments. As of 2023, XFS holds about 5% market share, which shows it retains a solid user base. But newer filesystems like Btrfs are gaining popularity and could potentially overtake XFS in the long run [3]. The XFS developers are working to keep it competitive by adding next-generation features.
