Hard drive formatting is the process of preparing a new,blank hard drive for use. When a hard drive is formatted, all existing data on the drive is permanently erased and cannot be recovered. The formatting process reorganizes the logical structures on the drive into an empty file system so that the operating system can read and write files to the disk. Formatting essentially resets the hard drive to a clean, fresh state ready to save new data.
There are a few main reasons to format a hard drive: to cleanly install a new operating system, to change the file system, to permanently erase all data for security or privacy reasons, to resolve file system errors or corruptions, or to partition and prepare a new drive for use. While formatting deletes all user files and programs stored on the drive, it does not physically erase or overwrite the data at the lowest level. So data remnants may exist until they are overwritten by new data written to the formatted drive.
What Happens During Formatting
Formatting a hard drive essentially prepares it for use by creating a file system and structures to organize data. There are two main types of formatting – low-level (also called full format) and quick format.
A low-level or full format completely erases any existing data on the drive and performs a magnetic-level scan of the entire drive to map out bad sectors. It divides the disk into tracks and sectors. This type of formatting can take several hours depending on the drive size. A full format is more thorough and may help resolve any underlying problems on the disk. Source
In contrast, a quick format simply creates a new file table without scanning the entire drive. It erases file references to make previously deleted data inaccessible, but does not actually remove the data itself. A quick format takes only a few seconds. However, data recovery software may still be able to recover files after a quick format. Quick formats are ideal when you need to quickly reuse a drive and don’t need to permanently erase data. Source
Overall, low-level or full formatting is more secure and robust, while quick formatting is faster but less thorough. The choice depends on your specific needs and how much time you can allot to the formatting process.
File System Overhead
One of the main reasons for lost disk space after formatting is due to file system overhead. File systems like FAT, exFAT, and NTFS all require some storage space to function properly. This overhead is used to store file system structures and file allocation tables.
The file allocation table (FAT) is a central data structure in FAT file systems like FAT16, FAT32, and exFAT that keeps track of used and unused clusters on the storage device [1]. The FAT contains entries for each cluster, indicating whether that cluster is used, free, or defective. This allows the operating system to easily locate free space and allocate new files without fragmentation.
However, the FAT itself takes up a significant amount of space on the drive. The size depends on the number of clusters but is typically between 32KB to 256KB. For large multi-terabyte hard drives, the FAT size can be several megabytes [2]. This overhead for the FAT partially explains why some disk space is “lost” after formatting.
Partition Alignment
One reason hard drives can lose space after formatting is due to partition misalignment. Most hard drives have a sector size of 512 bytes, but newer solid state drives (SSDs) and advanced format drives use a sector size of 4,096 bytes (4K) [1]. When partitions are created, the start and end points may not neatly align with these boundaries. This can lead to unaligned partitions that take up more space than necessary.
For example, a 1 MB partition on a 4K sector drive that starts at sector 1 and ends at sector 256 will take up the space from sector 1 to 257, wasting half of sector 257. Small misalignments like this can add up to several gigabytes of wasted storage space across a whole drive [2]. Aligning partitions properly ensures that no space is wasted at partition edges.
Idle Areas and Spare Sectors
Hard drives have spare sectors set aside to replace sectors that go bad during the drive’s lifetime. These spare sectors are part of the idle areas on the disk that are not visible to the operating system. As explained on the SuperUser forum, modern hard drives have a pool of spare sectors that get swapped in to replace damaged sectors, in a process called remapping (https://superuser.com/questions/1138940/where-do-replacement-hard-drive-sectors-come-from).
The number of spare sectors can range from 256 up to 2560, depending on the drive. When the drive is idle, it may proactively scan for bad sectors and remap them to the spare sector pool before data loss occurs, as described in this Level1Techs forum discussion (https://forum.level1techs.com/t/bad-sectors-only-increase-when-the-drive-is-idle-but-not-under-load-whats-it-doing/200178). This process of remapping bad sectors to spares makes the total available sectors shrink, using up part of the overhead spare area.
In summary, hard drive idle time allows the drive to scan itself and remap detected bad sectors, reducing the total available sectors. This is a normal function that takes place using spare sectors set aside for this purpose.
File System Structures
File systems require structures and metadata to organize and manage the data on a storage device. This overhead takes up disk space. For example, file systems use inodes or file records to track information about each file like permissions, timestamps, and data block pointers (https://superuser.com/questions/682809/how-much-space-do-directories-consume). Directories and folder structures also occupy space proportional to the length of their paths (https://www.bu.edu/tech/support/research/system-usage/using-file-system/file-system-structure/).
Additionally, journaling file systems like NTFS and ext4 utilize journals or log files that keep track of changes before committing them to the main file system. This provides greater integrity but requires more metadata. Finally, some space is used for boot sectors, partition tables, and other structures needed to initialize and manage the formatted drive.
Formatting Tools
Formatting utilities like FAT32 and NTFS have built-in overhead that takes up space on a drive. When a hard drive is formatted with FAT32 or NTFS, the formatting process creates various file system structures to organize and manage the data on the drive [1]. This file system overhead takes up disk space that is then no longer available for storing files.
For example, both FAT32 and NTFS use clusters or allocation units to divide the hard drive into logical blocks for storing data. The smaller the cluster size, the more clusters are needed to fully cover the drive space. More clusters require more overhead to track them, thus reducing available free space. NTFS is generally more efficient than FAT32, but still incurs some overhead [2].
In addition, NTFS maintains a Master File Table to track all files on the volume, as well as log files and metadata. This increases overhead but enhances NTFS capabilities. FAT32 has less features so requires less overhead, but at the cost of efficiency and performance.
Preinstalled Software
One reason for the discrepancy in disk space before and after formatting is the preinstalled software that comes on many hard drives, particularly those sold by OEMs (original equipment manufacturers). OEMs often create hidden partitions on the hard drive to store recovery tools, diagnostic utilities, backup software, device drivers, and even the operating system itself.
For example, Samsung may include partitions for recovery tools, while Dell or HP include a partition for restoring Windows to factory settings. These OEM partitions can take up a significant amount of disk space, from several GBs up to 15-20% of the drive capacity. However, they are hidden and not accessible to the end user in Windows Explorer.
According to this Microsoft forum post, OEM partitions may show up as simply “OEM Partition” and can take up a large chunk of disk space, even 10-15GB, without the user being able to access or delete that partition.
So when a drive is formatted, the OEM partitions remain untouched, while the main C drive partition is reset. This means the OEM software is still taking up space in those hidden partitions, while the visible C drive now starts empty and shows less available space.
Rounding Errors
One factor that contributes to the discrepancy between a drive’s advertised capacity and available space is rounding errors in file size approximations. Hard drives divide space into blocks or clusters, which have a set size like 4KB. When a file is saved, it occupies an entire block even if it is not completely full. For example, a 2KB file will take up a 4KB block.
This means that the actual space used is rounded up to the nearest block size. A 100KB file may display as 104KB for instance. While this difference seems trivial for one file, it adds up with thousands of files on a disk. All of the rounding approximations multiply and accumulate over time, resulting in lost available space.
According to this SuperUser post, rounding errors account for a significant portion of the “missing” space on a newly formatted drive. The post estimates that formatting could incur a loss of 7-10% of the total drive space due to rounding alone.
Conclusions
In summary, there are several main reasons why hard drives lose space after formatting:
File system overhead takes up space to store metadata about files and the disk structure. Common file systems like NTFS and exFAT require reserved space for this structural data.
Partition alignment to physical sector boundaries can result in unused gaps, especially for advanced format drives with 4k sectors.
Idle areas, spare sectors, and excess capacity set aside by manufacturers account for additional lost space during low-level formatting.
File system structures like the master file table in NTFS occupy drive space to organize data and track files.
Preinstalled software utilities, recovery partitions, and hidden system files also consume storage capacity.
Inaccuracies and rounding errors during size conversions from decimal to binary units lead to slightly less reported capacity.
In summary, formatting a drive requires reserving space for structural data, alignment gaps, and pre-loaded software, leading to a slight decrease in available storage space for the end user.