Data storage technology has advanced rapidly in recent years to keep up with the exponential growth of data. This article will provide an overview of the latest innovations in data storage, from new mediums like flash storage and DNA storage to improvements in traditional magnetic and optical storage. We will also discuss new storage architectures like cloud storage and edge computing that change how and where data is stored. The key criteria for evaluating new storage tech are capacity, speed, durability, portability, and cost. This article aims to summarize the state-of-the-art in each major category of storage technology and provide insights into where the industry is heading.
Flash Storage
Flash storage, also known as solid-state drives (SSDs), use flash memory rather than spinning platters like traditional hard disk drives (HDDs). Some key advantages of flash storage over HDDs include faster access times, higher throughput, improved reliability, and lower latency. However, flash storage tends to be more expensive and has lower capacities than HDDs.1
Some of the latest innovations in flash storage include:2
- Increased density using 3D NAND technology to stack flash memory cells vertically.
- PCIe 4.0 SSDs doubling the throughput of previous generations.
- QLC (4-bit) NAND providing higher capacities but lower endurance.
- Optane SSDs combining attributes of memory and storage.
- NVMe over Fabrics allowing flash storage to be shared over a network.
While price remains a limiting factor for flash storage, capacity and speed continue to increase while costs decline gradually. Enterprise and data center adoption is driving much of the innovation as flash becomes the preferred medium for primary storage performance.
Hard Disk Drives
Hard disk drives (HDDs) have been the dominant data storage technology for several decades. HDDs store data on rapidly rotating platters coated with magnetic material. A read/write head floats just above the platter to magnetically record data.
Key innovations in HDDs have focused on increasing areal density, which is the amount of data that can be stored per square inch of disk. Through improvements in magnetic recording heads, media materials, and error correction, areal density has increased exponentially over the years (PDF) Trends in Storage Technologies.
Two new technologies promise to continue increasing HDD areal density: shingled magnetic recording (SMR) and technologies like heat-assisted magnetic recording (HAMR) and microwave-assisted magnetic recording (MAMR). SMR overlaps data tracks to increase areal density at the cost of slower write speeds. HAMR and MAMR use external energy like lasers or microwaves to temporarily heat the magnetic medium, allowing data to be recorded on higher density media An Analysis Of Flash And HDD Technology Trends.
Magnetic Tape
Magnetic tape has seen a resurgence in recent years thanks to its low cost, high capacity, and durability for long-term archival storage. The Linear Tape-Open (LTO) format is the dominant standard, with roadmaps outlining capacity increases for future generations of drives. The latest LTO-9 drives provide a native capacity of 18 TB per tape cartridge and support compressed capacities up to 72 TB [1].
Compared to hard disk drives, magnetic tape has much higher latency when accessing data randomly. However, sequential read/write performance can match HDDs. The key advantages of tape are the low cost per terabyte, longevity with a 30+ year shelf life, and portability for offsite storage and backup [2]. Tape libraries with automation are commonly used for data backup and archiving.
Tape is well-suited for infrequently accessed “cold” data, while HDDs and SSDs are better for frequently accessed “hot” data. The high capacity and low cost of tape make it a popular choice for long-term data retention and compliance archives. However, restoring data from tape has longer lead times compared to spinning disks. Overall, tape fills an important role in a tiered storage architecture and strategy.
[1] https://www.techtarget.com/searchstorage/tip/Whats-driving-the-resurgence-in-tape-storage-use
[2] https://corodata.com/tape-backups-still-used-today
Optical Storage
Optical storage like CDs, DVDs, and Blu-ray discs were once very popular for consumer data storage. While declining in popularity, optical discs are still used today for archival storage thanks to their longevity and reliability. Blu-ray discs have capacities up to 128GB for BDXL discs using 3 layers (Tomcoughlin, 2019).
One of the most durable forms of optical media today is M-DISC, which uses inorganic materials to resist data degradation. M-DISC Blu-ray discs are rated to last up to 1,000 years if stored properly (Milloni, 2021). Optical discs also remain useful for cold storage of data that does not need frequent access.
For continued growth, optical storage technologies are increasing capacities and transfer speeds. Blu-ray recently reached 16X write speeds, allowing burning a 100GB disc in just 6-7 minutes. Multi-layer technologies are also expanding capacities up to 1TB for a single disc (M-DISC, 2020). While niche today, optical storage retains advantages for longevity, removability, and security.
Sources:
Tomcoughlin, M. (2019). Digital Storage Technology Roadmaps and Implications for Cloud Computing. https://tomcoughlin.com/wp-content/uploads/2019/06/Digital-Storage-Technology-Roadmaps-and-Implications-for-Cloud-Computing-iNEMI-061317r.pdf
Milloni, G. (2021). M-DISC Optical Media. https://www.mdisc.com/what-is-mdisc/
M-DISC. (2020). M-DISC 4.0: A Thousand Years in a Disc. https://www.mdisc.com/m-disc-4-0-a-thousand-years-in-a-disc/
DNA Storage
DNA storage is an emerging technology that encodes digital data into DNA molecules. It takes advantage of DNA’s density, durability, and longevity as a storage medium.
To store data in DNA, the digital data is first converted into binary code. The binary code is then translated into DNA base pairs, made up of adenine, cytosine, guanine and thymine. Short segments of synthetic DNA are produced with specific sequences that represent the encoded data. These short DNA strands can be dehydrated and preserved for long periods of time.
To retrieve the stored data, the DNA sequences are read using sequencing techniques. The base pair sequences are translated back into binary code, which is then converted into the original data format. Research shows DNA can store data with much higher density compared to existing storage media, with a maximum storage density of 1 exabyte per cubic millimeter (one billion gigabytes).
Current capabilities of DNA storage include encoding text files, images, videos, and entire computer operating systems into DNA. However, costs and limitations of DNA sequencing technology are still barriers to viable commercial use. As sequencing technology improves and costs decrease, experts predict DNA storage could become widespread within 5-10 years.
Cloud Storage
Cloud storage refers to the storage of data online in the cloud, typically using a cloud computing provider. Major cloud storage providers include Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, Dropbox, Box, and Apple iCloud (Enterprise Storage Forum). Cloud storage offers several benefits compared to traditional on-premises storage such as scalability, accessibility from anywhere, and cost savings. However, there are also potential drawbacks like reliance on an internet connection and risks related to data security.
Some of the latest services and trends in cloud storage include the growth of hybrid cloud storage solutions, increased use of cloud-native applications, multi-cloud strategies, edge computing integration, built-in ransomware defenses, and quantum-safe encryption (American Cloud). There is also a shift towards storing more data in the cloud than on-premises. Nearline “cold” cloud storage, which offers lower costs for less frequently accessed data, continues to grow in popularity.
Overall, cloud storage is seeing widespread adoption due to its flexibility, scalability, and cost-effectiveness. However, factors like security and compliance requirements still favor on-premises storage in some cases. The future is likely to see expanded use of hybrid cloud models combining the benefits of cloud and on-site storage.
Edge Storage
Local edge storage refers to storage devices located close to the edge of the network, near users and data sources. This allows for faster data processing and reduced latency compared to transferring data to distant centralized data centers. Edge storage is becoming increasingly important with emerging technologies like 5G, Internet of Things (IoT), and augmented/virtual reality (AR/VR) that require massive data transfer and ultra-low latency.
Key innovations in edge storage include new hardware optimized for edge environments, like micro data centers and ruggedized storage arrays. There is also development of software-defined storage architectures tailored for distributed edge infrastructure. Major storage vendors like Seagate are developing edge-specific products, like the Seagate Exos AP 5U84 storage array for edge applications.
Use cases benefiting from local edge storage include smart cities, autonomous vehicles, AR/VR, content delivery networks (CDNs), IoT networks, and industrial environments. By processing and storing data locally on edge devices rather than transmitting to the cloud, edge storage enables real-time data analytics and quicker response times for time-sensitive applications.
As noted by Anastasios Tsogkas, “During our discussions, we delved into cutting-edge storage trends, groundbreaking innovations, and the pressing challenges that lie ahead.” (Tsogkas)
Storage Class Memory
Storage Class Memory (SCM) is an emerging technology that aims to fill the gap between DRAM and NAND flash memory in terms of cost, performance, and persistence.[1] SCM combines the benefits of memory and storage into one solution.
An overview of SCM is that it provides near-DRAM performance at a lower cost per bit than DRAM, while also being non-volatile like flash memory.[2] This means it can provide faster access to data than NAND flash, but doesn’t get wiped when power is removed.
One implementation of SCM is 3D XPoint technology developed by Intel and Micron. 3D XPoint uses a crosspoint architecture that stacks memory cells vertically across a 3D grid.[2] This allows for greater density and performance.
Some benefits of SCM and 3D XPoint include faster read/write speeds than NAND flash, higher endurance, and data persistence. This makes it well-suited for applications like databases, analytics, and fast caching.[1]
However, SCM is still an emerging technology and has disadvantages like higher cost per GB than NAND flash and lower density.[1] Overall though, SCM represents an exciting memory innovation that can expand workload possibilities.
[1] https://www.linkedin.com/pulse/storage-class-memory-market-growth-research-6qbse/
[2] https://www.linkedin.com/pulse/storage-class-memory-market-latest-innovations-odpnf/
Conclusion
In summary, some of the key data storage technologies to emerge in recent years include flash storage, hard disk drives, magnetic tape, optical storage, DNA storage, cloud storage, edge storage, and storage class memory. Each technology offers its own advantages and use cases. Flash storage provides fast access times for hot data, while tape offers high capacities for cold data archiving. DNA storage holds promise as an ultra-dense long-term archival medium. Meanwhile, cloud and edge storage solutions allow data to be accessed from anywhere. Storage class memory sits between standard DRAM and SSDs, offering near-DRAM speeds.
Looking ahead, we can expect continued advances in all of these storage mediums. DNA storage in particular may see significant progress toward commercial viability. Cloud and edge computing will drive demand for distributed storage innovations. At the same time, time-tested technologies like tape and HDDs will co-exist for the foreseeable future thanks to continual density improvements. Overall, enterprises will benefit from having access to a diverse data storage ecosystem with options to match varying performance, cost and capacity requirements.
