08 Apr RESEARCH PAPER: Datacenter Memory and Storage: Past, Present, Future

The server industry is in the middle of a once-every-20-years shift. What has happened in the past is a good indicator of what is about to happen. Given radical past transitions, it is very likely that many of today’s key technologies probably will fade out of the market by the year 2020.

This paper focuses on server primary system memory and secondary storage technologies spanning the past, present, and future. We begin with an overview of historical server architectures going all the way back to the 1960s. Then we outline the current state of server hardware, software, and how they are used today. We conclude with a description of potentially important technologies upcoming in the next decade.

Table of Contents

  • Summary
  • History of Server Architecture
  • Hardware Technologies
  • Integration
  • Memory vs. Storage
  • Registers
  • Primary Memory
  • Secondary Storage
  • Tertiary Storage
  • Software Technologies
  • Physical Memory Model
  • Virtual Memory Model
  • Non-Uniform Memory Access (NUMA) Memory Model
  • RDMA (Remote Direct Memory Access) Memory Model
  • Virtual Machines (VM) Model
  • Usage Models
  • Current State of Server Architecture
  • Hardware Technologies
  • Application Specific Acceleration and Shared Primary System Memory
  • Memory and Storage Access Over Networks
  • Non-Volatile Chip Technologies Move onto Primary Memory Bus
  • Software Technologies
  • Solid State Drives (SSD) Will Replace Hard Disk Drives (HDD)
  • Impact of NVDIMM on the Operating System
  • Impact of NVDIMM on the BIOS
  • Usage Models
  • Future Datacenter Architecture Directions
  • Hardware Technologies
  • Integration Continues: Packaging Multiple Chips
  • Rack Scale Architecture
  • Non-Volatile Memories
  • Software Technologies
  • Usage Models
  • Conclusion
  • Glossary
  • Figure 1: Major Parts of a Server
  • Figure 2: Late 1980s Separate Chips
  • Figure 3: 1990s x86 Integration
  • Figure 4: Current Server SoC and Package Integration
  • Figure 5: Possible Future of Server SoC Integration
  • Figure 6: Mainframe Data Pyramid, Circa 1965
  • Figure 7: Updated Data Pyramid
  • Figure 8: DIMM (Dual In-Line Memory Module)
  • Figure 9: A Server Motherboard with DIMMs and Other Components
  • Figure 10: A Storage Server Motherboard with Many DAS Devices
  • Figure 11: Physical Memory Model
  • Figure 12: Virtual Memory Model
  • Figure 13: NUMA Memory Model
  • Figure 14: RDMA Memory Model
  • Figure 15: A Physical Server, Not Virtualized
  • Figure 16: A Virtualized Server
  • Figure 17: A. PCI-Express Processor Card, B. PCI-Express Storage Card, & C. PCI-Express Network Card
  • Figure 18: Server Processor Shipments Over Time
  • Figure 19: Micron Automata Processor Attaches to DIMM Connector
  • Figure 20: A. Common “Flash Drive”, B. Flash Drive with Exposed Flash Storage Chip, & C. Flash Storage Chip
  • Figure 21: Battery Backed Up DRAM General Architecture
  • Figure 22: NVDIMM-N General Architecture
  • Figure 23: NVDIMM-F General Architecture
  • Figure 24: NVDIMM-unnamed General Architecture
  • Figure 25: Overview of BIOS Boot Procedure
  • Figure 26: Competing Memory Technologies: Relative Performance
  • Figure 27: Competing Storage Technologies: Relative Performance
  • Figure 28: Potential Future Data Pyramid

You can download the paper here.

Companies Cited

  • Amazon
  • AMD
  • Dell
  • Diablo Technologies
  • Facebook
  • Google
  • HP
  • IBM
  • Intel
  • Lenovo
  • Micron
  • Microsoft
  • SanDisk
  • Storage Networking Industry Association (SNIA)
  • VMware