SCHEDULE: NOV 10-16, 2007

Zest: The Maximum Reliable TBytes/sec/$ for Petascale Systems

Session: Storage Challenge Finalists

Event Type: Challenge Finalist

Time: 1:50pm - 2:10pm

Session Chair: Raymond L. Paden

Team Member(s): Nathan T. B. Stone, Doug Balog, Paul Nowoczynski, Jason Sommerfield, Jared Yanovich

Location: A10 / A11

Abstract:
PSC has developed a prototype distributed file system infrastructure that vastly accelerates aggregated write bandwidth on large compute platforms. Write bandwidth, more than read bandwidth, is the dominant bottleneck in HPC I/O scenarios due to writing checkpoint data, visualization data and post-processing (multi-stage) data. We have prototyped a scalable solution on the Cray XT3 compute platform that will be directly applicable to future petascale compute platforms having of order 10^6 cores. Our design emphasizes high-efficiency scalability, low-cost commodity components, lightweight software layers, end-to-end parallelism, client-side caching and software parity, and a unique model of load-balancing outgoing I/O onto high-speed intermediate storage followed by asynchronous reconstruction to a 3rd-party parallel file system. The absence of a central metadata service further reduces latency, allowing for the maximum reliable performance per unit cost for petascale systems.

Chair/Team Member Details:

Raymond L. Paden (Chair)
IBM

Nathan T. B. Stone
Pittsburgh Supercomputing Center

Doug Balog
Pittsburgh Supercomputing Center

Paul Nowoczynski
Pittsburgh Supercomputing Center

Jason Sommerfield
Pittsburgh Supercomputing Center

Jared Yanovich
Pittsburgh Supercomputing Center

Home | About | Contact Us | Registration