Framework: Software: NSCI: Collaborative Research: Hermes: Extending the HDF Library to Support Intelligent I/O Buffering for Deep Memory and Storage Hierarchy Systems
posterposted on 03.02.2020 by Xian-He Sun
Poster sessions are particularly prominent at academic conferences. Posters are usually one frame of a powerpoint (or similar) presentation and are represented at full resolution to make them zoomable.
Modern HPC and distributed systems come equipped with deep memory and storage hierarchies (DMSH). The expert knowledge required to manage these multi-tier storage environments puts their benefits out of reach for most scientists and researchers. In this project, we propose the design and development of Hermes, a new, heterogeneous-aware, multi-tiered, dynamic, and distributed I/O buffering platform which provides: 1) Vertical and horizontal distributed buffering in the DMSH: 2) Selective buffering; 3) Adaptive and dynamic buffering via system and application profiling. We are developing new buffering algorithms and mechanisms that address the challenges of a DMSH ecosystem. This effort will eventually boost HDF5 core technology and facilitate an agile architecture that will allow the evolution of next generation I/O and will address the increasingly challenging scale and complexity of future systems. Hermes software is intended to support new scientific/engineering methodologies and will be carefully designed, implemented, and thoroughly tested.