10.6084/m9.figshare.11786787.v1 Xian-He Sun Xian-He Sun Framework: Software: NSCI: Collaborative Research: Hermes: Extending the HDF Library to Support Intelligent I/O Buffering for Deep Memory and Storage Hierarchy Systems figshare 2020 NSF-CSSI-2020-Talk Framework Software Datamanagement Multi-tiered I/O Computer System Architecture 2020-02-03 18:25:15 Poster https://figshare.com/articles/poster/Framework_Software_NSCI_Collaborative_Research_Hermes_Extending_the_HDF_Library_to_Support_Intelligent_I_O_Buffering_for_Deep_Memory_and_Storage_Hierarchy_Systems/11786787 Modern HPC and distributed systems come equipped with deep memory and storage hierarchies (DMSH). The expert knowledge required to manage these multi-tier storage environments puts their benefits out of reach for most scientists and researchers. In this project, we propose the design and development of Hermes, a new, heterogeneous-aware, multi-tiered, dynamic, and distributed I/O buffering platform which provides: 1) Vertical and horizontal distributed buffering in the DMSH: 2) Selective buffering; 3) Adaptive and dynamic buffering via system and application profiling. We are developing new buffering algorithms and mechanisms that address the challenges of a DMSH ecosystem. This effort will eventually boost HDF5 core technology and facilitate an agile architecture that will allow the evolution of next generation I/O and will address the increasingly challenging scale and complexity of future systems. Hermes software is intended to support new scientific/engineering methodologies and will be carefully designed, implemented, and thoroughly tested.