Start Over Save to list Export MARC Display MelCat
     
Author Zhang, Xuechen
Title Rethinking the design and implementation of the i/o software stack for high-performance computing [electronic resource] / by Xuechen Zhang.
Publication Info. 2012.
Location Call No. Status Notes
 Libraries Electronic Books  Electronic Resource - WSU ETD    AVAIL. ONLINE
Note Thesis supervisor: Song Jiang
Thesis Thesis (Ph.D.)--Wayne State University, 2012.
Summary Current I/O stack for high-performance computing is composed of multiple software layers in order to hide users from complexity of I/O performance optimization. However, the design and implementation of a specific layer is usually carried out separately with limited consideration of its impact on other layers, which could result in suboptimal I/O performance because data access locality is weakened, if not lost, on hard disk, a widely used storage medium in high-end storage systems. In this dissertation, we experimentally demonstrated such issues in four different layers, including operating system process management layer and MPI-IO middleware layer on compute server side, and parallel file system layer and disk I/O scheduling layer on data server side. This dissertation makes four contributions towards solving each of the issues. First, we propose a data-driven execution model for <italic>DualPar</italic> to explore opportunity of effective I/O scheduling to alleviate I/O bottleneck via cooperation between the I/O and process schedulers. Its novelty is on the ability to obtain a pool of pre-sorted requests to I/O scheduler in its data-driven execution mode by using process pre-execution and prefetching techniques. Second, realizing that well-formed locality for an MPI program by using collective I/O can be seriously compromised by non-determinism in process scheduling, we proposed <italic>Resonant I/O</italic>, to match the data request pattern with the pattern of file striping over multiple data servers to improve disk efficiency.
Third, since the conventional practice for I/O parallelism using file striping may compromise on-disk data access locality, we proposed <italic>IOrchestrator</italic> scheduling framework which is implemented in PVFS2 parallel file system to improve I/O performance of multi-node storage systems by orchestrating I/O services among programs when such inter-data-server coordination is dynamically determined to be cost effective. Fourth, we developed <italic>iTransformer</italic>, a scheme that employs a small SSD to schedule requests for the data on disk. Being less space constrained than with more expensive DRAM, iTransformer can buffer larger amounts of dirty data before writing it back to the disk, or prefetch a larger volume of data in a batch into the SSD. In both cases high disk efficiency can be maintained for highly concurrent requests.
System Details Mode of access: World Wide Web.
System requirements: Adobe Reader.
Added Author Song Jiang, advisor.
Permanent url for this catalog record: