Thursday, 4th of March 2021, 12:00 – 1:00

ndzip: A high-throughput parallel lossless compressor for scientific data


Fabian Knorr
Researcher at DPS, University of Innsbruck


Applications in High Performance Computing (HPC) commonly need to exchange large volumes of floating-point data. Fast, specialized compression schemes can speed up such workloads by reducing the time spent waiting for data transfers. We introduce ndzip, a high-throughput lossless compressor for multidimensional grids of single- and double-precision floating point data. With a parallelization scheme efficiently targeting modern SIMD-capable multicore processors, it achieves a compression throughput close to main memory bandwidth, significantly outperforming existing schemes.
We evaluate the compressor using a representative set of scientific data, demonstrating a competitive trade-off between compression effectiveness and throughput.


Nach oben scrollen