Portal de Periódicos da CAPES

Two High-Performance Alternatives to ZLIB Scientific-Data Compression

2014; Springer Science+Business Media; Linguagem: Inglês

10.1007/978-3-319-09147-1_45

1611-3349

Samuel Santos Almeida, Vítor Oliveira, António Manuel Silva Pina, Manuel Melle‐Franco,

Parallel Computing and Optimization Techniques

ZLIB is used in diverse frameworks by the scientific community, both to reduce disk storage and to alleviate pressure on I/O. As it becomes a bottleneck on multi-core systems, higher throughput alternatives must be considered, exploring parallelism and/or more effective compression schemes. This work provides a comparative study of the ZLIB, LZ4 and FPC compressors (serial and parallel implementations), focusing on CR, bandwidth and speedup. LZ4 provides very high throughput (decompressing over 1GB/s versus 120MB/s for ZLIB) but its CR suffers a degradation of 5-10%. FPC also provides higher throughputs than ZLIB, but the CR varies a lot with the data. ZLIB and LZ4 can achieve almost linear speedups for some datasets, while current implementation of parallel FPC provides little if any performance gain. For the ROOT dataset, LZ4 was found to provide higher CR, scalability and lower memory consumption than FPC, thus emerging as a better alternative to ZLIB.