Installing parallel I/O libraries

High-resolution simulations (e.g. modeling the entire Greenland ice sheet using the 900 m resolution and a grid of 1700 by 3000 points) can produce huge amounts of data and I/O can become a bottleneck.

PISM supports several parallel I/O approaches that take advantage of parallel NetCDF and PnetCDF. The administrators of your HPC system should be able to help you install these libraries, but it may be easier to install them in the “home” directory instead.

This section describes the steps needed to build

  • NetCDF with parallel I/O based on HDF5 (needed to use PISM’s option -o_format netcdf4_parallel),

  • PNetCDF (needed to use PISM’s option -o_format pnetfdf),

Scripts below install libraries in ~/local/library_name, using ~/local/build/library_name to build them.

Section Building PISM with libraries in non-standard locations explains how build PISM with these libraries.

Section PISM’s I/O performance explains how to use them in PISM.

Installing HDF5-based parallel NetCDF

Installing HDF5

 1# Install HDF5 1.12.0 with parallel I/O in ~/local/hdf5,
 2# using ~/local/build/hdf5 as the build directory.
 3
 4version=1.12.0
 5prefix=${prefix:-$HOME/local/hdf5}
 6build_dir=${build_dir:-$HOME/local/build/hdf5}
 7hdf5_site=https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.12
 8url=${hdf5_site}/hdf5-${version}/src/hdf5-${version}.tar.gz
 9
10mkdir -p ${build_dir}
11pushd ${build_dir}
12
13wget -nc ${url}
14tar xzf hdf5-${version}.tar.gz
15
16pushd hdf5-${version}
17
18export CC=${CC:-mpicc}
19
20CFLAGS=-w ./configure \
21  --disable-static \
22  --enable-parallel \
23  --prefix=${prefix} 2>&1 | tee hdf5_configure.log
24
25make all 2>&1 | tee hdf5_compile.log
26make install 2>&1 | tee hdf5_install.log
27
28popd
29popd

To compile parallel HDF5 one should use the MPI compiler wrapper mpicc and run configure with the option --enable-parallel. The flag -w is not important: it hides numerous compiler warnings emitted when building HDF5.

Installing NetCDF

 1# Install parallel NetCDF using parallel HDF5 in ~/local/hdf5 and
 2# ~/local/build/netcdf as a build directory.
 3
 4lib_dir=${lib_dir:-$HOME/local}
 5hdf5=${lib_dir}/hdf5
 6pnetcdf=${lib_dir}/pnetcdf
 7
 8version=4.9.2
 9prefix=${prefix:-$HOME/local/netcdf}
10build_dir=${build_dir:-$HOME/local/build/netcdf}
11url=https://github.com/Unidata/netcdf-c/archive/refs/tags/v${version}.tar.gz
12
13mkdir -p ${build_dir}
14cd ${build_dir}
15
16wget -nc ${url}
17tar zxf v${version}.tar.gz
18
19cd netcdf-c-${version}
20
21export CPPFLAGS="-I${hdf5}/include -I${pnetcdf}/include"
22export LDFLAGS="-L${hdf5}/lib -L${pnetcdf}/lib"
23export CC=${CC:-mpicc}
24
25./configure \
26  --enable-netcdf4 \
27  --enable-parallel4 \
28  --enable-pnetcdf \
29  --disable-dap \
30  --disable-libxml2 \
31  --disable-byterange \
32  --prefix=${prefix} 2>&1 | tee netcdf_configure.log
33
34make all 2>&1 | tee netcdf_compile.log
35make install 2>&1 | tee netcdf_install.log

Here we use the same compiler wrapper and set CPPFLAGS and LDFLAGS to select the parallel HDF5 library installed earlier. The option --enable-netcdf4 is required for parallel I/O; --disable-dap is not required (it disables a NetCDF feature not used by PISM).

Installing PnetCDF

 1# Install PnetCDF 1.12.1 in ~/local/pnetcdf,
 2# using ~/local/build/pnetcdf as a build directory.
 3
 4version=1.12.1
 5prefix=${prefix:-$HOME/local/pnetcdf}
 6build_dir=${build_dir:-$HOME/local/build/pnetcdf/}
 7url=https://parallel-netcdf.github.io/Release/pnetcdf-${version}.tar.gz
 8
 9mkdir -p ${build_dir}
10pushd ${build_dir}
11
12wget -nc ${url}
13tar xzf pnetcdf-${version}.tar.gz
14
15pushd pnetcdf-${version}
16
17export CC=${CC:-mpicc}
18
19./configure \
20      --prefix=${prefix} \
21      --enable-shared \
22      --disable-static \
23      --disable-cxx \
24      --disable-fortran
25
26make all
27make install
28
29popd
30popd

Here we disable PnetCDF’s C++ and Fortran APIs and build the shared library.

Previous Up Next