The intructions to build OpenFOAM (v1812) on ARM based processors using the ARM and CRAY-MPICH compilers are presented. Although OpenFOAM v1812 does come with the compilation rules for the ARM based systems few modifications as mentioned on the ARM developers page1,2 are still required to build OpenFOAM for better performance. Here the focus is to primarily compile OpenFOAM on the ARMv8-A (64 bit) cores. These instructions presented below are tested on the Cavium ThunderX2 ARM cores available on the Isambard (Phase2), which is a Tier 2 HPC facility for UK-based researchers3. The diff-based patch approach is used here to create the patch files that help you update the necessary files instead of editing them manually to avoid any mistakes.

  1. Download OpenFOAM-v1812 and ThirdParty-v1812
      mkdir $BUILD_DIR
      cd $BUILD_DIR
      tar -xvf OpenFOAM-v1812.tgz -C $BUILD_DIR 
      tar -xvf ThirdParty-v1812.tgz -C $BUILD_DIR

  2. Load ARM compilers
     # load ARM compiler if no compilers are loaded
     module load PrgEnv-allinea/6.0.5
     # or swap current cray compiler with arm compiler
     module switch PrgEnv-cray/6.0.5 PrgEnv-allinea/6.0.5

  3. Make modifications to the configuration files required to build OF
     # Download the two patch files and apply the patch running the below commands
     (cd $BUILD_DIR/OpenFOAM-v1812/; patch -p1 < $BUILD_DIR/OpenFOAM-v1812.patch)
     (cd $BUILD_DIR/ThirdParty-v1812/; patch -p1 < $BUILD_DIR/ThirdParty-v1812.patch)
     # create file with compiler options
     cd $BUILD_DIR/OpenFOAM-v1812
     # please note **cray-mpich** module is loaded by default in my case
     echo -e "WM_COMPILER=Arm \nWM_MPLIB=CRAY-MPICH \nWM_LABEL_SIZE=64" > $BUILD_DIR/OpenFOAM-v1812/etc/

  4. Compile OpenFOAM-v1812
     # source configuration file
     . etc/bashrc
     # This step is optional to check if everything is okay to build OpenFOAM
     # Compile openfoam using all available processors (-j) with reduced output (-s) and log the output (-l) to a file so that we can examine any compilation issues later.
     ./Allwmake -j -s -l