Build netCDF, IO/API and CMAQ


Build netCDF, IO/API and CMAQ

Following Ch. 3, 4, and 5 in the CMAQ manual , build and install the following packages in your UNIX account (e.g., ~/projects/cmaq/):
  • netcdf 4.1.2
  • ioapi 3.1
  • CMAQ 4.7.1

  1. Introduction
    Air quality simulation models for use at regional, state, and local scales within the United States. The models have been applied to estimate the ability of various control strategies to improve air quality and ensure cost-effective results.

    1. First-generation air quality models simulated air quality using simple chemistry at local scales, and Gaussian plume formulation was the basis for prediction.

    2. Second-generation models covered a broader range of scales (local, urban, regional) and pollutants, addressing each scale with a separate model that often focused on a single pollutant or issue (e.g., ozone, acid deposition).

    3. Third-generation models were needed that could treat multiple pollutants simultaneously and at scales up to continental or larger.

The EPACommunity Multiscale Air Quality (CMAQ) modeling system is a third-generation air quality model. It is available online at CMAQ is designed for applications ranging from regulatory and policy analysis to understanding the complex interactions of atmospheric chemistry and physics. It is a three-dimensional Eulerian (i.e., gridded) atmospheric chemistry and transport modeling system that simulates ozone, particulate matter (PM), toxic airborne pollutants, visibility, and acidic and nutrient pollutant species throughout the troposphere.

    • CMAQ uses coupled mathematical representations of actual chemical and physical processes to simulate air quality.

    • CMAQ belongs to the Eulerian class of mathematical models that calculate a mass balance within each grid cell by solving the transport across each cell boundary and chemical transformations within each cell during a given time period.

    • CMAQ thus requires two primary types of inputs: meteorological information, and emission rates from sources of emissions that affect air quality.

The five main CMAQ programs are  

    • The initial conditions processor (ICON)

The configuration options for ICON include selecting the chemical mechanism to model, defining the horizontal and vertical grids, and choosing whether the initial conditions are generated from an ASCII profile or from an existing CCTM output file.

    • The boundary conditions processor (BCON)

The configuration options for BCON include selecting the chemical mechanism to model, defining the horizontal and vertical grids, and choosing whether the boundary conditions are generated from an ASCII profile or from an existing CCTM output file.

    • The clear-sky photolysis rate calculator (JPROC)

The only configuration option required for JPROC is the selection of the chemical mechanism to use in the modeling. Output from JPROC is an ASCII look-up table of photolysis rates that CCTM uses to calculate gas-phase chemical transformations and pollutant concentrations.

    • The Meteorology-Chemistry Interface Processor (MCIP)

Configuration options for MCIP include the time periods over which to extract data from the meteorological model output files, horizontal and vertical grid definitions, and selections for calculating dry deposition velocities and integrating satellite cloud observations into MCIP output.

    • The CMAQ Chemistry-Transport Model (CCTM)  

CCTM integrates the output from the preprocessing programs described above (JPROC, BCON, ICON, and MCIP), as well as CMAQ-ready emissions inputs (e.g., output from SMOKE), to simulate continuous atmospheric chemical conditions.

Configuration options for CCTM include the temporal coverage of the simulation, the chemical mechanism to use in the modeling, the physics scheme to use for modeling pollutant transport, heterogeneous and aqueous chemistry options, and diagnostic options (such as process analysis, discussed in the next paragraph).

CCTM has the largest number of configuration options of all the CMAQ programs.

Ancillary support programs distributed with CMAQ include  

    • The code builder/manager (M3BLD)

It is the first program that needs to compiled after installing the CMAQ source code.

    • The chemical mechanism compiler (CHEMMECH)

Chemical mechanisms are represented in CMAQ through a series of INCLUDE files that contain mechanistic and kinetic parameters that describe a photochemical mechanism. CHEMMECH creates these INCLUDE files from an ASCII mechanism-definition file that represents the chemistry as sequential equations of reactants, products, and reaction rate information.

    • The process analysis preprocessor (PROCAN)

This preprocessor uses an input configuration file to select the process analysis options desired, and outputs three INCLUDE files that are used to compile a version of CCTM that is instrumented for process analysis.

  1. Overview of the Science in the CMAQ Modeling System

  1. CMAQ System Requirements and Installation

After benchmarking has been successfully completed, the CMAQ system can be configured for

other simulations.

Before installing any components of CMAQ, users must install CVS, netCDF, and I/O API.

The relation between the 'icc' command and 'icpc' command is the same as between 'gcc' and 'g++'. While both 'icc' and 'gcc' can compile C++ source, the don't know about the runtime libraries needed.

For gcc you can in fact use CXX=gcc when compiling the MySQL server sources, because work has been done to avoid C++ constructs that adds C++ runtime library dependencies. But this has not been done for icc, so you need to set CXX=icpc.

Setup Environmental Variables

# Setting compilers variables

export CC="/opt/intel/Compiler/11.1/046/bin/intel64/icc" #"/usr/bin/gcc"

export CXX="/opt/intel/Compiler/11.1/046/bin/intel64/icpc" #"/usr/bin/g++"

export FC=ifort

export F90=ifort

export F77=ifort

export PERL=/usr/bin/perl

export BIN=Linux2_x86_64ifort

# Setting code generation options

export CPPFLAGS="-O2 -DNDEBUG -DpgiFortran"

export CFLAGS="-g -O2"

export CXXFLAGS="-g -O2"

export FFLAGS='-g -O2 -mp -recursive'

# Setting CMAQ environmental variables

export M3HOME=~/CMAQ

export M3DATA=$M3HOME/data

export M3MODEL=$M3HOME/models

export M3LIB=$M3HOME/lib

export M3WORK=$M3HOME/work

# Setting ifort environmental variables

source /opt/intel/Compiler/11.1/046/bin/ intel64

source /opt/intel/Compiler/11.1/046/bin/ intel64

安装 netcdf-4.1.2.tar.gz

    1. 在netcdf的安装文件(tar.gz)的目录中,提示符中输入 tar -xvf netcdf-4.1.2
    2. 解压完毕后,输入

./configure --prefix=/home/michaelchen/CMAQ/NetCDF4.1.2 --disable-separate-fortran --disable-netcdf4 --disable-dap --enable-f77 --disable-compiler-recover --disable-cxx --disable-shared

    1. make
    2. make install
      You should see netCDF in /home/michaelchen/CMAQ/NetCDF4.1.2 as well as new files in /include, /lib, /share and /man.


libnetcdf.a 信息:

size: 2.44 MB (2,561,780 bytes)    size on disk: 2.44 MB (2,562,048 bytes)

安装  ioapi-3.1.tar.gz

    1. 在ioapi的安装文件(tar.gz)的目录中,提示符中输入 tar -xvf ioapi-3.1.tar.gz
    2. 进入ioapi-3.1目录,进行如下操作
    3. vim编辑Makefile (粗体是修改后的结果)

      BASEDIR    = ${HOME}/CUDA_exercise/ex2/ioapi-3.1

      INSTALL    = ${HOME}/CUDA_exercise/ex2/ioapi-3.1/ioapi

      LIBINST    = $(INSTALL)/$(BIN)

      BININST    = $(INSTALL)/$(BIN)

      CPLMODE    = nocpl

      IOAPIDEFS  =

      PVMINCL    = /dev/null #dev$(PVM_ROOT)/conf/$(PVM_ARCH).def

          1. 完成Makefile编辑后进行

      make configure

      make   在make的过程中会遇到这样的一些问题,每次遇到修改后重新make编译

          • /ioapi-3.1/ioapi/iobin3.c(1104) (col. 29): internal error: 0_1529
            COPTFLAGS = -O2 ${MFLAGS}
            FOPTFLAGS = -O2 -no-fp-port -unroll -stack_temps -safe_cray_ptr ${MFLAGS}
          • /ioapi-3.1/Linux2_x86_64ifort; ifort -auto -cm -w90 -w95 -warn notruncated_source -Bstatic  airs2m3.o -L/home/ckcheng/projects/gpu/epd/ioapi-3.1/Linux2_x86_64ifort -lioapi -lnetcdf  -lpvm3 -openmp -Vaxlib -Bstatic  -o airs2m3
            ld: cannot find -lnetcdf
          • 在编译m3tools中文件时,出现“cannot find -lpvm3”
            LIBS = -L${OBJDIR} -lioapi -lnetcdf  
            $(PVMLIBS) $(OMPLIBS) \
            $(ARCHLIB) $(ARCHLIBS)
          • 会有类似下面的现象出现多次。
            writeprofile.o: In function `main':
            writeprofile.f:(.text+0x0): multiple definition of `main'



      libioapi.a 信息:

      size: 5.74 MB (6,019,448 bytes)    size on disk: 5.74 MB (6,020,096 bytes)

      链接libnetcdf.alibioapi.a     (也可以不做这步,用拷贝替代)

      libnetcdf.a: /home/michaelchen/CMAQ/NetCDF4.1.2/lib/libnetcdf.a

      libioapi.a: /home/michaelchen/CUDA_exercise/ex2/ioapi-3.1/Linux2_x86_64ifort/libioapi.a



      ioapi  netCDF


      fixed_src  Linux2_x86_64ifort










      set CC = /opt/intel/Compiler/11.1/046/bin/intel64/icc

      set FC = /opt/intel/Compiler/1/1.1/046/bin/intel64/ifort

      set MPICH = /opt/openmpi-1.4.3_intel

      set INCL = /opt/openmpi-1.4.3_intel/include (stenex目录中)

      set FC = /opt/openmpi-1.4.3_intel/bin/mpif90  (pario目录中,cctm的bldit.cctm)


      Suppose you are running in multiprocessor mode, please make the following changes.

      1. Edit cctm/bldit.cctm and re-run bldit.cctm

      < # set ParOpt             # set for multiple PE's; comment out for single PE
      > set ParOpt             # set for multiple PE's; comment out for single PE

      2. Edit cctm/run.cctm and re-run run.cctm. You may need to set the path for MPIRUN

      < setenv NPCOL_NPROW "1 1"; set NPROCS   = 1 # single processor setting
      < # setenv NPCOL_NPROW "4 2"; set NPROCS  = 8
      > # setenv NPCOL_NPROW "1 1"; set NPROCS   = 1 # single processor setting
      > setenv NPCOL_NPROW "4 2"; set NPROCS  = 8

      < time $BASE/$EXEC
      < # time $MPIRUN -v -machinefile $TASKMAP -np $NPROCS $BASE/$EXEC
      > # time $BASE/$EXEC
      > time $MPIRUN -v -machinefile $TASKMAP -np $NPROCS $BASE/$EXEC

      3. After a while, you may encounter error:

      The curr_str buffer is not big enough!
      The curr_str buffer is not big enough!
      The curr_str buffer is not big enough!
      The curr_str buffer is not big enough!

      4. Go to the directory cctm/BLD_ela and change the mode of file cctm/BLD_ela/distr_env.c

      cd BLD_ela
      chmod 666 distr_env.c

      5. To remove this error, edit cctm/BLD_ela/distr_env.c

      < #define TEMP_BUF_SIZE   102400
      < #define CURR_STR_SIZE   1024
      > #define TEMP_BUF_SIZE   204800
      > #define CURR_STR_SIZE   2048

      6. Remove all .o files in the directory cctm/BLD_ela

      rm *.o

      7. Re-do steps 1 and 2.

      If the buffer is not big enough again, try doubling the value in the step 4. (If the problem still exists after doubling, try larger value. For example, 10 times value)




      aka "Bourne" shell, written by Steve Bourne at AT&T Bell Labs for Unix V7 (1979). Small, simple, and (originally) very few internal commands, so it called external programs for even the simplest of tasks. It is always available on everything that looks vaguely like Unix.


      The "C" shell. (Bill Joy, at Berkeley). Many things in common with the Bourne shell, but many enhancements to improve interactive use. The internal commands used only in scripts are very different from "sh", and similar (by design) to the "C" language syntax.


      The "TC" shell. Freely available and based on "csh". It has many additional features to make interactive use more convenient.


      The "Korn" shell, written by David Korn of AT&T Bell Labs (now AT&T Research). Written as a major upgrade to "sh" and backwards compatible with it, but has many internal commands for the most frequently used functions. It also incorporates many of the features from tcsh which enhance interactive use (command line history recall etc.).

      POSIX 1003.2 Shell Standard

      Standards committees worked over the Bourne shell and added many features of the Korn shell (ksh88) and C shell to define a standard set of features which all compliant shells must have.


      The "Bourne again" shell. Written as part of the GNU/Linux Open Source effort, and the default shell for Linux and Mac OS-X. It is a functional clone of sh, with additional features to enhance interactive use, add POSIX compliance, and partial ksh compatability.


      A freeware functional clone of sh, with parts of ksh, bash and full POSIX compliance, and many new interactive command-line editing features. It was installed as the default shell on early MacOSX systems.

      • setenv/unsetenv are commands that are built into the csh family of shells.
        Under bash the equivalent built ins are export and unset. For example:
        CSH:  setenv name value
        BASH: export name=value
        CSH:  unsetenv name
        BASH: unset name
      • openmpi 对 icc
        mpich 对 gcc
      • ls -l 显示更多属性