2. Installation
2.1. Third-Party Libraries
There are several libraries which we use on a routine basis for applications. The first two, Metis and MPI, are essential for medium-to-large production jobs, and the third (ParMetis) is necessary for pre-processing grids associated with very large jobs. The next three listed here, KSOPT, PORT, and NPSOL, are only necessary if you are going to be doing design optimization. SUGGAR and DiRTlib are only required if you need to use overset (chimera) meshes; typically overset grids are only used for certain types of moving-body problems. The TECPLOT tecio library, though not required, allows binary TECPLOT files to be written instead of larger ASCII files (ASCII files also load more slowly into TECPLOT). Finally, CGNS is necessary only if you are working with files written in the CGNS format.
| MPI |
| Metis v4.0.1 |
| Metis v5.0pre2 |
| ParMetis v3.1.1 |
| ParMetis v3.1.1 with OpenMPI |
| KSOPT |
| PORT |
| NPSOL |
| SUGGAR |
| DiRTlib |
| TECPLOT |
| CGNS |
MPI
Website: www-unix.mcs.anl.gov/mpi
Website: www.open-mpi.org
If you wish to execute the codes in the FUN3D suite across multiple
processors, you will need some implementation of the MPI libraries.
This may already be installed on your machine, or you may have to
acquire and install it yourself.
When using an MPI implementation, it will be necessary to directly
integrate your F95 compiler with the mpif90 script that ships with
the MPI implementation.
At the current time, we cannot support the pre-built Intel® MPI Library for Linux. The Intel® library is based on MPICH 2 from Argonne National Laboratory. We suggest downloading, building, and installing that version for support of the Intel® Fortran Compiler.
Metis v4.0.1
Two versions of Metis are available for use: Metis v4.0.1 and Metis v5.0pre2. Metis v4.0.1 is recommended for small to medium sized grids (16 million nodes or less); whereas Metis v5.0pre2 is recommended for grids larger than 16 million nodes. Metis v4.0.1 is internally limited to 32-bit addressing.
Website: www-users.cs.umn.edu/TILDAkarypis/metis
If you are planning on running in a multiprocessor environment, you will need this library, which partitions the connectivity information associated with an unstructured grid, so that the domain can be split into partitions and sent out to different processors.
Metis v5.0pre2
When using metis for large grids (i.e., internal numbers approaching the 32-bit limit, and/or the 2GB memory limit), then Metis v5 is recommended. This version supports portable I/O on both 32- and 64-bit architectures, and supports the larger grids. Download this version from the same site as metis v4.0.1.
ParMetis v3.1.1
Website: www-users.cs.umn.edu/TILDAkarypis/metis/parmetis
ParMetis is a parallel implementation of Metis (see above). The grid preprocessor known as party, which is included with the FUN3D suite, requires Metis to compile. This preprocessor is sufficient for most users. However, due to the sequential nature of party, preprocessing large grids (on the order of several million nodes) can be very cumbersome, if not impossible on a single machine with a relatively small amount of memory. For this reason, we now offer a distributed memory version of the preprocessor, known as pParty. If you wish to use this parallel version of party, you must download and install ParMetis. Moreover, on-the-fly adaptation capabilities that are being implemented for future releases of the suite will require the ParMetis library for dynamic load-balancing.
ParMetis v3.1.1 with OpenMPI
More recent versions of MPI internally manage the handles (e.g., communicators) differently from previous versions of MPI. Specifically, when interfacing between fortran and C, the more recent method uses MPI_Comm_f2c to convert the fortran handle into a C handle.
For example, if comm is a valid Fortran handle to a communicator, then MPI_Comm_f2c returns a valid C handle to that same communicator; if comm = MPI_COMM_NULL (Fortran value), then MPI_Comm_f2c returns a null C handle; if comm is an invalid Fortran handle, then MPI_Comm_f2c returns an invalid C handle.
ParMetis, a C library with fortran wrappers, is using the previous paradigm. Consequently, when FUN3D’s pParty is linked with ParMetis compiled with MPI using the newer paradigm, an error occurs. Currently within FUN3D’s pParty, only kmetis is called. The following workaround will bypass the error.
1. cd to the ParMetis source directory 2. /* save original kmetis.c */ cp ParMETISLib/kmetis.c ParMETISLib/kmetis.c_ORG 3. /* Replace the kmetis.c file with one in the FUN3D/utils source directory */ cp $FUN3D_dir/utils/ParMETISLib/kmetis.c ParMETISLib/kmetis.c /* where $FUN3D_dir is the path of your FUN3D directory */ 4. Compile ParMetis, and link in with pParty Alternatively, one can make the following changes directly to ParMETISLib/kmetis.c. /ParMETISLib/kmetis.c > >>>>>>>> 26c26 > >>>>>>>> < MPI_Comm *commF) > >>>>>>>> --- > > >>>>>>>>> MPI_Comm *comm) > >>>>>>>> 39,48d38 > >>>>>>>> < > >>>>>>>> < MPI_Comm comm_world; > >>>>>>>> < MPI_Comm *comm; > >>>>>>>> < > >>>>>>>> < comm_world = MPI_Comm_f2c((MPI_Fint)*commF); > >>>>>>>> < comm = &comm_world; > >>>>>>>> <
KSOPT
Contact: Greg Wrenn
Documentation: Wrenn, Gregory A., “An Indirect Method for Numerical
Optimization Using the Kreisselmeier-Steinhauser Function,” NASA CR 4220,
March 1989.
The KSOPT library is used for multiobjective and constrained
FUN3D-based design optimization. It is not required if you
do not plan on using the design optimization feature of FUN3D.
If you configure FUN3D to link to KSOPT, you must use the Fortran90
implementation of KSOPT, with its object files gathered to a
library called libksopt.a. Only FUN3D versions 10.7 and later
support the use of KSOPT.
PORT
Website: www.bell-labs.com/project/PORT
Website: netlib.sandia.gov/port/index.html
The PORT library is used for unconstrained FUN3D-based design optimization. It is not required if you do not plan on using the design optimization feature of FUN3D. Note that if you install both the PORT and NPSOL (below) libraries, you may have to comment out some of the low-level BLAS routines in one of the two packages; the libraries come with duplicate versions of some of the algebra routines which the linker may complain about.
NPSOL
Website: www.sbsi-sol-optimize.com (Must purchase)
The NPSOL library is used for constrained FUN3D-based design optimization. It is not required if you do not plan on using the design optimization feature of FUN3D. Note that if you install both the PORT (above) and NPSOL libraries, you may have to comment out some of the low-level BLAS routines in one of the two packages; the libraries come with duplicate versions of some of the algebra routines which the linker may complain about.
SUGGAR v2.41 or higher
Contact: Ralph Noack
SUGGAR is used to assemble composite meshes, cut holes, determine interpolation coefficients, etc. It is not reqiured unless you plan to use overset (chimera) meshes in FUN3D. SUGGAR may be compiled as a stand-alone executable and/or as a library. For static overset meshes you will need the stand-alone compilation; for moving body simulations using FUN3D Version 10.5 and above, you will need to compile both the stand-alone executable and the library. It is beyond the scope of this web page to provide details of the architecture-specific “make” flags needed to compile SUGGAR; see the documentation that comes with SUGGAR for more information.
In the suggar src directory
make make libs make libs_no_mpi
The first make above compiles the stand-alone suggar executable, while the 2nd and 3rd steps compile
an mpi-enabled library and a library without mpi linkage, respectively. You will need both libraries
to compile various parts of the FUN3D suite. Then wherever you keep the resulting .a files,
make the following soft links (this assumes you compile for 32-bit linux)
ln -s libsuggar_3d_no_mpi_opt_linux.a libsuggar.a ln -s libsuggar_3d_opt_linux.a libsuggar_mpi.a
If compiling 64-bit versions of the .a files under linux, the corresponding links would be:
ln -s libsuggar_3d_no_mpi_opt_linux-64.a libsuggar.a ln -s libsuggar_3d_opt_linux-64.a libsuggar_mpi.a
Assuming these .a files are located in a directory /path/to/libsuggar,
when configuring FUN3D, use
--with-suggar=/path/to/libsuggar
DiRTlib v1.25 or higher
Contact: Ralph Noack
The DiRTlib library must be linked to FUN3D in order to utilize the overset connectivity data computed by SUGGAR. It is not reqiured unless you plan to use overset (chimera) meshes in FUN3D. It is beyond the scope of this web page to provide details of the architecture-specific “make” flags needed to compile DiRTlib; see the documentation that comes with DiRTlib for more information.
In the dirtlib src directory
make serial make mpich+dc
Then wherever you keep the resulting .a files,
make the following soft link
ln -s libdirt+dc_mpich.a libdirt_mpich.a
Assuming these .a files are located in a directory /path/to/dirtlib, when configuring FUN3D, use
--with-dirtlib=/path/to/dirtlib
TECPLOT
Website: www.tecplot.com
By default, any TECPLOT output generated from the postprocessor code party, or from within
the flow solver itself, is written as an ASCII file. If you have a copy of TECPLOT, you were
provided with a library archive tecio.a (or tecio64.a for 64-bit versions). Provided you have
the tecio.a archive, you may configure the FUN3D suite to use that library via:
--with-tecio=/path/to/tecio
Then, TECPLOT solution data written out from the flow solver (FUN3D Version 10.7 and higher) and volumetric TECPLOT data written by party will be in binary form. This results in smaller file sizes and faster importation into TECPLOT. Note, however, that when you link against a particular tecio library, you will have to use the corresponding TECPLOT visualizer. For example, if you link against the tecio library from TECPLOT360-2008, you will not be able to use the resulting .plt files in TECPLOT360, only with TECPLOT360-2008.
Also note: the tecio library that was shipped with TECPLOT360-2008 had a bug that will result in error messages when the binary files are written. You must get the updated library from TECPLOT’s website (download.tecplot.com/tecio).
CGNS
Website: www.cgns.org
The CGNS library is used for working with files written in CGNS format. (CGNS is a convention for writing machine-independent, self-descriptive data files for CFD, and includes implementation software.) FUN3D currently has some limited capability for reading and writing CGNS files. You only need the CGNS library if you are planning to take advantage of this feature. Version 2.5 of the CGNS library or greater is required for adding CGNS capability to FUN3D.
2.2. Compiling
Version 10.4.x and Above
As of Version 10.4.x, the FUN3D suite of codes is configured and built via the GNU build system. Provided you have a Fortran 95 compiler available, a bare bones installation process would be,tar zxf fun3d-10.4.1.tgz cd fun3d-10.4.1 ./configure make
To add third-party library capabilities, you must specify where those libraries are located by passing command line options of the form
--with-LIB=/path/to/LIB to configure where LIB is the name of
a third party library.
./configure \
--with-metis=/usr/local/metis/lib \
--with-mpi=/usr/local/mpich-1.2.7_intel-9.1.045
make
This will provide executables for the serial pre-processor, party,
and the parallel flow solver, nodet_mpi. The executables are located
respectively in the Party/ and FUN3D_90/ subdirectories of the
configuration and are sufficient for most applications.
Use ./configure --help to see additional available options.
Details of Compiler Selection and Compiler Options
When the MPICH environment is selected, the build will use mpif90 located
in the bin directory of the given MPICH installation. Otherwise, the
system path is searched for one of the Fortran 95 compilers supported
by FUN3D. The search follows a predefined order and stops when the
first functional compiler is found. The user can explicitly specify
the desired Fortran 95 compiler via the FC environment variable.
Depending on the Fortran 95 compiler used, configure will select
compiler and linker options commonly used in production by the FUN3D team.
The process is referred to as Fortran 95 tuning. It is possible to
augment or completely override the compiler/linker options independent
of the Fortran 95 compiler selection. The FCFLAGS and LDFLAGS
environment variables are used for this purpose. The default behavior is
to augment by appending their values to the options defined by
Fortran 95 tuning. If the --disable-ftune option is given to
configure, developer options will be completely overridden by the
values given by FCFLAGS and LDFLAGS. When using MPICH,
Fortran 95 tuning is based on the Fortran 95 compiler that supports
mpif90 (see mpif90 -show).
-O3, -ip, and -Vaxlib options,
./configure \
FC=ifort \
FCFLAGS="-O3 -ip" \
LDFLAGS="-Vaxlib -lm" \
--disable-ftune
Note that quotes "" are used to protect values with spaces from the shell.
Also note that some developer options may be unconditionally required
for a given compiler as is the case of linking with the math library
(-lm above). The order of variables/options is inconsequential.
Version 10.3.1 and Below
To compile the FUN3D suite, you will need a Fortran95 compiler (and linker)
defined for F90COMPILER (and F90LINKER) in the file Makefile.env.
This file is generated by running ./Configure in the top directory,
and may need to be edited by hand if your local configuration differs.
Feel free to get in touch for suggested flags to use on many of the popular F95 compilers. Be sure to add the correct path to each of the appropriate libraries listed above as well. If you do not have a particular library installed, leave its variable blank.
To build the executables in the FUN3D suite, simply type make mpi or
make seq at the shell prompt in the top-level directory.
If your compilation worked correctly, you will end up with an executable
in each of the subdirectories of your source code distribution:
Party/party(sequential grid preprocessor)PartyMPI/pparty(parallel grid preprocessor)FUN3D_90/nodet_{seq,mpi}(flow solver)Adjoint/dual_{seq,mpi}(adjoint solver)GridMove/gridmove_{seq,mpi}(mesh movement/mesh adjoint)GetGrad/getgrad_{seq,mpi}(sensitivity evaluation)Design/opt_driver(design optimization driver)
Today's NASA Official:
Chris Rumsey, a member of
The FUN3D Development Team
Contact: FUN3D-support@lists.nasa.gov
NASA Privacy Statement