Creating & Using libraries in Scientific Code

For the discussion here we're going to restrict ourselves to Unix/Linux computing environments.

Suppose you have a program consisting of a collection source files. Typically scientific applications are constantly evolving, but parts of that code base, utilities, linear algebra solvers, IO routines, etc., may be static. This static code might be duplicated across numerous applications, and each application may have slightly differing versions. A simple way to control this static code is to create a library file out of it, and place it in a shared location.

Library Construction:
There are two types of libraries, static and dynamic, the latter also known as shareable. Both versions of these libraries are functionally equivalent. Lets look at each type.

Static Libraries:
A static library is just an archive of object files, usually indicated by a .a suffix. Using the Unix ar command a collection of object files may be combined to create a library. The benefit of a static library is that it gets attached to the executable of the target application. The executable is self-contained meaning the library's presence is not required when running the program. Under some circumstances, the executable may run faster.

Dynamic Libraries:
Dynamic libraries or shared libraries differ from static libraries in that the library is not part of the executable, but is linked at run time. Therefore dynamic libraries need to be present when the application is run. A dynamic library is typically indicated by the .so suffix. There main advantage of dynamic libraries are that since the executable does not include the library within it, the executable is significantly smaller. The Windows OS relies on dynamic libraries.

Lets look at static libraries and leave dynamic libraries for later. Creating a static library is pretty simple.

• Compile the source objects

$(FC) -c ftest1.f90 ftest2.f90 ftest3.f90

creating three object files ftest1.o ftest2.o ftest3.o, and if the source contains any Modules, a series of module symbol files with the suffix .mod. Keep track of these because we will need them for the linking stage.

• Create an archive from the object files using the Unix archive ar command.

ar -cvq libftest.a ftest1.o ftest2.o ftest3.o

which creates an archive named libftest.a

• Last step is to create and add a table of contents to te archive. This is done using the UNIX command ranlib. This last step differentiates between an archive of object files and an actual library.

ranlib libftest.a

See that was pretty easy.

Now you can see what's in the archive with the archive command

ar -t libftest.a

Running this produces the list:

ftest1.o
ftest2.o
ftest3.o

A second Unix command nm lists all the symbols contained in each of the object files contained in the archive. For our purposes, symbols are associations with routines, common data, and modules in the source code.

Library Use:
The last step here is actual library use. The exact procedure varies somewhat between languages and compilers. So we will focus on the case of Fortran 90/95 code.

Historically the compile and link and loads phase of creating an executable used separate commands. Modern Fortran compilers $(FC) combine all three steps in a single compiler call.

After the compiler compiles the source files, it uses the ld command to link the resulting .o files, any .o files that you specify as input files, and some of the .o and .a files in the product and system library directories. The compiler can then produce a single .o object file or a single executable output file from these object files.

If all the object files and libraries you need are local to the build, or are part of the library path environment variable, things should just build.

Typically this isn't the case and you need to specify the location and name of the libraries. Suppose our previous library called libftest.a is located in the /mydirectory/mylibrary/lib. This is done by adding two terms to the compile statement. The fisrt is the lowercase -l flag for the library name, and the second is the uppercase flag -L for the library path.

$(FC)-o myprogram myprogram.o -L/mydirectory/mylibrary/lib -lftest

This creates an executable called myprogram resulting from the source file myprogram.f90 linked to the library libftest.a located in the directory /mydirectory/mylibrary/lib.

Note two important things:

1. There are no spaces between the flags and the entries.
2. The library name excludes the prefix lib and the suffix .a

Now you are not necessarily finished. If the library libftest.a contains Fortran modules, and those modules are declared within the source code myprogram.f90, then you also need to specify the path to the .mod files created above during the compilation.

Module files contain an associated symbol file that holds information needed by program units, subprograms, and interface bodies that USE that module. By default, it is assumed that these symbol files exist in the current directory. For libraries, this is typically not the case, and a path to these .mod files must be specified.

Typically compilers provide a method to specify the module path. The -I flag is intended for specifying the path to any include and/or module files. Some compilers provide a second flag -M intended just for specifying the module files paths. The -I flag is the most common on across compilers. Some compilers, like Absoft, use -p instead, so check the compiler flag options for your compiler to be certain.

The following example creates an executable called myprogram.

$(FC)-o myprogram myprogram.o -I/mydirectory/mylibrary/mod -L/mydirectory/mylibrary/lib -lftest

The executable was created by linking the object file myprogram.o with the library libftest.a which lives in the directory /mydirectory/mylibrary/lib and the corresponding module files which live /mydirectory/mylibrary/mod.


For further info see:

YoLinux

Fortran Programming guide: Libraries