Here is a collection of miscellaneous information, advice, annotated references and web-page links, etc., on the general topic of numerical programming, a.k.a. "number crunching". This material should be relevant to a wide range of numerical programming projects, not just my own research field of numerical general relativity.
Note that many of the books listed here are listed in my BibTeX bibliography. For these books, I give the BibTeX key in the bibliography; you can search for this to get the full author names, publisher, ISBN, etc.
Table of Contents:
/usr/local/share2 Numerical Software for thp.univie.ac.at Machines
Link to my web page on C programming.
Link to my web page on C++ programming.
Link to
my web page on CVS (Concurrent Versions System).
Link back to my home page.
There's also various information on numerical algorithms and scientific programming in my web page for the Dynamics of Gravitational Collapse Privatissimum
Bruce Bush's
The Perils of Floating Point
This is an excellent article on some of the basic considerations which
one must keep in mind in numerical programming, caused by the fact that
floating point numbers/arithmetic are only approximations to real
numbers/arithmetic. It's written in terms of Fortran, but the author's
points are equally valid for other languages.
How Computers Handle Numbers - a.k.a. Computer Arithmetic Uncovered
This is an excellent 4-lecture short course on floating-point computation
for computer scientists. This page also links to course notes for
two numerical analysis courses (one basic, one more advanced).
Numerical Analysis - Numerical Methods Project
This is a nice set of web pages for two undergraduate numerical analysis
courses. There are lots of worked examples, some downloadable code,
and nice lists of annotated links to other web sites.
Forsythe, Malcolm, and Moler,
Computer Methods for Mathematical Computations
(BibTeX key
Forsythe-Malcolm-Moler)
(Software available online at Netlib
in double and
single precision)
Kahaner, Moler, and Nash,
Numerical Methods and Software
(BibTeX key Kahaner-Moler-Nash)
(Software available online in
double and
single
precision)
These are two excellent introductory books on numerical methods for
scientific computing. The authors of these books are all experts in
numerical analysis, and it shows: their explanations and advice are
authoritative, and their codes are of high quality. I particularly
like these books' "grey-box" philosophy of explaining the concepts
behind the numerical methods and subroutines, but not trying
to explain all the gory details that go into the high-quality codes
which accompany the books. These books alas don't cover as wide a
range of topics as
Numerical Recipes, but
in my opinion, these are the better books: their advice can always be
trusted, their software is superb and very easy to use, and their
software is freely available online and freely redistributable, too!
Numerical Recipes
(books, software,
books online,
and web site)
(BibTeX key Numerical-Recipes)
I have mixed feelings about Numerical Recipes. On one
hand, the books cover a wide range of areas in numerical methods for
scientific computing, in an easy-to-read and down-to-earth manner, and
they come with a substantial library of subroutines. And you can
view the books online
(either postscript or pdf format) at the
Numerical Recipes web site,
too. On the other hand, the Numerical Recipes authors aren't
really experts in numerical analysis, and alas it shows in the uneven
quality of the books' explanations, advice, and the accompanying codes.
Here are some (mostly critical)
reviews of Numerical Recipes and the accompanying software,
by people who genuinely are numerical analysis experts.
Overall, I'd say Numerical Recipes is a useful starting
point for learning about numerical methods, but it shouldn't be your
last book on the subject or on any given topic.
I also have
some objections to the
very restrictive
licensing terms
for the Numerical Recipes software.
Stoer and Bulirsch,
Introduction to Numerical Analysis
(BibTeX key Stoer-Bulirsch)
Although this is nominally a beginning undergraduate text on numerical
analysis, it's written at a fairly advanced level, and covers a lot of
material. It's an excellent reference book.
Ralston and Rabinowitz,
A First Course in Numerical Analysis
(BibTeX key Ralston-Rabinowitz)
Another classic advanced "introductory" numerical analysis textbook.
Despite being rather old, this is still an excellent reference book.
Jack Dongarra maintains an excellent list of Freely Available Software for Linear Algebra, covering dense-matrix, sparse-matrix direct, and sparse-matrix iterative solvers for linear equations and eigenproblems.
Barton and Nackman,
Scientific and Engineering C++
This book is a C++ textbook oriented towards scientific programming. It
starts with the basic features of C++ for either Fortran or C programmers,
then builds to some of the more advanced features. The later parts present
real-world examples of using the more advanced features to simplify
complicated scientific programming problems. I've only briefly skimmed
this book so far, but I like what I've seen.
The Netlib Software Archive, available from repositories in
USA (1),
USA (2),
UK,
Germany,
Norway,
and
Australia
Netlib is a huge archive of numerical software, almost all of
it free (public domain, GNU public licensed,
or otherwise legally available and redistributable without paying money),
and almost all of it of very high quality. Most of it is
written in Fortran, even Fortran 66
(Hollerith constants, no less!), but it's still fantastic software.
Numerical Analysis FAQ
This is an excellent and well-indexed collection of resources (books,
web pages, software archives) on numerical analysis and associated fields.
Guide to Available Mathematical Software (GAMS)
This is an excellent tree-structured index of numerical software.
galileo
and all the other thp.univie.ac.at machines under
/usr/local/share2/.
LAPACK
(Fortran, i.e. Fortran 77)
LAPACK90
(Fortran 90)
LAPACK++
(C++)
LAPACK is a subroutine library for
numerical linear algebra. It largely supplants the earlier
LINPACK and
EISPACK libraries, and offers
more features, (sometimes) improved accuracy, and (often) much better
performance. Unfortunately, LAPACK is very large,
so many people still use the earlier libraries ;=).
All the LAPACK subroutines come in separate variants for
single precision real, double precision real, single precision complex,
and double precision complex arithmetic. LAPACK comes in
3 basic variants: Fortran LAPACK,
but this is a clone of the Fortran version, and still uses the Fortran
calling sequences.) LAPACK includes a lot of
online documentation, and if you know where to look in it, it's
sufficient for almost all day-to-day programming with the subroutines.
There's also a nice printed book, the
LAPACK User's Guide.
QUADPACK
QUADPACK is a Fortran subroutine
library for (1-dimensional) numerical integration. It includes both
automatic integrators (which adaptively choose their evaluation points
based on where the function being integrated seems to be wiggling the
most) and non-automatic integrators (which use a fixed set of evaluation
points independent of what function you're integration). [As a starting
point, I suggest the QAGS automatic integrator, or
QAGI if your range of integration is infinite. If you're
doing further calculations with the integral such that you need a
smooth dependence of the integral on the input parameters,
then I suggest the QK61 non-automatic integrator.]
ODEPACK
(Double
or Single
Precision)
ODEPACK is a Fortran subroutine
library for the numerical solution of ODEs (both non-stiff and stiff).
That is, given a system of ODEs and the initial value of the state
vector, the ODEPACK routines will time-integrate the ODEs
to find the state vector at future times. ODEPACK includes
6 basic integrators, each of which offers a large selection of options.
Fortunately, all the integrators also provide easy-to-use default
options, and even separate "simple use" and "all the details"
documentation. [As a starting point, I suggest the LSODA
integrator. If you need to find zeros of user-defined functions
concurrently with the integration (eg to locate the integration endpoint),
then try LSODAR.] There are actually two separate
ODEPACK libraries, one for single and one for double
precision; alas they use the same subroutine names, so they can't be
combined into a single library :=(.
LINPACK
LINPACK is a Fortran subroutine
library for solving linear equations. It has been largely supplanted
by LAPACK, but is still widely used.
All the LINPACKsubroutines come in separate variants for
single precision real, double precision real, single precision complex,
and double precision complex arithmetic. Once you're familiar with
the general organization of the library, the header comments in the
source code are sufficient documentation for most day-to-day programming
with the subroutines. Further documentation, including the overall
structure of the library, and also a nice summary of conditioning in
numerical linear algebra, is in a separate (very nice) book
(search for the key LINPACK-book in
my BibTeX bibliography).
EISPACK
(Double
or Single
Precision)
EISPACK is a Fortran subroutine
library for finding eigenvalues and/or eigenvectors of matrices.
It's a bit awkward to use, and has been largely supplanted by
LAPACK, but is still quite widely
used. There are actually two separate EISPACK libraries,
one for single and one for double precision; alas they use the same
subroutine names, so they can't be combined into a single library :=(.
EISPACK includes (different) appropriate collections of
subroutines for both real and complex matrices. Solving an eigenproblem
generally requires calling several EISPACK subroutines,
but unfortunately the subroutine sequences involved aren't documented
anywhere online, but only in the EISPACK
(search for the key EISPACK-book in
my BibTeX bibliography). Therefore, for most
practical use of EISPACK, you need to have a copy of the
book at hand. (There's also a second EISPACK book, but
it mostly documents subroutines which overlap those provided by
LINPACK.)
My Web Page on Calling Fortran Subroutines from C
A lot of excellent numerical software is written in Fortran, so it's
often useful to use this software (i.e. to call Fortran subroutines
and/or functions) from C and/or C++ programs. This is a web page I
wrote on how to do this (C only, alas not yet C++), as well as how
to write C functions which can be called from Fortran code.
[
Link back to
text citation of this note.]
My objections to the very restrictive licensing terms for the
Numerical Recipes
software:
That is, if you use a Numerical Recipes routine in a larger code of your own, you're not allowed to distribute their routine in source form as part of a source distribution of your own larger code. Even worse, this restriction still holds even if you have to modify the Numerical Recipes routine to be suitable for your use. This makes the Numerical Recipes routines much less useful than they might otherwise be, and makes their suggestion that you experiment with and modify the routines somewhat hollow.
The Numerical Recipes licensing terms are all well within the authors legal rights, but I also find them completely antiethical to the usual academic conventions of freely sharing information and research software. This is particularly so since many of the Numerical Recipes routines are closely modeled after existing public domain routines written (and freely contributed) by other researchers.
[
Link back to text citation of this note.]
The C version of the LAPACK library
LAPACK,
CLAPACK.
This was obtained by running the Fortran version through the
f2c Fortran-to-C translator. CLAPACK's
goal is to provide (a clone of the Fortran version of)
LAPACK for systems without a Fortran compiler.
As far as use is concerned, CLAPACK is exactly
the same as the original Fortran LAPACK. In
particular, CLAPACK still uses the Fortran
calling sequences.
Link back to Jonathan Thornburg's home page.
$Revision: 1.5 $ of $Date: 2006/12/24 17:40:35 $.