Cloudy & Associates

Commit cd87346a authored by Chatzikos, Marios's avatar Chatzikos, Marios
Browse files

Convert README files in tsuite/ to Markdown

parent 1034cc0c
## Notes
These is the test harness the permits nightly automated testing of Cloudy.
Please see our wiki at http://trac.nublado.org/wiki/TestSuite for more details.
Please see our wiki at
[our Wiki page](https://gitlab.nublado.org/cloudy/cloudy/-/wikis/TestSuite)
for more details.
## Notes
```
# The test suites use the following TABLE SED commands to import stellar SEDs
# without requiring the user to install the corresponding grid:
#
......@@ -28,3 +31,4 @@ init "honly.ini"
ionization parameter -2
stop zone 1
constant temperature 4
```
# Cloudy tests readme file
Read me file for the standard tests directory
=============================================
**For more information visit the Cloudy web site,
[www.nublado.org](http://www.nublado.org)**
* * *
The standard tests
------------------
The standard test cases that are computed every night in Lexington are the set
of files with names ending in ".in".
Each contains the commands needed to compute a particular model.
When they are computed they produce output files with the same name but ending
in ".out".
Additional files are created too - these mostly results of all the monitor
commands, ending in ".asr", and overviews of the model results ".ovr".
The file [doc\_tsuite.htm](doc_tsuite.htm) gives a list of all test cases, the
commands contained in each, and a description of its purpose.
This file is automatically created by _doc\_tsuite.pl_
The purpose of each test case is given in the documentation that follows the
input commands.
Cloudy stops reading input commands if the end of file or a blank line is
encountered.
Each ```\*.in``` file begins with the commands, followed by the blank line to
tell the code to stop reading, then followed by a description of the purpose
of the test.
This description is totally ignored - the command parser stops when it
encounters the first empty line.
The first part of the name of many test scripts indicate its purpose.
For instance, all PDR models start with "pdr\_" and all quasar BLR models
start with "blr\_". 
The input scripts follow a particular order of commands although this is not
necessary.
The file template\_scripts.txt describes the various groups of commands and
gives the overall format for an input script.
New scripts should follow this format.
* * *
Asserts - reliability in the face of complexity
-----------------------------------------------
Cloudy is designed to be autonomous and self-aware.
The code uses extensive self-checking to insure that the results are valid.
This  philosophy is described in Ferland 2001, ASP Conference Series, Vol 247,
_Spectroscopic Challenges of Photoionized Plasmas_, G Ferland & D Savin,
editors ([astro-ph/0210161](http://xxx.lanl.gov/ftp/astro-ph/papers/0210/0210161.pdf)). 
Asserts provide the ability to automatically validate complex results.
There are two types of asserts here - the first are a set of commands that are
included in the input files and tell the code what answer to expect, and the
second are C++ language macros that are part of the source and confirm the
internal decisions made by the code.
To avoid confusion, the first type of asserts have been renamed to monitors
since C10.
All of the files in the test suite include **monitor** commands.
The **monitor** command is described in the **Miscellaneous Commands** section
of **Hazy I**, and provides the infrastructure needed for complete automatic
testing of the code.
This command tells the code what answer to expect.
If the computed results do not agree then the code prints a standard comment
and exits with an error condition.
These **monitor** commands have nothing to do with the simulation itself, and
would not be included in an actual calculation.
You should ignore them, or even delete all of them (a Perl script,
tests\_remove\_asserts.pl, is provided as part of the test suite to do this).
The source code also includes many C++ **assert** macros that are designed to
validate the internal decisions made by the code.
The **assert** macro only exists if the NDEBUG macro is not set when the code
is compiled.
(If the NDEBUG macro is set then **assert** macros within the source are
ignored by the compiler.)
The test cases should be run at least once with the **assert** macro active,
that is, do not include a compiler option to define the NDEBUG macro.
In most compilers the NDEBUG macro is set when compiler optimization is set to
a high level.
In practice this means that the entire code should be compiled with only low
optimization and the tests computed to validate the platform.
Then recompile with higher optimization for production runs, and recompute the
test cases.
* * *
The "run" command
-----------------
When executed as a stand-alone program the code expects to read commands from
standard input and write results to standard output.
I compute single models by defining a shell script called "**run**`".`
It is described in more detail in the
[Cloudy wiki](https://gitlab.nublado.org/cloudy/cloudy/-/wikis/RunningC22).
* * *
The Perl scripts
----------------
The test suite includes a series of Perl scripts that compute test cases and
then check for errors.
These provide an automatic way to validate the code.  
### Notes on Perl
Perl comments start with a sharp sign "#" and end with the end of line.
Perl variable names begin with a dollar sign "$".
A Perl script is executed by typing the name of the script, as in
**`run_parallel.pl`** or **`./run_parallel.pl`**
### run\_parallel.pl
This script was written by Peter van Hoof and will run the test suite using a
number of processors.
The beginning of the script says how to use it.
### rerun\_parallel.pl
This script is very similar to run\_parallel.pl, with two important differences.
First, it does not clean up the test suite directory before running, and second,
by default it only runs those sims that failed in an earlier run.
This script is mainly intended for developers.
### checkall.pl
This script searches for problems in all of the test cases (the \*.out files)
in the current directory.
It first looks for botched monitors and warnings.
These indicate very serious problems.
A list of any models with these problems is placed in the file **serious.txt**.
Next it whether the string PROBLEM was printed in any of the models, and writes
a list of these to the file **minor.txt**.
A few of these can occur in a normal series of models and they do not,
by themselves, indicate a serious problem.
Finally it looks for all models that did not end.. 
### doc\_tsuite.pl
This script creates two files that document the test suite.
The file **doc\_tsuite.html** is an html representation of the set of tests,
while **doc\_tsuite.txt** is a comma delimited table that can be incorporated
into a word processor.
* * *
If you find a problem
---------------------
Cloudy should run on all platforms without errors.
Botched monitors or outright crashes should never happen.
I cannot fix it if I do not know it is broken.
Please let me know of any problems.
My email address is [gary@pa.uky.edu](mailto:gary@cloud9.pa.uky.edu)
* * *
Visit [https://nublado.org](https://nublado.org) for details and latest updates.
Good luck,
Gary J. Ferland
Last edited: Nov 3, 2022
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta name="GENERATOR" content="Microsoft FrontPage 6.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<title>Cloudy tests readme file</title>
</head>
<body background="clouds.jpg">
<h1>Read me file for the standard tests directory</h1>
<p><b>For more information visit the Cloudy web site,
<a href="http://www.nublado.org">www.nublado.org</a> </b></p>
<hr>
<h2><a name="The standard tests">The standard tests</a></h2>
<p>The standard test cases that are computed every night in Lexington are the
set of&nbsp; files with names ending in &quot;.in&quot;. Each contains the
commands needed to compute a particular model.&nbsp; When they are computed they produce output files with the same name
but ending in &quot;.out&quot;. Additional files are created too -&nbsp; these mostly results of all the monitor
commands, ending in &quot;.asr&quot;, and overviews of the
model results &quot;.ovr&quot;.</p>
<p>The file <a href="doc_tsuite.htm">doc_tsuite.htm</a> gives a list of all
test cases, the commands contained in each, and a description of its purpose.&nbsp;
This file is automatically created by <i>doc_tsuite.pl</i></p>
<p>The purpose of each test case is given in the documentation that follows the
input commands. Cloudy stops reading input commands if the end of file or a
blank line is encountered. Each *.in file begins with the commands, followed by
the blank line to tell the code to stop reading, then followed by a description
of the purpose of the test.&nbsp; This description is totally ignored - the
command parser stops when it encounters the first empty line.</p>
<p>The first part of the name of many test scripts indicate its purpose.&nbsp;
For instance, all PDR models start with &quot;pdr_&quot; and all quasar BLR models start
with &quot;blr_&quot;.&nbsp;</p>
<p>The input scripts follow a particular order of commands although this is not
necessary.&nbsp; The file template_scripts.txt describes the various groups of
commands and gives the overall format for an input script.&nbsp; New scripts
should follow this format.</p>
<hr>
<h2><a name="Asserts - reliability in the face of complexity">Asserts -
reliability in the face of complexity</a> </h2>
<p>Cloudy is designed to be autonomous and self-aware.&nbsp; The code uses
extensive self-checking to insure that the results are valid.&nbsp; This&nbsp;
philosophy is described in Ferland 2001, ASP Conference Series, Vol
247, <i>Spectroscopic Challenges of Photoionized Plasmas</i>, G Ferland &amp; D
Savin, editors (<a href="http://xxx.lanl.gov/ftp/astro-ph/papers/0210/0210161.pdf">astro-ph/0210161</a>).&nbsp; </p>
<p>Asserts provide the ability to automatically validate complex results.&nbsp;
There are two types of asserts here - the first are a set of commands that are
included in the input files and tell the code what
answer to expect, and the second are C++ language macros that are part of the
source and confirm the internal decisions made
by the code. To avoid confusion, the first type of asserts have been renamed to monitors since C10.</p>
<p>All of the files in the test suite include <b>monitor</b> commands.&nbsp; The <font size="3"><b><kbd> monitor</kbd></b></font> command is described in the <b>Miscellaneous Commands</b> section of
<b>Hazy I</b>, and
provides the infrastructure needed for
complete automatic testing of the code.&nbsp; This command tells the code what answer to
expect.&nbsp; If the computed results do not agree then the code prints a
standard comment and exits with an
error condition.&nbsp; These <b>monitor</b> commands have nothing to do with the
simulation itself, and would not be included in an actual calculation.&nbsp; You
should ignore them, or even delete all of them (a Perl script,
tests_remove_asserts.pl, is provided is provided as part of the test suite to do
this).</p>
<p>The source code also includes many C++ <font size="3"><b><kbd>assert</kbd></b></font> macros that are designed to validate the code's
internal decisions.&nbsp; The <font size="3"><b><kbd>assert</kbd></b></font> macro only exists if the NDEBUG macro is
not set when the code is compiled.&nbsp; (If the NDEBUG macro is set then
<font size="3"><b><kbd>assert</kbd></b></font>
macros within the source are ignored by the compiler.)&nbsp; The test cases
should be run at least once with the <font size="3"><b><kbd>assert</kbd></b></font> macro active, that is, do not
include a compiler option to define the NDEBUG macro.&nbsp; In most compilers
the NDEBUG macro is set when compiler optimization is set to a high level.&nbsp;
In practice this means that the entire code should be compiled with only low
optimization and the tests computed to validate the platform.&nbsp; Then
recompile with higher optimization for production runs, and recompute the test
cases.</p>
<hr>
<h2><a name="The &quot;run&quot; command">The &quot;run&quot; command</a></h2>
<p>When executed as a stand-alone program the code expects to read commands from
standard input and write results to standard output.&nbsp; I compute single models by defining a shell script called &quot;<font size="3"><b><kbd>run</kbd></b></font><code>&quot;.</code>&nbsp;
It is described in more detail in the <a href="http://wiki.nublado.org/wiki/RunningC13">Cloudy wiki</a></p>
<hr>
<h2><a name="The Perl scripts">The Perl scripts</a></h2>
<p>The test suite includes a series of Perl scripts that compute test cases and then check for errors.&nbsp;
These provide an automatic way to validate the code.&nbsp;&nbsp;</p>
<h3>Notes on Perl</h3>
<p>Perl comments start with a sharp sign &quot;#&quot; and end with the end of
line.&nbsp;&nbsp;<br>
Perl variable names begin with a dollar sign &quot;$&quot;.<br>
A Perl script is executed by typing the name of the script, as in <b><code>run_parallel.pl</code></b>
or <b><code>./run_parallel.pl</code></b></p>
<h3>run_parallel.pl</h3>
<p>This script was written by Peter van Hoof and will run the test suite using a
number of processors.&nbsp; The beginning of the script says how to use it.</p>
<h3>rerun_parallel.pl</h3>
<p>This script is very similar to run_parallel.pl, with two important differences.
First, it doesn't clean up the test suite directory before running, and second,
by default it only runs those sims that failed in an earlier run. This script
is mainly intended for developers.</p>
<h3>checkall.pl</h3>
<p>This script searches for problems in all of the test cases (the *.out files)
in the current directory. &nbsp;
It first looks for botched monitors and warnings.&nbsp; These indicate very
serious problems.&nbsp; A list of any models with these problems is placed in
the file <font size="3"><b><kbd>serious.txt</kbd></b></font>.&nbsp; Next it
whether the string PROBLEM was printed in any of the models, and writes a list
of these to the file <font size="3"><b><kbd>minor.txt</kbd></b></font>.&nbsp; A
few of these can occur in a normal series of models and they do not, by
themselves, indicate a serious problem.&nbsp; Finally it looks for all models
that did not end..&nbsp; </p>
<h3>doc_tsuite.pl</h3>
<p>This script creates two files that document the test suite.&nbsp; The file
<b><font size="3"><kbd>doc_tsuite.html</kbd></font></b> is an html representation of the set of tests, while <b>
<font size="3"><kbd>doc_tsuite.txt</kbd></font></b> is a comma delimited table
that can be incorporated into a word processor.</p>
<hr>
<h2><a name="If you find a problem">If you find a problem</a></h2>
<p> Cloudy should run on all platforms without errors.&nbsp; Botched monitors or
outright crashes should never happen. I can't fix it if I don't know it's
broken.&nbsp; Please let me know of any problems. My email address is <a href="mailto:gary@cloud9.pa.uky.edu"> gary@pa.uky.edu</a></p>
<hr>
<p>Visit <a href="http://www.nublado.org">http://www.nublado.org</a>
for details and latest updates.&nbsp; </p>
<p>Good luck,<br>
Gary J. Ferland</p>
<p><!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%B %d, %Y" startspan -->Last edited: May 6, 2013<!--webbot bot="Timestamp" endspan i-checksum="16933" --></p>
</body>
</html>
## Notes
These test various abort modes. The tests are:
* abort.in - throw a cloudy_abort exception
* assert.in - the assert macro
* avx_domain.in - invalid arguments passed to a vectorized math function
* bounds_array.in - test array bounds
* bounds_auto.in
* bounds_avx_ptr.in
* bounds_multi.in
* bounds_multi_iter.in
* bounds_static.in
* bounds_vector.in
* exception.in - throws a C++ exception
* fpe_isnan_float.in - floating point error
* fpe_isnan.in
* fpe_longoverflow.in
* fpe_NaN.in
* fpe_overflow.in
* fpe_setnan_float.in
* fpe_setnan.in
* fpe_zero.in
* grid_corners.in - corners of grid exceed temperature limits of code, test that
control passes smoothly back to main with all punch produced after abort is
declared
* insanity.in - tests TotalInsanity
* limit_lte_he1_nomole_ste_nocoll2.in - a fairly large (but quick) grid where
random models fail due to various errors. this tests if the grid is resilient
to a variety of errors; also check if SAVE GRID output remains complete
despite these errors
* monitor.in - test various monitor commands
* segfault.in - test segmentation fault
* undef.in - variable is undefined
Scripts:
* run_parallel.pl - perl script to run all tests
* clean_tsuite.pl - clean up after run
* run_sequence.pl - helper script
* checkall.pl - empty stub
These test various abort modes. The tests are:
abort.in - throw a cloudy_abort exception
assert.in - the assert macro
avx_domain.in - invalid arguments passed to a vectorized math function
bounds_array.in - test array bounds
bounds_auto.in
bounds_avx_ptr.in
bounds_multi.in
bounds_multi_iter.in
bounds_static.in
bounds_vector.in
exception.in - throws a C++ exception
fpe_isnan_float.in - floating point error
fpe_isnan.in
fpe_longoverflow.in
fpe_NaN.in
fpe_overflow.in
fpe_setnan_float.in
fpe_setnan.in
fpe_zero.in
grid_corners.in - corners of grid exceed temperature limits of code, test that
control passes smoothly back to main with all punch produced after abort is
declared
insanity.in - tests TotalInsanity
limit_lte_he1_nomole_ste_nocoll2.in - a fairly large (but quick) grid where
random models fail due to various errors. this tests if the grid is resilient
to a variety of errors; also check if SAVE GRID output remains complete
despite these errors
monitor.in - test various monitor commands
segfault.in - test segmentation fault
undef.in - variable is undefined
run_parallel.pl - perl script to run all tests
clean_tsuite.pl - clean up after run
run_sequence.pl - helper script
checkall.pl - empty stub
## Notes
These are plots of results from various sims in tsuite done
using Jeremy Sanders' Veusz probram, http://home.gna.org/veusz/
......
## Notes
Read me file for the sample programs
These are a series of programs that show how Cloudy can be
......@@ -20,44 +22,48 @@ object files in source directory, and runs the programs.
See the instructions at the top of that script to find out more.
The shell script
```
complink.sh directory [gcc/clang]
```
will use gcc to build the cpp program within directory by default.
If a compiler is provided uses the given compiler and assumes that cloudy was build in the sys subdirectory.
If a compiler is provided uses the given compiler and assumes that
cloudy was build in the sys subdirectory.
template.cpp - This is a template you can use to write your own
* template.cpp - This is a template you can use to write your own
program that calls Cloudy as a subroutine. This provides a
template C++ main program that includes the needed exception handlers.
collion - This is an example of a gas in collisional ionization
* collion - This is an example of a gas in collisional ionization
equilibrium. There is no ionizing radiation. This is something
like the solar corona.
collcool - cooling in collisional ionization
* collcool - cooling in collisional ionization
comp4, This checks that the code is properly initialized when
* comp4 - This checks that the code is properly initialized when
it sets up. Two different models are computed twice and the
output goes to file1.txt (the first pair) and file2.txt (the
second). These files should be exactly the same.
hazy_coolingcurve This is an example of a gas in collisional
* hazy_coolingcurve - This is an example of a gas in collisional
ionization equilibrium in which the temperature is varied and
the heating and cooling derived.
hazy_kmt - This redoes the "S-curve" calculation originally
* hazy_kmt - This redoes the "S-curve" calculation originally
done by Krolick, McKee, & Tarter (1981; ApJ, 249, 422),
hizlte - a high metallicity gas in thermodynamic equilibrium.
* hizlte - a high metallicity gas in thermodynamic equilibrium.
mpi This is my main routine for running large grids on our HP
* mpi - This is my main routine for running large grids on our HP
cluster. It uses MPI to place a grid point on each processor.
vary_nete Both the electron density and temperature are varied
* vary_nete - Both the electron density and temperature are varied
over a broad range and several [O III] lines are predicted.
This shows how to make diagnostic diagrams where line ratios
are used to deduce temperature and density.
varyn This is a series of calculations in which the density
* varyn - This is a series of calculations in which the density
is varied and the intensities of some [OII] lines relative to
[OII] 3727 are printed.
Visit www.nublado.org for details and latest updates.
Visit [https://nublado.org](https://nublado.org) for details and
latest updates.
Read me for template.cpp
# Read me for template.cpp
A template for writing your own program to call Cloudy as a subroutine
OVERVIEW:
......@@ -24,9 +25,12 @@ or even use multiple nested loops.
COMPILATION:
------------
To compile the template into an executable, run the script
'./complink-template.sh'. It points to ../../source for the necessary
header files (*.h) and the static library (libcloudy.a). This is all
you need to do, if you compile Cloudy in source/ with g++.
```
./complink-template.sh
```
It points to ```../../source``` for the necessary
header files (\*.h) and the static library (libcloudy.a).
This is all you need to do, if you compile Cloudy in source/ with g++.
If you want to build the template with a compiler other than g++,
e.g., clang++, you must also build Cloudy with the same compiler.
......@@ -36,18 +40,19 @@ If you build Cloudy from one of the compiler-specific subdirectories
(path to library) option points to the appropriate directory, while
the SYS_CONFIG macro points to the cloudyconfig.h file therein, for
instance:
```
clang++ template.cpp \
-DSYS_CONFIG=\"/path/to/cloudy/source/sys_llvm/cloudyconfig.h\" \
-I ../../source \
-L ../../source/sys_llvm -lcloudy \
-o template.exe
```
This is VERY important.
The header file cloudyconfig.h is produced prior to compilation
of any source files, it holds information on the compiler's
capabilities, and fine-tunes the building process. Without the
of any source files, it holds information on the capabilities of
the compiler, and fine-tunes the building process. Without the
SYS_CONFIG macro, the compiler will pick up the, if any,
cloudyconfig.h file found in source/ (as directed by -I option),
which may have been produced for a different compiler. This
......
Read me for collion.cpp - a gas in collisional ionization equilibrium
# Read me for collion.cpp
a gas in collisional ionization equilibrium
This program varies the gas kinetic temperature in an environment where collisional
ionization will dominate. This is very similar to conditions in the solar corona.
......
## Notes
This checks that the code is properly initialized when it sets up.
Three different models are computed twice. The punch results
output goes to file1.txt (the first set) and file2.txt (the second).
......
Read me for Hazy cooling curve sample program
=============================================
This is an example of a gas in collisional ionization equilibrium in which
only the temperature is varied.
The gas temperature is varied over a very broad range and only the cosmic
background and cosmic rays are included.
The hydrogen density and gas kinetic temperature are specified and a single
zone is computed.
The temperature and cooling divided by _n_(H)<sup>2</sup> are printed.
This was used to create a figure in Hazy.
![](hazy_coolingcurve.jpg)
This figure shows the total cooling over the density squared.
The x-axis is the log of the gas temperature (K).
The y-axis is the cooling divided by the square of the density
(erg s<sup>-1</sup> cm<sup>3</sup>).
last modified 19 July, 2008
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<title>readme hazy cooling curve</title>
</head>
<body>
<h1>Read me for Hazy cooling curve sample program</h1>
<p>This is an example of a gas in collisional ionization equilibrium in which
only the temperature is varied.&nbsp; The gas temperature is varied over a very
broad range and only the cosmic background and cosmic rays are included.&nbsp; The hydrogen density and gas kinetic
temperature are specified and a single zone is computed.&nbsp; The temperature
and cooling divided by <i>n</i>(H)<sup>2</sup> are printed.&nbsp; This was used
to create a figure in Hazy. </p>
<p><img border="0" src="hazy_coolingcurve.jpg" width="911" height="685"></p>
<p>This figure shows the total cooling over the density squared..&nbsp; The x-axis is
the log of the gas temperature (K).&nbsp; The y-axis is the cooling divided by the
square of the density (erg s<sup>-1</sup> cm<sup>3</sup>).</p>
<p>last modified
<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%d %B, %Y" startspan -->19 July, 2008<!--webbot bot="Timestamp" endspan i-checksum="21191" --></p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
</body>
</html>
Read me for hazy\_kmt
=====================
hazy\_kmt.c
-----------
This redoes the "S-curve" calculation originally done by Krolick, McKee, &
Tarter (1981; ApJ, 249, 422).
The gas density is varied between 101 and 1012 cm\-3.
It is exposed to the cosmic background and an AGN continuum.
The output file _hazy\_kmt.txt_ gives the log of the temperature,
the temperature (K), and pressure.
![](hazy_kmt.jpg)
The x-axis is the log of the density, and the y-axis is the gas pressure.