simulated annealing is a global optimization method that distinguishes between different local optima. starting from an initial point, the algorithm takes a step and the function is evaluated. when minimizing a function, any downhill step is accepted and the process repeats from this new point. an uphill step may be accepted. thus, it can escape from local optima. this uphill decision is made by the metropolis criteria. as the optimization process proceeds, the length of the steps decline and the algorithm closes in on the global optimum. since the algorithm makes very few assumptions regarding the function to be optimized, it is quite robust with respect to non-quadratic surfaces. the degree of robustness can be adjusted by the user. in fact, simulated annealing can be used as a local optimizer for difficult functions.
| Type | Visibility | Attributes | Name | Initial | |||
|---|---|---|---|---|---|---|---|
| integer, | private, | parameter | :: | wp | = | real64 |
real kind used by this module [8 bytes] |
| integer, | public, | parameter | :: | simann_wp | = | wp |
for exporting from the module |
| real(kind=wp), | private, | parameter | :: | pi | = | acos(-1.0_wp) |
value of pi for this module's real kind |
| real(kind=wp), | private, | parameter | :: | twopi | = | 2.0_wp*pi | |
| integer, | public, | parameter | :: | n_distribution_modes | = | 5 |
number of modes |
| integer, | public, | parameter | :: | sa_mode_uniform | = | 1 | |
| integer, | public, | parameter | :: | sa_mode_normal | = | 2 | |
| integer, | public, | parameter | :: | sa_mode_cauchy | = | 3 | |
| integer, | public, | parameter | :: | sa_mode_triangular | = | 4 | |
| integer, | public, | parameter | :: | sa_mode_bipareto | = | 5 |
interface for distribution functions used for perturbations
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(in) | :: | lower |
lower bound |
||
| real(kind=wp), | intent(in) | :: | upper |
upper bound |
random value within [lower, upper]
interface to function to be maximized/minimized
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(in), | dimension(:) | :: | x | ||
| real(kind=wp), | intent(out) | :: | f | |||
| integer, | intent(out) | :: | istat |
status flag:
|
interface for reporting intermediate results to the user. This is called when iprint > 0. See iprint for when this is called.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(in), | dimension(:) | :: | x | ||
| real(kind=wp), | intent(in) | :: | f | |||
| integer, | intent(in) | :: | istat |
status flag:
|
interface for parallel function evaluations. This is used when the user wants to evaluate multiple points in parallel (e.g. on a GPU or with OpenMP) This function should return the number of input points to be received
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| integer, | intent(out) | :: | n_inputs |
number of inputs that can be received |
interface for parallel function evaluations. This function receives the input points for evaluation
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(in), | dimension(:,:) | :: | x |
input points (n x n_inputs), each column is one x vector |
interface for parallel function evaluations.
This function receives the output points from the
parallel evaluation. Returns both the x and f values
for a completed evaluation (typically from a queue).
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(out), | dimension(:) | :: | x | ||
| real(kind=wp), | intent(out) | :: | f | |||
| integer, | intent(out) | :: | istat |
status flag:
|
| Type | Visibility | Attributes | Name | Initial | |||
|---|---|---|---|---|---|---|---|
| integer, | private | :: | n | = | 0 |
number of variables in the function to be optimized. |
|
| logical, | private | :: | maximize | = | .false. |
denotes whether the function should be maximized or minimized. a true value denotes maximization while a false value denotes minimization. intermediate output (see iprint) takes this into account. |
|
| real(kind=wp), | private | :: | eps | = | 1.0e-9_wp |
error tolerance for termination. if the final function values from the last neps temperatures differ from the corresponding value at the current temperature by less than eps and the final function value at the current temperature differs from the current optimal function value by less than eps, execution terminates and ier = 0 is returned. |
|
| integer, | private | :: | ns | = | 20 |
number of cycles. after ns function evaluations, each element of
vm is adjusted according to the input |
|
| integer, | private | :: | nt | = | 100 |
number of iterations before temperature reduction. after ntns function evaluations, temperature (t) is changed by the factor rt. value suggested by corana et al. is max(100, 5n). see goffe et al. for further advice. |
|
| integer, | private | :: | neps | = | 4 |
number of final function values used to decide upon termination. see eps. suggested value is 4. |
|
| integer, | private | :: | maxevl | = | 10000 |
the maximum number of function evaluations. if it is exceeded, ier = 1. |
|
| logical, | private | :: | use_initial_guess | = | .true. |
if false, the initial guess is ignored and a random point in the bounds is used for the first function evaluation |
|
| integer, | private | :: | n_resets | = | 2 |
number of times to run the main loop (must be >=1) |
|
| real(kind=wp), | private, | dimension(:), allocatable | :: | lb |
the lower bound for the allowable solution variables. |
||
| real(kind=wp), | private, | dimension(:), allocatable | :: | ub |
the upper bound for the allowable solution variables. if the algorithm chooses x(i) < lb(i) or x(i) > ub(i), i = 1, n, a point is from inside is randomly selected. this this focuses the algorithm on the region inside ub and lb. unless the user wishes to concentrate the search to a particular region, ub and lb should be set to very large positive and negative values, respectively. note that the starting vector x should be inside this region. also note that lb and ub are fixed in position, while vm is centered on the last accepted trial set of variables that optimizes the function. |
||
| real(kind=wp), | private, | dimension(:), allocatable | :: | c |
vector that controls the step length adjustment. the suggested value for all elements is 2.0. |
||
| integer, | private | :: | iprint | = | 1 |
controls printing inside sa: |
|
| integer, | private | :: | iseed1 | = | 1234 |
the first seed for the random number generator. |
|
| integer, | private | :: | iseed2 | = | 5678 |
the second seed for the random number generator. different values for iseed1 and iseed2 will lead to an entirely different sequence of trial points and decisions on downhill moves (when maximizing). see goffe et al. on how this can be used to test the results of sa. |
|
| integer, | private | :: | step_mode | = | 1 |
how to vary |
|
| real(kind=wp), | private | :: | vms | = | 0.1_wp |
for |
|
| integer, | private | :: | iunit | = | output_unit |
unit number for prints. |
|
| integer, | private | :: | cooling_schedule | = | 1 |
temperature reduction schedule: |
|
| real(kind=wp), | private | :: | cooling_param | = | 1.0_wp |
parameter for cooling schedules 3 and 5. suggested value: 1.0 |
|
| logical, | private | :: | optimal_f_specified | = | .false. |
if the optional |
|
| real(kind=wp), | private | :: | optimal_f | = | 0.0_wp |
if |
|
| real(kind=wp), | private | :: | optimal_f_tol | = | 0.0_wp |
absolute tolerance for the |
|
| integer, | private, | dimension(:), allocatable | :: | distribution_mode |
distribution to use for perturbations for each variable: |
||
| real(kind=wp), | private, | dimension(:), allocatable | :: | dist_std_dev |
standard deviation for normal/truncated_normal |
||
| real(kind=wp), | private, | dimension(:), allocatable | :: | dist_scale |
scale parameter for cauchy/bipareto distributions |
||
| real(kind=wp), | private, | dimension(:), allocatable | :: | dist_shape |
shape parameter for bipareto distribution |
||
| procedure(sa_func), | private, | pointer | :: | fcn | => | null() |
the user's function |
| integer, | private | :: | ireport | = | 0 |
how often to report intermediate results to the user via |
|
| procedure(sa_report_func), | private, | pointer | :: | report | => | null() |
if associated, this function is called to report intermediate results to the user. |
| logical, | private | :: | parallel_mode | = | .false. |
if true, the user wants to evaluate multiple points in parallel (e.g. on a GPU or with OpenMP). if false, the user will evaluate one point at a time. if true, the user must provide all of n_inputs_to_send, fcn_parallel_input and fcn_parallel_output. |
|
| procedure(sa_func_parallel_inputs), | private, | pointer | :: | n_inputs_to_send | => | null() |
if the user wants to evaluate multiple points in parallel, this function should return the number of input points that will be sent for evaluation at a time. |
| procedure(sa_func_parallel_inputs_func), | private, | pointer | :: | fcn_parallel_input | => | null() |
this function receives the input points for evaluation |
| procedure(sa_func_parallel_output_func), | private, | pointer | :: | fcn_parallel_output | => | null() |
this function receives the output points from the parallel evaluation. The |
| real(kind=wp), | private | :: | vm_min | = | 1.0e-12_wp |
minimum allowable step size. if all vm values fall below this, terminate with ier=6 |
| procedure, public :: initialize => initialize_sa | |
| procedure, public :: optimize => sa | |
| procedure, public :: destroy => destroy_sa | |
| procedure, private :: func | |
| procedure, private :: perturb_and_evaluate | |
| procedure, public :: perturb_variable | this is public so we can use it in the tests |
if the function is to be minimized, switch the sign of the function. note that all intermediate and final output switches the sign back to eliminate any possible confusion for the user.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(in) | :: | me | |||
| real(kind=wp), | intent(in) | :: | f |
Perturb a single variable using its assigned distribution and parameters.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| integer, | intent(in) | :: | ivar |
variable index |
||
| real(kind=wp), | intent(in) | :: | x |
variable value to perturb |
||
| integer, | intent(in) | :: | mode |
perturbation distribution mode (see |
||
| real(kind=wp), | intent(in) | :: | lower |
lower bound |
||
| real(kind=wp), | intent(in) | :: | upper |
upper bound |
perturbed value
this function replaces exp() to avoid underflow and overflow.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | x |
Uniform random number on the interval [xl,xu].
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | xl |
lower bound |
||
| real(kind=wp), | intent(in) | :: | xu |
upper bound |
Normal (Gaussian) random number with specified mean and standard deviation. Uses the Box-Muller transform.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | mean |
mean of the distribution |
||
| real(kind=wp), | intent(in) | :: | std_dev |
standard deviation |
Cauchy random number with specified location and scale parameters. The Cauchy distribution has heavier tails than the normal distribution, which can be useful for occasional large jumps in simulated annealing.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | location |
location parameter (median) |
||
| real(kind=wp), | intent(in) | :: | scale |
scale parameter |
Truncated normal distribution within bounds [xl, xu]. Uses rejection sampling to ensure the value stays within bounds.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | mean |
mean of the underlying normal distribution |
||
| real(kind=wp), | intent(in) | :: | std_dev |
standard deviation |
||
| real(kind=wp), | intent(in) | :: | xl |
lower bound |
||
| real(kind=wp), | intent(in) | :: | xu |
upper bound |
Triangular distribution on [0,1] with specified mode. Uses inverse transform sampling - very efficient, no rejection needed.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | mode |
mode of the distribution (should be in [0,1]) |
Bi-polar (two-sided) Pareto distribution. Generates a symmetric heavy-tailed distribution centered at a location.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| real(kind=wp), | intent(in) | :: | center |
center location of the distribution |
||
| real(kind=wp), | intent(in) | :: | scale |
scale parameter for the Pareto distribution |
||
| real(kind=wp), | intent(in) | :: | shape |
shape parameter (smaller values = heavier tails) |
||
| real(kind=wp), | intent(in) | :: | xl |
lower bound |
||
| real(kind=wp), | intent(in) | :: | xu |
upper bound |
Destructor.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(out) | :: | me |
Initialize the class.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| integer, | intent(in) | :: | n | |||
| real(kind=wp), | intent(in), | dimension(n) | :: | lb | ||
| real(kind=wp), | intent(in), | dimension(n) | :: | ub | ||
| real(kind=wp), | intent(in), | optional, | dimension(n) | :: | c | |
| logical, | intent(in), | optional | :: | maximize | ||
| real(kind=wp), | intent(in), | optional | :: | eps | ||
| integer, | intent(in), | optional | :: | ns | ||
| integer, | intent(in), | optional | :: | nt | ||
| integer, | intent(in), | optional | :: | neps | ||
| integer, | intent(in), | optional | :: | maxevl | ||
| integer, | intent(in), | optional | :: | iprint | ||
| integer, | intent(in), | optional | :: | iseed1 | ||
| integer, | intent(in), | optional | :: | iseed2 | ||
| integer, | intent(in), | optional | :: | step_mode | ||
| real(kind=wp), | intent(in), | optional | :: | vms | ||
| integer, | intent(in), | optional | :: | iunit | ||
| logical, | intent(in), | optional | :: | use_initial_guess | ||
| integer, | intent(in), | optional | :: | n_resets | ||
| integer, | intent(in), | optional | :: | cooling_schedule |
temperature reduction schedule (1-5) |
|
| real(kind=wp), | intent(in), | optional | :: | cooling_param |
parameter for cooling schedules 3 and 5 |
|
| logical, | intent(in), | optional | :: | optimal_f_specified |
if the optional |
|
| real(kind=wp), | intent(in), | optional | :: | optimal_f |
if |
|
| real(kind=wp), | intent(in), | optional | :: | optimal_f_tol |
absolute tolerance for the |
|
| integer, | intent(in), | optional, | dimension(:) | :: | distribution_mode |
distribution for perturbations (per variable): |
| real(kind=wp), | intent(in), | optional, | dimension(:) | :: | dist_std_dev |
std dev for normal/truncated_normal (per variable or scalar) |
| real(kind=wp), | intent(in), | optional, | dimension(:) | :: | dist_scale |
scale for cauchy/pareto (per variable or scalar) |
| real(kind=wp), | intent(in), | optional, | dimension(:) | :: | dist_shape |
shape for pareto (per variable or scalar) |
| procedure(sa_func), | optional | :: | fcn |
the user's function for scalar evaluation. if not present, the user must provide all of n_inputs_to_send, fcn_parallel_input and fcn_parallel_output for parallel evaluation. |
||
| procedure(sa_func_parallel_inputs), | optional | :: | n_inputs_to_send |
if the user wants to evaluate multiple points in parallel, this function should return the number of input points that will be sent for evaluation at a time. |
||
| procedure(sa_func_parallel_inputs_func), | optional | :: | fcn_parallel_input |
this function receives the input points for evaluation |
||
| procedure(sa_func_parallel_output_func), | optional | :: | fcn_parallel_output |
this function receives the output points from the parallel
evaluation. The |
||
| integer, | intent(in), | optional | :: | ireport |
how often to report intermediate results to the user via |
|
| procedure(sa_report_func), | optional | :: | report |
if associated, this function is called to report intermediate results to the user. |
Continuous simulated annealing global optimization algorithm
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(inout), | dimension(me%n) | :: | x |
on input: the starting values for the variables of the function to be optimized. [Will be replaced by final point] |
|
| real(kind=wp), | intent(in) | :: | rt |
the temperature reduction factor. the value suggested by corana et al. is .85. see goffe et al. for more advice. |
||
| real(kind=wp), | intent(inout) | :: | t |
on input, the initial temperature. see goffe et al. for advice.
on output, the final temperature. Note that if |
||
| real(kind=wp), | intent(inout), | dimension(me%n) | :: | vm |
the step length vector. on input it should encompass the region of
interest given the starting value x. for point x(i), the next
trial point is selected is from x(i) - vm(i) to x(i) + vm(i).
since vm is adjusted so that about half of all points are accepted,
the input value is not very important (i.e. if the value is off,
sa adjusts vm to the correct value).
note: if |
|
| real(kind=wp), | intent(out), | dimension(me%n) | :: | xopt |
the variables that optimize the function. |
|
| real(kind=wp), | intent(out) | :: | fopt |
the optimal value of the function. |
||
| integer, | intent(out) | :: | nacc |
the number of accepted function evaluations. |
||
| integer, | intent(out) | :: | nfcnev |
the total number of function evaluations. in a minor point, note that the first evaluation is not used in the core of the algorithm; it simply initializes the algorithm. |
||
| integer, | intent(out) | :: | ier |
the error return number: |
Perturb the x vector and evaluate the function.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| class(simulated_annealing_type), | intent(inout) | :: | me | |||
| real(kind=wp), | intent(in), | dimension(:) | :: | x |
input optimization variable vector to perturb |
|
| real(kind=wp), | intent(in), | dimension(:) | :: | vm |
step length vector |
|
| real(kind=wp), | intent(out), | dimension(:) | :: | xp |
the perturbed |
|
| real(kind=wp), | intent(out) | :: | fp |
the value of the user function at |
||
| integer, | intent(inout) | :: | nfcnev |
total number of function evaluations |
||
| integer, | intent(inout) | :: | ier |
status output code |
||
| logical, | intent(in), | optional | :: | first |
to use the input |
Initialize the intrinsic random number generator.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | seed1 |
the first seed for the random number generator. |
||
| integer, | intent(in) | :: | seed2 |
the second seed for the random number generator. |
this subroutine prints the double precision vector named vector. elements 1 thru ncols will be printed. name is a character variable that describes vector. note that if name is given in the call to print_vector, it must be enclosed in quotes. if there are more than 10 elements in vector, 10 elements will be printed on each line.
| Type | Intent | Optional | Attributes | Name | ||
|---|---|---|---|---|---|---|
| integer, | intent(in) | :: | iunit | |||
| real(kind=wp), | intent(in), | dimension(ncols) | :: | vector | ||
| integer, | intent(in) | :: | ncols | |||
| character(len=*), | intent(in) | :: | name |