simulated_annealing_module Module

simulated annealing is a global optimization method that distinguishes between different local optima. starting from an initial point, the algorithm takes a step and the function is evaluated. when minimizing a function, any downhill step is accepted and the process repeats from this new point. an uphill step may be accepted. thus, it can escape from local optima. this uphill decision is made by the metropolis criteria. as the optimization process proceeds, the length of the steps decline and the algorithm closes in on the global optimum. since the algorithm makes very few assumptions regarding the function to be optimized, it is quite robust with respect to non-quadratic surfaces. the degree of robustness can be adjusted by the user. in fact, simulated annealing can be used as a local optimizer for difficult functions.

Reference

  • corana et al., "minimizing multimodal functions of continuous variables with the "simulated annealing" algorithm", september 1987 (vol. 13, no. 3, pp. 262-280), acm transactions on mathematical software.
  • goffe, ferrier and rogers, "global optimization of statistical functions with simulated annealing", journal of econometrics, vol. 60, no. 1/2, jan./feb. 1994, pp. 65-100.

History

  • based on reference by bill goffe : 1/22/94, version: 3.2 See: https://www.netlib.org/opt/simann.f
  • modifications by alan miller : fortran 90 version - 2 october 2013
  • Jacob Williams, 8/26/2019 : modernized Fortran

TODO

  • input rand distributation option
  • a way to specify that some variables are not to be changed... this could be part of the distributation selection (a constant option). each variable can have a different distribution.
  • get ideas from: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.basinhopping.html

Uses

  • module~~simulated_annealing_module~~UsesGraph module~simulated_annealing_module simulated_annealing_module ieee_exceptions ieee_exceptions module~simulated_annealing_module->ieee_exceptions iso_fortran_env iso_fortran_env module~simulated_annealing_module->iso_fortran_env

Used by

  • module~~simulated_annealing_module~~UsedByGraph module~simulated_annealing_module simulated_annealing_module module~simulated_annealing_module_c simulated_annealing_module_c module~simulated_annealing_module_c->module~simulated_annealing_module

Variables

Type Visibility Attributes Name Initial
integer, private, parameter :: wp = real64

real kind used by this module [8 bytes]

integer, public, parameter :: simann_wp = wp

for exporting from the module

real(kind=wp), private, parameter :: pi = acos(-1.0_wp)

value of pi for this module's real kind

real(kind=wp), private, parameter :: twopi = 2.0_wp*pi
integer, public, parameter :: n_distribution_modes = 5

number of modes

integer, public, parameter :: sa_mode_uniform = 1
integer, public, parameter :: sa_mode_normal = 2
integer, public, parameter :: sa_mode_cauchy = 3
integer, public, parameter :: sa_mode_triangular = 4
integer, public, parameter :: sa_mode_bipareto = 5

Abstract Interfaces

abstract interface

  • private function dist_func(me, lower, upper) result(r)

    interface for distribution functions used for perturbations

    Arguments

    Type IntentOptional Attributes Name
    class(simulated_annealing_type), intent(inout) :: me
    real(kind=wp), intent(in) :: lower

    lower bound

    real(kind=wp), intent(in) :: upper

    upper bound

    Return Value real(kind=wp)

    random value within [lower, upper]

abstract interface

  • public subroutine sa_func(me, x, f, istat)

    interface to function to be maximized/minimized

    Arguments

    Type IntentOptional Attributes Name
    class(simulated_annealing_type), intent(inout) :: me
    real(kind=wp), intent(in), dimension(:) :: x
    real(kind=wp), intent(out) :: f
    integer, intent(out) :: istat

    status flag:

    • 0 : valid function evaluation
    • -1 : invalid function evaluation. try a different input vector.
    • -2 : stop the optimization process

abstract interface

  • public subroutine sa_report_func(me, x, f, istat)

    interface for reporting intermediate results to the user. This is called when iprint > 0. See iprint for when this is called.

    Arguments

    Type IntentOptional Attributes Name
    class(simulated_annealing_type), intent(inout) :: me
    real(kind=wp), intent(in), dimension(:) :: x
    real(kind=wp), intent(in) :: f
    integer, intent(in) :: istat

    status flag:

    • 1 : intermediate report for a function evaluation
    • 2 : intermediate report for a new optimal value found

abstract interface

  • public subroutine sa_func_parallel_inputs(me, n_inputs)

    interface for parallel function evaluations. This is used when the user wants to evaluate multiple points in parallel (e.g. on a GPU or with OpenMP) This function should return the number of input points to be received

    Arguments

    Type IntentOptional Attributes Name
    class(simulated_annealing_type), intent(inout) :: me
    integer, intent(out) :: n_inputs

    number of inputs that can be received

abstract interface

  • public subroutine sa_func_parallel_inputs_func(me, x)

    interface for parallel function evaluations. This function receives the input points for evaluation

    Arguments

    Type IntentOptional Attributes Name
    class(simulated_annealing_type), intent(inout) :: me
    real(kind=wp), intent(in), dimension(:,:) :: x

    input points (n x n_inputs), each column is one x vector

abstract interface

  • public subroutine sa_func_parallel_output_func(me, x, f, istat)

    interface for parallel function evaluations. This function receives the output points from the parallel evaluation. Returns both the x and f values for a completed evaluation (typically from a queue).

    Arguments

    Type IntentOptional Attributes Name
    class(simulated_annealing_type), intent(inout) :: me
    real(kind=wp), intent(out), dimension(:) :: x
    real(kind=wp), intent(out) :: f
    integer, intent(out) :: istat

    status flag:

    • 0 : valid function evaluation
    • -1 : invalid function evaluation. try a different input vector.
    • -2 : stop the optimization process

Derived Types

type, public ::  simulated_annealing_type

Components

Type Visibility Attributes Name Initial
integer, private :: n = 0

number of variables in the function to be optimized.

logical, private :: maximize = .false.

denotes whether the function should be maximized or minimized. a true value denotes maximization while a false value denotes minimization. intermediate output (see iprint) takes this into account.

real(kind=wp), private :: eps = 1.0e-9_wp

error tolerance for termination. if the final function values from the last neps temperatures differ from the corresponding value at the current temperature by less than eps and the final function value at the current temperature differs from the current optimal function value by less than eps, execution terminates and ier = 0 is returned.

integer, private :: ns = 20

number of cycles. after ns function evaluations, each element of vm is adjusted according to the input step_mode the suggested value is 20.

integer, private :: nt = 100

number of iterations before temperature reduction. after ntns function evaluations, temperature (t) is changed by the factor rt. value suggested by corana et al. is max(100, 5n). see goffe et al. for further advice.

integer, private :: neps = 4

number of final function values used to decide upon termination. see eps. suggested value is 4.

integer, private :: maxevl = 10000

the maximum number of function evaluations. if it is exceeded, ier = 1.

logical, private :: use_initial_guess = .true.

if false, the initial guess is ignored and a random point in the bounds is used for the first function evaluation

integer, private :: n_resets = 2

number of times to run the main loop (must be >=1)

real(kind=wp), private, dimension(:), allocatable :: lb

the lower bound for the allowable solution variables.

real(kind=wp), private, dimension(:), allocatable :: ub

the upper bound for the allowable solution variables. if the algorithm chooses x(i) < lb(i) or x(i) > ub(i), i = 1, n, a point is from inside is randomly selected. this this focuses the algorithm on the region inside ub and lb. unless the user wishes to concentrate the search to a particular region, ub and lb should be set to very large positive and negative values, respectively. note that the starting vector x should be inside this region. also note that lb and ub are fixed in position, while vm is centered on the last accepted trial set of variables that optimizes the function.

real(kind=wp), private, dimension(:), allocatable :: c

vector that controls the step length adjustment. the suggested value for all elements is 2.0.

integer, private :: iprint = 1

controls printing inside sa:

Read more…
integer, private :: iseed1 = 1234

the first seed for the random number generator.

integer, private :: iseed2 = 5678

the second seed for the random number generator. different values for iseed1 and iseed2 will lead to an entirely different sequence of trial points and decisions on downhill moves (when maximizing). see goffe et al. on how this can be used to test the results of sa.

integer, private :: step_mode = 1

how to vary vm after ns cycles.

Read more…
real(kind=wp), private :: vms = 0.1_wp

for step_mode=3, the factor to adjust vm

integer, private :: iunit = output_unit

unit number for prints.

integer, private :: cooling_schedule = 1

temperature reduction schedule:

Read more…
real(kind=wp), private :: cooling_param = 1.0_wp

parameter for cooling schedules 3 and 5. suggested value: 1.0

logical, private :: optimal_f_specified = .false.

if the optional f value is known, it can be specified by optimal_f.

real(kind=wp), private :: optimal_f = 0.0_wp

if optimal_f_specified=True the solver will stop if this value is achieved.

real(kind=wp), private :: optimal_f_tol = 0.0_wp

absolute tolerance for the optimal_f check

integer, private, dimension(:), allocatable :: distribution_mode

distribution to use for perturbations for each variable:

Read more…
real(kind=wp), private, dimension(:), allocatable :: dist_std_dev

standard deviation for normal/truncated_normal

real(kind=wp), private, dimension(:), allocatable :: dist_scale

scale parameter for cauchy/bipareto distributions

real(kind=wp), private, dimension(:), allocatable :: dist_shape

shape parameter for bipareto distribution

procedure(sa_func), private, pointer :: fcn => null()

the user's function

integer, private :: ireport = 0

how often to report intermediate results to the user via report function:

Read more…
procedure(sa_report_func), private, pointer :: report => null()

if associated, this function is called to report intermediate results to the user.

logical, private :: parallel_mode = .false.

if true, the user wants to evaluate multiple points in parallel (e.g. on a GPU or with OpenMP). if false, the user will evaluate one point at a time. if true, the user must provide all of n_inputs_to_send, fcn_parallel_input and fcn_parallel_output.

procedure(sa_func_parallel_inputs), private, pointer :: n_inputs_to_send => null()

if the user wants to evaluate multiple points in parallel, this function should return the number of input points that will be sent for evaluation at a time.

procedure(sa_func_parallel_inputs_func), private, pointer :: fcn_parallel_input => null()

this function receives the input points for evaluation

procedure(sa_func_parallel_output_func), private, pointer :: fcn_parallel_output => null()

this function receives the output points from the parallel evaluation. The x here will be one of the ones sent via sa_func_parallel_inputs_func, The algorithm will only process one at the time.

real(kind=wp), private :: vm_min = 1.0e-12_wp

minimum allowable step size. if all vm values fall below this, terminate with ier=6

Type-Bound Procedures

procedure, public :: initialize => initialize_sa
procedure, public :: optimize => sa
procedure, public :: destroy => destroy_sa
procedure, private :: func
procedure, private :: perturb_and_evaluate
procedure, public :: perturb_variable

this is public so we can use it in the tests


Functions

private pure function func(me, f)

if the function is to be minimized, switch the sign of the function. note that all intermediate and final output switches the sign back to eliminate any possible confusion for the user.

Arguments

Type IntentOptional Attributes Name
class(simulated_annealing_type), intent(in) :: me
real(kind=wp), intent(in) :: f

Return Value real(kind=wp)

private function perturb_variable(me, ivar, x, mode, lower, upper) result(r)

Perturb a single variable using its assigned distribution and parameters.

Arguments

Type IntentOptional Attributes Name
class(simulated_annealing_type), intent(inout) :: me
integer, intent(in) :: ivar

variable index

real(kind=wp), intent(in) :: x

variable value to perturb

integer, intent(in) :: mode

perturbation distribution mode (see distribution_mode for details)

real(kind=wp), intent(in) :: lower

lower bound

real(kind=wp), intent(in) :: upper

upper bound

Return Value real(kind=wp)

perturbed value

private pure function exprep(x) result(f)

this function replaces exp() to avoid underflow and overflow.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: x

Return Value real(kind=wp)

private function uniform_random_number() result(f)

Get a new uniform random number from [0,1].

Read more…

Arguments

None

Return Value real(kind=wp)

private function uniform(xl, xu)

Uniform random number on the interval [xl,xu].

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: xl

lower bound

real(kind=wp), intent(in) :: xu

upper bound

Return Value real(kind=wp)

private function normal(mean, std_dev)

Normal (Gaussian) random number with specified mean and standard deviation. Uses the Box-Muller transform.

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: mean

mean of the distribution

real(kind=wp), intent(in) :: std_dev

standard deviation

Return Value real(kind=wp)

private function cauchy(location, scale)

Cauchy random number with specified location and scale parameters. The Cauchy distribution has heavier tails than the normal distribution, which can be useful for occasional large jumps in simulated annealing.

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: location

location parameter (median)

real(kind=wp), intent(in) :: scale

scale parameter

Return Value real(kind=wp)

private function truncated_normal(mean, std_dev, xl, xu)

Truncated normal distribution within bounds [xl, xu]. Uses rejection sampling to ensure the value stays within bounds.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: mean

mean of the underlying normal distribution

real(kind=wp), intent(in) :: std_dev

standard deviation

real(kind=wp), intent(in) :: xl

lower bound

real(kind=wp), intent(in) :: xu

upper bound

Return Value real(kind=wp)

private function triangular_dist(mode)

Triangular distribution on [0,1] with specified mode. Uses inverse transform sampling - very efficient, no rejection needed.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: mode

mode of the distribution (should be in [0,1])

Return Value real(kind=wp)

private function bipareto(center, scale, shape, xl, xu)

Bi-polar (two-sided) Pareto distribution. Generates a symmetric heavy-tailed distribution centered at a location.

Read more…

Arguments

Type IntentOptional Attributes Name
real(kind=wp), intent(in) :: center

center location of the distribution

real(kind=wp), intent(in) :: scale

scale parameter for the Pareto distribution

real(kind=wp), intent(in) :: shape

shape parameter (smaller values = heavier tails)

real(kind=wp), intent(in) :: xl

lower bound

real(kind=wp), intent(in) :: xu

upper bound

Return Value real(kind=wp)


Subroutines

private subroutine destroy_sa(me)

Destructor.

Arguments

Type IntentOptional Attributes Name
class(simulated_annealing_type), intent(out) :: me

private subroutine initialize_sa(me, n, lb, ub, c, maximize, eps, ns, nt, neps, maxevl, iprint, iseed1, iseed2, step_mode, vms, iunit, use_initial_guess, n_resets, cooling_schedule, cooling_param, optimal_f_specified, optimal_f, optimal_f_tol, distribution_mode, dist_std_dev, dist_scale, dist_shape, fcn, n_inputs_to_send, fcn_parallel_input, fcn_parallel_output, ireport, report)

Initialize the class.

Read more…

Arguments

Type IntentOptional Attributes Name
class(simulated_annealing_type), intent(inout) :: me
integer, intent(in) :: n
real(kind=wp), intent(in), dimension(n) :: lb
real(kind=wp), intent(in), dimension(n) :: ub
real(kind=wp), intent(in), optional, dimension(n) :: c
logical, intent(in), optional :: maximize
real(kind=wp), intent(in), optional :: eps
integer, intent(in), optional :: ns
integer, intent(in), optional :: nt
integer, intent(in), optional :: neps
integer, intent(in), optional :: maxevl
integer, intent(in), optional :: iprint
integer, intent(in), optional :: iseed1
integer, intent(in), optional :: iseed2
integer, intent(in), optional :: step_mode
real(kind=wp), intent(in), optional :: vms
integer, intent(in), optional :: iunit
logical, intent(in), optional :: use_initial_guess
integer, intent(in), optional :: n_resets
integer, intent(in), optional :: cooling_schedule

temperature reduction schedule (1-5)

real(kind=wp), intent(in), optional :: cooling_param

parameter for cooling schedules 3 and 5

logical, intent(in), optional :: optimal_f_specified

if the optional f value is known, it can be specified by optimal_f. [Default is False]

real(kind=wp), intent(in), optional :: optimal_f

if optimal_f_specified=True the solver will stop if this value is achieved.

real(kind=wp), intent(in), optional :: optimal_f_tol

absolute tolerance for the optimal_f check

integer, intent(in), optional, dimension(:) :: distribution_mode

distribution for perturbations (per variable):

Read more…
real(kind=wp), intent(in), optional, dimension(:) :: dist_std_dev

std dev for normal/truncated_normal (per variable or scalar)

real(kind=wp), intent(in), optional, dimension(:) :: dist_scale

scale for cauchy/pareto (per variable or scalar)

real(kind=wp), intent(in), optional, dimension(:) :: dist_shape

shape for pareto (per variable or scalar)

procedure(sa_func), optional :: fcn

the user's function for scalar evaluation. if not present, the user must provide all of n_inputs_to_send, fcn_parallel_input and fcn_parallel_output for parallel evaluation.

procedure(sa_func_parallel_inputs), optional :: n_inputs_to_send

if the user wants to evaluate multiple points in parallel, this function should return the number of input points that will be sent for evaluation at a time.

procedure(sa_func_parallel_inputs_func), optional :: fcn_parallel_input

this function receives the input points for evaluation

procedure(sa_func_parallel_output_func), optional :: fcn_parallel_output

this function receives the output points from the parallel evaluation. The x here will be one of the ones sent via sa_func_parallel_inputs_func, The algorithm will only process one at the time.

integer, intent(in), optional :: ireport

how often to report intermediate results to the user via report function:

Read more…
procedure(sa_report_func), optional :: report

if associated, this function is called to report intermediate results to the user.

private subroutine sa(me, x, rt, t, vm, xopt, fopt, nacc, nfcnev, ier)

Continuous simulated annealing global optimization algorithm

Read more…

Arguments

Type IntentOptional Attributes Name
class(simulated_annealing_type), intent(inout) :: me
real(kind=wp), intent(inout), dimension(me%n) :: x

on input: the starting values for the variables of the function to be optimized. [Will be replaced by final point]

real(kind=wp), intent(in) :: rt

the temperature reduction factor. the value suggested by corana et al. is .85. see goffe et al. for more advice.

real(kind=wp), intent(inout) :: t

on input, the initial temperature. see goffe et al. for advice. on output, the final temperature. Note that if t=0, then all downhill steps will be rejected

real(kind=wp), intent(inout), dimension(me%n) :: vm

the step length vector. on input it should encompass the region of interest given the starting value x. for point x(i), the next trial point is selected is from x(i) - vm(i) to x(i) + vm(i). since vm is adjusted so that about half of all points are accepted, the input value is not very important (i.e. if the value is off, sa adjusts vm to the correct value). note: if vm=0.0, then it is set to abs(xu-xl)`.

real(kind=wp), intent(out), dimension(me%n) :: xopt

the variables that optimize the function.

real(kind=wp), intent(out) :: fopt

the optimal value of the function.

integer, intent(out) :: nacc

the number of accepted function evaluations.

integer, intent(out) :: nfcnev

the total number of function evaluations. in a minor point, note that the first evaluation is not used in the core of the algorithm; it simply initializes the algorithm.

integer, intent(out) :: ier

the error return number:

Read more…

private subroutine perturb_and_evaluate(me, x, vm, xp, fp, nfcnev, ier, first)

Perturb the x vector and evaluate the function.

Read more…

Arguments

Type IntentOptional Attributes Name
class(simulated_annealing_type), intent(inout) :: me
real(kind=wp), intent(in), dimension(:) :: x

input optimization variable vector to perturb

real(kind=wp), intent(in), dimension(:) :: vm

step length vector

real(kind=wp), intent(out), dimension(:) :: xp

the perturbed x value

real(kind=wp), intent(out) :: fp

the value of the user function at xp

integer, intent(inout) :: nfcnev

total number of function evaluations

integer, intent(inout) :: ier

status output code

logical, intent(in), optional :: first

to use the input x the first time

private subroutine rand_init(seed1, seed2)

Initialize the intrinsic random number generator.

Read more…

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: seed1

the first seed for the random number generator.

integer, intent(in) :: seed2

the second seed for the random number generator.

public subroutine print_vector(iunit, vector, ncols, name)

this subroutine prints the double precision vector named vector. elements 1 thru ncols will be printed. name is a character variable that describes vector. note that if name is given in the call to print_vector, it must be enclosed in quotes. if there are more than 10 elements in vector, 10 elements will be printed on each line.

Arguments

Type IntentOptional Attributes Name
integer, intent(in) :: iunit
real(kind=wp), intent(in), dimension(ncols) :: vector
integer, intent(in) :: ncols
character(len=*), intent(in) :: name