Boltzmann sampler

Not to be confused with Boltzmann machine.

A Boltzmann sampler is an algorithm intended for random sampling of combinatorial structures. If the object size is viewed as its energy, and the argument of the corresponding generating function is interpreted in terms of the temperature of the physical system, then a Boltzmann sampler returns an object from a classical Boltzmann distribution.

The concept of Boltzmann sampler was proposed by Philippe Duchon, Philippe Flajolet, Guy Louchard and Gilles Schaeffer in 2004.

Description

The concept of Boltzmann sampling is closely related to the symbolic method in combinatorics. Let ${\mathcal {C}}$ be a combinatorial class with an ordinary generating function $C(z)$ which has a nonzero radius of convergence $\rho$, i.e. is complex analytic. Formally speaking, if each object $c\in {\mathcal {C}}$ is equipped with a non-negative integer size $\omega (c)$, then the generating function $C(z)$ is defined as

$C(z)=\sum _{c\in {\mathcal {C}}}z^{\omega (c)}=\sum _{n=0}^{\infty }a_{n}z^{n},$

where $a_{n}$ denotes the number of objects $c\in {\mathcal {C}}$ of size $n$. The size function is typically used to denote the number of vertices in a tree or in a graph, the number of letters in a word, etc.

A Boltzmann sampler for the class ${\mathcal {C}}$ with a parameter $z$ such that $0<z<\rho$, denoted as $\Gamma {\mathcal {C}}(z)$ returns an object $c\in {\mathcal {C}}$ with probability

$\mathbb {P} (\Gamma {\mathcal {C}}(z)=c)={\dfrac {z^{\omega (c)}}{C(z)}}.$

Construction

Finite sets

If ${\mathcal {C}}=\{c_{1},\ldots ,c_{r}\}$ is finite, then an element $c_{j}$ is drawn with probability proportional to $z^{\omega (c_{j})}$.

Disjoint union

If the target class is a disjoint union of two other classes, ${\mathcal {C}}={\mathcal {A}}+{\mathcal {B}}$, and the generating functions $A(z)$ and $B(z)$ of ${\mathcal {A}}$ and ${\mathcal {B}}$ are known, then the Boltzmann sampler for ${\mathcal {C}}$ can be obtained as

$\left(\operatorname {Bern} \left({\frac {A(z)}{C(z)}}\right)\longrightarrow \Gamma {\mathcal {A}}(z)\mid \Gamma {\mathcal {B}}(z)\right)$

where $(X\longrightarrow f\mid g)$ stands for "if the random variable $X$ is 1, then execute $f$, else execute $g$". More generally, if the disjoint union is taken over a finite set, the resulting Boltzmann sampler can be represented using a random choice with probabilities proportional to the values of the generating functions.

Cartesian product

If ${\mathcal {C}}={\mathcal {A}}\times {\mathcal {B}}$ is a class constructed of ordered pairs $(a,b)$ where $a\in {\mathcal {A}}$ and $b\in {\mathcal {B}}$, then the corresponding Boltzmann sampler $\Gamma {\mathcal {C}}(z)$ can be obtained as

$\Gamma {\mathcal {C}}(z)=(\Gamma {\mathcal {A}}(z),\Gamma {\mathcal {B}}(z)),$

i.e. by forming a pair $(a,b)$ with $a$ and $b$ drawn independently from $\Gamma {\mathcal {A}}(z)$ and $\Gamma {\mathcal {B}}(z)$.

Sequence

If ${\mathcal {C}}$ is composed of all the finite sequences of elements of ${\mathcal {A}}$ with size of a sequence additively inherited from sizes of components, then the generating function of ${\mathcal {C}}$ is expressed as $C(z)=\sum _{k=0}^{\infty }A(z)^{k}={\dfrac {1}{1-A(z)}}$, where $A(z)$ is the generating function of ${\mathcal {A}}$. Alternatively, the class ${\mathcal {C}}$ admits a recursive representation ${\mathcal {C}}=1+{\mathcal {A}}\times {\mathcal {C}}.$ This gives two possibilities for $\Gamma {\mathcal {C}}(z)$.

1. $\Gamma {\mathcal {C}}(z)=\Gamma (1+{\mathcal {A}}\times {\mathcal {C}})(z)=\left(\operatorname {Bern} \left({\frac {1}{C(z)}}\right)\longrightarrow 1\,{\Big |}\,(\Gamma {\mathcal {A}}(z),\Gamma {\mathcal {C}}(z))\right)$
2. $\Gamma {\mathcal {C}}(z)=\left(\operatorname {Geom} (A(z))\Longrightarrow \Gamma {\mathcal {C}}(z)\right)$

where $(X\Longrightarrow f)$ stands for "draw a random variable $X$; if the value $X=x$ is returned, then execute $f$ independently $x$ times and return the sequence obtained". Here, $\operatorname {Geom} (p)$ stands for the geometric distribution $\mathbb {P} (\operatorname {Geom} (p)=k)=p^{k}(1-p)$.

Recursive classes

As the first construction of the sequence operator suggests, Boltzmann samplers can be used recursively. If the target class ${\mathcal {C}}$ is a part of the system

${\begin{cases}{\mathcal {C}}_{1}=\Phi _{1}({\mathcal {C}}_{1},\ldots ,{\mathcal {C}}_{n},{\mathcal {Z}}),\\\qquad \vdots \\{\mathcal {C}}_{n}=\Phi _{n}({\mathcal {C}}_{1},\ldots ,{\mathcal {C}}_{n},{\mathcal {Z}}),\end{cases}}$

where each of the expressions $\Phi _{k}({\mathcal {C}}_{1},\ldots ,{\mathcal {C}}_{n},{\mathcal {Z}})$ involves only disjoint union, cartesian product and sequence operator, then the corresponding Boltzmann sampler is well defined. Given the argument value $z$, the numerical values of the generating functions can be obtained by Newton iteration.

Labelled structures

Boltzmann sampling can be applied to labelled structures. For a labelled combinatorial class ${\mathcal {C}}$, exponential generating function is used instead:

$C(z)=\sum _{c\in {\mathcal {C}}}{\dfrac {z^{\omega (c)}}{\omega (c)!}}=\sum _{n=0}^{\infty }a_{n}{\dfrac {z^{n}}{n!}},$

where $a_{n}$ denotes the number of labelled objects $c\in {\mathcal {C}}$ of size $n$. The operation of cartesian product and sequence need to be adjusted to take labelling into account, and the principle of construction remains the same.

In the labelled case, the Boltzmann sampler for a labelled class ${\mathcal {C}}$ is required to output an object $c\in {\mathcal {C}}$ with probability

$\mathbb {P} (\Gamma {\mathcal {C}}(z)=c)={\frac {1}{C(z)}}{\frac {z^{\omega (c)}}{\omega (c)!}}.$

Labelled sets

In the labelled universe, a class ${\mathcal {C}}$ can be composed of all the finite sets of elements of a class ${\mathcal {A}}$ with order-consistent relabellings. In this case, the exponential generating function of the class ${\mathcal {C}}$ is written as

$C(z)=\sum _{k=0}^{\infty }{\dfrac {A(z)^{k}}{k!}}=e^{A(z)}$

where $A(z)$ is the exponential generating function of the class ${\mathcal {A}}$. The Boltzmann sampler for ${\mathcal {C}}$ can be described as

$\Gamma {\mathcal {C}}(z)=\left(\operatorname {Poisson} (A(z))\Longrightarrow \Gamma {\mathcal {C}}(z)\right)$

where $\operatorname {Poisson} (\lambda )$ stands for the standard Poisson distribution $\mathbb {P} (\operatorname {Poisson} (\lambda )=k)=e^{-\lambda }{\dfrac {\lambda ^{k}}{k!}}$.

Labelled cycles

In the cycle construction, a class ${\mathcal {C}}$ is composed of all the finite sequences of elements of a class ${\mathcal {A}}$, where two sequences are considered equivalent if they can be obtained by a cyclic shift. The exponential generating function of the class ${\mathcal {C}}$ is written as

$C(z)=\sum _{k=0}^{\infty }{\frac {A(z)^{k}}{k}}=\log {\frac {1}{1-A(z)}}$

where $A(z)$ is the exponential generating function of the class ${\mathcal {A}}$. The Boltzmann sampler for ${\mathcal {C}}$ can be described as

$\Gamma {\mathcal {C}}(z)=\left(\operatorname {Loga} (A(z))\Longrightarrow \Gamma {\mathcal {C}}(z)\right)$

where $\operatorname {Loga} (\lambda )$ describes the log-law distribution $\mathbb {P} (\operatorname {Loga} (\lambda )=k)={\dfrac {1}{\log(1-\lambda )^{-1}}}{\dfrac {\lambda ^{k}}{k}}$.

Properties

Let $N$ denote the random size of the generated object from $\Gamma {\mathcal {C}}(z)$. Then, the size has the first and the second moment satisfying

1. $\mathbb {E} _{z}(N)=z{\dfrac {C'(z)}{C(z)}};$
2. $\mathbb {E} _{z}(N^{2})={\dfrac {z^{2}C''(z)+zC'(z)}{C(z)}};$
3. $z{\dfrac {d}{dz}}\mathbb {E} _{z}(N)=\operatorname {Var} _{z}(N)$.

Examples

Binary trees

The class ${\mathcal {B}}$ of binary trees can be defined by the recursive specification

${\mathcal {B}}={\mathcal {Z}}+{\mathcal {Z}}\times {\mathcal {B}}\times {\mathcal {B}}$

and its generating function $B(z)$ satisfies an equation $B(z)=z+zB(z)^{2}$ and can be evaluated as a solution of the quadratic equation

$B(z)={\dfrac {1-{\sqrt {1-4z^{2}}}}{2z}}$

The resulting Boltzmann sampler can be described recursively by

$\Gamma {\mathcal {B}}(z)=\left(\operatorname {Bern} \left({\frac {z}{B(z)}}\right)\longrightarrow {\mathcal {Z}}\mid \left({\mathcal {Z}},\,\Gamma {\mathcal {B}}(z),\,\Gamma {\mathcal {B}}(z)\right)\right)$

Set partitions

Consider various partitions of the set $\{1,2,\ldots ,n\}$ into several non-empty classes, being disordered between themselves. Using symbolic method, the class ${\mathcal {C}}$ of set partitions can be expressed as

${\mathcal {C}}=\operatorname {Set} (\operatorname {Set} _{>0}({\mathcal {Z}})).$

The corresponding generating function is equal to $C(z)=e^{e^{z}-1}$. Therefore, Boltzmann sampler can be described as

$\Gamma {\mathcal {C}}=\left(\operatorname {Poisson} (e^{z}-1)\Longrightarrow \left(\operatorname {Poisson} _{>0}(z)\Longrightarrow {\mathcal {Z}}\right)\right),$

where the positive Poisson distribution $\operatorname {Poisson} _{>0}(\lambda )$ is a Poisson distribution with a parameter $\lambda$ conditioned to take only positive values.

Further generalisations

The original Boltzmann samplers described by Philippe Duchon, Philippe Flajolet, Guy Louchard and Gilles Schaeffer only support basic unlabelled operations of disjoint union, cartesian product and sequence, and two additional operations for labelled classes, namely the set and the cycle construction. Since then, the scope of combinatorial classes for which a Boltzmann sampler can be constructed, has expanded.

Unlabelled structures

The admissible operations for unlabelled classes include such additional operations as Multiset, Cycle and Powerset. Boltzmann samplers for these operations have been described by Philippe Flajolet, Éric Fusy and Carine Pivoteau.

Differential specifications

Let ${\mathcal {C}}$ be a labelled combinatorial class. The derivative operation is defined as follows: take a labelled object $c\in {\mathcal {C}}$ and replace an atom with the largest label with a distinguished atom without a label, therefore reducing a size of the resulting object by 1. If $C(z)=\sum _{n=0}^{\infty }a_{n}{\dfrac {z^{n}}{n!}}$ is the exponential generating function of the class ${\mathcal {C}}$, then the exponential generating function of the derivative class ${\mathcal {C}}'$is given by

$$C'(z)={\dfrac {d}{dz}}C(z)=\sum _{n=0}^{\infty }na_{n}{\frac {z^{n-1}}{n!}}$$

A differential specification is a recursive specification of type

${\mathcal {T}}'=\Phi ({\mathcal {T}},{\mathcal {Z}})$

where the expression $\Phi ({\mathcal {T}},{\mathcal {Z}})$ involves only standard operations of union, product, sequence, cycle and set, and does not involve differentiation.

Boltzmann samplers for differential specifications have been constructed by Olivier Bodini, Olivier Roussel and Michèle Soria.

Multi-parametric Boltzmann samplers

A multi-parametric Boltzmann distribution for multiparametric combinatorial classes is defined similarly to the classical case. Assume that each object $c\in {\mathcal {C}}$ is equipped with the composition size $\omega (c)=(\omega _{1}(c),\ldots ,\omega _{d}(c))$ which is a vector of non-negative integer numbers. Each of the size functions $\omega _{j}(c)$ can reflect one of the parameters of a data structure, such as the number of leaves of certain colour in a tree, the height of the tree, etc. The corresponding multivariate generating function $C(z_{1},\ldots ,z_{d})$ is then associated with a multi-parametric class, and is defined as

$$C(z_{1},\ldots ,z_{d})=\sum _{c\in {\mathcal {C}}}z_{1}^{\omega _{1}(c)}\cdots z_{d}^{\omega _{d}(c)}.$$

A Boltzmann sampler for the multiparametric class ${\mathcal {C}}$ with a vector parameter ${\boldsymbol {z}}=(z_{1},\ldots ,z_{d})$ inside the domain of analyticity of $C(z_{1},\ldots ,z_{d})$, denoted as

$\Gamma {\mathcal {C}}(z_{1},\ldots ,z_{d})$ returns an object $c\in {\mathcal {C}}$ with probability

$\mathbb {P} (\Gamma {\mathcal {C}}(z_{1},\ldots ,z_{d})=c)={\frac {z_{1}^{\omega _{1}(c)}\cdots z_{d}^{\omega _{d}(c)}}{C(z_{1},\ldots ,z_{d})}}.$

Multiparametric Boltzmann samplers have been constructed by Olivier Bodini and Yann Ponty. A polynomial-time algorithm for finding the numerical values of the parameters $z_{1},\ldots ,z_{d}$ given the target parameter expectations, can be obtained by formulating an auxiliary convex optimisation problem

Applications

Boltzmann sampling can be used to generate algebraic data types for the sake of property-based testing.

Software

• Random Discrete Objects Suite (RDOS): http://lipn.fr/rdos/
• Combstruct package in Maple: https://www.maplesoft.com/support/help/Maple/view.aspx?path=combstruct
• Haskell package Boltzmann Brain: https://github.com/maciej-bendkowski/boltzmann-brain