Entropy is a fundamental quantity in many areas of knowledge, from physics to information science to biology. Originally put forward in the nineteenth century for very practical purposes (to quantify the reversibility of thermodynamic cycles, hence of thermal engines), entropy was the key concept that allowed Ludwig Boltzmann to bridge the gap between the (time irreversible) macroscopic thermodynamics and the (reversible) microscopic Newtonian physics. As defined by Boltzmann, the entropy S_B represents the number of microscopic states that are compatible with a given macroscopic realization:

$$S_{\mathrm{B}}=k_{\mathrm{B}}\ln \mathrm{\Omega },$$(1)

where k_B is Boltzmann’s constant and Ω is the relevant phase space volume, which is a measure of the number of microscopic states. Note that the logarithm in the above definition is required so that Boltzmann’s statistical entropy possesses the same additive properties as the thermodynamic entropy.

Later, Claude Shannon discovered that a similar formula to Boltzmann’s (albeit with the opposite sign) can be used to quantify the information content of a signal. Following Shannon’s work, it is customary to identify entropy with the (lack of) information or “disorder” of a system. As information is a concept that permeates many natural sciences, the concept of entropy quickly spread to other fields, such as biology and genetics.

It was John von Neumann who generalized the Boltzmann entropy to quantum physics. This is actually more than a mere generalization. Indeed, equation (1) is somewhat problematic, as Ω has the dimensions of a phase space volume, while the argument of the logarithm should be nondimensional – not to mention that S_B can become negative. But considering that quantum mechanics introduces a minimal action given by Planck’s constant h, Boltzmann’s formula can be rewritten as: S_B = kln(Ω/h^d) (where d is the number of dimensions of the system), which is always nonnegative as long as Ω ≥ h^d and vanishes only when the equality sign holds. In terms of discrete quantum states with occupation probabilities p_i, the von Neumann entropy reads as

$$S_{\mathrm{V}N}=-\sum _{i=1}^n\mathrm{}{p}_i\ln p_i,$$(2)

which is always positive and only vanishes when one state is occupied with probability equal to one.

While it is generally admitted that the Boltzmann/von Neumann/Shannon entropy is a measure of the lack of information of a system, it is also clear that such a measure is not unique. Given a set of events endowed with probabilities p_i, one can construct many other formulae that quantify our uncertainty about the actual observed outcome. And indeed, many other definitions of entropy have been proposed in the past [1], so many that it is virtually impossible to do justice to all of them. In physics, Tsallis’ entropies are a popular example [2], as are Rényi’s entropies [3]. In biology too, several measures of biological diversity have been put forward, see for instance the recent review [4].

How is one to navigate among this plethora of entropy definitions? An answer was provided recently by Ellerman [5], who emphasized the importance of making distinctions between elements of a given set U. If such a set is partitioned into a number n of subsets B_i (such that ${\cup }_{i=1}^n{B}_i=U$), each endowed with a probability p_i of finding an element of U in that subset, then the probability that in two independent draws one will obtain elements in distinct subsets B_i and $B_{j\ne i}$ is: $p_i(1-p_i)$. This is precisely the concept of distinction, i.e., the ability to establish that two independent draws are different from one another. Summing over all n subsets, we obtain the total number of distinctions, which is the definition of what Ellerman termed logical entropy:

$$S_{\mathrm{L}}=\sum _{i=1}^n\mathrm{}{p}_i(1-p_i)=1-\sum _{i=1}^n\mathrm{}{p}_i^2.$$(3)

The subsets B_i may contain one single element, in which case S_L represents the probability that two consecutive draws yield different elements of U. S_L varies in the interval [0,1], and the lower bound is reached when one element has probability p_i = 1, while for all others $p_{j\ne i}=0$. For equal probabilities ($p_i=1/n, \forall i$), one gets: $S_L=1-1/n\to 1$, when $n\to \mathrm{\infty }$.

Although the formula (3) is not itself original (for instance, it is a special case of the Tsalllis entropy), its interpretation in terms of partitions of a set is new and illuminating. Indeed, the logical entropy enjoys a number of intriguing features (e.g., it has the properties of a measure, in the precise mathematical sense) which single it out among the many other definitions proposed in the literature.

This Special Issue contains a collection of four papers devoted to the logical entropy and its applications to physics, particularly quantum mechanics.

The paper by David Ellerman [6] contains an exhaustive introduction, both conceptual and historical, to the logical entropy. In particular, it discusses how the latter relates to the more usual Shannon entropy in quantifying the informational content of a set.

The paper by Boaz Tamir et al. [7] discusses the possible extension of the logical entropy to the quantum domain. Indeed, the logical entropy can be expressed in terms of the density matrix ρ as: $S_{\mathrm{L}}=1-\mathrm{T}\mathrm{r}{\rho }^2$, and quantifies the purity of a quantum state. The authors prove several properties of this entropy for generic density matrices that are relevant to various areas of quantum mechanics and quantum information.

The paper by Denis Sunko [8] addresses the logical entropy in the context of many-body quantum mechanics. It shows how the logical entropy can be used to distinguish many-body fermion states by their information content, although they are pure states whose usual quantum entropies are equal to zero.

Finally, the paper by Giovanni Manfredi [9] points out that the definition of logical entropy (3) lends itself quite naturally to a generalization of probabilities to negative values, an idea that goes back to Feynman and Wigner. By combining negative probabilities with the definition of logical entropy, one can recover many intriguing properties that are typical of quantum systems.

We hope this Special Issue will foster research into a topic that has a long and distinguished past history, but still much potential for future developments.

References

© G. Manfredi, Published by EDP Sciences, 2022