Complex Systems Dynamics PDF

Preface

In 1982, a paper by John Hopfield entitled "Neural Networks and Physical Systems with Emergent Collective Computational Abilities" appeared in the Proceedings of the National Academy of Sciences. The "recipe" of the paper was the following:

• Start with a network composed of very simplified models of the components of the nervous system.

• Apply some ideas from statistical mechanics, and rely on numerical computer simulation to obtain results.

• Compare emergent collective properties to qualitatively similar properties of the brain.

• From this, propose new circuits and new computer architectures.

Although this was not a new subject, as is evident from the bibliography, this paper and a handful of others of comparable quality were the starting point of an extraordinary rebirth of this field of scientific activity. Researchers, funding, meetings, papers, and new machines rapidly materialized in ever greater numbers.

The goal of this book is to introduce the reader to this research field in order that he or she may understand its methods and concepts as well as the many varied fields of application. Among these, let us just mention the most important:

• Physics of disordered systems and of growth and forms.

• The biology of the brain, but also certain key problems concerning development, immunology, and the origin of life.

• Signal processing and the design of parallel computers.

This book is a general introduction aimed at readers with a university education in science or engineering. An understanding of elementary calculus is required. We have attempted to avoid modern mathematical formalism, and consequently, although most algorithms are explained in the appendix, we leave it to the reader to look up long calculations and elegant proofs in the appropriate references. In fact the work of the researchers in this field often involves numerical simulation, which is a "hybrid" of theory and experiment.

The reader will note that the subject is not only rich in methods, but also enables a number of significant concepts to be defined or stated in a different framework. All of these new concepts appear in italics in the paragraph of the book in which they are first defined. This paragraph is referenced in the index. The applications which are discussed do not require that the reader be simultaneously a physicist, a biologist, and a specialist in signal processing—all of the necessary concepts are in principle redefined in the text. However, it sometimes happens that for a given problem, the performance of automata networks is compared to that of more traditional methods; in this case the explanations of the latter are very succinct.

The organization of the book is as follows: In chapter 1, we define complex systems, to which we will apply the method of automata networks. A very brief overview will place new perspectives resulting from advances in theoretical biology and parallel computation in historical context. Fundamental definitions relating to automata, networks, and their dynamics will be given in chapter 2, along with several examples. One-dimensional cellular automata networks discussed in chapter 3 will give other simple examples which enable the dynamics of the networks to be understood and characterized. Chapter 4 discusses two-dimensional cellular automata networks and their application to the physics of growth and to fluid mechanics.

Chapter 5, which is on formal neurons, is the basis for the four following chapters. The explanation of the Hopfield model includes the fundamental notions of a formal neuron, recognition, associative memories, and learning. Chapter 6 deals with higher performance methods, which go beyond certain limitations of the Hopfield model. Along the same lines, chapter 7 discusses the back-propagation technique, which is one of the most powerful techniques to date in applications of signal processing.

In chapter 8 we tackle the domain of probabilistic automata. The notion of temperature is restated in this context, which enables a link to be established between the problems of associative memories and of spin glasses. Chapter 9 widens our perspective. Simulated annealing allows a parallel approach to combinatorical optimization, which is a class of problems of great technological importance. An application to image processing is discussed.

Chapter 10 brings us back to more general considerations on organization, chaos, and the genericity of the dynamical properties of automata networks. Finally, chapter 11 discusses the application of these ideas to population genetics in the broadest sense.

In the conclusion, in chapter 12, we first discuss several limitations of the network method, and then we give a few bibliographical references for the reader who would like more detailed information about certain subjects in the book, or about methods and applications which were not discussed.

We think that the automata network approach could shed light on many areas, from biology to parallel computation, including physics. This by no means implies that it can enable us to understand everything, and even less that this book covers all aspects mentioned in the introduction. We believe that modern science is loathe to accept global concepts, and consequently, the purpose of this book is to clarify some of these concepts by showing by examples how broadly they can be applied. We believe that the enlightenment brought about by concepts from other fields can only improve the understanding of a scientific field. However, readers pressed for time can concentrate their efforts according to their own interests. Chapters 1, 2, 3, and 5 are probably indispensable. Chapters 4, 8, and 10 focus on physics and dynamical systems. Chapters 6, 7, and 9 deal with signal processing and combinatorial optimization. Chapter 8 is recommended for non-physicists, as it will help them to understand chapter 9. Chapter 11 deals with developmental biology and evolution.

We have not attempted to give a complete bibliography, as the number of papers published in this field is already of the order of a few thousand. We do not even claim to have listed the most important works. The references are either remarkably clear books or papers, articles which inspired a passage in the book, or reference works which allow the reader to delve into a particular field.

A few references have been added to the English version.

The choice of discussed subjects reflects our own research interests. Many researchers with whom we have collaborated deserve thanks, whether or not they are at the Statistical Physics Laboratory of the Ecole Normale Superieure: Henri Atlan, Bernard Derrida, Francoise Fogelman-Soulie, Dominique d'Humieres, Werner Krauth, Jean-Pierre Nadal, Gerard Toulouse, and Jean Vannimenus. Their suggestions and corrections have greatly contributed to this book. The constant encouragement of Michele Leduc, director of the French series where this book was originally published, was invaluable throughout the preparation of this manuscript, from start to finish. Thanks are also due to Sylvie Ryckebusch and Ronda Butler- Villa for the English version of this book.

Author: Gerard Weisbuch