Information Theory:
"What is it?" and
"How can it inform ALife research?"
How do we measure complexity, cognition, and hidden structure in evolved systems?
Shannon Information Theory (IT) provides a powerful mathematical toolkit for analyzing similarities, dissimilarities, and structure within data. In Artificial Life (ALIFE) research, IT has been used to quantify population dynamics, classify cognition, and reveal the internal workings of evolved agents. However, its mathematics can be daunting — and misinterpretation is common.
This tutorial introduces IT concepts and methods in a way that is accessible and directly relevant to ALIFE research. The focus is on building intuition — understanding what IT measures reveal (and what they do not), and how to apply them effectively in practice.
Methods covered:
- Representation (quantification of knowledge)
- Fragmentation and Smearedness (distribution of knowledge)
- Data Flow and Transfer Entropy
- Cognitive complexity (PHI)
- Information Decomposition (PID and FID)
Applications of IT:
- qunatifying population diversity and structure
- classifying cognition based on information processing capacity
- understanding cognition using information flow/tracing
- encryption (what exactly is it)
- the relationship between IT and emergence
This tutorial is intended for ALIFE researchers at all levels, from newcomers curious about Information Theory to experienced practitioners seeking deeper insight. I will focuse on developing conceptual understanding and intuition. While some mathematical formalism is required, it will be minimized whenever possible. By the end of the session, attendees should have a clear sense of what IT can and cannot reveal, and how they may employ IT in their own research.
About the presenter
My name is Cliff Bohm. I’m a 7th year PhD student working with Chris Adami. Although at the time of this tutorial I will be a PostDoc working for Emily Dolson. I work on various topics in ALIFE, and have used Information Theory both to understand the operation of evolved digital cognitions as well as to quantify ecology. I’m currently working on a formal definition of emergence based on information theory.
Some papers with relevent content that will be covered
- Marstaller, L., Hintze, A. and Adami, C. (2013). The evolution of representation in simple cognitive networks. Neural computation, 25(8), pp.2079-2107. Link to Paper
- Bohm, Clifford, Douglas Kirkpatrick, Victoria Cao, and Christoph Adami (2022). Information fragmentation, encryption and information flow in complex biological networks. Entropy 24, no. 5 Link to Paper
- Williams, Paul L., and Randall D. Beer (2022). Nonnegative decomposition of multivariate information. arXiv:1004.2515 Link to Paper
- Schreiber, Thomas (2000). Measuring information transfer. Physical review letters 85.2 Link to Paper
- Dolson, E. L., Vostinar, A. E., Wiser, M. J., & Ofria, C. (2019). The MODES toolbox: Measurements of open-ended dynamics in evolving systems. Artificial life , 25(1) Link to Paper
Contact
If you have questions before or after the tutorial, feel free to contact me:
Email: cliff.bohm@gmail.com