“A few exciting words”: Information and entropy revisited
Open Access
- 13 May 2015
- journal article
- research article
- Published by Wiley in Journal of the Association for Information Science and Technology
- Vol. 66 (10) , 1965-1987
Abstract
A review is presented of the relation between information and entropy, focusing on two main issues: the similarity of the formal definitions of physical entropy, according to statistical mechanics, and of information, according to information theory; and the possible subjectivity of entropy considered as missing information. The paper updates the 1983 analysis of Shaw and Davis. The difference in the interpretations of information given respectively by Shannon and by Wiener, significant for the information sciences, receives particular consideration. Analysis of a range of material, from literary theory to thermodynamics, is used to draw out the issues. Emphasis is placed on recourse to the original sources, and on direct quotation, to attempt to overcome some of the misunderstandings and oversimplifications that have occurred with these topics. Although it is strongly related to entropy, information is neither identical with it, nor its opposite. Information is related to order and pattern, but also to disorder and randomness. The relations between information and the “interesting complexity,” which embodies both patterns and randomness, are worthy of attention.Keywords
This publication has 91 references indexed in Scilit:
- Physical Order Produces Healthy Choices, Generosity, and Conventionality, Whereas Disorder Produces CreativityPsychological Science, 2013
- Measurement of statistical evidence on an absolute scale following thermodynamic principlesTheory in Biosciences, 2013
- What Is Information?: Why Is It Relativistic and What Is Its Relationship to Materiality, Meaning and OrganizationInformation, 2012
- Cybersemiotics: An Evolutionary World View Going Beyond Entropy and Information into the Question of MeaningEntropy, 2010
- The meanings of entropyEntropy, 2005
- The Deep Physics Behind the Second Law: Information and Energy As Independent Forms of BookkeepingEntropy, 2004
- Some Observations on the Concepts of Information-Theoretic Entropy and RandomnessEntropy, 2001
- Insight into entropyAmerican Journal of Physics, 2000
- Remembering about documents: memory for appearance, format, and locationErgonomics, 1991
- Information Theory and Statistical MechanicsPhysical Review B, 1957