A thought occured to me recently while reading Borges' short story the Library of Babel. I had also been doing a lot of reading on information science as of late.

The story, which is worth a read, posits a bunch of librarians living in a massive (but finite!) library. Each book in the library has a given number of characters per page, and a given number of pages. The library doesn't repeat, it simply contains every possible combination of characters per book, in different books. This represents 10^4677 books. For comparison, estimates of the number of protons in the visible universe are around 10^78-82.

From the stand point of what can be put into language, or any human code, this Library of Babel likely represents the maximum entropy of any code possible.

You can take a look at the library here: https://libraryofbabel.info/About.html

An interesting fact that Borges notes in the story is that the pages of meaningless strings of characters that are far more likely to occur than real words, all actually mean something. Somewhere in the library is a book, or series of books, that explains a code through which said book can be translated into a meaningful message. Even a repeating string of "XLLLX XLLLX" could have multiple meanings, if we take that each block modifies the ones before or after it.

This led me to a few thoughts about information theory that, despite some desperate searching, I could not find answers to anywhere in the literature (and this is perhaps likely because I don't know the literature and its terms well enough, or because my ideas won't prove out).

I've introduced the Library because it is a useful stand-in for "all possible messages."

Looking at the Library of Babel a few things stand out:

1. The total number of meaningful messages is less than the total number of possible messages. The proof of this is that the same message can be sent using different codes, such that, once transcribed into meaning by the receiver, it is the same message. For an example, we can imagine whole books of English where every letter is simply shifted one space down, A becomes B, Z becomes A, etc.

2. While the maximum Shannon Entropy of a message is the number of possible configurations of the message (so for a three bit code it is 8), we often describe entropy as the amount of surprise in a message. For example, an all lower case text message with only spaces has 27 possible outcomes per character. So the maximum entropy is 27^n where n = the number of characters in said message. If there are limits on what a message can contain, surprise is reduced.

Now let's look at synonyms. Imagine you're getting a message from your doctor as a follow up to an exam. They're very pressed for time so they just have two buttons for sending you a complete message, two possible outcomes. However, the amount of surprise from the message is different if the buttons send either the message "you need to see an opthalmologist" or "you need to see a doctor who specializes in eyes" versus if the two potential messages are "you need to see an opthalmologist" and "you need to see a neurologist." In fact, in the first case, given we take opthalmologist and eye specialist to be perfect synonyms, the surprise is zero, only one possible message can be sent in terms of meaning. Synonymous messages don't provide additional information. Messages can also more or less synonymous obviously.

The above example is fairly odd, but you can also look at binary code. A three digit binary code has 8 outcomes. However, if we assume we code our transmissions with an incredibly feeble encryption method whereby the first bit of our message simply tells us to flip all 1s to 0s and all 0s to 1s if the first digit is a 1, or to ignore the first bit if it is a zero, we actually only have four possible messages. I am well aware that from the mathematical definition of surprise/entropy this still represents 8 potential outcomes, which I will return to, but the general point is that the maximum entropy of meaning for a message can be less than the total number of possible outcomes (i.e. x^n).

In reality, letters in English don't follow a random distribution, so an English message has a far lower surprise (surprise is simply a measure of probability, p for event x). Automatic text generators further reduce entropy, and get more accurate in producing believable text by using words as the unit of analysis, and then group words together in strings in which they are likely to occur. Modern text generators do one better, by generating their text from a sample you give them, so that they mimic the authors' style and content

The issue I see here is that every standard measure of surprise I've seen that looks at letter or word distributions do not take account of synonymity between messages for calculating surprise.

3. Certain information can not reach some receivers because of the way it is encoded. A good example is a series of text messages displayed by a projector. If the projector uses two shades of red that the human eye can't tell apart, one for the background and one for the text, someone watching just sees an unchanging red square. Essentially, the error for our channel is = 1. If you flip the text to a green wavelength, the message gets across fine, although some wavelength pairings that work for most people won't work for people who are colorblind.

This means that functional synonymity can depend on the recipient of a message. This is a different synonymity from two messages referring to identical ideas however.

Now what does this all have to do with complexity? Complexity is a poorly defined concept. Complex systems are systems that cannot be too chaotic or too orderly. They are generally robust (removing one part doesn't destroy the system), adaptive, have emergent properties, and are networked.

The problem is that these definitions aren't particularly firm. There are tons of examples of complex systems that seem to fail some of these test, or non-complex systems that pass them. The brain is a common example of a complex system, but it isn't particularly redundant. Sure, you can remove neurons, but you can also scrape off flakes of metal on a clock gear and still have it work. The fact is, the brain is much more like our uncomplex clock when you start talking about removing functional components, with the most common result being the death of the system when one part is removed.

https://arxiv.org/pdf/1502.03199 this has a good overview of the definition of complexity and its problems.

https://core.ac.uk › pdfPDF
What is a Complex System? - CORE

This poses one possible measure of complexity, statistical complexity. More statistically complex systems are those systems with more patterns, less randomness.

One I really liked was: https://adamilab.msu.edu › Re...PDF
What is complexity? - The Adami Lab.

This looks at the complexity of an organism as a function of how much information it holds about its enviornment.

---

So, my insight, or perhaps confusion, is that the degree of relative synonymity between various possible messages in a system seems to be highly analogous to complexity.

How is this?


To recap:
1. Codes of similar sizes can hold perfectly or roughly synonymous messages.

2. In terms of information producing meaning (or in thinking of information in the physical sciences, causing an effect), the important thing is the entropy of meaning in a message, not necessarily the number of possible messages or the probability of each individual message.

3. The degree of synonymity is determined by the receiver of the information.

The tie in to complexity is that, in complex systems, possible messages have very low rates of synonymity. Information in the form of light waves is far less synonymous for a human eye than it is for a rock.

To use the brain in a bit of a grizzly example, the possible states of my brain if I am sitting in a blank room, thinking, is far lower than the possible states if you throw my brain in a blender. Entropy, as measured by possible states, within the complex system of a functioning brain is orders of magnitude less, because there are far fewer states that represent a functioning brain than there are possible configurations of all the parts that make up the system. Just as coherent passages in the Library are far less common.

However, we have very little ability to measure thought by looking at the brain, or to predict behavior. Meanwhile, if we use our blender for a bit, we will have an extremely effective model for telling us exactly where certain lipids and proteins will eventually settle in the mush we have left. If we vaporize it, we can create an quite accurate model of what will happen to the components despite exponentially higher entropy.

The thing that springs to my mind is that, from an information perspective, this is because the synonymity between various components of the system goes up as the system is destroyed. A single proton's behavior in studying hydrogen gas is replaceable with any other proton. By contrast, the components of a neuron can't be changed much at all without the information it transmits becoming significantly less synonymous.

By way of another example, we can imagine a benzo molecule floating around the blood, bouncing into red blood cells, not doing a whole lot. It doesn't do much when it crosses the blood brain barrier either. It's physical contact with most areas of most cells is mostly synonymous with a plethora of other molecules. But then it bounces into a GABA receptor, and transmits significantly more information.

Synonymity explains how surprise can go up while entropy goes down. It also explains interconnectedness, since changes in key system components are less synonymous in more complex systems than they are in less complex ones. In a random series of characters, any new character is unlikely to change the meaning of a message, whereas in a novel, a plot twist at the end of the book can effect the meaning of all preceding words. The choice of words on the last pages becomes less likely to be synonymous as the complexity of the text before increases (this makes sense given our measure of surprise is context dependant).

It's certainly not like hydrogen molecules in a container don't interact, nor are they not connected. Equalized pressure is an emergent property coming from interactions across the volume of the gas. You also have "tipping points" in phase transitions and fusion, another property of "complexity." The equilibrium is also more robust, existing through a range of temperatures and through variations in gravity, continuing through degrees of variance in both that would eradicate more complex systems, and the system is also highly adaptive to enviornmental changes, reaching new equilibriums quite quickly. And indeed, these systems are considered "complex," just less complex than things like ecosystems and brains. The problem is that, by common definitions of complexity, gas equilibrium should be MORE complex than a brain.

And yet this can't be true for our definition of complexity to have pragmatic meaning, because predicting gas equilibriums is fairly easy, predicting changes in economics seemingly hopeless.

It seems apparent to me that the reason it is easier to model gas dynamics, despite much higher entropy, a larger number of interactions, more adaptability, etc. is because the components share a great deal of synonymity. Information exchange is greater in terms of Shannon Entropy, but lower in terms of meaning entropy. This is what allows you to study a small quantity of hydrogen gas and deduce laws that apply for massive volumes, while studying a single person gets you almost nowhere in determining human social behaviors.

----

Perhaps there isn't too much to this line of thinking, because synonymity isn't easy to model mathematically. After all, there are lots of good definitions of complexity, they just lack rigor.

However, it seemed to stand out to me when I thought about the intersection of information science and complexity.

I'd also love any recommendations in information science that deal with synonymity, I haven't found anything. The closest idea I've found is error, but they really aren't the same thing.