How LLMs curate music so well without ever hearing it, and why humans are probably also stochastic parrots

LLMs are deaf in that they've never heard anything — they're trained exclusively on text data. So even though they've read through tons of album reviews and discussions about music on forums like Reddit, it's fair to say that LLMs have never actually listened to music. And yet they can curate it extremely well with just a few examples of what you like.

Consider a scenario in which I tell you I've been deaf my whole life — literally since birth — but I'm going to try to sell you on my recommendations for songs and music you should listen to (and may have never listened to or heard of before). You would almost definitely not pay me for my recommendations, because I have no idea what music sounds like, let alone what you would want to listen to. Virtually no one would spend money on music recommendations from someone who's literally never heard music before. How could you know what sounds good if you've never heard a single song?

But this is precisely the situation we find ourselves in with LLMs. They are shockingly good at music recommendations (i.e., suggesting artists and songs you might like) if you give them a few examples to help them triangulate your taste.

I guess what I find so interesting about this is not unique to music — after all, Spotify's algorithms (which have actually heard music before, as they are trained on actual audio data in addition to large-scale correlational data between users) can recommend new music to me anytime (although I've found Spotify to be worse than ChatGPT).

What I find fascinating here is the power of intelligence on a very basic level — the power of things like pattern matching and inference, extrapolation, interpolation, and triangulation.

If you're not familiar with interpolation or triangulation, it's worth looking them up. But in short:

• Interpolation estimates unknown values between known data points.
• Triangulation figures out a location based on its relationship to known points.

Both are handy techniques for filling in the blanks with educated guesses.

Thinking, feeling, and doing may be fundamentally uncoupled

The ability of LLMs to recommend music — something that most of us intuitively believe requires some sort of auditory experience, like actually listening to music — might say a lot about the nature of intelligence more generally.

We typically associate intelligence, especially in tasks like music recommendation, with direct sensory input. The assumption here is that understanding and appreciating music requires hearing it (i.e. actually consciously listening to its rhythms, harmonies, and emotional tones). But LLMs challenge this assumption by providing accurate music recommendations without any auditory input. They know what music you might like despite themselves being deaf.

This suggests, to me at least, that intelligence may be less about what's happening on a conscious, experiential level, and more about pattern recognition, inference, prediction, and the like.

After all, attention is all you need.

Perhaps we're not so different after all

If we accept that LLMs can perform tasks like music recommendation without sensory experience, it seems to challenge our conventional beliefs about how human intelligence might function. We tend to think that our intelligence is deeply intertwined with our conscious experiences — our thoughts, emotions, and sensory perceptions. But we now know that much of our cognitive processing happens below the level of conscious awareness. Often, when speaking or writing, I don't consciously choose each word; many words seem to "come to me" without me having actually thought them in advance, or anticipated them in advance. They seem to be summoned by processes I'm not fully privy to.

I woud argue that this unconscious word-summoning is likely driven by the same kind of pattern recognition and statistical inference that powers LLMs. I sometimes use words I don't ever recall having learned or used before, and then I employ them in context in some way that I later find out to be exactly correct. I think this happens to all of us, and I also think it suggests that the brain is constantly conjuring up predictions (i.e. basically generating text!) based on past experiences and stored knowledge, even when we're not consciously aware of it.

If it's true that much of our intelligence operates subconsciously, then the role of consciousness in our cognitive processes may be less central than we usually assume. Consciousness could be more of a bystander — a narrative constructed by the brain to make sense of decisions and actions that have already been made subconsciously. This narrative gives us the illusion of control and intentionality, but it might not accurately reflect the underlying mechanisms of our cognition.

This directly challenges the traditional view that consciousness is a prerequisite for general intelligence. If intelligence is primarily about recognizing patterns and making inferences, then consciousness might be a completely orthogonal feature — an emergent property, not the driving force. Our conscious mind could simply be observing and rationalizing the outcomes of processes that occur without it being directly involved in any way. After all, if this were the case, how could anyone prove otherwise?

Chomsky, deBoer, Chollet, and other stochastic parrots

Now we get to the meaty part: what does this mean for AI? Critics like Noam Chomsky, Freddie deBoer, and François Chollet have argued that LLMs lack true intelligence because — and I'm massively simplifying/reducing their arguments here — there is no place within LLMs where the intelligence can be experienced, or where intelligent thoughts can be held.

Chollet, the most vocal "thinker" on this topic, has said that "intelligence is not just pattern recognition; it's about understanding the world, reasoning, and having a model of reality," while Chomsky argued in a New York Times op-ed that "true intelligence involves the capacity for reasoning and understanding, which machines, by their very nature, lack."

These critiques strike me as obviously false given recent accomplishments of AI. They assume that consciousness is a prerequisite for true intelligence and that without it, AI can only mimic human-like behaviors superficially. But what if human intelligence is itself largely driven by unconscious processes that don't require direct conscious input? Then this assumption might not hold.

Consciousness could be orthogonal to intelligence, meaning that AI systems, including LLMs, could exhibit forms of intelligence similar to human intelligence without necessarily possessing conscious experience.

You are not allowed to be conscious unless I say so

But what if these statistical machines actually are conscious? How would we know? Despite the fact that many people make bombastic claims to the contrary, consciousness is still one of the most profound mysteries in science and philosophy. (Full stop.) We know it's tied to brain activity, but we don't fully understand how or why it arises, or how to verify its existence in anyone besides ourselves.

So as long as we fundamentally don't understand how or why human consciousness arises, and given our long history of prematurely declaring that most non-human animals were not conscious, I think we should epistemically be very humble about whether LLMs might be experiencing the world in some way, either now or in the future. (Personally, I suspect that if they will be conscious in the future, they probably already are now — albeit possibly in some much weirder way than we can imagine or comprehend.)

The key point here is that we don't yet know enough about consciousness, on a fundamental level (i.e. why it's there at all as an epiphenomenon), to categorically state that LLMs can't have some sort of inner experience. And we've historically said "this is uniquely human" and "that can never be done by machines" and then been wrong so many times now that many people don't even find articles on the topic to be interesting.

It strikes me as entirely unlikely that humans happened to evolve at the exact evolutionary point where consciousness or certain rare forms of intelligence emerge. It seems far more likely that we're not so special after all.

If human intelligence relies heavily on subconscious pattern recognition and less on conscious deliberation than we usually believe, then AI might be capable of similar forms of intelligence, even without conscious experiences. This could mean that the divide between human and machine intelligence is narrower than we've traditionally thought, and that AI might not need to replicate human consciousness to achieve levels of intelligence and creativity that are super-impactful, or super-dangerous, or super-useful.

LLMs may eventually prove to be mirrors into humanity that we systematically avoid looking into

The actual capabilities demonstrated by today's LLMs already challenge our understanding of human and machine intelligence, as well as any categorical divide between the two. They suggest that intelligence might not require sensory experience or consciousness at all, but may rather be rooted in a raw ability to process information and recognize patterns.

We should be open to the possibility that intelligence and consciousness may be more complex and varied than we currently understand, and that AI systems (including LLMs) might operate in ways that run completely counter to our anthropocentric ideas about the uniqueness of our intelligence and/or consciousness. But by broadening our views of these phenomena, we may discover that AI has the potential to surprise us in ways we've yet to imagine — not just in terms of what it might do, but in terms of what it may reveal about us.