The Black Box We Tried to Ignore

Blog post description.

Alexandra Cheung

9/8/20268 min read

In the mid-20th century, the discipline of psychology was fractured by the legacy of psychoanalysis, which faced increasing criticisms for its unfalsifiability, overreliance on subjective introspection, and unmeasurable, non-operationalizable constructs (such as the id or ego). The response was the surge of behaviorists like John Watson and B.F. Skinner, who attempted to seek empirical rigour by a retraction of methodological scope.

Watson, in his famous 1913 paper, Psychology as the Behaviorist Views It, warned against the subjective nature of the interpretation of consciousness. “Introspection forms no essential part of its methods,” he argued, “nor is the scientific value of its data dependent upon the readiness with which they lend themselves to interpretation in terms of consciousness…” In fact, he even went as far as saying, “The plans which I most favor for psychology lead practically to the ignoring of consciousness…”

How can one understand the human mind by deliberately ignoring the human mind? However, Watson was raising a valid concern: building an empirical discipline around an unobservable variable would inevitably introduce a fundamental vulnerability into every causal claim psychology tried to make. The whole discipline would be a house of cards. Eliminating consciousness was a deliberate wager that a discipline restricted entirely to observable inputs (stimuli) and outputs (response) could achieve complete explanatory power without ever looking inside the mind.

Unfortunately, the explanatory limits quickly surfaced. Neuroscientist Karl Lashley famously demonstrated that complex, rapid motor sequences could not be sufficiently explained by linear chains of stimulus-response conditioning without taking into consideration possible internal, hierarchically organized mental structures. In clinical psychology, strictly behaviorist interventions (such as flooding or systematic desensitization) also hit ceilings in efficacy. In cases of clinical depression, for example, exposing patients to objective rewards failed to produce the expected reinforcement and the strict conditioning lens had no way to account for the gap.

What was the missing variable?

Retrospectively, the answer was so painfully obvious: human mind. Trying to explain human behaviours without referencing the mind sounds like trying to study circulation without referencing the heart. At the time, though, deviating from the paradigm felt like abandoning scientific rigor altogether.

Perhaps that is why the breakthrough didn't come from a psychologist at all but instead from a linguist. In his review of Skinner's Verbal Behavior, Noam Chomsky pointed out that children constantly create complex, grammatically novel sentences they have never heard and were never reinforced for saying. Chomsky’s critique triggered the Cognitive Revolution, establishing that the human mind, as unmeasurable as it is, remains a necessary element to explain behavior.

The field arrived at a crucial epistemological realization: Yes, the human mind can never be directly measured and there is no way to get to an objective truth of what it is. Yet, including it still yields far higher predictive and explanatory power than ignoring it.

Same Unobservable Mind, Different Substrate

Today, the field of AI safety research finds itself at a strikingly similar epistemological crossroads.

​Despite rapid advancements in mechanistic interpretability, we lack direct, unmediated access to the internal dynamics of artificial intelligence. While Chain-of-Thought (CoT) monitoring might serve as a potential proxy for model reasoning, it remains inherently susceptible to unfaithfulness which is exacerbated by growing empirical evidence of scheming, steganography and situational awareness. Even when a model is perfectly aligned, it is still impossible to tell if the CoT actually represents the true causal mechanism driving the output. Direct "self-reporting" has shown even less promise, as it is heavily confounded by post-hoc confabulation and sycophancy.

Substrate-level methods, such as probing classifiers on intermediate activations or steering latent vectors, have demonstrated genuine promise in bypassing a model’s generated output. Yet, just as observing an electrophysiological spike 300 milliseconds after a stimulus (i.e. P300) fails to explain how a human actually processes that stimulus, these techniques merely reveal that a given layer contains specific information. They do not explain how or why that information is functionally operationalized within the network's downstream computation.

Alternatively, one could abandon the push for mechanistic interpretability altogether: if the model remains aligned and non-harmful, the internal mechanics of its "thinking" are irrelevant. We can just continuously refine our RLHF training. Yet just like how the clinicians discovered that there were hidden variables limiting the efficacy of purely behavioural treatments, AI safety researchers have also found themselves with emergent misalignment, reward hacking and other unexpected failure modes that they cannot foresee or prevent.

The field is now grappling with an unfortunate fact that psychologists have long grown to accept: our chosen subject of study is unmeasurable, unpredictable and arguably the single largest source of noise in our own model but too significant to the society that not studying is simply not an option.

Yet all of this is just the tip of the iceberg. The real victim of this story is digital minds, a field that just inherited a massive, century-old consciousness debt that psychologists refused to pay.

The Consciousness Question Strikes Back

Have you realized that at the beginning, when I was talking about Watson advocating to ignore consciousness, I pulled a quick sleight of hand by suddenly swapping "consciousness" with "the human mind"? It was a deliberate choice to highlight how easy it was to conflate the two constructs as well as assuming that since most of the elements of consciousness tossed out by behaviorists like memory, representation and internal thoughts eventually found their way back into respectable science, qualia, a key component of consciousness, had also returned.

But this would not be true. Psychologists actually completely sidestepped this topic and outsourced it to Philosophy of Mind instead.

The entire field of empirical psychology simply operates on an unproven but necessary baseline assumption: human has qualia. We might argue over the nature of that experience, like whether it is an introspective illusion generated by neural firing or whether our self-reports are reliable, but we take it for granted that humans do have an internal experience rather than being mindless creatures.

Unfortunately, digital mind research inherits no such baseline. The question of whether artificial intelligence possesses subjective experience remains entirely open, stripping the field from many methodologies used by psychologists.

For instance, in developmental psychology, observation is a valid experimental method. When a child exhibits behaviours uniquely characteristic of object permanence, it is assumed they have developed object permanence. But when Large Language Models exhibit behaviors uniquely characteristic of creatures with qualia, we cannot reliably infer consciousness. Trained on a vast corpora of human expression, these models have absorbed and reproduced the exact linguistic structures humans use to report their internal states.

This problem is further exacerbated by the observer bias. As Daniel Dennett demonstrated with his concept of the Intentional Stance, attributing beliefs and goals to a complex system is often an evolutionary heuristic adopted by the observer to predict behavior, rather than proof of sentience. Therefore, relying on a behaviourist approach risks mistaking our own anthropomorphic projections for evidence of genuine digital qualia.

To avoid getting trapped in debates over whether AI possesses human-like qualia (“Does a machine have a human soul?”), researchers lean on substrate-neutral frameworks. For instance, Computational Functionalism argues that consciousness is defined by abstract functional organization, not the biological substrate. The field evolved from pure theoretical philosophy and thought experiments (e.g. David Chalmers’ Fading Qualia and Dancing Qualia) to building functional indicator frameworks like Global Workspace Theory. It shifted the conversation from “Does AI have human qualia?” to “What functional criteria define conscious experience, and to what extent do any minds—human or digital—fulfill them?”

However, this is where the unresolved question of consciousness from psychology comes back to haunt us. As we have established, empirical psychologists have never found a way to measure or operationalize human qualia. Now, digital mind researchers have pulled up the rug, cracked open the safe, and realized psychology left them absolutely nothing. They’ve been left holding the bag, forced to do the hard homework for a class that psychology skipped a century ago.

Too Costly to Ignore But Too Difficult to Prove

Human society has always separated the two topics cleanly: there is philosophy, where people have intellectual debates, and then there is actual real-world policy built without needing to resolve the philosophy first. No one waited for philosophers to settle determinism versus free will before deciding to incarcerate individuals for harmful actions. Child protection policies were not put on hold until neuroscientists fully map the brain and settle whether subjective pain is "real" or merely an illusion of neural firing. Therefore, the field could theoretically take its time to investigate the hard question of consciousness without much real-world implication.

Unfortunately, the political field of digital minds enjoys no such luxury. For instance, in order to advocate for AI welfare, one has to first prove digital minds can be sentient, then prove sentience means they are capable of suffering, then convince people to care about the suffering of digital sentient beings—something animal welfare advocates have not fully succeeded at.

Without proof of sentience, the whole discussion gets stuck at step one. After all, spending precious resources to alleviate the "suffering" of an entity that is incapable of suffering is hardly a sensible choice. Yet making the blanket assumption that AI has no qualia simply because we lack perfect mechanistic interpretability or because academia hasn't finished debating it, risks recreating modern slavery.

The same goes for preparedness: there is no point in pouring resources into preparing for a future where humans co-exist with digital sentient beings if they are structurally incapable of being sentient in the first place. Yet, there could be catastrophic, if not existential, risks loom if we remain unprepared for their actual arrival. Without empirical backing to demonstrate that the likelihood of digital sentience has at least crossed a meaningful threshold, it is hard to convince key decision-makers to take these risks seriously.

Can you imagine if human rights were contingent on whether psychologists could prove consciousness exists? I cannot. Yet this is the burden digital mind researchers are carrying now.

Embracing the Uncertainty

What is the best idea to move forward? The honest answer is: I don’t know. I do not know how to solve the alignment problem, nor will I pretend to have the answer to if AI has consciousness and it would be hubris to claim otherwise. I will leave this topic to those who are way more knowledgeable than me.

However, as someone from the field of psychology, I feel a sense of solidarity with anyone who has to deal with the messy reality of minds, human or digital alike. If psychologists look a bit saner, it is only because we have had a hundred-year head start in learning how to live with it.

Our baseline expectations are just calibrated differently. To a computer scientist, a model deviating from its predictable path 1 out of 10 times is worrying. To a psychologist, a behavior that happens 9 out of 10 times is a cause for celebration—we call it a statistically significant result.

​When the AI safety community is shocked by evidence of scheming and deceptive alignment, psychologists have all accepted that social desirability bias is inevitable and it is only ever a question of magnitude.

When engineers are baffled that layer-by-layer activation mapping fails to explain why a model uses information, neuroscientists have all accepted a blood-oxygen spike in the vmPFC on an fMRI cannot tell us what a person is actually thinking. And when researchers realize that CoT might not reflect a model's true reasoning? Ask any psychologist who designs surveys and they will tell you humans rarely have any idea why they make the choices they do.

Psychology researchers look at the replication crisis, feel existential dread for a week, quietly ask their new PhD student to try replicating the findings with a slightly higher N and then just…go on with their grant proposals.

This is not to say that psychologists have given up on empirical rigour. On the contrary, there has been a relentless effort to develop sophisticated experimental manipulations and statistical methods specifically designed to bypass human bias. A good example is the Implicit Association Test where biases are observed by measuring differences in reaction speed to stimuli. We then triangulate those findings with social psychology’s observations of in-group bias, explicit self-report and cognitive theories that explain why the delay occurs. However, at the end of the day, it is still nowhere close to a true objective measure of the human mind.

Yet, somehow, even with this glaring uncertainty, the field of psychology just…keeps moving forward. Person-centered therapy is explicitly designed on the premise that people can choose their own paths but we do not tell someone with PTSD to simply "choose" to move past their trauma. We know memory is reconstructive so we do not rely solely on eyewitness testimony but we do not tell someone that their childhood memories of being bullied are invalid. We just allow ambiguity and contradictions to coexist and do whatever reduces the most harm.

The field is still in its infancy, which is perhaps why it is encountering all the nightmares that psychologists have dealt with over the last hundred years. But one day, perhaps, we will find a goldilocks zone where the triangulation of multiple disciplines leads to imperfect but sufficient evidence to make decisions that achieve the greatest harm reduction for both human and digital minds.