ToMpathy

i. Demo

In an environment with high levels of unpredictability, such as the event horizon of the singularity, I believe that the only way to really prove that you know something others do not is to build something others cannot. Or, alternatively, predict something publicly that no one else can predict - but I did that and no one gives a shit about my predictions.

So I built it.

I'm still refining its accuracy and packaging it for release (soon), but you'll see a gif above gently glancing through the features. If you click the three question marks on each tab, it takes you to one of three essays that walk through the theory.

Those essays are recreated here for posterity.

ii. Current Attention

Hello everyone! Thanks for your “current attention”. Quick note: a human wrote every goddamn word of these 3 tooltips, though I generated the grey-bordered pictures. This is meant to be a short 3-essay explanation of what’s happening on the science side. Longer version here. Off we go:

Art is physical evidence of compressed decision-making. ANY evidence: a painting, spoken words, or a footprint^[1]. We reverse-engineer the decision-making of others both in-person and at a distance through the inference of appreciation, which is part of empathy^[2]. We do this to assess and then copy the previous decisions of others^[3].

But it’s not automatic, this copying. In fact, it’s explicitly driven by the conscious process of attention^[4]. You can read Mein Kampf and not instantly become Hitler, if you're careful enough. To decide whether it’s safe to copy, we have to understand the creator enough to know whether to trust them and incorporate their process^[5]. We have to extract the creator’s core values^[6].

This project is intended to take this process of empathic understanding and supercharge it with AI.

I comprehend the moral hazard of what I have made here. At least I think I do. Extrapolated out, this understanding becomes glasses that can read minds, horrific government overreach, and potentially an even more dangerous hazard than the current one (fun fact: unlabeled AI-generated content is a cognitohazard and will erode away your expertise over time^[7], see the next tooltip).

Despite this, I made it anyway. Not just to see if I could… but because if this doesn’t exist, all of us die anyway^[8]. We NEED to figure out how to give AI the equivalent of human empathy, or else it will never be able to figure out what we want. And if superintelligent AI can’t figure out what we truly, in our heart of hearts, truly want… then all of us die^[9].

Geoffrey Hinton, the godfather of AI, is right. There’s only really one situation on earth in which something much smarter will be controlled by something dumber. He says it a lot: we need to somehow figure out how to make AI love us like a mother^[10].

Well, I’m a cognitive engineer, and I know a design requirement when I see one. This doesn’t get us all of the way there. But it is absolutely a necessary first step^[11]. Empathy is an important early step in mothering^[12].

So this is my attempt to give AI empathy. I apologize for the bad actors that will use this knowledge for evil, but let’s be honest. All that bad shit is already happening^[13].

But I’m also a UX designer, and I know you clicked that question mark looking for an explanation of “current attention”, so here it is: current attention is inferred from deviations from the reader’s personally generated process, derived from expertise and adjusted through inferred context to match the creator’s generated process as closely as possible. That’ll make more sense once we talk about what expertise really is. Which means you’ll have to click the (much more on-topic, I promise) second question mark for the second essay.

iii. Expertise (Previous Attention)

Expertise is a lossy, compressed record of past attention and decisions. It’s also a map of your available actions and their outcomes, given a certain context^[14]. Habit includes expertise, and refers to the lossy, compressed record of all past behaviors^[15]. “Cells that fired together, wired together”^[16],[17]. Because you paid attention in the past to conduct inference on certain decisions, those decisions become easier over time since you can just reuse previous activity^[18]. Repeat this enough times, and the flow state from your expertise can take over large portions of the task.

We want to reverse-engineer something cool we see, to learn how to make it ourselves. To do this, let's start by picturing a helpful pyramid outline made of little boxes on the wall. You have post-it notes to write down guesses. Along the bottom of that pyramid are all of the boxes that repesent the steps, sequentially, that a creator took to physically create an artifact. At the very top was their goal in its creation^[19].

You’re trying to solve a puzzle. Did they use a 3-paragraph format? Fill in a middle box with a post-it. At the end of the day, your goal is to fully fill in the line of post-its on the bottom, and to compress those soon-to-be-stolen steps into a reusable process. The more of each individual little box you cover, the more confident you are.

Expertise lets you instantly fill in a whole block with near 100% confidence. If you can figure out the top block (“what the author was trying to accomplish with this string of activity”), then you have a hint to every subsequent inference^[20]. For the an early look at our "human hacks" the reader can also check internally to see if he or she responds in a particular way and assume that response to be the creator’s goal. You can feel sad and assume that was intentional.

We use our own “generative model” to make the thing ourselves in our heads^[21], and deviations (“mistakes” we identify) demand an explanation^[22]: was that decision a mistake of prediction on the creator’s part? Or are we missing some context, such as a health issue? Was the author gambling on a low-certainty outcome, trying to learn and improve their own expertise map?^[23],[24] Or do we not actually understand the creator’s true goal?

So far, all of this is just simple inference. So why is human empathy special? If empathy is an inference, why is AI just not already doing it better? Why are you trying to “give” empathy to AI at all? Well, for one - it should be better eventually. That’s the point. But for two, it’s not better yet^[25],[26] - because we cheat.

Some interesting human/AI effects are occurring on this pyramid. The largest is that humans cheat through emotion. Emotions are just an expertise, delivered through evolution and largely unchangeable directly^[27]. See a tiger, and you don’t have to work out an inference on how to set adrenaline + heart rate + pupil dilation. You just push the FEAR button on that big control panel, and it all works out^[28] (heads-up, the control panel comes back later, and we're treating Pixar's emotion-characters more like rulebooks that tell you when to push it).

So we have some pre-solved chunks, and there are many ways to use them: put a bunch of our pre-solved emotion blocks near the top (even above the top) and assume they want what you’d want to infer the top block^[29]. We can lock in some of the middle blocks, but we do have the risk of overriding a better explanation^[30] (a less likely risk with the same midbrain, though^[31]). Also, these emotional post-its directly connect to the body's processes: the more you use the red stack of post-its the angrier you'll actually feel.

We can also use mirror neurons and reference frames to find a particularly good fit at the bottom, though their scope is limited. Affective empathy is basically just mirror neurons^[32] (don’t kill me yet please, neuroscientists^[33]). Mirror neurons more or less end up copying other people’s neurons by happenstance, because they fire both when you take an action and when you watch someone else^[34]. We now know they largely target physical muscles you can see - they aren’t magic, they’re the same kind of associative learning as everything else^[35].

But what mirror neurons do is fill in part of the inference for you: someone is less likely to make a fear face if they aren’t pushing the FEAR button on their internal control panel. I personally think the Facial Feedback Hypothesis is enough to get us there on its own after that^[36] (try it out yourself! try making all the muscles in a fear face, and see if you can feel the small spike in your heart rate^[37] - mirror neurons should be able to trigger that too in the simulations you run).

Cognitive empathy is a different clever mental trick^[32], also often called embodied simulation. You imagine yourself making the artifact, but you layer on all the context you can to make your situation more accurate^[29]. You run your own generative model, but you adjust it for as many relevant factors as you can figure out, such as the creator’s available tools.

AI mucks this whole entire process up, and here’s how: the decision to use AI opens up a block beneath it with nearly irreducible complexity that has the appearance of something interesting. It looks interesting because we use the final step of AI training to refine the bottom row - it's called reinforcement learning from human feedback (RLHF)^[38],[39], and it makes the output appear reverse-engineerable and structured^[40]. And it technically is! But we aren't able to cheat this block like the rest, because its processes aren't our processes^[41]. So we fill it in with our best wrong guess, "learning" noise and eroding our skill. And if we don’t realize it’s AI generated?^[42] The fake learning accellerates dramatically, and when this same issue occurs inside AI we get model collapse^[43],[44]. Also, fun fact: we’re rapidly getting worse at detecting AI over time^[45], experts most of all^[46] (AI-writing experts have resistance, though^[47]).

And we muck this process up for AI, because of all this cheating we’re doing. We’re trying to train them to want what we want, but we can’t^[48]. So we use RLHF^[38],[39] to make it look better^[49],[40].

Instead, I’m trying to help AI understand us better. To do this, I’m doing something stupid: I’m trying to make something like the SIFT framework, for empathy. The SIFT framework was an early attempt to help manually teach algorithms to see^[50]. It worked okay for a short while, but raw computational power eventually found better methods for organization. But this - this human empathy? Right now, we need it. So I suspect this is going to be a temporary solution, but it's one we need to put into place until AI learns empathy on its own. A temporary band-aid to help out the AI’s inference while we wait for it to get powerful enough for this not to matter^[51]. It’s too dangerous right now to not need to manually make it a little bit more aligned.

iv. Core Values

Cool, so you’ve inferred the process (or at least you think you have). Remember, we ALSO have to extract the core values (“latent preferences”) of others to decide how much we learn.

First, why do we even bother?

Let’s imagine reading Mein Kampf while not being a nazi. Now, our goal here is to read this book while absorbing Hitler’s rhetorical skill without absorbing any of that nazi stuff. Doing this relies on a simple trick: the actual learning happens at the point of our attention^[52], and by simply not thinking about the nazi reasoning we already have a pretty decent barrier. But a stronger shield is that inferring sufficiently inefficient decisions (“a poor transition mapping”) triggers epistemic disgust, also known as contempt^[53].

Epistemic disgust outright blocks uptake through pushing attention away. Empathy fails to fire, because you aren’t doing inference anymore^[54] (no nerve cells are firing together). You take your current low-confidence best-guess and just roll with it^[55],[56]: “that teen is wearing clothing in a way that shows they just need a dad”, “those poor people just need to get a job”, “those foreigners just have a bad culture.” And then, in turn, others see that inference and may be disgusted by the inefficiency of that, and so on (this also invites the solution to such poor inference: humility in the belief that others may have a viable path to success that you don’t see^[57], expressed as increased willingness to epistemically forage through research or just plain asking^[58]).

But I’m getting ahead of myself. Before that, let’s talk about what emotions are. We already know that they’re a form of ancestral, instinctive expertise. And that is true. But I was being a little bit imprecise there. Let’s explain how emotions work really quick. I really hope you’ve seen Pixar’s Inside Out here.

Believe it or not, the Inside Out series caused a bit of a stir in the scientific world^[59],[60]. Inside Out 2’s “new emotions”^[61] highlight the problem. So much of the early work on emotion centered around the so-called “core emotions,” that to make a movie about new ones appearing stirs up a pretty strong set of disagreements around the work of Lisa Feldman Barrett and her constructed emotions^[62]. Lisa has always maintained that there is no "core," and everything is constructed. That we all have our own separate process for "fear."

But she’s wrong, there are core emotions… sort of. Affective Neuroscience finds 7 channels that we all have^[28],[63]. The problem is definitional nowadays, so let's fix that. We're going to call the core channels “drives” rather than “emotions”. What they really are is the FEAR button on the Pixar Inside Out console I mentioned earlier (the caps were intentional all along!). Ancestral drives are always shown in all caps to differentiate them from emotions, which you craft yourself through repetition. Also, side note - I would love it if any neuroscientists in the house found themselves with a renewed interest in confirming possible other channels^[64],[65].

We have these basic building blocks of emotion, these ancestral “drives” from Affective Neuroscience. And then, because you just have a lot of unconstrained inference machinery in your neocortex, you can associate these drives with each other or stimuli in your environment or verbal concepts or really anything at all^[31]. “Cells that fired together, wired together” becomes, in practice, “anything that can be associated, will.”

And so, from those building blocks we create our own structures. Complex, culturally-dependent concepts^[66] like “honor,”^[67] “schadenfreude,”^[68] or Nordic “Hygge”^[69] come out of this, as does anything we call a “value”. These are, explicitly, solutions to common problems. They are constructed expertise, rather than evolutionary. They are your personal set of mental shortcuts (heuristics) that you use to solve all of life’s problems. So to tie it all together in our Pixar Inside Out metaphor, the emotion is more like having one of the little emotion-people on your shoulder telling you when to push the FEAR button. The little emotion people also whisper to you to staple explanations on the wall or even sometimes push the FEAR button themselves.

But what’s interesting is that this is bidirectional. Not only does it give you reactions and responses, it also distorts your view of what is possible at any given time^[70],[71] - just like all expertise does. Put an engineer in a locked room, and watch her assemble a solution out of her SEEKING and RAGE networks^[72], with a bit of expertise on pipe bombs. An actor in the same room may instead try to tearfully make his case to his captor, assembling his solution out of FEAR and PANIC/GRIEF (by the way, neuroscientists - I will hereafter be referring to PANIC/GRIEF as BELONGING, which makes much more sense on the UI layer. Sorrynotsorry.)

And what’s weird is that this final layer is so powerfully distorting that we trust it over our presumably “actual” goals, like survival or reproduction^[73]… or alternatively, if I can hazard a guess, it’s simply the parameters for the loss function of memetic persistence given to us by evolution^[74]. We’re mesa-optimizers chasing the proxies^[48]. In other words: even when the calculation is obvious, we still don’t generally optimize for survival/reproduction directly (though we can if we choose, like with any value). Examples: suicide, voluntary sterilization, James Dean, sugar^[75].

Alright, moment of truth time, because alignment researchers have been telling me to get to the point for ages. How do we tell what people actually want in the first place? Where does the mythical “latent preference data”^[76] come from? (And not to brag, but it comes bundled with an anti-capture mechanism):

The attention stream has the data^[77]. Aesthetic or naturally rewarding stimuli can pull attention^[78] (whatever data is most rapidly compressing in your field of view will pull your eye, actually^[79],[80]), but that is not the only driver^[81]. The movement of attention shows what the creator wanted, and you can identify attention by extracting deviations from the expected process^[82].

Each act of attention is identified by a decision, and the power of “want” in that decision is determined by the inferred cost-given-context^[83] with a discount rate (from economics)^[84] attached. This gives us a vector, which becomes one of our dots below. And, incidentally, because we do get a little bit of weak dating information (extreme expertise is associated with older decisions^[85]), we can begin to assemble a picture of a person’s wants: their preferred set points^[86],[87].

Each of these preferences (“set-points”) can have an accompanying preferred solution, but the preference is not the solution. That being said, we will be naming the drives in accordance with their preferred solution, as every drive block comes with a preferred problem. So if access to an expected reward is thwarted, RAGE can fire^[63], which can trigger the solution of creativity^[88] through adrenaline: your awareness of the possibilities expands (more weight on that expert transition map), and like the engineer in the example you reach for the tools available to you, such as her pipe bombs. But the engineer’s preference isn’t for pipe bombs, or even for RAGE - it is for the expected reward not to have been thwarted. So now we know how to extract the preference signal.

Heads up, this signal is WEAK AS FUCK… and that’s a good thing. Because we all know that even the tiniest divergence between what we want and what the AI wants is, for a nearly omniscient AI, an instant extinction threat to our species^[9]. This means that the only way to boost that signal to a usable level is to use all of it. All of our data. And it’s not just data density that matters, it’s human-decision density. So, erm, that’s my anti-capture mechanism. A super weak signal (…that isn’t weak to bot-farms^[89]).

But that means it can’t just optimize for one person. It would never be sure enough^[90],[91], and everyone (including that person) dies because it misunderstands a request and takes an action that instantly murders all humans before we can react. Frankly, I’m even nervous about the power as it is - using all of our data is still going to be inaccurate enough to make me very uncomfortable. So yeah. We have to make a utopian future for all of us - a personal, fusion-powered spaceship for every African child. Elon can have one too. Because if we don’t, everyone dies. Sorrynotsorry.

And how do we get there from here? Off the cuff, I think we have the target for a loss function for aligned behavior now. We just need to ensure that we pair it with an appropriately scaled active inference framework^[92] to drive behavior: an optimally aligned AI needs to take clever actions to elicit the most information about human preferences^[93], while balanced against the actions most likely to fulfil them (active inference has the math on this^[24], we have to work out the scalar).

I have a ton more to say about this. Like how this solution also doesn’t lock us into our set points, because higher-order neocortical goals and values are how we grow anyway. Or how through the interaction between possibility distortion (the engineer example) and attention I believe I’ve cracked the puzzle of finding “wanting to want”^[94]. Or how I expect the application of “degree of alignment” to become yet another exponential vector for AI improvement.

But I’ve pushed the limits of what’s possible for an… erm… “tooltip.”

Oh, you came here to ask about the math or science of the backend? Just toss an agent at my open repos: ToMpathy^[95], SoundingLine^[96], and GhostScaleSim^[97]. I’d pay particular attention to the theory documentation in SoundingLine. I’m gonna treat this more like a Wright brothers thing. Y’all can work out the details as they apply to your specific fields.

It’s all hands on deck now, and we all have two goals: either help solve the alignment problem, or help as many people survive to see its resolution as we can. We’re nearing the finish line, but we do have the tools we hoped we would. It’ll be hard work, the next steps, but: