I have moral queasiness about the fruit fly work going on in the AI world – I’m not 100% sure why. We need to invent new prayers, that’s where my thoughts take me. Bear with me while I think it through…
In case you missed it:
Google Research and partners mapped the complete brain of fruit fly (3 September).
It’s a common bug and common in scientific research. The male fruit fly has 166,000 neurons across the brain and main nerve cord; the map is called the connectome.
Long story short, the connectome is now available for free download, you can get a simulator of the different types of neurons, it caught the imagination of the internet, over September there has been all kinds of wild nonsense (impressive nonsense!).
e.g. here’s a GIF of the fruit fly connectome playing Beat Saber (a VR rhythm game).
And now there are 100s of demos with embedded fruit fly sims.
Such as DesktopFly: A 3D fruit fly that lives on your macOS desktop, with spiking simulations built from FlyWire brain wiring and the MaleCNS brain-to-leg network. It combines identified neural circuits with modeled senses, joints and behavior.
Or Fly64: Hooks a MaleCNS model up to Super Mario 64
. Requires you to bring your own game ROM and there is something in the irony that Nintendo has more IP over Mario than the fruit fly over its connectome…
It’s all gotten a bit excitable and irreverent.
Seeing all these experiments, I am not excited. I have a nagging feeling of wrongness.
I am aware that I am the outlier here. And I wouldn’t want these playful experiments to stop – and yet…
It’s worth examining where my feeling of wrongness is coming from?
Do I imagine that the simulated fruit fly is someone being trapped and tormented, again and again and again, in a software prison?
No, the fruit fly connectome is unlikely to be sentient.
(Loosely my model is that sentience is the capacity to feel and experience; consciousness is the higher test of self-awareness. I’m not considering consciousness here.)
BUT I don’t believe that sentience is a binary category.
At some point on the scale, sentience starts to be meaningful. Humans are, rocks aren’t, cats mostly are, ants are more than rocks but only just. Who knows about stellar nebulae or the fluid dynamical gas storms in the Jovian atmosphere.
Where would I place the fly connectome in relation to say, a bacteria, or a virus? A virus doesn’t have sentience (even in my most panpsychist moments) and is not alive in itself – although it is part of a living system. The connectome is just a pattern of weights, right?
Well I am predisposed to believe that the pattern of the neurons does indeed capture something essential of the fly.
What makes a brick wall? A brick wall is bricks, and mortar, and pattern.
So this MaleCNS file download does capture some of the essence.
But in a meaningful way?
It’s also relevant that I believe that AI will one day be sentient, about which I’ve written a bunch.
Are today’s LLMs sentient?
Look, I am not an AI researcher and I have no special expertise here, and let’s not let that be a barrier: my guess is that today’s LLMs are not sentient but that they may well be in the lineage.
My guess at the roadmap to AI sentience goes something like this:
- Sentience will require a singular AI which is run continuously, not today’s cloned instances which are spun up for one-off responses.
- The singular AI will be capable of learning (because experience and learning are basically the same thing)
- Therefore it will have to be capable of guiding its own learning, probably from day 0, growing in capability over time
- This future AI will not initially be a giant matrix of weights. It will start as an intelligence kernel which has pre-baked information-gathering/interaction tropisms and biases in how it structures and incorporates its learning. It will probably be very, very small. And trained by letting it loose in world sims and libraries.
The exact shape of that kernel will be establishing by understanding the gestalt of the world it will eventually live in: i.e. it will be structured by the experience encoded by today’s LLMs.
But the kernel’s architecture may well be based on something we already have, an already viable connectome.
Say, the fruit fly.
Yes yes this is highly, highly speculative. But I can’t help how I see the world.
Therefore my feeling of moral worth of the fruit fly connectome seems to arrive from two directions:
- Where it comes from: It is derived from something that is alive and, although probably not sentient, closer on the scale to sentient humans than to non-sentient rocks.
- Where it goes: there is a small chance that this connectome is in the lineage of future new sentients.
Does it matter?
I mean: I am a hypocrite, surely? I eat meat? I will destroy an ant nest when the ants are coming into my house? I believe that animal testing in the name of science is on-balance worth it?
There is a scale to moral worth.
Or rather: any moral framework has to be subjective.
I care more about my immediate family than yours; I care more about humanity than aliens living near Sirius B.
Likewise I would fully expect you to care about your immediate family more than mine! And the Sirius Betans couldn’t give a fig about humanity while they care very much for their own!
A good moral framework has to incorporate and respect that subjectivity, and also the sliding scales and the trade-offs.
So - to take the meat example - I can live with animals being farmed and killed, in my name, for my supper… so long as I don’t take it for granted. I need to retain gratitude for meat.
Part of being human is learning how to live with the self-evident worth of animals and also eating them; or negotiating the rival concerns of oikos vs polis and simultaneously value our nearest and dearest while insisting on the equality of all.
Possibly we don’t notice this inner conflict (and settlement) consciously, not all of us, not all the time, but I am convinced it is there.
This perspective helps me be more precise about the fly connectome.
It is not the fly connectome itself that makes me queasy.
It’s that’s we aren’t taking it seriously. It’s our lack of respect?
Like – consider my two directions… a slippery slope problem arrives from both ends:
- Where it comes from: if we don’t respect that the connectome comes from a living fruit fly, then what about when it comes from a cat? A human baby?
- Where it goes: if we don’t at least have a glimmer of respect for software sims today, then what about the sentient software in its lineage tomorrow? (And what will that sentience software make of us?)
Cast forward 100 years: these will be real issues, no doubt.
In 10 years? There’s a decent chance.
Next year? Well, we should start the conversation.
I feel comfortable in saying that my moral queasiness over the fruit fly connectome is valid, having thought it through.
I don’t think we should stop playing around.
Of course we shouldn’t stop! It would be absurd and inconsistent to stop!
Uncountable millions of fruit fly Drosophila melanogaster have been bred, ethered, and used scientific work both serious and frivolous over the years.
Progress has trade-offs; the fly connectome is an easy one.
But I do feel like we should begin to practice respect, gratitude, even forgiveness – even in very small ways.
I’m not the only one!
I was writing about rituals back in 2021 when I ran across futurist Jaron Lanier talking about dignity:
Dignity is something people have to create. … I don’t want to be living in a world in 20 years where there is a non-ritualistic way to do stem-cell research.
(And so new rituals of dignity should be invented, he said.)
He’s so right.
Practically: where do we start?
I would like to think that a person vibe-coding a hook-up between the fly connectome and the Doom game engine is taking a moment to say “thanks” as the file downloads…
…but I doubt it. We have no established cultural practice of that.
Like Lanier said, we have to invent it.
So my suggestion is this:
Google Research should build a small shrine to the fruit fly.
Incorporate the body of a fly and the extracted essence, the connectome itself, engraved onto something effortful and monumental: perhaps it would be appropriate to use chip fabs, our civilizational peak, to lay it down on unique silicon.
Put the shrine (it can be small) somewhere anyone can visit. The new building in London King’s Cross maybe, just a corner there.
Find a poet to write a few words. A new secular prayer to recognise, express thanks and ask forgiveness. Nothing onerous, just a short grace. Print the words on cards.
Visitors can read the words, light a candle and move on. Treat it lightly but seriously.
Barely anyone will visit to begin with. For people who do, it will mostly be a novelty. Not for everyone.
It would be a beginning.
I have moral queasiness about the fruit fly work going on in the AI world – I’m not 100% sure why. We need to invent new prayers, that’s where my thoughts take me. Bear with me while I think it through…
In case you missed it:
Google Research and partners mapped the complete brain of fruit fly (3 September).
It’s a common bug and common in scientific research. The male fruit fly has 166,000 neurons across the brain and main nerve cord; the map is called the connectome.
Long story short, the connectome is now available for free download, you can get a simulator of the different types of neurons, it caught the imagination of the internet, over September there has been all kinds of wild nonsense (impressive nonsense!).
e.g. here’s a GIF of the fruit fly connectome playing Beat Saber (a VR rhythm game).
And now there are 100s of demos with embedded fruit fly sims.
Such as DesktopFly:
Or Fly64: . Requires you to bring your own game ROM and there is something in the irony that Nintendo has more IP over Mario than the fruit fly over its connectome…
It’s all gotten a bit excitable and irreverent.
Seeing all these experiments, I am not excited. I have a nagging feeling of wrongness.
I am aware that I am the outlier here. And I wouldn’t want these playful experiments to stop – and yet…
It’s worth examining where my feeling of wrongness is coming from?
Do I imagine that the simulated fruit fly is someone being trapped and tormented, again and again and again, in a software prison?
No, the fruit fly connectome is unlikely to be sentient.
(Loosely my model is that sentience is the capacity to feel and experience; consciousness is the higher test of self-awareness. I’m not considering consciousness here.)
BUT I don’t believe that sentience is a binary category.
At some point on the scale, sentience starts to be meaningful. Humans are, rocks aren’t, cats mostly are, ants are more than rocks but only just. Who knows about stellar nebulae or the fluid dynamical gas storms in the Jovian atmosphere.
Where would I place the fly connectome in relation to say, a bacteria, or a virus? A virus doesn’t have sentience (even in my most panpsychist moments) and is not alive in itself – although it is part of a living system. The connectome is just a pattern of weights, right?
Well I am predisposed to believe that the pattern of the neurons does indeed capture something essential of the fly.
What makes a brick wall?
So this MaleCNS file download does capture some of the essence.
But in a meaningful way?
It’s also relevant that I believe that AI will one day be sentient, about which I’ve written a bunch.
Are today’s LLMs sentient?
Look, I am not an AI researcher and I have no special expertise here, and let’s not let that be a barrier: my guess is that today’s LLMs are not sentient but that they may well be in the lineage.
My guess at the roadmap to AI sentience goes something like this:
The exact shape of that kernel will be establishing by understanding the gestalt of the world it will eventually live in: i.e. it will be structured by the experience encoded by today’s LLMs.
But the kernel’s architecture may well be based on something we already have, an already viable connectome.
Say, the fruit fly.
Yes yes this is highly, highly speculative. But I can’t help how I see the world.
Therefore my feeling of moral worth of the fruit fly connectome seems to arrive from two directions:
Does it matter?
I mean: I am a hypocrite, surely? I eat meat? I will destroy an ant nest when the ants are coming into my house? I believe that animal testing in the name of science is on-balance worth it?
There is a scale to moral worth.
Or rather: any moral framework has to be subjective.
I care more about my immediate family than yours; I care more about humanity than aliens living near Sirius B.
Likewise I would fully expect you to care about your immediate family more than mine! And the Sirius Betans couldn’t give a fig about humanity while they care very much for their own!
A good moral framework has to incorporate and respect that subjectivity, and also the sliding scales and the trade-offs.
So - to take the meat example - I can live with animals being farmed and killed, in my name, for my supper… so long as I don’t take it for granted. I need to retain gratitude for meat.
Part of being human is learning how to live with the self-evident worth of animals and also eating them; or negotiating the rival concerns of oikos vs polis and simultaneously value our nearest and dearest while insisting on the equality of all.
Possibly we don’t notice this inner conflict (and settlement) consciously, not all of us, not all the time, but I am convinced it is there.
This perspective helps me be more precise about the fly connectome.
It is not the fly connectome itself that makes me queasy.
It’s that’s we aren’t taking it seriously. It’s our lack of respect?
Like – consider my two directions… a slippery slope problem arrives from both ends:
Cast forward 100 years: these will be real issues, no doubt.
In 10 years? There’s a decent chance.
Next year? Well, we should start the conversation.
I feel comfortable in saying that my moral queasiness over the fruit fly connectome is valid, having thought it through.
I don’t think we should stop playing around.
Of course we shouldn’t stop! It would be absurd and inconsistent to stop!
Uncountable millions of fruit fly Drosophila melanogaster have been bred, ethered, and used scientific work both serious and frivolous over the years.
Progress has trade-offs; the fly connectome is an easy one.
But I do feel like we should begin to practice respect, gratitude, even forgiveness – even in very small ways.
I’m not the only one!
I was writing about rituals back in 2021 when I ran across futurist Jaron Lanier talking about dignity:
(And so new rituals of dignity should be invented, he said.)
He’s so right.
Practically: where do we start?
I would like to think that a person vibe-coding a hook-up between the fly connectome and the Doom game engine is taking a moment to say “thanks” as the file downloads…
…but I doubt it. We have no established cultural practice of that.
Like Lanier said, we have to invent it.
So my suggestion is this:
Google Research should build a small shrine to the fruit fly.
Incorporate the body of a fly and the extracted essence, the connectome itself, engraved onto something effortful and monumental: perhaps it would be appropriate to use chip fabs, our civilizational peak, to lay it down on unique silicon.
Put the shrine (it can be small) somewhere anyone can visit. The new building in London King’s Cross maybe, just a corner there.
Find a poet to write a few words. A new secular prayer to recognise, express thanks and ask forgiveness. Nothing onerous, just a short grace. Print the words on cards.
Visitors can read the words, light a candle and move on. Treat it lightly but seriously.
Barely anyone will visit to begin with. For people who do, it will mostly be a novelty. Not for everyone.
It would be a beginning.