For month 2 of my residency, I’m looking at how we interact with AIs online. I’m zooming in on this approach: what if an AI was like working with a teammate?
ℹ️ This is one of a series of notes from my summer 2023 invention residency with PartyKit. See all research notes here.
I want to investigate what it feels like to have nonhuman colleagues. And, although I could sketch that on paper, how would you build this kind of system? — perhaps there are lessons from the building that would inform the design, too.
Once there’s you plus one other (human or AI) then you’re in a multiplayer situation. So PartyKit is the ideal platform to prototype with.
These are my first sketches (all working code!) and I went pretty playful...
Summoning NPC teammates
How do we collaborate with AIs? What if they were just like our human colleagues, cursors and all? Well not exactly human
I’ve been sketching NPCs-as-dolphins — you build a little pool, summon them, and that’s where they hang out.
(NPCs = non-player characters, a term from video games)
Different dolphins do different things! Just like you work with different folks over a project, you'd pick diff NPCs for the task at hand
This one writes haikus (as a quick demo)
It’s an real, coded-up virtual user btw: the infinite canvas makes use of the open source tldraw and I’m using PartyKit to (a) host the tldraw collaboration back-end, and (b) connect to that same back-end as a legit user with presence and the ability to monitor/affect the document state.
A design advantage of cursors is you can play with DISTANCE to show ATTENTION
AI chatbots are tough. They don’t have affordances, you don’t know what to do
But this dolphin (who loves painting stars) can proactively offer its help whenever it sees an opportunity! Better
From building the proto, it’s pretty clear that apps will need serious re-engineering to become natively multiplayer (once you have user + NPC, you’re already multiplayer). Like, do you need a server-side machine API? What else?
Oh yeah, dolphins...
Dolphins! My go-to nonhuman companion species. So smart but very much not us, right? Like AIs obv.
The triangle and circle “embassy” shape I used is a schematic of the original Dolphin Embassy by architects Ant Farm (1974). it's amazing work, a barge floating on the ocean, containing a cetacean pool/portal, a perfect meeting point 🐬
For completeness, my initial sketch (tap to view)
I make software protos for one of three reasons: because I want to illustrate and bottom out an interaction; because I want to learn a new technique; because I can‘t stop laughing every time I think of it.
Sometimes all three? This is my first sketch of what I wanted to build with NPCs. What came out wasn't exactly what I expected - that’s the point! - but the dolphin metaphor gave me a frame and a goal, and I’ve now got decent foundations to build on. The “serious” applications of this pattern can come later.