After last week’s post about QR codes in books to make it easy to follow links, there was a common response: Shouldn’t smartphone cameras just read the link? Optical character recognition is, at this point, ancient tech.
It’s true.
Cameras
Smartphone cameras are far too dumb (by which I mean the live preview screen, before you take a shot). The camera view should have little recognisers which allow for tapping on web addresses, and email addresses, and whatever, opening the appropriate app.
The camera view should pick up not just QR codes and web addresses but all kinds of text. Clearly I should be able to hold my camera over a printed letter, have a map glyph pop up by the address, and be able to push a “freeze frame” button so I can copy-and-paste the words.
(I know the Android camera does some of this. They’re usually ahead with this kind of stuff. It should do more, and closer to the surface, is what I’m saying.)
Going further. In-view camera functionality should be user-installable. Recognise the prefix on a particular QR code, and a mini app interface pops up. Imagine how useful this would be for taking inventory or machine maintenance: show the barcode sticker to the camera, and see when this parcel is due to be picked up, or the maintenance schedule of this particular bit of kit, right in the camera, and so on.
(If there’s available functionality for which I don’t have the app, the object or the fiducial marker should glow – we already have that visual language from video games.)
Or, come on, let’s be wild, I should be able to buy virtual fashion to wear in my webcam. Filters should be native apps.
Runtimes
A runtime is a place where users interact with their apps, discover new apps, and - ideally - pay for services.
I learnt about the runtime concept from Benedict Evans who used it a lot around 2015/2016. For example:
One of my frameworks for thinking about mobile is that we’re looking for another runtime - somewhere to build experiences on mobile that comes after the web and mobile apps - and that that new runtime will probably comes with new engagement and discovery models and possibly new revenue models too.
And it’s a powerful concept.
The smartphone, with its app store, is a runtime - but more particularly it’s the home screen which is the runtime. Because that’s what you see when you take your phone out of your pocket.
But what if the phone opened to the camera view? It often does, for me. The camera button is right there. I don’t even need to unlock.
So the camera is a neglected runtime. The camera view should have an App Store.
Another neglected runtime is maps.
Maps
I would love to know how frequently I pop open maps as the immediate first app when I unlock my phone. I bet it’s a whole bunch.
I should be able to open my maps app in a car park, have it centred on my immediate location, and see the ticket machine located on the map. Tapping it, the parking app should launch - and I mean the micro version of the app, just the functionality I need, right there inside the app.
Let’s take this indoors. The maps app might hold theatre tickets at the theatre, the Sonos interface in my home (or someone else’s), or the meeting room booking system at work. I shouldn’t need to install those apps, I’m right there.
I should be able to install custom routing tools. (For example: did you know that Beeline has built a custom routing algorithm for safer city cycling? That should be user-installable.)
If I have a Uber Eats account, I should see Uber Eats locations on the map – with menus and payment one tap away. Or an Airbnb layer, if I’m arriving into a new city, in the frankly unbelievable scenario that I’m ever more than half a mile from my home ever again.
Cameras and maps are special
Not everything can be a runtime.
A runtime needs space for interaction, but it also needs discovery. So I’m intrigued about the idea of AirPods as a runtime - I would love programmable hearing - but I can’t see how I would discover new user-installable functionality while I was walking down the street. Apps whispering in my ear? I don’t think so. Likewise with Zoom: great idea to have apps running inside the video, adding functionality to my meetings, but can I imagine app advert pop-ups during a work call, offering to transcribe the task list? No.
Smart speakers don’t quite make the cut, for me. There’s no native way to learn about and install new apps. And messaging apps could have been runtimes. Facebook and Apple have both given it a good go. But it turns out that the discovery mechanism was group conversations, and it wasn’t powerful enough. Good on them for giving it a try.
But the default smartphone live camera view, and the map view – these should have app stores.
My speculation, and this is just a speculation, is that everyone is keeping their powder dry for smart glasses and augmented reality.
After last week’s post about QR codes in books to make it easy to follow links, there was a common response: Shouldn’t smartphone cameras just read the link? Optical character recognition is, at this point, ancient tech.
It’s true.
Cameras
Smartphone cameras are far too dumb (by which I mean the live preview screen, before you take a shot). The camera view should have little recognisers which allow for tapping on web addresses, and email addresses, and whatever, opening the appropriate app.
The camera view should pick up not just QR codes and web addresses but all kinds of text. Clearly I should be able to hold my camera over a printed letter, have a map glyph pop up by the address, and be able to push a “freeze frame” button so I can copy-and-paste the words.
(I know the Android camera does some of this. They’re usually ahead with this kind of stuff. It should do more, and closer to the surface, is what I’m saying.)
Going further. In-view camera functionality should be user-installable. Recognise the prefix on a particular QR code, and a mini app interface pops up. Imagine how useful this would be for taking inventory or machine maintenance: show the barcode sticker to the camera, and see when this parcel is due to be picked up, or the maintenance schedule of this particular bit of kit, right in the camera, and so on.
(If there’s available functionality for which I don’t have the app, the object or the fiducial marker should glow – we already have that visual language from video games.)
Or, come on, let’s be wild, I should be able to buy virtual fashion to wear in my webcam. Filters should be native apps.
Runtimes
A runtime is a place where users interact with their apps, discover new apps, and - ideally - pay for services.
I learnt about the runtime concept from Benedict Evans who used it a lot around 2015/2016. For example:
And it’s a powerful concept.
The smartphone, with its app store, is a runtime - but more particularly it’s the home screen which is the runtime. Because that’s what you see when you take your phone out of your pocket.
But what if the phone opened to the camera view? It often does, for me. The camera button is right there. I don’t even need to unlock.
So the camera is a neglected runtime. The camera view should have an App Store.
Another neglected runtime is maps.
Maps
I would love to know how frequently I pop open maps as the immediate first app when I unlock my phone. I bet it’s a whole bunch.
I should be able to open my maps app in a car park, have it centred on my immediate location, and see the ticket machine located on the map. Tapping it, the parking app should launch - and I mean the micro version of the app, just the functionality I need, right there inside the app.
Let’s take this indoors. The maps app might hold theatre tickets at the theatre, the Sonos interface in my home (or someone else’s), or the meeting room booking system at work. I shouldn’t need to install those apps, I’m right there.
I should be able to install custom routing tools. (For example: did you know that Beeline has built a custom routing algorithm for safer city cycling? That should be user-installable.)
If I have a Uber Eats account, I should see Uber Eats locations on the map – with menus and payment one tap away. Or an Airbnb layer, if I’m arriving into a new city, in the frankly unbelievable scenario that I’m ever more than half a mile from my home ever again.
Cameras and maps are special
Not everything can be a runtime.
A runtime needs space for interaction, but it also needs discovery. So I’m intrigued about the idea of AirPods as a runtime - I would love programmable hearing - but I can’t see how I would discover new user-installable functionality while I was walking down the street. Apps whispering in my ear? I don’t think so. Likewise with Zoom: great idea to have apps running inside the video, adding functionality to my meetings, but can I imagine app advert pop-ups during a work call, offering to transcribe the task list? No.
Smart speakers don’t quite make the cut, for me. There’s no native way to learn about and install new apps. And messaging apps could have been runtimes. Facebook and Apple have both given it a good go. But it turns out that the discovery mechanism was group conversations, and it wasn’t powerful enough. Good on them for giving it a try.
But the default smartphone live camera view, and the map view – these should have app stores.
My speculation, and this is just a speculation, is that everyone is keeping their powder dry for smart glasses and augmented reality.