1.
Ear2Face! It is possible, given a photograph of an ear, for an AI to take a really good guess at the person’s face.
Ref.
Yaman, D., Eyiokur, F. I., & Ekenel, H. K. (2020). Ear2Face: Deep Biometric Modality Mapping (arXiv:2006.01943). arXiv.
Just… just go to the PDF at the link about, and look for Figure 4 which is a grid of photographs of ears and actual faces and AI predicted faces. Please just do that for me. (Illustration preserved on my IG for posterity.)
(Being realistic for a sec - the ear photo also includes a bit of hair, around the side and at the top. So you can get a sense of hair colour, which way it falls etc. Then you also get a good idea of skin tone, age, etc, and who knows what else is being picked up in that photo… it’s not going to be exclusively down to ear whorls. And actually, after getting those coarse features that even you or I could take a stab at, once you look closely at the illustration, you can see the AI is doing an okaaaay job but it’s not magical.)
TANGENTIALLY:
Apple Support for Personalized Spatial Audio, by which the already incredible spatial audio on Apple AirPods is improved by photographing your ear geometry with the LIDAR-enhanced camera: To capture a view of your right ear, hold your iPhone with your right hand. Move your right arm 45 degrees to your right, then turn your head slowly to the left.
2.
Video deepfakes: ‘live’, real-time deepfakes in video calls have not been a major cause for concern until very recently.
I didn’t know they were a concern now! But apparently deepfakes of celebs are a thing on Tiktok - and the potential of a live video call on Zoom, say, which turns out to be a deepfake… it’s enough that there is some concern today. (It has never even occured to me that a WhatsApp/Facetime/Zoom from a loved one could be a deepfake.)
Anyway here’s the trick:
To Uncover a Deepfake Video Call, Ask the Caller to Turn Sideways.
Synths, deepfakes, are poor at rendering ears and profiles generally.
The point is not that there is a trick to identify synths on calls. The point is that, in order to get to the trick, first you have to suspect. And to have your synth-suspicion meter reach 100 on a call such that you ask the could-be-a-synth to turn to the side, that requires your synth-suspicion meter to be active always..
One day soon, in the back of head on all calls, we will all have to be asking ourselves: is this real? Nothing is real (2021).
(Research idea: some kind of high trust proof-of-identity protocol that bootstraps off two people meeting IRL and can be carried into calls, like 2FA for person-to-person interaction.)
3.
In-ear EEG!
Such headphones can then be used to perform electroencephalographies (EEGs). EEGs record the electrical activity on the surface of the brain. This activity may be affected by disease states such as tumors or strokes, as well as by normal functions such as eye-blinks or the auditory response. In the future, this technology can be used for lifesaving interventions by quickly diagnosing a stroke, or for mundane activities like pausing your music player with the blink of an eye.
Ref.
Kaveh, R., Doong, J., Zhou, A., Schwendeman, C., Gopalan, K., Burghardt, F. L., Arias, A. C., Maharbiz, M. M., & Muller, R. (2020). Wireless User-Generic Ear EEG. IEEE Transactions on Biomedical Circuits and Systems, 14(4), 727-737.
So that would be cool. There is a bunch of headroom in multimodal computer interactions (as discussed) and blinking is another channel.
SEE ALSO, an ooooold idea: FuelBand for alpha waves (2012).
4.
Bartosz Ciechanowski’s essay on Sound.
From a model of molecules bouncing in a box, to understanding pressure, then pressure waves, then individual notes, then sound. Read this!
Simple interactive simulations with a single slider, embedded in prose, are crazy powerful for understanding.
1.
Ear2Face! It is possible, given a photograph of an ear, for an AI to take a really good guess at the person’s face.
Ref.
Yaman, D., Eyiokur, F. I., & Ekenel, H. K. (2020). Ear2Face: Deep Biometric Modality Mapping (arXiv:2006.01943). arXiv.
Just… just go to the PDF at the link about, and look for Figure 4 which is a grid of photographs of ears and actual faces and AI predicted faces. Please just do that for me. (Illustration preserved on my IG for posterity.)
(Being realistic for a sec - the ear photo also includes a bit of hair, around the side and at the top. So you can get a sense of hair colour, which way it falls etc. Then you also get a good idea of skin tone, age, etc, and who knows what else is being picked up in that photo… it’s not going to be exclusively down to ear whorls. And actually, after getting those coarse features that even you or I could take a stab at, once you look closely at the illustration, you can see the AI is doing an okaaaay job but it’s not magical.)
TANGENTIALLY:
Apple Support for Personalized Spatial Audio, by which the already incredible spatial audio on Apple AirPods is improved by photographing your ear geometry with the LIDAR-enhanced camera:
2.
Video deepfakes:
I didn’t know they were a concern now! But apparently deepfakes of celebs are a thing on Tiktok - and the potential of a live video call on Zoom, say, which turns out to be a deepfake… it’s enough that there is some concern today. (It has never even occured to me that a WhatsApp/Facetime/Zoom from a loved one could be a deepfake.)
Anyway here’s the trick:
To Uncover a Deepfake Video Call, Ask the Caller to Turn Sideways.
Synths, deepfakes, are poor at rendering ears and profiles generally.
The point is not that there is a trick to identify synths on calls. The point is that, in order to get to the trick, first you have to suspect. And to have your synth-suspicion meter reach 100 on a call such that you ask the could-be-a-synth to turn to the side, that requires your synth-suspicion meter to be active always..
One day soon, in the back of head on all calls, we will all have to be asking ourselves: is this real? Nothing is real (2021).
(Research idea: some kind of high trust proof-of-identity protocol that bootstraps off two people meeting IRL and can be carried into calls, like 2FA for person-to-person interaction.)
3.
In-ear EEG!
Ref.
Kaveh, R., Doong, J., Zhou, A., Schwendeman, C., Gopalan, K., Burghardt, F. L., Arias, A. C., Maharbiz, M. M., & Muller, R. (2020). Wireless User-Generic Ear EEG. IEEE Transactions on Biomedical Circuits and Systems, 14(4), 727-737.
So that would be cool. There is a bunch of headroom in multimodal computer interactions (as discussed) and blinking is another channel.
SEE ALSO, an ooooold idea: FuelBand for alpha waves (2012).
4.
Bartosz Ciechanowski’s essay on Sound.
From a model of molecules bouncing in a box, to understanding pressure, then pressure waves, then individual notes, then sound. Read this!
Simple interactive simulations with a single slider, embedded in prose, are crazy powerful for understanding.