Using Image Recognition Software to Find Your Soulmate… and for Social Good

This week’s guest is Justin Long, a researcher and engineer at 3 Tier Logic. As you may have read in recent coverage on Justin’s Tinderbox project, he has figured out a way to automate dating apps like Tinder using image-recognition algorithms, saving time and, as we’ll hear about today, planting the seed for future work in image recognition outside of online dating.

BlackBoxPhD 3-Point Checklist: How to Leverage Image Recognition Technology



Image-recognition software can be used to make online dating more efficient. Justin used the Eigenfaces open source technology to improve his experience on Tindr and save him time.
This technology can also be used for social good. These software innovations can be scaled to detect things like child pornography and sexual abuse.
Eigenfaces is a “quick and dirty” image recognition solution. To recognize the faces across different pictures, it requires that the person is facing the same direction in each picture.

Justin – thanks so much for speaking with us! The recent coverage of your Tinderbox left us at the UCLA Center for Digital Behavior very intrigued. Could you share a bit about yourself and how you got involved with Tinderbox?

Sure. My name is Justin Long and I’m the Chief of Research and Special Projects at 3 Tier Logic.

Tinderbox began as an everyday problem. Tinder is a very widely used dating application that requires users to either swipe right (yes) or left (no) on a potential match’s picture, depending on if they like them or not.

One of my gripes with the app was that it was very time-consuming. My friends were getting sucked into it, I was getting sucked into it and I just wanted to automate the process. While it started out as a gripe, as I started building it and diving into the technology, it became more about seeing if I could build it and it turned into a really fun project.

Could you explain how the image-recognition algorithm, Eigenfaces, works? Is it open source?

Eigenfaces has been around for a long time (since 1987) and it is open source. The way that you do eigenfaces is you have a training set of faces who have already been identified – you already know who they belong to. You turn each image into a matrix, so that it is just a grid of numbersthat all represent the pixel intensity. Next, across each image in the entire training set, you take the average and calculate the mathematical difference between each individual image. Once you have the average across the entire grid, you can then compare each new image with that average, and find which image has the closest difference. As in, if you are identifying someone for facial recognition purposes, the algorithm will compare the differences between the new image and the average image – the mean – and is able to recognize person A or person B and differentiate that from person C.

Interesting. Now, with the image-recognition capabilities of Eigenfaces and the smart bot built into Tinderbox, where else do you see this being applied? How do you hope others will use what you’ve built for crime, health and medicine?

There are two immediate applications. One is related to prediction technology for health and forensics. If you’re an investigator, say on a child pornography case, it may be useful to use the same facial recognition to identify a perpetrator. That’s more of a policing use, but from a health use- it’s not every day somebody gets their photo taken of them. They do put pictures of them on social media, and I’m personally very interested in looking at if I were to average each face, and compared that average of someone with a history of abuse to someone who does not, would be very interested to see whether there’s a pattern that you can say “ there’s definitely a facial structure change over time”. A lot of our emotions are communicated via facial expressions and it is known that your facial structure changes over time.

Another is business-related. Anyone who runs a dating site is interested in this, because anyone who runs a dating site wants to optimize their matching algorithms.

What are potential barriers in using something like Eigenfaces for recognizing abuse and what are ways to overcome these hurdles?

You’ve got a couple problems with the algorithm itself. The first one is, Eigenvectors only works with black and white images. If you wanted to do any analysis on images that are color-dependent, like a bruise, it might not actually work. When you take a discoloration and turn it to grayscale, it’s not going to be significant enough to stand out. You couldn’t use Eigenvectors or Eigenfaces for something like recognition of a bruise. You’d have to have a very sensitive set and it would probably be impractical.

Second, you need to have relatively consistent images. With Tinder, people generally take similar profile pictures, i.e., from the same angle. With Tinderbox, I had the algorithm only examine images where there was a single face (as opposed to group pictures), and where the picture was taken from a balanced perspective – with the person’s face pointed straight at the camera. If pictures of the person are not consistent, then the algorithm may not be as successful in correctly identifying an individual.

In terms of overcoming these hurdles and possibilities for future work, one of the most interesting results that came out of the Tinderbox project was how the Eigenfaces looked when you averaged them together. The uncanny black and white faces that you saw on my blog – those are a representation of the average of each face merged together. I had an a-ha moment, where I thought how this might be used to predict faces of individuals who may be under stress or may be being abused. It made me think, what does a person from a background of abuse look like in comparison to a control – to a person who is not being abused and who is happy in their own lives? This could be very powerful.

If indicators of abuse (facial expression, change over time) were contained within the face, then you might be successful in detecting those indicators and identifying something like abuse.

Psychology, Products, People

Psychology, Products, People

Using Image Recognition Software to Find Your Soulmate… and for Social Good

Leave a comment Cancel reply

92 thoughts on “Using Image Recognition Software to Find Your Soulmate… and for Social Good”