You Are Using Computer Vision Every Day, But You Don’t Know it Yet
September 18, 2018Sep 18, 2018 | By Chang Xu

When you wake up in the morning, the very first thing you do is to authenticate into your iPhone using Face ID, which your phone can do using its front-facing camera.

1 nkJkx1nDng-2Hatqx OKwA

While you're still in bed, you open up your Nanit app, which gets its information from the camera above your baby's crib. Nanit gives you a report on how well your baby slept last night. You see that it took Lily 12 minutes to fall asleep. She slept for 10 and half hours, and you and your husband only had to visit her twice in the middle of the night. Pretty good!

1 ruN07K2CnHWNIbTZkahzGg

As you flip through your email, you get a notification from Waldo Photosthat they have new photos of your daughter, who's away at camp. As teachers at camp upload the photos they take of the campers, Waldo's image recognition works in the background to pick out just the photos including your daughter, saving you the trouble of slogging through everybody's photos.

1 FF38DZH KYEBeTrqTyXSg

Then as you are making breakfast, your son is playing with the iPad on the table. The iPad can see what's on the table because it has an Osmo camera on top. Your son puts together Lego-like puzzle pieces and makes the character on the screen of his iPad jump through obstacles, telling it to run right three steps, run down two steps, then jump.

1 mdexWum07xJiBPh9PQaKTQ

Your doorbell rings and you open up the Ring app to see that it's the delivery guy with a package for you. You tell him to leave the package there because your hands are full right now and you'll pick it up later.

1 6OJdzw6FcK9sZpq77GQQyA

You finally get to work and as you go into a conference room for your meeting, the Density device installed above the door counted the number of people in the room. Density devices gather space utilization data in this room and in every room in your building. Density reports the data to your facility manager, who can then better plan the spaces in all of their buildings.

1 d48O9Uxo-VA08oxvy77vHA

After work, you drive to the grocery store. You didn't pay attention for a second and you deviated from your lane, so your car beeped at you. Because your car's rear-view mirror has a camera on the back of it that can see the lane markings in front and can detect it when you veer away from your lane.

1 2Bq2gln6K6sfVgUIh2v2aA

You got to an Amazon Go store to pick up groceries. You simply pick up what you want from the shelves and put it in your bag. The dense mass of cameras on the ceiling and in the shelves identified you and the item you picked up. As you leave the store, you got a bill sent directly to your Amazon app.

1 6dTxmhFdGUKPeZcNs9E 1Q

1 Og-s82vXQIghparCk5FrQ

By the way, the lettuce that you bought at the store has been grown in a greenhouse with cameras installed on the ceiling. These cameras monitor the crops and if it sees any discoloration on the leaves that indicated certain diseases, it would alert the farmer to treat it immediately.

1 TSRCKY6iv56pwYOVXiAyGg

While you were in the store's parking lot, satellites orbiting the earth took a picture of the parking lot and counted the number of cars in it. They compared it with the day before, the month before and the year before. This data is then sent to a hedge fund. The hedge fund used it to make predictions on how this retailer is doing and traded based on that information.

1 pjbQt1JNv6e8Zl5F8U9 NA

In the evening, you turn on ESPN and you watch the Cleveland Cavaliers play the Golden State Warriors. Little do you know, in the background, GumGum is quietly observing the logos of sponsors on-screen, whether on the court, on the sides, or on players' jerseys. GumGum measures the logos by their clarity, prominence, visibility, size, share of voice, and placement in order to quantify the value each sponsor received during this game. The sponsors can then use this information to calculate their ROI as well as to support their negotiations.

1 7 3jUwjpKUzxGFCbAvO9qw

Everything you are doing uses computer vision. You just haven't thought about it in any of these ways. One of the reasons that computer vision is so useful is because it is pervasive.

What is computer vision?

Let's take a quick step back. You don't need to understand the math. You just need to understand that ---

Computer vision is being able to interpret the physical world through cameras and sensors.

In the past, we have to manually input instructions to tell machines to do one action and then another.

1 RQomhBXnju9ghFWukrtQhw

Then, computers gained the ability to infer the world through data such as Tweets, posts, and purchases. But they only understood numbers and words.

1 CdHRi Xl3rJq15w28LU2rA

Now, computers can understand what it is seeing and sensing. They can take inputs that are images and videos, which allows them to directly observe the physical world.

1 dwwcSL-vlKPm- GBUfG13Q

This means that we can have many many cameras that humans don't need to look at. Using cameras as their eyes, computers will become more knowledgeable and aware of the physical world and be able to directly interact with us.

1 rd9gzWmlcNmypATA2jOeJw

Scene from Big Hero 6. Baymax must have great computer vision hookups.


This story is the start of a series about computer vision. Stay tuned for more!

If you are building a startup using computer vision, I'd love to talk to you. Shoot me an email at chang@upfront.com.

When you wake up in the morning, the very first thing you do is to authenticate into your iPhone using Face ID, which your phone can do using its front-facing camera.

1 nkJkx1nDng-2Hatqx OKwA

While you're still in bed, you open up your Nanit app, which gets its information from the camera above your baby's crib. Nanit gives you a report on how well your baby slept last night. You see that it took Lily 12 minutes to fall asleep. She slept for 10 and half hours, and you and your husband only had to visit her twice in the middle of the night. Pretty good!

1 ruN07K2CnHWNIbTZkahzGg

As you flip through your email, you get a notification from Waldo Photosthat they have new photos of your daughter, who's away at camp. As teachers at camp upload the photos they take of the campers, Waldo's image recognition works in the background to pick out just the photos including your daughter, saving you the trouble of slogging through everybody's photos.

1 FF38DZH KYEBeTrqTyXSg

Then as you are making breakfast, your son is playing with the iPad on the table. The iPad can see what's on the table because it has an Osmo camera on top. Your son puts together Lego-like puzzle pieces and makes the character on the screen of his iPad jump through obstacles, telling it to run right three steps, run down two steps, then jump.

1 mdexWum07xJiBPh9PQaKTQ

Your doorbell rings and you open up the Ring app to see that it's the delivery guy with a package for you. You tell him to leave the package there because your hands are full right now and you'll pick it up later.

1 6OJdzw6FcK9sZpq77GQQyA

You finally get to work and as you go into a conference room for your meeting, the Density device installed above the door counted the number of people in the room. Density devices gather space utilization data in this room and in every room in your building. Density reports the data to your facility manager, who can then better plan the spaces in all of their buildings.

1 d48O9Uxo-VA08oxvy77vHA

After work, you drive to the grocery store. You didn't pay attention for a second and you deviated from your lane, so your car beeped at you. Because your car's rear-view mirror has a camera on the back of it that can see the lane markings in front and can detect it when you veer away from your lane.

1 2Bq2gln6K6sfVgUIh2v2aA

You got to an Amazon Go store to pick up groceries. You simply pick up what you want from the shelves and put it in your bag. The dense mass of cameras on the ceiling and in the shelves identified you and the item you picked up. As you leave the store, you got a bill sent directly to your Amazon app.

1 6dTxmhFdGUKPeZcNs9E 1Q

1 Og-s82vXQIghparCk5FrQ

By the way, the lettuce that you bought at the store has been grown in a greenhouse with cameras installed on the ceiling. These cameras monitor the crops and if it sees any discoloration on the leaves that indicated certain diseases, it would alert the farmer to treat it immediately.

1 TSRCKY6iv56pwYOVXiAyGg

While you were in the store's parking lot, satellites orbiting the earth took a picture of the parking lot and counted the number of cars in it. They compared it with the day before, the month before and the year before. This data is then sent to a hedge fund. The hedge fund used it to make predictions on how this retailer is doing and traded based on that information.

1 pjbQt1JNv6e8Zl5F8U9 NA

In the evening, you turn on ESPN and you watch the Cleveland Cavaliers play the Golden State Warriors. Little do you know, in the background, GumGum is quietly observing the logos of sponsors on-screen, whether on the court, on the sides, or on players' jerseys. GumGum measures the logos by their clarity, prominence, visibility, size, share of voice, and placement in order to quantify the value each sponsor received during this game. The sponsors can then use this information to calculate their ROI as well as to support their negotiations.

1 7 3jUwjpKUzxGFCbAvO9qw

Everything you are doing uses computer vision. You just haven't thought about it in any of these ways. One of the reasons that computer vision is so useful is because it is pervasive.

What is computer vision?

Let's take a quick step back. You don't need to understand the math. You just need to understand that ---

Computer vision is being able to interpret the physical world through cameras and sensors.

In the past, we have to manually input instructions to tell machines to do one action and then another.

1 RQomhBXnju9ghFWukrtQhw

Then, computers gained the ability to infer the world through data such as Tweets, posts, and purchases. But they only understood numbers and words.

1 CdHRi Xl3rJq15w28LU2rA

Now, computers can understand what it is seeing and sensing. They can take inputs that are images and videos, which allows them to directly observe the physical world.

1 dwwcSL-vlKPm- GBUfG13Q

This means that we can have many many cameras that humans don't need to look at. Using cameras as their eyes, computers will become more knowledgeable and aware of the physical world and be able to directly interact with us.

1 rd9gzWmlcNmypATA2jOeJw

Scene from Big Hero 6. Baymax must have great computer vision hookups.


This story is the start of a series about computer vision. Stay tuned for more!

If you are building a startup using computer vision, I'd love to talk to you. Shoot me an email at chang@upfront.com.