What's impossible to see?

Computer vision has the ability to solve for what humans cannot see or process upon seeing. There are four ways in which computer vision can augment:

  • if something travels faster than what humans can see
  • if something is smaller than what humans can see
  • if something is outside of our normal cognitive pattern, ie. things that we normally don't, so we can't recognize it
  • if something doesn't have a recognizable pattern upon first glance, but in fact it does

The first two ways are relatively simple to solve for and relatively uninteresting. The latter two are harder to do and relatively more interesting. The third way relies on taking a visual input, categorizing it correctly, then retrieving the right data to compare and contrast. The fourth way relies on taking a visual input, then categorizing and structuring a pattern around it. I find the fourth way immeasurably harder in many respects.

I default to the same framework when it comes to computer vision for autonomous driving. Technologies that can help recognize a pattern to better aid autonomous driving systems is where my mind gravitates. For example, perhaps helping ADAS' define traffic and accident data to understand how accidents happen and find the strongly correlated behaviors. Or helping ADAS' define different city driving behaviors so that autonomous cars are able to better react to differences in city driving behavior. This is important because, as Mercedes Benz USA CEO says, the real constraint of autonomous driving becoming ubiquitous is humans.

There are some companies working on this problem now, or potentially could do. Nauto, a startup offering a network of cloud-connected dashboard cameras, could start analyzing traffic and accident data. Nexar and Mapillary could also do that based on the data they have on local roads and driving behavior. For the time being however, they are only focused on data aggregation rather than data analysis.

But rather than working on yet another car camera, there are other ways to get driving data. You could take existing dash cam videos; Russia has the largest amount of dash cams, but Asian countries like Taiwan and Korea are experiencing high growth as well. Data upload could be incentivized and immediately there's a ton of data available. You could also place fixed cameras, like Placemeter, in various city spots to measure high congestion areas and see how humans behave around cars.

Data aggregation isn't where the sweet spot is. Data aggregation is merely the gateway to where the interesting stuff could come from. The more interesting stuff is taking that data and interpreting patterns where no one expects.

Thanks Nathan Benaich, Pete Kane, and Jung-hee Ryu for thoughts.