Canesta: time-of-flight CMOS sensor

Recently, Canesta agree to be acquired by Microsoft.

Canesta, a California company, is founded by Cyrus Bamji and Abbas Rafii in 1999. It focus on 3D sensing technology.

(Left: James Spare, CEO; Right: Cyrus Bamji, CTO, founder)

Startup - virtual keyboard

Virtual Keyboard by projection was first invented by IBM. In 2002, Canesta developed a projection keyboard using their propietary "electronic perception technology". Electronic perception technology is the 3D sensing CMOS camera by time-of-flight (TOF) technique. This was very marvelous  product but too advanced in that time. It was praised as tomorrow's digital device.

But Canesta licence these patents to Celluon, a Korea company.

• Method & Apparatus for Entering Data Using a Virtual Input Device
• Quasi-Three-Dimensional Method and Apparatus to Detect & Localize Interaction of User-Object & Virtual Transfer Device
• Method for Enhancing the Performance in a System Utilizing an Array of Sensors that Sense at Least Two Dimensions
• Method & Apparatus for Approximating a Source Position of a Sound Causing Event for Determining & Input Used in Operating an Electronic Device
• Method & Apparatus for Approximating Depth of an Objects Placement into a Monitored Region with Application to Virtual Input Devices

Now smart phone and pico projector rise the demand of input interface. We can image that virtual keyboard would be applied in many field.

Time-of-Flight (TOF)
Base on the relationship between velocity and time, TOF is very common in daily life. Distance can be calculated by multiplying velocity and time. Moving object and acoustic wave are easy understood, but light speed is too fast. Laser rangefinder get the distance by comparing the time delay between the pulse of emitter and receiver, but it is appropriate in "m" such long distance level.  In 3C products, devices  are usually far from several "cm", which causes time delay too short for measurement.
(TOF: http://en.wikipedia.org/wiki/Time-of-flight )
(Laser rangefinder http://en.wikipedia.org/wiki/Laser_rangefinder )

Modulation measurement
If you are familiar with lock-in (in small signal measurement) or PLL (in communication field), you must hear "homodyne technique". Canesta module use IR LED emit pulse signal, then CMOS sensor receive the reflected pulse light from object. From the block diagram, reflected pulse light has the same frequency with emitter. Mixer, a nonlinear circuit, multiply the received signal with reference signal. Low pass filter would let the signal which has the same frequency with reference pass. Another block mixer with a phase delay - 90 degree - support the cosine item in phase calculation. Distance can be extracted from phase.The phase can be determined to  better than 1/300, or less than 50 pico second. 50 ps means its distance resolution is 15mm.

Furthermore, Canesta has put much effort in solving those issues :

• phase ambiguity - use different modulation frequency
• noise filter 
• calibration 
• ambient effect
• dynamic range - for different reflectivity

Canesta CMOS chip

For the TOF measurement, Canesta integrate the electric, described above, into the area CMOS sensor. Each pixel,synchronized with the light source, has its circuits that contain low noise amplifier, mixer, low pass filter, integrator, variable phase delay. For real-time, it gets phase signal by setting neighbor pixel with different phase delay then binning them as a cluster pixel.

Patent
Caneta  has invented many fundamental technology for 3D Time of flight detection and manufacture. They issued many patents making 3D TOF robust, like  ambient light, aliasing, motion blur, dynamic range, timing...etc.  It is not easy to implement many kind of circuits in sensor. Canesta's technology is specifically designed for CMOS process in that it is immune to surface defects  and the performance improves as feature size get smaller.

We can see how Microsoft will use the 3D sensing technology of Canesta in the future.