Mesa Imaging: SwissRanger

Mesa Imaging, spun from CSEM (Swiss Center for Electronics and Microtechnology), specializes in commercializing 3D time-of-flight (TOF) depth camera. Their TOF device debuts for a long time so that those field had used SwissRanger for 3D information requirements, like robotics field.

SwissRanger -SR4000
SR4000 is MESA Imaging's latest product, which is more compact, different with its previous generation. It has higher modulation frequency to get higher resolution.

• Pixel size: 176 x 144
• FOV: 43.6 x 34.6
• Pixel pitch: 40um
• Modulation freq.: 14.5~31MHz
• Range: 0.8~8m
• Resolution: 10mm

Measuring Principles - TOF
Time-of-Flight (TOF) technique, well adopted by making use of the inherent properties of "wave", has two major principles to obtain depth information -- time domain and phase domain (all related to time but different in representation). SwissRange use phase domain to calculate distance by demodulation technique. Demodulation is very common, and makes operation to extract phase shift by comparing obtained signal with reference signal.

The basic mathematics formula for demodulation is correlation with reference signals of different phase delay. So conventional method is lock-in structure by electronic circuitry like PLL, multiplier....etc. But SwissRanger has a very special view to convert the basic concept - infinity time space correlation to discrete time space. Signal measurement in digital always use "sampling" and need a minimum requirement that at least double sampling frequency than measured signal because of Nyquist criterion. Each sampling data has the same time interval and integration duration. Based on discrete Fourier transform (DFT), cross-correlation in infinity time space has the same effect with discrete sampling integration. Sampling is synchronized with reference signal (same as IR pulse emitter), and it extract phase shift by sorting each sampling signal and integrating each sampling with the same time delay with reference. SwissRanger adopts 4-folder sampling, which means 20MHz modulation IR pulse needs 80MHz sampling sensor. Phase is calculated from arctan [(Integration(270)-Integration(90))/(Integration(0)-Integration(180))]

CCD Pixel Design
For the correlation principle by DFT, it need a special pixel design to satisfy the requirement - subsampling and integration for each.

1. Detection of light - convert photon to electron
2. Fast Separation - subsampling and divide each to 0/90/180/270
3. Repeated addition - integration for each subsampling
4. In-pixel storage - during the sampling period, all subsampling and photocharge is in pixel and for the amp and A/D in next stage

Sensor
There are 3 basic parts for TOF sensor:

1. Photodetection -PD with sampling
2. Demodulation - for fast separation ,repeated addition, and in-pixel storage
3. Operation - MUX, column amplifier, and A/D converter

Each pixel outputs 4 different subsampling signal -0/90/180/270 , amplified in column Amp and then all converted to digital. Processor will use digital signal to calculate phase shift for each pixel.

In the publication paper from mesa imaging, they show different pixel types as below which all contain the 4 function -from detection, separation, addition to storage. 4-tap CCD has the highest aperture ratio; 1-tap CCD has the highest frequency; multitap photodiode has better performance in low signal level and other wavelength.

Issues

• Sampling by gate control
  CCD is good at its low noise in charge transportation which controlled by gate. For high sampling frequency, each photocharge integration time is short even though the repetition numbers is high (Ex. 20MHz IR Pulse--needs 80MHz sampling rate ;For 20 fps frame rate, it has 50ms integration - 10^6 repeated additions for each subsampling.) For fast and stable charge transportation, each charge converted from photon needs to move to floating node immediately. Mesa imaging use transparent gate above buried photodiode to construct electronic potential to move charge more stably and less power consumption. Only the outer gate has electronic contact and wire so that inner gates are floating gates coupled capacitively without any wire.

• Demodulation by gate control
  Demodulation area is separated geometrically from photodetection area for compact size requirement which can decrease the parasite capacitive and enhance demodulation frequency. Its electronic potential is similar with above figure. For a 2D arrangement to separate 4 section like 4-tap CCD, gate control can operate to move each subsampling photocharge to its storage.

• Illumination
  SR4000 is very terse to not see the IR illumination from the appearance (SR3000 has LED array encompassed the lens in front view.) Mesa imaging use pyramid reflector to guide LED to make the illumination beam shape has the most overlap with field of view (FOV) with camera lens. This design can optimize the IR LED and reduce the multi-path problem.

• Noise level
  The resolution is determined by the ability to resolve the phase from digital signal, which can be improved by increase the power of LED, f-number of lens and the average number (the numbers of addition) The minimum phase resolution is lambda/200, which is equivalent to 7.5cm for 20MHz modulation frequency (lambda=15m).If the noise level is handled well, the resolution can achieve sub-cm.
• Aperture ratio
  SwissRange's aperture ratio is very low, compared with CCD/CMOS image sensor.Each pixel contains photodiode, demodulation circuitry and transporting CCD. The area designed for photodetection does not occupy most area, especially as demodulation area is very important in TOF measurement.
• Array size
  176x144 is very low resolution and 40um in size is very huge in image sensor. CCD process is higher cost than CMOS. 176 pixels would need 7mm in horizontal. The same area but with much less pixels has the limitation in high resolution image requirement. 
• Power consumption
  For 20MHz modulation IR light, the sensor will operate in 80MHz sampling frequency and each floating gate operates in high voltage above 5V, which make CCD such high power consumption more worse. And sensor need PLL circuitry to multiply the reference function to control gate to separation the photocharge. If they do not consider the heat dispersion, the temperature drift would cause error in phase measurement.

[Related articles]

• Canesta: time-of-flight CMOS sensor
• 3DV systems : ZCam - depth camera