Digital Imaging Fundamentals
Ilario Grijalva
For the past fifty years, the primary medium for photomicrography has been film, which has served the scientific community well by faithfully reproducing countless images from the optical microscope. It has only been in the past decade that improvements in electronic camera and computer technology have made digital imaging cheaper and easier to use than conventional photography.
Illustrated in Figure 1 is a Nikon Eclipse 600 transmitted/reflected light microscope equipped with an aftermarket Peltier-cooled digital camera capable of integrating images over a long accumulation period. The camera system is controlled by a separate unit that interfaces to a FireWire port housed in an IBM-compatible personal computer. Integration periods and other image acquisition parameters are selected by a proprietary Windows-based software program.
When a camera having charge coupled device (CCD) imaging sensor incorporates an analog-to-digital (A/D) converter on the sensor or in close proximity, it is generally referred to as a digital camera. Because CCD chips, like all optical sensors, are analog devices that produce a stream of varying voltages, the term digital is used only when those voltages are digitized in the camera and output in a computer compatible format. In a 12-bit digital camera, the analog signal from the CCD is digitized with 12-bit depth by the on-board A/D converter. Whether or not the output can actually be resolved into 4096 discrete intensity levels (12 bits) depends on the camera noise. In order to discriminate between individual intensity levels, each gray level step should be about 2.7 times larger than the camera noise. Otherwise, the difference between steps 2982 and 2983, for example, cannot be resolved with any degree of certainty. Some so-called 12-bit cameras have so much camera noise that 4096 discrete steps cannot be discriminated.
If the signal is analog to start with, why digitize it in the camera rather than somewhere downstream? There are two benefits to using an in-camera A/D converter: reduced noise and direct computer-compatible output. In general, the closer the A/D is to the sensor, the lower the noise level. Low-level analog signals from the CCD are far more readily corrupted by noise than their high-level digital counterparts. In the ideal case, the A/D is on the CCD chip, immediately adjacent to the output amplifier of the sensor. The lower the noise, the more gray levels can be identified and, therefore, the more bits that can be meaningfully used for the intensity measurement.
A digital camera has several advantages over its analog counterpart. Digital cameras produce a progressive scan output unlike the interlaced signal generated by video cameras. Digitization of interlaced video signals requires specialized capture boards and frame buffers. The output of progressive scan cameras can be interfaced directly to the computer (e.g., IEEE-1394, RS-422 or SCSI interfaces). In a progressive scan camera, the entire image is first acquired during the exposure time (also denoted as the integration period) and then read out, line by line from the top of the image to the bottom. Modern, high-speed amplifiers and A/D converters permit digital cameras to produce full-frame images at rates that equal or exceed the video framing rate.
Another advantage to digital cameras is that the output perfectly suits the format of a computer monitor. Because the signal is already digitized, image storage, manipulation and display are greatly simplified in comparison with similar maneuvers using video signals. The difficulties of dealing with prints, slides and negatives are eliminated in digital photography because many scientific journals now accept digital image files. The result is improved quality both of published images and those shown in presentations. The digitized image can be processed, compressed, transmitted via the Internet, pasted into documents or tuned into a poster.
Two CCD designs are commonly used in digital cameras: interline transfer and frame transfer. The interline-transfer CCD incorporates charge transfer channels beside each photodiode so that the accumulated charge can be efficiently and rapidly shifter over to them (Figure 2). Interline-transfer sensors can also be electronically shuttered "off" by dumping the stored charge instead of shifting it into the transfer channels. The frame-transfer CCD uses a two-part sensor in which the top half is covered by a light-tight mask and is used as a storage region. Light is allowed to fall on the uncovered portion, and the accumulated charge is then rapidly shifted into the marked storage region. While the signal is being integrated on the light sensitive portion of the sensor, the stored charge is read out.
Two types of color digital cameras are used for scientific applications-a single CCD camera with a wavelength selection filter or a three-sensor camera. Both use filters to produce red, green and blue versions of the field-of-view. The single-sensor cameras uses a filter wheel or liquid-crystal tunable filter to acquire the red, green and blue images in sequence. The three-sensor camera has a beam-splitting prism and trim filters that enable each sensor to image the appropriate color and to acquire all three images simultaneously. Invariably, color cameras are less sensitive than their monochrome counterparts because of the additional beam-splitting and wavelength selection components. In some applications, particularly immunofluorescence, the loss of sensitivity is offset by the ability to capture multiple wavelengths simultaneously or in rapid succession. In addition, some color cameras achieve a higher resolution by diagonally offsetting the red, green and blue sensors, each by one-third of a pixel, thereby tripling the number of samples obtained.
Although CCD camera manufacturers and users routinely refer to each photodiode as a pixel (picture element), there is no requisite correspondence between the number and position of the pixels in the sensor and those in the computer monitor or printer. However, the display or printer resolutions should always be at least as high as that of the sensor.
Quantum efficiency (QE) refers to the percentage of incident photons that are detected. (For reference purposes, the QE of our photopic vision is about 3 percent; Figure 3). Silicon photodiodes, the basic building blocks of the CCD, have a high QE (80 percent) across a broad range of the visible spectrum and into the near infrared, as illustrated in Figure 3. The spectral sensitivity of a CCD is lower than that of a simple silicon photodiode because the CCD has charge transfer channels on its surface that reduce peak QE to about 40 percent.
Recently, the transparency of the channels of some scientific-grade CCDs has been increased and the QE in the blue-green range improved to nearly 70 percent. The losses from the surface channels are completely eliminated in the back-illuminated CCD. In this design, light falls onto the back of the CCD in a region that has been thinned by etching until it is transparent. A quantum efficiency as high as 90 percent can be realized. However, back-thinning results in a delicate, relatively expensive sensor that, to date, has only been employed in scientific-grade, slow-scan CCD cameras.
There are two major sources of noise in CCD cameras-dark noise and read-out noise. Although great improvements have been made over the past few years in the reduction of CCD dark noise at room temperature, cooling the chip further reduces the noise tenfold per 20 C decrease. Dark noise is most evident as "hot" pixels (white dots) in images obtained with room temperature CCD cameras after integration periods of 4 or 5 seconds. Cooling to 0 C is usually sufficient for integration periods up to 30 seconds. Experiments requiring very long exposures (e.g., chemiluminescence) need even lower sensor temperatures. Digital cameras are available in cooled or uncooled versions.
Noise sources vary in digital cameras, and several common types are presented as oscilloscope traces in Figure 4. Photon noise, dark current, fixed pattern noise, and photo response nonuniformity are generated on the CCD itself, while reset noise, I/f noise, and quantization noise occur during amplification and conversion of the analog signal to a digital output. Read-out noise is generated in the amplifier on the CCD chip that converts the stored charge of each photodiode (i.e., pixel) into an analog voltage to be quantified by A/D conversion. Read-out noise may be viewed as a "toll" that must be paid for reading the stored charge. The size if this toll has decreased steadily in the past few years to 5-10 electrons/pixel because of improvements in CCD design, clocking and sampling methods. Read-out noise increases in proportion to read-out speed. The cost of going faster is more noise and hence, more uncertainty in the voltage determination and lower number of bits of resolution. This is why slow-scan cameras generally exhibit lower read-out noise than faster output detectors and have higher number of useful bits. Digital came4ras range from those with 8-12 bit depth at 30 frames per second output to 16-bit depth at 1-2 frames per second.
One solution to the speed/read-out noise problem is the use of multiple output amplifiers (taps) on a large CCD. Instead of reading the stored charge from the entire CCD through one output amplifier, the sensor is divided into four or eight sections each of which has its own amplifier. The image is read out in parts and then stitched together in software at rates of several frames per second. The required speed and associated noise of each amplifier are reduced accordingly.
Since photons randomly arrive at the sensor surface, their numbers fluctuate with a noise as described by Poisson statistics that is equal to the square root of the number of detected photons. Of course, camera noise adds to this photon statistical noise and further reduces the S/N. The highest S/N that can be achieved by a digital camera is the square root of the maximum accumulated charge (the full-well capacity). A simple estimate dot the S/N of any homogeneous region in an image is the average intensity of the region of interest divided by the standard deviation of the intensity of that region.
c80f0f1006