New method for assessing colorimetric properties of luminaries

[Tom Frisch]

Years ago I was hoping to use the plots spectrometer to measure the color accuracy of the lights I use for video and photo work.  Currently manufacturers use something called CRI to quantify the accuracy of the light you get from a light source.  In my research I learned that CRI was based on 1960's technology and math.  It looks at a series of pastel paint chips, and measures how accurately they are illuminated by a test light source.  Turns out those pastel chips don't really represent all the colors that cameras are capturing, so it makes for a pretty inaccurate measurement of color accuracy.  Making maters worse, fluorescent and LED bulbs often have big spikes of certain wavelengths of color, and those spikes are not always caught by the limited CRI test.  I also read that bulb makers are not interested in a more accurate test because it shows just how lame a lot of bulbs really are.  

During this period of research, I wrote to a Dr. Yoshi Ohno who worked at NIST.  He was promoting another system of measurement called CQS that uses far more colors in the test, and should be much more accurate.  Even better, he was able to produce a series of equations built into an excel spreadsheet that could take a spectrum and produce CCT, CRI, and CQS numbers from the raw spectrum.  I figured this was exactly what I needed, and he was even so kind as to share the spreadsheet with me.  Now 3 years later, life got in the way, and I didn't do anything with the information.  

Today I was reading my usual video production blogs, and I came across a wonderful interview with a British camera tester named Alan Roberts, formerly of the BBC.  

http://www.newsshooter.com/2015/02/27/bve-2015-how-accurate-are-your-led-lights-ex-bbc-expert-alan-roberts-has-the-surprising-answers/

It seems he came up against this same issue, but being much smarter and dedicated than I, he went ahead and designed his own test.  Like Ohno's, it also uses an expanded set of color chips, and from what I can tell, just like Ohno's spreadsheet, his also can simulate lighting those chips from a spectrogram.  He has gone ahead and tested many of the name brand light sources, and posted his results online, as well as his testing methodology (posted here: https://tech.ebu.ch/docs/tech/tech3355.pdf.  He also said his software is free, but I haven't dug around to find it.

I figured I'd post the info here, in case anyone wants to pick up the torch and consider implementing either Ohno's or Rober's work on Spectralworkbench.

[Terry Relph-Knight]

Hi Tom,

This is an interesting topic and it's very useful to know about Alan Roberts work. Thank you for posting it.
 
The problem with measuring anything like this with the PL Spectrometer and Spectral Workbench is one of calibration. The fluorescent lamp wavelength calibration seems to provide a calibration repeatability of about 4nm within the visible range and, since it uses the emission spectra of mercury vapour, is I suppose potentially quite accurate. One problem at the moment is that with a 0.09mm slit and careful set up you can start to resolve the dual peak in the main green spike/s. At present the Spectral Workbench wavelength calibration procedure assumes you will not be able to resolve both peaks and that they will be seen as a single peak.

There is NO amplitude calibration - NONE - and the amplitude response is not flat. Camera settings (even if you don't make any camera settings the camera electronics will default to certain values and these will not produce comparable results from different cameras) make a considerable difference in the amplitude response from blue to red. Since relative colour rendering measurements depend on having a reasonably flat amplitude response this is a problem.

There are a number of factors that contribute to the non-linearity of the PL Spectrometer amplitude response. If you are using a DVD-R derived diffraction grating then whether or not you have removed the blue dye from its surface makes a measurable difference in amplitude response.

From what I can tell, without better calibration methods, you could not use the Public Lab Spectrometers to make accurate enough measurements to satisfy Alan Roberts methods. One way forward might be to investigate ways of making a relatively broadband light source with a halogen or xenon lamp and a filter as a Public Lab product and getting that calibrated. Such a light source might be use to produce an amplitude compensation for the spectrometer. Flattening out the PL Spectrometer amplitude response is one problem. Calibrating it for absolute measurement of light power is likely to be even harder.

I'm not saying you can't use the PL Spectrometer as a useful tool to investigate colour rendering and lighting, or for all the other things you might use a spectrometer for. Just that you have to be very aware of its current limitations, which, I guess is true of using any bit of kit to measure anything.

[Tom Frisch]

Thanks Terry for that informative post.  That all makes sense to me.  I'm not sure that absolute light power would be as important, at least for color rendition, but you make many good points about how the current PL Spectrometer might not be the right tool for the job.

I also emailed with Alan Roberts, mainly asking if he thought I could run his test with date from the PL Spectrometer, and also asking the difference between CQS and his TLCI rating system.  He sent me this reply, which I'm pasting below for anyone who might want to explore this further.

Also, I looked up the spetral radiometer that he uses Ocean Optics USB2000+VR- I found the following on a $1200 new unit on ebay:

USB 2000+ Spectrometer

*Sony ILX511 2048-element linear silicon CCD array detector

*Sensitivity of up to 75 photons/count at 400 nm; 41 photons/count at 600 nm

*Integration times from 1 ms to >60 seconds

*16 bit, 3MHz A/D Converter* 4 triggering modes

*22-pin connector for interfacing to external products

Installed with:

*25 um slit,

*spectral range 200-850 nm, 

* Integration time:  1ms to > 60s

*2048 CCD Array

*Single-piece, multi-bandpass detector coating to eliminate second-order effects from 200-850 nm

*Resoluiton: ~ 1.5 nm (FWHM)

*Stray light:  

                   < 0.05% at 600 nm

                   < 0.10% at 435 nm

                   < 0.10% at 250 nm

*Fiber Optic Connection:  SMA 905 with NA: 0.22

*Requires Spectrasuite Software or Ocean View (sold separately) 

* Included:  USB cable.

*30 day warranty

-----------------Alan's response here--------------------

First, you can't get meaningful results from a DVD as a grating, because it disperse far too much. You can certainly identify wavelength peaks and troughs, using other sources for calibration (e.g. the Hg lines in flu tubes) but that gives you no clue as the the actual light magnitude at any wavelength. For that, you need a spectroradiometer and calibration.

I use an Ocean Optics USB2000+VR and have a calibration directly traceable to NPL in the UK. Currently, we're in the process of constructing a more modern calibration source which will be funded by the EBU and calibrated by NPL. With all this, I can measure the SPD accurately, in a way which others can check. You can't do this with a DVD.

The CRI is truly ancient. It uses the colour science of the 1960s. At best estimate, it is about 55% reliable in determining whether one source is better than another. The colour-difference metric it uses is CIE1964, which has been superceded by 5 CIE difference metrics since then. The CIE, in Tech 15.2004 states that all the older difference metrics are included for historical purposes, and that only CIEDE2000 should be used for colour-difference work. Also, the CRI was originally designed as a means of assessing lighting for public places, building, offices etc, and it is not all all suitable for use in television.

I know the CQS. Yoshi Ohno at NIST is one of the inventors, Wendy Davies is the other but she's moved on now. Both were on the CIE committee which was expect to approve the CQS. It wasn't approved. The CQS uses CIELAB as the difference metric, another of the banned metrics in Tech 15.2004. I asked why they used that and was told, effectively, that they didn't want to frighten people. CIELAB dates from 1976. In the subjective tests we ran before formalising the TLCI, the CELAB metric performed worst of the lot, CIEDE2000 was best. You can see all this work in EBU document Tech.3354, which is freely available at www.tech.ebu.ch. Again, the CQS was designed to upodate the CRI and is of little use in television, because it doesn't tell the truth.

The TLCI is fully specified in EBU Tech.3355, and my software to run it is available from the EBU (www.tech.ebu.ch) isn the published file TLCI-2012.zip. I also keep beta versions of improvements on-line in a Dropbox folder, which you're welcome to share if you wish. Rather than go into all the explanations here, I suggest you start downloading and reading.  

I deliberately avoided going to the CIE for approval, because I know that they're not really interested in television. The TLCI is specifically aimed at lighting in television. Not only that, but neither the CRI nor the CQS tell you what the numbers mean, the TLCI tells you exactly what the numbers mean since we ran subjective tests using professional colourists in the UK and Norway (they agreed with my original intentions almost without exception.

[Terry Relph-Knight]

I wonder what Alan means by - "First, you can't get meaningful results from a DVD as a grating, because it disperse far too much."

It seems to me there will certainly be some wavelength distortion of spectra because the diffracted light is radiating from a point, but is falling onto the flat plane of the camera sensor. So as you move away from the normal of the grating the wavelengths gradually get further apart at the point where they meet the sensor. You would either need some optical correction or a curved sensor to correct this.

However this is true of any grating. Or is my understanding of how the optics work perhaps wrong?

[Dave Stoft]

Perhaps he 'misspoke' as email has it's limitations. Dispersion is just the spread with wavelength as a result of diffraction and yes the spread on a flat image area is non-linear (though the rate of change is linear). However that can be corrected (and I proposed that SWB do this in a recent research note) - it works; it's just math.

His real point is getting an accurate spectra with amplitude cal. While I did show a rough gain-correction via a solar spectra works and improves the PLab device (see another new research note) it is limited. In addition, the PLab device has limited dynamic range. That said, one probably could make some reasonable _relative_ comparison measurements between sources using the PLab device if it has been properly calibrated (with dispersion adj) and (at least solar gain-correction) and some care in doing the measurements. The spectral limis are 400-650nm.

Cheers,
Dave

[Terry Relph-Knight]

Hi Dave,

I have read your research notes, some interesting ideas. And yes I did realise that amplitude calibration or rather lack off, was the main problem - I discussed it in my first post replying to Tom. The question is, can a general method be arrived at to accurately characterise the PL spectrometer design in order to produce a calibration correction for both wavelength and amplitude and is it worth it? It would add quite a bit to the complexity of Spectral Workbench and I would think it would mean that Public Lab would have to get its hands on a high quality commercial spectrometer and perhaps a broadband light source in order to check results.

I can see how you might apply a radial to flat correction if you assume you know the geometry of the spectrometer. I would think it rather more difficult to come up with a correction that somehow measures the angle between the grating and the camera in each individual spectrometer. 

[Dave Stoft]

Actually, as long as the PLab device is using a 45-deg incident angle, the residual offset could be accomodated from the image data -- i.e. any residual shift in the 0-deg (524nm) point; it would be possible to add-uin the small offset from center. The calculation I use finds the pixel:nm relationship directly from the pixel positions for 436 and 546nm peaks. I'm sure the SWB code could be changed from the simple 2-pt linear approximation since it only affects that isolated section of code. The peak-finding code would not have to change nor the display code.

Yes, the harder part is intensity cal and dynamic range. 8-bits is fairly limited though with the PLab 3.0, the background noise is lower which helps. A known, "flat" source is not an easy task to find and I doubt the PLab device is stable enough to consider much more than my solar-based gain-correction. What I was suggesting is that relative measurements have an advantage -- but probably not for attempting a real CRI type measurement. It's good to improve the PLab device but equally good to know it's real limits.

[Tom Frisch]

As far as a known "flat" source, which I believe you mean an SPD without a lot of massive spikes, Alan's chart lists a tungsten halogen bulb (or fixtures that use one) and a  candle "flame, white wax" as 100 on his scale.  

Certainly doesn't help the intensity calibration.  On that front, I wonder if comparing a few samples of the same wattage tungsten halogen bulb with a known slit width and webcam could give repeatable results?

Check out Alan's article, it has quite a bit about spectrometers and the potential pitfalls:

http://www.gtc.org.uk/media/fm/Zerb%20articles/TLCI_web.pdf

In it he states: "Since the mathematics of the TLCI-2012 makes calculations over the visible range from 380nm to 760nm in 5nm steps, it follows that the calibration must cover at least that range in 5nm steps. Typically, the responsivity is not flat, and the resulting compensation to obtain a flat response can emphasise noise, dark-current variations and higher-order spectral content"

Food for thought.  Tons of interesting stuff in there.

-Tom

--
Post to this group at plots-sp...@googlegroups.com
 
Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
---
You received this message because you are subscribed to a topic in the Google Groups "plots-spectrometry" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/plots-spectrometry/mFGgO5a5cDI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to plots-spectrome...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---------------------------------------
Tom Frisch  /  t...@tom11.com  /  mobile: 503.473.2801  /  www.tom11.com

[Dave Stoft]

Yes, it's good to discuss these topics and yes, an ideal flat source would not have spike or notches but also no "roll-off" at the ends of the detectable wavelengh range so there is high SNR for lower noise data. Yes, halogen and incandescent sources are "smooth" but they are not, themselves, of known output. I have a halogen with a "typical" curve but not a calibrated source which is why I used the solar spectra as it is available most anywhere and professionally measured data is readily available on the web. Solar data does work but gain-correction at 400nm and above 650nm gets a bit noisy as the solar spectrum rolls-off. However, it does show that the PLab devices are inherantly limiting at those ends anyway. Dark current tends to not be an issue with PLab as other noise sources generally swamp that error. The math can be performed at various resolutions and PLab can reach 1-2nm under good conditions but it lacks in dynamic range. All that said, performing measurements for CRI and the like becomes more a question of precision. With careful setup and some Matlab code Alan's measurement techniques could be performed using the PLab spectrometer. Would the results be a accurate as his with his using OceanOptics? Not likely. Could one get sufficient data to selectively grade sources relative to each other? Probably. Alan is right that to perform accurate, repeatable measurements with results traceable to NPL requires significant effort to characterize the system and remove sources of uncertainty -- and finally, have proof to support the measurement technique. My recent research notes (http://publiclab.org/notes/stoft/02-25-2015/plab-spectrometer-gain-correction) (http://publiclab.org/notes/stoft/03-01-2015/optimal-slit-width) (http://publiclab.org/notes/stoft/03-02-2015/cfl-cal-error-explained) (http://publiclab.org/notes/stoft/02-26-2015/plab-3-sanm-camera-focus) are directed at finding and improving those measurment limits for PLab.

On Tuesday, March 3, 2015 at 8:19:13 AM UTC-8, Tom Frisch wrote:

As far as a known "flat" source, which I believe you mean an SPD without a lot of massive spikes, Alan's chart lists a tungsten halogen bulb (or fixtures that use one) and a  candle "flame, white wax" as 100 on his scale.  

Certainly doesn't help the intensity calibration.  On that front, I wonder if comparing a few samples of the same wattage tungsten halogen bulb with a known slit width and webcam could give repeatable results?

Check out Alan's article, it has quite a bit about spectrometers and the potential pitfalls:

http://www.gtc.org.uk/media/fm/Zerb%20articles/TLCI_web.pdf

In it he states: "Since the mathematics of the TLCI-2012 makes calculations over the visible range from 380nm to 760nm in 5nm steps, it follows that the calibration must cover at least that range in 5nm steps. Typically, the responsivity is not flat, and the resulting compensation to obtain a flat response can emphasise noise, dark-current variations and higher-order spectral content"

Food for thought.  Tons of interesting stuff in there.

-Tom

[Terry Relph-Knight]

Hi Dave,

It seems to me the problem with using published curves for sunlight is that such curves are only illustrative and not definitive. I suspect that if you lined up half a dozen different commercial and calibrated spectrometers (which would probably have slightly different optics - different slit widths for example) you would likely get six curves that were not identical. Similarly if you plot curves for sunlight using the same spectrometer, but at different times of day, you get different curves.

I don't see how you can regard sunlight curves taken from the internet as somehow being "of known output". Or have you perhaps gathered sunlight curves from multiple internet sources and found that they are all the same?

I don't disagree with your method of it being possible to produce an amplitude correction for the PL spectrometer, but for it to be useful you have to be sure that it's possible to provide a global correction in Spectral Workbench that will always give more accurate results than obtained without the correction. Crudely speaking, if without correction measurement error is minus 20% and by applying (over) correction the error becomes plus 20% then the "correction" isn't helping.

It seems to me the only way to proceed with producing any error correction is if you can prove that the correction will improve the accuracy for all and any builds of the Public Lab spectrometer design (as you say in your reference to Alan's work -  and finally, have proof to support the measurement technique). And the only way to do that in this case is to compare results against a known reference.

To produce a proven amplitude correction with a known degree of improvement I think you would either have to take simultaneous measurements of the same broad spectrum light source (lets say this would be daylight) using a high quality commercial spectrometer with known calibration and perhaps half a dozen examples of the PL spectrometer. Either that or use a calibrated laboratory wideband light source with a known response and capture spectra with half a dozen PL spectrometers. Then you could produce an averaged correction that would give you the most accurate and consistent results across all six (or however many PL specs you used).

[Jeffrey Warren]

Great discussion here and I want to remind all that we're just beginning a major rewrite of Spectral Workbench, so if a clear methodology can get us closer to absolute amplitude calibration, that can definitely be a feature in the new version, and we should make the interface for doing that as easy as possible.

Likewise for nonlinear calibration.

Dave I saw your excellent suggestions for SW 2.0 and am working on some boring upgrades before moving on to the good stuff, so hang in there. Thanks!!!

--

Post to this group at plots-sp...@googlegroups.com
 
Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
---

You received this message because you are subscribed to the Google Groups "plots-spectrometry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to plots-spectrome...@googlegroups.com.

[Dave Stoft]

Terry, good observations. In general, I'm agreeing with you. As I tried to suggest, it's a question of degree. I'll try to clarify.

Solar:
Yes, the solar spectra (at the earth surface) does vary with time of day and time of year. However, spectra of clear mid-day sun are generally close to 5600K so the over-all curve shape for those conditions does look nearly identical in solar plots for that temperature. (I  extracted the plot from a research paper on the effects of humidity on solar spectra and used their 5780K plot data.) Now, I also ignored, and they did not plot, the zillions of small absorption 'holes' in the solar spectra -- simply because the PLab device is not capable of that resolution. Is there an exact match between the 5780K solar plot and the solar energy I was observing via clear, mid-day sun? Yes. Was that difference substantial? No. Why? Coffee ready ... read on.

Silicon:
The webcam image sensor is silicon based which means it has an inherent bandwidth limitation. It also has the much-criticized Bayer filter and JPG lossy format for converting the RGGB data to RGB pixel information. The Bayer filter has a more dramatic effect on flatness than the silicon (you can find the curves on the web) but the Bayer filters also cut-off sensitivity on either end of the spectrum as well. So, the first point is to accept that possibly-valid data, and calibrations, are limited to 400-650(maybe 700)nm span. While you might get some response to light at wavelengths outside that range, the limited dynamic range of the camera will prevent any usable gain calibration beyond these limits.

Bayer:
If you look at the RGB filter curves for digital cameras, they all look the same -- or at least really close. Yes, my DSLR has much better defined curves, but they are essentially the same peak wavelengths and same relative peak ratios and same Q (kind of like sharpness) and similar cross-over effect between them. Now, the summation of these filters will produce a smoother and a little less 'peaked' response than the web-cam but they all have something in common -- the accumulative response of the digital camera (any type) is of a broad-peaked bandpass filter with steep sides at 400nm through 650-700nm. Yes, there is a small added 'broad lump' in the 700+nm region because the IR filter was removed from the PLab camera but that effect is small.

Gain:
We don't have a case where camera and source are both close to flat so we need only "smooth out" some ripples by finding the difference. It would be great to have a readilly available, flat source but at least mid-day sun is likely more similar than comparing incandescent lights with no cal curve. I do have a Solux 4700K halogen and their 'typical' curve. It is at least smooth, but again no cal curve. With wavelength, calibration is mostly 1) associating pixel-span with wavelength span and 2) correcting for specific dispersion related to diffraction angle. With amplitude, it is really gain-correction to offset this huge effect in the sensitivity curve of silicon + Bayer filter. Since we know that virtually all PLab spectrometers have this broadband RGB-Bayer curve to their sensitivity, virtually any correction that offsets that sensitivity curve with the opposite (higher gain at the ends and broad dip in the middle) is going to help flatten the PLab sensitivity curve -- it's not going to make it worse. Yes, it will increase the effect of noise at the ends and, since that is noise, it should probably be averaged as only the average carries information. The question is to what degree; i.e. how accurate, what's the error and how individual does it need to be.

Standards:
Ok, let's consider the opposite end -- traceability to NPL or NIST. Yes, that is what Alan needed/wanted to do. NIST/NPL use independent methods to painstakingly measure response at a series of carefully configured wavelengths and could provide sensitivity vs wavelength plus the associated measurement error. True, that is the ultimate and I've had such measurements and calibrations performed for other devices but PLab is a long way from that world. Fortunately, there's a continuum of accuracy values and room for improvement with the present PLab 3.0 device and SWB which does not require nearly as much cost / energy / equipment.

Meanwhile.....
Ok Jeff, back to PLab reality. As we know, the webcam spectrometer is not flat, the average user might do cal but may not use the 546nm peak because they cannot even resolve the double-green peak as other then one broad line. So, step one is to make sure devices are stable enough to resolve the double-green and can/will select the 546nm (#2) peak. Next, retrofit SWB with the ability to accept 1600pix line data and then include the better calculations for linear dispersion so the wavelength cal is accurate. Then, massage the pix-to-wavelength data to uniform wavelength values so comparing plots (inside or outside of SWB) is correct. ( Since the device resolution is between 1-2nm at very best, 0.5nm increments would be sufficient as data from older 640x480 devices would require interpolation anyway.) Finally, apply a nominal gain-correction which is representative of the average webcam device sensitivity and maybe ... provide the user an option for taking a solar reference (but asking the user to learn how to do this could be very difficult and, instead, add huge errors if done wrong).

Accuracy:
Ok, you might have spilled you coffee over that last one. Sorry 'bout that, but, in the face of the present overwhelming unflatness of the PLab detector, almost any 'nominal' correction could be of help. The point of having a first-order gain-correction built-in to only to offset the huge non-flatness of the existing device. The "trick" here is not to lie -- but users are already being misled just by the complexity of making spectral measurements with this 'magic box'. They see a spectra and, naturally, assume it's precisely what it is -- but it isn't. So, providing even a 'nominal' gain correction is an improvement. There was a recent discussion on-line here about a spectral curve of an LED source and the complaint that the PLab curve didn't look enough like the web-published curves. I believe a 'nominal-cal' would help. Next comes the associated flatness spec. Terry, one approach, similar to what you suggested, is to apply the gain-correction curve and then make many measurements of the same source and look for the variations. There is also the device-to-device repeatability error so a number of devices, all built the same way, could take spectra for comparison. The result of all of this will still likely be significant error, but I'm suggesting that is better than "unknown" which is what the accuracy spec is now. Also, remember that we're discussing flatness accuracy, not absolute sensitivity nor relative measurement precision or repeatability.

Future Specs?:
So, what might the PLab 3.X device have for specs? I'll toss out some possible numbers: Detection Bandwidth: 380nm-750nm, Usable Bandwidth: 400-650nm, Resolution: 2nm FWHM, Flatness: +/- 10% (450-600nm), +/- 25% (400-650nm), Dynamic Range: 30dB. Limited? Yes, but an improvement and added value can clarity for users.

Cheers,
Dave

[Jeffrey Warren]

Cool, so just posted this: https://github.com/publiclab/spectral-workbench/issues/104 (upgrade capture resolution to camera max #104)

and I like the idea of an "nominal absolute amplitude calibration" -- what could we call it? "Adjust ...." and have a notice something like:

"This will not make your device truly absolutely brightness calibrated, but will help to offset the color sensitivity variability of your webcam and ease comparisons. Click here to learn more about absolute amplitude calibration."

?

--

Post to this group at plots-sp...@googlegroups.com
 
Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
---

You received this message because you are subscribed to the Google Groups "plots-spectrometry" group.

To unsubscribe from this group and stop receiving emails from it, send an email to plots-spectrome...@googlegroups.com.

[Dave Stoft]

I think a button might be [GainComp] for gain compensation with some note like you suggest.

Also, since using such a feature will likely appear to add noise at the ends (and the ends, of necessity, will be less and less accurate the further out your go) there are two other related considerations: 1 - Include some averaging (at least at the ends) just to dampen the noise and 2 - consider adding a couple vertical-line markers to show the effective bandwidth of the compensation and help the user understand the limitations.

[Terry Relph-Knight]

Dave & Jeff,

This is all starting to sound like the next version of SW could be a huge step forward -

1/. Greater resolution along the horizontal axis

2/. A calibration procedure that allows for the double green peak

3/. Perhaps some correction for the 'radial to flat' distortion that occurs when the light from the grating hits the flat plane of the camera sensor

4/. Amplitude correction to flatten the frequency response

If at all possible I'd like to see some way to allow using camera control software within SW. I can flip back and forth now from the control software to SW, but it is a bit clumsy.
I think camera settings should be born in mind when attempting the above tweaks. The camera settings need to be in a known and specific state. Auto WB off and so on. Maybe there should be a warning note that appears in SW?

[Dave Stoft]

Terry, yes, good reminders....

The Syba webcam was not controllable (that I found, anyway) and so it ran with AGC open-loop. Since the total amount of light from just the spectra in the black field, video gain went to max automatically and the images, and thus spectra, had a lot of unnecessary noise. With the TLapse interface, some parms can be controlled with the newer Sanm camera so it would be great to have SWB set some basic conditions (pixel resolution and depth,WB, Gama, exposure, etc.) which will remain stable throughout the measurement. Nothing like a device wandering around under it's own whim to make it harder to get meaningful data. I'm also of the view that the only setting which might be allowed under user control would be exposure -- but even that would be nice to have S/W control to later add HDR for improved dynamic range.

[Jeff Hecht]

To elaborate a bit on Dave's comments on variations in the solar spectrum -- an important thing to remember is the difference between the solar spectrum in space and the solar spectrum on the ground, which is affected by light scattering and absorption in the atmosphere. The solar spectrum in space is fairly constant, but the atmospheric effects vary with the angle of illumination and the state of the atmosphere.  The end result is quite a variation in the quality of sunlight during the day; it is most blue at midday, but as the sun dips lower more and more of the blue light is scattered, so daylight becomes redder. The human visual system has evolved to adapt to this variation, so we don't notice the gradual change. I found an interesting web site that shows some of the changes at http://www.geog.ucsb.edu/ideas/Insolation.html

What that means for calibrating the spectrometer is that it gets very tricky for sunlight, but is easier with artificial lighting, where you can store spectra of some common light sources, or provide ways of recording the light source spectrum and then comparing it with reflected or transmitted light. -- Jeff Hecht

[Dave Stoft]

Jeff, yes, that's true. The solar spectra does shift -- some with altitude, some with humidity, quite a bit with time of day/year where the deg-K shift is high. However, relative to the gross sensitivity vs wavelength curve of the webcam, using clear, mid-day sun it may be possible to measure a spectra that is quite close to a published curve obtained under those same conditions. Yes, a known (professionally measured or calibrated) broadband source would be preferable. Any suggestions? The closest I've found was a Solux 4700K Halogen with a published 'typical' curve. The concept is that a one-time gain correction of the basic webcam sensitivity curve might provide a significant improvement to the now rather nonuniform spectra from PLab spectrometers. Both sunlight and artificial light are tricky; the atmosphere is variable and so are manufacturing specs, voltage, ambient temp, etc. A transfer standard would be easier -- measure the same source with both a PLab 3.X and an expensive processional spectrometer and then extract the data to generate the correction curve. Yes, the resulting spec must be downgraded from the professional device but even that would be significantly better then the present 'unknown'.

[Terry Relph-Knight]

"A transfer standard would be easier -- measure the same source with both a PLab 3.X and an expensive processional spectrometer and then extract the data to generate the correction curve."

Well this is the point I have been making all along. Ideally any calibration correction needs to be proven to be effective and remaining errors need to be quantified. To do that requires either a commercial spectrometer or standard light source with a known calibration and simultaneous tests carried out on a sample group of PL spectrometers. Probably even recursive tests and adjustments would need to be made.

[Dave Stoft]

Right ... though the devil is in the details. Right now, the PLab devices are too unstable to be worthy of too much effort / expense of comparison against a commercial spectrometer so, although the theory is right, the practical aspects still trump. It's still a matter of degree. A commercial device can have sub-nm resolution, 80dB of dynamic range and have been corrected to within 1% over 350-1200nm. It would certainly work as a transfer standard. No argument there, but it's expensive and not readily available to the general user. I'm not in any disagreement with what you propose. Instead, I'm looking for improvement increments that are more accessible and could make the PLab device much closer to qualifying as a viable tool. When even CFL spectra are frequently inconsistent and of low resolution, collected data is very suspect at best.

I'm working on a technique using a relatively inexpensive source for which I believe I can get reasonably accurate output data. If so, it would allow gain-correction to be a reproducible process; going a long way toward verification as well as improving the flatness spec from "unknown" to something maybe better then 25%. At each step, the specs can get revised as the errors are reduced. I'm in agreement with you but so far, no specs have been written; I think just obtaining sufficient measurements to have specs would be a good step.

[Jeff Hecht]

Dave -
I agree a commercial spectrometer could be useful as a transfer standard. Has anybody studied the instabilities in the PLab spectrometers carefully enough to quantify them and their causes? (I  lurk on the list but don't read all of the discussions here, so I easily could have missed it.) It might also help to pin down wavelength-stable sources that are readily available.

The only inexpensive laser sources I know that should be wavelength-stable are green laser pointers, because the green light at 532 nanometers comes from frequency-doubling of a narrow near-infrared line of neodymium at 1064 nm. Red and violet (nominally 405 nm) pointers use diode lasers that can vary more in wavelength, both because of manufacturing differences and because of shifts during operation (the wavelength of diode lasers is temperature sensitive).  The best wavelength-stable laser sources in the red are helium-neon gas lasers, but they're considerably more expensive. (You need a power supply as well as a tube.)

I don't know the nature of the wavelength variability in fluorescent bulbs, so I can't compare it to that of the red and violet laser pointers. It would be worth measuring those variations with a professional instrument, to see how significant they are for use with the PLab spectrometers.

-- Jeff Hecht

[Dave Stoft]

I believe the major instabilities come from mechanical design (slit/grating/camers not mechanical rigid configuation), camera settings and noise. These cam be improved. As for spectral line source stability, I think that even the CFL will probably always be better then the PLab device just because several of the CFL lines are very stable and very narrow and the best PLab FWHM resolution is typically 1-2nm (the CFL lines are something like <0.05nm FWHM). Lasers and laser diodes are quite stable but laser diodes have much wider FWHM.