do the spectra of different oil samples differ enough to make a comparison to an unknown sample

[Benjamin Sugar]

Odd subject heading but what're gonna do?

Anyway.

I'm gong to be setting up a workshop using the oil testing kits.  The workshop that was recently posted as so helpful, thank you to all who put that together.

I'd like to give participants a fictional scenario in which they'll try to match oil recovered from an area with a oil samples from known suppliers using the sample packs that can be purchased.  Sort of a whodunnit.  Again.  A completely fictional ploy to give us an opportunity to make comparisons. 

However, the image of the spectra captured at the following link, (the image where all of the tested samples are overlayed) don't look very discernibly different from one another.

page: http://publiclab.org/wiki/oil-testing-kit
image: http://i.publiclab.org/system/images/photos/000/005/453/original/Screen_Shot_2014-07-23_at_12.04.33_AM.png

when not overlayed, are they different enough that given mystery samples, and known samples, it would be possible to discern the difference between oils?

[Yagiz Sutcu]

Hi Ben,

I was planning to provide some feedback on oil testing kit page Jeff created but could not find some free time to do it yet, sorry Jeff :(

But I guess, you need some answers quickly...So, let me try to help:

Fluorescence spectroscopy (like used in oil testing kit) generates a peak at laser's wavelength unlike absorbance spectroscopy which uses broadband light source. Therefore, peak height normalization is much more suitable for comparison purposes (rather than area under the curve). So, once you have a set of spectra obtained from different oil samples, you should be able to see the difference after peak height normalization.

But, if you want the create an algorithm to give you the answer (type of the unknown oil), you need to do little more math :) Basically, you need to have reference spectra from oils you have and a matching algorithm to match the unknown sample spectra to one of the known sample. Find similar tool may work in this case. (But keep in mind that things get little complicated when oil concentration also a non-controllable variable)

I hope this helps little bit.

Cheers,
Yagiz

--
Post to this group at plots-sp...@googlegroups.com
 
Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
---
You received this message because you are subscribed to the Google Groups "plots-spectrometry" group.
To unsubscribe from this group and stop receiving emails from it, send an email to plots-spectrome...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Benjamin Sugar]

thanks yagiz.

i only need people to compare them by eyes, not by computer.

what is this peak height normalization you speak of?

do you know of a place on the wiki where you can find info on it?

[Yagiz Sutcu]

Sorry for the late reply Ben,

It is basically dividing whole spectra by the peak height.

So, you determine the value at the laser peak for the spectra and divide whole spectra by this value. Unfortunately, you cannot do it using Spectral Workbench but you can download the data and play with using Excel or any computation software.

Let me know if you need any help.

Yagiz

[Benjamin Sugar]

would it be possible to just have images of different spectra and have people compare the images they getting using the spectral workbench.  I wan't to keep it as simple as possible for people.  I won't have time to educate on any math at the workshop.

also, is the peak height the identifier then?

[Jeffrey Warren]

Yagiz - do you think the laser peak height is going to be linearly related to the rest of the curve? I worry that it's *so* bright that it's clipping in the blue channel, as in this image from http://spectralworkbench.org/analyze/spectrum/14919

Benjamin - I think the big issues we face right now are:

* getting enough light in so that the curve is closer to 25-75% brightness, so it's not overwhelmed by noise

* adjusting the height of the curves to be ​comparable, by algorithm or by visual comparison, which gets at Yagiz's distinction between area and height, and my post about it: http://publiclab.org/notes/warren/07-30-2014/equalizing-area-of-spectral-graphs-for-comparison

On Mon, Aug 11, 2014 at 11:50 AM, Benjamin Sugar <goo...@skilfullycurled.org> wrote:

would it be possible to just have images of different spectra and have people compare the images they getting using the spectral workbench.  I wan't to keep it as simple as possible for people.  I won't have time to educate on any math at the workshop.

also, is the peak height the identifier then?

--

[Dan Beavers]

Perhaps another way to think about it is to consider the red channel an
attenuator and normalize all channels based on the red channel amplitude
at the laser frequency.

On 2014-08-11 12:56, Jeffrey Warren wrote:
> Yagiz - do you think the laser peak height is going to be linearly
> related to the rest of the curve? I worry that it's *so* bright that
> it's clipping in the blue channel, as in this image from
> http://spectralworkbench.org/analyze/spectrum/14919
>
>
>

> Benjamin - I think the big issues we face right now are:
>
> * getting enough light in so that the curve is closer to 25-75%
> brightness, so it's not overwhelmed by noise
> * adjusting the height of the curves to be ​comparable, by algorithm or
> by visual comparison, which gets at Yagiz's distinction between area and
> height, and my post about it:
> http://publiclab.org/notes/warren/07-30-2014/equalizing-area-of-spectral-graphs-for-comparison
>
>
>
>
>
>
>
>
>
> On Mon, Aug 11, 2014 at 11:50 AM, Benjamin Sugar
> <goo...@skilfullycurled.org <mailto:goo...@skilfullycurled.org>> wrote:
>
> would it be possible to just have images of different spectra and
> have people compare the images they getting using the spectral
> workbench. I wan't to keep it as simple as possible for people. I
> won't have time to educate on any math at the workshop.
>
> also, is the peak height the identifier then?
>
> --
> Post to this group at plots-sp...@googlegroups.com

> <mailto:plots-sp...@googlegroups.com>

>
> Public Lab mailing lists (http://publiclab.org/lists) are great for
> discussion, but to get attribution, open source your work, and make
> it easy for others to find and cite your contributions, please
> publish your work at http://publiclab.org
> ---
> You received this message because you are subscribed to the Google
> Groups "plots-spectrometry" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to plots-spectrome...@googlegroups.com

> <mailto:plots-spectrome...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
>

> --
> Post to this group at plots-sp...@googlegroups.com
>
> Public Lab mailing lists (http://publiclab.org/lists) are great for
> discussion, but to get attribution, open source your work, and make it
> easy for others to find and cite your contributions, please publish your
> work at http://publiclab.org
> ---
> You received this message because you are subscribed to the Google
> Groups "plots-spectrometry" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to plots-spectrome...@googlegroups.com

> <mailto:plots-spectrome...@googlegroups.com>.

[Jeffrey Warren]

ah, that's really clever, Dan! Yagiz, do you have a reference we can look at for why total peak height is more appropriate than area under curve? Thanks!

On Mon, Aug 11, 2014 at 2:16 PM, Dan Beavers <dan.b...@acm.org> wrote:

Perhaps another way to think about it is to consider the red channel an attenuator and normalize all channels based on the red channel amplitude at the laser frequency.

On 2014-08-11 12:56, Jeffrey Warren wrote:

Yagiz - do you think the laser peak height is going to be linearly
related to the rest of the curve? I worry that it's *so* bright that
it's clipping in the blue channel, as in this image from
http://spectralworkbench.org/analyze/spectrum/14919

Benjamin - I think the big issues we face right now are:

* getting enough light in so that the curve is closer to 25-75%
brightness, so it's not overwhelmed by noise
* adjusting the height of the curves to be ​comparable, by algorithm or
by visual comparison, which gets at Yagiz's distinction between area and
height, and my post about it:
http://publiclab.org/notes/warren/07-30-2014/equalizing-area-of-spectral-graphs-for-comparison









On Mon, Aug 11, 2014 at 11:50 AM, Benjamin Sugar

<goo...@skilfullycurled.org <mailto:google@skilfullycurled.org>> wrote:

    would it be possible to just have images of different spectra and
    have people compare the images they getting using the spectral
    workbench.  I wan't to keep it as simple as possible for people.  I
    won't have time to educate on any math at the workshop.

    also, is the peak height the identifier then?

    --

    Post to this group at plots-spectometry@googlegroups.com

    <mailto:plots-spectometry@googlegroups.com>

    Public Lab mailing lists (http://publiclab.org/lists) are great for
    discussion, but to get attribution, open source your work, and make
    it easy for others to find and cite your contributions, please
    publish your work at http://publiclab.org
    ---
    You received this message because you are subscribed to the Google
    Groups "plots-spectrometry" group.
    To unsubscribe from this group and stop receiving emails from it,

    send an email to plots-spectrometry+unsub...@googlegroups.com

    <mailto:plots-spectrometry+unsub...@googlegroups.com>.

    For more options, visit https://groups.google.com/d/optout.


--

Post to this group at plots-spectometry@googlegroups.com

Public Lab mailing lists (http://publiclab.org/lists) are great for
discussion, but to get attribution, open source your work, and make it
easy for others to find and cite your contributions, please publish your
work at http://publiclab.org
---
You received this message because you are subscribed to the Google
Groups "plots-spectrometry" group.
To unsubscribe from this group and stop receiving emails from it, send

an email to plots-spectrometry+unsub...@googlegroups.com

<mailto:plots-spectrometry+unsub...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
Post to this group at plots-spectometry@googlegroups.com

Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
--- You received this message because you are subscribed to the Google Groups "plots-spectrometry" group.

To unsubscribe from this group and stop receiving emails from it, send an email to plots-spectrometry+unsub...@googlegroups.com.

[Yagiz Sutcu]

Actually, it is not more important than AUC, it's another way to compare different spectra. And more suitable if you have a characteristic peak at some wavelength independent of the sample.

For the example image you mentioned, you can play with webcam output rather than one of the RGB channels. Once you divide whole spectra by the peak height, all spectra (you want to compare) will have a unity peak at laser's wavelength and the remaining part of the spectra will be more easy to visually compare. (Here we talk about quantitative comparison, not any mathematical model)

You can play with RGB channels as well and in this case, spectral workbench is not suitable for visualization since it cuts out anything larger than 1. You have do it in another platform.

Yagiz

On 8/11/2014 2:36 PM, Jeffrey Warren wrote:

ah, that's really clever, Dan! Yagiz, do you have a reference we can look at for why total peak height is more appropriate than area under curve? Thanks!

On Mon, Aug 11, 2014 at 2:16 PM, Dan Beavers <dan.b...@acm.org> wrote:

Perhaps another way to think about it is to consider the red channel an attenuator and normalize all channels based on the red channel amplitude at the laser frequency.

On 2014-08-11 12:56, Jeffrey Warren wrote:

Yagiz - do you think the laser peak height is going to be linearly
related to the rest of the curve? I worry that it's *so* bright that
it's clipping in the blue channel, as in this image from
http://spectralworkbench.org/analyze/spectrum/14919

Benjamin - I think the big issues we face right now are:

* getting enough light in so that the curve is closer to 25-75%
brightness, so it's not overwhelmed by noise
* adjusting the height of the curves to be ​comparable, by algorithm or
by visual comparison, which gets at Yagiz's distinction between area and
height, and my post about it:
http://publiclab.org/notes/warren/07-30-2014/equalizing-area-of-spectral-graphs-for-comparison









On Mon, Aug 11, 2014 at 11:50 AM, Benjamin Sugar

<goo...@skilfullycurled.org <mailto:goo...@skilfullycurled.org>> wrote:

    would it be possible to just have images of different spectra and
    have people compare the images they getting using the spectral
    workbench.  I wan't to keep it as simple as possible for people.  I
    won't have time to educate on any math at the workshop.

    also, is the peak height the identifier then?

    --

    Post to this group at plots-sp...@googlegroups.com

    <mailto:plots-sp...@googlegroups.com>

    Public Lab mailing lists (http://publiclab.org/lists) are great for
    discussion, but to get attribution, open source your work, and make
    it easy for others to find and cite your contributions, please
    publish your work at http://publiclab.org
    ---
    You received this message because you are subscribed to the Google
    Groups "plots-spectrometry" group.
    To unsubscribe from this group and stop receiving emails from it,

    send an email to plots-spectrome...@googlegroups.com

    <mailto:plots-spectrome...@googlegroups.com>.

    For more options, visit https://groups.google.com/d/optout.


--

Post to this group at plots-sp...@googlegroups.com

Public Lab mailing lists (http://publiclab.org/lists) are great for
discussion, but to get attribution, open source your work, and make it
easy for others to find and cite your contributions, please publish your
work at http://publiclab.org
---
You received this message because you are subscribed to the Google
Groups "plots-spectrometry" group.
To unsubscribe from this group and stop receiving emails from it, send

an email to plots-spectrome...@googlegroups.com

<mailto:plots-spectrome...@googlegroups.com>.

For more options, visit https://groups.google.com/d/optout.

--
Post to this group at plots-sp...@googlegroups.com

Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
--- You received this message because you are subscribed to the Google Groups "plots-spectrometry" group.

To unsubscribe from this group and stop receiving emails from it, send an email to plots-spectrome...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Post to this group at plots-sp...@googlegroups.com

 
Public Lab mailing lists (http://publiclab.org/lists) are great for discussion, but to get attribution, open source your work, and make it easy for others to find and cite your contributions, please publish your work at http://publiclab.org
---
You received this message because you are subscribed to the Google Groups "plots-spectrometry" group.

To unsubscribe from this group and stop receiving emails from it, send an email to plots-spectrome...@googlegroups.com.

[Jeffrey Warren]

Ah, for comparison, i just created a macro: http://spectralworkbench.org/macro/warren/autosize_y

It can be run in the Analyze page to resize the y-axis. 

[Benjamin Sugar]

Am I understanding this correctly (doubtful by here goes):

Peak height is associated with fluorescence spectroscopy not absorbance spectroscopy.

The peak that you see is from the laser.  Everything after the peak is produced by the excited materials in the sample. (source: Oil Testing Kit page)  

Samples are compared by area under the curve.  The peak height of the laser can throw off the area under the curve as it is independant of the fluorescence from the sample.

In order to a compensate, you can normalize the data using peak height normalization or using the red channel as an attenuator.

Follow up question to Jeff's note: http://publiclab.org/notes/warren/07-30-2014/equalizing-area-of-spectral-graphs-for-comparison

When you say the y-axis doesn't matter anymore, you're referring to the y-axis of the peak, right?  Presumably it matters for the area under the curve?

[Yagiz Sutcu]

Sorry I read as "important" instead of "appropriate" :(

For example Figure 9 in the following document is a simple illustration of the idea:

http://www.lifetechnologies.com/us/en/home/references/molecular-probes-the-handbook/introduction-to-fluorescence-techniques.html
 
Also Figure 9 (at page 16) of this one:

http://www.opticsinfobase.org/view_article.cfm?gotourl=http%3A%2F%2Fwww.opticsinfobase.org%2FDirectPDFAccess%2FC26F3F66-920D-668A-E3E4F286FAB7D8E6_199776%2Foe-18-12-12436.pdf%3Fda%3D1%26id%3D199776%26seq%3D0%26mobile%3Dno&org=

If the peak height at some carefully-selected wavelength is not changing with concentration of sample or absorption properties of the sample, it works quite well for visual comparison. Of course, it may not work all the time especially if samples coming from different devices with different calibrations etc.

Yagiz

[Yagiz Sutcu]

I tried to answer in-line below in red.
yagiz

On 8/11/2014 4:24 PM, Benjamin Sugar wrote:

Am I understanding this correctly (doubtful by here goes):

Peak height is associated with fluorescence spectroscopy not absorbance spectroscopy. YES

The peak that you see is from the laser.  Everything after the peak is produced by the excited materials in the sample. (source: Oil Testing Kit page)  YES

Samples are compared by area under the curve.  The peak height of the laser can throw off the area under the curve as it is independant of the fluorescence from the sample. Let's say "shape" rather than area. This is for visula examination. You may have very differet shape but same area. To be able to use appropriate metric for comparison (mathematically), little more analysis required.

In order to a compensate, you can normalize the data using peak height normalization or using the red channel as an attenuator.

Follow up question to Jeff's note: http://publiclab.org/notes/warren/07-30-2014/equalizing-area-of-spectral-graphs-for-comparison

When you say the y-axis doesn't matter anymore, you're referring to the y-axis of the peak, right?  Presumably it matters for the area under the curve?

[Jeffrey Warren]

> When you say the y-axis doesn't matter anymore, you're referring to the y-axis of the peak, right?  Presumably it matters for the area under the curve?

For this, i just meant that since you divide by the area under the curve, your y-axis is only plotting a ratio, with no absolute units. So the scale bar can be unlabeled. But the values are still important - they're just relative only, not absolute units.