Blind Calibration and measurement theory

922 views
Skip to first unread message

NeilH

unread,
Jun 4, 2012, 1:57:31 PM6/4/12
to airqua...@googlegroups.com
I’m starting a new thread on something David Holstius referenced “Blind Calibration” (BC) in  “Calibration Gas (NO2) sensors”
The reference is 

And I posted what I decoded from the paper and then thought it would be a good opportunity to discuss it on a separate thread, as it is the most coherent framework that I have heard of so far for .

It seems to me though that Balzano (UCLA)  and Nowak(U.Wisc) haven’t covered the fundamentals  of measurement theory.

So I’d like to explore my understandings of it here, and if anybody has some thoughts to jump in.

The BC use temperature, and so I would like to start with that – since everyone has a “feeling” for temperature and at the same time has the experience of translating between a number of world views of temperature (Fahrenheit Celsius Kelvin).
There are a number of references scales of temperature Fahrenheit, Celsius/Centigrade and Kelvin.
They all have a direct linear relationship to each other, and all have been successful in their own domain.
The Fahrenheit scale evolved by German physicist Daniel Fahrenheit (1686-1736)– proposed in a paper in 1724 was based on his perception of calibration points.
The Celsius scale named after Aders Celsius (1701-1744) used an SI notation of 100 divisions between freezing point of water and boiling point of water.
The Kelvin scale named after William Thomson Kelvin (1824-1907) references an absolute zero, and a unit of degree Celsius.
The absolute zero is equivalent to -273C.
 (details from wikipedia.org/wiki/Kelvin)

Subsequent analysis has resulted in refinement of the definitions, and national standards to maintain those definitions to the degree of accuracy required by scientific (& economic) standards .

So my understanding is that fundamentals of a scale are defined. 
They could be anything – but a system is chosen and widely/taught.

In the case of temperature- the core definitions is two physical transition states of water  – boiling point of water, melting point of pure ice – and a scale defined with a 100 units called Celsius between the two points. 
Measuring temperature is so economically & scientifically valuable – and a fundamental energy property of the universe that all sorts of methods of been discovered to transform the real world energy to a measurable temperature scale. 
From expansion of alcohol (and mercury) in a narrow capillary tube, to properties of materials (thermistors, thermocouples) that can be measured electrically.  
As modern history shows it doesn’t matter too much what scale is used (Fahrenheit in the public forums of the US – Celsius in the rest of the world) – so long as everyone understands what scale is being used and when there is a need to translate.
An alcohol thermometer doesn’t have a natural relationship to freezing point of water – it has a constructed relationship – usually over a scale that its target market wants. Eg healthcare
A thermistor is a type of resistor who’s resistance varies significantly with temperature – and is engineered and characterized to have electrical properties that can then be mathematically transformed to units of temperature – your choice.

What I’m detailing for temperature is there is traceability back to the two fundamental transition points of water, and a arbitrary scale of 100units between those points

Now looking at another common set of measurements – length  – say 1meter or 1foot – a ruler, or tape measure – they all essentially have two points  - a beginning – ‘0’ units – and an end. 1m or  1ft. They have units to make it useful. Metric or imperial.

Now coming to an Air Quality parameter – a specific gas (eg NO2) the units appear to be natural units defined by atomic relationships – Avogadro constant (http://en.wikipedia.org/wiki/Avogadro_constant)
Eg for NO2 with a range of interest between 0ppb and 250ppb for any given volume
So the molecular science for a constant 1 liter (or scaled appropriately)  there would be  6.02214X×10**23. Molecules (or elementary entities of Synthetic Air)
So for 0 ppb NO2 – or possibly including measurement error purpose less than 0.5ppb of NO2 – 0.5*6.02214X×10**14. 
So for there to be 250ppb of NO2 in 1L this would be 250*6.02214X×10**14

If we could snapshot a volume and then have an molecular counter – then it would be possible to directly relate counted molecular in a given volume to  the parts per billion in that volume.

However the MiCS2710/2714 innovative low cost NO2 sensor - defines that it has a physical property  of resistance that is proportional to the ratio of NO2 on its surface best measured at 220C  – and further more it explicitly states for absolute measurements, interpret from my resistive scale at least two measurements. 
One synthetic air – giving a 0ppb on its scale
And another one of the designers choice – say 100ppb or 250ppb.

So coming back to blind calibrations from raw sensors – I think the papers are missing a fundamental description of measurement theory. 

I’d really like to hear from anybody investing the computational side  if I have missed something.

Cesar Garcia

unread,
Jun 4, 2012, 5:54:51 PM6/4/12
to airqua...@googlegroups.com
NeilH,

I truly appreciate your insights about these topics. I will try to
investigate them further.

Best,
César
--
Cesar García - @elsatch

Ando con encolamiento para responder correos y los proceso lunes,
miércoles y viernes. Si es algo urgente/rápido contáctame por Twitter.
Gracias!

David Broadwell

unread,
Jun 6, 2012, 9:32:02 PM6/6/12
to airqualityegg
I don't have the time for a lengthy reply now but I am an engineer at
a calibration lab, and might be able to offer some insights into
scaled sensor deployments ... I will read that paper in more detail
and see what pops out.

On Jun 4, 1:57 pm, NeilH <neil...@sonic.net> wrote:
> I’m starting a new thread on something David Holstius referenced “Blind
> Calibration” (BC) in  “Calibration Gas (NO2) sensors”
> The reference ishttp://sunbeam.ece.wisc.edu/publications/bcbook.pdf

Cesar Garcia

unread,
Jun 7, 2012, 3:37:51 AM6/7/12
to airqua...@googlegroups.com
It might be possible to find additional information around this topic,
including some background for neophites in Laura Balzano thesis:

http://sunbeam.ece.wisc.edu/publications/balzanomsthesis.pdf

There is also a paper called System-level Calibration for Data Fusion
in Wireless Sensor Networks
(http://www.cse.msu.edu/publications/tech/TR/MSU-CSE-09-5.pdf) that
includes this information in the intro:

Balzano et al. [3] theoretically prove that the sensors can be
partially calibrated using blind measurements. However, the blind
approaches often require that the deployment is dense enough [3, 4].
As a compromise, the semi-blind calibration approaches [15, 20, 21]
require partial ground truth information. In [15, 21], the sensor
locations are calibrated using accurate or coarse position information
of a subset of nodes. In [20], an uncalibrated sensor calibrates
itself when rendezvousing a calibrated sensor. The calibration
approach presented in this paper falls into the semi-blind category,
in
which sensors can calibrate their energy measurements as long as the
physical position (instead of the accurate energy profile) of the
target is known


I also found interesting information in the procedings of this year
Wireless Sensor Network Conference of IEEE. There two particularly
interesting articles On the Fly Calibration of Low Cost Gas Sensors
(see pg 241 on Google Books preview) and Energy-Aware Gas Sensors
Using Wireless Sensor Networks (for cool optimization techniques)

http://books.google.es/books?id=uXRGyfExmzsC&pg=PA228&dq=On-the-Fly+Calibration+of+Low-Cost+Gas+Sensors&hl=es&sa=X&ei=31bQT4W4LsGGhQeQ4-WKDA&redir_esc=y#v=onepage&q=On-the-Fly%20Calibration%20of%20Low-Cost%20Gas%20Sensors&f=false

I hope it helps!
Best,
César

David Holstius

unread,
Jun 7, 2012, 1:42:24 PM6/7/12
to airqua...@googlegroups.com
Thanks Cesar! The Balzano thesis is a great find. Looks like the full version of "On-the-Fly Calibration of Low-Cost Gas Sensors" (Hasenfratz et al 2012) is available here:


There are a couple of other interesting papers and slide decks linked from his Publications page. He seems to have done a substantial bit of work with the OpenSense project / ETH Zurich.

NeilH

unread,
Aug 7, 2012, 4:24:12 PM8/7/12
to airqua...@googlegroups.com
Since I started this thread, and there has been only a short conversation on "Blind Calibration" or "Network Calibration" and some occasional references in other threads,  I'll summarize my understanding.

There is no useful theory that anyone has put forward or a technical paper identified  that supports the concept of "Blind Calibration" for a simple stationery sensor - that is making sense out of raw data read from sensors with no calibrated source.
There appears to be a lot of wishful thinking.
David Hasenfratz's paper
discusses mobile mounted sensors WITH GPS that periodically correlate (calibrated) with a "perfect sensor".
The paper is based on ways of reducing the costs of having good confidence in low cost sensors that are periodically "automatically calibrated" by reference to close proximity to a perfect sensor. The papers have identified the "air gap" - the unknown value of the difference in measurements between the sensors as an issue.

The following is my understanding  of the basics of this project and how it applies to the sensors  Mics 2710(NO2)  Mics5525 (CO).

The desired measurement units are based on parts per million - a well understood dilution of atoms in a defined volume.
The sensors physical measurement is a wire that varies its resistance based on absorption of elements. 
These are dominated by the specific gas they are defined for - but can include other gasses. eg for NO2 Mic2710/2714 - can also indicate presence of NO and H
Typical calibration methods are "teaching the electronics" - capturing the constants for an algorithm - to translate the measured resistance in ohms to units (ppm) that are desired. 
An analogy for un-calibrating is like buying a measuring tape  which has numbers on it, but no units - ie not stated if its inches, feet, leagues, thumbs or some other unit. Typical raw sensors are non-linear, especially at the end of a specific sensors range. Sensors often have to be selected for the desired range, or batches of sensors may already have been partially filtered by the manufacturer but still fit within the original specification.
So can acquiring a 100 measuring tapes - all with different units - be processed by an algorithm to provide defined units. 

If the basic measurement technique was counting specific molecules - then there could be an algorithm from molecules counted to ppm.  

Measurements with numbers & units can be hard to relate to in a meaningful way. Temperature is a common measurement - and translations between Celsius and Fahrenheit is simple. Temperature can be HOT WARM or COLD - the numbers are different on the different scales. 

Thresholds evolve or are defined to indicate whether a number is "dangerous or acceptable" - "good or bad" - "desirable or undesirable" 
Accuracy of measurements is needed when thresholds are decisive - for whatever reason - if the threshold is only a relative indicator then the (in)accuracy of the measurement is not that important. 
For a gas NO2/CO/?? measuring device - the numbers need to map to know units for thresholds to be used to define whether the "Air Quaility" is good, poor or bad.

Typically calibration happens in a controlled environment (the factory) where it can be standardized. From a financial point of view and added value to an instrument, the volume calibration at the factory is supposed to be the cheapest place to do it. 
By calibrating and defining specifications that the instrument is supposed to meet, it means innovation can happen where there are skilled people who are used to making it happen.
IMHO, from what I've seen of this project,  its not that hard (or expensive) to provide a basic calibration of the instrument/sensor readings  at the factory.  
It may be expensive to define an instrument to fully meet defined national standards - but that is a different instrument than one that has basic calibration to measure an air quality threshold.
 
Neil

Richard Beckwith

unread,
Sep 20, 2012, 7:16:59 PM9/20/12
to airqua...@googlegroups.com

> I’d really like to hear from anybody investing the computational side  
> if I have missed something.

I think that there are two independent issues to consider here.

First, there is the issue of how much of something we have measured. This is related to knowing what the lengths the markings on your ruler are -- this is at 5 cm., that is at 10... This is the scale used in the measurements. If we have this data for every sensor in our network, we can both compare the levels across the full site and talk about how these levels compare to measurements elsewhere.

With calibrated measurements and a scale, if we are taking ambient temperature measurements, we can compare the temperature of the different locations in our network as well as how those temperatures compare to locations not in our network as long as we can translate those measurements into a shared scale.

Having a scale for our measurements that allows us to compare to others isn't the only thing that counts as calibration. In fact, it's kind of not what defines calibration. We can have (what I'll call) a scale-free calibration. (Yeah, I know that makes it sound like complexity theory but I like it.)

With the second type of calibration, we can know how the measurements of the sensors in our network compare to each other. Given a set of calibrated sensors, we can know that Location A is cooler than Location B without knowing how these temperatures compare to locations outside of our network. With this type of calibration, we have not calibrated with respect to a scale but rather with respect to the set of sensors in our network.

We can know that walking along one path will be warmer than walking along another one within our network but we won't know how this compares to a path outside our network. We won't have the convenience of a scale with which to facilitate that comparison. However, this doesn't mean it's not calibrated.

It's worth pointing out that Joe has made the interesting observation that even uncalibrated sensors can provide interesting data. If I know that some location is higher than it usually is, that can be interesting. We might even be able to use such data for point source location of pollutants. It's better if the sensors are calibrated though, because then we can do things like characterize paths through space.

In the literature, it shows that what's needed for this type of calibration is a guarantee of oversampling. That is, the data coming from the sensors need to be correlated. We don't need a scale to share with known measurements but, interestingly, one can be introduced scale into the issue by bringing into this madness a calibrated sensor with a known scale.

So, I don't think that it's true that "there is no useful theory...that supports the concept of 'Blind Calibration'" I would agree that there is no way that you can introduce a scale into that calibrated systems blindly but that is another issue.

Richard

CleanAirUK

unread,
Sep 21, 2012, 8:22:56 AM9/21/12
to airqua...@googlegroups.com
Hi,

I think the underlying issue of data quality/integrity is really quite important. As I understand it then the argument about the AQE is that there's mass deployment and so that helps in correcting inconsistencies, and the lack of calibration of the devices is not a limitation in the overall accuracy of the readings i.e you can compensate.

But, I can't see that helps in most cases. Air pollution can vary quite substantially on a street by street basis. Traffic is the cause of much pollution in urban areas and even 100 meters makes an big difference. Take a look at the slides in this presentation and you'll see one which shows what I'm talking about: http://www.cleanairuk.org/cities-for-clean-air-london-2012-ian-mudway.html (There's a also nice audio accompaniment on Slideshare via the webpage)

So, the levels of air pollution aren't constant in an area, which means that the variability of the sensors can't be factored out by making that assumption that it might be invariant in a small area with two of more AQE. Indeed, even suppose that you pulled in the data from the UK Government's 300 or so air monitoring stations (which are calibrated and maintained properly), that isn't a huge help for the reason I've given earlier. Furthermore, the sensors will degrade over time. I hope that makes sense?! Perhaps, I missing something?

I guess there is another way round this: create or use some sort of calibration service for the AQEs. Perhaps universities would have the equipment for that? So, have a certification/ calibration service and when the data logging is undertaken then record whether the AQE is certified/calibrated.

Anyhow, this is a point of discussion. Any further thoughts welcome. Thanks.

Andrew

Network for Clean Air

Twitter: @CleanAirUK
Facebook: www.facebook.com/cleanairuk
Web: www.cleanairuk.org

--------------------------
--------------------------------


Hi,

I think the underlying issue of data quality/integrity is really quite important. As I understand it then the argument about the AQE is that there's mass deployment and so that helps in correcting inconsistencies, and the lack of calibration of the devices is not a limitation in the overall accuracy of the readings i.e you can compensate.

But, I can't see that helps in most cases. Air pollution can vary quite substantially on a street by street basis. Traffic is the cause of much pollution in urban areas and even 100 meters makes an big difference. Take a look at the slides in this presentation and you'll see one which shows what I'm talking about: http://www.cleanairuk.org/cities-for-clean-air-london-2012-ian-mudway.html (There's a also nice audio accompaniment on Slideshare via the webpage)

So, the levels of air pollution aren't constant in an area, which means that the variability of the sensors can't be factored out by making that assumption that it might be invariant in a small area with two of more AQE. Indeed, even suppose that you pulled in the data from the UK Government's 300 or so air monitoring stations (which are calibrated and maintained properly), that isn't a huge help for the reason I've given earlier. Furthermore, the sensors will degrade over time. I hope that makes sense?! Perhaps, I missing something?

I guess there is another way round this: create or use some sort of calibration service for the AQEs. Perhaps universities would have the equipment for that? So, have a certification/ calibration service and when the data logging is undertaken then record whether the AQE is certified/calibrated.

Anyhow, this is a point of discussion. Any further thoughts welcome. Thanks.

Andrew

Network for Clean Air

Twitter: @CleanAirUK
Facebook: www.facebook.com/cleanairuk
Web: www.cleanairuk.org

--------------------------

Richard Beckwith

unread,
Sep 21, 2012, 6:36:52 PM9/21/12
to airqua...@googlegroups.com
> I think the underlying issue of data quality/integrity is really quite important.

Absolutely.

> So, the levels of air pollution aren't constant in an area, which means that the variability of the sensors
> can't be factored out by making that assumption that it might be invariant in a small area with two of more AQE.

Actually, this isn't quite what I said but the difference is revealing
since it implicates the substantial variability that you point to.
The way that I understand Blind Calibration to work (from my reading
of Balzano and Nowak IPSN '07) is basically that if you are
OVERsampling, you can not only accurately do interpolation (which is
why we usually oversample) but you can use the correlation between
measurements as a means of calibrating the output values. This
doesn't give you the ability to convert to a different scale (such as
a metric scale) but it does allow you to compare the values from one
device to those from another. You can do this across multiple pairs
of objects until you get a large number of devices calibrated to each
other.

The trick here is that you have to oversample. You can't simply have
"a bunch" of sensors that are spatially distributed. They need to be
correlated. In many areas, cities say, the intrasite variation
associated with air quality is such that sampling would need to be
quite dense (as you mentioned) to guarantee oversampling.

> Furthermore, the sensors will degrade over time.

This is actually a very scary prospect since degradation isn't
guaranteed to be equivalent across devices so, not only would scale
shift but the accuracy of intrasite variation measurement could
plummet.

NeilH

unread,
Sep 23, 2012, 7:14:23 PM9/23/12
to airqua...@googlegroups.com
Hello Richard
Interesting to have some thoughts on this;
As a precursor to the discussion I'm assuming that in the end we are talking about "air quality outside someones window" that would be represented in some fashion as Good/Poor/Bad with an approxiamate number in known units and be derivable back to some work by the standards bodies that is backed by solid science.
It may not be exactly the way environmental standards would define it (be EPA approved for legal leverage), but at least be traceable.
When it comes to calibrated sensors/instruments are we talking about the same usage of the word?
eg http://en.wikipedia.org/wiki/Calibration
"Calibration is a comparison between measurements – one of known magnitude or correctness made or set with one device and another measurement made in as similar a way as possible with a second device.
The device with the known or assigned correctness is called the standard. The second device is the unit under test, test instrument, or any of several other names for the device being calibrated."


Sounds to me like there is a different definition of calibration evolving and I'd be interested on what your definition is.
To me, sounds like your are monitoring of the empirical sensor readings across some time period and using that as a reference data set?

Questions below

On Thursday, September 20, 2012 4:16:59 PM UTC-7, Richard Beckwith wrote:

>
> Having a scale for our measurements that allows us to compare to others isn't the only thing that counts as calibration. In fact, it's kind of not what defines calibration. We can have (what I'll call) a scale-free calibration. (Yeah, I know that makes it sound like complexity theory but I like it.)
>

OK - so you can heuristically store the sensors output readings creating a data-set across time.
However is it going to be able to provide a useful connection to thresholds for the GOOD/POOR/BAD indication.

>
>
> With the second type of calibration, we can know how the measurements of the sensors in our network compare to each other. Given a set of calibrated sensors, we can know that Location A is cooler than Location B without knowing how these temperatures compare to locations outside of our network. With this type of calibration, we have not calibrated with respect to a scale but rather with respect to the set of sensors in our network.
>
Neil:OK - but you have the magic words - given a set of calibrated sensors - where do they come from, a relationship between the scales of the individual sensors has to be determined to make a viable comparison. I think you have skipped how that step happens.
Can this scale now be useful ? Can an approximate error be generated to a traceable scale?

For the real world of NO2 sensors, how does this work?
Looking at real values derived from the NO2 MicS2710/2714 data sheet I suggest using three physical sensors – oversampling.
These three physical sensors that are placed next to each – a simple oversampling - could return the following values for NO2 response under the following conditions
NO2 0ppb 100ppb 250ppb
Sensor1 - 2.2 39.0 125 (this is a nominal reading of 2.2K for R0 and typ guess Rs/R0)
Sensor2 - 7.8 100.2 700 (this is at the high end of R0 with a high value guestimate Rs/R0)
Sensor3 – 0.8 4.7 4.8 (this is at the low end of R0 with an early saturation for the high end of the scale)
(0ppb is required by the manufacturer/data sheet, 100pb is an air quality threshold, 250ppb is used as a reference in the data sheet – atmospheric range for NO2 is defined elsewhere as 0-400ppb)

So is there a way of using the raw sensor readings without translating them to defined units with a believable error?


DD

unread,
Nov 23, 2013, 11:12:05 AM11/23/13
to airqua...@googlegroups.com
> Neil:OK - but you have the magic words - given a set of calibrated sensors - where do they come from, a relationship between the scales of the individual sensors has to be determined to make a viable comparison. I think you have skipped how that step happens.

Neil, if you live in a metro area that has calibrated sensors (the expensive kind), couldn't that be where the "given set of calibrated sensors" comes from?

Given the blind calibration discussion above and the desire to ground truth the group of inexpensive sensors, does this sound like a potential field calibration method?:

1.) Locate a nearby known calibrated (expensive type) of air quality sensors. In my city there are 9 of these that are maintained.
2.) Work with local manager of said calibrated sensor(s) to place a quantity of cheap sensor units "near" the calibrated sensor (for ground truthing)
3.) Maybe wait until the right weather is forecasted for conditions that minimize near-field errors between the "nearby" cluster and reference. An empirical amount of minimizing of error may still not be possible.
4.) Set all on a more rapid data acquisition mode to sample a lot of data points. I'm not sure if this would be possible with the calibrated sensor, but work with the manager to check if it could be accommodated.
5.) Leave devices to experience a full range of normal particulate and emissions extremes over a normal work day (minimum dip probably around 3,4am, maximum probably around rush hours).
6.) Use collected data and calibrated sensor data to get a rough idea of error and offset.
7.) Repeat as often as manager is willing. This could also give some idea of how the individual sensors have aged in the field conditions they're normally deployed in.

This would test entire inexpensive sensor system(s) against ground truth system in the field, and this dataset could be used to present at least one error and offset trial. You wouldn't even have to quantify anecdotal numbers for error and offset -- let whomever has a beef with the accuracy do that work if they want.

Ok, I took a shot at defining a rough method. Go ahead and poke holes in it!
:)


NeilH

unread,
Nov 25, 2013, 11:34:40 PM11/25/13
to airqua...@googlegroups.com
Welll..... thats calibration.
Figuring out a technique to "teach" the sensors the range - translate the sensors unique relationship of the gas to defined units.
There are a few algorithims and assumptions that need to be incoporated in the "teach the sensors"
If the variation in the concentration of the gas is fairly slow, and the reference sensor reports in real time, then there should be some baselines measurenments that can be made.
Typically "to calibrate" the sensors you'll want two+ measurements - one at each end of the scale, and with non-linear sensor response some in the middle.
Accuracy comes at a cost .... so what accuracy is desired for the sensors.
The non-calibrated version could be considered close to a junk measurement.
1) low value "normal" calibration measurement baselines the sensor
2) a higher value measurement especially round a pollution threshold may define a point on a non linear curve that a lot more measurements can be interpolated from. Thats where some sensor magic may come from.
Hope that is useful.

BenB

unread,
Dec 9, 2013, 1:02:40 PM12/9/13
to airqua...@googlegroups.com
The easiest way to calibrate all the AQEs would be to do this. (imo)
Calibrate some eggs in a lab to known values of CO, NO2, etc.. concentrations, relevant to the range seen in the environments, (no sense calibrating for extreme values) Then setup a few of these eggs indoors and a few outdoors at the AQE factory site. An example would be 3 indoors and 3 outdoors (multiple would be better for catching a malfunctioning sensor), these reference eggs would then provide the standard for all new eggs and any old eggs that need calibrating. This provides two reference points. An additional reference point could be created by using a clean room/compartment, actively removing the sensor gases from an area, providing a lower reference point closer to zero. Then every year or so re-calibrate some of the reference eggs in a lab. 2 or 3 different points on the curve should be enough to get some accurate measurements for all the eggs, at least lower the uncertainty observed in the data right now.


Reply all
Reply to author
Forward
0 new messages