Fundamental Issue
Walker
Having skimmed the latest CAP v1.2 doc, and the Alert Hub subscription
interface, a fundamental problem seems unanswered.
Specifically, how an entity (person, company, city, state, country)
can automatically (programmatically), without human intervention,
answer the fundamental question: does this alert affect me?
The problem is that even though the entire CAP concept is a spatial
one, that is to say that all the information in the the problem domain
is spatial, there is no spatial normalization, or even in fact
mandatory machine readable spacial information in the CAP v1.2 spec.
CAP v1.2 3.2.4 "area" Element and Sub-elements, specifies the only
mandatory element, is areaDesc
The text describing the affected area of the alert message (REQUIRED)
A text description of the affected area.
A text description is generally not considered machine readable.
...
There are several additional elements that are optional in 3.2.4
"area" Element and Sub-elements, including coordinate point and
radius, polygon of coordinate points, and geocodes of seemingly any
arbitrary system, FIPS codes, postal codes etc.
This is all fine and good and understandable, however what an entity
needs to do to answer the fundamental question programmatically, does
this alert affect me, remains difficult to answer.
Now of course every entity could implement a solution to the problem,
including a normalization of spatial information from every
encountered geocode system, for example converting all zipcodes, FIPS
codes, possibly countries, states, counties, etc. to coordinate pair
polygons. Then finding the intersection of the polygon of what the
entity considers the bounds of itself or area of interest. Combined
possibly with message type filtering, this will finally answer the
question, does this alert affect me.
...
It seems from cursory glance, that the Google Public Alerts homepage
at
http://www.google.org/publicalerts
has to solve the problem described above (spatial normalization) in
order to present the variety of CAP system messages on a map display.
Now, in my opinion, one entity solving the spatial normalization
problem well and sharing the result, is preferable to thousands or
millions of entities having to tackle the same problem, and all of its
associated ongoing update issues.
I am not advocating removing any original information from an alert,
any entity could, if they so desire, implement any system of spatial
normalization they wish using the unaltered original spatial
information.
If Google is normalizing the spatial data, can and will Google enrich
and add value to the CAPS data feed with that information?
That is for example, if an alert is originated with FIPS codes,
ZIPCODES, States and Counties as spatial information, and Google is
already converting it to coordinate pair polygons in order to display
on a map based display, can that enriched data be appended to the
alert so subscribers can use it to answer the question; does this
alert affect me, without having to tackle the 600 pound problem of
normalization themselves.
If this value added data can be appended, the only thorny remaining
issue would be of resolution, for example if a zipcode or a county
takes 100kB of data to describe every nook and cranny of its border
polygon, what is an acceptable level of smoothing of detail to reduce
the poly count to manageable levels. How is it indicated?
.....
The corresponding process of filtering alerts to an entity is enabled
by normalization of spatial information.
How much bandwidth does a feed consume? Consider an unfiltered
publication of worldwide implementation of CAP, everybody gets all
messages for the entire world, which may or may not be OK, after all
the bandwidth is determined by the number of alerts worldwide and that
is determined by the granularity of the alerts themselves and the
level of alert activity.
Even with no filtering, bandwidth of a subscription may never become
an issue in any real way as long as the link speed and capacity is
high.
However if an entity, person, company, city, county, state, etc. only
was interested in CAPS data that satisfied the criteria, does this
alert affect me, or does this alert have a normalized polygon in the
spatial domain that intersects my polygon of spatial interest, then it
would be possible for the subscription service to filter CAPS messages
to subscribers based on the subscribers polygon of interest.
Again this is not advocating that a subscriber cannot receive an
unfiltered worldwide feed, but if an entity desired, for example, the
EOC of Anytown, USA only wanted to receive alerts for their city,
county, state, or the USA, they could do so via specifying a polygon
of interest.
Chris Walker
Art Botterell
I can't speak to Google's implementation, but the point you raise about CAP itself is a good one... with, I'm afraid, a somewhat vexed history.
Ideally, IMHO, every CAP message would include an explicit geospatial geometry (a polygon or circle). Unfortunately, at the time the CAP standard was developed, and to some extent even today, a lot of warning originators simply didn't think in geospatial terms. And it wasn't just the non-technical officials who issue alerts; for a lot of the IT folks who support them GIS was (and still is) a foreign universe. Thus a lot of the early implementors tried to rely on more familiar string-matching techniques based on various arbitrary codes including ZIP and FIPS as well as various home-grown systems of locally pre-defined areas.
In addition to serving the recipients poorly as you describe, that also injects spurious artifacts into the CAP message. Hazards rarely... as in pretty much never... align themselves neatly with political or administrative or any kind of pre-defined boundaries. Thus this practice actually distorts the semantics of the CAP message itself by including area codings that describe the bounds of the originator's jurisdiction or some other preconception rather than the area actually at risk from the subject hazard. Both over- and under-warning errors can result... a problem we're already familiar with from previous generations of warning systems such as EAS and weather radio.
I do have a bit of concern that trying to supplement such string-geocodes or -described area specifications with automated look-ups of the corresponding geometries might have the unintended consequence of reinforcing that anti-pattern of originating authorities describing their own political bounds in CAP messages rather than the hazard. In any event it would introduce some ambiguity as to who was responsible for the content of the CAP message, which is something I don't think any distributor of CAP alerts would want.
So I'm afraid the right solution here isn't the easy one. It would be nice if we could get one organization, e.g. Google, to compensate for poor CAP origination practices... but that's not really possible, as the CAP ecosystem extends well beyond the Google distribution. And while it certainly would be desirable to get OASIS to tighten up the standard to require explicit geometries instead of just recommending them... well, folks acquainted with standards work will appreciate that that's much quicker said than done.
Ultimately the right solution, IMHO, is to provide feedback to the individual CAP originators as to what the consumers want and need. At present there are only a limited number of CAP originators, but with the rollout of the U.S. federal IPAWS framework that number is liable to explode in the next couple of years.
So if there's a desire to advocate for best practice and deprecate merely descriptive text or a-priori zones in CAP messages I'm thinking the time for that will be sooner rather than later.
- Art
Walker
I understand, I have seen the pattern over and over, however something
needs to be done.
Mandatory coordinate pair based spatial information would be ideal and
preferred, point/radius, poly etc. however as long as spatial data is
allowed that is not normalized, it needs to be processed to be
useful.
How much of an impact on adoption of CAP do you think the difference
between having the end user needing to normalize the spatial component
themselves makes? It's not a trivial problem.
I am not advocating specifically that Google is responsible for
spatial normalization, I am advocating that the specification be
extended to ALLOW the normalization of spacial information by a third
party and appending it to the message.
Face it, filtering needs to be done somewhere by something or
somebody.
What message go out on a pager system, which message are even
candidates for a human to make a decision about to go out on a pager
system, which messages consume precious bandwidth on emergency radio
links? When you are getting an unfiltered feed of the world's alerts,
with non normalized data, those types of decisions are not easily
implemented.
Every engineering effort is an opportunity for improvement, or a
missed opportunity.
Chris
James Hamilton
Peter
geographic information about the extent and intensity of hazards if
there is no way to also express the probabilistic nature of these
forecasts. Forecasts are based on models that predict the intensity
of hazardous conditions as a function of space and time. Inevitably,
such models will predict a contour that separates “dangerous” areas
from “safe” ones. Representing this contour as a series of geographic
points can be misleading. A coordinate pair has decimal places, and
decimal places imply precision.
I suspect that at least part of the rationale for using ZIP codes,
FIPS codes, or other zone-based forecasts stems from the desire to
surround the area of true hazard with a large buffer that crudely
approximates the diffuse uncertainty of the prediction. Obviously,
this “approximation” breaks down in edge cases and creates needless
worry for large groups of people who happen to live in a large zone.
I’ve seen this happen during the eruptions of Redoubt and Augustine
volcanoes in Alaska. The NWS released forecasts for volcanic ash fall
using the same geographic zone system it used for storms and other
weather products. The ash fall danger actually existed in a narrow
sector only a few kilometers across, while the forecast fell across
the entire zone, in this case the Kenai Peninsula, which is 250 km
long.
In spite of their limitations, zone-based forecast systems are
deterministic and consistent with respect to their geographic content.
They also never appear ad hoc, which is not necessarily the case for a
contour that, from the public’s perspective, arbitrarily divides a
town into safe and dangerous halves. The “not ad hoc” feature is one
worth saving, and is close kin to the principle of not implying more
precision than you actually have. I would argue that any extension to
the CAP protocol to include detailed geographic information must also
carry with it a means to express the uncertainty associated with the
encapsulated hazards forecast.
Peter
--
Peter Cervelli
U.S. Geological Survey
Volcano Science Center
345 Middlefield Road, MS 910
Menlo Park, CA 94025
6503295188 (office)
6502084414 (cell)
pcer...@usgs.gov
On Feb 2, 8:14 am, James Hamilton <james.e.hamilton...@gmail.com>
wrote:
>
> read more »
Art Botterell
Or to keep things simple one can simply apply a consistent buffer around the forecast impact area. The problem with using pre-defined "zones" is that they'll almost never provide a consistent buffer. Thus using them actually adds to the uncertainty we're concerned about.
That zones can be mapped to geometries is true but misses the point. The problem is that real-world events generally don't map back to pre-determinted zones. By using such zones we sacrifice accuracy for convenience; that's understandable, maybe, but not good.
In the past our dissemination media, being mostly broadcast in nature, largely masked this issue. Since the delivery of the alerts was quite imprecise, nobody noticed that imprecise targeting of the alerts... we just figured that was the way things were, and accepted that warning systems were really only usable for very large-scale events. But now we're getting the technical opportunity to target alert delivery quite precisely, both by way of more agile dissemination channels and of intelligent and location-aware receiving devices that can filter out alerts that don't affect their individual operators.
This is a huge opportunity to improve the performance and utility of our warning systems, but it can be lost if alert originators cling to legacy procedures that don't reflect the advancing state of the art. False precision is a legitimate concern, but our ability to forecast precisely and detect hazards at much greater resolution is improving and will continue to do so, and we don't want to cling to procedures that keep us from leveraging those advances.
NOAA, I should mention, has started to address this. While they continue to use forecast zones for a lot of their products, they've been including explicit LAT...LON polygons in their "short-fused" warning products for many years now.
Not sure I understand the "not ad-hoc" principle as suggested, though. If part of a town is at risk and the other isn't, we definitely want to express that, both to minimize the side effects of overreaction (clogged highways, emptied stores, jammed 9-1-1, etc.) and as guidance to responders who may use resources in one part of town to assist the other. Emergencies are "ad-hoc" by definition.
- Art
PS - Since a lot of folks on this list may not know me, and since I'm not speaking ex-officio, maybe I'd better introduce myself a tiny bit to put my remarks in context. I was the original proponent of the Common Alerting Protocol, and shepherded it through its early iterative development and subsequent formalization. Since then I've built and operated several CAP-based alerting systems and consulted on a number of others in the U.S. and internationally. My career has been mainly in emergency telecommunications and emergency public information. Currently I'm involved in research on at a major university. (Peter, I'm just down the road from you at Moffett Field.)
Matthias Lendholt
We have developed the following workflow which is not only applicable to tsunami early warning systems but also to other hazards:
i) incoming sensor events: in-situ sensors (tsunami: seismic system, buoys, ...) send notifications via SWE SAS standard
ii) forecasting: based on incoming measurements and bathymetry data, forecasts and predictions are generated based on scientific models (domain specific, e.g. tsunami, bushfire, ...). The results are stored shapefiles (e.g. points of interest with estimated time of arrival and severity (tsunami: maximum wave height))
iii) classification of administrative units: a web service named AAIS (affected area identification service) intersects forecasts with geometries of administrative units and classifies them based on a worst-case approach
iv) generation of warning message: for each administrative unit one CAP info element is generated. Criticality values are set for each region.
However, depending on the hazard and the geographic region the proper administrative unit level has to be selected. The federal state level might be suited for storm warnings in the Netherlands but in other countries the second, third or even fourth level should be taken.
For addressing administrative units we are using geo codes. We are not storing the geometries in CAP messages. This is a nightmare and would end up in huge messages with thousdands of vertices. Since our projects deal in an international context, we had to find a geocode standard that serves in all regions. Neither FIPS nor SALB (UNESCO) nor ISO3166 fulfilled our requirements. We decided to use the non-official HASC standard.
If you are interested in our work, you will find some more details in these publications:
"Tailoring spatial reference in early warning systems to administrative units" In Earth Science Informatics, http://www.springerlink.com/content/5k71277584761771/
"Addressing administrative units in international tsunami early warning systems: shortcomings in international geocode standards" In International Journal of Digital Earth, http://www.tandfonline.com/doi/abs/10.1080/17538947.2011.584574
If someone is interested in one of these publications.... please send me an email.
Matthias
--
Matthias Lendholt
GFZ German Research Centre for Geosciences
CeGIT Centre for GeoInformation Technology
Telegrafenberg A20, 14473 Potsdam, Germany
http://www.dews-online.org/
Art Botterell
> The general public will not be able to understand any warning with some lat/lon values or complex polygon outlines.
We may not have communicated effectively the difference in purpose between the areaDesc element and the geocodes and geometries. The areaDesc is the part that's intended to be understood by human recipients, whereas the geocodes and geometries are intended to enable automated routing and/or filtering of alerts... e.g., by a location-aware receiving device that can calculate whether it's in the designated alerting area. (The purpose of such routing or filtering is to minimize the problem of "warning spill" where recipients become annoyed and perhaps even sensitized to alerts that aren't actually meant for them.)
Of course it's only been recently that we've had media capable of delivering alerts with any degree of geographic precision, wo this distinction didn't really arise in the age of wide-area broadcast alerting. Also, given that tsunamis tend to affect quite large areas relative to administrative boundaries, it might be that you haven't encountered the requirement for more precise alert targeting that arises in other hazard types such as fires, severe storms, hazardous materials and of course man-made hazards.
> This is a nightmare and would end up in huge messages with thousdands of vertices.
Not necessarily, and indeed not typically. Very complex polygons are usually the result of the extraction from a high-resolution GIS representation of a highly irregular "fractal" boundary such a water feature such as a coastline or river. Since hazards only rarely respect such boundaries, one can infer that this usually happens because the warning originator actually started with a previously surveyed political or administrative boundary rather than a characterization of the threat.
On the other hand, polygons plotted in real time by an individual (such as a weather forecaster here in the U.S.) tend to be quite simple, and even polygons derived from hazard models such as those applicable to hazardous material releases tend to be relatively simple (and also tend to be convex, which makes automated buffering and/or simplification easy if its required.)
- Art
