Infections in Travellers
7 views
Skip to first unread message
michaeledwardmarks
Oct 26, 2011, 4:44:41 PM10/26/11
to oss-uk-health
Hi,
My name is Michael Marks. I am a friend of carl's and a junior doctor
working in London - I do research at the Hospital for Tropical
Diseases.
I have two related problems I would like to throw out to the group
Firstly:
One of the things that I think would be really helpful would be a
central portal for information about outbreaks in different parts of
the world.
For example a number of organisations issue travel alerts with some
but not complete overlap of what they report:
1) Center for Disease Control in the US - http://wwwnc.cdc.gov/travel/
(listed under travel notices)
2) WHO -http://www.who.int/csr/don/en/index.html (already available I
think as an RSS feed)
3) The HPA - http://www.hpa.org.uk/NewsCentre/ (although this one is
less clean as the HPA do stuff other than infections)
4) MD Travel Health - http://mdtravelhealth.com/recent_health_alerts.php
There is a website called http://healthmap.org/en/ which does
something along these lines - and displays it on a map. The problems
with the site include the fact that it clearly doesn't suck all data
sources as not all the alerts on the individual websites come up here
and that lots of the alerts are from newspaper articles etc rather
than relevant institutions like CDC/HPA
In the ideal world you would go to a website/App and it would
1)Display a merged list of all the alerts
2)Allow you to sort/search the list for specific countries and disease
- so for example it might display the last 5 outbreak alerts about
China if that is where your patient had been.
3)Have a function similar to healthmaps in displaying the data on a
map
Advantages:
Merges multiple disparate sources of data
Helpful for A+E/Medics seeing returning travellers
Helpful for GPs in advising people of whats going on where
Secondly:
The CDC and WHO websites have lots of cool and helpful stuff on it.
One of the things the CDC has is a page for nearly every infectious
diseases on which the epidemiology is described in terms of incidence
by country, month etc.
Each disease has a page and on these pages there tends to be a link to
this section - either labelled Epidemiology and Statistics or
sometimes just Statistics
What would be nice would be a scraper type app that just listed all
the different diseases for which one of these info pages was available
so you can easily access it.
Equally the WHO has guidelines for most infectious diseases which
should be amenable to a scraper approach similar to the NICE/Green Top
scrapers.
In a dream world the 2 problems described would be solved via one
interface.
i.e
go to website:
search dengue
Get list/map of recent outbreak alerts for dengue
Get direct link to CDC epidemiology page on CDC (could also link to
main CDC page for said disease)
Get direct link to relevant WHO guidelines
Some of the stuff described is already done in parts across different
sites but a central hub which pulled all of the different elements
together would be great
Thoughts on practicality would be greatly appreciated.
Happy to provide more detail/examples if needed.
Michael
My name is Michael Marks. I am a friend of carl's and a junior doctor
working in London - I do research at the Hospital for Tropical
Diseases.
I have two related problems I would like to throw out to the group
Firstly:
One of the things that I think would be really helpful would be a
central portal for information about outbreaks in different parts of
the world.
For example a number of organisations issue travel alerts with some
but not complete overlap of what they report:
1) Center for Disease Control in the US - http://wwwnc.cdc.gov/travel/
(listed under travel notices)
2) WHO -http://www.who.int/csr/don/en/index.html (already available I
think as an RSS feed)
3) The HPA - http://www.hpa.org.uk/NewsCentre/ (although this one is
less clean as the HPA do stuff other than infections)
4) MD Travel Health - http://mdtravelhealth.com/recent_health_alerts.php
There is a website called http://healthmap.org/en/ which does
something along these lines - and displays it on a map. The problems
with the site include the fact that it clearly doesn't suck all data
sources as not all the alerts on the individual websites come up here
and that lots of the alerts are from newspaper articles etc rather
than relevant institutions like CDC/HPA
In the ideal world you would go to a website/App and it would
1)Display a merged list of all the alerts
2)Allow you to sort/search the list for specific countries and disease
- so for example it might display the last 5 outbreak alerts about
China if that is where your patient had been.
3)Have a function similar to healthmaps in displaying the data on a
map
Advantages:
Merges multiple disparate sources of data
Helpful for A+E/Medics seeing returning travellers
Helpful for GPs in advising people of whats going on where
Secondly:
The CDC and WHO websites have lots of cool and helpful stuff on it.
One of the things the CDC has is a page for nearly every infectious
diseases on which the epidemiology is described in terms of incidence
by country, month etc.
Each disease has a page and on these pages there tends to be a link to
this section - either labelled Epidemiology and Statistics or
sometimes just Statistics
What would be nice would be a scraper type app that just listed all
the different diseases for which one of these info pages was available
so you can easily access it.
Equally the WHO has guidelines for most infectious diseases which
should be amenable to a scraper approach similar to the NICE/Green Top
scrapers.
In a dream world the 2 problems described would be solved via one
interface.
i.e
go to website:
search dengue
Get list/map of recent outbreak alerts for dengue
Get direct link to CDC epidemiology page on CDC (could also link to
main CDC page for said disease)
Get direct link to relevant WHO guidelines
Some of the stuff described is already done in parts across different
sites but a central hub which pulled all of the different elements
together would be great
Thoughts on practicality would be greatly appreciated.
Happy to provide more detail/examples if needed.
Michael
Mathena Pavan
Oct 26, 2011, 4:48:13 PM10/26/11
to oss-uk...@googlegroups.com, oss-uk-health
Hi Mark
Sorry I couldn't make it to drinks tonight but as it is Diwali was with family. I have been giving feedback to the WHO Europe who are changing their website as we speak. A few times i have messaged them on twitter and their response about details on outbreaks has been swift and really helpful. I wonder if you have used that before?
Mathena
Sent from my iPhone
Wai Keong
Oct 26, 2011, 5:24:04 PM10/26/11
to oss-uk...@googlegroups.com
Hi Mark. Welcome to the Group!
Your idea sounds great. Does definitely sound like a scrapper type app/ platform.
BW
Wai Keong
michaeledwardmarks
Oct 26, 2011, 5:26:56 PM10/26/11
to oss-uk-health
I haven't used that before although it is pretty cool.
I don't think it solves the problems I described above though - you
are waiting for someone else to respond, you can't search by country/
disease, no links to the CDC page or WHO guidelines......and most
importantly it assumes your place of work hasn't blocked twitter which
I imagine lots of hospital have.
Michael
On Oct 26, 9:48 pm, Mathena Pavan <mathena.pa...@gmail.com> wrote:
> Hi Mark
>
> Sorry I couldn't make it to drinks tonight but as it is Diwali was with family. I have been giving feedback to the WHO Europe who are changing their website as we speak. A few times i have messaged them on twitter and their response about details on outbreaks has been swift and really helpful. I wonder if you have used that before?
>
> Mathena
>
> Sent from my iPhone
>
> > (listed under travel notices)
>
> > 2) WHO -http://www.who.int/csr/don/en/index.html(already available I
>
> > There is a website calledhttp://healthmap.org/en/which does
I don't think it solves the problems I described above though - you
are waiting for someone else to respond, you can't search by country/
disease, no links to the CDC page or WHO guidelines......and most
importantly it assumes your place of work hasn't blocked twitter which
I imagine lots of hospital have.
Michael
On Oct 26, 9:48 pm, Mathena Pavan <mathena.pa...@gmail.com> wrote:
> Hi Mark
>
> Sorry I couldn't make it to drinks tonight but as it is Diwali was with family. I have been giving feedback to the WHO Europe who are changing their website as we speak. A few times i have messaged them on twitter and their response about details on outbreaks has been swift and really helpful. I wonder if you have used that before?
>
> Mathena
>
> Sent from my iPhone
>
> On 26 Oct 2011, at 21:44, michaeledwardmarks <michaeledwardma...@gmail.com> wrote:
>
>
>
>
>
>
>
> > Hi,
>
> > My name is Michael Marks. I am a friend of carl's and a junior doctor
> > working in London - I do research at the Hospital for Tropical
> > Diseases.
>
> > I have two related problems I would like to throw out to the group
>
> > Firstly:
>
> > One of the things that I think would be really helpful would be a
> > central portal for information about outbreaks in different parts of
> > the world.
>
> > For example a number of organisations issue travel alerts with some
> > but not complete overlap of what they report:
>
> > 1) Center for Disease Control in the US -http://wwwnc.cdc.gov/travel/
>
>
>
>
>
>
>
> > Hi,
>
> > My name is Michael Marks. I am a friend of carl's and a junior doctor
> > working in London - I do research at the Hospital for Tropical
> > Diseases.
>
> > I have two related problems I would like to throw out to the group
>
> > Firstly:
>
> > One of the things that I think would be really helpful would be a
> > central portal for information about outbreaks in different parts of
> > the world.
>
> > For example a number of organisations issue travel alerts with some
> > but not complete overlap of what they report:
>
> > (listed under travel notices)
>
> > 2) WHO -http://www.who.int/csr/don/en/index.html(already available I
> > think as an RSS feed)
>
> > 3) The HPA -http://www.hpa.org.uk/NewsCentre/(although this one is
>
> > less clean as the HPA do stuff other than infections)
>
> > 4) MD Travel Health -http://mdtravelhealth.com/recent_health_alerts.php
>
>
> > There is a website calledhttp://healthmap.org/en/which does
Ross Jones
Oct 26, 2011, 5:36:37 PM10/26/11
to oss-uk...@googlegroups.com
Hi Michael,
It definitely does look like a job for a scraper, and there is a good chance that we could find someone to do the work, a lot of the community who use the ScraperWiki mailing list are often found writing scrapers for people - especially if it is a worthwhile task.
Do you think this problem would be best served by a new app, or do you think it would be worth talking to the healthmap.org team and seeing if they are interested in pulling in data from CDC rather than just their current sources? That way we can just point them at the scrapers and they can use that for more accurate data. It does seem with the 'submit an outbreak' button that they are attempting an ushahidi (http://ushahidi.com/) type platform (assuming it isn't based on that already) but I can't be sure.
Ross.
Mathena Pavan
Oct 27, 2011, 2:02:46 AM10/27/11
to oss-uk...@googlegroups.com
Dear Micheal
Very true. I think that combining resources in a similar way to the current projects and maybe including social media feeds on the site might combine the two to make it even more useful.
Mathena
Sent from my iPhone
Message has been deleted
michaeledwardmarks
Oct 27, 2011, 5:30:30 AM10/27/11
to oss-uk-health
Grant,
That would be great and might address the first of the two problems I
discussed above - so I would definitely be keen to speak to her.
In terms of the second problem I described on my initial post I would
still be keen to see if a scraper type app could pull the CDC
epidemiology data sheets and WHO guidelines and make them available
via a shared interface - as they are also very useful. I guess a
scraping app could potentially provide that data to a dropdown on
healthmap fairly easily? So again if that was possible we might be
able to integrate the second problem with the first.
Michael
On Oct 27, 7:18 am, Grant Hill-Cawthorne <ga...@cam.ac.uk> wrote:
> I can put you in touch with Rumi Chunana who is a Harvard research
> fellow on the healthmap.org team. I met her at a conference and she has
> previously visited our visualisation lab in Saudi. They produced a nice
> add-on to healthmap that focused on the countries that the Hajj pilgrims
> would come from to help plan ahead for potential outbreaks during the Hajj.
>
> I've found healthmap to be pretty good when faced with ID patients who
> have been travelling. Although I receive all the Promed alerts I can't
> keep all these in my head.
>
> I know from speaking to the healthmap team they always felt that they
> would identify potential outbreaks quicker because they rely on local
> media. However this does pre-suppose that you have an open media in your
> country. Many countries purposefully suppress disease outbreaks simply
> because of the impact it may have on tourism, etc. I'm sure healthmap
> would be interested in trying to incorporate some of the CDC data - as
> long as the CDC don't have some kind of copyright over it.
>
> Grant
> >> (listed under travel notices)
>
> >> 2) WHO -http://www.who.int/csr/don/en/index.html(already available I
>
> >> There is a website calledhttp://healthmap.org/en/which does
> >> Michael- Hide quoted text -
>
> - Show quoted text -
That would be great and might address the first of the two problems I
discussed above - so I would definitely be keen to speak to her.
In terms of the second problem I described on my initial post I would
still be keen to see if a scraper type app could pull the CDC
epidemiology data sheets and WHO guidelines and make them available
via a shared interface - as they are also very useful. I guess a
scraping app could potentially provide that data to a dropdown on
healthmap fairly easily? So again if that was possible we might be
able to integrate the second problem with the first.
Michael
On Oct 27, 7:18 am, Grant Hill-Cawthorne <ga...@cam.ac.uk> wrote:
> I can put you in touch with Rumi Chunana who is a Harvard research
> fellow on the healthmap.org team. I met her at a conference and she has
> previously visited our visualisation lab in Saudi. They produced a nice
> add-on to healthmap that focused on the countries that the Hajj pilgrims
> would come from to help plan ahead for potential outbreaks during the Hajj.
>
> I've found healthmap to be pretty good when faced with ID patients who
> have been travelling. Although I receive all the Promed alerts I can't
> keep all these in my head.
>
> I know from speaking to the healthmap team they always felt that they
> would identify potential outbreaks quicker because they rely on local
> media. However this does pre-suppose that you have an open media in your
> country. Many countries purposefully suppress disease outbreaks simply
> because of the impact it may have on tourism, etc. I'm sure healthmap
> would be interested in trying to incorporate some of the CDC data - as
> long as the CDC don't have some kind of copyright over it.
>
> Grant
>
> On 26/10/2011 22:36, Ross Jones wrote:
>
>
>
> > Hi Michael,
>
> > It definitely does look like a job for a scraper, and there is a good
> > chance that we could find someone to do the work, a lot of the
> > community who use the ScraperWiki mailing list are often found writing
> > scrapers for people - especially if it is a worthwhile task.
>
> > Do you think this problem would be best served by a new app, or do you
> > think it would be worth talking to the healthmap.org
> > <http://healthmap.org> team and seeing if they are interested in
> On 26/10/2011 22:36, Ross Jones wrote:
>
>
>
> > Hi Michael,
>
> > It definitely does look like a job for a scraper, and there is a good
> > chance that we could find someone to do the work, a lot of the
> > community who use the ScraperWiki mailing list are often found writing
> > scrapers for people - especially if it is a worthwhile task.
>
> > Do you think this problem would be best served by a new app, or do you
> > think it would be worth talking to the healthmap.org
> > pulling in data from CDC rather than just their current sources? That
> > way we can just point them at the scrapers and they can use that for
> > more accurate data. It does seem with the 'submit an outbreak' button
> > that they are attempting an ushahidi (http://ushahidi.com/) type
> > platform (assuming it isn't based on that already) but I can't be sure.
>
> > Ross.
>
> > On 26 Oct 2011, at 21:44, michaeledwardmarks wrote:
>
> >> Hi,
>
> >> My name is Michael Marks. I am a friend of carl's and a junior doctor
> >> working in London - I do research at the Hospital for Tropical
> >> Diseases.
>
> >> I have two related problems I would like to throw out to the group
>
> >> Firstly:
>
> >> One of the things that I think would be really helpful would be a
> >> central portal for information about outbreaks in different parts of
> >> the world.
>
> >> For example a number of organisations issue travel alerts with some
> >> but not complete overlap of what they report:
>
> >> 1) Center for Disease Control in the US -http://wwwnc.cdc.gov/travel/
> > way we can just point them at the scrapers and they can use that for
> > more accurate data. It does seem with the 'submit an outbreak' button
> > that they are attempting an ushahidi (http://ushahidi.com/) type
> > platform (assuming it isn't based on that already) but I can't be sure.
>
> > Ross.
>
> > On 26 Oct 2011, at 21:44, michaeledwardmarks wrote:
>
> >> Hi,
>
> >> My name is Michael Marks. I am a friend of carl's and a junior doctor
> >> working in London - I do research at the Hospital for Tropical
> >> Diseases.
>
> >> I have two related problems I would like to throw out to the group
>
> >> Firstly:
>
> >> One of the things that I think would be really helpful would be a
> >> central portal for information about outbreaks in different parts of
> >> the world.
>
> >> For example a number of organisations issue travel alerts with some
> >> but not complete overlap of what they report:
>
> >> (listed under travel notices)
>
> >> 2) WHO -http://www.who.int/csr/don/en/index.html(already available I
> >> think as an RSS feed)
>
> >> 3) The HPA -http://www.hpa.org.uk/NewsCentre/(although this one is
>
> >> less clean as the HPA do stuff other than infections)
>
> >> 4) MD Travel Health -http://mdtravelhealth.com/recent_health_alerts.php
>
>
> >> There is a website calledhttp://healthmap.org/en/which does
>
> - Show quoted text -
Ross Jones
Oct 27, 2011, 10:15:44 AM10/27/11
to oss-uk...@googlegroups.com
Hi Michael,
I have just spoken to Aidan at ScraperWiki, and he mentioned that he is intending to visit EHI with some developers to run some sort of scraping hack-day. I mentioned your needs and this might be a good opportunity to get your scrapers written and usable in a single day. Are you planning to attend?
Ross.
michaeledwardmarks
Oct 28, 2011, 4:31:32 PM10/28/11
to oss-uk-health
Hi ross,
I need to check my work schedule. Unlike Carl my day to day job still
requires me to be in a particular place 9-5 most days but if I am free
then I will be coming - and that sounds like a great opportunity to
make some headway with this,
Michael
> >>>> 2) WHO -http://www.who.int/csr/don/en/index.html(alreadyavailable I
I need to check my work schedule. Unlike Carl my day to day job still
requires me to be in a particular place 9-5 most days but if I am free
then I will be coming - and that sounds like a great opportunity to
make some headway with this,
Michael
> >>>> think as an RSS feed)
>
> >>>> 3) The HPA -http://www.hpa.org.uk/NewsCentre/(althoughthis one is
>
michaeledwardmarks
Nov 2, 2011, 2:44:33 PM11/2/11
to oss-uk-health
Unfortunately work precludes attedance and can't get anyone to cover
me for the EHI.
Hmmm. Any chance someone can press my case for me?
Equally Grant - I'm still keen to try and see if the healthmap people
are interested in any collaboration.
Michael
On Oct 28, 8:31 pm, michaeledwardmarks <michaeledwardma...@gmail.com>
wrote:
> > >>>> 3) The HPA -http://www.hpa.org.uk/NewsCentre/(althoughthisone is
me for the EHI.
Hmmm. Any chance someone can press my case for me?
Equally Grant - I'm still keen to try and see if the healthmap people
are interested in any collaboration.
Michael
On Oct 28, 8:31 pm, michaeledwardmarks <michaeledwardma...@gmail.com>
wrote:
Message has been deleted
Ross Jones
Nov 7, 2011, 2:48:17 PM11/7/11
to oss-uk...@googlegroups.com
Hi,
We hacked on a few things today at EHI Live, I managed to get a lot of the outbreak information from WHO global alert and response. A lot of this is historical from 1996-present, and there is still some data cleaning to be done as they tended to change the format over the years :(
The scraper is at https://scraperwiki.com/scrapers/who_outbreaks/ although I suspect either the download spreadsheet ( https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=who_outbreaks&query=select+*+from+`swdata`&apikey= ) or the API ( https://scraperwiki.com/docs/api?name=who_outbreaks#sqlite ) or a HTML table view ( https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable&name=who_outbreaks&query=select%20*%20from%20%60swdata%60%20order%20by%20year desc ) might be useful.
Hopefully those links will be clickable, and at some point I suspect the JSON export will be most useful. If I can find a useful lat/lon for each country we can try and add those too.
I'll try and find some time tomorrow, or more realistically later in the week, to try and clean up the data.
Ross
Message has been deleted
Ross Jones
Nov 8, 2011, 4:11:55 AM11/8/11
to oss-uk...@googlegroups.com
If you export the data as CSV you can load it into Google Fusion Tables ( http://www.google.com/fusiontables/ ) and it will let you show them on a map (although it takes a short while to geocode the countries and some are obviously missing). The heatmap doesn't seem to work but you can filter the view by year and click on dots to get the record it corresponds to. I tried to share the one I created but I think I broke Google.
Ross
On 7 Nov 2011, at 20:00, Grant Hill-Cawthorne wrote:
This is a really helpful resource. It would be great to have a historical googlemap of this data. There are certainly some surprises - like Ebola in South Africa.
This is the most useful thing I have seen come out of the WHO recently :)
G
On 07/11/2011 19:48, Ross Jones wrote:Hi,We hacked on a few things today at EHI Live, I managed to get a lot of the outbreak information from WHO global alert and response. A lot of this is historical from 1996-present, and there is still some data cleaning to be done as they tended to change the format over the years :(
The scraper is at https://scraperwiki.com/scrapers/who_outbreaks/ although I suspect either the download spreadsheet ( https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=who_outbreaks&query=select+*+from+ <https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=who_outbreaks&query=select+*+from+>`swdata`&apikey= ) or the API ( https://scraperwiki.com/docs/api?name=who_outbreaks#sqlite ) or a HTML table view ( https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable&name=who_outbreaks&query=select%20*%20from%20%60swdata%60%20order%20by%20year <https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable&name=who_outbreaks&query=select%20*%20from%20%60swdata%60%20order%20by%20year> desc ) might be useful.
michaeledwardmarks
Nov 8, 2011, 5:58:37 PM11/8/11
to oss-uk-health
Hi Ross,
Sounds great (I think Scraperwiki must be down as none of the links
nor even the main site are working). I will have another go in the
morning.
And the google fusion table idea for displaying on a map also sounds
excellent.
Will feedback in more detail once I have got the links to work
tomorrow hopefully,
Michael
On Nov 8, 9:11 am, Ross Jones <r...@servercode.co.uk> wrote:
> If you export the data as CSV you can load it into Google Fusion Tables (http://www.google.com/fusiontables/) and it will let you show them on a map (although it takes a short while to geocode the countries and some are obviously missing). The heatmap doesn't seem to work but you can filter the view by year and click on dots to get the record it corresponds to. I tried to share the one I created but I think I broke Google.
Sounds great (I think Scraperwiki must be down as none of the links
nor even the main site are working). I will have another go in the
morning.
And the google fusion table idea for displaying on a map also sounds
excellent.
Will feedback in more detail once I have got the links to work
tomorrow hopefully,
Michael
On Nov 8, 9:11 am, Ross Jones <r...@servercode.co.uk> wrote:
> If you export the data as CSV you can load it into Google Fusion Tables (http://www.google.com/fusiontables/) and it will let you show them on a map (although it takes a short while to geocode the countries and some are obviously missing). The heatmap doesn't seem to work but you can filter the view by year and click on dots to get the record it corresponds to. I tried to share the one I created but I think I broke Google.
>
> Ross
>
> On 7 Nov 2011, at 20:00, Grant Hill-Cawthorne wrote:
>
>
>
>
>
>
>
> > This is a really helpful resource. It would be great to have a historical googlemap of this data. There are certainly some surprises - like Ebola in South Africa.
>
> > This is the most useful thing I have seen come out of the WHO recently :)
>
> > G
>
> > On 07/11/2011 19:48, Ross Jones wrote:
> >> Hi,
>
> >> We hacked on a few things today at EHI Live, I managed to get a lot of the outbreak information from WHO global alert and response. A lot of this is historical from 1996-present, and there is still some data cleaning to be done as they tended to change the format over the years :(
>
> >> The scraper is athttps://scraperwiki.com/scrapers/who_outbreaks/although I suspect either the download spreadsheet (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...>`swdata`&apikey= ) or the API (https://scraperwiki.com/docs/api?name=who_outbreaks#sqlite) or a HTML table view (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...> desc ) might be useful.
> Ross
>
> On 7 Nov 2011, at 20:00, Grant Hill-Cawthorne wrote:
>
>
>
>
>
>
>
> > This is a really helpful resource. It would be great to have a historical googlemap of this data. There are certainly some surprises - like Ebola in South Africa.
>
> > This is the most useful thing I have seen come out of the WHO recently :)
>
> > G
>
> > On 07/11/2011 19:48, Ross Jones wrote:
> >> Hi,
>
> >> We hacked on a few things today at EHI Live, I managed to get a lot of the outbreak information from WHO global alert and response. A lot of this is historical from 1996-present, and there is still some data cleaning to be done as they tended to change the format over the years :(
>
michaeledwardmarks
Nov 14, 2011, 11:59:23 AM11/14/11
to oss-uk-health
Right. Off nights and had a proper look.
Really cool. Especially when displayed on a map via google fusion
table. That is a long way towards what I was thinking of. So many many
thanks.
Can I ask
1) Would it be easy/difficult to replicate on CDC data and merge the
two streams?
2) Would it be possible to add a function that allows you to set the
time frame within which notifications are displayed e.g only endemics
between 2005-7 etc?
These two functionalities would be very very helpful - but have to say
what already exists is very nice.
Michael
On Nov 8, 10:58 pm, michaeledwardmarks <michaeledwardma...@gmail.com>
wrote:
Really cool. Especially when displayed on a map via google fusion
table. That is a long way towards what I was thinking of. So many many
thanks.
Can I ask
1) Would it be easy/difficult to replicate on CDC data and merge the
two streams?
2) Would it be possible to add a function that allows you to set the
time frame within which notifications are displayed e.g only endemics
between 2005-7 etc?
These two functionalities would be very very helpful - but have to say
what already exists is very nice.
Michael
On Nov 8, 10:58 pm, michaeledwardmarks <michaeledwardma...@gmail.com>
wrote:
> Hi Ross,
>
> Sounds great (I think Scraperwiki must be down as none of the links
> nor even the main site are working). I will have another go in the
> morning.
> And the google fusion table idea for displaying on a map also sounds
> excellent.
>
> Will feedback in more detail once I have got the links to work
> tomorrow hopefully,
>
> Michael
>
> On Nov 8, 9:11 am, Ross Jones <r...@servercode.co.uk> wrote:
>
>
>
>
>
>
>
> > If you export the data as CSV you can load it into Google Fusion Tables (http://www.google.com/fusiontables/) and it will let you show them on a map (although it takes a short while to geocode the countries and some are obviously missing). The heatmap doesn't seem to work but you can filter the view by year and click on dots to get the record it corresponds to. I tried to share the one I created but I think I broke Google.
>
> > Ross
>
> > On 7 Nov 2011, at 20:00, Grant Hill-Cawthorne wrote:
>
> > > This is a really helpful resource. It would be great to have a historical googlemap of this data. There are certainly some surprises - like Ebola in South Africa.
>
> > > This is the most useful thing I have seen come out of the WHO recently :)
>
> > > G
>
> > > On 07/11/2011 19:48, Ross Jones wrote:
> > >> Hi,
>
> > >> We hacked on a few things today at EHI Live, I managed to get a lot of the outbreak information from WHO global alert and response. A lot of this is historical from 1996-present, and there is still some data cleaning to be done as they tended to change the format over the years :(
>
> > >> The scraper is athttps://scraperwiki.com/scrapers/who_outbreaks/althoughI suspect either the download spreadsheet (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...>`swdata`&apikey= ) or the API (https://scraperwiki.com/docs/api?name=who_outbreaks#sqlite) or a HTML table view (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...> desc ) might be useful.
>
> Sounds great (I think Scraperwiki must be down as none of the links
> nor even the main site are working). I will have another go in the
> morning.
> And the google fusion table idea for displaying on a map also sounds
> excellent.
>
> Will feedback in more detail once I have got the links to work
> tomorrow hopefully,
>
> Michael
>
> On Nov 8, 9:11 am, Ross Jones <r...@servercode.co.uk> wrote:
>
>
>
>
>
>
>
> > If you export the data as CSV you can load it into Google Fusion Tables (http://www.google.com/fusiontables/) and it will let you show them on a map (although it takes a short while to geocode the countries and some are obviously missing). The heatmap doesn't seem to work but you can filter the view by year and click on dots to get the record it corresponds to. I tried to share the one I created but I think I broke Google.
>
> > Ross
>
> > On 7 Nov 2011, at 20:00, Grant Hill-Cawthorne wrote:
>
> > > This is a really helpful resource. It would be great to have a historical googlemap of this data. There are certainly some surprises - like Ebola in South Africa.
>
> > > This is the most useful thing I have seen come out of the WHO recently :)
>
> > > G
>
> > > On 07/11/2011 19:48, Ross Jones wrote:
> > >> Hi,
>
> > >> We hacked on a few things today at EHI Live, I managed to get a lot of the outbreak information from WHO global alert and response. A lot of this is historical from 1996-present, and there is still some data cleaning to be done as they tended to change the format over the years :(
>
Ross Jones
Nov 14, 2011, 12:07:00 PM11/14/11
to oss-uk...@googlegroups.com
I believe there was a chap on Twitter, @carlplant who was waiting for the cleaned data so that he can do some visualisations. It's on my list, but it shouldn't be too hard if anyone else wanted to have a go.
If you can find a link to the CDC data I'll see if I can get a scraper done for it.
Ross
Carl Reynolds
Nov 15, 2011, 12:32:26 AM11/15/11
to oss-uk...@googlegroups.com
Its on my list too...
Best, Carl
michaeledwardmarks
Nov 15, 2011, 11:33:15 AM11/15/11
to oss-uk-health
The CDC data is on this site which lists all the CDC RSS feeds.
http://www2c.cdc.gov/podcasts/rss.asp
The relevant one is Travel Notices
http://www2c.cdc.gov/podcasts/createrss.asp?t=r&c=66
Its available as XML and RSS.
Hope that helps.
Cheers for all your great work - I think this has potential to be very
helpful to people in a whole range of clinical situations.
Mike
> >>>>>> The scraper is athttps://scraperwiki.com/scrapers/who_outbreaks/althoughIsuspect either the download spreadsheet (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...>`swdata`&apikey= ) or the API (https://scraperwiki.com/docs/api?name=who_outbreaks#sqlite) or a HTML table view (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...> desc ) might be useful.
http://www2c.cdc.gov/podcasts/rss.asp
The relevant one is Travel Notices
http://www2c.cdc.gov/podcasts/createrss.asp?t=r&c=66
Its available as XML and RSS.
Hope that helps.
Cheers for all your great work - I think this has potential to be very
helpful to people in a whole range of clinical situations.
Mike
Ross Jones
Nov 15, 2011, 11:36:59 AM11/15/11
to oss-uk...@googlegroups.com
Oh, I think I already made a start on that…. https://scraperwiki.com/scrapers/cdc_travel_notices_1/
Ross
Carl Reynolds
Jan 7, 2012, 3:41:25 PM1/7/12
to oss-uk-health
Hi guys, just an update on this, I've begun having a look at cleaning
the WHO data using google refine which is an awesome tool that I can
thoroughly recommend.
I took the problem/idea to the london pydojo this Thursday and there
was a lot of interest but implementing the mastermind board game in
python with AI pipped infections in travellers.
However, I did hear about biocaster http://born.nii.ac.jp/ which is an
interesting project and there was a lot of interest in working on this
problem another time.
Best, Carl
On Nov 15 2011, 4:36 pm, Ross Jones <r...@servercode.co.uk> wrote:
> Oh, I think I already made a start on that….https://scraperwiki.com/scrapers/cdc_travel_notices_1/
> >>>>>>>> The scraper is athttps://scraperwiki.com/scrapers/who_outbreaks/althoughIsuspecteither the download spreadsheet (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=...>`swdata`&apikey= ) or the API (https://scraperwiki.com/docs/api?name=who_outbreaks#sqlite) or a HTML table view (https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...<https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable...> desc ) might be useful.
the WHO data using google refine which is an awesome tool that I can
thoroughly recommend.
I took the problem/idea to the london pydojo this Thursday and there
was a lot of interest but implementing the mastermind board game in
python with AI pipped infections in travellers.
However, I did hear about biocaster http://born.nii.ac.jp/ which is an
interesting project and there was a lot of interest in working on this
problem another time.
Best, Carl
On Nov 15 2011, 4:36 pm, Ross Jones <r...@servercode.co.uk> wrote:
> Oh, I think I already made a start on that….https://scraperwiki.com/scrapers/cdc_travel_notices_1/
Message has been deleted
Mike Marks
Jan 30, 2012, 1:23:08 PM1/30/12
to oss-uk...@googlegroups.com
Apologies for my tardiness in following up on this - ICU night shifts
are currently wrecking havoc on me.
Google refine is indeed very nice.
Biocaster looks interesting - similar but broader in some respects
than healthmap; it wasn't clear to me what range of sources they take
their data from and how this compares to healthmap - but will look
into this.
are currently wrecking havoc on me.
Google refine is indeed very nice.
Biocaster looks interesting - similar but broader in some respects
than healthmap; it wasn't clear to me what range of sources they take
their data from and how this compares to healthmap - but will look
into this.
I have just dropped the people at healthmap another email on this to
follow up so will see if that leads anywhere as well as that project
is quite well fleshed out already and should in theory be the easiest
for us to build on if they were to be interested.
Michael
On 07/01/2012, ga...@cam.ac.uk <ga...@cam.ac.uk> wrote:
> Biocaster looks good. I particularly like the trending graphs on flu. Thanks
> for raising ID again at the meeting.
>
> As much as I like infections, the mastermind game does sound cool.
> Sent from my BlackBerry®
Reply all
Reply to author
Forward
0 new messages