As many of you know, we used to share API snapshots, zipped files that include all the loan and lender API objects. Their publication was always a little buggy, and it ceased entirely at some point in 2010. Anyone who's downloaded a snapshot since then has had only very stale data to look at.
Last night, we got the API snapshot script to run cleanly for the first time since 2010! It's not yet configured to run on a regular schedule, but for now, there are fresh snapshots available.
Awesome, thanks! Looks like this snapshot has more than twice the data the last one had; the previous snapshot (from Jan 2010) had over 165k loans, and this one has over 410k. Expect an updated world map visualization in... several months :'(
On Wed, Apr 11, 2012 at 3:48 PM, Noah Balmer <n...@kiva.org> wrote: > As many of you know, we used to share API snapshots, zipped files that > include all the loan and lender API objects. Their publication was always > a little buggy, and it ceased entirely at some point in 2010. Anyone who's > downloaded a snapshot since then has had only very stale data to look at.
> Last night, we got the API snapshot script to run cleanly for the first > time since 2010! It's not yet configured to run on a regular schedule, but > for now, there are fresh snapshots available.
> -- > You received this message because you are subscribed to the Google Groups > "build-kiva" group. > To post to this group, send email to build-kiva@googlegroups.com. > To unsubscribe from this group, send email to > build-kiva+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/build-kiva?hl=en.
On Wed, Apr 11, 2012 at 1:48 PM, Noah Balmer <n...@kiva.org> wrote: > As many of you know, we used to share API snapshots, zipped files that > include all the loan and lender API objects. Their publication was always > a little buggy, and it ceased entirely at some point in 2010. Anyone who's > downloaded a snapshot since then has had only very stale data to look at.
> Last night, we got the API snapshot script to run cleanly for the first > time since 2010! It's not yet configured to run on a regular schedule, but > for now, there are fresh snapshots available.
> -- > You received this message because you are subscribed to the Google Groups > "build-kiva" group. > To post to this group, send email to build-kiva@googlegroups.com. > To unsubscribe from this group, send email to > build-kiva+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/build-kiva?hl=en.
> As many of you know, we used to share API snapshots, zipped files that
> include all the loan and lender API objects. Their publication was always
> a little buggy, and it ceased entirely at some point in 2010. Anyone who's
> downloaded a snapshot since then has had only very stale data to look at.
> Last night, we got the API snapshot script to run cleanly for the first
> time since 2010! It's not yet configured to run on a regular schedule, but
> for now, there are fresh snapshots available.
As promised, here's an updated "Kiva world map." I was a little
disappointed to find the visualization hadn't changed much since the last
snapshot (from 2010), despite a lot of new unique data points being added.
They will look different, though; I realized that the previous map didn't
look to everyone else as it did to me (my monitor's colors were a little
messed up I guess), so this time around it should look how I originally
intended...
I was toying with the idea of making a poster out of it; if anyone would be
interested in something like that, let me know and I can generate a much
higher-res image.
On Wed, Apr 18, 2012 at 2:24 PM, Bryan Stroube <bryan.stro...@gmail.com>wrote:
> On Apr 11, 4:48 pm, Noah Balmer <n...@kiva.org> wrote:
> > As many of you know, we used to share API snapshots, zipped files that
> > include all the loan and lender API objects. Their publication was
> always
> > a little buggy, and it ceased entirely at some point in 2010. Anyone
> who's
> > downloaded a snapshot since then has had only very stale data to look at.
> > Last night, we got the API snapshot script to run cleanly for the first
> > time since 2010! It's not yet configured to run on a regular schedule,
> but
> > for now, there are fresh snapshots available.
> --
> You received this message because you are subscribed to the Google Groups
> "build-kiva" group.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
Nicely done. I've talked with some people about how we could put these on
lender pages or team pages.
If you could at some point create an Amazon EC2 image that I could load up
and run to generate these, then I could get it working towards running on
our side. I see there have been some minor adjustments since the last time
I looked at it:
https://github.com/sndurkin/kiva-map/commit/0bbb1238379626c5d577c888b...
I think we could pretty easily script an install that would be able to
generate these.
On Thu, Oct 4, 2012 at 6:18 PM, Sean Durkin <sndur...@gmail.com> wrote:
> As promised, here's an updated "Kiva world map." I was a little
> disappointed to find the visualization hadn't changed much since the last
> snapshot (from 2010), despite a lot of new unique data points being added.
> They will look different, though; I realized that the previous map didn't
> look to everyone else as it did to me (my monitor's colors were a little
> messed up I guess), so this time around it should look how I originally
> intended...
> I was toying with the idea of making a poster out of it; if anyone would
> be interested in something like that, let me know and I can generate a much
> higher-res image.
> On Wed, Apr 18, 2012 at 2:24 PM, Bryan Stroube <bryan.stro...@gmail.com>wrote:
>> This is excellent news, very much appreciated!
>> Bryan
>> On Apr 11, 4:48 pm, Noah Balmer <n...@kiva.org> wrote:
>> > As many of you know, we used to share API snapshots, zipped files that
>> > include all the loan and lender API objects. Their publication was
>> always
>> > a little buggy, and it ceased entirely at some point in 2010. Anyone
>> who's
>> > downloaded a snapshot since then has had only very stale data to look
>> at.
>> > Last night, we got the API snapshot script to run cleanly for the first
>> > time since 2010! It's not yet configured to run on a regular schedule,
>> but
>> > for now, there are fresh snapshots available.
>> --
>> You received this message because you are subscribed to the Google Groups
>> "build-kiva" group.
>> To post to this group, send email to build-kiva@googlegroups.com.
>> To unsubscribe from this group, send email to
>> build-kiva+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/build-kiva?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "build-kiva" group.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
Is there still interest in generating maps for lenders & teams? If so, I
could look into how to get the script running on Amazon EC2 (maybe you
could give me a couple pointers?), but I figured you would want to change
the script, because it just uses the API to fetch all the data (which, for
teams, can be a lot). I think it would be faster and more efficient if it
made specific queries directly to the DB. Or are you talking about more of
a proof-of-concept?
On Thu, Oct 4, 2012 at 9:47 PM, Kevin O'Brien <ke...@kiva.org> wrote:
> Nicely done. I've talked with some people about how we could put these on
> lender pages or team pages.
> If you could at some point create an Amazon EC2 image that I could load up
> and run to generate these, then I could get it working towards running on
> our side. I see there have been some minor adjustments since the last time
> I looked at it:
> https://github.com/sndurkin/kiva-map/commit/0bbb1238379626c5d577c888b...
> I think we could pretty easily script an install that would be able to
> generate these.
> Anyways, again, nice job!
> On Thu, Oct 4, 2012 at 6:18 PM, Sean Durkin <sndur...@gmail.com> wrote:
>> As promised, here's an updated "Kiva world map." I was a little
>> disappointed to find the visualization hadn't changed much since the last
>> snapshot (from 2010), despite a lot of new unique data points being added.
>> They will look different, though; I realized that the previous map didn't
>> look to everyone else as it did to me (my monitor's colors were a little
>> messed up I guess), so this time around it should look how I originally
>> intended...
>> I was toying with the idea of making a poster out of it; if anyone would
>> be interested in something like that, let me know and I can generate a much
>> higher-res image.
>> On Wed, Apr 18, 2012 at 2:24 PM, Bryan Stroube <bryan.stro...@gmail.com>wrote:
>>> This is excellent news, very much appreciated!
>>> Bryan
>>> On Apr 11, 4:48 pm, Noah Balmer <n...@kiva.org> wrote:
>>> > As many of you know, we used to share API snapshots, zipped files that
>>> > include all the loan and lender API objects. Their publication was
>>> always
>>> > a little buggy, and it ceased entirely at some point in 2010. Anyone
>>> who's
>>> > downloaded a snapshot since then has had only very stale data to look
>>> at.
>>> > Last night, we got the API snapshot script to run cleanly for the first
>>> > time since 2010! It's not yet configured to run on a regular
>>> schedule, but
>>> > for now, there are fresh snapshots available.
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "build-kiva" group.
>>> To post to this group, send email to build-kiva@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> build-kiva+unsubscribe@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/build-kiva?hl=en.
>> --
>> You received this message because you are subscribed to the Google Groups
>> "build-kiva" group.
>> To post to this group, send email to build-kiva@googlegroups.com.
>> To unsubscribe from this group, send email to
>> build-kiva+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/build-kiva?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "build-kiva" group.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
You're right that we would probably use direct db queries, although if we
could do things with the API to make the data available in more easily
digestable chunks that'd be good.
We've also got the API snapshot on our Github page (with hopefully soon to
be even more updated data), so makes it easier to pull down changes:
https://github.com/kiva/API
On Fri, Oct 5, 2012 at 10:07 AM, Sean Durkin <sndur...@gmail.com> wrote:
> Thanks!
> Is there still interest in generating maps for lenders & teams? If so, I
> could look into how to get the script running on Amazon EC2 (maybe you
> could give me a couple pointers?), but I figured you would want to change
> the script, because it just uses the API to fetch all the data (which, for
> teams, can be a lot). I think it would be faster and more efficient if it
> made specific queries directly to the DB. Or are you talking about more of
> a proof-of-concept?
> Sean
> On Thu, Oct 4, 2012 at 9:47 PM, Kevin O'Brien <ke...@kiva.org> wrote:
>> Nicely done. I've talked with some people about how we could put these on
>> lender pages or team pages.
>> If you could at some point create an Amazon EC2 image that I could load
>> up and run to generate these, then I could get it working towards running
>> on our side. I see there have been some minor adjustments since the last
>> time I looked at it:
>> https://github.com/sndurkin/kiva-map/commit/0bbb1238379626c5d577c888b...
>> I think we could pretty easily script an install that would be able to
>> generate these.
>> Anyways, again, nice job!
>> On Thu, Oct 4, 2012 at 6:18 PM, Sean Durkin <sndur...@gmail.com> wrote:
>>> As promised, here's an updated "Kiva world map." I was a little
>>> disappointed to find the visualization hadn't changed much since the last
>>> snapshot (from 2010), despite a lot of new unique data points being added.
>>> They will look different, though; I realized that the previous map didn't
>>> look to everyone else as it did to me (my monitor's colors were a little
>>> messed up I guess), so this time around it should look how I originally
>>> intended...
>>> I was toying with the idea of making a poster out of it; if anyone would
>>> be interested in something like that, let me know and I can generate a much
>>> higher-res image.
>>> On Wed, Apr 18, 2012 at 2:24 PM, Bryan Stroube <bryan.stro...@gmail.com>wrote:
>>>> This is excellent news, very much appreciated!
>>>> Bryan
>>>> On Apr 11, 4:48 pm, Noah Balmer <n...@kiva.org> wrote:
>>>> > As many of you know, we used to share API snapshots, zipped files that
>>>> > include all the loan and lender API objects. Their publication was
>>>> always
>>>> > a little buggy, and it ceased entirely at some point in 2010. Anyone
>>>> who's
>>>> > downloaded a snapshot since then has had only very stale data to look
>>>> at.
>>>> > Last night, we got the API snapshot script to run cleanly for the
>>>> first
>>>> > time since 2010! It's not yet configured to run on a regular
>>>> schedule, but
>>>> > for now, there are fresh snapshots available.
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "build-kiva" group.
>>>> To post to this group, send email to build-kiva@googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> build-kiva+unsubscribe@googlegroups.com.
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/build-kiva?hl=en.
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "build-kiva" group.
>>> To post to this group, send email to build-kiva@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> build-kiva+unsubscribe@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/build-kiva?hl=en.
>> --
>> You received this message because you are subscribed to the Google Groups
>> "build-kiva" group.
>> To post to this group, send email to build-kiva@googlegroups.com.
>> To unsubscribe from this group, send email to
>> build-kiva+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/build-kiva?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "build-kiva" group.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
