is GET /loans/search supposed to only return a partial result-set? if not, there is a bug where it does not return information like occupation, loan_count, invite_count, etc. if it is designed to only return this partial data, is there a way to request a change that might support the use of a parameter to return the entire data-set? This change would make querying lender data so much more efficiently. Thanks.
http://api.kivaws.org/v1/loans/search.json is paginated, so returns one
page at a time. There have recently been discussions about adding a
?page=all option to the loans page, but that is still in its infancy. You
can also use the snapshots for a relatively recent data dump.
Regards,
Martin
On 15 August 2012 10:08, thepip3r <cameron.wils...@gmail.com> wrote:
> is GET /loans/search supposed to only return a partial result-set? if
> not, there is a bug where it does not return information like occupation,
> loan_count, invite_count, etc. if it is designed to only return this
> partial data, is there a way to request a change that might support the use
> of a parameter to return the entire data-set? This change would make
> querying lender data so much more efficiently. Thanks.
> --
> You received this message because you are subscribed to the Google Groups
> "build-kiva" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/build-kiva/-/FVHSFIzglSAJ.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
To start, this is supposed to cover the 'lender' portion of the api, not the loans. And if by paginated, you mean that the results are returned in pages or chunks, that's fine. I really have no problem with that at all.
My problem with search.json is that the all of the fields are NOT returned. As an example, if I run this:
{"lenders":[{"lender_id":"thewilsons","name":"Cameron & Michelle","image":{"id":281368,"template_id":1},"whereabouts":"Spokane WA","country_code":"US","uid":"thewilsons","member_since":"2009-02-17T07:01 :15Z","personal_url":"
www.argumentforatheism.org","occupation":"Enterprise IT Engineer & Insurance Underwriter","loan_because":"the concept behind this site is fantastic -- we love being able to directly make a positive impact on other peoples' lives in ways not previously possible.","occupational_info":"Cameron works with large computer systems.\r\n\r\nMichelle researches and provides commercial insurance to businesses.\r\n\r\n\r\n","loan_count":231,"invitee_count":2,"inviter_id":0} ]}
If I run, http://api.kivaws.org/v1/lenders/search.json, I only get back the lender_id, name, whereabouts, country_code, and uid -- only a fraction of the fields. As it stands, the only way I can see to get all of the lender data (besides using the old data dumps (back in April)), is to query search.json to get all the users and then run the same queries over again for groups of users to get all of their fields using the http://api.kivaws.org/v1/lenders/lender1,lender2,lender3,....json method.
Is anyone familiar with a more efficient way to acquire this information?
On Wednesday, August 15, 2012 2:19:30 PM UTC-4, Martin Butt wrote:
> Hi Cameron,
> http://api.kivaws.org/v1/loans/search.json is paginated, so returns one > page at a time. There have recently been discussions about adding a > ?page=all option to the loans page, but that is still in its infancy. You > can also use the snapshots for a relatively recent data dump.
> Regards,
> Martin
> On 15 August 2012 10:08, thepip3r <cameron...@gmail.com <javascript:>>wrote:
>> is GET /loans/search supposed to only return a partial result-set? if >> not, there is a bug where it does not return information like occupation, >> loan_count, invite_count, etc. if it is designed to only return this >> partial data, is there a way to request a change that might support the use >> of a parameter to return the entire data-set? This change would make >> querying lender data so much more efficiently. Thanks.
>> -- >> You received this message because you are subscribed to the Google Groups >> "build-kiva" group.
>> To view this discussion on the web visit >> https://groups.google.com/d/msg/build-kiva/-/FVHSFIzglSAJ.
>> To post to this group, send email to build...@googlegroups.com<javascript:>
>> .
>> To unsubscribe from this group, send email to >> build-kiva+...@googlegroups.com <javascript:>.
>> For more options, visit this group at >> http://groups.google.com/group/build-kiva?hl=en.
There is a summary view and a detailed view to various entities, loans,
lenders, etc. I agree that it would be nice to be able to specify detail
levels... KivaLens first pages through every fundraising loan and if
there's a new one it has to request the detail page separately to get the
repayment info and store it in my DB. Being able to just get a list of IDs
as the most basic result all the way up to full details included in a
result would be quite handy.
Martin misunderstood what was meant by "partial results."
On Thu, Aug 16, 2012 at 5:03 AM, thepip3r <cameron.wils...@gmail.com> wrote:
> Hi Martin,
> To start, this is supposed to cover the 'lender' portion of the api, not
> the loans. And if by paginated, you mean that the results are returned in
> pages or chunks, that's fine. I really have no problem with that at all.
> My problem with search.json is that the all of the fields are NOT
> returned. As an example, if I run this:
> {"lenders":[{"lender_id":"thewilsons","name":"Cameron &
> Michelle","image":{"id":281368,"template_id":1},"whereabouts":"Spokane
> WA","country_code":"US","uid":"thewilsons","member_since":"2009-02-17T07:01 :15Z","personal_url":"
> www.argumentforatheism.org","occupation":"Enterprise IT Engineer &
> Insurance Underwriter","loan_because":"the concept behind this site is
> fantastic -- we love being able to directly make a positive impact on other
> peoples' lives in ways not previously
> possible.","occupational_info":"Cameron works with large computer
> systems.\r\n\r\nMichelle researches and provides commercial insurance to
> businesses.\r\n\r\n\r\n","loan_count":231,"invitee_count":2,"inviter_id":0} ]}
>> On 15 August 2012 10:08, thepip3r <cameron...@gmail.com> wrote:
>>> is GET /loans/search supposed to only return a partial result-set? if
>>> not, there is a bug where it does not return information like occupation,
>>> loan_count, invite_count, etc. if it is designed to only return this
>>> partial data, is there a way to request a change that might support the use
>>> of a parameter to return the entire data-set? This change would make
>>> querying lender data so much more efficiently. Thanks.
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "build-kiva" group.
>>> To view this discussion on the web visit https://groups.google.com/d/** >>> msg/build-kiva/-/FVHSFIzglSAJ<https://groups.google.com/d/msg/build-kiva/-/FVHSFIzglSAJ>
>>> .
>>> To post to this group, send email to build...@googlegroups.com.
>>> To unsubscribe from this group, send email to build-kiva+...@**
>>> googlegroups.com.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
-- Sent from my computer. No excuses for misspellings or missing words.
Thanks for the help Martin and Paul. I'm sorry about incorrectly posting the title to indicate this was for loans. And yes, the reason for my question is that the script I wrote to sync my DB with Kivas is to use search.json to essentially get back a page of lenders, grab their lender ids, and construct a new URL with each lender_id comma-separated to requery Kiva's DB for the full dataset. This seems rather inefficient to me but I don't see another way around the problem and from what you're saying, it sounds like there isn't an answer to the problem as the api currently stands.
On Thursday, August 16, 2012 2:00:54 PM UTC-4, Paul wrote:
> There is a summary view and a detailed view to various entities, loans, > lenders, etc. I agree that it would be nice to be able to specify detail > levels... KivaLens first pages through every fundraising loan and if > there's a new one it has to request the detail page separately to get the > repayment info and store it in my DB. Being able to just get a list of IDs > as the most basic result all the way up to full details included in a > result would be quite handy.
> Martin misunderstood what was meant by "partial results."
> Paul
> On Thu, Aug 16, 2012 at 5:03 AM, thepip3r <cameron...@gmail.com<javascript:>
> > wrote:
>> Hi Martin,
>> To start, this is supposed to cover the 'lender' portion of the api, not >> the loans. And if by paginated, you mean that the results are returned in >> pages or chunks, that's fine. I really have no problem with that at all.
>> My problem with search.json is that the all of the fields are NOT >> returned. As an example, if I run this:
>> {"lenders":[{"lender_id":"thewilsons","name":"Cameron & >> Michelle","image":{"id":281368,"template_id":1},"whereabouts":"Spokane >> WA","country_code":"US","uid":"thewilsons","member_since":"2009-02-17T07:01 :15Z","personal_url":"
>> www.argumentforatheism.org","occupation":"Enterprise IT Engineer & >> Insurance Underwriter","loan_because":"the concept behind this site is >> fantastic -- we love being able to directly make a positive impact on other >> peoples' lives in ways not previously >> possible.","occupational_info":"Cameron works with large computer >> systems.\r\n\r\nMichelle researches and provides commercial insurance to >> businesses.\r\n\r\n\r\n","loan_count":231,"invitee_count":2,"inviter_id":0} ]}
>>> On 15 August 2012 10:08, thepip3r <cameron...@gmail.com> wrote:
>>>> is GET /loans/search supposed to only return a partial result-set? if >>>> not, there is a bug where it does not return information like occupation, >>>> loan_count, invite_count, etc. if it is designed to only return this >>>> partial data, is there a way to request a change that might support the use >>>> of a parameter to return the entire data-set? This change would make >>>> querying lender data so much more efficiently. Thanks.
>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "build-kiva" group.
>>>> To view this discussion on the web visit https://groups.google.com/d/** >>>> msg/build-kiva/-/FVHSFIzglSAJ<https://groups.google.com/d/msg/build-kiva/-/FVHSFIzglSAJ>
>>>> .
>>>> To post to this group, send email to build...@googlegroups.com.
>>>> To unsubscribe from this group, send email to build-kiva+...@**
>>>> googlegroups.com.
>> To post to this group, send email to build...@googlegroups.com<javascript:>
>> .
>> To unsubscribe from this group, send email to >> build-kiva+...@googlegroups.com <javascript:>.
>> For more options, visit this group at >> http://groups.google.com/group/build-kiva?hl=en.
> -- > Sent from my computer. No excuses for misspellings or missing words.
> Thanks for the help Martin and Paul. I'm sorry about incorrectly posting
> the title to indicate this was for loans. And yes, the reason for my
> question is that the script I wrote to sync my DB with Kivas is to use
> search.json to essentially get back a page of lenders, grab their lender
> ids, and construct a new URL with each lender_id comma-separated to requery
> Kiva's DB for the full dataset. This seems rather inefficient to me but I
> don't see another way around the problem and from what you're saying, it
> sounds like there isn't an answer to the problem as the api currently
> stands.
> Thanks all,
> Cameron
> On Thursday, August 16, 2012 2:00:54 PM UTC-4, Paul wrote:
>> There is a summary view and a detailed view to various entities, loans,
>> lenders, etc. I agree that it would be nice to be able to specify detail
>> levels... KivaLens first pages through every fundraising loan and if
>> there's a new one it has to request the detail page separately to get the
>> repayment info and store it in my DB. Being able to just get a list of IDs
>> as the most basic result all the way up to full details included in a
>> result would be quite handy.
>> Martin misunderstood what was meant by "partial results."
>> Paul
>> On Thu, Aug 16, 2012 at 5:03 AM, thepip3r <cameron...@gmail.com> wrote:
>>> Hi Martin,
>>> To start, this is supposed to cover the 'lender' portion of the api, not
>>> the loans. And if by paginated, you mean that the results are returned in
>>> pages or chunks, that's fine. I really have no problem with that at all.
>>> My problem with search.json is that the all of the fields are NOT
>>> returned. As an example, if I run this:
>>> {"lenders":[{"lender_id":"**thewilsons","name":"Cameron &
>>> Michelle","image":{"id":**281368,"template_id":1},"**whereabouts":"Spokane
>>> WA","country_code":"US","uid":**"thewilsons","member_since":"**
>>> 2009-02-17T07:01:15Z","**personal_url":"www.**argumentforatheism.org","*
>>> *occupation":"Enterprise IT Engineer & Insurance
>>> Underwriter","loan_because":"**the concept behind this site is
>>> fantastic -- we love being able to directly make a positive impact on other
>>> peoples' lives in ways not previously possible.","occupational_info"**:"Cameron
>>> works with large computer systems.\r\n\r\nMichelle researches and provides
>>> commercial insurance to businesses.\r\n\r\n\r\n","**
>>> loan_count":231,"invitee_**count":2,"inviter_id":0}]}
>>>> On 15 August 2012 10:08, thepip3r <cameron...@gmail.com> wrote:
>>>>> is GET /loans/search supposed to only return a partial result-set? if
>>>>> not, there is a bug where it does not return information like occupation,
>>>>> loan_count, invite_count, etc. if it is designed to only return this
>>>>> partial data, is there a way to request a change that might support the use
>>>>> of a parameter to return the entire data-set? This change would make
>>>>> querying lender data so much more efficiently. Thanks.
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "build-kiva" group.
>>>>> To view this discussion on the web visit https://groups.google.com/d/* >>>>> *ms**g/build-kiva/-/FVHSFIzglSAJ<https://groups.google.com/d/msg/build-kiva/-/FVHSFIzglSAJ>
>>>>> .
>>>>> To post to this group, send email to build...@googlegroups.com.
>>>>> To unsubscribe from this group, send email to build-kiva+...@**
>>>>> googlegroups.**com.
>>> To post to this group, send email to build...@googlegroups.com.
>>> To unsubscribe from this group, send email to build-kiva+...@**
>>> googlegroups.com.
>>> For more options, visit this group at http://groups.google.com/** >>> group/build-kiva?hl=en <http://groups.google.com/group/build-kiva?hl=en>
>>> .
>> --
>> Sent from my computer. No excuses for misspellings or missing words.
> To post to this group, send email to build-kiva@googlegroups.com.
> To unsubscribe from this group, send email to
> build-kiva+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/build-kiva?hl=en.
-- Sent from my computer. No excuses for misspellings or missing words.
