Re: [KoBo Users] Pull data using Briefcase / humanitarianresponse.info ?

[ronald munjoma]

Hi Francis,

On 15 April 2015 at 15:47, Francis Vachon <francisv...@gmail.com> wrote:

Hi all,

I'm having trouble pulling forms using briefcase (1.4.5, latest version) from

​​

humanitarianresponse.info. I keep getting FAILED, with following details:

Fetching form definition

resolving against briefcase form definitions

Fetching form manifest

preparing to retrieve instance data

retrieving next chunk of instances from server...

NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch of submission download chunk failed.  Detailed error: FORBIDDEN (403) while accessing: https://kc.humanitarianresponse.info/franck26/view/submissionList?formId=encryptedForm2&cursor=&numEntries=100

Please verify that the URL, your user credentials and your permissions are all correct.

FAILED.

​I do not use Kobotoolbox, but came across the same error whilst pulling from ona.io. I managed to solve it by adding my user name to the url (briefcase endpoints are not yet implemented in ona.io)

​In your case I think it should look like: humanitarianresponse.info/[account_usernane_here]​

Hope this will work for you.

Best and Regards

Ronald Munjoma

So it does seem to get the form definition (I can actually check the XML in the Briefcase storage). If I check the provided address for the request I get:

I only found FormHub's page on using briefcase (http://blog.formhub.org/2013/06/27/formhub-supports-odk-briefcase/), but it doesn't offer much help. I could pull from Aggregate no problem for those same forms. The specific form shown here is encrypted, however I have also an unencrypted form, which leads to the same result.

The address is connects to is good (since I do get the form definition list), & credentials seem good as well (I tried deliberately entered wrong credential, but then Briefcase just wouldn't connect).

--
You received this message because you are subscribed to the Google Groups "Kobo Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kobo-users+...@googlegroups.com.
To post to this group, send email to kobo-...@googlegroups.com.
Visit this group at http://groups.google.com/group/kobo-users.
For more options, visit https://groups.google.com/d/optout.

--

Best and Regards

Ronald Munjoma

+263 775 625 283, +263 776 444 309

[Tino Kreutzer]

Hi Francis,

Pulling from https://kc.humanitarianresponse.info with Briefcase definitely works (I just tested it again), so we'll have to see what the issue is in your particular case. 

It sounds like you might have an issue with a specific form, but as Ronald just pointed out, in Briefcase's "Connect..." window make sure you enter https://kc.humanitarianresponse.info/username [no trailing slash at the end]

Did you try to pull data from other forms/projects to see if this is a particular issue with this form you're trying to pull? 

Best,

Tino

[Francis Vachon]

Hi guys,

So I have been trying again and no luck. There is something, at some point, that blocks Briefcase from pulling anything from Kobo. I tried with 2 different Kobo account and many different forms, to no avail.

Yes, for connection I do have https://kc.humanitarianresponse.info/myaccount (not trailing slash)

I tried both with & without the credentials. I always get the very same result in all cases (from the "Details" section in Briefcase):

Fetching form definition

resolving against briefcase form definitions

Fetching form manifest

preparing to retrieve instance data

retrieving next chunk of instances from server...

NOT ALL SUBMISSIONS RETRIEVED: Error fetching list of submissions: Fetch of submission download chunk failed.  Detailed error: FORBIDDEN (403) while accessing: https://kc.humanitarianresponse.info/franck26/view/submissionList?formId=GLO-PH-RepWomen-ref-27-XLS&cursor=&numEntries=100

Please verify that the URL, your user credentials and your permissions are all correct.

FAILED.

Also, if I go into the briefcase storage area that I have defined, I notice:

• The Form definition has been retrived (that's consistant with the details shown above, that bit did work fine)
• under C:/BriefcaseStorageArea/ I have 2 temp files (xml files) that are actually my submissions. E.g. if I open those, I do see my filled forms with the appropriate answers listed.

So it would seem that for some reasons, Briefcase did retrieve the data. It just couldn't/wouldn't get them under CSV format for some reason. Under BriefcaseStorage/forms/myForm/instances/, there is indeed nothing.

If I try to export what Briefcase "retrieved", then I just get empty output CSV files (with only the headers - as if I had pulled form definitions without any submissions, but then that's probably expected since the pull didn,t actually succeed)....

Any thoughts?

[Francis Vachon]

Another detail I just noticed:

If I try to pull once, then I get the form definition (without any of the temp files mentionned above). If I pull gain, I get one. Then again I get another one. Attached one of those temps files.

Ok I don't know what's wrong with me today but the file uploader doesn't work either (e.g. I can't upload the temp file. In fact I can't even uplaod a screenshot of it, or anything...)

On Wednesday, April 15, 2015 at 4:16:57 PM UTC+2, Ronald Munjoma wrote:

[Francis Vachon]

Additionnal info:

So looking back the above-mentionned details, I tried to copy-past into my address bar the request for the form submission list:

https://kc.humanitarianresponse.info/franck26//view/submissionList?formId=GLO-PH-RepWomen-ref-27-XLS

Then I get to a pop-up asking for username & password (DJANGO). Logical: this would be what you specify in Briefcase when you try to  connect, so the server can authenticate you & allow or deny access.

However, if I enter my credential in that pop, access is denied with "wrong credential". I tried to log out of mytest account & re-logging (to ensure I do indeed use the right credentials). Those work. But the very same info entered at the submission list step is denied.

Am I really the only one experiencing this? Or just few people are using briefcase with Kobo?

[Francis Vachon]

Additional info:

So in the end it seems to be related to newly created account (e.g. since some point int 2015). I have written an email to the Kobo team on it to report the bug, as I am pretty sure this is indeed a bug.

Basically I have tried with quite a few different accounts, and they all failed in the same fashion (see above). Then at cartONG they tried as well on CartONG's test account, and everything worked fine. THen I tried as well, but with an older account created back in 2014 (late in the year) and it work fine as well.

All the failling accounts had in common that they have been created in 2015. I tried setting up a brand new account and it failed as well.

Therefore if others are experiencing the same problem, it has a high chance of being related to when you account was created. I didn't find a work around about it, it might be a back-end issue actually.... So stay tune for further info.

Francis 

[Patrick Vinck]

Thank you Francis, we will be checking on this as soon as possible.Response o the Nepal earthquake has slowed us down a little but we are mindful of the problem.

Best,

Patrick

--

[Tino Kreutzer]

Hi Francis,

Thanks for sending this detailed report. That's very useful. We'll look into and fix whatever prevents newer users from downloading data through ODK Briefcase. I'll send a message back here when that's done. 

Best,

Tino

--
Tino Kreutzer

Program Manager KoBoToolbox 

Tel +1 617 372 7051 | Skype tinokreutzer
tino.k...@kobotoolbox.org

--

[Francis Vachon]

Hi all,

Just giving this guy a kick. I'm also noticing that since Briefcase won't work, you can't use encryption on newer accounts eithers (as it relies on Briefcase). Well assuming that this is indeed about newer accounts, but I could manager to make it work it olders ones without fail therefore it has to be something of the kind...

Francis

On Wednesday, April 15, 2015 at 10:16:57 AM UTC-4, Ronald Munjoma wrote:

[Tino Kreutzer]

Hi Francis,

I've tested this on both new and older accounts and I'm able to pull with Briefcase without any problems. This leads me to believe that this is an issue either with your username or with a specific form. In order to test this it would help if you could send us your user credentials to sup...@kobotoolbox.org (change your password first if need be) so we can try to reproduce this issue. 

Best,

Tino

[GARCIN Yann]

Hello everybody,

 

I have the same problem as Francis, using briefcase to pull from my kobo account created last week. Each time I tried to pull some data from the kobo server it failed, and briefcase returned me a "FORBIDDEN (403)" error.

 

I also tried with the CartONG account which was created last summer in August and everything worked well.

 

If you have some new information on this topic I would be interested.

 

Best,

 

Yann

[Tino Kreutzer]

Hi Francis and Yann,

I just wanted to let you know that the Briefcase permissions issue was fixed on our servers. All out tests indicate that this issue is now fixed for all users on both servers. 

All the best,

Tino

--

[Dyala Rousan]

Hi Tino,

Can you please assist me withe the ODK briefcase I really can make ODK briefcase to pull the kobo forms at all i tried so many things but none worked whit me. I have 2 kobo forms one has encryptoin and the other one donst have any encryption just normal form. 

I am trying to use ODK briefcase to pull the kobo forms but i am getting same error when pulling any of these 2 forms.

The error is as follows:

"Fetching form definition

resolving against briefcase form definitions

Error parsing form definition: org.opendatakit.aggregate.exception.ODKIncompleteSubmissionData: The data model does not have an id or xmlns attribute.  Add an id="your.domain.org:formId" attribute to the top-level instance data element of your form."

I am attaching to you the screen shot of ODK briefcase and the kobo form.

Please if you can assist me in this urgent issue.

Thank you.

Dyala

To unsubscribe from this group and stop receiving emails from it, send an email to kobo-users+unsubscribe@googlegroups.com.