Re: [Firebase] Push an array of JSON objects one by one from a text file using cURL and jq

[Kato Richardson]

I replied to this same post before, but can't find the original to reference here.

The issue is that while read line is literally reading the file one line at a time and then trying to push that line up to the server as JSON content. It isn't of course, a valid JSON structure unless one includes the entire structure in the request.

The solution is to use a command line tool or node script that can stream the file or, if it's fairly small, to just load it into memory and push it all at once.

Cheers,

Kato

On Sat, Mar 7, 2015 at 5:46 PM, Jacob Wenger <ja...@firebase.com> wrote:

[I accidentally deleted this message and I don't see any way to undo that, so I'm posting it myself.]

Hi,

I've been trying to mix those guys

https://www.firebase.com/docs/rest/api/#section-post

https://stedolan.github.io/jq/manual/?#Invokingjq

with the ultimate goal of pushing objects one by one from a JSON

array stored inside a text file.

Given a test.json like this.

[
  {
    "name": "Joe",
    "title": "hacker"
  },
  {
    "name": "Jen",
    "title": "hacker"
  }
]



I've tried without success a number of combinations on the command line such as:

$ jq -r '.[] | tostring | @sh' test.json | while read line; do curl -vX POST -d ${line} https://my-firebase.firebaseio.com/test.json; done;

Which prints the following:

* Hostname was NOT found in DNS cache

 

*   Trying 75.126.166.164... 

* Connected to my-firebase.firebaseio.com (75.126.166.164) port 443 (#0) 

* TLS 1.2 connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 

* Server certificate: *.firebaseio.com 

* Server certificate: RapidSSL SHA256 CA - G3 

* Server certificate: GeoTrust Global CA 

> POST /test.json HTTP/1.1 

> User-Agent: curl/7.37.1 

> Host: my-firebase.firebaseio.com 

> Accept: */* 

> Content-Length: 33 

> Content-Type: application/x-www-form-urlencoded 

>  

* upload completely sent off: 33 out of 33 bytes 

< HTTP/1.1 400 Bad Request 

< Content-Length: 136 

< Strict-Transport-Security: max-age=31556926; includeSubDomains; preload 

< Content-Type: application/json; charset=utf-8 

< Cache-Control: no-cache 

<  

{ 

  "error" : "Invalid data; couldn't parse JSON object, array, or value. Perhaps you're using invalid characters in your key names." 

} 

* Connection #0 to host my-firebase.firebaseio.com left intact 

* Hostname was NOT found in DNS cache 

*   Trying 75.126.166.173... 

* Connected to my-firebase.firebaseio.com (75.126.166.173) port 443 (#0) 

* TLS 1.2 connection using TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 

* Server certificate: *.firebaseio.com 

* Server certificate: RapidSSL SHA256 CA - G3 

* Server certificate: GeoTrust Global CA 

> POST /test.json HTTP/1.1 

> User-Agent: curl/7.37.1 

> Host: my-firebase.firebaseio.com 

> Accept: */* 

> Content-Length: 33 

> Content-Type: application/x-www-form-urlencoded 

>  

* upload completely sent off: 33 out of 33 bytes 

< HTTP/1.1 400 Bad Request 

< Content-Length: 136 

< Strict-Transport-Security: max-age=31556926; includeSubDomains; preload 

< Content-Type: application/json; charset=utf-8 

< Cache-Control: no-cache 

<  

{ 

  "error" : "Invalid data; couldn't parse JSON object, array, or value. Perhaps you're using invalid characters in your key names." 

} 
* Connection #0 to host my-firebase.firebaseio.com left intact

Can anybody help me to figure out what's going on?

This output already points out the content length however I'm afraid it would help best if I could manage to see what's actually being transferred within the request body.

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/4a0c49f1-4529-408c-818f-6bf9c29edfc0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.