Trying to replicate command line curl binary post

[Howard Ding]

Hi,

I'm trying to use Typhoeus to post some binary data, ala the following command line curl:

curl -X POST --post301 --location-trusted --user "my-user:my-password" --digest --data-binary @pic.jpg  -H "Some-Header: Value" http://my.post.url/path

When this is made at some point it will be asked for the auth information, it may be redirected from the machine it initially makes contact with to one that will actually handle the request, and then finally the post will happen (with a 100 expect continue for good measure). (This is interacting with an HTTP based storage device, just for context.)

I've tried a variety of permutations of options in a Typhoeus request with :body, :postfieldsize, :upload, :readdata, :infilesize, and :copypostfields, but I can't seem to find a magic combination that will make this work. The following request seems to work for text files but fails for binary (and also has the disadvantage of having to read the file into memory, which I'd like to avoid but can live with if I have to):

    Typhoeus::Request.new(file_url(bit_file),

                          :postfieldsize => File.size(file_path),

                          :body => File.open(file_path, 'rb') { |f| f.read },

                          :method => :post,

                          :headers => ingest_headers(bit_file, file_path, opts)

                          :followlocation => 1,

                          :unrestricted_auth => true,

                          :httpauth => :digest,

                          :userpwd => "#{self.user}:#{self.password}",

                          :verbose => true)

Reading the docs I suspect the problem may be that the binary file is being treated as a null terminated string somewhere and hence is perceived as being shorter than it is. So I expected that replacing the :body argument with :copypostfields would stand a good chance of working. However, when I do this the POST does happen, but with an empty body, so a stream is stored on the device, but it's not the bits that I'm trying to send, but rather empty.

Any insight appreciated.

Thanks,

Howard

[Hans Hasselberg]

Good evening,

sorry for the late response - somehow your message disappeared from my inbox and I forgot about it.
You actually discovered that Typhoeus cannot properly handle when the body is _only_ a file handle. It knows what to do with strings: https://github.com/typhoeus/ethon/blob/master/lib/ethon/easy/queryable.rb#L31 and with Array or Hash: https://github.com/typhoeus/ethon/blob/master/lib/ethon/easy/queryable.rb#L58..L78. This is the reason why only passing the file didn't work out for you - which it should! Could you file an issue for ethon?

curl -X POST --post301 --location-trusted --user "my-user:my-password"
--digest --data-binary @pic.jpg  -H "Some-Header: Value"
http://my.post.url/path

I just added the support for the postredir option: http://curl.haxx.se/libcurl/c/curl_easy_setopt.html#CURLOPTPOSTREDIR. It should now work like that (point to ethon master):

Typhoeus.post(
"http://my.post.url/path",
body: File.open("pic.jpg", "r").read,
postredir: :post_301,
unrestricted_auth: true,
httpauth:  :digest,
userpwd: "my-user:my-password",
headers: {"Some-Header" => "Value"}
)

Cheers,
Hans