reading binary blob

[Brian Craft]

I need to parse some floats from a binary blob. The whole blob is floats, so the number of floats can be found from the blob size.

 Is gloss the best way to do it? Is there anything simpler? gloss doesn't seem to have a nice way to saying "read floats until you run out of bytes". I also couldn't get the (repeat) call to work for this. It seems to have its own ideas about how data should look, rather than being a general parser. I was eventually able to get it to work by doing

(gloss.core/compile-frame {:scores (repeat count :float32)})

before decoding each record, which creates a gloss codec for each record, which seems a bit much.

[Steffen]

Hi,

I wrote a binary parser combinator(ish) library that uses java's input/outputstreams (https://github.com/smee/binary) that may help (add [org.clojars.smee/binary "0.2.0"] to your project's leiningen dependencies).

As I use streams, I have no way of knowing the size of a binary blob beforehand, so in principal you have three options: Using a length prefix, using an arbitrary prefix or iteratively read floats until you run out of bytes:

 

(use 'org.clojars.smee.binary.core))

(def codec-prefix (repeated :float :prefix :int))

(def codec-length (repeated :float :length 50))

(def codec-simple (compile-codec :float))

;; call with either

(defn by-prefix [^java.io.InputStream in]

  (decode codec-prefix in))

(defn by-length [^java.io.InputStream in]

  (decode codec-length in))

(defn by-greedy [^java.io.InputStream in]

  (take-while (complement nil?) 

              (repeatedly #(try 

                             (decode codec-simple in)

                             (catch java.io.EOFException e nil)))))  

Each call would return a sequence of floats. You can write a sequence of floats back with "encode", of course.

Currently the documentation is nothing to be proud of, but it is in the pipeline. Instead please refer to the beginning of an mp3 idv2 parser at https://github.com/smee/binary/blob/master/src/org/clojars/smee/binary/demo/mp3.clj or the tests of the project. 

Steffen

[Brian Craft]

If I'm holding a binary blob from a db query result, is clojure.java.io/input-stream the right way to present it as a stream?

I'm doing 

  (:require [org.clojars.smee.binary.core :as binary])

  (:require [clojure.java.io :as io])

...

(defn score-decode [blob]

  (binary/decode codec-length (io/input-stream blob)))

which seems to work.