[ANN] Tupelo Clojure I/O Utils - tupelo.io - 0.9.185
74 views
Skip to first unread message
Alan Thompson
Jan 17, 2020, 8:04:55 PM1/17/20
to clojure
I've build up some file input/output utils over the past year that may be of use to the Clojure community. They allow one to do fine-grained I/O with native Java numeric types (both signed & unsigned), as well as create & delete temporary files & directories.
Full API docs can be found:
- on CljDoc.org: https://cljdoc.org/d/tupelo/tupelo/0.9.186
- and GitHub Pages: https://cloojure.github.io/doc/tupelo/tupelo.io.html
The unit tests show the code in action. For all the primitive numeric types, plus byte-array and String, the library can write and read to a DataOutputStream. Note that there are unsigned variants of writing/reading as well as the normal signed numeric versions.
(with-open [dos (DataOutputStream.
(FileOutputStream. dummy-file))]
(doto dos
(write-string-bytes "hello")
(write-bytes (byte-array [1 2 3 4]))
(write-byte 42)
(write-byte -42)
(write-byte Byte/MIN_VALUE)
(write-byte Byte/MAX_VALUE)
(write-byte-unsigned 142)
(write-byte-unsigned types/BYTE_UNSIGNED_MIN_VALUE)
(write-byte-unsigned types/BYTE_UNSIGNED_MAX_VALUE)
(write-short 9999)
(write-short -9999)
(write-short Short/MIN_VALUE)
(write-short Short/MAX_VALUE)
(write-short-unsigned 55999)
(write-short-unsigned types/SHORT_UNSIGNED_MIN_VALUE)
(write-short-unsigned types/SHORT_UNSIGNED_MAX_VALUE)
(write-integer int-val)
(write-integer Integer/MIN_VALUE)
(write-integer Integer/MAX_VALUE)
(write-integer-unsigned int-val)
(write-integer-unsigned types/INTEGER_UNSIGNED_MIN_VALUE)
(write-integer-unsigned types/INTEGER_UNSIGNED_MAX_VALUE)
(write-long long-val)
(write-long Long/MIN_VALUE)
(write-long Long/MAX_VALUE)
(write-long-unsigned long-val)
(write-long-unsigned types/LONG_UNSIGNED_MIN_VALUE)
(write-long-unsigned types/LONG_UNSIGNED_MAX_VALUE)
(write-float pi-float)
(write-float Float/MIN_VALUE)
(write-float Float/MAX_VALUE)
(write-double pi-double)
(write-double Double/MIN_VALUE)
(write-double Double/MAX_VALUE) ))
and the corresponding reads:
(with-open [dis (DataInputStream. (io/input-stream dummy-file))]
(is= (read-string-bytes 5 dis) "hello")
(is= (vec (read-bytes 4 dis)) [1 2 3 4])
(is= (read-byte dis) 42)
(is= (read-byte dis) -42)
(is= (read-byte dis) Byte/MIN_VALUE)
(is= (read-byte dis) Byte/MAX_VALUE)
(is= (read-byte-unsigned dis) 142)
(is= (read-byte-unsigned dis) types/BYTE_UNSIGNED_MIN_VALUE)
(is= (read-byte-unsigned dis) types/BYTE_UNSIGNED_MAX_VALUE)
(is= (read-short dis) 9999)
(is= (read-short dis) -9999)
(is= (read-short dis) Short/MIN_VALUE)
(is= (read-short dis) Short/MAX_VALUE)
(is= (read-short-unsigned dis) 55999)
(is= (read-short-unsigned dis) types/SHORT_UNSIGNED_MIN_VALUE)
(is= (read-short-unsigned dis) types/SHORT_UNSIGNED_MAX_VALUE)
(is= (read-integer dis) int-val)
(is= (read-integer dis) Integer/MIN_VALUE)
(is= (read-integer dis) Integer/MAX_VALUE)
(is= (read-integer-unsigned dis) int-val)
(is= (read-integer-unsigned dis) types/INTEGER_UNSIGNED_MIN_VALUE)
(is= (read-integer-unsigned dis) types/INTEGER_UNSIGNED_MAX_VALUE)
(is= (read-long dis) long-val)
(is= (read-long dis) Long/MIN_VALUE)
(is= (read-long dis) Long/MAX_VALUE)
(is= (read-long-unsigned dis) long-val)
(is= (read-long-unsigned dis) types/LONG_UNSIGNED_MIN_VALUE)
(is= (read-long-unsigned dis) types/LONG_UNSIGNED_MAX_VALUE)
(is= (read-float dis) pi-float)
(is= (read-float dis) Float/MIN_VALUE)
(is= (read-float dis) Float/MAX_VALUE)
(is= (read-double dis) pi-double)
(is= (read-double dis) Double/MIN_VALUE)
(is= (read-double dis) Double/MAX_VALUE)) )
Other tests show that an Exception is thrown if the supplied value is out of range:
;-----------------------------------------------------------------------------
(throws? (write-byte dos (inc Byte/MAX_VALUE)))
(throws? (write-byte dos (dec Byte/MIN_VALUE)))
(throws? (write-short dos (inc Short/MAX_VALUE)))
(throws? (write-short dos (dec Short/MIN_VALUE)))
(throws? (write-integer dos (inc Integer/MAX_VALUE)))
(throws? (write-integer dos (dec Integer/MIN_VALUE)))
(throws? (write-long dos (* 2M (bigdec Long/MAX_VALUE))))
(throws? (write-long dos (* -2M (bigdec Long/MIN_VALUE))))
(throws? (write-byte-unsigned dos (inc types/BYTE_UNSIGNED_MAX_VALUE)))
(throws? (write-byte-unsigned dos (dec types/BYTE_UNSIGNED_MIN_VALUE)))
(throws? (write-short-unsigned dos (inc types/SHORT_UNSIGNED_MAX_VALUE)))
(throws? (write-short-unsigned dos (dec types/SHORT_UNSIGNED_MIN_VALUE)))
(throws? (write-integer-unsigned dos (inc types/INTEGER_UNSIGNED_MAX_VALUE)))
(throws? (write-integer-unsigned dos (dec types/INTEGER_UNSIGNED_MIN_VALUE)))
(throws? (write-long-unsigned dos (inc types/LONG_UNSIGNED_MAX_VALUE)))
(throws? (write-long-unsigned dos (dec types/LONG_UNSIGNED_MIN_VALUE)))
Other useful functions include the ability to make temporary disk files & directories, and to recursively delete a dir:
(dotest
; Create nested dirs & files, then delete recursively
(let [tmp-path (create-temp-directory "some-stuff")
tmp-name (str tmp-path)
dir-one (create-temp-directory tmp-path "dir-one")
dir-two (create-temp-directory dir-one "dir-two")
tmp-one-a (create-temp-file dir-one "aaa" nil)
tmp-one-b (create-temp-file dir-one "bbb" ".dummy")
tmp-two-a (create-temp-file dir-two "aaa" ".tmp")
tmp-two-b (create-temp-file dir-two "bbb" ".tmp")
count-files-fn (s/fn [dir-name :- s/Str]
(let [dir-file (io/file dir-name)
counts (for [file (file-seq dir-file)]
(if (.exists file)
1
0))
total (apply + counts)]
total))]
(when false ; debug printouts
(spyx (str tmp-path))
(spyx (str tmp-name))
(spyx (str dir-one))
(spyx (str tmp-one-a))
(spyx (str tmp-one-b))
(spyx (str tmp-two-a))
(spyx (str tmp-two-b))
(pprint/pprint (vec (sort
(map str
(file-seq (.toFile tmp-path))))))
; Sample debug output:
; (str tmp-path) => "/tmp/some-stuff-562114607264734833"
; (str tmp-name) => "/tmp/some-stuff-562114607264734833"
; (str dir-one) => "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970"
; (str tmp-one-a) => "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/aaa-345552808102662294.tmp"
; (str tmp-one-b) => "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/bbb-14875687905791190659.dummy"
; (str tmp-two-a) => "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/dir-two-15168249174986120265/aaa-14487327636081006800.tmp"
; (str tmp-two-b) => "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/dir-two-15168249174986120265/bbb-8158500208162744706.tmp"
;
; File names produced:
; ["/tmp/some-stuff-562114607264734833"
; "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970"
; "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/aaa-345552808102662294.tmp"
; "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/bbb-14875687905791190659.dummy"
; "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/dir-two-15168249174986120265"
; "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/dir-two-15168249174986120265/aaa-14487327636081006800.tmp"
; "/tmp/some-stuff-562114607264734833/dir-one-15514198984069907970/dir-two-15168249174986120265/bbb-8158500208162744706.tmp"]
)
(is= 7 (count-files-fn tmp-name))
(is= 7 (delete-directory tmp-name))
(is= 0 (count-files-fn tmp-name))
(is= 0 (delete-directory tmp-name)) ; idempotent
))
There are also a few stream type-testing predicates:
(let [in-stream (io/input-stream dummy-file)
out-stream (io/output-stream dummy-file)
dis (DataInputStream. in-stream)
dos (DataOutputStream. out-stream)]
(isnt (data-input-stream? in-stream))
(is (input-stream? in-stream))
(is (input-stream? dis))
(is (data-input-stream? dis))
(isnt (data-output-stream? out-stream))
(is (output-stream? out-stream))
(is (output-stream? dos))
(is (data-output-stream? dos))))
Enjoy!
Alan Thompson
Reply all
Reply to author
Forward
0 new messages