Encoding/decoding binary data
Nicolas Neuss
what do people here use for encoding/decoding binary data as text (for
storing it safely in a database)?
Nicolas
RG
Nicolas Neuss <last...@kit.edu> wrote:
Is there a reason you don't want to use BLObs?
rg
Thomas A. Russ
> what do people here use for encoding/decoding binary data as text (for
> storing it safely in a database)?
I would think that one would use a BLOB or the equivalent for storing
binary data directly. That way you avoid the overhead of having to
encode and decode it.
--
Thomas A. Russ, USC/Information Sciences Institute
Nicolas Neuss
Mainly that I don't know how to achieve it using CLSQL/Postgresql. Does
anyone know if that is supported?
[The kludge I use at the moment is encoding/decoding with Hunchentoot's
url-encode/url-decode, but this is quite wasteful.]
Nicolas
RG
Nicolas Neuss <last...@kit.edu> wrote:
> RG <rNOS...@flownet.com> writes:
>
> > In article <8739ru9...@ma-patru.mathematik.uni-karlsruhe.de>,
> > Nicolas Neuss <last...@kit.edu> wrote:
> >
> >> Hello,
> >>
> >> what do people here use for encoding/decoding binary data as text (for
> >> storing it safely in a database)?
> >>
> >> Nicolas
> >
> > Is there a reason you don't want to use BLObs?
>
> Mainly that I don't know how to achieve it using CLSQL/Postgresql. Does
> anyone know if that is supported?
Can't help you there I'm afraid.
> [The kludge I use at the moment is encoding/decoding with Hunchentoot's
> url-encode/url-decode, but this is quite wasteful.]
Youch! You should probably be using b64encode.
rg
Petter Gustad
> Mainly that I don't know how to achieve it using CLSQL/Postgresql. Does
> anyone know if that is supported?
I don't think it's supported. Postmodern might support blobs. Elephant
can store most Lisp objects, at least with the Berkeley DB back-end.
Petter
--
.sig removed by request.
Nicolas Neuss
>> [The kludge I use at the moment is encoding/decoding with Hunchentoot's
>> url-encode/url-decode, but this is quite wasteful.]
>
> Youch! You should probably be using b64encode.
Yes, that would be better. Is this already in some CL library?
Nicolas
Nicolas Neuss
Rob Warnock
+---------------
| Nicolas Neuss <last...@kit.edu> writes:
| > Mainly that I don't know how to achieve it using CLSQL/Postgresql.
| > Does anyone know if that is supported?
|
| I don't think it's supported. Postmodern might support blobs. Elephant
| can store most Lisp objects, at least with the Berkeley DB back-end.
Eric Marsden's "pg.lisp" <http://www.cliki.net/Pg> supports
PostgreSQL blobs [or used to, I dunno 'bout the current version].
See the source for documentation & unit test code.
-Rob
p.s. The above CLiki page deprecates "pg" in favor of Postmodern,
but I couldn't find any mention of large object (blob) support in
the online docs for Postmodern.
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Raymond Toy
> RG <rNOS...@flownet.com> writes:
>
>> In article <8739ru9...@ma-patru.mathematik.uni-karlsruhe.de>,
>> Nicolas Neuss <last...@kit.edu> wrote:
>>
>>> Hello,
>>>
>>> what do people here use for encoding/decoding binary data as text (for
>>> storing it safely in a database)?
>>>
>>> Nicolas
>>
>> Is there a reason you don't want to use BLObs?
>
> Mainly that I don't know how to achieve it using CLSQL/Postgresql. Does
> anyone know if that is supported?
Don't know about CLSQL, but PostgreSQL[1] definitely supported BLObs. I
used to store whole pdf files and gzipped ps and html files in a database.
Ray
[1] That was version 6.1 or so. I assume it still supports BLObs.
Robert Maas, http://tinyurl.com/uh3t
> what do people here use for encoding/decoding binary data as text
ConVow notation (Consonants alternating Vowel, always ending with
vowel, hence starting with consonent if even number of digits or
vowel if odd number of digits), which is a representation of base
20 of a big integer. For example 8*9 = 72 = 3*20 + 12 so the digits
are [3,12], so 3 is converted to a consonant while 12 is converted
to a vowel, per these tables:
(defparameter g*r20-consonants
'("b" "c" "d" "f" "g"
"h" "j" "k" "l" "m"
"n" "p" "r" "s" "t"
"v" "w" "z" "sh" "ch"))
(defparameter g*r20-vowels
'("a" "e" "i" "o" "u"
"aa" "ai" "au" "ea" "ee"
"ei" "eu" "ie" "oa" "oi"
"oo" "ui" "ay" "ey" "oy"))
Thus the preliminary result is "fie". If that is more than 5
characters, it's randomly brokwn into "words" by inserting spaces.
Finally, the first letter of the pseudo-sentence is capitalized and
a period (full stop) is appended at the end. Multiple sentences can
be appended to make paragraphs, and single sentences can be broken
at word boundaries to avoid too-long lines, and this doesn't create
a problem when decoding because the capital start and final period
clearly mark the sentences. I use this for my public-key
cryptosystem, where each code-block (big integer which is exponent
modulo product of two primes) is converted to one sentence, and all
the blocks in a single message are appended to a sentence. This
also makes it possible to use Twitter, which is limited to 140
characters per message, to build long sentences by multiple
consecutive tweets, where the first tweet in a group starts with a
capital letter and the corresponding last tweet in a group ends
with period and any middle tweets in that group have neither
capital nor period.
> (for storing it safely in a database)?
If you have a table set up to support long-enough strings to hold
your largest possible ConVow(bigInteger) you'll ever use, it would
work directly one sentece per record. If your tables support only
255 characters per string field but your ConVow sentences are
longer, but you have a way of identifying several
related-in-sequence records, such as your table having a group-ID
plus a part-sequence-number, then you can use the same break-up
approach as I use on Twitter (except you have 255 instead of 140 as
your maximum-piece-length).