Re: J vs Kdb+/Q
Jack Andrews
---------- Forwarded message ----------
From: Stevan Apter <s...@nsl.com>
Date: Fri, Sep 19, 2008 at 10:56 PM
Subject: Fw: [personal kdb+] Re: J vs Kdb+/Q
To: eff...@gmail.com
hi jack. i'm having difficulty posting to the personal-kdb list, so
if you wouldn't mind posting this for me ...
kdb+/q has two components: k4, the underlying symbolic language, and
q, a simple syntactic overlay on k4. q is exactly as powerful as k4.
the overlay is simple: k4 has +x (transpose) and x+y (addition), q
has flip x and x+y. i.e. no ambivalence in q.
k (i.e. k3) has atoms, lists, and dictionaries, q has atoms, lists,
dictionaries, tables, and keytables. for the purist: q tables are
lists of dictionaries having the same domain, and keytables are
dictionaries where the domain and range are tables with the same
cardinality.
this simple extension, viz. having the interpreter recognize table
and keytable as derived types, has powerful implications. since +
understands atoms and lists, it also understands tables (since a table
is a list) and dictionaries, and hence keytables. so the set of
permissible operations in q is vastly larger than in k.
since tables and keytables are types, the sql-like language of ksql
(select, exec, update, delete, upsert, &c.) is a sublanguage of q
rather than the reverse, as sql programmers might expect. in other
words, the data language is a fragment of the programming language.
in k we use a dictionary of equal-length lists to simulate a table.
in q, we revert to the more intuitive idea that a table is a list
of records, where a record is a dictionary of atoms. as a side-
benefit, the indexing of a q table is what we would expect: table[rows;fields].
in k, every global variable is a triple: a symbol (name), a value,
and an attribute dictionary. since a dictionary is a whole whose
parts are variables, global variables are "right recursive" (i.e.
variables in an attribute dictionary can have attributes.) this
is powerful, but often unwieldy. q takes a simpler approach: no
attributes.
q's dependency and namespace model is simpler and more limited
than k's, but not intrinsically so. i expect that arthur will
eventually flesh these out.
q's connection to the outside world is much richer and more highly
developed than k's. this is important for people using q to develop
real-world applications.
and of couse k has a built-in mil-spec gui, whereas q does not.
if you know k, you can't do better than consult jeff borror's
"q for mortals."
i came to q after 15 years programming in k (k1,k2,k3). whenever
i think i've got arthur figured he manages to surprise me. i'd
call it a relentless drive to simplify.
----- Original Message ----- From: "Jack Andrews" <eff...@gmail.com>
To: <personal...@googlegroups.com>
Sent: Friday, September 19, 2008 8:43 AM
Subject: [personal kdb+] Re: J vs Kdb+/Q
>
> i'm familiar with, but not an experienced user of kdb, k and j. q is
> designed to look a little like sql (select ...) but has quite a rich
> set of primitives. q is a cut-down k -- i don't think the rank
> operator comes up in q (but it does in k). there's a page in wikipedia
> for k (search for k programming language) that is a great overview. k
> includes hash maps (or dictionaries) explicitly. i think k is great
> and arthur is an amazing inventor.
>
> did you notice the announcement of JDB on the j mailing lists?
>
> ta, jack
>
> >
Jack Andrews
> (select, exec, update, delete, upsert, &c.) is a sublanguage of q
and i thought that q /was/ the sql-like language of kdb+. i didn't
realize that q was not ksql.
> and of couse k has a built-in mil-spec gui, whereas q does not.
i thought the gui was ripped out of k... what is a mil-spec gui?
ta, jack
Aaron Davies
information in stevan's message was somewhat hidden and easy to miss:
> k (i.e. k3)
all his "k" remarks from there on applied to k3, the obsolete
predecessor language to k4 and q.
specific to your questions:
q is everything defined in q.k--english names for monadic functions
and various utility functions for db operations (and other things).
"qsql" is sometimes used as the name for the four-arg "sql style"
command construct for dealing directly with tables
"(select|exec|update|delete) by from where". there's no such thing as
ksql anymore (thank god!). the boundaries between "kdb+" and "q" are
kind of fuzzy, at least imho. i suppose q is the language, and kdb+ is
the system, but there's really not much point to distinguishing them.
--
Aaron Davies
aaron....@gmail.com
Aaron Davies
http://en.wikipedia.org/wiki/Mil-spec
not sure what sa meant by it
On Sat, Sep 20, 2008 at 9:22 PM, Aaron Davies <aaron....@gmail.com> wrote:
> specific to your questions:
> On Sat, Sep 20, 2008 at 9:15 PM, Jack Andrews <eff...@gmail.com> wrote:
>>> and of couse k has a built-in mil-spec gui, whereas q does not.
>>
>> i thought the gui was ripped out of k... what is a mil-spec gui?
>>
>> ta, jack
>>
>> >>
>>
>
>
>
> --
> Aaron Davies
> aaron....@gmail.com
>
--
Aaron Davies
aaron....@gmail.com
Felix Lungu
jph
> q, a simple syntactic overlay on k4. q is exactly as powerful as k4.
> the overlay is simple: k4 has +x (transpose) and x+y (addition), q
> has flip x and x+y. i.e. no ambivalence in q.
Thanks for all this great information - it's really helped me
understand what Q is. Could you provide some similar description of
what KDB+ is? How about kdb+tick?
Aaron Davies
> understand what Q is. Could you provide some similar description of
> what KDB+ is? How about kdb+tick?
as i said earlier, exactly what kdb+ is is somewhat debatable (at least imho)
i guess you could say it's the database system implemented by the q
interpreter and managed using the q language?
kdb+tick is a set of scripts written in q (as of last week; previously
written in k4) which implement a basic real-time market data
system--ticks go from a feed (reuters, bloomberg, etc.) to a
feedhandler, then to a tickerplant which batches them up and publishes
them to any clients (real-time databases, statistics generators, etc.)
which have subscribed. the real-time database then add its tables to
the historical database at the end of the day.
--
Aaron Davies
aaron....@gmail.com
sa
contributed
to the mystery.
k1 k2 k3 k4 ...
as someone said, most of the examples on the net (e.g. my site
nsl.com) is k3.
k3 is obsolete. [there are a few k4 examples on nsl -- e.g. the ray-
tracer.]
the current k is k4 (ambivalence)/q (no ambivalence). as someone
said, q is
defined by the k4 script q.k. you can define your own syntactic
overlay by
redefining q.k. i wonder if anyone's done this?
the confusion may be due to the fact that people sometimes use "k" to
refer to the *whole research program* from k1 to k4 (and beyond), and
sometimes to refer to the current product offering from kx. i think i
prefer
the former usage, unless context makes it clear that some other
meaning
is intended. (arthur used to say that every language he wrote was
called
"a".)
kdb+ is (i guess) the official name of the base product offered by
kx. at least,
that's how i use the term, which i rarely do. someone from kx can
scold me
if i'm still adding to the confusion.
my interest (and i assume it's shared by most people here) is in the
language.
did someone say that k3 contains a rank operator (and that k4/q
doesn't?)
that's certainly mistaken. A+ (the precursor to k) had one, but none
of the k's
ever did. the technical reason is that APLs after APL\360 had nested
arrays, i.e. arrays of (enclosed) arrays, while k has lists.
in k, there's no difference between
matrix:(1 2 3;4 5 6)
listofvectors:(1 2 3;4 5 6)
a function can applied to the sublists of m (f'm) or to the cells of m
(f''m), but not
the "columns" of m except by flipping the matrix. depth is a weak
form of rank.
of course, the rank *operator* may make a return appearance someday.
Slobodan Blazeski
programming language, q code IS beautiful.
I like j, and it has a great community,many cool ideas and a
collection of excellent books and articles, (j for c programmers is my
favourite) but I really don't like a "programs that look like a three-
year-old with a particular fondness for periods and colons was set
before the keyboard."
bobi