simple, terse reference to nested hash values in Common Lisp

[ccc31807]

In Perl, if you have a nested hash, you can get a particular value by doing this:
$nested_hash{$key1}{$key2} gets the value.

In Python, you can do nestedHash[key1][key2] gets the appropriate value.

Is there a simple, terse way of getting values for nested hash elements in Common Lisp? Or is one stuck with (gethash (gethash key2 nested-hash) nested-hash)?

Thanks, CC.

[XeCycle]

Sounds easy to implement one yourself. Just don't forget the
setf expander.

--
Carl Lei (XeCycle)
Department of Physics, Shanghai Jiao Tong University
OpenPGP public key: 7795E591
Fingerprint: 1FB6 7F1F D45D F681 C845 27F7 8D71 8EC4 7795 E591

[WJ]

newLISP:

(set 'nested-alist
'((foo (name "Rob") (sex "male"))
(bar (name "Chad") (sex "male"))))

> (assoc '(foo sex) nested-alist)
(sex "male")

[Pascal J. Bourguignon]

http://groups.google.com/group/comp.lang.lisp/msg/190d771c0e033a1c



--
__Pascal Bourguignon__ http://www.informatimago.com/
A bad day in () is better than a good day in {}.

[WJ]

Pascal J. Bourguignon wrote:

> ccc31807 <cart...@gmail.com> writes:
>
> > In Perl, if you have a nested hash, you can get a particular value by doing this:
> > $nested_hash{$key1}{$key2} gets the value.
> >
> > In Python, you can do nestedHash[key1][key2] gets the appropriate value.
> >
> > Is there a simple, terse way of getting values for nested hash elements in Common Lisp? Or is one stuck with (gethash (gethash key2 nested-hash) nested-hash)?
>
> http://groups.google.com/group/comp.lang.lisp/msg/190d771c0e033a1c

So CL (COBOL-Like) is similar to assembly language.
It doesn't have anything built into it, but you can
invest large chunks of your life into trying to add
the commands that you need.

[amh...@gmail.com]

You don't have to invest large chunks of your life into adding the commands that you need. You need to invest small ones. In Lisp you can develop pretty fast, that's why I and many others love the languages (all the Lisps). For example:


(defun nested-hash (hash-table &rest keys)
(labels ((f (hash-table keys)
(if (and (hash-table-p hash-table) (car keys))
(f (gethash (car keys) hash-table) (cdr keys))
hash-table)))
(f hash-table keys)))

(defparameter *hash* (make-hash-table :test 'equal))
(setf (gethash "feliz" *hash*) (make-hash-table :test 'equal))
(setf (gethash "navidad" (gethash "feliz" *hash*)) "merry christmas")
(nested-hash *hash* "feliz" "navidad")

I build this function in like 10 minutes and I got a pretty good solution. You just need to call (nested-hash *the-nested-hash* "key1" "key2" "key3" "etc"). If you're worried about writing too much, just rename it to NH. If you want to be really Pythonic or Perlish, use reader macros to build a small Python or Perl in Common Lisp that reads nested hash the way you want.

[Barry Margolin]

In article <a139a151-c77a-4676...@googlegroups.com>,

Lisp doesn't have terse notation for any types of data access, why would
you expect this to be any different? I suppose the exception would be
the abbreviations like CADAR.

If GETHASH didn't have the <default> argument, we could have defined it
to be (gethash table &rest keys), where multiple keys mean to drill down
recursively. But recursive hash tables were not a common use pattern at
the time we defined things. Perl didn't have them, either, until
references were added in later versions.

Basically, at the time CL was being designed, hash tables tended to be
used only for "big" data structures. Little aggregates were always done
with structures or objects. I don't think it was until Perl caught on
that it became popular to use hashes as the universal container for
non-uniform data (PHP took this further, unifying arrays and hashes).

I think processor speeds are a major reason for the change. In the 80's,
if you had to calculate a hash function for almost every data access,
we'd still be waiting for some of those programs to finish.

--
Barry Margolin, bar...@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[ccc31807]

On Wednesday, December 26, 2012 7:22:49 PM UTC-5, Barry Margolin wrote:
> Lisp doesn't have terse notation for any types of data access, why would
> you expect this to be any different? I suppose the exception would be
> the abbreviations like CADAR.
>
> If GETHASH didn't have the <default> argument, we could have defined it
> to be (gethash table &rest keys), where multiple keys mean to drill down
> recursively. But recursive hash tables were not a common use pattern at
> the time we defined things. Perl didn't have them, either, until
> references were added in later versions.>
>
> Basically, at the time CL was being designed, hash tables tended to be
> used only for "big" data structures. Little aggregates were always done
> with structures or objects. I don't think it was until Perl caught on
> that it became popular to use hashes as the universal container for
> non-uniform data (PHP took this further, unifying arrays and hashes).>
>
> I think processor speeds are a major reason for the change. In the 80's,
> if you had to calculate a hash function for almost every data access,
> we'd still be waiting for some of those programs to finish.

Thank you for your answer.

I use Perl as my everyday language for data wrangling. With Perl, the syntax for nested hashes and hash refs can look as forbidding as ancient Sumerian cuneiform, but once you figure it out it becomes a very nice way to get the work done.

I'm in the process of reading 'Collective Intelligence' by Toby Segaran and am attempting to translate his Python code to Common Lisp. I have found much of Perl to be Lisp-like, and am finding much of Python to be Perl-like especially with regard to manipulation of the values of nested data structures and list processing.

I had never thought about this before, but the absence of easy syntax to get at the values of nested data structures is really crippling, so much so that I'm beginning to think that Common Lisp isn't a good candidate for a general purpose language.

Obviously, this isn't the kind of problem Lisp evolved to solve, and my question would be, why not? A good design for nested data structures can't be any harder than CLOS and probably would be a lot easier -- after all, Lisp handles nested lists natively.

I'm really sort of at a loss here, I don't know what to say. I can see using nested structures as a hack for nested hashes, but I dunno.

Thoughts?

CC.

[Pascal Costanza]

Just implement it. It's not hard.


Pascal

--
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
The views expressed are my own, and not those of my employer.

[Barry Margolin]

In article <a14ee8cb-1f1c-4fd1...@googlegroups.com>,

ccc31807 <cart...@gmail.com> wrote:

> I had never thought about this before, but the absence of easy syntax to get
> at the values of nested data structures is really crippling, so much so that
> I'm beginning to think that Common Lisp isn't a good candidate for a general
> purpose language.

You get used to the verbosity after a while, it's not that big a deal.
A good editor with symbol completion helps.

Just be glad you're not programming in COBOL (although if you use LOOP,
it can feel like it).

[Paul Rubin]

Barry Margolin <bar...@alum.mit.edu> writes:
> Basically, at the time CL was being designed, hash tables tended to be

> used only for "big" data structures... I don't think it was until

> Perl caught on that it became popular to use hashes as the universal
> container for non-uniform data (PHP took this further, unifying arrays
> and hashes).

Awk used hashes for everything, iirc. Lisp tended to use association
lists or plists. One difference that in Lisp you can often use interned
symbols as keys, so you can compare them with eq. If you're using
strings as keys, string comparisons are probably about as expensive as
hashing.

[Pascal J. Bourguignon]

ccc31807 <cart...@gmail.com> writes:

> I had never thought about this before, but the absence of easy syntax
> to get at the values of nested data structures is really crippling, so
> much so that I'm beginning to think that Common Lisp isn't a good
> candidate for a general purpose language.

Are you dumb?

How crippling is writing:

(-> hash-of-hash :key1 :key2)

???




(let ((hash-of-hash (make-hash-table)))
(setf (-> hash-of-hash :key1) (make-hash-table))
(setf (-> hash-of-hash :key3) (make-array 33 :initial-element 33))
(setf (-> hash-of-hash :key1 :key2) 42)
(list (-> hash-of-hash :key1 :key2)
(-> hash-of-hash :key3 3)))
--> (42 33)

> I'm really sort of at a loss here, I don't know what to say.

The shut up, learn the language, write programs in lisp!

[Daniel Rupis]

Another solution with loop:

(defun nested-hash(hash-table &rest keys)
(loop for k in keys
for h = (gethash k hash-table) then (gethash k h)
finally (return h)))

[budden]

Hi CC!

CL is a bad _language_ with nasty names and nasty syntax. But CL is an excellent _platform_ with its threads, native code compilers, static typing, fast precise GC and hot code swapping. Perl, Python, Ruby, PHP are "toy" platforms while CL is "serious" platform. As a platform, Lisp is, at least, comparable with Java/.NET.

I believe one should not waste a time defining such things for himself. At best, this would lead to yet another single-user programming language.

For now, I think the solution is some other "good" language on CL platform, which should be designed so that all strong points of CL are available to the end user. CL would be used as an intermediate language.

E.g. I think of modifying cl-javascript to be more straightforward compiler, so that it would be just the javascript-like syntax sugar for entire CL.

[Daniel Rupis]

Another shorter solution without loop:

(defun nested-hash (h &rest ks)
(dolist (k ks h)
(setq h (gethash k h))))

[ccc31807]

On Dec 27, 6:24 am, "Pascal J. Bourguignon" <p...@informatimago.com>
wrote:
> Are you dumb?

Maybe. ;-)

> How crippling is writing:
>
>    (-> hash-of-hash :key1 :key2)
>
> ???
>
> (let ((hash-of-hash (make-hash-table)))
>   (setf (-> hash-of-hash :key1) (make-hash-table))
>   (setf (-> hash-of-hash :key3) (make-array 33 :initial-element 33))
>   (setf (-> hash-of-hash :key1 :key2) 42)
>   (list  (-> hash-of-hash :key1 :key2)
>          (-> hash-of-hash :key3 3)))
> --> (42 33)

Okay, let's play this game. Suppose you have a data file that has
thousands of records and dozens of columns, but for the sake of
brevity, let's say your data file looks like this:

1,George,Washington,Virginia,1788,2 terms
2,John,Adams,Massachusetts,1796,1 term
3,Thomas,Jefferson,Virginia,1800,2 terms
...
44,Barack,Obama,Illinois,2008,2 terms

With Perl, you would do something like this to create the hash:
my %nested_hash_of_presidents;
while (<IN>)
{
my ($key,$first,$last,$home,$year,$terms) = split $_;
$nested_hash_of_presidents{$key} = {
first => $first,
last => $last,
home => $home,
year => $year,
terms => $terms,
};
}

To write out the data, you would do something like this
(ABBREVIATE nested_hash_of_presidents as pres)
foreach my $key (keys %pres)
{
print OUT qq($key,$pres{$key}{first},...,$pres{$key}{terms}\n);
}

What I'd expect Common Lisp to have is something like this:

(defparameter *nested-hash-of-presidents*)

(do ((line (read-line STREAM)))
(('eol nil))
(multiple-value-bind (key first last home year terms) (parse-csv
line))
(setf *nested-hash-of-presidents* key
(:first first :last last :home home :year year :terms terms))))

Yeah, I know my syntax is off -- please consider this as some kind of
lispy pseudo-code.

And yes, I've decided to roll my own, with grateful thanks to some
others who have posted suggestions. This will be my first real macro,
and I'm sure that I'll have many false starts before I get it right.

But ... I'm still surprised that something like this isn't already
available. This is a common task for me and I suspect for many others,
and I'm baffled at the prospect of writing my own in 2012, fifty plus
years after the birth of Lisp.

CC.

[ccc31807]

On Dec 27, 3:51 am, Paul Rubin <no.em...@nospam.invalid> wrote:
> Awk used hashes for everything, iirc.  Lisp tended to use association
> lists or plists.  One difference that in Lisp you can often use interned
> symbols as keys, so you can compare them with eq.  If you're using
> strings as keys, string comparisons are probably about as expensive as
> hashing.

One value of the hash is that you have key/value pairs.

A second value is that both the insertion and the lookup of a
particular item is very fast.

I'm not really comparing strings, just the hashed values of the
strings. Not sure how this works under the covers, but when I think of
comparing strings I think of regular expressions, and when I think of
hash manipulation I think of hashed values in a particular memory
location.

CC.

[Barry Margolin]

In article
<2145281a-66f9-479c...@d4g2000vbw.googlegroups.com>,

Paul was talking about the way Lisp programs used to do this before hash
tables, which was association lists. These required comparisons, not
hashing.

[Barry Margolin]

In article
<dfa79f7e-37a5-4332...@n9g2000vbv.googlegroups.com>,

The traditional Lispy way would be to use DEFSTRUCT, not hash tables,
for the presidents. And it does have a terse notation very similar to
Perl, PHP, Javascript, etc.

`#S(PRESIDENT :FIRST ,first
:LAST ,last
:HOME ,home
:YEAR ,year
:TERMS ,terms)

And before you embark on using CL hash tables similarly to the way you
would in Perl, I strongly suggest you do some benchmarks. Since this
type of use is not common in CL, implementors may not have optimized
them for lots of small tables.

[Daniel Rupis]

---------------------------------------------

An example:

Use quicklisp for loading libraries, cl-ppcre is a Perl Portable Compatible Regular Expression library for common lisp.

File presidents.csv contains the three lines of your example.

You can look for information with look-for, for example

CL-USER> (look-for :last "Washington" :home)
"Virginia"

Example code follows:
------------------------------------------------
(ql:quickload :cl-ppcre)
(use-package :cl-ppcre)

(defparameter *hash-of-presidents* (make-hash-table)
"contains information about the presidents")

(defun read-csv()
(setq *hash-of-presidents* (make-hash-table))
(with-open-file (f "presidents.csv" :direction :input)
(loop for line = (read-line f nil nil) while line do
(process line))))

(defun process(line)
(multiple-value-bind (key first last home year term)
(values-list (cl-ppcre:split #\, line))
(setf (gethash (parse-integer key) *hash-of-presidents*)
(list :first first :last last :home home :year year :term term))))

(defun look-for(known-field known-value question)
(maphash (lambda(k v)
(if (string= (second (member known-field v)) known-value)
(return-from look-for (second (member question v)))
"Unknown")) *hash-of-presidents*))


(defun show-hash(h)
(maphash (lambda(k v)(format t "~s=>~s~%" k v)) h))

[Pascal J. Bourguignon]

No, you're still surprised by your ignorance. Something like this is
already available. It is not a common task for most other lisp
programmers, and if you're not able to write it yourself, perhaps you
should consider another line of work.


cl-user> (defparameter *nested-hash-of-presidents*
(com.informatimago.common-lisp.cesarum.utility:hashtable
:test 'equal
:elements (mapcar (lambda (record)
(list (first record)
(com.informatimago.common-lisp.cesarum.utility:hashtable
:test 'equal
:elements (mapcar 'list '(:key :first :last :home :year :terms) record))))
(com.informatimago.common-lisp.csv.csv:load-records #P"/tmp/ccc.data"))))
*nested-hash-of-presidents*
cl-user> (-> *nested-hash-of-presidents* "1" :first)
"George"
cl-user>

[Pascal J. Bourguignon]

ccc31807 <cart...@gmail.com> writes:

> On Dec 27, 3:51 am, Paul Rubin <no.em...@nospam.invalid> wrote:
>> Awk used hashes for everything, iirc.  Lisp tended to use association
>> lists or plists.  One difference that in Lisp you can often use interned
>> symbols as keys, so you can compare them with eq.  If you're using
>> strings as keys, string comparisons are probably about as expensive as
>> hashing.
>
> One value of the hash is that you have key/value pairs.
>
> A second value is that both the insertion and the lookup of a
> particular item is very fast.
>
> I'm not really comparing strings, just the hashed values of the
> strings. Not sure how this works under the covers,

Yes, that's the problem…

> but when I think of comparing strings I think of regular expressions,
> and when I think of hash manipulation I think of hashed values in a
> particular memory location.

Oh my!

[Dimitri Fontaine]

ccc31807 <cart...@gmail.com> writes:
> Okay, let's play this game. Suppose you have a data file that has
> thousands of records and dozens of columns, but for the sake of
> brevity, let's say your data file looks like this:
>
> 1,George,Washington,Virginia,1788,2 terms
> 2,John,Adams,Massachusetts,1796,1 term
> 3,Thomas,Jefferson,Virginia,1800,2 terms
> ...
> 44,Barack,Obama,Illinois,2008,2 terms

I'm not too sure what the key is useful for as a hash key here, but
reading other comments I sure wanted to try something myself, with a
hash table holding structures in our case.

Here's what I quickly got:

;; defstruct et csv
(defparameter *test-file* #p"/Edited/Path/To/test-file.csv")

(defparameter *test-hash* (make-hash-table))

(defstruct (president
(:constructor make-president (first-name
last-name
home
year
terms)))
first-name last-name home year terms)

(defun process-csv-row (fields)
"Parse a single CSV line: buffer it locally"
(destructuring-bind (id &rest data) fields
(let ((pres (apply #'make-president data)))
(setf (gethash (parse-integer id) *test-hash*) pres))))

(with-open-file (input *test-file* :direction :input)
(cl-csv:read-csv input :row-fn #'process-csv-row :separator #\,))



CL-USER> (gethash 44 *test-hash*)
#S(PRESIDENT :FIRST-NAME "Barack" :LAST-NAME "Obama" :HOME "Illinois" :YEAR "2008" :TERMS "2 terms")
T
CL-USER> (president-first-name (gethash 44 *test-hash*))
"Barack"
CL-USER> (with-slots (first-name last-name) (gethash 44 *test-hash*)
(format nil "~a ~a" first-name last-name))
"Barack Obama"

You will notice that I added an explicit boa constructor for the
president structure so that I could use apply easily to build it from
the data read in the CSV file.

I've been using the QuickLisp available package "cl-csv" here, that
offers a quite easy way to use a function to process each csv row read,
as you can see, and that's where we tell appart the id from the rest of
the data that goes into the structure.

We could keep the id as a string and use (make-hash-table :test 'equal),
but I decided that you certainly want the id to be an fixnum if possible.

And finally, some quick usage example of accessing either the whole
structure or just some parts of it. Apparently it's quite easy to need
more power than available with a struct, and then you use a class.

Regards,
--
dim

[WJ]

Barry Margolin wrote:

> In article <a139a151-c77a-4676...@googlegroups.com>,
> ccc31807 <cart...@gmail.com> wrote:
>
> > In Perl, if you have a nested hash, you can get a particular value by doing
> > this:
> > $nested_hash{$key1}{$key2} gets the value.
> >
> > In Python, you can do nestedHash[key1][key2] gets the appropriate value.
> >
> > Is there a simple, terse way of getting values for nested hash elements in
> > Common Lisp? Or is one stuck with (gethash (gethash key2 nested-hash)
> > nested-hash)?
> >
> > Thanks, CC.
>
> Lisp doesn't have terse notation for any types of data access, why would
> you expect this to be any different? I suppose the exception would be
> the abbreviations like CADAR.
>
> If GETHASH didn't have the <default> argument, we could have defined it
> to be (gethash table &rest keys), where multiple keys mean to drill down
> recursively. But recursive hash tables were not a common use pattern at
> the time we defined things. Perl didn't have them, either, until
> references were added in later versions.

From the tCL docs:

dict get dictionaryValue ?key ...?

Given a dictionary value (first argument) and a key (second argument),
this will retrieve the value for that key. Where several keys are
supplied, the behaviour of the command shall be as if the result of
dict get $dictVal $key was passed as the first argument to dict get
with the remaining arguments as second (and possibly subsequent)
arguments. This facilitates lookups in nested dictionaries. For
example, the following two commands are equivalent:

dict get $dict foo bar spong
dict get [dict get [dict get $dict foo] bar] spong

[WJ]

Barry Margolin wrote:

> In article <a139a151-c77a-4676...@googlegroups.com>,
> ccc31807 <cart...@gmail.com> wrote:
>
> > In Perl, if you have a nested hash, you can get a particular value by doing
> > this:
> > $nested_hash{$key1}{$key2} gets the value.
> >
> > In Python, you can do nestedHash[key1][key2] gets the appropriate value.
> >
> > Is there a simple, terse way of getting values for nested hash elements in
> > Common Lisp? Or is one stuck with (gethash (gethash key2 nested-hash)
> > nested-hash)?
> >
> > Thanks, CC.
>
> Lisp doesn't have terse notation for any types of data access,

CL (COBOL-Like) is not Lisp.

Lisp (Clojure) does have terse notation for nested hash-table access.


;; Create hash-table.
(def table
{:find {:the {:moron "bonehead", :imbecile "CL worshipper"}}
:foo "bar" })

user=> (get-in table [:find :the :imbecile])
"CL worshipper"

[Mirko Vukovic]

On Thursday, October 3, 2013 7:35:57 AM UTC-4, WJ wrote:

stuff deleted

> >
> > Lisp doesn't have terse notation for any types of data access,
>
> CL (COBOL-Like) is not Lisp.
>
> Lisp (Clojure) does have terse notation for nested hash-table access.
>
>
> ;; Create hash-table.
> (def table
> {:find {:the {:moron "bonehead", :imbecile "CL worshipper"}}
> :foo "bar" })
>
> user=> (get-in table [:find :the :imbecile])
> "CL worshipper"

Your has table misses several entries:

:rude "WJ", :abusive "WJ", :unhelpful "WJ"

[WJ]

newLISP:

(set 'alist
'((find (the (moron "bonehead") (imbecile "CL worshipper")))
(foo "bar")))

(assoc '(find the imbecile) alist)
===>
(imbecile "CL worshipper")


> (ref 'moron alist)
(0 1 1 0)
> (alist 0 1 1 0)
moron
> (alist '(0 1 1 0))
moron

[WJ]

Barry Margolin wrote:

> In article <a139a151-c77a-4676...@googlegroups.com>,
> ccc31807 <cart...@gmail.com> wrote:
>
> > In Perl, if you have a nested hash, you can get a particular value by doing
> > this:
> > $nested_hash{$key1}{$key2} gets the value.
> >
> > In Python, you can do nestedHash[key1][key2] gets the appropriate value.
> >
> > Is there a simple, terse way of getting values for nested hash elements in
> > Common Lisp? Or is one stuck with (gethash (gethash key2 nested-hash)
> > nested-hash)?
> >
> > Thanks, CC.
>
> Lisp doesn't have terse notation for any types of data access, why would
> you expect this to be any different? I suppose the exception would be
> the abbreviations like CADAR.

McCarthy insisted that "Lisp" should not be used as a synonym for
CL (COBOL-Like), and yet you do it anyway. You have nothing but
contempt for McCarthy and for Lisp.

In Gauche Scheme, you can use ~ (the universal accessor):

(let ((inner (make-hash-table))
(outer (make-hash-table)))
(hash-table-put! outer 'foo inner)
(dotimes (i 5)
(hash-table-push! (~ outer 'foo) 'bar (* i 100)))
(print (~ outer 'foo 'bar))
(print (~ outer 'foo 'bar 2)))

===>
(400 300 200 100 0)
200

Even more terse:

(define-method object-apply ((h <hash-table>) (s1 <symbol>) (s2 <symbol>))
(~ h s1 s2))


(let ((inner (make-hash-table))
(outer (make-hash-table)))
(hash-table-put! outer 'foo inner)
(dotimes (i 5)
(hash-table-push! (~ outer 'foo) 'bar (* i 100)))
(outer 'foo 'bar))

===>
(400 300 200 100 0)

--
If Lisp is to live, then CL must die.