Generating a listing of all symbols (16K+) and labeling subsets

[Hans BKK]

Total noob here - and non-programmer to boot, as will become
immediately apparent from the code below - so please be gentle.

Before I start getting to know emacs as an end-users - which I'm
highly motivated to do, despite the amazingly steep learning curve to
do the most basic things - I plan to of course highly customize my
emacs environment to suit my needs, before starting the muscle-memory
training required to become efficient.

I want to experiment with "out of the box" newbie-friendly mods like
cua-mode, Starter Kit, Prelude, Evil maybe even Ergoemacs, but to
start with not so much in a hands-on manner, but systematically
investigating the packages they install, keybinding mods etc.

To that end I'd like to generate standardized-format text "symbols
reports" on the relevant values of all functions, variables, key
bindings etc, so that I can do A-B comparison via diff between stock
vanilla emacs vs after installing these bundles, or in fact any
packages in the future.

I wrote the following function "list-hh-symbols" toward this end,
based on the list-options function from the now-obsolete options.el.
Please anyone noobier than me - don't use this for anything other than
learning by reading, I suspect it's of laughable quality.

Obviously any feedback at all would be most welcome, but the ideal
would be for someone to point me to something that already does this
out of the box. Or write it for me of course 8-)

Failing that, I'd specifically like to ask about the logic of the
categorizing breakdown and for better taxonomy terms.

And most of all, for code in line with the below scheme that allows
for the "other" categories to be more precisely broken down,
specifically identifying those symbols which are constant-value
variables, keymaps and macros.

And any others I might have missed among the 16,000+ that are out there.

------------------------------------------
(defun list-hh-symbols ()
"Display a list of Emacs symbols - names only"
(interactive)
(message "Looking up all symbols...")
(with-output-to-temp-buffer "*List Symbols*"
(let (vars)
(mapatoms (function (lambda (sym)
(setq vars (cons sym vars)))))
(setq vars (sort vars 'string-lessp))
(while vars
(let ((sym (car vars)))
(cond
((fboundp sym) ; ALL functions
(cond
((commandp sym)
(cond
((subrp (symbol-function sym))
(princ "=============================\n")
(princ "command - built-in primitive:\t\t")
(prin1 sym)
(princ "\n\n"))
((byte-code-function-p (symbol-function sym))
(princ "====================\n")
(princ "command - byte-code:\t\t\t")
(prin1 sym)
(princ "\n\n"))
((functionp sym)
(princ "================\n")
(princ "command - elisp:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t
(princ "===============\n")
(princ "keymap command:\t\t\t\t")
(prin1 sym)
(princ "\n\n"))
)
)
(t ; (non-command) functions
(cond
((subrp (symbol-function sym))
(princ "==============================\n")
(princ "function - built-in primitive:\t\t")
(prin1 sym)
(princ "\n\n"))
((byte-code-function-p (symbol-function sym))
(princ "=====================\n")
(princ "function - byte-code:\t\t\t")
(prin1 sym)
(princ "\n\n"))
((functionp sym)
(princ "=================\n")
(princ "function - elisp:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t
(princ "================================\n")
(princ "other ~function (keymap, macro):\t")
(prin1 sym)
(princ "\n\n"))
)
)
)
)
((boundp sym) ; ALL variables
(cond
((custom-variable-p sym)
(princ "=====================\n")
(princ "user (custom) option:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t
(princ "=========================\n")
(princ "non-user (setq) variable:\t\t")
(prin1 sym)
(princ "\n\n"))
)
)
((facep sym)
(princ "=====\n")
(princ "\n")
(princ "face:\t\t\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t
(princ "====================\n")
(princ "other (misc) symbol:\t\t\t")
(prin1 sym)
(princ "\n\n"))
)
(setq vars (cdr vars))))
(message "Looking up all symbols...done")
(with-current-buffer "*List Symbols*"
(setq buffer-read-only t)))))

;;;;;;;;;;;;;;;;;;;;;;;

(provide 'hh-list-symbols)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; hh-list-symbols.el ends here
-------------------------------------------------------

and as a complete side note, which is better for navigating this
stuff, Helm or Icicles?

or has anyone got both of them working together? I assume from my
reading that they'd have so much overlap as to most likely interfere
with each other. . .

Thanks in advance.

[Thien-Thi Nguyen]

() han...@gmail.com
() Thu, 17 Apr 2014 22:09:35 -0400

I'd specifically like to ask about the logic of the
categorizing breakdown and for better taxonomy terms.

It is a category error to categorize Emacs Lisp symbols into
(single) buckets, unless you wish to defer understanding. :-D

OTOH, sometimes one is the right number to strive for; i see
three copies of your message, and i haven't even been drinking!

--
Thien-Thi Nguyen
GPG key: 4C807502
(if you're human and you know it)
read my lisp: (responsep (questions 'technical)
(not (via 'mailing-list)))
=> nil

[Thorsten Jolitz]

Thien-Thi Nguyen <t...@gnu.org> writes:

> () han...@gmail.com
> () Thu, 17 Apr 2014 22:09:35 -0400
>
> I'd specifically like to ask about the logic of the
> categorizing breakdown and for better taxonomy terms.
>
> It is a category error to categorize Emacs Lisp symbols into
> (single) buckets, unless you wish to defer understanding. :-D

I wrote something similar (but much simpler) recently for getting a list
of all mode commands with keybindings, and detecting keybinding
conflicts between two modes:

* Compare Keymaps

#+begin_src emacs-lisp
(defun omm-get-cmd-symbols-with-keys (rgxp mode &optional req)
"Return alist of (key . sym) pairs where sym matches RGXP.

Require REQ and load MODE in temp buffer before doing the real
work. Push the intermediary results of `mapatoms' to
`omm-tmp-storage'.

Usage example:
(pp (omm-get-cmd-symbols-with-keys
\"\\(^org-\\|^orgtbl-\\)\" 'org-mode 'org))"
(setq omm-tmp-storage nil)
(mapatoms
(lambda (--sym)
(eval
`(and (commandp --sym)
(string-match ,rgxp (symbol-name --sym))
(with-temp-buffer
(when ,req
(require (quote ,req)))
(funcall (quote ,mode))
(let ((cmd-key (substitute-command-keys
(concat
"\\[" (symbol-name --sym) "]"))))
(push
(cons
(if (string-match "^M-x " cmd-key)
nil cmd-key)
--sym)
omm-tmp-storage)))))))
(delq nil
(mapcar
(lambda (--pair) (if (car --pair) --pair nil))
(delq nil omm-tmp-storage))))
#+end_src

#+begin_src emacs-lisp
(defun omm-get-keybinding-conflicts (cmd-syms1 cmd-syms2)
"Return alist with common keys of CMD-SYMS1 and CMD-SYMS2.

The return-list consists of sublists of this form

(event definition-map1 definition-map2)

Usage example:

(pp (omm-get-keybinding-conflicts
(omm-get-cmd-symbols-with-keys \"^magit-\" 'magit-mode)
(omm-get-cmd-symbols-with-keys \"^dired-\" 'dired-mode)))"
(let ((keys2 (map 'car cmd-syms2))) ; FIXME with org-mode-map
(delq nil
(mapcar
(lambda (--pair)
(when (member (car --pair) keys2)
(list (car --pair)
(cdr --pair)
(cdr (assoc (car --pair) cmd-syms2)))))
cmd-syms1))))
#+end_src


--
cheers,
Thorsten

[Hans BKK]

On Friday, April 18, 2014 3:19:40 AM UTC-4, Thien-Thi Nguyen wrote:
> I'd specifically like to ask about the logic of the
> categorizing breakdown and for better taxonomy terms.
>
>
>
> It is a category error to categorize Emacs Lisp symbols into
>
> (single) buckets, unless you wish to defer understanding. :-D

I realize there is overlap, hence the duplication of tests for primitive vs lisp etc within multiple buckets. And within a given cond level, order is by (my judgement) of priority - commands first, ID as a functions before values, user options before setq ones, etc.

This logic flow is exactly what I was looking for feedback on, as well as how to identify more specifically those currently falling into the "t" or "other" buckets - currently over half the symbols are neither functions nor variables nor faces. Too bad apparently predicates don't exist to cover all possible symbol types.

Once I have the "skeleton" to my satisfaction, I plan to output the relevant docstrings as well, so when I'm looking at a package's "diff report" everything's in one place.

[Hans BKK]

On Friday, April 18, 2014 6:09:59 AM UTC-4, Thorsten Jolitz wrote:

> I wrote something similar (but much simpler) recently for getting a list
>
> of all mode commands with keybindings, and detecting keybinding
>
> conflicts between two modes:

That's very useful thanks, I'll try to adapt bits of that to ID keymaps!

Anyone have ideas on how to detect/ID macros?

Is it reasonable to ignore the possibility that either of those are primitives or byte-code?

Any additional symbol types likely to be of interest?

[Nicolas Richard]

Hans BKK <han...@gmail.com> writes:
> I realize there is overlap, hence the duplication of tests for
> primitive vs lisp etc within multiple buckets. And within a given cond
> level, order is by (my judgement) of priority - commands first, ID as
> a functions before values, user options before setq ones, etc.

Some symbols are just symbols. every time the lisp reader reads
something, symbols are interned.

Currently I have 76920 symbols in obarray.

If you want to have a report of useful symbols used in a package you can
do e.g. for smerge-mode:
M-x apropos RET ^smerge- RET

--
Nico.

[Hans BKK]

On Fri, Apr 18, 2014 at 11:27 AM, Nicolas Richard

> Some symbols are just symbols. every time the lisp reader reads
> something, symbols are interned.
>
> Currently I have 76920 symbols in obarray.

Aha. OK, so those "just symbols" can go to dev null.

Question remains - how to separate out and ID - in the absence of a
predicate - any that actually may be of interest remaining in my
current "other" - which I presume macros should be, and having got
code for keymaps already above.

Any other function types? Or out of those that are neither fboundp nor boundp?

> If you want to have a report of useful symbols used in a package you can
> do e.g. for smerge-mode:
> M-x apropos RET ^smerge- RET

Thanks for that, looks useful. But looks to only pick up those
starting with the package string? And I think apropos only displays a
limited subset, e.g. only Customized variables? and/or only those with
docstrings?

In this context I'm aiming more for an all-in-one standard "state
report" I can diff between any arbitrary emacs-config-A and
emacs-config-B, showing all changes, including to existing system
variables.

[Nicolas Richard]

Hans BKK <han...@gmail.com> writes:
> On Fri, Apr 18, 2014 at 11:27 AM, Nicolas Richard
>> Some symbols are just symbols. every time the lisp reader reads
>> something, symbols are interned.
>>
>> Currently I have 76920 symbols in obarray.
>
> Aha. OK, so those "just symbols" can go to dev null.

I don't know what that should mean. Some symbols will never be reused,
some I don't know if they will be reused, some are still used (e.g.
because they serve as identifier), and many probably serve a purpose I
am not aware of.

If you like numbers, here are some :
(yf/count-symbols)
=> 77064

(yf/count-symbols #'fboundp)
=> 27896

(yf/count-symbols #'boundp)
=> 13713

;; the intersecton of the two previous sets:
(yf/count-symbols (lambda (x) (and (fboundp x) (boundp x))))
=> 792

(yf/count-symbols
(lambda (x)
(and
(symbol-plist x)
(not (fboundp x))
(not (boundp x)))))
=> 3136

(yf/count-symbols #'facep)
=> 673
(there's obviously some overlapping with bound and fbound symbols)

(yf/count-symbols #'keywordp)
=> 1510

So, are all the other symbols unneeded ? Dunno.

FWIW, here's the yf/count-symbols that I used:
(defun yf/count-symbols (&optional predicate)
(let ((count 0))
(mapatoms
(lambda (x)
(when (or (not predicate)
(funcall predicate x))
(incf count))))
count))

> Question remains - how to separate out and ID - in the absence of a
> predicate - any that actually may be of interest remaining in my
> current "other" - which I presume macros should be, and having got
> code for keymaps already above.

macros are fboundp.
(defmacro asymbollikenoother () t)
(fboundp 'asymbollikenoother)
=> t

>
>> If you want to have a report of useful symbols used in a package you can
>> do e.g. for smerge-mode:
>> M-x apropos RET ^smerge- RET
>
> Thanks for that, looks useful. But looks to only pick up those
> starting with the package string?

That was the idea, yes. Most packages are namespaced that way, those
that aren't probably should not exist in an ideal world ;)

> And I think apropos only displays a
> limited subset, e.g. only Customized variables? and/or only those with
> docstrings?

C-h f a p r o p o s RET
=>
Show all meaningful Lisp symbols whose names match PATTERN.
Symbols are shown if they are defined as functions, variables, or
faces, or if they have nonempty property lists.

> In this context I'm aiming more for an all-in-one standard "state
> report" I can diff between any arbitrary emacs-config-A and
> emacs-config-B, showing all changes, including to existing system
> variables.

I'm not sure if you can get meaningful data, doing that.

--
Nico.

[Hans BKK]

On Fri, Apr 18, 2014 at 4:47 PM, Nicolas Richard wrote:
> Hans BKK writes:
>> On Fri, Apr 18, 2014 at 11:27 AM, Nicolas Richard
>>> Some symbols are just symbols. every time the lisp reader reads
>>> something, symbols are interned.
>>>
>>> Currently I have 76920 symbols in obarray.
>>
>> Aha. OK, so those "just symbols" can go to dev null.
>
> I don't know what that should mean.

I meant that after my code selects for display those symbols likely to
be of interest, I can leave the final "(t" case to simply not list
those likely to be irrelevant to my purpose.

> ;; the intersecton of the two previous sets:
> (yf/count-symbols (lambda (x) (and (fboundp x) (boundp x))))
> => 792

> (there's obviously some overlapping with bound and fbound symbols)

Yes, I'm testing for function first as I consider that primary.
Strange (to me) that such a high % of functions have no associated
value.

> FWIW, here's the yf/count-symbols that I used:
> (defun yf/count-symbols (&optional predicate)
> (let ((count 0))
> (mapatoms
> (lambda (x)
> (when (or (not predicate)
> (funcall predicate x))
> (incf count))))
> count))

cool - thanks

>> Question remains - how to separate out and ID - in the absence of a
>> predicate - any that actually may be of interest remaining in my
>> current "other" - which I presume macros should be, and having got
>> code for keymaps already above.
>
> macros are fboundp.
> (defmacro asymbollikenoother () t)
> (fboundp 'asymbollikenoother)

yes, but I'm looking for a way to ID them specifically within the
fbound cond rather than having them lumped in with the final (t
"other" grouping:

(cond
((fboundp sym) ; ALL functions
(cond
((commandp sym)
(cond

((keymapp sym)
(princ "=======\n")
(princ "\n")
(princ "command - keymap:\t\t\t\t\t")

(prin1 sym)
(princ "\n\n"))

((subrp (symbol-function sym))
(princ "=============================\n")
(princ "command - built-in primitive:\t\t")
(prin1 sym)
(princ "\n\n"))
((byte-code-function-p (symbol-function sym))
(princ "====================\n")
(princ "command - byte-code:\t\t\t")
(prin1 sym)
(princ "\n\n"))
((functionp sym)
(princ "================\n")
(princ "command - elisp:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t
(princ "===============\n")

(princ "other command:\t\t\t\t")

(prin1 sym)
(princ "\n\n"))
)
)
(t ; (non-command) functions
(cond

((keymapp sym)
(princ "==================\n")
(princ "\n")
(princ "function - keymap:\t\t\t\t")

(prin1 sym)
(princ "\n\n"))

((subrp (symbol-function sym))
(princ "==============================\n")
(princ "function - built-in primitive:\t\t")
(prin1 sym)
(princ "\n\n"))
((byte-code-function-p (symbol-function sym))
(princ "=====================\n")
(princ "function - byte-code:\t\t\t")
(prin1 sym)
(princ "\n\n"))
((functionp sym)
(princ "=================\n")
(princ "function - elisp:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t

(princ "=============================\n")
(princ "other ~function (macros +??):\t")

(prin1 sym)
(princ "\n\n"))
)

>> Thanks for that, looks useful. But looks to only pick up those
>> starting with the package string?
>
> That was the idea, yes. Most packages are namespaced that way, those
> that aren't probably should not exist in an ideal world ;)

I wish we lived in such a place 8-)

>> And I think apropos only displays a
>> limited subset, e.g. only Customized variables? and/or only those with
>> docstrings?
>
> C-h f a p r o p o s RET
> =>
> Show all meaningful Lisp symbols whose names match PATTERN.
> Symbols are shown if they are defined as functions, variables, or
> faces, or if they have nonempty property lists.

Note that in any given state,
(apropos-variable "." t)
shows much fewer results than the auto-completion listing from
M-x describe-variable

Since I'm only looking at the diff, the changes made by activating a
given feature or package, it doesn't matter if there's a lot of
less-meaningful kruft as long as it's kruft that doesn't change.

[John Mastro]

Hans BKK <han...@gmail.com> wrote:
> yes, but I'm looking for a way to ID them specifically within the
> fbound cond rather than having them lumped in with the final (t
> "other" grouping:

The currently-in-development Emacs 24.4 includes `macrop'[1]. I can't
vouch for what (if any) other versions it's appropriate for, but it did
succeed in a couple very basic experiments on 24.3.1.

[1]
http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/subr.el?h=emacs-24#n2664

--
John Mastro

[Hans BKK]

Excellent, thanks!

I'd rather not start playing with dev monolithically at this early stage of my knowledge, and also need a stable platform for doing my testing over a few months.

So should I try replacing that whole file or better to try just copying that function into my init.el?

[Hans BKK]

> > The currently-in-development Emacs 24.4 includes `macrop'[1]. I can't
> > vouch for what (if any) other versions it's appropriate for, but it did
> > succeed in a couple very basic experiments on 24.3.1.
> > [1]
> > http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/subr.el?h=emacs-24#n2664

OK, just dropped that snippet at the top of my listing.el file and macrop works a treat, thanks!


Here's a shortened listing of the cond-tree for each symbol type and my taxonomy if anyone cares to comment further on my terminology:

((fboundp sym) ; ALL functions
((commandp sym)
((keymapp sym)
(princ "command - keymap:\t\t\t\t\t")
((subrp (symbol-function sym))

(princ "command - built-in primitive:\t\t")

((byte-code-function-p (symbol-function sym))

(princ "command - byte-code:\t\t\t")

((functionp sym)

(princ "command - elisp:\t\t\t")

(t ; (non-command) functions
((macrop sym)
(princ "function - macro:\t\t\t")
((keymapp sym)
(princ "function - keymap:\t\t\t\t")
((subrp (symbol-function sym))

(princ "function - built-in primitive:\t\t")

((byte-code-function-p (symbol-function sym))

(princ "function - byte-code:\t\t\t")

((functionp sym)

(princ "function - elisp:\t\t\t")

(t
(princ "other ~function (aliases??):\t")))

)
((boundp sym) ; ALL variables

((custom-variable-p sym)

(princ "user (custom) option:\t\t\t")

((keywordp sym)
(princ "keyword:\t\t\t\t\t")
(t
(princ "non-user (setq) variable:\t\t"))
)
((facep sym)

(princ "face:\t\t\t\t\t")

(t

(princ "other (misc) symbol:\t\t\t")

)

----------------------
And current counts - note will probably drop outputting the first category to the list unless someone suggests another meaningful symbol type to track

other (misc): 8734
keyword: 323
faces: 50
variables: 2542
user (custom) option: 659
non-user (setq) variable: 1183
functions
(non-command): 4277
keymap: 34
built-in primitive: 1022
byte-code: 2342
other (bound?) function: 335

command: 2373
keymap:3
built-in primitive: 88
byte-code: 762
other command: 1561
macro:298

===================
and finally the code itself if anyone wants to suggest other improvements or to give it a whirl on their setup.


-------------------
(defun macrop (object)
"Non-nil if and only if OBJECT is a macro."
(let ((def (indirect-function object t)))
(when (consp def)
(or (eq 'macro (car def))
(and (autoloadp def) (memq (nth 4 def) '(macro t)))))))

(defun list-hh-symbols ()
"Display a list of Emacs symbols - names only"
(interactive)
(message "Looking up all symbols...")
(with-output-to-temp-buffer "*List Symbols*"
(let (vars)
(mapatoms (function (lambda (sym)
(setq vars (cons sym vars)))))
(setq vars (sort vars 'string-lessp))
(while vars
(let ((sym (car vars)))
(cond
((fboundp sym) ; ALL functions
(cond
((commandp sym)
(cond

((keymapp sym)
(princ "=======\n")
(princ "\n")
(princ "command - keymap:\t\t\t\t\t")

(prin1 sym)
(princ "\n\n"))

((subrp (symbol-function sym))
(princ "=============================\n")
(princ "command - built-in primitive:\t\t")
(prin1 sym)
(princ "\n\n"))
((byte-code-function-p (symbol-function sym))
(princ "====================\n")
(princ "command - byte-code:\t\t\t")
(prin1 sym)
(princ "\n\n"))
((functionp sym)
(princ "================\n")
(princ "command - elisp:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t
(princ "===============\n")

(princ "other command:\t\t\t\t")

(prin1 sym)
(princ "\n\n"))
)
)
(t ; (non-command) functions
(cond

((macrop sym)
(princ "================\n")
(princ "function - macro:\t\t\t")

(prin1 sym)
(princ "\n\n"))

((keymapp sym)
(princ "==================\n")
(princ "\n")
(princ "function - keymap:\t\t\t\t")

(prin1 sym)
(princ "\n\n"))

((subrp (symbol-function sym))
(princ "==============================\n")
(princ "function - built-in primitive:\t\t")
(prin1 sym)
(princ "\n\n"))
((byte-code-function-p (symbol-function sym))
(princ "=====================\n")
(princ "function - byte-code:\t\t\t")
(prin1 sym)
(princ "\n\n"))
((functionp sym)
(princ "=================\n")
(princ "function - elisp:\t\t\t")
(prin1 sym)
(princ "\n\n"))
(t

(princ "============================\n")
(princ "other ~function (aliases??):\t")

(prin1 sym)
(princ "\n\n"))
)
)
)
)
((boundp sym) ; ALL variables
(cond
((custom-variable-p sym)
(princ "=====================\n")
(princ "user (custom) option:\t\t\t")
(prin1 sym)
(princ "\n\n"))

((keywordp sym)
(princ "========\n")
(princ "\n")
(princ "keyword:\t\t\t\t\t")

[Drew Adams]

> > The currently-in-development Emacs 24.4 includes `macrop'[1]...
>
> Excellent, thanks! ...

>
> should I try replacing that whole file or better to try just copying
> that function into my init.el?

Ensure that the function works OK with your Emacs version.
If so, copy just that function to one of your own files.

[Robert Thorpe]

han...@gmail.com writes:

> Total noob here - and non-programmer to boot, as will become
> immediately apparent from the code below - so please be gentle.
>
> Before I start getting to know emacs as an end-users - which I'm
> highly motivated to do, despite the amazingly steep learning curve to
> do the most basic things - I plan to of course highly customize my
> emacs environment to suit my needs, before starting the muscle-memory
> training required to become efficient.

There's no one right way to learn Emacs. But, I think the way you're
choosing is a lot of work.

You can start off using it for everyday editing, that's what I did and
what lots of people do. I expect you've done the tutorial and learned
the keybindings, that's very useful. Then read a bit of the manual
and the internet resources occasionally and learn more.

You only really need to looks for customizations, enable non-default
packages, etc. when you run into a problem or you feel something is
inefficient. Why change the standard behaviour if it's not a problem?

Don't worry too much about the customizing before learning the keychords.
The keys don't really change that much. A lot of non-standard packages
don't change the keychords at all, they just add new commands. Of the
rest lots of them use the default keychords to do slightly different
things. There are a few packages that add new keychords that you'll
want to use instead of default ones, but not many. There are very few
packages that override default keybindings with totally different
things. Mostly you can ignore that case.

You might want to add a shorter-keybinding for C-x o (other-window) and
get used to that. But that's about it.

There are lots of keychords that are undefined by default, those are
your playground to do what you want. As far as I know every keybinding
of the form C-c <alphabetic character> is free in all modes packaged
with Emacs.

BR,
Robert Thorpe

[Hans BKK]

On Saturday, April 19, 2014 12:34:31 PM UTC-4, Robert Thorpe wrote:
> Total noob here - and non-programmer to boot, as will become
> immediately apparent from the code below - so please be gentle.
>
> Before I start getting to know emacs as an end-users - which I'm
> highly motivated to do, despite the amazingly steep learning curve to
> do the most basic things - I plan to of course highly customize my
> emacs environment to suit my needs, before starting the muscle-memory
> training required to become efficient.

There's no one right way to learn Emacs. But, I think the way you're
choosing is a lot of work.

You can start off using it for everyday editing, that's what I did and
what lots of people do. I expect you've done the tutorial and learned
the keybindings, that's very useful. Then read a bit of the manual
and the internet resources occasionally and learn more.

You only really need to looks for customizations, enable non-default
packages, etc. when you run into a problem or you feel something is
inefficient. Why change the standard behaviour if it's not a problem?

<snip>

In the interest of keeping this thread focused on the implementation-specific issues rather than the diversions likely from my answer, I've started a new thread for the "pig picture" issues here:

http://groups.google.com/forum/#!topic/gnu.emacs.help/GLhpDW6Ksa8

[Hans BKK]

On Friday, April 18, 2014 1:15:38 AM UTC-4, Hans BKK wrote:
> To that end I'd like to generate standardized-format text "symbols
> reports" on the relevant values of all functions, variables, key
> bindings etc, so that I can do A-B comparison via diff between stock
> vanilla emacs vs after installing these bundles, or in fact any
> packages in the future.

Updated code, now including docstrings and variable values (including for functions)

Will likely kill the final "(t / leftovers / other (misc) symbol" bucket if it's true changes in that category are unlikely to be of interest.

Exploring the code and resulting output has been a decent learning exercise in its own right.

Any and all feedback welcome.

----------------------

(defun macrop (object)
"Non-nil if and only if OBJECT is a macro."
(let ((def (indirect-function object t)))
(when (consp def)
(or (eq 'macro (car def))
(and (autoloadp def) (memq (nth 4 def) '(macro t)))))))

(defun list-hh-symbols ()
"Display a list of Emacs symbols and their docstrings"

(interactive)
(message "Looking up all symbols...")
(with-output-to-temp-buffer "*List Symbols*"
(let (vars)
(mapatoms (function (lambda (sym)
(setq vars (cons sym vars)))))
(setq vars (sort vars 'string-lessp))
(while vars
(let ((sym (car vars)))
(cond
((fboundp sym) ; ALL functions
(cond

((commandp sym) ; interactive commands
(cond
((keymapp sym)
(princ "=================\n")
(unless (documentation sym)
(princ "\n"))
(princ "command - keymap: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))
)

((subrp (symbol-function sym))
(princ "=============================\n")

(unless (documentation sym)
(princ "\n"))
(princ "command - built-in primitive: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
((byte-code-function-p (symbol-function sym))
(princ "====================\n")

(unless (documentation sym)
(princ "\n"))
(princ "command - byte-code: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
((functionp sym)
(princ "================\n")

(unless (documentation sym)
(princ "\n"))
(princ "command - elisp: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))
)
(t
(princ "==============\n")
(unless (documentation sym)
(princ "\n"))
(princ "other command: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))
)
)
)
(t ; non-interactive functions
(cond
((macrop sym)
(princ "================\n")
(unless (documentation sym)
(princ "\n"))
(princ "function - macro: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
((keymapp sym)
(princ "==================\n")

(unless (documentation sym)
(princ "\n"))
(princ "function - keymap: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))
)

((subrp (symbol-function sym))
(princ "==============================\n")

(unless (documentation sym)
(princ "\n"))
(princ "function - built-in primitive: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
((byte-code-function-p (symbol-function sym))
(princ "=====================\n")

(unless (documentation sym)
(princ "\n"))
(princ "function - byte-code: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
((functionp sym)
(princ "=================\n")

(unless (documentation sym)
(princ "\n"))
(princ "function - elisp: ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
(t
(princ "============================\n")

(unless (documentation sym)
(princ "\n"))
(princ "other ~function (aliases??): ")

(prin1 sym)
(princ "\n\n")

(when (documentation sym)
(princ (substitute-command-keys
(documentation sym)))
(princ "\n\n"))
(when (boundp sym)
(princ "-------\n")
(princ "as value: ")
(prin1 (symbol-value sym))
(princ "\n\n"))
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n"))
(when (documentation sym)
(princ "\n"))

)
)
)
)
)
((boundp sym) ; ALL variables
(cond
((custom-variable-p sym)
(princ "=====================\n")

(unless (documentation-property sym
'variable-documentation)
(princ "\n"))
(princ "user (custom) option: ")

(prin1 sym)
(princ "\n\n")

(princ "---------------------\n")
(princ "value: ")
(prin1 (symbol-value sym))
(princ "\n\n")
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n\n"))
)
((keywordp sym)
(princ "========\n")
(princ "\n")
(princ "keyword: ")

(prin1 sym)
(princ "\n\n")
)

(t
(princ "=========================\n")

(unless (documentation-property sym
'variable-documentation)
(princ "\n"))
(princ "non-user (setq) variable: ")

(prin1 sym)
(princ "\n\n")

(princ "-------------------------\n")
(princ "value: ")
(prin1 (symbol-value sym))
(princ "\n\n")
(when (documentation-property sym
'variable-documentation)
(princ (substitute-command-keys
(documentation-property sym
'variable-documentation)))
(princ "\n\n\n")))

)
)
((facep sym)
(princ "=====\n")
(princ "\n")

(princ "face: ")

(prin1 sym)
(princ "\n\n")
)

(t
(princ "====================\n")

(princ "\n")
(princ "other (misc) symbol: ")

(prin1 sym)
(princ "\n\n")
)

)
(setq vars (cdr vars))))
(message "Looking up all symbols...done")
(with-current-buffer "*List Symbols*"
(setq buffer-read-only t)))))

;;;;;;;;;;;;;;;;;;;;;;;

(provide 'list-hh-symbols)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; list-hh-symbols.el ends here

[Florian v. Savigny]

> Exploring the code and resulting output has been a decent learning exercise in its own right.

I thought so. Even if -- worst case -- you end up saying that your
work has not been useful /at all/, you should have learned a good deal
about what symbols are, which is useful knowledge when you program in
elisp. (Efficiency, of course, is another matter.) I think a lot of
unorthodox approaches do not succeed, but if you do have the time,
there is nothing to suggest that there is no value in trying them out.

I could imagine that much of the veteran response is due to the
entirely natural and inevitable fact that the more experienced you
get, the older you necessarily get, and the older you get, the less
time you want to waste. (And, of course, the better you know how not
to waste it.) IOW, usefulness and efficiency are naturally developing
criteria. But they are not, as you have suggested, part of a dogma.

> Any and all feedback welcome.

OK, I'll take that literally, because I am not sure I have completely
understood the purpose of your package:

If you have icicle-mode, apparently, list-hh-symbols fails:
"Symbol's function definition is void: cycle-icicle-image-file-thumbnail".

Thus, I have tried your function with emacs -q --no-site-file, and got
a buffer of some 100K lines. I understand your intention is to diff
it, rather than browse it with your eyes, but then it would seem to me
the docstrings are not of much value, as I should think customisation
would not normally change them.

Also, is it practical to order them alphabetically, rather than, say,
by file from which they were loaded? As has been mentioned elsewhere,
most packages are "namespaced" by prefixing all their symbols with a
unique ID -- which should keep the symbols from one package together
in an alphabetical listing -- but that is not true for all of them (I
seem to recall some very fundamental ones.)

Are you intending to enable different kinds of listings? Scoping and
different kinds of ordering would seem important to me to make such
"reports" manageable.

What about customisations that advise existing functions, or even
redefine them?

> Will likely kill the final "(t / leftovers / other (misc) symbol"
> bucket if it's true changes in that category are unlikely to be of
> interest.

Whatever, there are a lot of "other (misc) symbol" entries all through
the listing, which do not at all look informative to me. (And a lot of
which, frankly, amaze me. But this is probably due to my limited
understanding of Elisp.) But who knows. Maybe it is sometimes
informative to simply know whether a symbol exists or not.

The challenge for you is to test your function in real life, and
demonstrate inhowfar it helps you to understand some given
customisation. Icicle-mode, for one, is a customisation that seems
impressive, but at the same time changes a lot of behaviour I was used
to. ;-)

--

Florian von Savigny
Melanchthonstr. 41
33615 Bielefeld

[Hans BKK]

On Wed, Apr 23, 2014 at 3:40 AM, Florian v. Savigny wrote:
> entirely natural and inevitable fact that the more experienced you
> get, the older you necessarily get, and the older you get, the less
> time you want to waste. (And, of course, the better you know how not
> to waste it.) IOW, usefulness and efficiency are naturally developing
> criteria. But they are not, as you have suggested, part of a dogma.

I beg to differ - most cultures in the world don't put a material value on their time with quite the same desperation as the western/material/consumerist one currently marching across the globe, which, once you're aware of alternatives, makes them IMO much more pleasant to live within.

Thanks very much for "wasting" the time with your response though Florian 8-)

> Thus, I have tried your function with emacs -q --no-site-file, and got
> a buffer of some 100K lines. I understand your intention is to diff
> it, rather than browse it with your eyes, but then it would seem to me
> the docstrings are not of much value, as I should think customisation
> would not normally change them.

Just to save me looking them up when scanning the diffs.

> Also, is it practical to order them alphabetically, rather than, say,
> by file from which they were loaded? As has been mentioned elsewhere,
> most packages are "namespaced" by prefixing all their symbols with a
> unique ID -- which should keep the symbols from one package together
> in an alphabetical listing -- but that is not true for all of them (I
> seem to recall some very fundamental ones.)
>
> Are you intending to enable different kinds of listings? Scoping and
> different kinds of ordering would seem important to me to make such
> "reports" manageable.
>
> What about customisations that advise existing functions, or even
> redefine them?

I think maybe you missed my very first statement of confession:

>> Total noob here - and non-programmer to boot, as will become immediately apparent from the code below - so please be gentle.

I've done a little batch file/macro coding but never worked with a proper language before, so my abilities to do any of this are very limited. Have no idea what scoping is, nor what "advising" a function might mean. Simple alpha-ordering is what was in the "list-options.el" code I based this on, so from my POV it certainly was "practical" to leave it that way.

Plus, it does seem to best further the goal of having a single standard function that runs against any arbitrary customized emacs to enable A-B comparisons with vanilla or a differently-customized one.

I would of course be very happy if anyone wants to refactor this to have it make more sense for their purposes, ideally keeping this ordering as an option.

> > Will likely kill the final "(t / leftovers / other (misc) symbol"
> > bucket if it's true changes in that category are unlikely to be of
> > interest.
>
> Whatever, there are a lot of "other (misc) symbol" entries all through
> the listing, which do not at all look informative to me. (And a lot of
> which, frankly, amaze me. But this is probably due to my limited
> understanding of Elisp.) But who knows. Maybe it is sometimes
> informative to simply know whether a symbol exists or not.

Yes if nothing else I'm sure it will help me in trying to understand a given package's source code.

> If you have icicle-mode, apparently, list-hh-symbols fails:
> "Symbol's function definition is void: cycle-icicle-image-file-thumbnail".

This message was very common as I was working on the different symbol categories, apparently icicles uses a type I didn't come across in my testing, so far only against plain vanilla.

See the "when" conditions testing for the presence (non-voidness) of a symbol's attribute before trying to query/output the value.

Perhaps this one is defined as "fboundp" but doesn't actually have a value in the function cell? I would have thought that wasn't even possible.

> Icicle-mode, for one, is a customisation that seems impressive, but at the same time changes a lot of behaviour I was used to. ;-)

Well I certainly won't have that problem since I haven't started to actually use emacs yet, in the sense of actually editing any text with it. 8-)

Icicles-specific question, as opposed to Helm, here:
https://groups.google.com/forum/#!topic/gnu.emacs.help/GSa1bqSKe9E

[Drew Adams]

> If you have icicle-mode, apparently, list-hh-symbols fails:
> "Symbol's function definition is void: cycle-icicle-image-file-thumbnail".

This was a bug - a copy+paste typo. This was the code:

(defalias 'cycle-icicle-image-file-thumbnail
'icicle-toggle-show-image-file-thumbnail)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(defun icicle-cycle-image-file-thumbnail ...)

The part underlined ^^^ should be icicle-cycle-image-file-thumbnail.
Fixed now.

So in addition to whatever help Hans's exercise has provided him
and others wrt learning Emacs, this thread has at least had the
added benefit of reporting this Icicles bug. ;-) Thx.

[Hans BKK]

On Wed, Apr 23, 2014 at 11:07 AM, Drew Adams wrote:
>> If you have icicle-mode, apparently, list-hh-symbols fails:
>> "Symbol's function definition is void: cycle-icicle-image-file-thumbnail".
>
> This was a bug - a copy+paste typo. This was the code:

...

> So in addition to whatever help Hans's exercise has provided him
> and others wrt learning Emacs, this thread has at least had the
> added benefit of reporting this Icicles bug. ;-) Thx.

Well I never, tickle me pink and call me Nancy!

Most welcome you are. . .