Invalid-token when dereferencing namespaced keywords.

[Philip Markgraf]

I am moving some code to use spec and namespaced keywords under Clojure 1.10.0 (release). One group of keywords starts with a numeric character after the colon, which has worked fine in the non-namespaced context. Creating and using the namespaced keyword works correctly in the local namespace (using only the double-colon), but fails with "Invalid Token" when dereferencing from another workspace.

I'm not sure if this is a bug or if I have been taking advantage of an undocumented/unsupported feature.

Having a leading-digit keyword has been very useful, as the names are an exacting fit of the problem domain and don't suffer from the addition of any visual pollution.

user=> (def example-a {:015-00 "015-00"})
#'user/example-a
user=> (def example-b {::015-00 "015-00"})
#'user/example-b
user=> (:015-00 example-a)
"015-00"
user=> (:015-00 example-b)
nil
user=> (::015-00 example-b)
"015-00"
user=> (::015-00 example-a)
nil
user=> (ns try)
nil
try=> (:015-00 user/example-a)
"015-00"
try=> (::user/015-00 user/example-b)

Syntax error reading source at (REPL:1:15).
Invalid token: ::user/015-00
Syntax error reading source at (REPL:1:31).
Unmatched delimiter: )
try=>

The current behavior is certainly inconsistent, even if it is not a serious bug.

Thank you!

[Justin Smith]

you are misusing the :: alias resolution operator, user is not an alias

Clojure 1.9.0
(ins)user=> (ns foo)
nil
(ins)foo=> ::user/a
RuntimeException Invalid token: ::user/a
clojure.lang.Util.runtimeException (Util.java:221)
(ins)foo=> :user/a
:user/a

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Philip Markgraf]

Thank you. I apologize that my condensed example was in error. (Is the error you are showing that you can't create a namespaced keyword in a namespace you are not in?)

However, the condition I originally describe occurs when the original namespace is properly created with ns:

user=> (ns name1)
nil
name1=> (def ex1 {:015-00 "1"})
#'name1/ex1
name1=> (def ex2 {::015-00 "2"})
#'name1/ex2
name1=> (::015-00 ex2)
"2"
name1=> (ns name2)
nil
name2=> (::name1/015-00 name1/ex2)

Syntax error reading source at (REPL:1:16).
Invalid token: ::name1/015-00
Syntax error reading source at (REPL:1:27).
Unmatched delimiter: )
name2=>

[Andy Fingerhut]

I believe the original intent was that keywords with a digit immediately after the colon would not be supported, but due to a bug in the Clojure reader they were permitted.  There was a brief time in 2013 where this bug was fixed, but there were enough extant Clojure code bases that used such keywords, that the change was reverted.  See the comments on the Clojure ticket below for some history:

https://dev.clojure.org/jira/browse/CLJ-1252

I believe that this bug has never existed for namespaced keywords, and I am not aware of any desire to change Clojure to allow such keywords.

Andy

[Alex Miller]

There are two interrelated issues here.

First, when you are using autoresolved keywords, the qualifier part must be an alias. Here it is a fully-qualified namespace (user). Instead, you should be using :user/015-00. Note that before Clojure 1.10, this would not produce an error - this was an oversight that was tightened up in 1.10.

:user/015-00 introduces the second error. Keywords with a leading digit were not originally legal (according to the reader reference), but have been grandfathered in (like :1st, etc). However, qualified keywords with a digit name have never been accepted by the reader and this is kind of an unresolved issue.

[Philip Markgraf]

Thank you, Alex and Andy. This answers my question regarding leading digit in keywords.

Alex,

You describe an error with autoresolved keywords in my example. Is this also true of the second example I posted (in response to Justin), which does not use the user namespace? Am I correct to use double-quote inside a namespace, and double-quote, namespace-alias, slash when using the keyword in another namespace?

Thank you!

[Alex Miller]

On Tuesday, January 29, 2019 at 10:07:29 PM UTC-6, Philip Markgraf wrote:

Thank you, Alex and Andy. This answers my question regarding leading digit in keywords.

Alex,

You describe an error with autoresolved keywords in my example. Is this also true of the second example I posted (in response to Justin), which does not use the user namespace?

Yes

 

Am I correct to use double-quote inside a namespace, and double-quote, namespace-alias, slash when using the keyword in another namespace?

No, in all of these cases you are using fully-qualified namespaces, not aliases. The double-colon is only for autoresolving based on the aliases of the current namespace, and you are not using aliases in any of your examples.

For example, you could have done the following to use an autoresolved keyword based on the alias n1:

(ns name2 (:require [name1 :as n1]))

(::n1/015-00 name1/ex2)  ;; invalid token

Although note that you will still run into the second problem of qualified keywords with leading digit names, so the latter won't work. For something like this, you'll need to avoid the leading digit in the keyword name.

Or you can programmatically create the keyword if you must:

((keyword "name1" "015-00") n1/ex2) . ;; "2"

[Philip Markgraf]

My actual case is using the keyword across two files (namespaces), using the ns/:require/:as constructs in the file that did not define the keyword. (The REPL example was created to give a minimal replication of the leading digit issue.)

So, now I am really wondering what the intent with spec and namespaced keywords is. When should we use single-colon keywords and when should we use the double-colon variety?

[Thomas Heller]

To expand on what Alex already mentioned. There is no such thing as double-colon keywords. Double-colon is a reader alias mechanism that let the reader resolve them so you can type less.

(ns foo.bar.xyz)

::hello

this is resolved at read-time and identical to actually writing

:foo.bar.xyz/hello

:: without a slash will use the current namespace as the namespace for the keyword.

If you have a require for another namespace and use :: with a slash you can use it to resolve your require alias.

(ns some.thing

  (:require [foo.bar.xyz :as x]))

::x/hello

again becomes

:foo.bar.xyz/hello

in your actual program. Nothing :: will ever exist at runtime.

So if you want you can always use the fully qualified keyword directly or you can let the require resolve them based on your ns require's. That means you'll get an error if there is no alias to resolve

(ns user2)

::user1/foo

This is invalid because user2 does not have an alias to user1, instead it is an actual full namespace.

:user1/foo

would be fine in this case (and identical to ::foo in user1). Or

(ns user2

  (:require [user1 :as u1]))

::u1/foo

Hope that makes things clearer. 

Cheers,

Thomas

[Philip Markgraf]

Thank you Justin, Andy, Alex and Thomas.

I now understand both the root of my issue (leading digits in keyword names is not allowed) and have a greater understanding of how keywords work, especially the reader's role in expanding double-colon keywords.

[Dieter Komendera]

We have the common use-case that we want to use double-colon keywords for well known namespaces, without actually requiring the namespaces.

(defmacro ns-alias
  "Set up a custom alias for use with namespace keywords."
  [ns as]
  `(do
     (create-ns '~ns)
     (alias '~as '~ns)))

This proved useful to prevent cyclic requires while still being able to use the shorthand syntax for long and common namespaces.

Cheers,

Dieter