More DrRacket binding arrow woes

[Alexis King]

This is not the first time I have asked about the binding arrows on this
mailing list, but I seem to have run into a new problem that has me
completely stumped. I have written a #lang that uses an entirely custom
implementation of read-syntax (it does not wrap Racket’s read-syntax in
any way), and it seems to work alright. I can write some simple
programs, and they run properly.

However, I am not seeing any of the Check Syntax binding arrows at all.
Neither the arrows from imports nor arrows between definitions show up.
Inspecting the syntax objects at read-time and after expansion in the
macro stepper reveals quite clearly that the syntax objects I am
producing are syntax-original?, and adding the
'original-for-check-syntax syntax property doesn’t affect things,
either. The bindings are clearly correct, since the program runs, and
comparing the scopes and syntax properties to an equivalent program in
#lang racket doesn’t reveal any meaningful differences. I have no idea
what I’m doing wrong.

Any ideas on how to diagnose this issue? I am using syntax/module-reader
along with the #:whole-body-readers? option, if that is meaningful (I’m
also not configuring #:language-info, if that matters). I can also post
the code I have somewhere if people think that would help, but it’s a
bit much without reducing it to a self-contained example, so I figured
I’d ask first to see if I’m forgetting something silly.

Alexis

[Alex Knauth]

> On May 7, 2016, at 5:18 AM, Alexis King <lexi....@gmail.com> wrote:

> However, I am not seeing any of the Check Syntax binding arrows at all.
> Neither the arrows from imports nor arrows between definitions show up.
> Inspecting the syntax objects at read-time and after expansion in the
> macro stepper reveals quite clearly that the syntax objects I am
> producing are syntax-original?, and adding the
> 'original-for-check-syntax syntax property doesn’t affect things,
> either. The bindings are clearly correct, since the program runs, and
> comparing the scopes and syntax properties to an equivalent program in
> #lang racket doesn’t reveal any meaningful differences. I have no idea
> what I’m doing wrong.
>

> Any ideas on how to diagnose this issue? I can also post

> the code I have somewhere if people think that would help, but it’s a
> bit much without reducing it to a self-contained example, so I figured
> I’d ask first to see if I’m forgetting something silly.
>
> Alexis

So, the bindings are fine, and the syntax-original? property is fine. Is the source location good, including the `syntax-source` field?

I had a similar issue once, and it turned out that it was because the `syntax-source` part of the source location was wrong. I had been using things like `(input-port-append (open-input-string ....) ....)`, which were resulting in `syntax-source` fields of `'string` for some of the syntax objects. So that's something silly that can happen.

Other than that I don't know very much of what can go wrong to mess up the check-syntax arrows. Oh, also, they don't show up from background expansion, but did you mean that they also don't show up when you click the actual Check Syntax button? Sometimes that works when background expansion doesn't (it did for the `'string` case).

Alex Knauth

[Robby Findler]

It's not great, but one of the things I do to try to diagnose issues
like this is to put printfs into the check syntax implementation
itself. They will show up in the stdout put of DrRacket (so start it
from a Terminal window on the mac, say). To test Alex's hypothesis, I
suggest putting one that shows the inputs being passed to `add-id` on
this line:

https://github.com/racket/drracket/blob/master/drracket-tool-lib/drracket/private/syncheck/traversals.rkt#L505

You should be able to just stick the printf in there and then restart
DrRacket without recompiling anything (assuming zos were already up to
date).

I also think that building a tool around this idea (that used logging
instead of printfs) could be quite useful. There are a few places
where one could put some logging information that shows what's going
on to help out language designers and see why check syntax bails out.

Robby

> --
> You received this message because you are subscribed to the Google Groups "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Alexis King]

> So, the bindings are fine, and the syntax-original? property is fine. Is the source location good, including the `syntax-source` field?
>
> I had a similar issue once, and it turned out that it was because the `syntax-source` part of the source location was wrong. I had been using things like `(input-port-append (open-input-string ....) ....)`, which were resulting in `syntax-source` fields of `'string` for some of the syntax objects. So that's something silly that can happen.

Yup, this seems to have been the problem. Interestingly enough, I had
thought of this before, but faking out the source field with what
appeared to be the proper value didn’t seem to work. I’m not entirely
sure why, but Robby’s suggestion helped track down the issue, and based
on the behavior I saw, it seems like it might be due to bad caching of a
.zo file that for some reason seemed to be sporadically picked up when
running the expander in different ways (Check Syntax, Macro Stepper,
just running the thing and printing things out).

I’m not really sure about why I was having difficulty getting those
changes to reload before, but that’s a separate issue. It would be a
little nice to mention the complete list of fields that DrRacket looks
at before drawing arrows, given that I had no idea which things I needed
to check. At least having a checklist of properties would help with
that, and the logging idea would be helpful, too.

Thanks to both of you!

[Robby Findler]

There are bugs in DrRacket's handling of the automatic-zo compilation
mechanism that I know about and have been trying to find time to fix.
With the current buggy state, you should always be able to get to a
non-buggy use if you do a "raco make x.rkt" in the shell (I think).
Or, if you disable DrRacket's automatic zo creation, that should also
avoid this problem (but then you have to manage the .zo files
yourself, of course, and racket's default settings can also lead to
confusing behavior when zo files are out of date.)

Glad you got it fixed!

Robby

[Sorawee Porncharoenwase]

I have a similar problem but I can't figure out a way to fix it yet :(

So, when I hover on `sum` on line 3, I get "no bound occurrences". That's definitely false since the program runs fine, and `sum` has a bound occurrence below (e.g., line 5).

I print the syntax objects passed into `#%module-begin` in the REPL. The srcloc seems right to me.

What baffles me further is that when I follow Robby's suggestion to insert a printf inside check-syntax code:

(printf "~a ~a ~a\n" stx-obj (syntax-source stx-obj) (syntax-line stx-obj))

I get the following result (only relevant lines shown)

#<syntax temp1> #f #f
#<syntax temp1> #f #f
#<syntax:3:20 s> string 3
#<syntax:3:23 id> string 3
#<syntax temp1> #f #f
#<syntax:3:9 id> string 3
#<syntax f3> #f #f
#<syntax:5:0 sum> string 5
#<syntax temp9> #f #f
#<syntax temp9> #f #f
#<syntax:7:16 sum> string 7
#<syntax:7:21 id> string 7
#<syntax:7:29 id> string 7
#<syntax temp9> #f #f
#<syntax:7:10 z> string 7
...

So: the sum on line 3 is either missing or somehow becomes `#<syntax temp1> #f #f`

And sum on line 7 has a correct srcloc, except that the syntax-source somehow changes from `'unsaved-editor` to `string`?!?

FYI, I use megaparsack's `parse-syntax-string` like this:

(define (parse src in)
  (define len (string-length "#lang recursive-language"))
  (parse-result!
   (parse-syntax-string toplevel/p
                        (datum->syntax #f (port->string in)
                                       (list src 1 len (add1 len) 0))))

And this is my module reader

(module reader racket
  (require syntax/strip-context)
  (require "parser.rkt")
  (provide (rename-out [my-read-syntax read-syntax]
                       [my-read read]))
  (define (my-read in) (syntax->datum (my-read-syntax #f in)))
  (define (my-read-syntax _ in)
    (datum->syntax #f
      `(module src recursive-language
        ,@(parse (object-name in) in)))))

Any idea how to fix this? Also feel free to suggest anything that you find wrong -- this is the first time I touch the reader!

Thank you :)

[Alexis King]

It is worth pointing out that Check Syntax does not see the result of
your reader, it sees the fully-expanded module and performs its analysis
from there. Therefore, it can be useful to use the macro stepper with
macro hiding disabled to inspect the fully-expanded module to make sure
that the information at least appears to be in place.

In the case of your language, I would use the macro stepper to ensure
the following three things are happening:

1. The `sum` identifier appears in binding position in the
fully-expanded program, and it has the appropriate source location
information.

2. The `sum` identifier also appears in use position in the
fully-expanded program, and it has distinct source location
information from the identifier in binding position.

3. Both identifiers are syntax-original?.

If those things are all the case, it’s difficult for me to say much more
without looking at the code itself.

(As an aside, you should probably be providing the first argument of
my-read-syntax as the first argument to `parse` instead of (object-name
in), but I’m not sure if that’s at all related to the issue here.)

> On Feb 13, 2018, at 23:33, Sorawee Porncharoenwase

[Sorawee Porncharoenwase]

It works now! I think I simply need a `strip-context` and avoid `datum->syntax`.

Thank you very much for your help!

[Matthias Felleisen]

Instead of parsing the raw strings, you might consider the following alternative: 

 — use read tables to nail down identifier syntax and literal constant syntax 

 — define all keywords and operators and whatever as plain syntax ids that raise a syntax error 

 — all remaining things are your identifiers  

 — now use module-begin to turn the sequence of tokens into a tree where each of your identifiers becomes an actual Racket id 

I just wrote a neat little paren-free language for optmizing network flows this way and it’s a rather viable strategy. 

[Sorawee Porncharoenwase]

Thank you for your advice!

If your code is online and you don't mind, could you point me to the project or any other tutorial so that I can see the sample code? Right now I have https://docs.racket-lang.org/guide/hash-reader.html but the grammar is rather simple. It would be nice if there is a more complex example.

[Matthias Felleisen]

Here is a snippet of code in the language: 

#lang network-flow

;; sample graph

problem: maximize from a to d 

node a -- 10 --> b

node a -- 20 --> c

node b -- 30 --> d

node c -- 30 --> d

Here is a link to the language 

http://www.ccs.neu.edu/home/matthias/4620-s18/network-syn-lang.rkt

(The github repo for the course is private. Sorry)

— Matthias