Julia grammar

[nikolai...@icloud.com]

Hi all!

I want to parse julia in java, so i came looking to the sources and now i have some questions:

1) do i get it right that julia lexer and parser are both here https://github.com/JuliaLang/julia/blob/master/src/julia-parser.scm and that there is no BNF grammar or something like that?

2) is there currently any work going on to rewrite it or to bring some major changes to the parser?

3) is there any convenient way to parse julia in julia and then print the AST? I have seen the parse function but it works only parses expressions, not whole files.

Thanks

[Patrick O'Leary]

On Monday, August 25, 2014 7:13:45 AM UTC-5, nikolai...@icloud.com wrote:

Hi all!

I want to parse julia in java, so i came looking to the sources and now i have some questions:

1) do i get it right that julia lexer and parser are both here https://github.com/JuliaLang/julia/blob/master/src/julia-parser.scm and that there is no BNF grammar or something like that?

That is correct.
 

2) is there currently any work going on to rewrite it or to bring some major changes to the parser?

There's stuff being played with, but the most radical proposal is my answer to (3).
 

3) is there any convenient way to parse julia in julia and then print the AST? I have seen the parse function but it works only parses expressions, not whole files.

https://github.com/jakebolewski/JuliaParser.jl

[Tobi]

But does it really matter for 3) if the Julia parser or the Scheme parser is used? I would say just load the entire file into a string and use the regular parsing function.

Cheers,

Tobi

[Patrick O'Leary]

On Monday, August 25, 2014 8:19:36 AM UTC-5, Tobi wrote:

But does it really matter for 3) if the Julia parser or the Scheme parser is used? I would say just load the entire file into a string and use the regular parsing function.

I don't believe that works. Base.parse() won't handle strings with more than one expression. I want to say this has come up before but I don't recall what the resolution was.

[Stefan Karpinski]

There's a method of parse that takes a starting position into the string and returns where it finished parsing. That version can be used to parse as many expressions from a string as one needs. Example:

julia> text = """

       1 + 2

       [ 1//(i + j - 1) for i=1:10, j=1:10 ]

       println("Hello, world.")

       """

"1 + 2\n[ 1//(i + j - 1) for i=1:10, j=1:10 ]\nprintln(\"Hello, world.\")\n"

julia> i = 1

1

julia> while !done(text,i)

           ex, i = parse(text,i)

           @show ex

       end

ex = :(1 + 2)

ex = :($(Expr(:comprehension, :(1 // ((i + j) - 1)), :(i = 1:10), :(j = 1:10))))

ex = :(println("Hello, world."))

julia> i

70

[Tobi]

Well, in https://github.com/tknopp/Julietta.jl/blob/master/src/sourcedocument.jl#L72
I had called Base.parse() in a while loop until the file reaches the end (actually its a GtkTextBuffer but this should not matter)
But I was not looking at the AST but was only interested if a parsing error occured.

[nikolai...@icloud.com]

On Monday, August 25, 2014 5:45:28 PM UTC+4, Stefan Karpinski wrote:

There's a method of parse that takes a starting position into the string and returns where it finished parsing. That version can be used to parse as many expressions from a string as one needs. 

Thanks Stefan

On Monday, August 25, 2014 4:55:29 PM UTC+4, Patrick O'Leary wrote:

 3) is there any convenient way to parse julia in julia and then print the AST? I have seen the parse function but it works only parses expressions, not whole files.
https://github.com/jakebolewski/JuliaParser.jl 

Thanks Patrick, this is a very nice link.

I guess that's enough for me to start going. Hopefully, I'll report back soon

[Jake Bolewski]

Ping me if I can be any help.  I haven't added the parse method with a fixed offset argument but it should be easy to add, in fact it would make a nice contribution :-)

Jake