Go scanner using Rob Pike's lexer

1,582 views
Skip to first unread message

Archos

unread,
Nov 17, 2011, 7:55:17 PM11/17/11
to golang-nuts
Are there plans to refactore the Go scanner using the Rob Pike's
lexical scanning (used in template package)?


http://golang.org/pkg/go/scanner/

Rob 'Commander' Pike

unread,
Nov 18, 2011, 12:39:58 AM11/18/11
to Archos, golang-nuts
Why?

-rob

Archos

unread,
Nov 18, 2011, 6:32:20 AM11/18/11
to golang-nuts
Because it would easier to "translate" Go to another language.
Although I've found an awesome lexer that could help me:

https://github.com/iNamik/lexer.go

On Nov 18, 5:39 am, "Rob 'Commander' Pike" <r...@golang.org> wrote:
> Why?
>
> -rob

Jim Whitehead II

unread,
Nov 18, 2011, 6:39:57 AM11/18/11
to Archos, golang-nuts
On Fri, Nov 18, 2011 at 11:32 AM, Archos <raul...@sent.com> wrote:
> Because it would easier to "translate" Go to another language.
> Although I've found an awesome lexer that could help me:
>
> https://github.com/iNamik/lexer.go

As we've said to you several times, there is already a lexer and
parser in the Go standard libraries. What you're linked is a framework
for creating a lexer using Go, which it seems quite clearly is not
what you want to do. You should look at the go/parser [1] package.

[1]: http://golang.org/pkg/go/parser

- Jim

unread,
Nov 18, 2011, 6:44:31 AM11/18/11
to golang-nuts
On Nov 18, 12:32 pm, Archos <raul....@sent.com> wrote:
> Because it would easier to "translate" Go to another language.

Why would it be easier?

Archos

unread,
Nov 18, 2011, 6:52:41 AM11/18/11
to golang-nuts
I have been working with the Go lexer. And I can translate the
constants of single lines (step to step)

Does anybody could say me if the design of my function Compile (loop
for) is right? Or is there any better way to get it?

https://github.com/kless/GotoJS/blob/master/tojs/tojs.go#L38

On Nov 18, 11:39 am, Jim Whitehead II <jnwhi...@gmail.com> wrote:

Evan Shaw

unread,
Nov 18, 2011, 2:49:42 PM11/18/11
to Archos, golang-nuts
On Sat, Nov 19, 2011 at 12:52 AM, Archos <raul...@sent.com> wrote:
> I have been working with the Go lexer. And I can translate the
> constants of single lines (step to step)
>
> Does anybody could say me if the design of my function Compile (loop
> for) is right? Or is there any better way to get it?

Why use the scanner directly? The parser's already been written for
you. You'd be much better off using that.

- Evan

Jeremy Shute

unread,
Nov 18, 2011, 3:52:48 PM11/18/11
to Evan Shaw, Archos, golang-nuts
Almost all of the work will be in doing stuff like type-checking, binding/scoping, and converting between the semantics of Go and your target language, anyway.  The lexer is probably the least difficult part.

Jeremy

Archos

unread,
Nov 18, 2011, 7:25:22 PM11/18/11
to golang-nuts
By fear. If the parser is refactored (since there are many changes in
Go) then I'd have to start almost from scratch. In change, my code
woud be easier to modify if I use the lexer.

On Nov 18, 7:49 pm, Evan Shaw <eds...@gmail.com> wrote:

Evan Shaw

unread,
Nov 18, 2011, 7:36:40 PM11/18/11
to Archos, golang-nuts
On Sat, Nov 19, 2011 at 1:25 PM, Archos <raul...@sent.com> wrote:
> By fear. If the parser is refactored (since there are many changes in
> Go) then I'd have to start almost from scratch. In change, my code
> woud be easier to modify if I use the lexer.

That strikes me as very paranoid. We're on the path to Go 1, a stable
version of the language, and all changes to non-experimental packages
come with gofix rewriters.

It's your project, though. Have fun.

- Evan

davidp...@gmail.com

unread,
Apr 9, 2012, 6:44:46 PM4/9/12
to golan...@googlegroups.com
Hi Archos,

Although it doesn't sound like my lexer project is right for your plan, I appreciate the mention.

If you end up using my lexer and/or parser in your own projects, I'd love to hear about it.

FYI - I've discovered a pitfall in naming your projects *.go, and have renamed my go projects to go_*, so the new URL for my lexer project is:

https://github.com/iNamik/go_lexer

Thanks again for the mention and good luck on your project!

-DFarrell (iNamik)

aditya....@gmail.com

unread,
Mar 17, 2014, 2:04:03 PM3/17/14
to golan...@googlegroups.com
how can i implement a GO lexer in c??any ideas?

Mateusz Czapliński

unread,
Mar 18, 2014, 7:59:26 AM3/18/14
to golan...@googlegroups.com, aditya....@gmail.com
On Monday, March 17, 2014 7:04:03 PM UTC+1, aditya....@gmail.com wrote:
how can i implement a GO lexer in c??any ideas?

I believe there's a chance this may help you:
  http://golang.org/src/cmd/gc/lex.c
and/or other files in the containing directory, i.e. in:
  http://golang.org/src/cmd/gc/

/M.
Reply all
Reply to author
Forward
0 new messages