mksyscall windows

[Archos]

The Windows call systems used in Go are generated automatically though mksyscall_windows.pl

http://golang.org/src/pkg/syscall/syscall_windows.go#L117
http://golang.org/src/pkg/syscall/zsyscall_windows_386.go

Is there any reason to don't add it to the directory "bin" together to the Go tools? For that it can be used by developers that need to use another system calls/functions in Windows.

[brainman]

On Monday, 20 August 2012 07:26:29 UTC+10, Archos wrote:

...
Is there any reason to don't add it to the directory "bin" together to the Go tools? ...

Hardly anyone uses it. I do. But it is as easy to run it from $GOROOT/src/pkg/syscall/msysscall_windows.pl. I just create a Makefile for it and forget. On the other hand, I do not think many Windows developers have perl installed. So, if to be used widely, it should be, probably, rewritten in Go.

Alex

[Archos]

I have filled in an issue about it:

http://code.google.com/p/go/issues/detail?id=3977

[Archos]

From mksyscall_windows.pl is focused for bootstrap Go I've found some time to build it in Go, so it can be used by developers that want deal with Windows system calls and they ha not Perl installed.

It's smarter than Perl's version from that it creates automatically the files for different architectures, if were necessary. No more flag "-l32".

Note that the code size is more than the double of the Perl's code size, but the Go version uses a scanner to parsing the function prototypes.

https://github.com/kless/syscall2go

[brainman]

On Friday, 24 August 2012 07:31:51 UTC+10, Archos wrote:

...
https://github.com/kless/syscall2go

I knew it should not be hard to write one :-).

There is one problem with your program. I run it like:

./syscall2go $GOROOT/src/pkg/syscall/syscall_windows.go

It display this:

1:164: error condition: expected integer
1:119: parameters: expected type
1:170: parameters: expected type
1:57: out: expected left parenthesis
1:148: error condition: expected integer
1:75: expected to finish
1:165: expected to finish

I do not think it is intended to do that.

I would write a test that input $GOROOT/src/pkg/syscall/syscall_windows.go file into your program and compare its output with the contens of $GOROOT/src/pkg/syscall/zsyscall_windows_386.go. Once this test PASS, I would know that my program works. It would also be helpful next time I need to change my code, because I would re-run this test to make sure it still works.

Alex

[Archos]

El viernes, 24 de agosto de 2012 01:41:46 UTC+1, brainman escribió:

On Friday, 24 August 2012 07:31:51 UTC+10, Archos wrote:

...
https://github.com/kless/syscall2go

I knew it should not be hard to write one :-).

There is one problem with your program. I run it like:

./syscall2go $GOROOT/src/pkg/syscall/syscall_windows.go

It display this:

1:164: error condition: expected integer
1:119: parameters: expected type
1:170: parameters: expected type
1:57: out: expected left parenthesis
1:148: error condition: expected integer
1:75: expected to finish
1:165: expected to finish

I do not think it is intended to do that.

Thanks for report it.
Those errors are got at parsing the prototypes lines; I'll check them and I'll add better error messages. 

 

I would write a test that input $GOROOT/src/pkg/syscall/syscall_windows.go file into your program and compare its output with the contens of $GOROOT/src/pkg/syscall/zsyscall_windows_386.go. Once this test PASS, I would know that my program works. It would also be helpful next time I need to change my code, because I would re-run this test to make sure it still works.

It it not as easy because syscall2go writes several files from the source. You can check like the Perl version repeats the same code for different architectures:

http://golang.org/src/pkg/syscall/zsyscall_windows_386.go#L148
http://golang.org/src/pkg/syscall/zsyscall_windows_amd64.go#L148

The Go version generates:

  source.go                  => where the prototype lines are
  zsyscall_source.go    => it is always created; // +build windows

These one are built only for prototypes that have an int64 in the input arguments:

  zsyscall_source_32b => use // +build windows,!amd64
  zsyscall_source_64b => use // +build windows,amd64

Although I could add a flag -t or -d to create all into a same file allowing to be compared.

[Archos]

Does anybody could confirm that the next prototype line is right? (using **CertContext)

//sys   CertAddCertificateContextToStore(store Handle, certContext *CertContext,
addDisposition uint32, storeContext **CertContext) (err error) = crypt32.CertAddCertificateContextToStore

http://golang.org/src/pkg/syscall/syscall_windows.go#L192

[minux]

why do you think it's not correct?

it's used by /src/pkg/crypto/x509/root_windows.go 

[Archos]

Because it was the only prototype using **.
I'll update the scanner to allow it.

[Archos]

brainman, any different suggestion to name the files generated?

Note that source.go will be almost always source_windows.go, so the files generated according to my scheme would be: zsyscall_source_windows.go, and optionally zsyscall_source_windows_32b.go and zsyscall_source_windows_64b.go

[brainman]

I should not be telling you what to do - it is your project. But, if I would write program like that, I would allow for my program to accept multiple input file names on command line and output to stdout. Just like current perl program does.

Alex

[Archos]

El viernes, 24 de agosto de 2012 23:19:10 UTC+1, brainman escribió:

I should not be telling you what to do - it is your project. But, if I would write program like that, I would allow for my program to accept multiple input file names on command line and output to stdout. Just like current perl program does.

 

The program already did that.

[brainman]

Then I didn't understand your question. You were asking about "generated file names". But if you output to stdout, then it is not applicable.

Alex

[Archos]

It has the flag -w to create files ;)
At the end, the names genearated are i.e. from a file named source_windows.go:
zsource_windows.go, and optionally zsource_windows_amd64.go and zsource_windows_386.go

[Archos]

The prototype on the next file [1] to create system calls in Windows, should have a windows API name in tittle case. It has now: "kernel32.getCurrentProcessId" but it should be "kernel32.GetCurrentProcessId" [2]

//sys getCurrentProcessId() (pid uint32) = kernel32.getCurrentProcessId

[1]: http://golang.org/src/pkg/syscall/syscall_windows.go#L204
[2]: http://winapi.freetechsecrets.com/win32/WIN32GetCurrentProcessId.htm

Or am I wrong? because "mksyscall_windows.pl" says:

>>> If go func name needs to be different from it's winapi dll name,
>>> the winapi name could be specified at the end, after "=" sign, like
>>>  //sys LoadLibrary(libname string) (handle uint32, err error) = LoadLibraryA

http://golang.org/src/pkg/syscall/mksyscall_windows.pl

[minux]

On Sat, Aug 25, 2012 at 9:51 PM, Archos <raul...@sent.com> wrote:

The prototype on the next file [1] to create system calls in Windows, should have a windows API name in tittle case. It has now: "kernel32.getCurrentProcessId" but it should be "kernel32.GetCurrentProcessId" [2]

//sys getCurrentProcessId() (pid uint32) = kernel32.getCurrentProcessId

the name is changed to avoid introducing new API in Go 1.0.x point releases. 

[Archos]

The code generated by syscall2go is correct.

I've checked it compiling Go in Windows with the system calls generated by this program. The test included with the program does it.

By the way, this is a good moment to rename the project, maybe "mksyscall"? just like the Perl version

[Archos]

https://github.com/kless/mksyscall

Note: I added a new flag to enable to use MustLoadDLL instead of NewLazyDLL.
If somebody has some doubt about those functions read the doc.:

    godoc syscall MustLoadDLL   [NewLazyDLL]

http://golang.org/src/pkg/syscall/dll_windows.go

[brainman]

On Saturday, 25 August 2012 10:23:25 UTC+10, Archos wrote:

El viernes, 24 de agosto de 2012 23:19:10 UTC+1, brainman escribió:

..., I would allow for my program to accept multiple input file names on command line and output to stdout. Just like current perl program does.

 
The program already did that.

Please, show me how to run it, so it inputs both $GOROOT//src/pkg/syscall/syscall_windows.go and  $GOROOT//src/pkg/syscall/zsyscall_windows_386.go and outputs contents of $GOROOT//src/pkg/syscall/zsyscall_windows_386.go to stdout.

Alex

[Archos]

Ok, at the first, we show how it was generated to know the files where it got the input:
 
$ head -n1 $GOROOT/src/pkg/syscall/zsyscall_windows_386.go
// mksyscall_windows.pl -l32 syscall_windows.go security_windows.go syscall_windows_386.go

$ head -n1 $GOROOT/src/pkg/syscall/zsyscall_windows_amd64.go
// mksyscall_windows.pl syscall_windows.go security_windows.go syscall_windows_amd64.go

And we only have to run:

$ mksyscall $GOROOT/src/pkg/syscall/syscall_windows.go $GOROOT/src/pkg/syscall/security_windows.go

Note: I don't add the another files (syscall_windows_amd64.go, syscall_windows_386.go) because they're empty.

// * * *

By the way, sorry but the project is in: https://github.com/kless/cutil/tree/master/mksyscall

To install: go get github.com/kless/cutil/mksyscall

Sorry, but I've several projects related and I prefer have them like sub.projects.

[brainman]

On Monday, 27 August 2012 18:26:33 UTC+10, Archos wrote:
>
> And we only have to run:
> $ mksyscall $GOROOT/src/pkg/syscall/syscall_windows.go $GOROOT/src/pkg/syscall/security_windows.go

Yeah, that is what I have tried too. And I expect the output of this command

# diff $GOROOT/src/pkg/syscall/zsyscall_windows_386.go <(./mksyscall $GOROOT/src/pkg/syscall/syscall_windows.go $GOROOT/src/pkg/syscall/security_windows.go)

to be empty. But it is not. So, whatever your program does, it is different to what mksyscall_windows.pl does.

Like I said at the start of this thread, I would use this rule to write my test. For example, if I would implement all generation in this function:

// mksyscall reads multiple go files, extracts //sys comments
// describing dll function call, generates correspondent
// syscall functions and writes them to dst stream.
func mksyscall(dst io.Writer, src []io.Reader) error {
    // ... to be implemented ...
    return nil
}

then I would write this test:

package main

import (
    "io"
    "io/ioutil"
    "os"
    "path/filepath"
    "testing"
)

func TestMksyscall(t *testing.T) {
    goroot := os.Getenv("GOROOT")
    if len(goroot) == 0 {
        t.Fatalf("GOROOT environment variable is not set")
    }

    d := filepath.FromSlash(goroot + "/src/pkg/syscall")
    src1, err := os.Open(filepath.Join(d, "syscall_windows.go"))
    if err != nil {
        t.Fatalf("Failed to open security_windows.go: %v", err)
    }
    defer src1.Close()
    src2, err := os.Open(filepath.Join(d, "security_windows.go"))
    if err != nil {
        t.Fatalf("Failed to open security_windows.go: %v", err)
    }
    defer src2.Close()

    dst, err := ioutil.TempFile("", "zzz")
    if err != nil {
        t.Fatalf("Failed to create temp file: %v", err)
    }
    defer dst.Close()
    defer os.Remove(dst.Name())

    err = mksyscall(dst, []io.Reader{src1, src2})
    if err != nil {
        t.Fatalf("mksyscall failed: %v", err)
    }

    golden, err := os.Open(filepath.Join(d, "zsyscall_windows_386.go"))
    if err != nil {
        t.Fatalf("Failed to open zsyscall_windows_386.go: %v", err)
    }
    defer golden.Close()

    dstfi, err := dst.Stat()
    if err != nil {
        t.Fatalf("Failed to stat output file: %v", err)
    }
    goldnefi, err := golden.Stat()
    if err != nil {
        t.Fatalf("Failed to stat zsyscall_windows_386.go: %v", err)
    }
    if dstfi.Size() != goldnefi.Size() {
        t.Errorf("Wrong output file size of %d, expected %d", dstfi.Size(), goldnefi.Size())
    }
    _, err = dst.Seek(0, 0)
    if err != nil {
        t.Fatalf("Failed to seek to the begining of output file: %v", err)
    }
    for i := dstfi.Size(); i > 0; i-- {
        var goldenb, dstb [1]byte
        _, err = golden.Read(goldenb[:])
        if err != nil {
            t.Fatalf("Error reading from zsyscall_windows_386.go: %v", err)
        }
        _, err = dst.Read(dstb[:])
        if err != nil {
            t.Fatalf("Error reading from output file: %v", err)
        }
        if goldenb[0] != dstb[0] {
            t.Fatalf("Unexpected output file contents")
        }
    }
}

and wouldn't even worry about any flags, options, features and such until test passes.

I would also use templates to write the output. It would be simpler, and would make it easier to change things quickly when I need to.

I would also avoid using any "flags" unless I am really, really need them. Simplicity is important.

Alex

[Archos]

El martes, 28 de agosto de 2012 03:23:06 UTC+1, brainman escribió:

On Monday, 27 August 2012 18:26:33 UTC+10, Archos wrote:
>
> And we only have to run:
> $ mksyscall $GOROOT/src/pkg/syscall/syscall_windows.go $GOROOT/src/pkg/syscall/security_windows.go

Yeah, that is what I have tried too. And I expect the output of this command

# diff $GOROOT/src/pkg/syscall/zsyscall_windows_386.go <(./mksyscall $GOROOT/src/pkg/syscall/syscall_windows.go $GOROOT/src/pkg/syscall/security_windows.go)

to be empty. But it is not. So, whatever your program does, it is different to what mksyscall_windows.pl does.

Of course that the output is different since both programs work of different way:

+ The Perl version: it takes all files of input and generates *one only file of output*; in addition you have to run it for each architecture (386, amd64), being able to generate the same code for both arhitectures like happens on the next case because the prototypes have not arguments with type int64.:

http://golang.org/src/pkg/syscall/zsyscall_windows_386.go
http://golang.org/src/pkg/syscall/zsyscall_windows_amd64.go

+ The Go version: it takes all files of input and generate *one otput file for each input file*; but it generates the files for the architectures only when it is necessary (because there is some argument whose type is int64)

As I said, the code generated is correct, since it has been tested by compiling Go using the code generated by that tool; if you want, run go test.
 

and wouldn't even worry about any flags, options, features and such until test passes.

I would also use templates to write the output. It would be simpler, and would make it easier to change things quickly when I need to.

I would also avoid using any "flags" unless I am really, really need them. Simplicity is important.

Some flags are set to default values; but mos of people only have to use flag -w to write output to files, and -lazy=false if you don't need to use the LazyDLL function.