SYSCALL vs CGO

[Luke Mauldin]

If I need to call a native C library, it looks like I have two options: SYSCALL and CGO.  On the surface, it looks like SYSCALL is a bit easier to use.  Is one of the solutions expected to be substantially faster than the other?  Are there other advantages/disadvantages that I am not seeing?  

Luke

[bryanturley]

On Wednesday, November 14, 2012 9:02:49 AM UTC-6, Luke Mauldin wrote:

If I need to call a native C library, it looks like I have two options: SYSCALL and CGO.  On the surface, it looks like SYSCALL is a bit easier to use.  Is one of the solutions expected to be substantially faster than the other?  Are there other advantages/disadvantages that I am not seeing?  

Luke

syscall talks to the kernel not native c libraries.
libc has some syscall wrappers which is why it might be confusing you.

You could cgo the libc syscall wrappers but you can not syscall say a zlib function.

[Luke Mauldin]

Is that restriction only on Linux?  I used SYSCALL in the ODBC driver I wrote: https://github.com/LukeMauldin/lodbc/  I used SYSCALL to call the odbc32.dll to call functions and it worked well.

Luke

[Aram Hăvărneanu]

Syscall is for doing system calls -- communicating with the kernel.
You can't use it to interface an arbitrary C library (except on Windows).

--
Aram Hăvărneanu

[Aram Hăvărneanu]

On windows, the syscall package doesn't actually do or call any system
calls (since most entry points are undocumented, albeit stable), it
just loads the Win32 library and everything is stacked on that. It has
a generic way of loading shared libraries so it can be used to load
any other shared library you want. On other systems, the mechanism is
different.

--
Aram Hăvărneanu

[bryanturley]

On Wednesday, November 14, 2012 9:24:37 AM UTC-6, Luke Mauldin wrote:

Is that restriction only on Linux?  I used SYSCALL in the ODBC driver I wrote: https://github.com/LukeMauldin/lodbc/  I used SYSCALL to call the odbc32.dll to call functions and it worked well.

Well it is more of Microsoft deciding to make software engineering more confusing.
If I am remembering correctly (doubt it) in windows there is a SYSCALL c macro/builtin that changes the way parameters are put onto the stack/reqs when calling that function.
But I almost never write windows code so don't trust me on that ;)

This is what syscall means in go and the unix world http://en.wikipedia.org/wiki/Syscall
You should avoid using the syscall package directly if you can, as your code won't be cross os compatible.

The best/supported way to call c libraries is cgo.

[bryanturley]

If you used the go syscall package to talk to odbc32.dll it must be a kernel component in windows.

mysql.so or mysql_client.so in unix would not be a kernel component you would need cgo to talk to it.

[Aram Hăvărneanu]

> If you used the go syscall package to talk to odbc32.dll it must be a kernel
> component in windows.

NO!

Stop spreading this FUD.

--
Aram Hăvărneanu

[bryanturley]

On Wednesday, November 14, 2012 12:13:42 PM UTC-6, Aram Hăvărneanu wrote:

> If you used the go syscall package to talk to odbc32.dll it must be a kernel
> component in windows.

NO!

Stop spreading this FUD.

Eh... the sentence did start with *IF*
To quote myself from this thread "But I almost never write windows code so don't trust me on that ;)"
Definitely FUD...

Does go's syscall package call non system calls (native api or whatever they call it) in windows? 
Including database drivers?
If your referring to my obvious dislike of MS products that would be accurate ;)

[bryanturley]

On Wednesday, November 14, 2012 11:22:17 AM UTC-6, bryanturley wrote:

On Wednesday, November 14, 2012 9:24:37 AM UTC-6, Luke Mauldin wrote:

Is that restriction only on Linux?  I used SYSCALL in the ODBC driver I wrote: https://github.com/LukeMauldin/lodbc/  I used SYSCALL to call the odbc32.dll to call functions and it worked well.

Well it is more of Microsoft deciding to make software engineering more confusing.
If I am remembering correctly (doubt it) in windows there is a SYSCALL c macro/builtin that changes the way parameters are put onto the stack/reqs when calling that function.
But I almost never write windows code so don't trust me on that ;)

This is what I was remembering from mingw's   windef.h, luckily I have never used MSVC
Replace SYSCALL in my above sentence with WINAPI

#define STDCALL __stdcall
#define FASTCALL __fastcall
#define WINAPI __stdcall
#define WINAPIV __cdecl

But this is off topic so...

[Aram Hăvărneanu]

> Eh... the sentence did start with *IF*

There's no question of if, this is what the author does. He even
provided a link for it:
https://github.com/LukeMauldin/lodbc/blob/master/odbc/apisys.go

> Does go's syscall package call non system calls (native api or whatever they
> call it) in windows?

The syscall package calls the regular Win32 API. I already said that.
If you don't know how the syscall package works under Windows, why do
you offer advice about it? Your inferences are based on a purely
speculative model (that happens to be false), for the Windows and
syscall package architecture.

System calls on Windows on IA-32(E) are done just like on any other
operating system, either via an interrupt (2E on Nt) or via
SYSENTER/SYSCALL. Unlike many other operating systems, this interface
is undocumented. There is an (almost) bijective mapping between system
calls and the user-level entry points exported in ntdll.dll. This
interface is only partially documented. The Win32 subsystem is
implemented as a user-level process on top of ntdll.dll and exports a
documented interface from the unfortunately named kernel32.dll shared
library.

Since the only documented interface to the system is a shared library,
no wonder the syscall package uses it, and since any user-level,
system provided, shared library is in no way different from an
arbitrary shared library, it shouldn't be a surprise there's a generic
mechanism available in the syscall package that allows you to use any
DLL.

--
Aram Hăvărneanu

[Harley Laue]

To quote an earlier post:

I'll elaborate on this a bit. I'll first suggest giving a quick look to these two files:
https://code.google.com/p/go/source/browse/src/pkg/syscall/dll_windows.go
https://code.google.com/p/go/source/browse/src/pkg/syscall/zsyscall_windows_386.go

This is in stark contrast to Linux (and by the looks of it other UNIX systems as well) that calls the syscall directly from assembly:
https://code.google.com/p/go/source/browse/src/pkg/syscall/asm_linux_386.s

The first link for Windows is where the DLL (any DLL) is loaded and the second is where it actually pieces that together for the standard syscalls available on Windows. I think this is likely due to the fact that Windows, while it has standard kernel functions, are seemingly spread out over several DLL's. They just happen to expose the generic DLL loading functionality for Windows. Most POSIX systems have a pretty centralized place for syscalls, typically the kernel. ;)

This behavior on Windows, IMO, isn't actually great and poorly documented to say the least. Though, I can admit that I could see why this might be needed to lessen the maintenance required to keep the Windows syscall list up-to-date with new releases. As an example of this point, say Windows 8 added X, Y, & Z in DLL XYZ.dll that wasn't known previous known at the Go 1.0 release. If Go didn't load Windows "syscalls" this way, XYZ.dll would have to be bound with CGo until a new Go release could be made to update the Windows syscall list. Thus, Go 1.0.0 will be able to load any extra syscalls that may have been added to Windows if you know the function and DLL that it's in.

I'm sure if someone really wanted to, they could do a elf loader in Go to be able to call into external shared libraries. I'll note that Go already has an elf loader in debug/elf, I haven't looked at it close enough to know if it's just reading the symbols or if it can actually load the symbols into memory to call into (though my initial guess is the former.)

[minux]

On Thu, Nov 15, 2012 at 11:47 PM, Harley Laue <losingge...@gmail.com> wrote:

I'm sure if someone really wanted to, they could do a elf loader in Go to be able to call into external shared libraries. I'll note that Go already has an elf loader in debug/elf, I haven't looked at it close enough to know if it's just reading the symbols or if it can actually load the symbols into memory to call into (though my initial guess is the former.) 

you can always write a cgo binding for rtld (dlopen, dlsym, dlclose, etc), and then Go programs

on unix can load and call functions in shared libraries just like windows.

IIRC, there is even a libffi binding for Go to facilitate calling foreign functions dynamically.

[bryanturley]

On Thursday, November 15, 2012 9:39:07 AM UTC-6, Aram Hăvărneanu wrote:

The syscall package calls the regular Win32 API. I already said that.
If you don't know how the syscall package works under Windows, why do
you offer advice about it? Your inferences are based on a purely
speculative model (that happens to be false), for the Windows and
syscall package architecture.

The inference was that syscall is not used to load random libraries and make calls into them actually.
And this mailing list is slow your post hadn't appeared for me yet when I replied.
But thanks for taking out all that tension and anger you get from windows coding on me.

It seems the real problem here is that windows is once again a miserable broken place to write code and should be avoided whenever possible.

[Harley Laue]

dl{open,sym,close,etc}: Yeah, that's another way to go about it. I guess my first thought was that a pure Go implementation might be cool :)
libffi: I didn't even consider that one. I think that's actually a good idea if you require this sort of functionality.

[Sebastien Binet]

it's over there:
https://github.com/sbinet/go-ffi/tree/master/pkg/dl
https://github.com/sbinet/go-ffi/tree/master/pkg/ffi

the "doc":
https://github.com/sbinet/go-ffi

hth,
-s

[chris dollin]

On 15 November 2012 15:59, bryanturley <bryan...@gmail.com> wrote:

> But thanks for taking out all that tension and anger you get from windows
> coding on me.

Please do not snark like that. It manages to be all of impolite,
insulting, and condescending, on the basis of no real evidence.

Were you wanting to turn them from their current choice of
development system to one you approve of, I assure you that
being gracious is more likely to work than being unpleasant.

Chris

--
Chris "allusive" Dollin

[bryanturley]

Maybe read the thread again?

I was ignorant that syscall is used in windows as a dlopen which seems wrong to me but it is windows so... whatever wrong is par for the course.
But I was accused of "spreading FUD" which is BS.
If you are going to wrongly accuse me of something I am going to defend myself. PERIOD
No worries though I am done with this thread.

[chris dollin]

On 15 November 2012 19:57, bryanturley <bryan...@gmail.com> wrote:

> But I was accused of "spreading FUD" which is BS.
> If you are going to wrongly accuse me of something I am going to defend
> myself. PERIOD

I made no claim that you should not defend yourself.
I asked you not to be rude.

> No worries though I am done with this thread.

Politeness is not thread-specific.