Conditional imports

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I was trying to do some profiling of my code using the example from the
very nice blog post. However, pprof isn't available on Windows. This is
known at build time (it gives a "can't find import: pprof" error trying
to compile).

What is an appropriate way to write conditional compile code. Something
that says "if you're on a platform that supports pprof, then import it,
and hook up the machinery to activate it when the flag is supplied".

Is it just having an external module named "usepprof_linux.go" and
changing the Makefile to include "usepprof_$GOOS.go". And then
"usepprof_windows.go" would have the same 'installpprof' function, but
it would just be a no-op there?

As an aside, what is the blocker to having pprof work on Windows? I
would think that the ASM would be pretty similar, but maybe I'm
completely wrong.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4S+LYACgkQJdeBCYSNAAM2vQCfXy0+J+zGfpHDVzklMfLx+sa4
am0AniAsIQNVxfKGwCxIhVtAmwkriWTc
=VnvL
-----END PGP SIGNATURE-----

[David Symonds]

Profiling is, to some degree, inherently system dependent. Having said
that, though, it seems reasonable to implement a stub pprof package on
Windows so the program can at least build.

Dave.

[brainman]

If you mean

import "runtime/pprof",

then, you should be able to build this on windows with no problem. Looking at your error message, make sure you have $GOROOT/pkg/windows_386/runtime/pprof.a present.

StartCPUProfile/StopCPUProfile aren't doing anything useful, because they are not implemented for windows (linux implementation just sends signals to the process to collect CPU statistics, not sure how to do something of that kind on windows). WriteHeapProfile on the other hand should work just fine to generate proper output.

You should be able to run gopprof, if you have mingw installed (with perl). Unfortunately, it doesn't convert addresses into names :-). I suspect nm program doesn't handle windows pe executables.

Alex

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/5/2011 2:14 PM, brainman wrote:
> If you mean
>
> import "runtime/pprof",
>
> then, you should be able to build this on windows with no problem. Looking
> at your error message, make sure you have
> $GOROOT/pkg/windows_386/runtime/pprof.a present.

Yeah, typo on my part. I saw the "pprof.XXX" in the code snippet, and
not the "we have to import runtime/pprof" in the text.

>
> StartCPUProfile/StopCPUProfile aren't doing anything useful, because they
> are not implemented for windows (linux implementation just sends signals to
> the process to collect CPU statistics, not sure how to do something of that
> kind on windows). WriteHeapProfile on the other hand should work just fine
> to generate proper output.

As in, how to use a timer to generate interrupts? Could you forcibly
start a goroutine in another thread (via runtime.LockOSThread()), and
just have it use time.At()? (or time.Sleep() I guess, since that also
forces a thread.)

>
> You should be able to run gopprof, if you have mingw installed (with perl).
> Unfortunately, it doesn't convert addresses into names :-). I suspect nm
> program doesn't handle windows pe executables.
>
> Alex
>

Assuming you mean "6prof" to collect the stats vs "gopprof" (to analyze
them?)
$ 6prof -P test.prof ./main.exe
prof: crack header for ./main.exe: unknown header type

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4TFNgACgkQJdeBCYSNAANKXQCcCKzmm4AN+9ofBRwwDhdXC6Lp
RAUAnivhFggZttOXku1Izs7EOiBzAmmB
=VV0y
-----END PGP SIGNATURE-----

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/5/2011 2:14 PM, brainman wrote:

> If you mean
>
> import "runtime/pprof",
>
> then, you should be able to build this on windows with no problem. Looking
> at your error message, make sure you have
> $GOROOT/pkg/windows_386/runtime/pprof.a present.
>
> StartCPUProfile/StopCPUProfile aren't doing anything useful, because they
> are not implemented for windows (linux implementation just sends signals to
> the process to collect CPU statistics, not sure how to do something of that
> kind on windows). WriteHeapProfile on the other hand should work just fine
> to generate proper output.

After a fair amount of googling, I did come across a couple of
interesting threads. In the end, it looks like you have to have a
HAL-level driver to get the appropriate interrupts. However, it looks
like Intel has already programmed one for you:

http://software.intel.com/sites/products/documentation/hpc/amplifierxe/en-us/lin/ug_docs/olh/common/installing_sep_driver.html
http://www.osronline.com/showthread.cfm?link=186286

Of course, then you might suffer from this bug. The sampling driver
isn't digitally signed, so it defaults to being disabled on boot.
http://software.intel.com/en-us/articles/intel-vtune-performance-analyzer-for-windows-known-issues-on-windows-vista-and-windows-longhorn-server-systems/

Basically, not very easy to do at all, but somehow possible. :)

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4THD8ACgkQJdeBCYSNAAMJoQCgmowHYKM0OE7FDEAUKyUB/JL+
U2cAn169B5U/VKLLLw6Kj4Mj5ysbOW2V
=HFHi
-----END PGP SIGNATURE-----

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/5/2011 4:14 PM, John Arbash Meinel wrote:
> On 7/5/2011 2:14 PM, brainman wrote:
>> If you mean
>
>> import "runtime/pprof",
>
>> then, you should be able to build this on windows with no problem. Looking
>> at your error message, make sure you have
>> $GOROOT/pkg/windows_386/runtime/pprof.a present.
>
>> StartCPUProfile/StopCPUProfile aren't doing anything useful, because they
>> are not implemented for windows (linux implementation just sends signals to
>> the process to collect CPU statistics, not sure how to do something of that
>> kind on windows). WriteHeapProfile on the other hand should work just fine
>> to generate proper output.
>
> After a fair amount of googling, I did come across a couple of
> interesting threads. In the end, it looks like you have to have a
> HAL-level driver to get the appropriate interrupts. However, it looks
> like Intel has already programmed one for you:
>

A bit more digging (actually just reading further on one of those
links), and I came across "Event Tracing for Windows":
http://msdn.microsoft.com/en-us/library/bb968803%28VS.85%29.aspx

I'm still digging through the documentation, and it is a bit confusing
because the system seems to be designed around one application emitting
events that the other one consumes. I'm also trying to figure out if
there are already some providers that would do enough of what you might
want. (EVENT_TRACE_FLAG_PROFILE seems to be part of that.)

It looks like you want something with the NT Kernel Logger Session:
http://msdn.microsoft.com/en-us/library/aa363691%28VS.85%29.aspx

It also looks like you need to write a custom "Consumer" otherwise it
defaults to just logging the trace information to a file.
http://msdn.microsoft.com/en-us/library/aa363692%28VS.85%29.aspx

You also can only do realtime processing (without a log file) if you are
in the Administrators or Performance Log Users group. Though that seems
a reasonable constraint.

The other option is something with ZwCreateProfile, with an example-ish
here:
http://www.ureader.com/msg/14741084.aspx

Though it seems a little tricky to get real information on this. This
PDF looks really interesting, though:
http://read.pudn.com/downloads81/sourcecode/windows/system/315403/11%20Profile.pdf

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4TJvIACgkQJdeBCYSNAAMuMgCfSOhODhv6dbEb0oXnZK5Cm6pD
o9IAn1Yu0DwdKqq0I8bk6pItqRWQ+tef
=kXu2
-----END PGP SIGNATURE-----

[Paulo Pinto]

You seem to be on the right track.

I wonder if it would be possible to use performance counters for the
same goal.

--
Paulo

> defaults to just logging the trace information to a file.http://msdn.microsoft.com/en-us/library/aa363692%28VS.85%29.aspx

>
> You also can only do realtime processing (without a log file) if you are
> in the Administrators or Performance Log Users group. Though that seems
> a reasonable constraint.
>
> The other option is something with ZwCreateProfile, with an example-ish
> here:http://www.ureader.com/msg/14741084.aspx
>
> Though it seems a little tricky to get real information on this. This

> PDF looks really interesting, though:http://read.pudn.com/downloads81/sourcecode/windows/system/315403/11%...

>
> John
> =:->
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Cygwin)

> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

[Gustavo Niemeyer]

> Is it just having an external module named "usepprof_linux.go" and
> changing the Makefile to include "usepprof_$GOOS.go". And then
> "usepprof_windows.go" would have the same 'installpprof' function, but
> it would just be a no-op there?

That's right. There are several examples of that in the standard
library (os, syscall, exec, etc).

--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

[John Meinel]

Perf counters would let you run a busy loop and sample at appropriate times. But to avoid impacting the code under test, you really need a callback.

John
=:->

[brainman]

Thank you for suggestions. Documented (http://code.google.com/p/go/issues/detail?id=2041) for posterity :-)

Alex