weekly.2011-04-13

[Andrew Gerrand]

We've just tagged a new Go release, weekly.2011-04-13. As usual, you
can update by running:
hg pull
hg update weekly

This weekly snapshot includes major changes to the reflect package and
the os.Open function. Code that uses reflect or os.Open will require
updating, which can be done mechanically using the gofix tool.

The reflect package's Type and Value types have changed. Type is now
an interface that implements all the possible type methods. Instead
of a type switch on a reflect.Type t, switch on t.Kind(). Value is
now a struct value that implements all the possible value methods.
Instead of a type switch on a reflect.Value v, switch on v.Kind().
See the change for the full details:
http://code.google.com/p/go/source/detail?r=843855f3c026

The os package's Open function has been replaced by three functions:
OpenFile(name, flag, perm) // same as old Open
Open(name) // same as old Open(name, O_RDONLY, 0)
Create(name) // same as old Open(name, O_RDWR|O_TRUNC|O_CREAT, 0666)

To update your code to use the new APIs, run "gofix path/to/code".
Gofix can’t handle all situations perfectly, so read and test the
changes it makes before committing them.

Other changes:
* archive/zip: add func OpenReader, type ReadCloser (thanks Dmitry Chestnykh).
* asn1: Implement correct marshaling of length octets (thanks Luit van
Drongelen).
* big: don't crash when printing nil ints.
* bufio: add ReadLine, to replace encoding/line.
* build: make the build faster, quieter.
* codereview: automatically port old diffs forward,
drop Author: line on self-clpatch,
recognize code URL without trailing slash.
* crypto/block: remove deprecated package.
* crypto/des: new package implementating DES and TDEA (thanks Yasuhiro
Matsumoto).
* crypto/ecdsa, crypto/rsa: use io.ReadFull to read from random source
(thanks Dmitry Chestnykh).
* crypto/rsa: add 3-prime support,
add support for precomputing CRT values,
flip the CRT code over so that it matches PKCS#1.
* crypto/x509: expose complete DER data (thanks Mikkel Krautz).
* doc: new "Functions" codewalk (thanks John DeNero).
* doc/roadmap: add sections on tools, packages.
* fmt: allow %U for unsigned integers.
* gc: fixes and optimizations.
* go/printer, gofmt: use blank to separate import rename from import path.
* go/scanner: better TokenString output.
* go/types: new Go type hierarchy implementation for AST.
* godashboard: show packages at launchpad.net (thanks Gustavo Niemeyer).
* gofix: add -diff, various fixes and helpers.
* gotest: fix a bug in error handling,
fixes for [^.]_test file pattern (thanks Peter Mundy),
handle \r\n returned by gomake on Windows (thanks Alex Brainman).
* gotype: use go/types GcImporter.
* govet: make name-matching for printf etc. case-insensitive.
* http: allow override of Content-Type for ServeFile,
client gzip support,
do not listen on 0.0.0.0 during test,
flesh out server Expect handling + tests.
* image/ycbcr: new package.
* image: allow "?" wildcards when registering image formats.
* io: fixes for Read with n > 0, os.EOF (thanks Robert Hencke).
* ld: correct Plan 9 compiler warnings (thanks Lucio De Re),
ELF header function declarations (thanks Lucio De Re),
fix Mach-O X86_64_RELOC_SIGNED relocations (thanks Mikkel Krautz),
fix Mach-O bss bug (thanks Mikkel Krautz),
fix dwarf decoding of strings for struct's fieldnames (thanks
Luuk van Dijk),
fixes and optimizations (25% faster).
* log: generalize getting and setting flags and prefix.
* misc/cgo/life: enable build and test on Windows (thanks Alex Brainman).
* misc/vim: add plugin with Fmt command (thanks Dmitry Chestnykh),
update type highlighting for new reflect package.
* net: disable multicast tests by default (thanks Dave Cheney),
sort records returned by LookupMX (thanks Corey Thomasson).
* openpgp: Fix improper := shadowing (thanks Gustavo Niemeyer).
* os: rename Open to OpenFile, add new Open, Create,
fix Readdir in Plan 9 (thanks Fazlul Shahriar).
* os/inotify: use _test for test files, not _obj.
* pkg/path: enable tests on Windows (thanks Alex Brainman).
* reflect: new Type and Value API.
* src/pkg/Makefile: trim per-directory make output except on failure.
* syscall: Add DT_* and MADV_* constants on Linux (thanks Albert Strasheim),
add Mmap, Munmap on Linux, FreeBSD, OS X,
fix StartProcess in Plan 9 (thanks Fazlul Shahriar),
fix Windows Signaled (thanks Alex Brainman).
* test/bench: enable build and test on Windows (thanks Alex Brainman).

Apologies if we missed anyone in the list above. We appreciate all your help.

To see a full list of changes between this and the previous release,
after updating, run:
hg log -r weekly.2011-04--04:weekly.2011-04-13

Enjoy.

Andrew

[⚛]

I am unable to compile it with GCC 4.6:

src/lib9/create.c:40:25: error: variable 'rdwr'
set but not used [-Werror=unused-but-set-variable]

cc1: all warnings being treated as errors

[Andrew Gerrand]

I just created this CL:

http://codereview.appspot.com/4395044

If you patch it in, does it fix the build for you?

Andrew

[Andrew Gerrand]

On 14 April 2011 14:21, Andrew Gerrand <a...@google.com> wrote:
> The os package's Open function has been replaced by three functions:
>        OpenFile(name, flag, perm) // same as old Open
>        Open(name) // same as old Open(name, O_RDONLY, 0)
>        Create(name) // same as old Open(name, O_RDWR|O_TRUNC|O_CREAT, 0666)

Omitted from these notes is another notable change: the O_CREAT flag
has been removed in favor of the correctly-spelled O_CREATE.

Hooray!

Andrew

[Rob 'Commander' Pike]

It's what ken always wanted.

-rob

[⚛]

The patch works, but there are additional errors of the same type:

src/libmach/executable.c:994:11: error: variable 'swab' set but not
used [-Werror=unused-but-set-variable]

I temporarily "solved" the problem by modifying the file quietgcc.bash
on my computer.

(I would patch the C files myself and send the patch to the Go team,
but unfortunately the historical record shows that this option would
not work ...)

On Apr 14, 6:05 am, Andrew Gerrand <a...@golang.org> wrote:
> I just created this CL:
>
> http://codereview.appspot.com/4395044
>
> If you patch it in, does it fix the build for you?
>
> Andrew
>

[Andrew Gerrand]

On 14 April 2011 17:26, ⚛ <0xe2.0x...@gmail.com> wrote:
> The patch works, but there are additional errors of the same type:
>
> src/libmach/executable.c:994:11: error: variable 'swab' set but not
> used [-Werror=unused-but-set-variable]
>
> I temporarily "solved" the problem by modifying the file quietgcc.bash
> on my computer.
>
> (I would patch the C files myself and send the patch to the Go team,
> but unfortunately the historical record shows that this option would
> not work ...)

If you're not able to sign the CLA, then perhaps you could just
post the list of warnings?

Andrew

[peterGo]

Are these the warnings that you encountered?

GCC warnings - golang-dev
http://groups.google.com/group/golang-dev/browse_thread/thread/9b8dc063550e7763

Peter

[⚛]

Yes.

On Apr 14, 11:55 am, peterGo <go.peter...@gmail.com> wrote:
> Are these the warnings that you encountered?
>

> GCC warnings - golang-devhttp://groups.google.com/group/golang-dev/browse_thread/thread/9b8dc0...
>
> Peter

[Steven]

On Thursday, April 14, 2011, Andrew Gerrand <a...@google.com> wrote:

> The reflect package's Type and Value types have changed.  Type is now
> an interface that implements all the possible type methods.  Instead
> of a type switch on a reflect.Type t, switch on t.Kind().  Value is
> now a struct value that implements all the possible value methods.
> Instead of a type switch on a reflect.Value v, switch on v.Kind().
> See the change for the full details:
>        http://code.google.com/p/go/source/detail?r=843855f3c026

I feel a little uncomfortable with the reduced compile time checking.
Before you had to state what you thought the value was and then try to
call a method. If there was a mismatch, the compiler would stop you.
You could define functions that accepted only specific kinds of types.
And you could easily see only the functions relevant to your kind of
type. What were the developers', particularly Russ's, thoughts on
these issues when the API decision was made?

[rh]

In the link you quoted, there is a link to the code review which discusses what you are looking for.

[⚛]

Some questions which are similar to yours, including Russ's answers:
http://groups.google.com/group/golang-dev/browse_thread/thread/cc0a55ce898fdaa1

[Rob 'Commander' Pike]

The changes are almost entirely driven by a need to reduce the allocations required by the reflect library to improve performance.

-rob

[Paulo Pinto]

Is there a Windows build already available?

[peterGo]

Paulo,

about gomingw - golang-nuts
http://groups.google.com/group/golang-nuts/browse_thread/thread/e50b3cadcf8e09a0

Peter

[ziutek]

> Instead of a type switch on a reflect.Type t, switch on t.Kind().

Is it safe to check that value is any signed Int this way?

vk := v.Kind()
if vk >= reflect.Int && vk <= reflect.Int64 {
// IntXX
}

Is it safe to check that value is a number this way?

if vk >= reflect.Int && vk <= reflect.Complex128 {
// Any number: IntXX, FloatXX, ComplexXX
}

[Russ Cox]

> Is it safe to check that value is any signed Int this way?
>
>  vk := v.Kind()
>  if vk >= reflect.Int && vk <= reflect.Int64 {
>   // IntXX
>  }

It's probably not a great idea.
I've been thinking about adding a few methods to Kind for
IsInt
IsUint
IsFloat
IsComplex

> Is it safe to check that value is a number this way?
>
>  if vk >= reflect.Int && vk <= reflect.Complex128 {
>    // Any number: IntXX, FloatXX, ComplexXX
>  }

Why would this be useful? In order to get or set
the value you need to know which class it is so
you can call the right methods.

Russ

[ziutek]

> > Is it safe to check that value is a number this way?
>
> >  if vk >= reflect.Int && vk <= reflect.Complex128 {
> >    // Any number: IntXX, FloatXX, ComplexXX
> >  }
>
> Why would this be useful?  In order to get or set
> the value you need to know which class it is so
> you can call the right methods.

In my code I have something like this:

func parse2(par interface{}) interface{} {
switch pv := par.(type) {
case *reflect.IntValue, *reflect.FloatValue:
// Do nothing
return par

case []Element:
// Do something with pv to obtain result
return result
}
panic("Unknown type")
}

[ziutek]

We have universal method which works with any IntXX:

func (v Value) Int() int64

It'll be useful to have something universal counterpart to Kind, maybe
something like this:

func (v Value) Genre() Genre

type Genre uint8
const (
Invalid Genre = iota
GenreBool
GenreInt
GenreUint
GenreUintptr
GenreFloat
GenreComplex
GenreArray
//...
)

Then we can build more efficient switch:

switch v.Genre() {
case GenreInt:
// Use Int() method if it is enough.
a := v.Int()

// or switch on Kind if you need specific type.
switch v.Kind() {
case Int8:
b := v.Interface().(int8)
case Int16:
c := v.Interface().(int16)
//...
}

case GenreFloat:
//...
}

Currently we must use flat switch which need O(n) comparisons.

[Russ Cox]

If you write

case Int, Int8, Int16, Int32, Int64:

then there's no reason the compiler can't do
that just as or more efficiently than the
range you asked about before, without
having to add a second variant of Kind.

Russ

[ziutek]

If I write:

switch v.Kind() {
case Uint, Uint8, Uint16, Uint32, Uint64:
a := v.Int()
//..

case Int, Int8, Int16, Int32, Int64:

a := v.Uint()
//..

case Float32, Float64:
a := v.Float()
//..
}

I believe that complier will generate code with 12 comparisons. If
probability of any kind is equal, then there will be average 6
comparisons for any value passed to this switch.

If I write:

switch v.Genre() {
case GenreUint:
a := v.Int()
//..

case GenreInt:
a := v.Uint()
//..

case GenreFloat:
a := v.Float()
//..
}

compiler will generate code with 3 comparisons and then there will be
average 1.5 comparisons for any value passed to this switch.

1.5 < 6

[ziutek]

Sorry for v.Uint <-> v.Int exchange.

[Steven]

Just a thought. What if kind was defined something like this? Then you could just switch on v.Kind() & reflect.MaskSize

type Kind uint8

const (

    Invalid Kind = iota

    Bool

    Int

    Uint

    Uintptr

    Float

    Complex

    Array

    Chan

    Func

    Interface

    Map

    Ptr

    Slice

    String

    Struct

    UnsafePointer

)

const MaskSize Kind = ^Kind(07 << 5)

const (

Int8 Kind = Int | (iota+1) << 5

Int16

Int32 

Int64 

)

const (

Uint8 Kind = Uint | (iota+1) << 5

Uint16

Uint32

Uint64

)

const (

Float32 Kind = Float | (iota+1) << 5

Float64

)

const (

Complex64 Kind = Complex | (iota+1) << 5

Complex128

)

[ziutek]

I think that any definition that permit group match will be better
than unstructured consts.

If definition will permit using a mask for check that some Kind is
Int, Uint, Int/Uint, Float or Complex it will be very god for me.
Using comparison operators with current definition also have their
advantages but only if current order will be preserved in future.

[⚛]

I would like to confirm that the changeset "8036:4fb77130a7ef"
entitled "8g: optimize byte mov" seems to work pretty well.

[Russ Cox]

On Fri, Apr 15, 2011 at 14:48, ⚛ <0xe2.0x...@gmail.com> wrote:
> I would like to confirm that the changeset "8036:4fb77130a7ef"
> entitled "8g: optimize byte mov" seems to work pretty well.

Thanks for the help.

Russ

[ziutek]

> Why would this be useful? In order to get or set
> the value you need to know which class it is so
> you can call the right methods.

The next real example from my Kasia package. Function which
determining if some value should be treat as boolean false or as true:

func getBool(val reflect.Value) bool {
switch val.Kind() {
case reflect.Invalid:
return false
case reflect.Array, reflect.Slice:
return val.Len() != 0
case reflect.Bool:
return val.Bool()
case reflect.Chan:
return !val.IsNil()
case reflect.Complex64, reflect.Complex128:
return val.Complex() != complex(0, 0)
case reflect.Float32, reflect.Float64:
return val.Float() != 0.0
case reflect.Func:
return !val.IsNil()
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32,
reflect.Int64:
return val.Int() != 0
case reflect.Interface:
return !val.IsNil()
case reflect.Map:
return val.Len() != 0
case reflect.Ptr:
return !val.IsNil()
case reflect.String:
return len(val.String()) != 0
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32,
reflect.Uint64:
return val.Uint() != 0
}
return true
}

[Chip Camden]

Quoth ziutek on Friday, 15 April 2011:

Forgive my impertinence, but I have to question the wisdom of wanting to
treat all these types as boolean. Surely you have to know anyway, for
each type, the conditions under which it will be considered "true".

I think there's a good reason why the Go language itself doesn't support
automatic conversion to boolean. It has been one of the more insipid pitfalls
for other languages, especially JavaScript and PHP.

--
.O. | Sterling (Chip) Camden | http://camdensoftware.com
..O | ster...@camdensoftware.com | http://chipsquips.com
OOO | 2048R/D6DBAF91 | http://chipstips.com

[Rob 'Commander' Pike]

const bool FALSE = "false"; // legal, wrong.

-rob

[ziutek]

> Forgive my impertinence, but I have to question the wisdom of wanting to
> treat all these types as boolean.  Surely you have to know anyway, for
> each type, the conditions under which it will be considered "true".
>
> I think there's a good reason why the Go language itself doesn't support
> automatic conversion to boolean.  It has been one of the more insipid pitfalls
> for other languages, especially JavaScript and PHP.

I am happy with the fact that Go doesn't do automatic conversions. But
if you want to implement JavaScript, PHP, Python in Go you should do
that whether you like this convention or not.

If you want to discuss whether it is needed in Kasia templates, you're
welcome.

[Chip Camden]

Quoth Rob 'Commander' Pike on Friday, 15 April 2011:

> const bool FALSE = "false"; // legal, wrong.
>
> -rob

It gets even worse in PHP (as most things do):

if (strpos("xyzzy", "x")) // Fails, because 0 is false
if (strpos("xyzzy", "x") != false) // Still fails, because 0 == false
if (strpos("xyzzy", "x") !== false) // Simon says... now it works

Automatic conversions are supposed to be a convenience, but sometimes they
end up being a source of confusion instead.

[Steven]

[Paulo Pinto]

Thanks

On Apr 15, 3:11 pm, peterGo <go.peter...@gmail.com> wrote:
> Paulo,
>
> about gomingw - golang-nutshttp://groups.google.com/group/golang-nuts/browse_thread/thread/e50b3...

[Eleanor McHugh]

On 14 Apr 2011, at 05:21, Andrew Gerrand wrote:
> The reflect package's Type and Value types have changed. Type is now
> an interface that implements all the possible type methods. Instead
> of a type switch on a reflect.Type t, switch on t.Kind(). Value is
> now a struct value that implements all the possible value methods.
> Instead of a type switch on a reflect.Value v, switch on v.Kind().
> See the change for the full details:
> http://code.google.com/p/go/source/detail?r=843855f3c026

Had I not been without internet the last couple of days I could have incorporated these changes into the Golang tutorial I've just given at ACCU in the UK. Guess I'll have to update the code in the reflection section over the weekend so that people playing along offline can get it to work :)

Ellie

Eleanor McHugh
Games With Brains
http://feyeleanor.tel
----
if _, ok := reality.(Reasonable); !ok { panic(reality) }

[Eleanor McHugh]

On 14 Apr 2011, at 08:04, Rob 'Commander' Pike wrote:
> On Apr 13, 2011, at 11:55 PM, Andrew Gerrand wrote:

>> On 14 April 2011 14:21, Andrew Gerrand <a...@google.com> wrote:
>>> The os package's Open function has been replaced by three functions:
>>> OpenFile(name, flag, perm) // same as old Open
>>> Open(name) // same as old Open(name, O_RDONLY, 0)
>>> Create(name) // same as old Open(name, O_RDWR|O_TRUNC|O_CREAT, 0666)
>>

>> Omitted from these notes is another notable change: the O_CREAT flag
>> has been removed in favor of the correctly-spelled O_CREATE.
>
> It's what ken always wanted.

He's not the only one :)

[rma...@gmail.com]

I had a difficult time getting this to compile.  I had this error:

go/src/lib9/create.c:40:25: error: variable 'rdwr' set but not used [-Werror=unused-but-set-variable]

cc1: all warnings being treated as errors

So I put the changes to go/src/lib9/create.c as suggested but that only moves the problem to:

go/src/libmach/executable.c:994:11: error: variable 'swab' set but not used [-Werror=unused-but-set-variable]

cc1: all warnings being treated as errors

Someone posted above that they fixed this by editing the quietgcc.bash file, for those who didn't get what to change in that file, on line 38 of quietgcc.sh you'll see this line:

-Werror \

Delete this line and run make.bash again.  Could someone verify that this was a change that isn't going to give me problems later on down the line?

[peterGo]

Did you update to the tip to get the latest fixes

$ hg pull -u

and see if you still had your problem?

Peter