release.2011-02-01

[Andrew Gerrand]

We've just tagged a new Go release, release.2011-02-01. As usual, you
can update by running:
hg pull
hg update release

This release includes significant changes to channel operations and minor
changes to the log package. Your code will require modification if it uses
channels in non-blocking communications or the log package's Exit functions.

Non-blocking channel operations have been removed from the language.
The equivalent operations have always been possible using a select statement
with a default clause. If a default clause is present in a select, that clause
will execute (only) if no other is ready, which allows one to avoid blocking on
a communication.

For example, the old non-blocking send operation,

if ch <- v {
// sent
} else {
// not sent
}

should be rewritten as,

select {
case ch <- v:
// sent
default:
// not sent
}

Similarly, this receive,

v, ok := <-ch
if ok {
// received
} else {
// not received
}

should be rewritten as,

select {
case v := <-ch:
// received
default:
// not received
}

This change is a prelude to redefining the 'comma-ok' syntax for a receive.
In a later release, a receive expression will return the received value and an
optional boolean indicating whether the channel has been closed. These changes
are being made in two stages to prevent this semantic change from silently
breaking code that uses 'comma-ok' with receives.
There are no plans to have a boolean expression form for sends.

Sends to a closed channel will panic immediately. Previously, an unspecified
number of sends would fail silently before causing a panic.

The log package's Exit, Exitf, and Exitln functions have been renamed Fatal,
Fatalf, and Fatalln respectively. This brings them in line with the naming of
the testing package.

The port to the "tiny" operating system has been removed. It is unmaintained
and untested. It was a toy to show that Go can run on raw hardware and it
served its purpose. The source code will of course remain in the repository
history, so it could be brought back if needed later.

This release also changes some of the internal structure of the memory
allocator in preparation for other garbage collector changes.
If you run into problems, please let us know.
There is one known issue that we are aware of but have not debugged yet:
http://code.google.com/p/go/issues/detail?id=1464&.

Other changes in this release:
* 5l: document -F, force it on old ARMs (software floating point emulation)
* 6g: fix registerization of temporaries (thanks Eoghan Sherry),
fix uint64(uintptr(unsafe.Pointer(&x))).
* 6l: Relocate CMOV* instructions (thanks Gustavo Niemeyer),
windows/amd64 port (thanks Wei Guangjing).
* 8l: add PE dynexport, emit DWARF in Windows PE, and
code generation fixes (thanks Wei Guangjing).
* bufio: make Flush a no-op when the buffer is empty.
* bytes: Add Buffer.ReadBytes, Buffer.ReadString (thanks Evan Shaw).
* cc: mode to generate go-code for types and variables.
* cgo: define CGO_CFLAGS and CGO_LDFLAGS in Go files (thanks Gustavo Niemeyer),
windows/386 port (thanks Wei Guangjing).
* codereview: fix windows (thanks Hector Chu),
handle file patterns better,
more ASCII vs. Unicode nonsense.
* crypto/dsa: add support for DSA.
* crypto/openpgp: add s2k.
* crypto/rand: use defer to unlock mutex (thanks Anschel Schaffer-Cohen).
* crypto/rsa: correct docstring for SignPKCS1v15.
* crypto: add package, a common place to store identifiers for hash functions.
* doc/codelab/wiki: update to work with template changes, add to run.bash.
* doc/spec: clarify address operators.
* ebnflint: exit with non-zero status on error.
* encoding/base32: new package (thanks Miek Gieben).
* encoding/line: make it an io.Reader too.
* exec: use custom error for LookPath (thanks Gustavo Niemeyer).
* fmt/doc: define width and precision for strings.
* gc: clearer error for struct == struct,
fix send precedence,
handle invalid name in type switch,
special case code for single-op blocking and non-blocking selects.
* go/scanner: fix build (adjust scanner EOF linecount).
* gob: better debugging, commentary,
make nested interfaces work,
report an error when encoding a non-empty struct with no public fields.
* godoc: full text index for whitelisted non-Go files,
show line numbers for non-go files (bug fix).
* gofmt -r: match(...) arguments may be nil; add missing guards.
* govet: add Panic to the list of functions.
* http: add host patterns (thanks Jose Luis Vázquez González),
follow relative redirect in Get.
* json: handle capital floating point exponent (1E100) (thanks Pieter
Droogendijk).
* ld: add -I option to set ELF interpreter,
more robust decoding of reflection type info in generating dwarf.
* lib9: update to Unicode 6.0.0.
* make.bash: stricter selinux test (don't complain unless it is enabled).
* misc/vim: Import/Drop commands (thanks Gustavo Niemeyer),
set 'syntax sync' to a large value (thanks Yasuhiro Matsumoto).
* net: fix race condition in test,
return cname in LookupHost.
* netchan: avoid race condition in test,
fixed documentation for import (thanks Anschel Schaffer-Cohen).
* os: add ETIMEDOUT (thanks Albert Strasheim).
* runtime: generate Go defs for C types,
implementation of callback functions for windows (thanks Alex Brainman),
make Walk web browser example work (thanks Hector Chu),
make select fairer,
prefer fixed stack allocator over general memory allocator,
simpler heap map, memory allocation.
* scanner: fix Position returned by Scan, Pos,
don't read ahead in Init.
* suffixarray: use binary search for both ends of Lookup (thanks Eric Eisner).
* syscall: add missing network interface constants (thanks Mikio Hara).
* template: treat map keys as zero, not non-existent (thanks Roger Peppe).
* time: allow cancelling of After events (thanks Roger Peppe),
support Solaris zoneinfo directory.
* token/position: added SetLinesForContent.
* unicode: update to unicode 6.0.0.
* unsafe: add missing case to doc for Pointer.

Apologies if we missed anyone in the list above. We appreciate all your help.

To see a full list of changes between this and the previous release,
after updating, run:
hg log -r release.2011-01-20:release.2011-02-01.1

Enjoy.

Andrew

[Joseph Poirier]

FYI - the windows release may be delayed due to an intermittent
broadband connection. My service provider says the outages
are because of the extreme weather we're having here in North
Texas. I'll get it uploaded asap.

-joe

[Skip Tavakkolian]

and this:
_ = ch <- v

changed to this, i think:

select {
case ch <- v:
}

-Skip

[Mateusz Czapliński]

On Feb 2, 3:45 am, Andrew Gerrand <a...@google.com> wrote:
> * 8l: add PE dynexport, emit DWARF in Windows PE [...] (thanks Wei Guangjing).
> * cgo: [...]

>         windows/386 port (thanks Wei Guangjing).

> * runtime: [...]

>         implementation of callback functions for windows (thanks Alex Brainman),
>         make Walk web browser example work (thanks Hector Chu),

Wow, as for Windows, that's MAJOR! :)
Big thanks to Alex, Wei, Hector, and all other Win32 magicians! :)

greetings
Mateusz

[Gustavo Niemeyer]

> and this:
>                _ = ch <- v
>
> changed to this, i think:
>
>                select {
>                case ch <- v:
>                 }

No, Andrew described this case:

> should be rewritten as,
>
> select {
> case ch <- v:
> // sent
> default:
> // not sent
> }

Without the default, the select statement becomes blocking.

--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter

[Andrew Gerrand]

Indeed. It's great to see so many contributions from the community.
Thanks again everyone! :-)

Andrew

[Johann Höchtl]

On Feb 2, 3:45 am, Andrew Gerrand <a...@google.com> wrote:

There has been some discussion if base32 warrants an inclusion as a
core package. I recently discovered geohash
http://en.wikipedia.org/wiki/Geohash
where base32 just fits fine; (Nodding towards Gustavo ;) )

[Gustavo Niemeyer]

> There has been some discussion if base32 warrants an inclusion as a
> core package. I recently discovered geohash
> http://en.wikipedia.org/wiki/Geohash
> where base32 just fits fine; (Nodding towards Gustavo ;) )

I thought about this as well when the support was going in. :-)

[⚛]

On Feb 2, 3:45 am, Andrew Gerrand <a...@google.com> wrote:

> This change is a prelude to redefining the 'comma-ok' syntax for a receive.
> In a later release, a receive expression will return the received value and an
> optional boolean indicating whether the channel has been closed. These changes
> are being made in two stages to prevent this semantic change from silently
> breaking code that uses 'comma-ok' with receives.
> There are no plans to have a boolean expression form for sends.
>
> Sends to a closed channel will panic immediately. Previously, an unspecified
> number of sends would fail silently before causing a panic.

I am a little confused by this. The 'close' operation is *not*
required for construction of general-purpose concurrent programs (and
Go has a garbage collector). Why not remove it completely?

[Andrew Gerrand]

On 3 February 2011 23:36, ⚛ <0xe2.0x...@gmail.com> wrote:
> I am a little confused by this. The 'close' operation is *not*
> required for construction of general-purpose concurrent programs (and
> Go has a garbage collector). Why not remove it completely?

Close is a convenience that simplifies sending a finite set of values
down a channel and ranging over it at the receiving end.

The upcoming changes make it possible to detect a channel is closed
only at the receiving end. This should allow us to keep the
convenience of close while removing tricky the handling of race
conditions and confusion created by its misuse.

Andrew

[⚛]

On Feb 3, 1:52 pm, Andrew Gerrand <a...@golang.org> wrote:

> On 3 February 2011 23:36, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>
> > I am a little confused by this. The 'close' operation is *not*
> > required for construction of general-purpose concurrent programs (and
> > Go has a garbage collector). Why not remove it completely?
>
> Close is a convenience that simplifies sending a finite set of values
> down a channel and ranging over it at the receiving end.

Yes. But supposing closing a channel is removed from the language, the
"for ... range channel" construct could still be used. The difference
would be that the programmer would have to explicitly break from the
loop when the last element is detected.

> The upcoming changes make it possible to detect a channel is closed
> only at the receiving end. This should allow us to keep the
> convenience of close while removing tricky the handling of race
> conditions and confusion created by its misuse.
>
> Andrew

In my opinion, the proposed future addition (= a receive expression

will return the received value and an optional boolean indicating

whether the channel has been closed) will leave the tricky race
conditions and confusions due to misuse at approximately the same
level as they are right now. Maybe it will improve the correctness of
some codes by a small margin - but it seems to me the bulk of the
concurrency-related problems will remain as tricky as before. I am
talking about the case of a larger concurrent application (multiple
goroutines which are using channels and selects in a non-trivial way).

In addition, if the Go language wants to support arbitrary coding
patterns with channels, the only way of how it can be done is to
introduce the operations "lock a channel" and "unlock a channel". If
each channel has a lock that is accessible to the programmer, it would
enable the programmer to examine the channel's state in a controlled
manner. Without locks, it is impossible to implement codes such as:

if !closed(ch) { some-actions; close(ch) }

because the access to the channel is not atomic. You have to do:

var chLock sync.Lock
chLock.Lock()
if !closed(ch) { some-actions; close(ch); chLock.Unlock() }
else { chLock.Unlock() }

Which brings me to the following point: the problem is not the fact
that you can send/receive values via a channel, but the problem is
that a channel has some user-visible per-channel *state* at all. If
the Go team wants to remedy the problem entirely, I would suggest to
remove the state. Consequently, there would be no "close(ch)" and no
"if closed(ch) {}". Since the channel would have no state, there
cannot be any concurrency problems with it. From the viewpoint of the
programmer, it would require to send a special value telling the user
code not to use the channel any further - the benefit of this is that
the information "I am closing this channel now" is sent through a
channel and that it is explicit - there is no such thing as a "shared
state" which needs to be locked/unlocked or taken special care of. (I
am *not* claiming it would make programming easy. Programming is hard,
and always will be.)

[Gustavo Niemeyer]

> Without locks, it is impossible to implement codes such as:
>
> if !closed(ch) { some-actions; close(ch) }
>
> because the access to the channel is not atomic. You have to do:

My understanding is that this race is going away entirely because
there will be no more closed() when the new v, ok := <-c
is introduced.

[Andrew Gerrand]

On 4 February 2011 03:13, ⚛ <0xe2.0x...@gmail.com> wrote:
> On Feb 3, 1:52 pm, Andrew Gerrand <a...@golang.org> wrote:
>> On 3 February 2011 23:36, ⚛ <0xe2.0x9a.0...@gmail.com> wrote:
>>
>> > I am a little confused by this. The 'close' operation is *not*
>> > required for construction of general-purpose concurrent programs (and
>> > Go has a garbage collector). Why not remove it completely?
>>
>> Close is a convenience that simplifies sending a finite set of values
>> down a channel and ranging over it at the receiving end.
>
> Yes. But supposing closing a channel is removed from the language, the
> "for ... range channel" construct could still be used. The difference
> would be that the programmer would have to explicitly break from the
> loop when the last element is detected.
>
>> The upcoming changes make it possible to detect a channel is closed
>> only at the receiving end. This should allow us to keep the
>> convenience of close while removing tricky the handling of race
>> conditions and confusion created by its misuse.
>>
>> Andrew
>
> In my opinion, the proposed future addition (= a receive expression
> will return the received value and an optional boolean indicating
> whether the channel has been closed) will leave the tricky race
> conditions and confusions due to misuse at approximately the same
> level as they are right now.

These tricky race conditions are caused by people using closed() on
the sending side to coordinate multiple senders.

When the closed state is available only to the receiving side (by way
of comma-ok) programmers won't be able to use close to coordinate
senders. Instead they will reach for the next most logical option:
additional channels.

Andrew

[Ryanne Dolan]

"Close is a convenience that simplifies sending a finite set of values

down a channel and ranging over it at the receiving end."

I'm sure this has been asked elsewhere, but why not change the semantics of 'range' to stop at a nil value?

Value types don't have a nil value, I realize.  But range could be restricted to work over channels of interfaces and pointers rather easily, since range-ing over a container would typically fall into that category.

In other words, maybe we shouldn't be able to range over 'chan int' at all, but 'chan *int' or 'chan interface{}' would range until a nil was received.

This would eliminate the extra closed state of channels without introducing any extra values.

Thanks.
Ryanne

--
www.ryannedolan.info

[Rob 'Commander' Pike]

Because nil is a fine value to send on a channel. Why take it away?
It seems less magic to me to have a separate close() operator that,
being explicit rather than implicit, is likelier to have heavier
semantics, as indeed it does.

-rob

[Ryanne Dolan]

"Because nil is a fine value to send on a channel.  Why take it away?"

Just to be clear, I don't think you shouldn't be able to send a nil.  You could send nils all day on any channel you want.

The difference would be that range would stop iterating upon receiving a nil.  It would be analogous to fmt.Printf stopping at a null character.  I think this would be simpler semantics than the current closed state and the future v,ok goofiness.

Thanks.
Ryanne

--
www.ryannedolan.info

[Gustavo Niemeyer]

> The difference would be that range would stop iterating upon receiving a
> nil.  It would be analogous to fmt.Printf stopping at a null character.  I

That's an excellent analogy, actually. Have you noticed it doesn't? :-)

[Ryanne Dolan]

Yeah yeah... nor am I saying it should.  Just an analogy.

Thanks.
Ryanne

--
www.ryannedolan.info

[Rich]

Having issues compiling the new release on Macintosh:

quietgcc -I"/usr/local/go/include" -ggdb -O2 -c "/usr/local/go/src/cmd/
6l/optab.c"
quietgcc -I"/usr/local/go/include" -ggdb -O2 -c "/usr/local/go/src/cmd/
6l/pass.c"
quietgcc -I"/usr/local/go/include" -ggdb -O2 -c -I. ../ld/pe.c
quietgcc -I"/usr/local/go/include" -ggdb -O2 -c "/usr/local/go/src/cmd/
6l/prof.c"
quietgcc -I"/usr/local/go/include" -ggdb -O2 -c "/usr/local/go/src/cmd/
6l/span.c"
quietgcc -I"/usr/local/go/include" -ggdb -O2 -c -I. ../ld/symtab.c
quietgcc -o 6l -L"/usr/local/go"/lib asm.o data.o dwarf.o elf.o enam.o
go.o ldelf.o ldmacho.o ldpe.o lib.o list.o macho.o obj.o optab.o
pass.o pe.o prof.o span.o symtab.o -lbio -l9 -lm
ld: symbol dyld_stub_binding_helper not defined (usually in crt1.o/
dylib1.o/bundle1.o)
collect2: ld returned 1 exit status
make: *** [6l] Error 1
rm-macbook:src root# uname -a
Darwin rm-macbook 10.6.0 Darwin Kernel Version 10.6.0: Wed Nov 10
18:13:17 PST 2010; root:xnu-1504.9.26~3/RELEASE_I386 i386

On Feb 3, 1:17 pm, Ryanne Dolan <ryannedo...@gmail.com> wrote:
> Yeah yeah... nor am I saying it should.  Just an analogy.
>
> Thanks.
> Ryanne
>
> --www.ryannedolan.info
>

[ygl]

On Feb 1, 6:45 pm, Andrew Gerrand <a...@google.com> wrote:

> Non-blocking channel operations have been removed from the language.

Maybe it is not related. In "reflect" pkg, there are methods
ChanValue.trySend()/TryRecv(). Will these methods remain in future?

Thanks
yigong

[Russ Cox]

>> Non-blocking channel operations have been removed from the language.
>
> Maybe it is not related. In "reflect" pkg, there are methods
> ChanValue.trySend()/TryRecv(). Will these methods remain in future?

A more precise version of Andrew's statement is that
special syntax for non-blocking channel operations has been
removed. You can still do non-blocking operations via select
statements.

I think it is safe to assume that we won't remove TrySend or
TryRecv from reflect unless we first add a general Select.
And even then, I think we'd leave the special cases, because
when you have to spell everything out using function calls
(as opposed to custom syntax like the select statement)
writing out trivial one-channel selects can get very tedious.

Russ

[⚛]

close() on a channel is analogous to free() on a memory location. Go
does not have free(), one reason is that it is error prone and complex
when sharing memory across multiple goroutines. Applying the same
reasoning to channels: Go should not have close().

Shouldn't close() be removed and handled automatically due to the same
reasons free() was removed and is being handled automatically?

If we can very easily forget about free() when dealing with memory,
*what* is preventing us from forgetting about close()?

[Gustavo Niemeyer]

> Shouldn't close() be removed and handled automatically due to the same
> reasons free() was removed and is being handled automatically?

The use of free() is enforced for all memory allocated, even though it has no
use besides giving resources back to the runtime/system for reuse. This
can be done automatically without altering the semantics of the application.
close() is optional, and its intention is altering the semantics of channel use.

[Andrew Gerrand]

On 4 February 2011 20:08, ⚛ <0xe2.0x...@gmail.com> wrote:
> close() on a channel is analogous to free() on a memory location. Go
> does not have free(), one reason is that it is error prone and complex
> when sharing memory across multiple goroutines. Applying the same
> reasoning to channels: Go should not have close().
>
> Shouldn't close() be removed and handled automatically due to the same
> reasons free() was removed and is being handled automatically?
>
> If we can very easily forget about free() when dealing with memory,
> *what* is preventing us from forgetting about close()?

This analogy is (very) faulty.

If you fail to free() memory you get a memory leak.

You only need to close a channel if your algorithm requires it. And if
you do require close, you would notice straight away if you forgot to
use it.

Close is very useful in a few circumstances, but it isn't necessary in
*most* channel usage scenarios.

Andrew