Repeating the billion dollar mistake?

1719 views
Skip to first unread message

Sebastian Sylvan

unread,
Nov 13, 2009, 6:52:43 AM11/13/09
to golang-nuts
On slide 14 of the Go Talk the first design principle is "Keep
concepts orthogonal", and yet Go violates this principle by repeating
Hoare's "Billion Dollar Mistake" (
http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare
) by combining the two orthogonal concepts of "reference to" and
"nullable" into a single construct.

I know many language designers have said in retrospect that they
regret getting it wrong (e.g. several from the C# team), so if you're
going to make a new language why not do it right this time and let
these two features be two separate features like they're supposed to
be?

Ian Lance Taylor

unread,
Nov 13, 2009, 11:33:43 AM11/13/09
to Sebastian Sylvan, golang-nuts
Go doesn't have references.

Pointers are not references, as can be seen by the fact that special
syntax is required to access the value that a pointer points to.

Ian

Sebastian Sylvan

unread,
Nov 13, 2009, 1:15:51 PM11/13/09
to Ian Lance Taylor, golang-nuts
That's an entirely irrelevant distinction. The word "reference" is used in my post (and in Hoare's talk) to refer to all kinds of reference constructs, pointers included.

The point is that there's no reason to take two orthogonal concepts ("reference/pointer to" and "nullable") and slap together in one construct (producing one of the most common sources of bugs in the process - a billion dollar's worth if you believe Hoare) when we know perfectly well how to avoid it (see e.g. Eiffel's Void safety).

There should be two orthogonal concepts. One which is "reference to" (a pointer), and one of which is "may be null". You can have a reference that's not nullable (i.e. it's statically known to never ever be null, which means you don't have to check it), and you can have regular values that are nullable, and indeed you can have nullable references (which is the only thing Go has today).



--
Sebastian Sylvan

Peter Froehlich

unread,
Nov 13, 2009, 1:27:32 PM11/13/09
to Ian Lance Taylor, Sebastian Sylvan, golang-nuts
Hi all,
This is terminology confusion at work (now a Go person commits the sin
of thinking in C++ or Java I guess?). Reference = Pointer in the Hoare
talk, there's no difference. I like that you call C's NULL "nil" in
Go, another piece of Oberon at work (it's NIL in Oberon). :-D

Cheers,
Peter
--
Peter H. Froehlich <http://www.cs.jhu.edu/~phf/>
Senior Lecturer | Director, Johns Hopkins Gaming Lab

levi

unread,
Nov 13, 2009, 2:29:25 PM11/13/09
to golang-nuts
On Nov 13, 5:33 pm, Ian Lance Taylor <i...@google.com> wrote:
> Sebastian Sylvan <sebastian.syl...@gmail.com> writes:
> > On slide 14 of the Go Talk the first design principle is "Keep
> > concepts orthogonal", and yet Go violates this principle by repeating
> > Hoare's "Billion Dollar Mistake" (
> >http://www.infoq.com/presentations/Null-References-The-Billion-Dollar...
> > ) by combining the two orthogonal concepts of "reference to" and
> > "nullable" into a single construct.
>
> > I know many language designers have said in retrospect that they
> > regret getting it wrong (e.g. several from the C# team), so if you're
> > going to make a new language why not do it right this time and let
> > these two features be two separate features like they're supposed to
> > be?
>
> Go doesn't have references.
>
> Pointers are not references, as can be seen by the fact that special
> syntax is required to access the value that a pointer points to.

Well, its not only pointers. Function values can be nil and map values
can be nil as well. Or channel values. And so on. The OP is right,
Hoare's mistake is indeed repeated.

-Levi

Ian Lance Taylor

unread,
Nov 13, 2009, 2:48:39 PM11/13/09
to Sebastian Sylvan, golang-nuts
Sorry for my terminology confusion.

Go doesn't have nullable types in general. We haven't seen a real
desire for them.

I personally think that in a systems programming language, a pointer
is a reasonable concept: it intuitively represents a memory address.
I don't personally think that permitting pointers to be nil is a
billion dollar mistake. In my C/C++ programming I've never noticed
that NULL pointers are a noticeable source of bugs.

Ian

Sebastian Sylvan

unread,
Nov 13, 2009, 3:00:02 PM11/13/09
to Ian Lance Taylor, golang-nuts
A pointer is a reasonable concept, but there's no reason combine the concept of a pointer with the concept of "nullable". Representing a memory address is perfectly fine and useful, but from there it doesn't follow that we should also allow it to point to an invalid memory address that crashes if you try to use it. You've already decided to opt for safety with regards to array slices, why stray from both that, and the principle of orthogonality, in this case? It doesn't remove any flexibility, it just pulls out unrelated concepts into two orthogonal features rather than one mashed together feature.

I'd say that in my experience probably the majority, or at least a very large minority, of asserts/crashes when programming C/C++ are null pointer asserts.

--
Sebastian Sylvan

Valentin Nechayev

unread,
Nov 13, 2009, 3:10:48 PM11/13/09
to golang-nuts
On 13 ноя, 21:29, levi <greenspan.l...@googlemail.com> wrote:

> > Pointers are not references, as can be seen by the fact that special
> > syntax is required to access the value that a pointer points to.
>
> Well, its not only pointers. Function values can be nil and map values
> can be nil as well. Or channel values. And so on. The OP is right,
> Hoare's mistake is indeed repeated.

The problem is that any real programming can't avoid this "mistake":
all alternatives are more opaque and horrible. If to insist on having
all pointers of real values, one can't correctly show some of them as
not having real value. If to use separate validness flag, one would
forget to check it. If to point to fictive data area, it would become
global area to exchange with garbage data. And so on. Do we all
respect Hoare, but this itself isn't bug, but only way to trigger
another bugs.

Sebastian Sylvan

unread,
Nov 13, 2009, 3:33:42 PM11/13/09
to Valentin Nechayev, golang-nuts
I'm not sure what you mean. This is a solved problem in many existing "real" languages (e.g. Eiffel, Haskell, etc.). Have you looked at e.g. Eiffel's Void safety? In what way does that not make sense to you?

You simply have *two* "decorators" for types. One for "pointer to" which can never be null (except in well defined scenarios, such as constructors, which end with a dynamic test to ensure the "not null" assumption is valid in the rest of the program). The other decorator is for "nullable" which makes any value (including pointers) potentially null. 

There obviously needs to be a way to convert a nullable value to a non-nullable one. There are various forms of this that you can look up (e.g. Eiffel merely detects that you've checked that the pointer isn't null in an if statement, and inside the if clause the type of the pointer is changed to a non-nullable one).

You don't lose any expressiveness. You can still have a "nullable pointer" if that's what you need to indicate that you don't have a "real" value. So in C syntax it might look something like:

int* x = &p; // pointer - always valid to derefence after initialization is complete
int? y = NULL; // nullable int, either an int or NULL
int*? z = NULL; // nullable pointer, identical to * in standard C - either a pointer or NULL


The benefit is that since for probably 90% of pointers you do *not* want them to be null, most of the code would be using non-nullable pointers, which means that you can omit all pointer checks from your code (and any that the runtime performs). Most functions arguments have non-nullness as precondition, for example. This means that non-nullable pointers would spread and the nullness would be confined to very few areas, and these nullable pointers would be checked and converted to a non-null pointers (or handled if it's null) as soon as possible.

--
Sebastian Sylvan

Peter Froehlich

unread,
Nov 13, 2009, 3:38:14 PM11/13/09
to Ian Lance Taylor, golang-nuts
Hi all,

On Fri, Nov 13, 2009 at 2:48 PM, Ian Lance Taylor <ia...@google.com> wrote:
> Go doesn't have nullable types in general.  We haven't seen a real
> desire for them.

Huh? nil is a valid pointer value, right? So they are "nullable".

> I personally think that in a systems programming language, a pointer
> is a reasonable concept: it intuitively represents a memory address.

Personally I think that much of the trouble with pointers stems from
the idea that "pointers are memory addresses". Abstractly pointers are
values that denote variables, "addresses" are simply a way to
implement pointers; you could implement them as URLs as well.

Since you don't have pointer arithmetic in Go (right?) there's really
no reason to ever think of them as specifically memory addresses. In
any case, there's really very little you can do aside from adding a
"nonnull" qualifier to the language or adding different pointer types
with different constraints. Without more mechanisms, the problem is
not going away.

Note that personally, I don't think there's a major problem with
pointers the way they are. Sure, it would be nice to have better ways
to express their varied uses (indeed Wirth found a cute way to do
type-safe and memory-safe address arithmetic in Oberon-SA) but if in
doubt and you want to have a minimal language, there's a lot of other
stuff you could worry about.

> I don't personally think that permitting pointers to be nil is a
> billion dollar mistake.  In my C/C++ programming I've never noticed
> that NULL pointers are a noticeable source of bugs.

No, the biggest problem are memory management issues and array bounds.
Of course to some extent both of these are related to pointers, but
you can also solve them and still have pointers (without address
arithmetic anyway).

Valentin Nechayev

unread,
Nov 13, 2009, 3:43:42 PM11/13/09
to Sebastian Sylvan, golang-nuts
On Fri, Nov 13, 2009 at 10:33 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

I'm not sure what you mean. This is a solved problem in many existing "real" languages (e.g. Eiffel, Haskell, etc.). Have you looked at e.g. Eiffel's Void safety? In what way does that not make sense to you?

You simply have *two* "decorators" for types. One for "pointer to" which can never be null (except in well defined scenarios, such as constructors, which end with a dynamic test to ensure the "not null" assumption is valid in the rest of the program). The other decorator is for "nullable" which makes any value (including pointers) potentially null. 

Well, you're talking about some different thing. The approach you showed is really language support of runtime checking of pointer to be non-null, and if you prohibit direct dereferencing of "nullable" type, this would be good protection (gathered in one conversion instead of each dereferencing). But I've told about general approach to show "placeholder" instead of real value: null pointer is the easiest method to show absense of value, all another methods are much more cumbersome and error-prone.

--
-netch-

Peter Froehlich

unread,
Nov 13, 2009, 3:44:02 PM11/13/09
to Sebastian Sylvan, Valentin Nechayev, golang-nuts
Tracking whether a pointer is or isn't NULL is a good idea, I agree.
But once you've reached the "not NULL" state, you may have to do
*other* checks at runtime. Take a look at this:

if (p != NULL) {
// p promoted to nonnull type qualifier
p = q;
}

Inside the then part you claim p will be treated as "nonnull" and
therefore no more null pointer checks are needed. Good. But you have
two options when it comes to the assignment: (a) allow it regardless
of whether q is nonnull, then you'll need a dynamic check or (b) only
allow it if q is nonnull already. In the first case you end up with
dynamic checks, in the second you end up with the "const pollution"
problem that Go otherwise tries to avoid (if I understand it
correctly).

Sebastian Sylvan

unread,
Nov 13, 2009, 3:53:01 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 8:43 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Fri, Nov 13, 2009 at 10:33 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

I'm not sure what you mean. This is a solved problem in many existing "real" languages (e.g. Eiffel, Haskell, etc.). Have you looked at e.g. Eiffel's Void safety? In what way does that not make sense to you?

You simply have *two* "decorators" for types. One for "pointer to" which can never be null (except in well defined scenarios, such as constructors, which end with a dynamic test to ensure the "not null" assumption is valid in the rest of the program). The other decorator is for "nullable" which makes any value (including pointers) potentially null. 

Well, you're talking about some different thing. The approach you showed is really language support of runtime checking of pointer to be non-null,

Well, I would phrase it as: language support for static guarantees that a pointer is always non-null at the dereference site, with language-enforced dynamic checks in the few places where you do have a nullable pointer (which can't be dereferenced) and you want to convert it to a regular pointer.
 
and if you prohibit direct dereferencing of "nullable" type, this would be good protection (gathered in one conversion instead of each dereferencing). But I've told about general approach to show "placeholder" instead of real value: null pointer is the easiest method to show absense of value, all another methods are much more cumbersome and error-prone.

Yes, you'd still have null, it just wouldn't be conflated with the concept of pointers - it would work for *any* type, not just pointers. 

These are really two totally different things ("nullable" vs "pointer"). It's an accident of history that they've been combined into one construct in so many languages, and there now seems to be a pretty broad consensus among language designers that it's a mistake that we unfortunately have to live with for many existing languages and VMs because it's so ingrained. Go is a new language, though, so it seems a shame for yet another group of language designers to make the same mistake again (especially when they have a stated goal to avoid just this kind of thing!), when so many others have already lived to regret it.

--
Sebastian Sylvan

Sebastian Sylvan

unread,
Nov 13, 2009, 4:00:19 PM11/13/09
to Peter Froehlich, Valentin Nechayev, golang-nuts
I definitely don't think you should allow any implicit conversions between nullable and non-nullable values! Your example would have to be something like:

if (p != NULL) {
 // p promoted to nonnull type qualifier
 if ( q != NULL ) { // if q is declared outside the outer if, you could just check it there
     p = q;
 }
}

You definitely *want* pointers to propgate non-nullness. You can always convert a regular pointer to a nullable one easily, and converting a nullable one to a regular pointer is just a simple check. What happens in practice though is that 90%+ of your pointers are non-nullable so you rarely have to check at all. You have a few places where you get a nullable one back to signify something special, and what you do is you almost immediately convert it to a non-nullable one (in order to actually do anything useful with it, like pass it to a function that usually takes non-nullable pointers) and from there on in you don't have to do any more checks on it.
--
Sebastian Sylvan

Ian Lance Taylor

unread,
Nov 13, 2009, 4:06:49 PM11/13/09
to Sebastian Sylvan, Valentin Nechayev, golang-nuts
Sebastian Sylvan <sebastia...@gmail.com> writes:

> You simply have *two* "decorators" for types. One for "pointer to" which can
> never be null (except in well defined scenarios, such as constructors, which
> end with a dynamic test to ensure the "not null" assumption is valid in the
> rest of the program). The other decorator is for "nullable" which makes any
> value (including pointers) potentially null.

This leads us in the direction of the const type qualifier. In my
personal opinion, this kind of thing should not be part of the type.
I think this amounts to a language design choice. I think that
calling it a billion dollar mistake amounts to hyperbole.

Ian

Valentin Nechayev

unread,
Nov 13, 2009, 4:12:54 PM11/13/09
to Sebastian Sylvan, golang-nuts
On Fri, Nov 13, 2009 at 10:53 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

Yes, you'd still have null, it just wouldn't be conflated with the concept of pointers - it would work for *any* type, not just pointers. 

These are really two totally different things ("nullable" vs "pointer"). It's an accident of history that they've been combined into one construct in so many languages, and there now seems to be a pretty broad consensus among language designers that it's a mistake that we unfortunately have to live with for many existing languages and VMs because it's so ingrained.

Saying again, you insist that combining these two concepts was incorrect. I try to show it was correct because is the simplest and supported way to show pointer without value (comparing it to literal 0 is just psychologic artifact). This doesn't avoid us from using another kind of pointers - non-nullable - as you showed here.

But, as soon as it's incorrect to do _checking_ without assigning of null pointer as you show:

var pp *foo nullable;
var p *foo;
if pp != nil { p = pp; }

(it's quite easy to write code without such check)

you shall combine checking and assignment in one action:

try {
  p = pp;
  do_something(p);
}

This means _generating exception_ because you can't move further. And, Go doesn't have exceptions yet.
 
Go is a new language, though, so it seems a shame for yet another group of language designers to make the same mistake again (especially when they have a stated goal to avoid just this kind of thing!), when so many others have already lived to regret it.

Well, it's rather good idea to invent "non-nullable" pointers as language construct... but let's be correct and avoid to blame for very early release.

--
-netch-

Sebastian Sylvan

unread,
Nov 13, 2009, 4:20:01 PM11/13/09
to Ian Lance Taylor, Valentin Nechayev, golang-nuts
I'm sorry but this makes no sense to me. In what way does this have a relation to the const qualifier? You can convert between them at any time, you just have to make sure you handle the failure case in the unsafe direction.
It's not about adding information to types, it's about having a less error prone view of what a pointer *is*. Your argument seems to me to be just as applicable to the distinction between ints and bools as well - why don't we just stick those under the same type? Answer: Because they're fundamentally different! Just like the concept of pointers is different to the concept of nullable values.

It's his mistake, he can call it what he wants. I don't think he's wrong in that a "feature" that introduces potential runtime crashes all over the place has been an incredibly expensive mistake.

--
Sebastian Sylvan

Sebastian Sylvan

unread,
Nov 13, 2009, 4:27:15 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 9:12 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Fri, Nov 13, 2009 at 10:53 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

Yes, you'd still have null, it just wouldn't be conflated with the concept of pointers - it would work for *any* type, not just pointers. 

These are really two totally different things ("nullable" vs "pointer"). It's an accident of history that they've been combined into one construct in so many languages, and there now seems to be a pretty broad consensus among language designers that it's a mistake that we unfortunately have to live with for many existing languages and VMs because it's so ingrained.

Saying again, you insist that combining these two concepts was incorrect. I try to show it was correct because is the simplest and supported way to show pointer without value (comparing it to literal 0 is just psychologic artifact). This doesn't avoid us from using another kind of pointers - non-nullable - as you showed here.

But, as soon as it's incorrect to do _checking_ without assigning of null pointer as you show:

var pp *foo nullable;
var p *foo;
if pp != nil { p = pp; }

This would be invalid, p would have to be initialized at the declaration site. Something like:

if pp != nil { 
   var p *foo;
   p = pp; 
   // rest of code using p in this clause
}
 

(it's quite easy to write code without such check)

you shall combine checking and assignment in one action:

try {
  p = pp;
  do_something(p);
}


I'm sorry, I just don't understand what you mean here. Could you try to explain it differently?  


--
Sebastian Sylvan

Sebastian Sylvan

unread,
Nov 13, 2009, 4:32:53 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 9:27 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:


On Fri, Nov 13, 2009 at 9:12 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Fri, Nov 13, 2009 at 10:53 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

Yes, you'd still have null, it just wouldn't be conflated with the concept of pointers - it would work for *any* type, not just pointers. 

These are really two totally different things ("nullable" vs "pointer"). It's an accident of history that they've been combined into one construct in so many languages, and there now seems to be a pretty broad consensus among language designers that it's a mistake that we unfortunately have to live with for many existing languages and VMs because it's so ingrained.

Saying again, you insist that combining these two concepts was incorrect. I try to show it was correct because is the simplest and supported way to show pointer without value (comparing it to literal 0 is just psychologic artifact). This doesn't avoid us from using another kind of pointers - non-nullable - as you showed here.

But, as soon as it's incorrect to do _checking_ without assigning of null pointer as you show:

var pp *foo nullable;
var p *foo;
if pp != nil { p = pp; }

This would be invalid, p would have to be initialized at the declaration site. Something like:

if pp != nil { 
   var p *foo;
   p = pp; 
   // rest of code using p in this clause
}

Ugh, this, obviously:

if pp != nil { 
   var p *foo = pp; 
   // rest of code using p in this clause
}
 

--
Sebastian Sylvan

Valentin Nechayev

unread,
Nov 13, 2009, 4:41:40 PM11/13/09
to Sebastian Sylvan, golang-nuts
On Fri, Nov 13, 2009 at 11:27 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:


This would be invalid, p would have to be initialized at the declaration site. Something like:

if pp != nil { 
   var p *foo;
   p = pp; 
   // rest of code using p in this clause
}

Well, let's the programmer make mistake:

if pq != nil { // pq is some another variable

  var p *foo;
  p = pp;
  // rest of code using p in this clause
}

You simply missed the real check and allowed `p' to be null. Congrats.

I'm sorry, I just don't understand what you mean here. Could you try to explain it differently?  

You shall NOT allow to continue execution if  conversion fails.

--
-netch-

Sebastian Sylvan

unread,
Nov 13, 2009, 4:49:17 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 9:41 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Fri, Nov 13, 2009 at 11:27 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:


This would be invalid, p would have to be initialized at the declaration site. Something like:

if pp != nil { 
   var p *foo;
   p = pp; 
   // rest of code using p in this clause
}

Well, let's the programmer make mistake:

if pq != nil { // pq is some another variable

  var p *foo;
  p = pp;
  // rest of code using p in this clause
}
 
You simply missed the real check and allowed `p' to be null. Congrats.

This would simply fail to compile with type errors on the assignment (p is non-nullable after all, and pp is nullable -- as different as strings and floats are).
 

I'm sorry, I just don't understand what you mean here. Could you try to explain it differently?  

You shall NOT allow to continue execution if  conversion fails. 

Execution should not even start if conversion could "fail" in an unsafe way! The compiler would stop you.

--
Sebastian Sylvan

Valentin Nechayev

unread,
Nov 13, 2009, 4:57:21 PM11/13/09
to Sebastian Sylvan, golang-nuts
On Fri, Nov 13, 2009 at 11:49 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

if pp != nil { 
   var p *foo;
   p = pp; 
   // rest of code using p in this clause
}

Well, let's the programmer make mistake:

if pq != nil { // pq is some another variable

  var p *foo;
  p = pp;
  // rest of code using p in this clause
}
 
You simply missed the real check and allowed `p' to be null. Congrats.

This would simply fail to compile with type errors on the assignment (p is non-nullable after all, and pp is nullable

The same for your example, exactly. Compiler don't know whether pp isn't null. If you doubt, compare with the following:

var i int;
for i := 0; i < 10; i++ {
   if i == 5 {
      if pp == nil { return; }
  }
}
p = pp;

then replace 5 with 12.

And, for any your attempt to prove that compiler can detect fact of checking `pp' I will create counterexample :)

OTOH if you teach compiler only to compare pointers with conditions in if-block, this would be simply inapplicable in real life.
 

--
-netch-

podperson

unread,
Nov 13, 2009, 5:04:12 PM11/13/09
to golang-nuts
> Personally I think that much of the trouble with pointers stems from
> the idea that "pointers are memory addresses". Abstractly pointers are
> values that denote variables, "addresses" are simply a way to
> implement pointers; you could implement them as URLs as well.

This is certainly my problem with pointers, and I wonder why you're so
allergic to the idea that -- by default -- when you allocate a pointer
you would also allocate a valid thing of appropriate size for it to
point to. If you explicitly want to deal with actual memory addresses
to blobs of stuff handed to you from a foreign API or whatever then
you can explicitly declare a potentially unsafe pointer, but the rest
of the time you're free from this rubbish. Just as GC frees you from
free, Sylvan is proposing to free you from checking for null pointers
_everywhere_. It's a big win and what's the price? The compiler shoves
some extra space into function headers and initializes pointer
variables to point to them -- chances are you were going to do the
same thing manually anyway.

If you're going to have pointers that could be urls -- as you suggest
-- then by all means do away with pointers entirely. But it seems to
me a URL is a very different beast from a "variable". I have natural
expectations that a url might not "be there", whereas I pretty much
expect my variables to be there.

Sebastian Sylvan

unread,
Nov 13, 2009, 5:06:37 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 9:57 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Fri, Nov 13, 2009 at 11:49 PM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

if pp != nil { 
   var p *foo;
   p = pp; 
   // rest of code using p in this clause
}

Well, let's the programmer make mistake:

if pq != nil { // pq is some another variable

  var p *foo;
  p = pp;
  // rest of code using p in this clause
}
 
You simply missed the real check and allowed `p' to be null. Congrats.

This would simply fail to compile with type errors on the assignment (p is non-nullable after all, and pp is nullable

The same for your example, exactly. Compiler don't know whether pp isn't null. If you doubt, compare with the following:

I made a mistake in my example that I subsequently corrected. The compiler would've failed on that one too. Sorry. 

You should always have to initialize any non-null pointers to a valid value. So the compiler would always know that something is potentially null, or definitely not null by virtue of regular old type checking (no magic needed!).

It's just how you can't assign an int to a string, you have to convert it to a string first. This is exactly the same thing. They're different types, so the compiler would stop you doing any of that.


--
Sebastian Sylvan

Valentin Nechayev

unread,
Nov 13, 2009, 5:10:40 PM11/13/09
to Sebastian Sylvan, golang-nuts
On Sat, Nov 14, 2009 at 12:06 AM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

I made a mistake in my example that I subsequently corrected. The compiler would've failed on that one too. Sorry. 

You should always have to initialize any non-null pointers to a valid value.

Well, this is more closer. But what shall the code do if initializing value is null and so initialization shall fail? This code shall not execute further...
 
So the compiler would always know that something is potentially null, or definitely not null by virtue of regular old type checking (no magic needed!).

It's just how you can't assign an int to a string, you have to convert it to a string first. This is exactly the same thing. They're different types, so the compiler would stop you doing any of that.

It can't  stop me at compile time because doesn't know real source pointer value.

--
-netch-

Sebastian Sylvan

unread,
Nov 13, 2009, 5:17:11 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 10:10 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Sat, Nov 14, 2009 at 12:06 AM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

I made a mistake in my example that I subsequently corrected. The compiler would've failed on that one too. Sorry. 

You should always have to initialize any non-null pointers to a valid value.

Well, this is more closer. But what shall the code do if initializing value is null and so initialization shall fail? This code shall not execute further...

It wouldn't let you initialize it to a null value because a null value would have a different type and would therefore give a type error at compile time. Again, there's no magic here, these are simply two variables with different types. You don't need any runtime checks to ensure that you haven't accidentally initialized a string with an int because the compiler prevents you from doing that. Same thing here.
 
So the compiler would always know that something is potentially null, or definitely not null by virtue of regular old type checking (no magic needed!).

It's just how you can't assign an int to a string, you have to convert it to a string first. This is exactly the same thing. They're different types, so the compiler would stop you doing any of that.

It can't  stop me at compile time because doesn't know real source pointer value.

It knows what type it is. That's all it needs. I think you may have missed what I wrote above about how Eiffel actually does the "null cast". That would explain the confusion. Look at this:
 
var p *int nullable = foo; //p can potentially be null

if p != nil{
  // inside this clause, the compiler knows p is not nil, so strips away the "nullness" from the type, giving it the type *int
  var q *int = p; // no type error!
}

var w *int = p; // TYPE ERROR. Here p is nullable.

Personally, I'm not a big fan of piggy-backing on the if statement to do this type promotion. It feels a bit too magic to me. I'd prefer a separate statement, maybe something like:

null_cast p {
  // we only end up here if p was non-null, and its type in this clause is non-nullable
}
else{ // optional
  // do something else? Here p is still nullable
}

--
Sebastian Sylvan

Sverre Rabbelier

unread,
Nov 13, 2009, 5:16:55 PM11/13/09
to Valentin Nechayev, Sebastian Sylvan, golang-nuts
Heya,

On Fri, Nov 13, 2009 at 23:10, Valentin Nechayev <net...@gmail.com> wrote:
> Well, this is more closer. But what shall the code do if initializing value
> is null and so initialization shall fail? This code shall not execute
> further...

You mean like this?

var p *int nullable = nil;
if p != nil {
var pp *int = p;
} else {
// can't do anything with p since it's nil
}

> It can't  stop me at compile time because doesn't know real source pointer
> value.

But it does, it either knows that it's a non-null pointer (since
that's it's type), or that it's a possibly null pointer, in which case
you can't make the assignment.

--
Cheers,

Sverre Rabbelier

Yegor

unread,
Nov 13, 2009, 5:20:07 PM11/13/09
to golang-nuts
Seems like Go's switch statement might be powerful enough to implement
Scala's Some/None optional value idiom (http://www.artima.com/forums/
flat.jsp?forum=276&thread=228428). As opposed to null/not-null
reference/pointer, an Option either holds a value or it doesn't. The
only way to get the value is to explicitly cast the option to Some or
None case classes, so there's no way you can forget to handle the no-
value situation. While Go doesn't have case-classes, its switch
statement can be used for dynamic type discovery. So in Go it might
look something like this:

o := getOptionalValue() // never returns null
switch o.(type) {
case Some:
return Some(o).value;
case None: // but may return no value
return "No value";
}

... with corresponding performance penalties, of course.

Yegor

On Nov 13, 2:32 pm, Sebastian Sylvan <sebastian.syl...@gmail.com>
wrote:
> On Fri, Nov 13, 2009 at 9:27 PM, Sebastian Sylvan <

Jon Harrop

unread,
Nov 13, 2009, 5:45:38 PM11/13/09
to golan...@googlegroups.com
On Friday 13 November 2009 22:20:07 Yegor wrote:
> Seems like Go's switch statement might be powerful enough to implement
> Scala's Some/None optional value idiom (http://www.artima.com/forums/
> flat.jsp?forum=276&thread=228428). As opposed to null/not-null
> reference/pointer, an Option either holds a value or it doesn't. The
> only way to get the value is to explicitly cast the option to Some or
> None case classes, so there's no way you can forget to handle the no-
> value situation. While Go doesn't have case-classes, its switch
> statement can be used for dynamic type discovery. So in Go it might
> look something like this:
>
> o := getOptionalValue() // never returns null
> switch o.(type) {
> case Some:
> return Some(o).value;
> case None: // but may return no value
> return "No value";
> }
>
> ... with corresponding performance penalties, of course.

You really want an option and non-null pointers in a high-level language with
compilation down to nullable pointers when the option type is parameterized
over a reference type. F# does this, for example.

--
Dr Jon Harrop, Flying Frog Consultancy Ltd.
http://www.ffconsultancy.com/?e

Valentin Nechayev

unread,
Nov 13, 2009, 5:49:24 PM11/13/09
to Sebastian Sylvan, golang-nuts
On Sat, Nov 14, 2009 at 12:17 AM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

if p != nil{
  // inside this clause, the compiler knows p is not nil, so strips away the "nullness" from the type, giving it the type *int
  var q *int = p; // no type error!

var w *int = p; // TYPE ERROR. Here p is nullable.

All this means too strict code style limitation which isn't applicable for real tasks.
 
Personally, I'm not a big fan of piggy-backing on the if statement to do this type promotion. It feels a bit too magic to me. I'd prefer a separate statement, maybe something like:

null_cast p {
  // we only end up here if p was non-null, and its type in this clause is non-nullable
}

The same story.
 
--
-netch-

Sverre Rabbelier

unread,
Nov 13, 2009, 5:56:02 PM11/13/09
to Valentin Nechayev, Sebastian Sylvan, golang-nuts
Heya,

On Fri, Nov 13, 2009 at 23:49, Valentin Nechayev <net...@gmail.com> wrote:
> All this means too strict code style limitation which isn't applicable for
> real tasks.

No, what Sebastian is saying is that since you don't _want_ a nullable
pointer 9 out of 10, it actually makes your code safer.

--
Cheers,

Sverre Rabbelier

Sebastian Sylvan

unread,
Nov 13, 2009, 5:58:30 PM11/13/09
to Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 10:49 PM, Valentin Nechayev <net...@gmail.com> wrote:
On Sat, Nov 14, 2009 at 12:17 AM, Sebastian Sylvan <sebastia...@gmail.com> wrote:

if p != nil{
  // inside this clause, the compiler knows p is not nil, so strips away the "nullness" from the type, giving it the type *int
  var q *int = p; // no type error!

var w *int = p; // TYPE ERROR. Here p is nullable.

All this means too strict code style limitation which isn't applicable for real tasks.

I strongly disagree. 
There are plenty of languages out there today that already do this and it just isn't an issue.

Furthermore, these checks would only be in the cases where you're already checking for null anyway (or at least should be!), such as functions that may indicate failure by returning null. Any pointer where you today wouldn't check for null before using it must be a pointer you consider "guaranteed to be non-null", and in this system that pointer wouldn't have a nullable type to begin with, which means you don't need to check it (it's guaranteed to be safe).

Also, check out any Haskell or F# programs and see how often you see Maybe/Option (which is what their "nullable" is called) in them. They're there, but they're extremely rare. The vast majority of variables are not nullable.

--
Sebastian Sylvan

roger peppe

unread,
Nov 13, 2009, 6:05:53 PM11/13/09
to golang-nuts
the difficulty with doing something like this in go
as it stands is that go relies on zero values
all over the place. zero values are cheap to create
and cheap on memory, as well as being a
convenient default initial value.

if you had non-nullable pointers, then creating the "zero value"
for a struct containing such a thing would also
entail allocating space for all its references,
only for the code to write the correct values over them again
and leave the original to be deallocated.

not great for a language that's trying to be efficient.

moreover it's not even clear that in the presence of
discriminated unions the compiler could know which
arm of the union to choose for the zero value.

in haskell you can't create a value out of nothing - hence
the existence of concepts like MonadZero.
i don't know what go would like with that kind
of restriction, but i suspect it would be a very different
language.

levi

unread,
Nov 13, 2009, 6:06:24 PM11/13/09
to golang-nuts
On Nov 13, 9:10 pm, Valentin Nechayev <net...@gmail.com> wrote:
> On 13 ноя, 21:29, levi <greenspan.l...@googlemail.com> wrote:
>
> > > Pointers are not references, as can be seen by the fact that special
> > > syntax is required to access the value that a pointer points to.
>
> > Well, its not only pointers. Function values can be nil and map values
> > can be nil as well. Or channel values. And so on. The OP is right,
> > Hoare's mistake is indeed repeated.
>
> The problem is that any real programming can't avoid this "mistake":
> all alternatives are more opaque and horrible.

It is horrible to have maps, functions, channels which are nil. And it
is horrible when language designers permit things for themselves that
language users are not allowed to, like in these ad-hoc cases having
reference semantics for *certain* built-in types [1].

The other important question is - why pointers at all?

-Levi

------------------
[1] Not to mention the fixed set of parameterized types in Go, but
that's another story.

Jon Harrop

unread,
Nov 13, 2009, 6:15:21 PM11/13/09
to golan...@googlegroups.com
What is wrong with .NET's solution to this problem?

Ian Lance Taylor

unread,
Nov 13, 2009, 6:17:44 PM11/13/09
to Sebastian Sylvan, Valentin Nechayev, golang-nuts
Sebastian Sylvan <sebastia...@gmail.com> writes:

> On Fri, Nov 13, 2009 at 9:06 PM, Ian Lance Taylor <ia...@google.com> wrote:
>
>> This leads us in the direction of the const type qualifier. In my
>> personal opinion, this kind of thing should not be part of the type.
>> I think this amounts to a language design choice. I think that
>> calling it a billion dollar mistake amounts to hyperbole.
>>
>
> I'm sorry but this makes no sense to me. In what way does this have a
> relation to the const qualifier? You can convert between them at any time,
> you just have to make sure you handle the failure case in the unsafe
> direction.

If you don't push nonnull through your program, then what benefit have
you really gained?


> It's not about adding information to types, it's about having a less error
> prone view of what a pointer *is*. Your argument seems to me to be just as
> applicable to the distinction between ints and bools as well - why don't we
> just stick those under the same type? Answer: Because they're fundamentally
> different! Just like the concept of pointers is different to the concept of
> nullable values.
>
> It's his mistake, he can call it what he wants. I don't think he's wrong in
> that a "feature" that introduces potential runtime crashes all over the
> place has been an incredibly expensive mistake.

Go currently crashes if you dereference a nil pointer. Presumably
with a nonnull qualifier, it would crash if you assign a nil pointer
to a pointer type with the nonnull qualifier. What is the fundamental
difference? Either way your program is taking an invalid action, and
it crashes.

Ian

Sebastian Sylvan

unread,
Nov 13, 2009, 6:23:10 PM11/13/09
to Ian Lance Taylor, Valentin Nechayev, golang-nuts
On Fri, Nov 13, 2009 at 11:17 PM, Ian Lance Taylor <ia...@google.com> wrote:
Sebastian Sylvan <sebastia...@gmail.com> writes:

> On Fri, Nov 13, 2009 at 9:06 PM, Ian Lance Taylor <ia...@google.com> wrote:
>
>> This leads us in the direction of the const type qualifier.  In my
>> personal opinion, this kind of thing should not be part of the type.
>> I think this amounts to a language design choice.  I think that
>> calling it a billion dollar mistake amounts to hyperbole.
>>
>
> I'm sorry but this makes no sense to me. In what way does this have a
> relation to the const qualifier? You can convert between them at any time,
> you just have to make sure you handle the failure case in the unsafe
> direction.

If you don't push nonnull through your program, then what benefit have
you really gained?

You don't "push" it through your program any more than you "push" booleans through your programs. Some things are pointers, some things are nullable, some things are nullable pointers. You could store all your booleans as ints and convert them to bools when you need to, but that would be silly, of course you store them as bools everywhere since that's what they actually are.
 


> It's not about adding information to types, it's about having a less error
> prone view of what a pointer *is*. Your argument seems to me to be just as
> applicable to the distinction between ints and bools as well - why don't we
> just stick those under the same type? Answer: Because they're fundamentally
> different! Just like the concept of pointers is different to the concept of
> nullable values.
>
> It's his mistake, he can call it what he wants. I don't think he's wrong in
> that a "feature" that introduces potential runtime crashes all over the
> place has been an incredibly expensive mistake.

Go currently crashes if you dereference a nil pointer.  Presumably
with a nonnull qualifier, it would crash if you assign a nil pointer
to a pointer type with the nonnull qualifier.  What is the fundamental
difference?  Either way your program is taking an invalid action, and
it crashes.

No, with my suggestion the compiler would catch it. No runtime crash. That's a huge win.
A pointer type wouldn't be nullable, so it would be a different type from a nullable type, which would give a type error if you try to assign one to the other (although an implicit conversion from non-nullable to nullable may be convenient).


--
Sebastian Sylvan

Intrigued Goer

unread,
Nov 13, 2009, 6:30:08 PM11/13/09
to golang-nuts
On Nov 13, 1:52 pm, Sebastian Sylvan <sebastian.syl...@gmail.com>
wrote:
> On slide 14 of the Go Talk the first design principle is "Keep
> concepts orthogonal", and yet Go violates this principle by repeating
> Hoare's "Billion Dollar Mistake" (http://www.infoq.com/presentations/Null-References-The-Billion-Dollar...
> ) by combining the two orthogonal concepts of "reference to" and
> "nullable" into a single construct.
>
> I know many language designers have said in retrospect that they
> regret getting it wrong (e.g. several from the C# team), so if you're
> going to make a new language why not do it right this time and let
> these two features be two separate features like they're supposed to
> be?

Oi, I get all warm fuzzy from the thought of non-nullable pointers.
The amount of Java software I've seen printing uncaught null pointer
exceptions makes me shake my head. The amount of stupid, repetitive,
error-prone, potentially never-executed null checking code I've
written in C(++) makes me want to never code in C(++) again. And then
there's all the code which has just an assert (or just assumption that
it'll segfault if it's null...), where it's been assumed that any null
pointer bugs were caught in testing, and more importantly any future
code will also be so thoroughly tested (hahaa) that these functions
never get called with null parameters...

Go has no exceptions to handle null pointers nicely, and not even C++-
like references to indicate that something probably shouldn't be null
(though C++ doesn't enforce this, a pity), so handling null pointers
is as much work as in C. But go does have multiple return values
(alternative to returning null to signal error), while it does not
have pointer arithmetic to mess up pointers. And checking this would
just be a type check, so no compile time penalty (type checking is
done anyway), and no run time penalty (nothing done at run time).

To the language developers: Please please implement this. Go has the
other features to make this integrate well with the language, and null
pointer handling is such a huge PITA and such a huge source of errors.
Don't mess up this chance to do things right! And make sure non-
nullable is the default for pointers. You'll kill a million bugs
before they're even made.

For the "null_cast" thing I propose syntax:
ifsafe pointer1[, pointer2...] {
// pointers temporarily non-nullable in here
} else {
// some of the pointers known to be null
}

Sebastian Sylvan

unread,
Nov 13, 2009, 6:30:41 PM11/13/09
to roger peppe, golang-nuts
On Fri, Nov 13, 2009 at 11:05 PM, roger peppe <rogp...@gmail.com> wrote:
the difficulty with doing something like this in go
as it stands is that go relies on zero values
all over the place. zero values are cheap to create
and cheap on memory, as well as being a
convenient default initial value.

if you had non-nullable pointers, then creating the "zero value"
for a struct containing such a thing would also
entail allocating space for all its references,
only for the code to write the correct values over them again
and leave the original to be deallocated.


I would prefer if you didn't allocate the initial space when the variable is guaranteed to be initialized before it's used. E.g. in Eiffel the references are nullable in the constructor (so you can initialize them at your leisure), and then there's a dynamic check after the constructor has finished to make sure everything has been assigned. 

For Go the idiomatic solution may be to just initialize anything that won't be explicitly initialized to a default instance... Not sure, don't have a good enough "feel" for Go yet to know what the correct thing to do about those pointers are.

Of course, you wouldn't be *removing* nullable pointers, so you're free to use them in cases where it would be too expensive to initialize them to a default target.


moreover it's not even clear that in the presence of
discriminated unions the compiler could know which
arm of the union to choose for the zero value.

I don't think you necessarily need full discriminated unions in the language. Just "nullable" as hard coded special case. "Nullable pointer" would probably compile down to a regular old pointer with a magic value for "null" so it's identical to the current system in that respect.


--
Sebastian Sylvan

Jon Harrop

unread,
Nov 13, 2009, 6:38:42 PM11/13/09
to golan...@googlegroups.com
On Friday 13 November 2009 23:17:44 Ian Lance Taylor wrote:
> Sebastian Sylvan <sebastia...@gmail.com> writes:
> > On Fri, Nov 13, 2009 at 9:06 PM, Ian Lance Taylor <ia...@google.com> wrote:
> >> This leads us in the direction of the const type qualifier. In my
> >> personal opinion, this kind of thing should not be part of the type.
> >> I think this amounts to a language design choice. I think that
> >> calling it a billion dollar mistake amounts to hyperbole.
> >
> > I'm sorry but this makes no sense to me. In what way does this have a
> > relation to the const qualifier? You can convert between them at any
> > time, you just have to make sure you handle the failure case in the
> > unsafe direction.
>
> If you don't push nonnull through your program, then what benefit have
> you really gained?

You can write code that cannot ever suffer from null pointer/reference issues,
eliminating an important class of bugs and making your code more reliable.

Ian Lance Taylor

unread,
Nov 13, 2009, 6:46:35 PM11/13/09
to Sebastian Sylvan, Valentin Nechayev, golang-nuts
So it is that conversion that will crash at runtime.

I'm sorry, I'm not seeing the huge win. If I wanted to tackle this
problem, I would have the compiler interally annotate every function
which dereferenced a pointer. Then I would have the compiler check
those annotations on every parameter pass. Then I would have the
compiler warn whenever a nil pointer gets passed to a function which
may eventually dereference it. I wouldn't touch the type system.

Ian

Sebastian Sylvan

unread,
Nov 13, 2009, 6:54:50 PM11/13/09
to Ian Lance Taylor, Valentin Nechayev, golang-nuts
No, nothing will crash at runtime. That's the whole point. You are statically guaranteed that every single pointer access is always 100% safe. There are no runtime crashes involved in any aspect of this.

If you have a nullable pointer and you want to actually do anything with it (e.g. dereference it) you have to cast it to a regular pointer - this is a safe cast though, again no crashes. You just have to specify what you want to do in the case it's actually not null, and what to do if it is null. E.g.

if p != nil {
 // here you can use p with 100% statically guaranteed safety, because you can only get here if p is not null at runtime
 // the compiler would change p's type to a pointer here
}
else
{
  // here you have to deal with the fact that p was null... 
  // p is still a nullable here...
}

You can of course choose to invoke a crash in the else clause, but then at least you're being naughty explicitly. The whole rest of your program can completely ignore the possibility of nulls because they're just not possible.

--
Sebastian Sylvan

Sverre Rabbelier

unread,
Nov 13, 2009, 7:08:53 PM11/13/09
to Sebastian Sylvan, Ian Lance Taylor, Valentin Nechayev, golang-nuts
Heya,

On Sat, Nov 14, 2009 at 00:54, Sebastian Sylvan
<sebastia...@gmail.com> wrote:
> You can of course choose to invoke a crash in the else clause, but then at
> least you're being naughty explicitly. The whole rest of your program can
> completely ignore the possibility of nulls because they're just not
> possible.

How would you do that if it is not allowed to dereference a nullable
pointer (which would be the sensible thing to do IMO)?

--
Cheers,

Sverre Rabbelier

Sebastian Sylvan

unread,
Nov 13, 2009, 7:15:47 PM11/13/09
to Sverre Rabbelier, Ian Lance Taylor, Valentin Nechayev, golang-nuts
I meant choose to invoke a crash in the general sense, not by dereferencing the nullable pointer (which is indeed not possible). E.g. just call  exit(1) or some such.

The point is that at no point do you need to worry about a pointer being null. It may still be the case that a nullable value would indicate fatal errors so that you must crash, but at least it's not being passed around the program for god knows how long until it finally crashes miles from the original problem.
 

--
Sebastian Sylvan

Jon Harrop

unread,
Nov 13, 2009, 7:31:55 PM11/13/09
to golan...@googlegroups.com
On Friday 13 November 2009 23:46:35 Ian Lance Taylor wrote:
> So it is that conversion that will crash at runtime.
>
> I'm sorry, I'm not seeing the huge win...

The huge win occurs when there is no conversion.

Sverre Rabbelier

unread,
Nov 13, 2009, 7:43:10 PM11/13/09
to Sebastian Sylvan, Ian Lance Taylor, Valentin Nechayev, golang-nuts
Heya,

On Sat, Nov 14, 2009 at 01:15, Sebastian Sylvan
<sebastia...@gmail.com> wrote:
> I meant choose to invoke a crash in the general sense, not by dereferencing
> the nullable pointer (which is indeed not possible). E.g. just call  exit(1)
> or some such.

Ah, I misunderstood your "being naughty" for "trying to dereference
the pointer anyway". But yes, it does make sense that a nullable
pointer turning out to actually be null could signal an error
condition.

--
Cheers,

Sverre Rabbelier

jqb

unread,
Nov 14, 2009, 1:55:57 AM11/14/09
to golang-nuts
On Nov 13, 3:17 pm, Ian Lance Taylor <i...@google.com> wrote:
> Sebastian Sylvan <sebastian.syl...@gmail.com> writes:
> > On Fri, Nov 13, 2009 at 9:06 PM, Ian Lance Taylor <i...@google.com> wrote:
>
> >> This leads us in the direction of the const type qualifier.  In my
> >> personal opinion, this kind of thing should not be part of the type.
> >> I think this amounts to a language design choice.  I think that
> >> calling it a billion dollar mistake amounts to hyperbole.
>
> > I'm sorry but this makes no sense to me. In what way does this have a
> > relation to the const qualifier? You can convert between them at any time,
> > you just have to make sure you handle the failure case in the unsafe
> > direction.
>
> If you don't push nonnull through your program, then what benefit have
> you really gained?

Your thinking is mired in the non-analogous "const" case. const is
about how a variable will be used, not what values it can take on.

A pointer is nonnull (it's a reference to an object) and most pointer
variables are not intended to take nil values, thus nonnull should be
the default, and nothing has to be "pushed through". Pointers that are
intended to be able to have a nil value should be declared nullable
(or nilable). Since the issue is what the set of allowed values is,
*of course* it should be part of the type -- your statement that it
shouldn't suggests some very confused thinking.

> > It's not about adding information to types, it's about having a less error
> > prone view of what a pointer *is*. Your argument seems to me to be just as
> > applicable to the distinction between ints and bools as well - why don't we
> > just stick those under the same type? Answer: Because they're fundamentally
> > different! Just like the concept of pointers is different to the concept of
> > nullable values.
>
> > It's his mistake, he can call it what he wants. I don't think he's wrong in
> > that a "feature" that introduces potential runtime crashes all over the
> > place has been an incredibly expensive mistake.
>
> Go currently crashes if you dereference a nil pointer.  Presumably
> with a nonnull qualifier, it would crash if you assign a nil pointer
> to a pointer type with the nonnull qualifier.

We're talking about static typing here. In a language that implements
the nonnull/nullable distinction, you can't assign nil to a non-
nullable pointer variable -- that isn't possible, any more than it's
possible to assign the address of an int to a pointer to float
variable; they aren't type-compatible. And it's not a nonnull
qualifier -- it's a *nullable* qualifier; nonnull is the default.

>  What is the fundamental
> difference?  Either way your program is taking an invalid action, and
> it crashes.

The fundamental difference is that static typing turns runtime
failures into compile-time failures. By distinguishing a type that
includes only pointers (references to objects) from a type that
includes both pointers and nil, one can prevent ever treating nil as
if it were a pointer (i.e., dereferencing it) at runtime.

k...@golang.org

unread,
Nov 14, 2009, 2:15:05 PM11/14/09
to golang-nuts

it is clear that you all have heard of the billion dollar blunder,
but few are aware of the lesser known, but more important, $1.98
mis-step. suppose you want to divide by the variable n, wouldnt
it be nice to guarantee that n is not zero. thus the qualifier
type non-zero, (abbreviated nz with apology to new zealand). so
we add a type adjective to the types int, uint, byte, etc. now

var n nz uint = 0;

would be flagged by the compiler while

var n nz uint;

would assign 1 to n. that way, you can divide by n without worries.

but there is another "bad" value that you might want to avoid, 1<<31
for signed int32. so i guess we come up with some sort of parametric
adjective to accommodate this.

var n butnot(256) byte;

but this will get an error since 256 is not a byte, so we change
to

var n butnot(-256) byte;

several things now work out, we can repeat the parametric type to
exclude many values. thus we can avoid all of the poles/zeroes in
an equation. we can also drop the special declaration of non-null
with this more general form.

lets now fall back to simple nz since we are still debating generic
types. in practice, we will want to sometimes pass an int to a
function that requires an nz-int. the compiler could do the check
for you by compiling inline a compare and panic. i dont think that
is very good, but it could be an option flag to the compiler or
maybe an ifdef or environment variable.

a better solution would be to flag it as an error so that the
programmer could convert the value from one type to another before
calling. thus the programmer would rewrite

f(v);

into

n := (nz int)v; f(n)

where the conversion would check for zero and panic. we have a
solution for that too.

n,ok := (nz int)v; if ok { f(n) }

where n gets the value v if v is not zero and gets the value 1 if
v is zero. since n is now a valid nz-int, we could just write

n,_ := (nz int)v;
f(n);


thus all the pieces are in place for f() to divide
without testing.

Sebastian Sylvan

unread,
Nov 14, 2009, 2:40:36 PM11/14/09
to k...@golang.org, golang-nuts
I understand what you're trying to do, but you should try to find an example that has a little more relevance.

For one thing the whole purpose of null is to signify "invalid" and it's always incorrect to dereference it, while zero is a perfectly valid integer in most cases. Second, once a pointer is non-null it doesn't generally become potentially null on most of its operations (+,-,%,* and others could return zero). Third, nullness is an orthogonal concept to "pointerness", while zero is a subset of integers.

In short: Your facetious example would lead to tons of annoying checks whenever you try to do anything - which I assume is your "point",  but we know from real world experience that separating pointers from null-ness does not carry that same penalty at all. This isn't an academic argument - plenty of languages do this. You can look at them and evaluate how it works (hint: extremely well).

This isn't about splitting apart the domain of a type to avoid bad things, it's about not munging two different concepts together into the same type for no good reason other than an historical accident. It's analogous to how C treats just about anything as booleans, while Go (and most other languages) has a proper boolean type. There's no reason why an int should be treated like a boolean, and there's no reason why a pointer should be treated as a nullable.

If you have any actual arguments about why these two orthogonal concepts should be conflated, I think everyone would be happy to hear them, but I don't see how this kind of reply is productive.

--
Sebastian Sylvan

Ian Lance Taylor

unread,
Nov 14, 2009, 2:59:10 PM11/14/09
to jqb, golang-nuts
jqb <jqba...@gmail.com> writes:

> A pointer is nonnull (it's a reference to an object) and most pointer
> variables are not intended to take nil values, thus nonnull should be
> the default, and nothing has to be "pushed through". Pointers that are
> intended to be able to have a nil value should be declared nullable
> (or nilable). Since the issue is what the set of allowed values is,
> *of course* it should be part of the type -- your statement that it
> shouldn't suggests some very confused thinking.

Sorry for my confusion.

What is the value of an uninitialized variable of pointer type?

Ian

Sebastian Sylvan

unread,
Nov 14, 2009, 3:09:11 PM11/14/09
to Ian Lance Taylor, jqb, golang-nuts
Someone suggested that in keeping with Go's policy on uninitialized values that it would simply point to a "zero value" of whatever type it's pointing to. I like this idea more and more. Of course, you'd make sure to try to initialize it at construction to avoid creating that zero value (just like for any other field at the moment).

Most other languages that do this tend to ensure that it's impossible to construct an object with a pointer that hasn't been initialized by the way they set up object construction. This would be more challenging and require more thought for Go (especially since there's no built in concept of constructors), I think.

Yet others, primarily Eiffel I think, do it with a dynamic check at the end of construction IIRC (which is less than ideal, but at least the crash happens where the problem is, not miles away from it hours later). Even if you go the Eiffel route, at least you narrow down the potential problem area to the initialization - once you're through that you don't have to worry about it any more.


--
Sebastian Sylvan

Ola Fosheim Grøstad

unread,
Nov 14, 2009, 3:33:33 PM11/14/09
to golang-nuts
On 13 Nov, 20:48, Ian Lance Taylor <i...@google.com> wrote:
> billion dollar mistake.  In my C/C++ programming I've never noticed
> that NULL pointers are a noticeable source of bugs.

The cost of a single NULL-dereference after deployment of a server can
be quite big.

Not all programmers are equally careful. You are probably a better
programmer than most. I agree with Hoare: language designers should
take the responsibility for errors made by their users. Rule #1 for
usability: you cannot use yourself as a model for the users of your
own product.

Anyway, C++ has references (&) which kinda are (not quite) like non-
null reference that you cannot assign to... So they have something
better than nullable pointers, at least. If I had a vote I'd vote for
non-null references, but then again I'd also vote for contracts
(invariant checking) with the ability to require some of them to be
established at compile time. I think it would be nice for debugging to
be able to attach contracts to types and have them checked everywhere.
Much better than asserts. NULL-dereferencing is just a special case of
what should be adressed IMO.

Ola.

Hans

unread,
Nov 14, 2009, 3:59:26 PM11/14/09
to golang-nuts


On Nov 13, 3:52 am, Sebastian Sylvan <sebastian.syl...@gmail.com>
wrote:
> ... yet Go violates this principle by repeating
> Hoare's "Billion Dollar Mistake" (http://www.infoq.com/presentations/Null-References-The-Billion-Dollar...
> ) by combining the two orthogonal concepts of "reference to" and
> "nullable" into a single construct.

I watched the video, or at least as much as I could before getting
really sick of hearing Hoare pontificate.

It's only a billion dollar mistake if you assume all programmers never
advance beyond the novice level. After listening to Hoare, I can
believe that he would make that assumption.

Peter Froehlich

unread,
Nov 14, 2009, 4:07:56 PM11/14/09
to Hans, golang-nuts
This discussion must have deteriorated pretty far while I wasn't
looking: We've reached the "Hoare is stupid!" level of the argument.
Amazing. :-D

Ola Fosheim Grøstad

unread,
Nov 14, 2009, 4:18:10 PM11/14/09
to golang-nuts
> What is the value of an uninitialized variable of pointer type?

You may require the compiler to ensure that it is provably not used
until it has obtained a valid reference.

Dwight Schauer

unread,
Nov 14, 2009, 4:24:33 PM11/14/09
to Ola Fosheim Grøstad, golang-nuts
This might catch some obvious cases.

But for pointer elements of a data structure that is passed some where
else, that might be difficult to track at compile time. How does the
compiler know if you are passing it elsewhere for them to be
initialized or used?

Ola Fosheim Grøstad

unread,
Nov 14, 2009, 4:31:46 PM11/14/09
to golang-nuts
On 14 Nov, 00:54, Sebastian Sylvan <sebastian.syl...@gmail.com> wrote:
if p != nil {
>  // here you can use p with 100% statically guaranteed safety, because you
> can only get here if p is not null at runtime
>  // the compiler would change p's type to a pointer here}
>
> else
> {
>   // here you have to deal with the fact that p was null...
>   // p is still a nullable here...
>
> }
>
> You can of course choose to invoke a crash in the else clause, but then at
> least you're being naughty explicitly. The whole rest of your program can
> completely ignore the possibility of nulls because they're just not
> possible.

If the compiler actually KNOWS that p is null then it should of course
issue an error. Why should the compiler allow programmers to be
suicidal? Even with separate compiled units you should be able to
propagate that kind of information in situations where functions
always assume that a parameter is non-null, even without a non-null
pointer type.

I never quite understood why compilers should just meet the specs of
the language and not do more if they are capable of infering more...?
I think Go will have to either settle for being non-suicidal or
suicidal. If they go for the latter, then give me inline asm plz...

Ola.

Hans

unread,
Nov 14, 2009, 4:31:51 PM11/14/09
to golang-nuts

> We've reached the "Hoare is stupid!" level of the argument.
> Amazing. :-D

I didn't say he was stupid. I do, however, feel confident in saying
that to claim his mistake cost a billion dollars, he must have a very
low opinion of all other software developers.

Dwight Schauer

unread,
Nov 14, 2009, 4:43:34 PM11/14/09
to Ola Fosheim Grøstad, golang-nuts
On Sat, Nov 14, 2009 at 3:31 PM, Ola Fosheim Grøstad
<ola.foshe...@gmail.com> wrote:
> If the compiler actually KNOWS that p is null then it should of course
> issue an error. Why should the compiler allow programmers to be
> suicidal? Even with separate compiled units you should be able to
> propagate that kind of information in situations where functions
> always assume that a parameter is non-null, even without a non-null
> pointer type.
>
> I never quite understood why compilers should just meet the specs of
> the language and not do more if they are capable of infering more...?
> I think Go will have to either settle for being non-suicidal or
> suicidal. If they go for the latter, then give me inline asm plz...

The more I think about this issue, the more I agree that it is
something the compiler should handle, and not be suicidal.

But this issue extends to index range checking at compile time as
well, which is a different subject, which would force the use of
ranged integer types.