Immediate values
Eustaquio Rangel de Oliveira Jr.
Hash: SHA1
Hi!
I heard that immediate values holds the object, not a reference to it, is
that right?
I mean:
s1 = "test" # a String located on for ex 0xCC53D5DF
s2 = s1 # points to the same place as s1
s3 = "test" # ANOTHER String, locate on for ex 0xC0DD54D0
n1 = 1 # Fixnum here, located on ... ?
n2 = n1 # points to the same place as n1
n3 = 1 # points to the same place as n1
So, Fixnum (as true, false and nil) objects uses the same object for all
over the program, but Strings, for ex, does not, even if the value are the
same there are distinct objects, right?
On the end, n1, n2 and n3 are not reference to this only one object
allocated there, shared by all? Variables are not all references, even on
the Fixnum case, pointing to an allocated single object there?
And I think the mark-and-sweep garbage collector works on the same way as
other objects to Fixnum, true, false and nil right?
Thanks! :-)
- ----------------------------
Eustáquio "TaQ" Rangel
eustaqu...@yahoo.com
http://beam.to/taq
Usuário GNU/Linux no. 224050
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFB4lJNb6UiZnhJiLsRAqS1AJ9PG+dTod2hRCEJAo71ciqIY+KiMQCdGbxU
yj5Dg/5NfkpW+Qnislw9Nz4=
=DRR4
-----END PGP SIGNATURE-----
Austin Ziegler
<eustaqu...@yahoo.com> wrote:
> I heard that immediate values holds the object, not a reference to
> it, is that right?
>
> I mean:
>
> s1 = "test" # a String located on for ex 0xCC53D5DF
> s2 = s1 # points to the same place as s1
> s3 = "test" # ANOTHER String, locate on for ex 0xC0DD54D0
> n1 = 1 # Fixnum here, located on ... ?
> n2 = n1 # points to the same place as n1
> n3 = 1 # points to the same place as n1
>
> So, Fixnum (as true, false and nil) objects uses the same object
> for all over the program, but Strings, for ex, does not, even if
> the value are the same there are distinct objects, right?
>
> On the end, n1, n2 and n3 are not reference to this only one
> object allocated there, shared by all? Variables are not all
> references, even on the Fixnum case, pointing to an allocated
> single object there?
Up until this paragraph I was with you and completely agree.
Variables are all references, even to Fixnum and Symbol objects.
It's an implementation detail that all Fixnum and Symbol object
instances s are references to the same object, IMO.
-austin
--
Austin Ziegler * halos...@gmail.com
* Alternate: aus...@halostatue.ca
Kaspar Schiess
I have already answered to this on the irc channel, but will try to
collect the answers given as an answer for the history:
Ruby variables hold references to objects. Objects are allocated on the
heap, so a variable saves a pointer to the heap.
n1 = "test"
n2 = n1
# n1 and n2 point to the _same_ object.
n1.object_id == n2.object_id # > true
So you do have to take care not to modify n1, otherwise you will be
modifying n2 too. This is actually less often a problem than one would
think.
# look here
n1 += 'test' # creates new object and leaves n2 alone
# of course
n1.sub! /est/, 'ry' # will modify n1 and n2
Look also here: http://www.rubycentral.com/book/tut_classes.html,
starting at the title 'Variables'.
Of course that is what you need to think about objects, rather than what
it looks like behind the facade. If you are interested in the underlying
reality I guess you need to look at some code.
best regards,
kaspar
Robert Klemme
"Austin Ziegler" <halos...@gmail.com> schrieb im Newsbeitrag
news:9e7db91105011...@mail.gmail.com...
Totally agree! And it doesn't make a difference from the usage point of
view since instances of Symbol, Fixnum, TrueClass, FalseClass, NilClass
are immutable.
Addenum: it's especially noteworthy that strings literals are treated
differently from true, false, Fixnums and Symbols. They create a new
string instance on each evaluation.
>> 5.times { p ["foo", 'foo', :foo, 1, 1.2, 10000000000000, true, false,
nil].map {|o| o.id} }
[134663704, 134663692, 3938574, 3, 134665492, 134665228, 2, 0, 4]
[134661376, 134661316, 3938574, 3, 134665492, 134665228, 2, 0, 4]
[134660428, 134660392, 3938574, 3, 134665492, 134665228, 2, 0, 4]
[134659348, 134659336, 3938574, 3, 134665492, 134665228, 2, 0, 4]
[134656588, 134656372, 3938574, 3, 134665492, 134665228, 2, 0, 4]
=> 5
Kind regards
robert
georgesawyer
at 07:00 PM wrote:
>I heard that immediate values hold the object, not a reference to it; is
that right? I mean:
>s1 = "test" # a String located on for ex 0xCC53D5DF
>s2 = s1 # points to the same place as s1
>s3 = "test" # ANOTHER String, locate on for ex 0xC0DD54D0
>n1 = 1 # Fixnum here, located on ... ?
>n2 = n1 # points to the same place as n1
>n3 = 1 # points to the same place as n1
over the program, but Strings, for ex, do not: even if the values are the
same, they are distinct objects; right?
Moreover, during program execution, each execution of the same line
containing a literal String creates a different object:
->for i in (1..2) do a='test'; p a.object_id end
20816438
20816414
->b=a; b.object_id
20816414
>On the end, are not n1, n2 and n3 references to this [single] object
allocated there, shared by all? Are not variables all references, even in
the Fixnum case, pointing to an allocated single object there?
Fixnum objects are immutable. A variable can refer to different Fixnum's.
That's the ideal, and Ruby's behavior. About implementation efficiency and
understanding, from _Programming Ruby (1st ed.)_, chapter "Extending Ruby",
section "Ruby Objects in C":
"[I]mmediate values are not pointers: Fixnum, Symbol, true, false, and nil
are stored directly in VALUE. Fixnum values are stored as 31-bit numbers[Or
63-bit on wider CPU architectures.] that are formed by shifting the
original number left 1 bit and then setting the least significant bit (bit
0) to ``1.'' When VALUE is used as a pointer to a specific Ruby structure,
it is guaranteed always to have an LSB of zero; the other immediate values
also have LSBs of zero. Thus, a simple bit test can tell you whether or not
you have a Fixnum."
->for i in (1..2) do a=10; p a.object_id end
21
21
->b=a; p b.object_id; b=nil; b.object_id
21
4
>I think the garbage collector works the same way as other objects on
Fixnum, true, false and nil, right?
Naturally, it ignores immutable objects. You can create millions of Fixnum
objects, yet they take no storage. Not so for String or Bignum objects:
->a=1_000; GC.disable; for i in (1..100_000_000) do b=a*2 end
1..100000000
->a='z'*1_000; GC.disable; for i in (1..1_000_000) do b=a*2 end
(irb):2:in `*': failed to allocate memory (NoMemoryError)
->a=1_000_000_000_000_000_000_000; GC.disable; for i in (1..10_000_000)
do b=a*2 end
[FATAL] failed to allocate memory
->system 'ruby --version'
ruby 1.8.2 (2004-07-29) [i386-mswin32]
Florian Gross
> Totally agree! And it doesn't make a difference from the usage point of
> view since instances of Symbol, Fixnum, TrueClass, FalseClass, NilClass
> are immutable.
Note that there is a small difference for Symbols and Fixnums. You can
not define singleton methods on those objects. Maybe external singleton
classes that work like exivars would help with that.
> Addenum: it's especially noteworthy that strings literals are treated
> differently from true, false, Fixnums and Symbols. They create a new
> string instance on each evaluation.
But note that literals with the same content will actually share the
String buffer.
Robert Klemme
"Florian Gross" <fl...@ccan.de> schrieb im Newsbeitrag
news:34fjg2F...@individual.net...
> Robert Klemme wrote:
>
> > Totally agree! And it doesn't make a difference from the usage point
of
> > view since instances of Symbol, Fixnum, TrueClass, FalseClass,
NilClass
> > are immutable.
>
> Note that there is a small difference for Symbols and Fixnums. You can
> not define singleton methods on those objects.
Yep.
> Maybe external singleton
> classes that work like exivars would help with that.
Uh, oh... What's an "exivar"?
> > Addenum: it's especially noteworthy that strings literals are treated
> > differently from true, false, Fixnums and Symbols. They create a new
> > string instance on each evaluation.
>
> But note that literals with the same content will actually share the
> String buffer.
True (yet another optimization). But only as long as they are not
modified. It's about the same behavior as #dup was called on some string
held behind the scenes. Still another object instance has to be allocated
which is why in performance critical parts you're usually better off
defining a frozen string constant if the string doesn't change anyway.
class Dummy
SAMPLE = "foo".freeze
def call_often(str)
str.include? SAMPLE
end
end
Kind regards
robert
ts
R> Uh, oh... What's an "exivar"?
Look at [ruby-talk:17321]
Guy Decoux
Bertram Scharpf
Am Montag, 10. Jan 2005, 21:09:03 +0900 schrieb Kaspar Schiess:
> Ruby variables hold references to objects.
Stricly spoken, Fixnums don't. They are treated a special
way.
> If you are interested in the underlying
> reality I guess you need to look at some code.
And here it comes (paste to `irb'):
max = 2**30
a = Array.new 5_000 do rand max end ; nil
a.all? { |i| i == i.object_id >> 1 }
a.any? { |i| (i.object_id & 0x1).zero? }
Bertram
--
Bertram Scharpf
Stuttgart, Deutschland/Germany
http://www.bertram-scharpf.de
Eustaquio Rangel de Oliveira Jr.
Hash: SHA1
Hi!
| "[I]mmediate values are not pointers: Fixnum, Symbol, true, false, and nil
| are stored directly in VALUE. Fixnum values are stored as 31-bit numbers[Or
| 63-bit on wider CPU architectures.] that are formed by shifting the
| original number left 1 bit and then setting the least significant bit (bit
| 0) to ``1.'' When VALUE is used as a pointer to a specific Ruby structure,
| it is guaranteed always to have an LSB of zero; the other immediate values
| also have LSBs of zero. Thus, a simple bit test can tell you whether or not
| you have a Fixnum."
Forgive me if I misunderstood, but so VALUEs are variables?
a = 1
a is the VALUE, with 32 bit length?
So, to find the object_id of *all* the Fixnum, all I have to do is
something like:
[taq@~]irb
irb(main):001:0> i = 3
=> 3
irb(main):002:0> s = "0b" << i.to_s(2) << "1"
=> "0b111"
irb(main):003:0> s.to_i(2)
=> 7
The object_id of 3 always will be 7 right? But i does not point to
somewhere on memory there? i is the own object?
And when I have a bit 1 there on LSB is *always* a Fixnum?
| Naturally, it ignores immutable objects. You can create millions of Fixnum
| objects, yet they take no storage. Not so for String or Bignum objects:
Ok, but how it does that? I mean, I have 31 (32?) bit numbers there on my
one thousand vars on the local scope. Where they are stored? They are
somewhere on the memory, right? How it can take no storage? Automatically
killed after it scopes ends? But before this, where they are? :-)
Sorry if I'm asking a lot about that, but immediate values made me curious
about it. :-)
Regards,
- ----------------------------
Eustáquio "TaQ" Rangel
eustaqu...@yahoo.com
http://beam.to/taq
Usuário GNU/Linux no. 224050
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFB4qQkb6UiZnhJiLsRAvBsAJ9NdntqBt0ZoPKAyrfnpr2J8aQRJwCfS4DX
KO3f8rtIznKnp5pICtOc/+U=
=vBrT
-----END PGP SIGNATURE-----
Eustaquio Rangel de Oliveira Jr.
Hash: SHA1
Btw, I'm trying to understand how these things works to see if it avoids
this
http://evanjones.ca/python-memory.html
kind of things. Maybe something on topic right now also. ;-)
- ----------------------------
Eustáquio "TaQ" Rangel
eustaqu...@yahoo.com
http://beam.to/taq
Usuário GNU/Linux no. 224050
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFB4qa/b6UiZnhJiLsRAik7AKCdcRhL6q2jgeu2YHc30haonwd1awCgr/7b
aRNxWaC9MACCcfgYSUUTcqk=
=X3Fl
-----END PGP SIGNATURE-----
Florian Gross
> Forgive me if I misunderstood, but so VALUEs are variables?
>
> a = 1
>
> a is the VALUE, with 32 bit length?
a is mapped to a VALUE. VALUEs are pointers to Ruby Object Structs.
Immediate objects are not represented by any Ruby Object Struct -- they
are identified directly by the pointer's destination. If Ruby sees such
a special pointer it does not need to resolve it.
Because there is no actual Ruby Object Struct there are no flags (which
means you can't (un)taint them), no instance variables and no singleton
classes for immediate values. (false, true and nil act as if FalseClass,
TrueClass and NilClass were their meta classes.)
Floats and Bignums are not actually immediate values, but they are
immutable and act as if they were immediate by default in some other
contexts. (You can't define singleton methods on them because they use a
custom singleton_method_added callback that raises an exception. It's
implemented by Numeric#singleton_method_added.)
false, nil and true and the special undef value that is not visible
anywhere in Ruby are represented by VALUEs of 0, 2, 4 and 6.
Fixnums are represented by VALUEs with bit 0 set to 1. This means that
there can be up to 2147483648 possible values (-1073741824 ..
1073741823) on 32 bit and up to 9223372036854775808 possible values
(-4611686018427387904 .. 4611686018427387903) on 64 bit systems.
Symbols are represented by VALUEs with bit 0 to 7 set to 01110000. This
means that there can be up to 16777216 different Symbols on 32 bit and
up to 72057594037927936 different Symbols on 64 bit systems.
If I'm wrong with any of those please correct me. It would also be
interesting if somebody could find out what values VALUEs that are not
immediate Objects can have.
itsme213
"Bertram Scharpf" <li...@bertram-scharpf.de> wrote
> Am Montag, 10. Jan 2005, 21:09:03 +0900 schrieb Kaspar Schiess:
> > Ruby variables hold references to objects.
>
> Stricly spoken, Fixnums don't. They are treated a special
> way.
You can try to work the angle that value-like things (Fixnums) are
fundamentally different from objects, but I think there is a simpler
explanation. Along the general lines some others have said in this thread,
but with some differences.
A variable always 'holds' ('contains', 'its value is') a _reference_ to an
object. This is true for local variables, instance variables, encoded
instance variables (see below), integer-indexed variables array members,
key-indexed variabels like hash entries, global variables. In fact, it is
even true for Constants; I like to think of all of these as 'slots'; some
slots may be 'frozen', ... i.e. they can never refer to a different object
(see below).
Most references are encoded as in-memory pointers to heap-allocated storage
corresponding to that object. That heap allocated storage, in turn, contains
a its own 'variables' (called instance variables), which also always 'hold'
references to objects. Methods on these objects can refer to their instance
variables (and to methd args, and to globals and constants).
Some references are optimized and encoded differently.
Fixnums are encoded as 2's complement bit strings (for example).
Thus, '0000' is an encoded _reference_ to the fixnum '0'.
And '0001' is an encoded _reference_ to the fixnum '1'.
Such encodings also encode a set of 'instance variables' that those objects
would have explicitly stored had they been represented on the heap like
normal objects. For example, the encoding of Fixnum '1' implicitly encodes
references to its adjacent fixnums '0' and '2'. The number '0' existed well
before the bit pattern '0001' ever appeared in your Ruby program.
Of course, methods on such objects need access to these 'instance
variables', so the methods are also optimized so they can simply work with
the encoded reference (rather than with the object itself), and again
guarantee to return (possibly encoded) references to objects.
Fixnum# + (other_fixnum)
return a (encoded, optimized) reference to another fixnum
e.g. '0001' + '0001' => '0010'
The discussion of Fixnums are 'immutable' is right in spirit, but in detail
bears closer inspection. Certain 'encoded' instance variables of Fixnums are
frozen. Specifically, their relationships to all other fixnums dictated by
the laws of math. i.e. these encoded instance variables are automaticaly
'frozen' (there was a long earlier thread on this topic). After all, you
would not want 2+1 to change from 3 to 73 in the middle of your program.
However, Fixnums can certainly have other mutable 'instance variables'; we
just have to handle the internal implementation differently because the
objects themselves are not heap allocated, so we need some other means to
get to these 'inst-vars'.
class Fixnum
@@foos = {}
def foo; return @@foos[self]; end
def foo=(x); @@foos[self]=x; end
end
irb(main):028:0> 1
=> 1
irb(main):029:0> 1.foo
=> nil
irb(main):030:0> 1.foo= 5
=> 5
irb(main):031:0> 1.foo
=> 5
Whatver works for you ...
Robert Klemme
"itsme213" <itsm...@hotmail.com> schrieb im Newsbeitrag
news:MtyEd.23363$3m6....@fe2.texas.rr.com...
>
> "Bertram Scharpf" <li...@bertram-scharpf.de> wrote
> > Am Montag, 10. Jan 2005, 21:09:03 +0900 schrieb Kaspar Schiess:
> > > Ruby variables hold references to objects.
> >
> > Stricly spoken, Fixnums don't. They are treated a special
> > way.
>
> You can try to work the angle that value-like things (Fixnums) are
> fundamentally different from objects, but I think there is a simpler
> explanation. Along the general lines some others have said in this
thread,
> but with some differences.
>
> A variable always 'holds' ('contains', 'its value is') a _reference_ to
an
> object. This is true for local variables, instance variables, encoded
> instance variables (see below), integer-indexed variables array members,
> key-indexed variabels like hash entries, global variables. In fact, it
is
> even true for Constants; I like to think of all of these as 'slots';
some
> slots may be 'frozen', ... i.e. they can never refer to a different
object
> (see below).
Although I agree with the rest of your detailed explanation I beg to
differ on this one: "they" refers to variables and constants and *every*
variable and constant can be made to point to another object:
>> Foo = 1
=> 1
>> Foo
=> 1
>> Foo = 2
(irb):2: warning: already initialized constant Foo
=> 2
>> Foo
=> 2
You probably wanted to say that for some classes there is just one
instance of a specific value:
>> 1.id
=> 3
>> (3 - 2).id
=> 3
>> (200 / 100 / 2).id
=> 3
>> (0.5 * 2).to_i.id
=> 3
So the fixnum representing the number 1 is always represented by the same
instance. (As you point out below, technically it's more complicated
because the "instance" is different from other classes instances, but from
the perspective of the user this doesn't matter as long as he does not try
to define methods for 1.)
Although this works I'd rather not have this kind of instance variables
for fixnums because for me fixnums are atoms (i.e. building blocks of the
language) without internal structure (ok, you may replace "atom" with
"quark"...). Also there is the issue with garbage collection of these
"instance variables"...
Kind regards
robert
Florian Gross
> You can try to work the angle that value-like things (Fixnums) are
> fundamentally different from objects, but I think there is a simpler
> explanation. Along the general lines some others have said in this thread,
> but with some differences.
Fixnums are Objects, even if they are not represented by an actual
Object structure in Ruby. You can do method calls on them, access their
instance variables etc.
> A variable always 'holds' ('contains', 'its value is') a _reference_ to an
> object. This is true for local variables, instance variables, encoded
> instance variables (see below), integer-indexed variables array members,
> key-indexed variabels like hash entries, global variables. In fact, it is
> even true for Constants; I like to think of all of these as 'slots'; some
> slots may be 'frozen', ... i.e. they can never refer to a different object
> (see below).
I'd just say that Objects are in most cases represented by VALUEs. I'm
not sure what you mean with "encoded instance variables".
> Most references are encoded as in-memory pointers to heap-allocated storage
> corresponding to that object. That heap allocated storage, in turn, contains
> a its own 'variables' (called instance variables), which also always 'hold'
> references to objects. Methods on these objects can refer to their instance
> variables (and to methd args, and to globals and constants).
It does not only contain instance variables. There's also information
like klass, flags, string buffer pointers and more in the internal
Structs. (And not all Objects store their instance variables in those
internal Structs, see exivars.)
> Fixnums are encoded as 2's complement bit strings (for example).
> Thus, '0000' is an encoded _reference_ to the fixnum '0'.
> And '0001' is an encoded _reference_ to the fixnum '1'.
I don't quite get this. How is '0000' different from '0' and '0001' from
'1'? If you're talking about object ids you should have used object_id
= 1 + (fixnum << 1). (just prepend a '1' in binary representation.)
> Such encodings also encode a set of 'instance variables' that those objects
> would have explicitly stored had they been represented on the heap like
> normal objects. For example, the encoding of Fixnum '1' implicitly encodes
> references to its adjacent fixnums '0' and '2'. The number '0' existed well
> before the bit pattern '0001' ever appeared in your Ruby program.
I'm not sure if you're using an analogy there, but Fixnums certainly
don't store their neighbors at all. It's a well-known fact that 2+1 ==
3, you don't need to store that in an instance variable.
> Of course, methods on such objects need access to these 'instance
> variables', so the methods are also optimized so they can simply work with
> the encoded reference (rather than with the object itself), and again
> guarantee to return (possibly encoded) references to objects.
Methods take Objects and return Objects. They don't care if that Object
is encoded in a VALUE or actually backed up by an actual Object struct.
> The discussion of Fixnums are 'immutable' is right in spirit, but in detail
> bears closer inspection. Certain 'encoded' instance variables of Fixnums are
> frozen. Specifically, their relationships to all other fixnums dictated by
> the laws of math. i.e. these encoded instance variables are automaticaly
> 'frozen' (there was a long earlier thread on this topic). After all, you
> would not want 2+1 to change from 3 to 73 in the middle of your program.
Again, not sure if this is supposed to be an analogy or simplistic view
of things, but the result of 2+1 is certainly not dictated by instance
variables and there are no 'special frozen instance variables' in Ruby.
And 2+1 can be changed from 3 to 73. After all 2+1 is just calling the
plus method on 2 with the argument 1. And methods can be changed:
class Fixnum
alias :old_plus :+
def +(other)
return 73 if self == 2 and other == 1
old_plus(other)
end
end
> However, Fixnums can certainly have other mutable 'instance variables'; we
> just have to handle the internal implementation differently because the
> objects themselves are not heap allocated, so we need some other means to
> get to these 'inst-vars'.
> class Fixnum
> @@foos = {}
> def foo; return @@foos[self]; end
> def foo=(x); @@foos[self]=x; end
> end
This are no instance variables, this are class variables.
Why don't just use instance variables directly? (After all Fixnums can
have instance variables. They are not stored in the Object struct of
Fixnums, of course, as Fixnums have no Object structs. So where do they
go? There's a global exivar table for Objects that can not store their
instance variables in Object structs. They go there. You don't notice
this, of course, and that's a good thing.)
> irb(main):001:0> 1.instance_variable_set("@chosen_one", true)
> => true
> irb(main):002:0> 1.instance_variable_get("@chosen_one")
> => true
> irb(main):003:0> 2.instance_variable_get("@chosen_one")
> => nil
ts
F> Symbols are represented by VALUEs with bit 0 to 7 set to 01110000. This
F> means that there can be up to 16777216 different Symbols on 32 bit and
2**23 - 1
F> up to 72057594037927936 different Symbols on 64 bit systems.
Guy Decoux
Florian Gross
> F> Symbols are represented by VALUEs with bit 0 to 7 set to 01110000. This
> F> means that there can be up to 16777216 different Symbols on 32 bit and
>
> 2**23 - 1
Hm, why not 2 ** 24 - 1? (Is one bit used for a special purpose or am I
overlooking something?)
Thanks.
ts
F> Hm, why not 2 ** 24 - 1? (Is one bit used for a special purpose or am I
F> overlooking something?)
It must make the transformation ID ==> Symbol, Symbol ==> ID and if I'm
right it lost a bit ("sign" bit)
This give one free bit, if one day you want to hack ruby :-)
Guy Decoux
Ruth A. Kramer
> I'd just say that Objects are in most cases represented by VALUEs. I'm
> not sure what you mean with "encoded instance variables".
(Not "picking" on Florian in particular, just needed a convenient quote
to respond to.)
This thread confuses me, and I think it's at least partly by
"overloaded" use of the word "value". ;-)
For me, I'd like to separate address and value to the extent possible.
In my (surely incorrect) words (and based on my understanding of Ruby),
a variable representing an object holds the memory address of that
object. The value of that object is something different, for example,
if the object is the number 2, the value of that object is its numeric
value, 2.
Yes, you can say that the value of the variable holding the reference to
the object is its memory address, but, to me, that just increases the
(potential for) confusion.
Is there an agreed upon vocabulary for these things in Ruby? If so,
what is it, if not, should there be?
Randy Kramer
Florian Gross
> Florian Gross wrote:
>
>>I'd just say that Objects are in most cases represented by VALUEs. I'm
>>not sure what you mean with "encoded instance variables".
>
> (Not "picking" on Florian in particular, just needed a convenient quote
> to respond to.)
>
> This thread confuses me, and I think it's at least partly by
> "overloaded" use of the word "value". ;-)
Oh, note that me talking about VALUE was just the way Object's are
presented. VALUEs are a frequently-used way to refer to Objects. It's a
C type that Ruby uses internally. It's usually just a pointer to some
object data in the form of a RBasic-compatible struct. (Or a magic
number in the case of immediate objects.)
> For me, I'd like to separate address and value to the extent possible.
I guess one could say that addresses are represented by VALUEs.
I'm not sure of the relationship between values and VALUEs. Maybe you
could say that the value of a variable is indicated by a VALUE. But all
this VALUE stuff does not matter anyway when you're coding in Ruby. I
think it can safely be ignored until you write a C extension or decide
to hack at Ruby's internals via Ruby/DL. ;)
I hope I did not confuse anybody, even if this was not directed to me in
particular. :)
Florian Gross
> F> Hm, why not 2 ** 24 - 1? (Is one bit used for a special purpose or am I
> F> overlooking something?)
>
> It must make the transformation ID ==> Symbol, Symbol ==> ID and if I'm
> right it lost a bit ("sign" bit)
>
> This give one free bit, if one day you want to hack ruby :-)
Hmmm, maybe this is can one day be used for representing Characters in
an efficient way. 0 .. 8388607 might not be full Unicode range, but it
would be a nifty optimization for the UCS2 space at least.
Now I'm only wondering if the number of Symbols expands to
36028797018963967 on 64 bit systems or if it is fixed independently of
native bit width.
Thanks for your correction and explanation!
itsme213
"Florian Gross" <fl...@ccan.de> wrote in message
news:34frpjF...@individual.net...
> itsme213 wrote:
> > Fixnums are encoded as 2's complement bit strings (for example).
> > Thus, '0000' is an encoded _reference_ to the fixnum '0'.
> > And '0001' is an encoded _reference_ to the fixnum '1'.
>
> I don't quite get this. How is '0000' different from '0' and '0001' from
Very sorry, I mis-typed something crucial. I meant to say:
*References* to Fixnums are encoded as 2's complement bit strings (for
example)
Thus, the bitstring 0000 is an encoded _reference_ to the fixnum 0
And 0001 is an encoded _reference_ to the fixnum 1
And Fixnum#+
takes 2 references to fixnum objects
manipulates just the references
implicitly uses information about referenced Fixnum
encoded in these references
e.g. that 0001 refers to 1, 0010 refers to 2
determines that reference 0001 + 0010 = reference 0010
which, magically, is the reference to fixnum 3
and returns a _reference_ to fixnum 3.
> > Most references are encoded as in-memory pointers to heap-allocated
storage
> It does not only contain instance variables. There's also information
> like klass, flags, string buffer pointers and more in the internal
> Structs. (And not all Objects store their instance variables in those
> internal Structs, see exivars.)
I agree. And I think it would help to have a uniform term to refer to all
those that are relevant for explaining the _behavior_ of any object. For
example, I would like to understand object behavior as always conforming to
the following:
When any object x executes method m, it can only access the <local ivars,
array members, hash entries, class reference, ..., and the parameters passed
to the method call, and globals and constants>, and things accessible
through these. There is no other magic in an the behavior of any object. And
imho, it would be nice to have a uniform term for the <.....> above.
> Again, not sure if this is supposed to be an analogy or simplistic view
> of things, but the result of 2+1 is certainly not dictated by instance
> variables and there are no 'special frozen instance variables' in Ruby.
An analogy. To show that underneath some differences in implementation
optimizations, it is all about objects with ... hmmm. hmmm. what should I
call them? Those things that are sometimes "@name" variables, sometimes x[5]
indexed variables, sometimes x[:k], and sometimes the very highly optimized
2's-comp-0001.next = 2's-comp-0010.
In my mental framework I think of those them as 'slots'.
Even better, and I should have used this instead. Then clearly Fixnums are
_not_ immutable in general, since their instance variables (which **happen**
to be stored in a global exivar table for Objects, just as I **happened** to
choose an effectively global class_variable on Fixnums) can be modified.
Instead, just **some** of the information about any Fixnum is immutable
(those things that are dictated by the laws of math, and are encoded in the
e.g.2's complement representation of references to Fixnums).
And when I do your version of 1.instance_variable_set ...
I have set the i-var for **the** fixnum 1; there is NO other fixnum 1
anywhere, no matter how many other variables I have that also refer to
fixnum 1.
If we do nail this down, we can collectively have a consistent way to
describe the state of a collection of Ruby objects at any point in time
(including arrays, hashes, fixnums, symbols, strings, klass, ...) as: a set
of objects with slots that refer to other objects. And the only possible
changes to this state is (a) creation of new objects, and (b) changes to
slots.
But, all this is not necessary unless we want a clean and consistent answers
to some otherwise tricky underlying questions. We can quite happily and
productively code in Ruby without it, and either develop some alternative
consistent underlying explanation or just don't bother. I have no problem
with that either.
itsme213
> ts wrote:
>
> > F> Hm, why not 2 ** 24 - 1? (Is one bit used for a special purpose or am
I
> > F> overlooking something?)
> >
> > It must make the transformation ID ==> Symbol, Symbol ==> ID and if I'm
> > right it lost a bit ("sign" bit)
> >
> > This give one free bit, if one day you want to hack ruby :-)
>
> Hmmm, maybe this is can one day be used for representing Characters in
> an efficient way. 0 .. 8388607 might not be full Unicode range, but it
> would be a nifty optimization for the UCS2 space at least.
>
That would be great.
[btw: imho, what we are doing is exploiting the space of valid object
references to efficiently encode references to objects that we don't want to
every create explicitly].
itsme213
"Florian Gross" <fl...@ccan.de> wrote in message
news:34fqajF...@individual.net...
> Eustaquio Rangel de Oliveira Jr. wrote:
>
> > Forgive me if I misunderstood, but so VALUEs are variables?
> >
> > a = 1
> >
> > a is the VALUE, with 32 bit length?
>
> a is mapped to a VALUE. VALUEs are pointers to Ruby Object Structs.
> Immediate objects are not represented by any Ruby Object Struct -- they
> are identified directly by the pointer's destination. If Ruby sees such
> a special pointer it does not need to resolve it.
Fine. As long as we understand that the immediate object is not the value
stored in variable a, but the _implicit_ object that resides at the
destination pointed to by 'a'. This is just to have a consistent underlying
explanation:
x = y
# x and y refer to the _same_ object
u = 5
v = u
# u and v refer to the _same_ object
u + v
# special pointer, no need to de-reference; + knows how to interpret these
special pointers to do integer arithmetic, returning other special pointers.
Otherwise we would have needless tricky problems:
u = 5 # one immediate object
v = u # another immediate object ??
v.instance_variable_get "@foo" #=> nil
u.instance_variable_set "@foo", 20 # the 1st immediate object
v.instance_variable_get "@foo" #=> 20 ??? how come ?
> Because there is no actual Ruby Object Struct there are no flags (which
> means you can't (un)taint them), no instance variables
No instance variables ... unless created by Ruby code would be better.
> Floats and Bignums are not actually immediate values, but they are
> immutable and act as if they were immediate by default in some other
> contexts. (You can't define singleton methods on them because they use a
> custom singleton_method_added callback that raises an exception. It's
> implemented by Numeric#singleton_method_added.)
How are floats and bignums represented? I guess bignums are heap allocated
and gc'd. Floats too?
x = 2.5
y = 100 ** 100
> false, nil and true and the special undef value that is not visible
> anywhere in Ruby are represented by VALUEs of 0, 2, 4 and 6.
false, true, and nil are objects _referred to_ by 0, 2, 4, 6. Same reason as
above.
x = false; y = x
# x and y refer to the same object
Does object.id generally correspond to the way a reference to object is
represented? What are the exceptions?
> Symbols are represented by VALUEs with bit 0 to 7 set to 01110000. This
> means that there can be up to 16777216 different Symbols on 32 bit and
> up to 72057594037927936 different Symbols on 64 bit systems.
Presumably the symbols themselves (or their corresponding strings) are
allocated somewhere within, or mapped from, the address block with that
0...7bit mask?
Thanks for sharing your very useful insights on these!
itsme213
>
> > Forgive me if I misunderstood, but so VALUEs are variables?
> >
> > a = 1
> >
> > a is the VALUE, with 32 bit length?
>
> a is mapped to a VALUE. VALUEs are pointers to Ruby Object Structs.
> Immediate objects are not represented by any Ruby Object Struct -- they
> are identified directly by the pointer's destination. If Ruby sees such
> a special pointer it does not need to resolve it.
That works better if readers understand that the immediate object is not the
pointer value in 'a', but _implicitly_ lives at the destination referred to
by 'a'. This is what I meant by references to Fixnums are encoded in an
optimized way e.g. 2's complement bit strings.
Otherwise
x = 5
y = 5
Would mean two immediate objects. How then to explain
x.instance_variable_set "@foo", 20
y.instance_variable_get "@foo" #=> 20 ??
x = y # x refers to the same object y referred to
u = 5 # u refers to the fixnum 5
v = u # v refers to the same object that u referred to
C = v # C refers to the same object that u and v refer to
> Because there is no actual Ruby Object Struct there are no flags (which
> means you can't (un)taint them), no instance variables
No instance variables? You probably mean "no instance variables unless
created by Ruby code".
> Floats and Bignums are not actually immediate values, but they are
> immutable
How are Floats and Bignums represented?
x = 2.5
y = 10 ** 10
> false, nil and true and the special undef value that is not visible
> anywhere in Ruby are represented by VALUEs of 0, 2, 4 and 6.
Great to know. That is what I guessed from false.id. I would say that
_references_ to the objects false, true, and nil are represented by the bit
strings 0, 2, and 4 (since all Ruby code can do is handle references to
objects).
Is the stored reference to an object generally the same as object.id ? How
do they correspond?
> Fixnums are represented by VALUEs with bit 0 set to 1.
_references_ to Fixnums ?
Good to know. So some fixnums might need some bit manipulation before
passing off to, say, C-routines for integer arithmetic.
> Symbols are represented by VALUEs with bit 0 to 7 set to 01110000.
_references_ to Symbols ? Presumably symbols (or their associated strings)
are actually allocated somewhere within (or mapped from) a block of memory
addresses with the bits 0..7 mask.
> If I'm wrong with any of those please correct me. It would also be
> interesting if somebody could find out what values VALUEs that are not
> immediate Objects can have.
_references_ to non-immediate Objects?
Thanks for sharing all these valuable insights!
why the lucky stiff
> Florian Gross wrote:
> > I'd just say that Objects are in most cases represented by VALUEs. I'm
> > not sure what you mean with "encoded instance variables".
>
> (Not "picking" on Florian in particular, just needed a convenient quote
> to respond to.)
>
> This thread confuses me, and I think it's at least partly by
> "overloaded" use of the word "value". ;-)
>
Hi, Ruth. So, when someone on the list uses uppercase VALUE, they are
refering to an object's symbol table id. The symbol table being the
global dictionary that pairs up a unique id with pointers to the actual
objects. VALUE is a C typedef.
Sometimes when I'm conversing with other languages, such as in my Syck
extension, I'll rename VALUE as SYMID, in the hopes of clearing things
up a bit.
So, yeah, it's a generic word, but it's also referring to the most
common and generic form of data used inside Ruby.
_why
Florian Gross
>>a is mapped to a VALUE. VALUEs are pointers to Ruby Object Structs.
>>Immediate objects are not represented by any Ruby Object Struct -- they
>>are identified directly by the pointer's destination. If Ruby sees such
>>a special pointer it does not need to resolve it.
>
> Fine. As long as we understand that the immediate object is not the value
> stored in variable a, but the _implicit_ object that resides at the
> destination pointed to by 'a'. This is just to have a consistent underlying
> explanation:
I dislike this whole "stored at" model. It's too complex. Let's express
it this way:
variables are names for Objects. In Ruby's implementation Objects are
represented by VALUEs.
>>Because there is no actual Ruby Object Struct there are no flags (which
>>means you can't (un)taint them), no instance variables
>
> No instance variables ... unless created by Ruby code would be better.
They're not saved in the Object struct, however. They're stored externally.
> How are floats and bignums represented?
Pointers to RFloat (klass, flags, value as double) and RBignum (klass,
flags, sign, length, pointer to digits). Whenever you create a new Float
or Bignum you get a new Object.
>>false, nil and true and the special undef value that is not visible
>>anywhere in Ruby are represented by VALUEs of 0, 2, 4 and 6.
>
> false, true, and nil are objects _referred to_ by 0, 2, 4, 6. Same reason as
> above.
I have no idea what distinction you are trying to make there.
> Does object.id generally correspond to the way a reference to object is
> represented? What are the exceptions?
I think .object_id just returns the destination of the VALUE pointer as
a Numeric. Don't trust me on this, though.
>>Symbols are represented by VALUEs with bit 0 to 7 set to 01110000. This
>>means that there can be up to 16777216 different Symbols on 32 bit and
>>up to 72057594037927936 different Symbols on 64 bit systems.
>
> Presumably the symbols themselves (or their corresponding strings) are
> allocated somewhere within, or mapped from, the address block with that
> 0...7bit mask?
No, the remaining bits are used for storing the symbol ID which is based
on the symbol type (global, instance variable...), an id counter and
some bit shifting. Don't ask me why the symbol ID is generated that way.
I don't know. There's two tables, one for going from a String to a
symbol ID and one for the other way. Those are used for Symbol#inspect
and so on.
itsme213
> >>Immediate objects are not represented by any Ruby Object Struct -- they
> >>are identified directly by the pointer's destination. If Ruby sees such
> >>a special pointer it does not need to resolve it.
> >
> > Fine. As long as we understand that the immediate object is not the
value
> > stored in variable a, but the _implicit_ object that resides at the
> > destination pointed to by 'a'. This is just to have a consistent
underlying
> > explanation:
>
> I dislike this whole "stored at" model. It's too complex. Let's express
> it this way:
I used "value stored in variable" because you said:
'VALUEs are pointers to ...'
'they are identified directly by the pointer's destination'
Everything I said works fine with 'the value of a variable' instead of
'value stored in variable'.
> variables are names for Objects.
That's ok too. But you will probably need to allow somewhere for inst-vars
and arr[i] and hash[k] and ... to also be (names for? a different term
here?) Objects. What would state in this object model be? a mapping of names
to Objects? I think not ... each object would itself be a mapping of names
to objects (via inst vars, arr[i]...). And state change and behavior of
methods would in turn have to be explained consistent. A name and name-map
explanation can be made to work consistently, and some formal object model
have such a basis.
But I have found that "slot that refers to an object", where slots can be
named, indexed, or other special slots, is simpler for most (including
learners); I'm now thinking that either my experience was an aberration or I
am doing a dismal job of explaining it even to Ruby wizards like yourself.
Let's go with "variables are names for Objects"
x = y
x and y are now names for the same object. Entirely independent of what
class of object, or of internal implementation. Would you agree?
> In Ruby's implementation Objects are
> represented by VALUEs.
But this throws me off. If Objects are represented by VALUEs, and VALUEs are
pointers to Ruby Object Structs, then creating a new object means creating a
new "pointer to Ruby Object Struct". Does x=Object.new create a new "pointer
to Ruby Object Struct"? To which one does the new pointer point? Then does
x=5 and y=5 create two pointers? Just trying to understand how your
terminology would work.
Could this not also lead to difficulty with your 'Immediate objects'.
x = 5
# is x a name for an immediate object here?
y = 5
# is y a name for an immediate object? the same immediate object? when did
this immediate object come into existence?
I really not trying to be dense, or obtuse, or academic :-( . just trying to
get the bottom of it clear.
> >>Because there is no actual Ruby Object Struct there are no flags (which
> >>means you can't (un)taint them), no instance variables
> >
> > No instance variables ... unless created by Ruby code would be better.
>
> They're not saved in the Object struct, however. They're stored
externally.
So? I think we are interleaving and mixing a discussion of Ruby's internal
implementation (which I appreciate your insights into, but which only a
minority of Ruby programmers will need to understand) with an attempt to
come up with a consistent explanation of the conceptual object model of Ruby
e.g. for someone not used to a pure object model.
> > Does object.id generally correspond to the way a reference to object is
> > represented? What are the exceptions?
>
> I think .object_id just returns the destination of the VALUE pointer as
> a Numeric. Don't trust me on this, though.
Ok. Is VALUE a struct or typedef defined in the C code?
> No, the remaining bits are used for storing the symbol ID which is based
> on the symbol type (global, instance variable...), an id counter and
> some bit shifting. Don't ask me why the symbol ID is generated that way.
> I don't know. There's two tables, one for going from a String to a
> symbol ID and one for the other way. Those are used for Symbol#inspect
> and so on.
Thanks for that.
Back to the conceptual object model: If we now try to explain
x = :foo
x.inspect
in terms of an implementation-independent object model, we would have to
pretend that there were Symbol objects (just like any other), and that they
had some instance variables (or 'names', if Object is a map from name to
objects) for the string. In fact, we could easily even implement reflective
methods like Symbol#instance_variable_get :@theString (in C) to poke around
those tables and actually return a (frozen) string. We would not want to
correspondingly implement #set because Symbols need that "instance variable"
to be frozen (yes, I know it is not an inst-var at the C-level; and that
Ruby does not have a notion of freezing an "instance variable").
Time to go kick my explanation skills into some semblance of respectability.
And write some Ruby code.
Tim Sutherland
[...]
>Hi, Ruth. So, when someone on the list uses uppercase VALUE, they are
>refering to an object's symbol table id. The symbol table being the
>global dictionary that pairs up a unique id with pointers to the actual
>objects. VALUE is a C typedef.
There is no such global dictionary, all the information is encoded in the
VALUE.
See http://www.rubygarden.org/ruby?GCAndExtensions:
Ruby objects are represented by VALUE in the Ruby C API. This is a typedef
for unsigned long. The value 0 is used for false and 4 is used for nil.
Otherwise, if the two least significant bits are 00 it is a pointer to a
structure on the heap which represents the Ruby object. (i.e. if "value &
0x03 == 0.") Other values are immediate objects or special constants (small
integers, symbols, true). This works because the structures are always
quad-aligned in memory. (So the pointer is always a multiple of 4, i.e.
has lower bits 00.) Then for example FIX2INT converts a Fixnum to a C int
by right-shifting the VALUE.
Florian Gross
>>I dislike this whole "stored at" model. It's too complex. Let's express
>>it this way:
>
> I used "value stored in variable" because you said:
> 'VALUEs are pointers to ...'
> 'they are identified directly by the pointer's destination'
But VALUE != value. VALUEs are in Ruby's internals what Objects are in Ruby.
>>variables are names for Objects.
>
> That's ok too. But you will probably need to allow somewhere for inst-vars
> and arr[i] and hash[k] and ... to also be (names for? a different term
> here?) Objects. What would state in this object model be? a mapping of names
> to Objects? I think not ... each object would itself be a mapping of names
> to objects (via inst vars, arr[i]...). And state change and behavior of
> methods would in turn have to be explained consistent. A name and name-map
> explanation can be made to work consistently, and some formal object model
> have such a basis.
Of course there's also other things that refer to Objects. Arrays
contain Objects of course. Hashs map Objects to other Objects. Instance
variables are just a funky version of variables that are local to an Object.
I'm not sure where you're going with that whole Slot model. I think it
only applies for languages like Python and JavaScript where everything
is based on fields. (In JavaScript you can do obj["foo"] = function() {
... } and then call it via obj.foo().)
> Let's go with "variables are names for Objects"
> x = y
> x and y are now names for the same object. Entirely independent of what
> class of object, or of internal implementation. Would you agree?
Yup, that's how I'd express it.
>>In Ruby's implementation Objects are
>>represented by VALUEs.
>
> But this throws me off. If Objects are represented by VALUEs, and VALUEs are
> pointers to Ruby Object Structs, then creating a new object means creating a
> new "pointer to Ruby Object Struct". Does x=Object.new create a new "pointer
> to Ruby Object Struct"? To which one does the new pointer point? Then does
> x=5 and y=5 create two pointers? Just trying to understand how your
> terminology would work.
VALUEs are pointers to Objects. But sometimes they mean something
different just on where they point. Ruby sees that the pointer has a
special format and can then skip dereferencing it because it already
knows what Object it stands for. This is the case with immediate
objects. They are stored directly in the VALUE. Tim Sutherland explained
this fairly well in his reply to _why's posting.
> Could this not also lead to difficulty with your 'Immediate objects'.
> x = 5
> # is x a name for an immediate object here?
> y = 5
> # is y a name for an immediate object? the same immediate object? when did
> this immediate object come into existence?
There's no actual C struct behind the Object to back it up. Yet in Ruby
you won't even feel the difference, because it's just an Object. It acts
like an Object and all that. Because of how immediate values work in
current Ruby you will always get the same Fixnum for the above.
>>>>Because there is no actual Ruby Object Struct there are no flags (which
>>>>means you can't (un)taint them), no instance variables
>>>No instance variables ... unless created by Ruby code would be better.
>>They're not saved in the Object struct, however. They're stored externally.
>
> So? I think we are interleaving and mixing a discussion of Ruby's internal
> implementation (which I appreciate your insights into, but which only a
> minority of Ruby programmers will need to understand) with an attempt to
> come up with a consistent explanation of the conceptual object model of Ruby
> e.g. for someone not used to a pure object model.
Yes, I agree in that this whole discussion does not make much sense for
regular Ruby programmers. All you need to know is that everything is an
Object, that Ruby does provide a few built-in (mostly) immutable Objects
like Numerics and Symbols. (Immutable Objects are usually called value
objects in OOP terminology.) There's lots of ways that you or Objects
can refer to Objects, but you'd expect this anyway, and if you pass an
Object around you will not magically get a copy of it like in other
languages.
> Ok. Is VALUE a struct or typedef defined in the C code?
typedef for long. This is why Ruby can only be compiled when
sizeof(long) == sizeof(void*).
> Back to the conceptual object model: If we now try to explain
> x = :foo
> x.inspect
> in terms of an implementation-independent object model, we would have to
> pretend that there were Symbol objects (just like any other), and that they
> had some instance variables (or 'names', if Object is a map from name to
> objects) for the string. In fact, we could easily even implement reflective
> methods like Symbol#instance_variable_get :@theString (in C) to poke around
> those tables and actually return a (frozen) string. We would not want to
> correspondingly implement #set because Symbols need that "instance variable"
> to be frozen (yes, I know it is not an inst-var at the C-level; and that
> Ruby does not have a notion of freezing an "instance variable").
Actually for the conceptual object model there's just Symbols which are
immutable Objects. Everything else is pedantry.
Ruth A. Kramer
Just for the record, it's Randy. (I've been stuck using my wife's email
client on the fast.net address since my Linux mail server crashed.)
> So, when someone on the list uses uppercase VALUE, they are
> refering to an object's symbol table id. The symbol table being the
> global dictionary that pairs up a unique id with pointers to the actual
> objects. VALUE is a C typedef.
Thanks, that helps! (I'll probably just try to ignore VALUE when I see
it.)
Randy Kramer
itsme213
"Florian Gross" <fl...@ccan.de> wrote ...
I grok what you mean and its certainly good enough, thanks.
georgesawyer
at 12:50 AM wrote,
>Ok, but how it does that? I mean, I have 31 (32?) bit numbers there on my
one thousand vars on the local scope. Where they are stored? They are
somewhere on the memory, right? How it can take no storage?
Well, I think it would be a good suggestion to take a quick look at
machine language instructions and assembly!
Cordially,