Integers are stored inconsistently - Perl driver

[Manoj]

Hi Mongos,

Here is my issue, When I try to store large integers in to mongo it
store it differently based on how large the number is. Here is a
simple program.

PS: My mongo server is running in 64bit and Perl API running in 32
bit.


use MongoDB;
use Data::Dumper;
my $connection = MongoDB::Connection->new(host => "mongodb://
mongo01:27017, find_master => 1);
#my $ext = { num => 253335, };
#my $ext = { num => 2533359821, };
my $ext = { num => 28347197234178, };
my $database = $connection->foo;
my $collection = $database->bar;
my $id = $collection->insert($ext);
my $data = $collection->find_one({ _id => $id });
warn Dumper($data);


In this the first number and last number stored correctly in mongo.
But the second number stored as some wrong negative integer.

Any idea why?

Regards
Manoj

[Riyad]

Manoj,

Does it happen to be storing it as -385876174 or roughly there abouts? If so it looks like it is treating that 2nd number as a 32-bit int (max value of 2,147,483,647) where as it is correctly treating the 2nd number as a 64-bit long (And thus storing it correctly).

I am not familiar with perl or the mongo-perl driver, but if you can force the type as long, it should store correctly.

[Manoj]

The value I can see through the mongo shell is -1761607475.
What you mean by force the type as log? how can I do it in such a way
that it wont force 2 digit number as long.
If you are talking about "NumberLong()" then I dont want that, because
http://groups.google.com/group/mongodb-user/browse_thread/thread/d64556b1bda2b455/5f4da4e08fe17238

[Kristina Chodorow]

You have to store it as a 64-bit int (NumberLong), it doesn't fit in
32 bits. To force it to be stored as a 64-bit int, you can use bigint
or the Math::BigInt class (see http://search.cpan.org/dist/MongoDB/lib/MongoDB/DataTypes.pod#Numbers).
Try to avoid using $where, NumberLongs will compare correctly doing
queries like {num => 123} (instead of {'$where' => 'this.num ==
123'}).

On Oct 25, 7:35 pm, Manoj <mano...@gmail.com> wrote:
> The value I can see through the mongo shell is  -1761607475.
> What you mean by force the type as log? how can I do it in such a way
> that it wont force 2 digit number as long.

> If you are talking about "NumberLong()" then I dont want that, becausehttp://groups.google.com/group/mongodb-user/browse_thread/thread/d645...

[Manoj]

Thanks Kristina.

I dont want to use Math::BigInt because we are trying to avoid too
many modules imported in to the script because of some performance
issue. We probably store it as a string or try to convert to Mongo's
MongoDB::BSON::Long.

One more thing I noticed...

When I store a number (from 32bit client to 64 bit server) say {a =>
2} mongo shell display as "a" : 2. But when I do the same in (64bit
client to 64bit server) mongo shell displays as "a" : NumberLong(2).

Any idea why? or is this just a javascript display issue?

On Oct 28, 1:54 am, Kristina Chodorow <k.chodo...@gmail.com> wrote:
> You have to store it as a 64-bit int (NumberLong), it doesn't fit in
> 32 bits.  To force it to be stored as a 64-bit int, you can use bigint

> or the Math::BigInt class (seehttp://search.cpan.org/dist/MongoDB/lib/MongoDB/DataTypes.pod#Numbers).

> Try to avoid using $where, NumberLongs will compare correctly doing
> queries like {num => 123} (instead of {'$where' => 'this.num ==
> 123'}).
>
> On Oct 25, 7:35 pm, Manoj <mano...@gmail.com> wrote:
>
>
>
>
>
>
>
> > The value I can see through the mongo shell is  -1761607475.
> > What you mean by force the type as log? how can I do it in such a way
> > that it wont force 2 digit number as long.
> > If you are talking about "NumberLong()" then I dont want that, becausehttp://groups.google.com/group/mongodb-user/browse_thread/thread/d645...
>
> > On Oct 26, 10:22 am, Riyad <rka...@gmail.com> wrote:
>
> > > Manoj,
>

> > > Does it happen to bestoringit as -385876174 or roughly there abouts? If

> > > so it looks like it is treating that 2nd number as a 32-bit int (max value
> > > of 2,147,483,647) where as it is correctly treating the 2nd number as a

> > > 64-bit long (And thusstoringit correctly).

[Manoj]

Some more large number related differences..

When I store (64bit client, 64 bit server)

my $num = 2147483649;
my $ext = { '_id' => 15 ,num => bless (\$num, "MongoDB::BSON::Long"),
ast => 56 };

Mongo shell displays

{ "_id" : NumberLong(15), "ast" : NumberLong(56), "num" :
BinData(2,"CgAAADIxNDc0ODM2NDk=") }

Data::Dumper displays

$VAR1 = {
'num' => '2147483649',
'_id' => 15,
'ast' => 56
};

I imagine mongo shell displays that as object but while retrieving, it
is converted as string?

What is your opinion about storing large number as object of
MongoDB::BSON::Long? Any performance issue? Any problem in querying,
sorting, comparison, indexing?

[Kristina Chodorow]

> When I store a number (from 32bit client to 64 bit server) say {a => > 2}  mongo shell display as "a" : 2. But when I do the same in (64bit > client to 64bit server) mongo shell displays as "a" : NumberLong(2). > Any idea why? or is this just a javascript display issue? 

Yes, sort of. 64-bit Perl will store numbers as 64-bit numbers. 32-
bit Perl will store numbers as 32-bit numbers.

Javascript only has one number type. MongoDB has three (32-bit int,
64-bit int, 64-bit floating point). The shell will display floating
point numbers and 32-bit ints "normally" because there's no risk of
loss of precision. NumberLong(987) means it's stored as a 64-bit int.

There is no MongoDB::BSON::Long type in the Perl driver, it's just
storing it as a binary string. This will cause it to behave like a
string, not a number, in the database (e.g., range queries and sorts
won't work properly).

[Kristina Chodorow]

They might either be doubles or 32-bit integers.  You could use the $type operator to check which: http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24type

As far as I can think of, there's no way to control which is saved.

It shouldn't have much impact on document size.  Doubles are 64-bits, so they are the same size as 64-bit integers.  Integers are 32 bits, so they'd be 2 bytes instead of 4.

On Thursday, September 20, 2012 12:57:17 PM UTC-4, Dylan Doxey wrote:

I have taken up the practice of adding zero to my numeric values before inserting them via the MongoDB Perl driver.

Being that everything is running on 64 bit architecture it's no surprise that most all the int type values are appearing as NumberLong(...).

However today I'm surprised to see some ints appearing in documents as non-string numeric values which are also not NumberLong.

Does this imply a storage difference?

Is there a way to control this explicitly?

How does this impact document size?