Force data type

[J Y]

Is there a way to convert a value to a type manually? I have a string
that has the value of "5.1", and not to be interpreted as a float.
But it got interpreted as a double when I try to send this into the DB
(5.099999904632568). It seems like CFStrictTyper is doing a regex to
try and detect numbers, is there a way to override and have it go in
as an actual string?

[Marc Esher]

Hey JY,

Give me the weekend to fix this. Thanks.

Marc

> --
> You received this message because you are subscribed to the Google Groups "CFMongoDB" group.
> To post to this group, send email to cfmo...@googlegroups.com.
> To unsubscribe from this group, send email to cfmongodb+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/cfmongodb?hl=en.
>
>

[J Y]

Wow, thanks! I would be happy to help if there's anything I can do.
Right now I am putting a "!" in front of the numbers to force it to go
in as a string.

[Marc Esher]

Give it a shot now.

Marc

[J Y]

Thanks for the update. I played with this for a bit, and while the
number did go in as 5.1 (which means comparisons would work
correctly), it is not what my original intentions were. The issue I
have right now is that 5.1 should be a string; that means if I have
5.1, 5.1.1, and 5.2, and I choose to perform a sort, the order should
be:

5.1
5.1.1
5.2

But right now, because the numbers are still interpreted as numeric, I
end up getting:

5.1
5.2
5.1.1

I hope this is clear. I think what's needed is a way to bypass the
automatic type conversion, so that the types can be set manually.

[Marc Esher]

Right now, to bypass cfmongodb's type conversion, you'll want to get a
com.mongodb.BasicDBObject from the object factory and populate it with
your struct, something like:

var dbo = mongo.getMongoConfig().getMongoFactory().getObject("com.mongodb.BasicDBObject");
dbo.put( {yourstruct} );

[phofmann]

Hi,

Is there a way to bypass cfmongodbs type conversion when using .query().$eq(...).find()? My problem is, that I want to use $eq to search for a string consisting only of numbers...

Cheers
Pascal

[Marc Esher]

Hi Pascal,

Have you tried the tip above for creating a regular Mongo BasicDBObject?

var dbo = mongo.getMongoConfig().getMongoFactory().getObject("com.mongodb.BasicDBObject");
dbo.put( {yourstruct} );

--

You received this message because you are subscribed to the Google Groups "CFMongoDB" group.

To view this discussion on the web visit https://groups.google.com/d/msg/cfmongodb/-/nkYargepBboJ.

[phofmann]

Hi,

I just learned that MongoDb also uses indexes when using regex() (with '^value$'). So I can use regex('field', '^value$') instead of eq('field', 'value').

Cheers
Pascal

[Marc Esher]

Good to know!

On Fri, Jan 4, 2013 at 2:35 AM, phofmann <phof...@trustinternational.com> wrote:

Hi,

I just learned that MongoDb also uses indexes when using regex() (with '^value$'). So I can use regex('field', '^value$') instead of eq('field', 'value').

Cheers
Pascal

On Wednesday, January 2, 2013 4:47:10 PM UTC, Marc Esher wrote:

Hi Pascal,

Have you tried the tip above for creating a regular Mongo BasicDBObject?

var dbo = mongo.getMongoConfig().getMongoFactory().getObject("com.mongodb.BasicDBObject");
dbo.put( {yourstruct} );

On Wed, Jan 2, 2013 at 10:47 AM, phofmann <phof...@trustinternational.com> wrote:

Hi,

Is there a way to bypass cfmongodbs type conversion when using .query().$eq(...).find()? My problem is, that I want to use $eq to search for a string consisting only of numbers...

Cheers
Pascal

--
You received this message because you are subscribed to the Google Groups "CFMongoDB" group.

To view this discussion on the web visit https://groups.google.com/d/msg/cfmongodb/-/nkYargepBboJ.

To post to this group, send email to cfmo...@googlegroups.com.
To unsubscribe from this group, send email to cfmongodb+...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/cfmongodb?hl=en.

--
You received this message because you are subscribed to the Google Groups "CFMongoDB" group.

To view this discussion on the web visit https://groups.google.com/d/msg/cfmongodb/-/ywUE2Aq_BlQJ.

[Peter Zhang]

Hi Marc,

I modified cfmongodb a little bit to have the ability to turn on/off auto data conversion, as well as setting data type manually. It works for insert/save/saveAll/update/remove operations.

save(doc, handleValueType, def)

saveAll(doc, handleValueType, def)

insert(doc, handleValueType, def)

update(doc, handleValueType, def)

remove(doc, handleValueType, def)

handleValueType:

0 - manually handle data type conversion.

1 - auto handle all data type conversion (default)

2 - auto handle only simple data type conversion (String)

3 - auto handle only complex data type conversion (Array, Struct)

def:

A list of key:value pair that defines how data conversion should be handled if we don't need auto conversion. It support one level, all level and specific level.

Example:

doc = {

   "key1" = "1",

   "key2" = "2",

   "keys" = {

      "key1" = "1",

      "key2" = "2"

   }

}

// one level

save(doc, 0, "key2:int")

// result in db: doc = { "key1" = "1", "key2" = 2, "keys" = { "key1" = "1", "key2" = "2" } }

// specific level

save(doc, 0, "key2:int,keys/key1:int");

// result in db: doc = { "key1" = "1", "key2" = 2, "keys" = { "key1" = 1, "key2" = "2" } }

// all levels

save(doc, 0, "*/key2:int")

// result in db: doc = { "key1" = "1", "key2" = 2, "keys" = { "key1" = "1", "key2" = 2 } }

[Marc Esher]

Peter,

Can you submit these changes as a pull request in github?

Also, have you run all the tests and example code to ensure that these changes do not at all affect the existing API?

Thanks,

Marc

To unsubscribe from this group and stop receiving emails from it, send an email to cfmongodb+...@googlegroups.com.

To post to this group, send email to cfmo...@googlegroups.com.

Visit this group at http://groups.google.com/group/cfmongodb?hl=en.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Peter Zhang]

Hi Marc,

I updated the code and simplified the process. I've submitted the changes as a pull request with detailed description.

I've run all the examples and all work. I've not run tests yet.

Thanks,

Peter

[Peter Zhang]

Hi Marc,

Update: ran run.cfm with all 62 successes

[Marc Esher]

Thanks Peter. We're reviewing this and should have responses within a few days. This is a huge change and we need to be sure it's the correct approach. We do appreciate all the effort you put into this!

Marc

[Marc Esher]

Peter,

Again, I appreciate your contribution here. I've added a lot of commentary on the pull request: https://github.com/marcesher/cfmongodb/pull/36

Feedback from all is welcome!

Marc

[J Y]

Hi Marc,

Disclaimer: Peter is my colleague and we are working on this problem together.

Thank you for your feedback.  I agree with a lot of what you said and am thinking of ways on how to implement what you described.  I appreciate the effort that you have put into this.  And thank you Peter for taking the time to work on this and figuring this out.

A lot of the work we have been doing involves strings that look a lot like numbers sometimes, but other times it's obvious it's a string.  It's for this reason that prompted us to look into more granular type control. Before, I would append non-printable characters to the string to force cfmongodb to see it as a string, but this is a hack at best and can hurt performance when we had to process large amounts of data to remove the appended character.  This is the reason that made us look into this problem.

For the most part, the default type determination mechanism in cfmongodb works perfectly.  It's just those 1% that we are trying to account for, without having to go outside of cfmongodb completely so we can process one collection where this is needed.  And as you said, I would hate to litter my code with javacast all over the place just because one field is the bad apple.

Having said that, I want to bang heads with you on your suggestions.  I like your  idea of being able to subclass CFStrictTyper with another class, as this would not affect existing performance.  If your preference is to go this route, we can certainly try this way and see how it goes.

Given your preference for minimal change and impact, I had another idea that may be a little counter-intuitive to use, but would require less change to the existing implementation.  The way I see it, the only types that could cause ambiguity are strings, numbers, and dates.  Arrays and structs are fairly set.  Dates are already handled with the date object.  So that leaves only strings and numbers.  Since CFStrictTyper.toJavaType() only deals with string, list, and map, we could add a 4th type, as an example, StringBuffer, that just return the value as a string.

  public Object toJavaType(Object value){
      ...
      } else if ( value instanceof StringBuffer ) { // force a value to be string
          return setAsString(value);
      }

      return value;
  }



 /* handle string */
  public Object setAsString(Object value) {
          return value.toString();
  }


In the struct, when we want to force a type to be string, we would just manually create it as StringBuffer (through a convenience method in MongoUtil, perhaps).  This way, only CFStrictTyper is affected, and we can put the conditional for StringBuffer checking after all the others, so it should not affect performance of existing types. 

The problem I see with this is that it's a potential behavior change if someone had been using StringBuffer to create the number, this will break their code.   This could be mitigated by creating a new class to use for this purpose specifically and package it as a part of cfmongodb for the sole purpose of setting a string.  A convenience method in MongoUtil will hide this implementation from the user.

Ultimately, we would like this capability in the official distribution, as oppose to forking our own implementation and miss out on future improvements of the library, so we would be happy to help make this happen.

Looking forward to suggestions and comments.