Varchars

[jtuchel]

HI,

a few weeks ago, I was complaining about the fact that GLORP truncates constant length chars.

Now, I am in another boat. I am porting my app from Access to DB2, just to find out how generous Access is to developers ;-)

One problem is that it seems to forgive too long strings on inserting to varchar fields. DB2 wants me to respect the rules. If the definition says 80 characters, it won't accept an INSERT with 85 characters, because I said so. While this is somehow very correct of DB2 and I somehow tend to respect that, because after all it was me who told Mr. DB Manager that there will be strings of no more than 80 characters.

But still I wonder if it is possible to let Glorp do the dirty work and cut off my longer strings before inserting...

Joachim

[Alan Knight]

Not only is it possible, but it ought to happen fairly automatically. If
you've set a width on the varchar type in Glorp, then it should go
through padString:for:, which will attempt to trim it to the required
length. It can get messy on some databases if you use multi-byte
encodings, but the width of varchar fields is measured in bytes, but it
tries to compensate even for that.

> --
> You received this message because you are subscribed to the Google
> Groups "glorp-group" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/glorp-group/-/RophJSfOx3MJ.
> To post to this group, send email to glorp...@googlegroups.com.
> To unsubscribe from this group, send email to
> glorp-group...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/glorp-group?hl=en.

[jtuchel]

Alan,

Am Montag, 5. März 2012 15:07:40 UTC+1 schrieb AlanKnight:

Not only is it possible, but it ought to happen fairly automatically. If
you've set a width on the varchar type in Glorp,

You mean in the decriptorForXXX: method? Ho would I do that?

I have of course set a length in the tableForYYY: using (platform varchar: 80). But this obviously isn't sufficient.

 

Joachim

I also encountered another nice little problem: I cannot use CHAR(1) as discriminator field for a type resolver on DB2, because AbtIbmCliFixedCharField from VAST converts that CHAR(1) to a character and Glorp uses Strings as keys for the type resolution. That's not a GLORP bug, it's just incompatible with VAST's extra cleverness. I have been looking for a nice place to fix that but neither found one in GLORP nor in VASTs DB code. I will just try to avoid CHAR(1) for subclass resolution. Wherever I looked at to fix it, it seemed like too dangerous to break lots of existing code....

[Alan Knight]

No, I think that ought to be sufficient. At least in the current versions, if the types are both strings, then the it gets a stringToStringConverter which would call padString:for: which would in turn call trimString:for:. Maybe the version you've got is old enough not to do that, or maybe there's something odd going on. But I'd try breakpointing one of those methods and see if it's getting called.

[Alan Knight]

Oh, and I don't have any brilliant thoughts about the type resolver issue. Although I'm surprised that Glorp cares if it's a string or not.

Alan Knight wrote:

--

You received this message because you are subscribed to the Google Groups "glorp-group" group.

[jtuchel]

Alan,

I am stuck in a totally different problem than I thought. My DB server is using UTF-8 and my client also. But VAST is using IS08859-15. So the String handed from within the Smalltalk image fits well into the field, but DB2's conversion to UTF-8 expands it by 1 byte for each umlaut. So If I Update a varchar(3) from GLORP with a String 'ÄÖÜ', Glorp will perfectly correctly send this string to the DB client and do its job nicely. But there is some conversion going on between the CLI client and the database that expands the String to 6 Bytes and therefor I get the SQLCode.

So it's nothing you have to worry about, other than maybe feel a bit of pity for me ;-)

Does anyone have a good idea what to do in this situation? IBMs Documentation is so full of non-information that I seem to not see the trees there.

My current idea is this: There's not much use in trying to offload work to DB2 by changing my client's code set, because the server still needs two bytes for each umlaut (or other special character like the Euro sign). So I either make my fields longer by a theoretical factor of, say, 1.5 (It's unlikely more than 50% of German words are Umlauts)  (DB2 proposes a factor of 3 in its SQLERRD fields, but I guess that's because they take the 4 bytes as a base and use the same rule of thumb about the percentage of real 4-byte characters), or I recreate the DB on the server with code set 8859-15. This is probably my best bet for the moment, because it should make the lengths of Strings equal for both sides of the connection. But it also brings up the question of what to do with other clients in other programming languages...

I will first try the latter option. Seems like less effort for now ;-)

[Alan Knight]

That's the bad behaviour by the databases that I referred to in an earlier message. You can have character fields with varying-width encodings like UTF-8, but the length of the field is measured in bytes. The trimming code in current Glorp attempts to deal with that, but it can be quite ugly. Basically you have to encode it, see if it's too long. If it is, trim some characters off the end and try again, and do a sort of binary search to find the maximum length that will fit. As far as sizing the fields, yes, you just have to make the fields large enough to hold as many characters as you want, on the assumption that each character might be multiple bytes.