Heads-up: wee ColdFusion incompat bug(?) in listToArray()

[Adam Cameron]

G'day, this is just an FYI.

I've just raised https://issues.jboss.org/browse/RAILO-2228, which says:

See steps to reproduce.

On ColdFusion, this yields a five-element array with each character as an element. On Railo it's just a single-element array with the whole string in the first element.

ColdFusion's behaviour dates back to at least CFMX7 (that's the oldest version I have on tap).

There is no suggestion one way or the other in the ColdFusion docs (http://help.adobe.com/en_US/ColdFusion/10.0/CFMLRef/WSc3ff6d0ea77859461172e0811cbec22c24-7f0f.html) whether an empty string should be allowable as a delimiter like this, but CF has worked this way since before Railo existed, so that's perhaps an adequate precedent? I could see arguments either way, but if a zero-length delimiter is illegal, then this should perhaps raise an exception. If it's valid, it should work the same as ColdFusion does, I think?

Cheers.

Repro:
<cfscript>
chars = "abcde";
array = listToArray(chars, "");
writeDump(variables);
</cfscript>

--
Adam

[Bruce Kirkpatrick]

It is a valid compatibility difference.

I just noticed that the new 4th argument can hang railo with an empty delimiter - multiCharacterDelimiter.

array = listToArray("abcde", "", true, true);  // I'm going to crash or timeout!

I submitted a bug on that.  Perhaps they'll both be addressed at the same time.

[Igal Sapir]

In my book the definition of a bug is when behavior strays from documentation.

I personally don't think that we should support this, as empty-string is Not a delimiter (while I do like the ability to iterate over the chars in a string).

Let's see what Micha thinks.

--
typos, misspels, and other weird words brought to you courtesy of my mobile device.

[Matt Quackenbush]

I can't imagine a world in which I'd even think to try to use an empty string as a delimiter. That defies the fundamental meaning of the word delimiter.

a blank space, comma, or other character or symbol that indicates the beginning or end of a character string, word, or data item

A delimiter is a sequence of one or more characters used to specify the boundary between separate, independent regions in plain text or other data streams.

Definitely *not* a Railo bug; another ACF silly behavior/idiosyncrasy/bug that Railo correctly fixes, though.

[Igal Sapir]

At the risk of repeating myself -- I'm with Matt on this one :-)

--
typos, misspels, and other weird words brought to you courtesy of my mobile device.

[Charlie Griefer]

On Dec 29, 2012, at 12:49 PM, Igal Sapir wrote:

In my book the definition of a bug is when behavior strays from documentation.

I personally don't think that we should support this, as empty-string is Not a delimiter (while I do like the ability to iterate over the chars in a string).

Let's see what Micha thinks.

This may be my ColdFusion bias showing, but as somebody who has used CF for a long time, I've always thought of a space as a valid delimiter. Using spaces between words was an easy way to create an array of words in a sentence with listToArray().

Curious as to why some would not consider it a valid delimiter?

In either case, as has been pointed out, Adobe CF has supported this for ages. At the very least, it should probably be listed somewhere as an incompatibility. 

--

Charlie Griefer

http://charlie.griefer.com

"Give light, and the darkness will disappear of itself." 

-- Desiderius Erasmus

[Matt Quackenbush]

A *space* __is__ a valid delimiter. An empty string (""), however, is *not*.

[Bruce Kirkpatrick]

Please consider that <script type="text/javascript>alert("apple".split(""));</script> becomes an array with 5 values. It's not uncommon to have this behavior. It would be nice if we had fewer syntax differences between languages. I'd rather see cfscript nearly identical to javascript so that my front-end and back-end code library could grow at the same time. It seems that Java is closer to javascript then CFML is especially with strings, math functions and object creation.

[Matt Quackenbush]

listToArray() != string.split()

[Bruce Kirkpatrick]

Are you suggesting that CFML have a new function called split?  At the moment, member functions were only added to array, query and struct.  http://blog.getrailo.com/post.cfm/railo-4-0-previews-member-functions     - Perhaps string member functions could be considered for a future release that mimic javascript or java behavior.

listtoarray is the only way to emulate string.split() unless you want to write a loop.

[Igal Sapir]

Space IS indeed a very valid delimiter.  We're not talking about a Space here but about an empty-string.

--
typos, misspels, and other weird words brought to you courtesy of my mobile device.

[Igal Sapir]

String.split() is a Java method and in the absence of a method with that name in ACF/Railo it is resolved via reflection at runtime.

--
typos, misspels, and other weird words brought to you courtesy of my mobile device.

[Bruce Kirkpatrick]

I wasn't aware that we could call any method in the java string class.  I suppose this makes it impossible/pointless to change that then.   Since we are able to run "abc".split("") and get an array with 3 elements, that means java itself is doing that behavior too.   I tested php and they don't allow empty delimiter, but it seems like the java world does.  It seems worth having.

[Bruce Kirkpatrick]

I should have said 4 elements.

[Igal Sapir]

Whenever you try to access a property/method of an object, if it is not found then Railo/ACF attempts to resolve it in java via reflection.

This is very useful when you use a 3rd party library or ad-hoc java classes.

--
typos, misspels, and other weird words brought to you courtesy of my mobile device.

[Charlie Griefer]

Ah, yes. Sorry, my bad. Carry on :)

Will throw my hat into the ring and agree that an empty-string delimiter is… oogy.

[Denny]

In principal I agree (standards are great), but as you've already noted,
even with the ones that "agree", there are slight differences, which of
course can make all the difference in the world. :)

And there's the huge difference between splitting with a regular
expression and converting a list to an array using a character
delimiter. We have refind instead of split, for instance, which is
actually a little more explicit than split.

Rolling with the cfscript split idea, would one expect the javascript
split or the java split? I would think java, because of the context of
being in a servlet container vs. a browser, myself.

FWIW, the behavior of returning a toCharArray() type of deal for
listToArray("abc","") seems more useful than returning one list element,
but I would intuitively guess the full string in 1 element would be
returned vs. the array of chars.

So in sum, I guess didn't really add much to the conversation besides
pointing out the regular expression functions. :)p

:denny

--
Railo Technologies: getrailo.com Professional Open Source
Skype: valliantster (505)510.1336 de...@getrailo.com
GnuPG-FP: DDEB 16E1 EF43 DCFD 0AEE 5CD0 964B B7B0 1C22 CB62

[Kym Kovan]

Here is a scenario where a null delimiter is legitimate, admittedly in
the opposite direction to the original post.

Working with large blocks of text strings we put each part in an array
as we manipulate things and at the end do a:

ArrayToList(contentArray, "") with the delimiter being null

so the output is a big string. Very fast for large strings.

Kym

On 30/12/2012 7:05 AM, Matt Quackenbush wrote:
> I can't imagine a world in which I'd even think to try to use an empty
> string as a delimiter. That defies the fundamental meaning of the word
> delimiter.
>
> a blank space, comma, or other character or symbol that indicates
> the beginning or end of a character string, word, or data item
>
>

> A *delimiter* is a sequence of one or more characters used to

> specify the boundary between separate, independent regions in plain
> text or other data streams.
>
>
> Definitely *not* a Railo bug; another ACF silly
> behavior/idiosyncrasy/bug that Railo correctly fixes, though.
>
>
>
>
> On Sat, Dec 29, 2012 at 1:49 PM, Igal Sapir <ig...@getrailo.org
> <mailto:ig...@getrailo.org>> wrote:
>
> In my book the definition of a bug is when behavior strays from
> documentation.
>
> I personally don't think that we should support this, as
> empty-string is Not a delimiter (while I do like the ability to
> iterate over the chars in a string).
>
> Let's see what Micha thinks.
>
> --
> typos, misspels, and other weird words brought to you courtesy of my
> mobile device.
>
> On Dec 29, 2012 5:00 AM, "Adam Cameron"
> <adamcamero...@gmail.com

> <mailto:adamcamero...@gmail.com>> wrote:
>
> G'day, this is just an FYI.
>
> I've just raised https://issues.jboss.org/browse/RAILO-2228,
> which says:
>

> /See steps to reproduce./
>
> //
>
> /On ColdFusion, this yields a five-element array with each

> character as an element. On Railo it's just a single-element

> array with the whole string in the first element./
>
> //
>
> /ColdFusion's behaviour dates back to at least CFMX7 (that's the
> oldest version I have on tap)./
>
> //
>
> /There is no suggestion one way or the other in the ColdFusion

> docs
> (http://help.adobe.com/en_US/ColdFusion/10.0/CFMLRef/WSc3ff6d0ea77859461172e0811cbec22c24-7f0f.html)
> whether an empty string should be allowable as a delimiter like
> this, but CF has worked this way since before Railo existed, so
> that's perhaps an adequate precedent? I could see arguments
> either way, but if a zero-length delimiter is illegal, then this
> should perhaps raise an exception. If it's valid, it should work

> the same as ColdFusion does, I think?/
>
> //
>
> /Cheers./

>
> Repro:
> <cfscript>
> chars = "abcde";
> array = listToArray(chars, "");
> writeDump(variables);
> </cfscript>
>
> --
> Adam
>
>

--

Yours,

Kym Kovan
mbcomms.net.au

[Michael Offner]

For me the discussion is a little bit like, "how does the shape of a car change when you drive against a wall?"

We are speaking about a site effect of a function we never have even think off and I'm sure adobe either.

Perhaps adobe simply use a StringTokanizer or a similar helper for this job and this helper has this outcome.

We have used a StringTokanizer in the beginning, but for better performance we have switched to a handmade solution.

You should avoid String.split, the argument passed to this method is a regex, as long you do not plan to pass a regex.

Adding a bif called split as alias for listtoarray makes sense from my point of view.

Micha

--

/micha

Michael Offner CTO Railo Technologies GmbH

[Igal @ getRailo.org]

On 12/31/2012 12:25 AM, Michael Offner wrote:

For me the discussion is a little bit like, "how does the shape of a car change when you drive against a wall?"

LOL!  :)

Adding a bif called split as alias for listtoarray makes sense from my point of view.

IMO it will be confusing to add such an alias, especially since the impl will be a little diff from what people are used to in Java and JS, which both accept a regex.


Igal

[Michael Offner]

Good point, but regex function in CFML always start with "re", so we should have resplit and split.

Most java programmers are not aware that split use regex as input or they don't care.

Most CFML developer are not to familiar with java, so I think this is not a big problem.

Do you have a idea for a other name?

Btw: CFML use perl regex dialect for regex, so this is incompatible as well to java/JavaScript.

Micha

[Igal @ getRailo.org]

I honestly don't think that we need an alias like that. we have
listToArray() which everyone is familiar with.

if you really want to add an alias then Split() is as good name as any,
but if it will be used also as a Member Function then it will hide the
Java String.split() method which Can cause a problem and break existing
code for users who used Java's String.split() by way of reflection.


Igal

[Michael Offner]

Yes for People familiar with cfml, but not for people new to cfml. I use list functions all the time for string operations like listlast, listfirst ... But mostly this has nothing to do with lists, simply because I never store information in lists.
We do not support member functions for the string class and there are no plans to do so.
Many bif names are starting with the name of the object type they are made for (structkeyexists, arraylen, stringlen ...). What we could do is a function stringsplit, but I don't really like this, I just wanna point to the possibility.

Micha

Von meinem iPhone gesendet

[Igal Sapir]

I would actually love it if we had member functions like isEmpty() and len() on strings

--
typos, misspels, and other weird words brought to you courtesy of my mobile device.

[Adam Cameron]

I find Micha's statement a curious one, although perhaps just due to lack of explanation as to why.

It strikes me as odd/incomplete/PHPish to offer member functions on other data types, but not on strings. Surely it's "in for a penny, in for a pound"?

Indeed I just expected that along with the member functions on structs etc, they were there on strings too (I don't write Railo-specific code, so the whole enterprise is out of bounds for me currently, so I've not looked that thoroughly).

--
Adam

On Monday, 31 December 2012 11:15:01 UTC, Igal wrote:

I would actually love it if we had member functions like isEmpty() and len() on strings

[Adam Cameron]

Yep, precisely. And indeed it's for the reverse than I use the listToArray() approach. And it's not like it's something I myself invented: it's a not-uncommon technique, surely?

I think a large percentage of list operations performed out there are leveraging the fact that anything can be a list as long as there's something that can be used to identify the tokens discretely. One can treat a sentence as a list, using whitespace and some punctuation characters as token-separators, but no-one in the "real world" would consider a sentence to be a type of list. Similarly I think it's legit to consider the token-separator to be [nothing]. It's less obvious if one thinks in terms of "delimiters", as it's less easy to comprehend things being delimited by [nothing], I guess (if this thread is anything to go by, anyhow ;-)

--
Adam

[Adam Cameron]

Yeah. listToArray() is a regex-character-type-safe way of effecting split(). So what's good for split() should be good for listToArray().  listToArray() is just a wrapper for split(), under the hood (CF's hood, anyhow).

--
Adam

[Adam Cameron]

Oh FFS. I replied to this but either Firefox or Google seem to have eaten it.  I'll try again. The upside for everyone is that it'll be briefer this time, as I can't be arsed typing it all in again.

There are a number of possibilities here:
1) The ColdFusion behaviour is by design, and it's simply overlooked in the docs.
2) The ColdFusion behaviour is not specifically by design, but it's not like it's not intended.
3) The ColdFusion behaviour is actually a bug. It is not intended for a zero-length delimiter to be valid.
4) The ColdFusion behaviour is by design, but the design is incorrect.

Let's take each separately
Firstly: CF is working the way it intends to, there's just an omission in the docs. In a way the proof is in the pudding here: it works, has done for some time, no-one has suugested it shouldn't work that way, and it makes sense the way it works. But it's definitely not in the docs. I'd say Occam's Razor would imply this is the case here.

Secondly: no specific measure was taken to make it work the way it does, one way or the other. This is completely legit, and as likely to be the case as the former.

A digression into the docs: there are three "sizes" of delimiters mentioned in the docs:
* in almost all cases, the list delimiter is a single character, and this can be represented by a list of one or more characters that are taken as single-character delimiters (this is, btw, an example of a list with a zero-length string as a delimiter ;-). No mention is made in any of the docs for these functions about zero-length strings as delimiters.
* in some cases, the delimiter can be multi-character. This too is explicitly documented in these cases. And there's no mention of zero-length delimiters
* in one case that I found - listChangeDelims() - it is specifically documented that the new delimiter can be an empty string,

This points to me to suggest that there was no specific intent for these functions to work with a zero-length delimiter, and in a way suggest it is specifically not the intent. Given that when it is the intent, it's specifically documented as such.

So, on the basis of the docs, the third option seems likely: zero-length delimiters are not supposed to be supported, and shouldn't be. This implies a bug in ColdFusion: zero-length strings should not be permitted, ie: an exception should be raised.

The fourth option: there's no evidence to conclude this.

Now, from Railo's perspective:
If the ColdFusion behaviour is by design, and it's simply an omission in the docs, Railo should emulate CF's behaviour.
If ColdFusion's behaviour is not explicitly by design, but equally is not explicitly considered incorrect, then Railo should emulate CF's behaviour
If ColdFusion's behaviour is actually buggy and a zero-length delimiter is invalid, then there should be a bug raised for ColdFusion, and if Railo decided to pre-emptively fix this, the fix would be to reject zero-length delimiters, via an exception.
To decide that the correct behaviour here is that a zero-length delimiter is legit, but ColdFusion's handling of it is "wrong" and Railo's handling of it is "right" is a special pleading, and there's absolutely no evidence to support this (from top to bottom). I am not suggesting anyone has suggested this, but it's kinda what we've ended up with.

I think the reality is that CF's behaviour is serendipitous, but Railo's incongruity here is incorrect. It should either do it the same as CF, or it should disallow it; it should not allow it but do something different. That's just wrong.  All things being equal the former (mimic CF correctly) is preferable as it reduces incompat, and there's simply no reason to make incompatibilities for the sake of pedantry. But if one wanted to take the pedantry route: then doing it properly and raising the exception would be a legit thing to do here.

--
Adam

[Peter Boughton]

Adam wrote:
> The upside for everyone is that it'll be briefer this time

Heh, if that's a briefer version, I think they did us all a favour! ;)


I agree though: the sensible choice is between throwing an error or being compatible with CF.


In the situation that people are utilising Railo's current [mis]behaviour, they can do:

    len(delim) ? ListToArray(string,delim) : string

(Or there's also the option of Railo's custom BIF or FLD functionality to restore original behaviour.)

[Adam Cameron]

On Monday, 31 December 2012 13:21:33 UTC, Peter Boughton wrote:

Adam wrote:
> The upside for everyone is that it'll be briefer this time

Heh, if that's a briefer version, I think they did us all a favour! ;)

Well yeah. I try to pre-empt as much of the likely facile/specious reaction I figure I'll get by being as explicit and thorough in my answers as possible. Otherwise someone will say "but [specious justification here]", which stands up to idle scrutiny (so will seem reasonable to a lot of people), but is basically wrong or misleading. Which doesn't help getting to the nub of a situation.

NB: I don't mean the specious / facile thing specifically about this list, it's just a general thing about online discussions I have learned to try to circumvent.  And it definitely is not something that applies to a lot of the feedback here. Some: yes. Not most though.
 

I agree though: the sensible choice is between throwing an error or being compatible with CF.

I had a wee bit more of a look at this, and documented it:
http://adamcameroncoldfusion.blogspot.co.uk/2012/12/lists-with-empty-string-delimiters.html

I'm more of an opinion now that empty-string list delimiters should be rejected wholesale. Sadly... they're not, and the handling of them is a bit wayward on both CF and Railo.

Whether or not it's worth doing anything about is another thing, though.Still: the info is there.

--
Adam

[Jeroen Knoef]

I can imagine the reason for this is because a Railo string is a java.lang.String, which is final. So the member functions would have to be implemented in a decorator class, and all strings in Railo would from then on have to be represented by an instance of the decorator. This becomes complicated fast. Apart from the slight performance hit of having 2 object instances instead of 1 for each string, one of the things that would become difficult is calling a java method that expects a string. You can't pass that decorator in there, so Railo has to do additional stuff, for just this data type. There are probably more reasons for not implementing list member functions, which is after all a Railo specific feature that is only going to be used by developers who don't have to worry about other engines.

For my projects I develop only for Railo, and I must say I really like member functions. Having to use ListToArray et al. is a pity, but understandable IMO.

Op maandag 31 december 2012 12:26:08 UTC+1 schreef Adam Cameron het volgende:

[Adam Cameron]

Hi Jeroen:
That sounds entirely reasonable, good thinking.

--
Adam

[Ed @ ODI Consulting]

It seems like this should be really easy for the Railo team to fix.  Just add a line of code to the listToArray() function to check for null delimiter and return .split(//) in that case.

[Mark Drew]

Many things are EASY to fix, but the consequences are bad for the language. I don't think any of this discussion is about whether it is difficult or not, but whether it should be done or not. 

One of the things we see with CFML is striking the balance between compatibility and what "makes sense in the language".

Regards

Mark Drew

[Ed @ ODI Consulting]

Is there some reason such a fix would be bad for the language ( splitting the string to chars kind of seems like the expected behavior ) ?
Not that this specific feature matters to me much - just trying to figure out how the process gets decided in general.
I guess the question I'm really asking, is just, how compatible can I expect the stable version of Railo 4 to be (or not be) with legacy CF 7/8/9 code.
How many behaviors will have been modified based on what "should be", vs, what is?

[Matt Quackenbush]

On Wed, Jan 2, 2013 at 11:42 AM, Ed @ ODI Consulting <e...@odiconsulting.com> wrote:

Is there some reason such a fix would be bad for the language ( splitting the string to chars kind of seems like the expected behavior ) ?

listToArray( myArray, '' ) returning anything other than an array with a single element *is not* expected behavior. The second argument is a "delimiter", and a delimiter _cannot_ be an empty string. A delimiter is by definition "one or more characters". That could be a space, a tab, two spaces, four spaces, an ;, an X, an 4... whatever, but definitely cannot be an empty string (i.e. '').

 

Not that this specific feature matters to me much - just trying to figure out how the process gets decided in general.
I guess the question I'm really asking, is just, how compatible can I expect the stable version of Railo 4 to be (or not be) with legacy CF 7/8/9 code.
How many behaviors will have been modified based on what "should be", vs, what is?

That question I can't definitively answer. I can say that the vast majority of CF 7/8/9 code has been, in my experience, quite compatible.  However, if one relies on underlying ACF classes/actions/silliness, then one should expect to change code in order to run on Railo (any release).

The biggest area of change that I can think of off the top of my head is the correct handling of arrays in Railo. That said, if I'm not mistaken there is a setting in the administrator to switch to ACF compatibility on arrays. There are probably others, but since it has been years since I've used ACF I'm not really sure. I don't care if any CFML I write runs on ACF. :-)

 

[Mark Drew]

I think this feature was missed from the documentation and we do strive to keep it very compatible, and if not, be able to offer good work rounds. 

How compatible? well you really should perform your own tests in all honesty. Maybe I am jaded but I have seen some insane code over the last few years that much of the time makes me say "Surely that cannot work, it's obviously wrong" but turns out it works, despite my eyes bleeding. 

How many behaviors will have been modified based on what "should be", vs, what is?

It's a tough decision, do we do a great product or just a compatible one? I would prefer a great product that makes sense for all concerned would be first and compatibility second, but that is *MY* personal opinion. 

I would say much of your code should work as expected but there *ARE* edge cases that crop up from time to time. 

I am sorry if I cannot be more precise, but with something like 130 tags and 597 functions, keeping track on how compatible they are with 4 different versions of another product is fairly *hard*. 

Let's say that CF 7/8/9 code should be PRETTY compatible. 

And by the way, I never knew you could do ListToArray("bladadada", "") and get an array of the characters, I would have never done that personally and aside from this thread, never have come across it (and I see a lot of code) (but I know you will all prove me wrong by saying you do this day in day out and I am an idiot for not knowing such a handy function) 

[Matt Quackenbush]

On Wed, Jan 2, 2013 at 11:52 AM, Mark Drew <ma...@getrailo.com> wrote:

I think this feature was missed from the documentation and we do strive to keep it very compatible, and if not, be able to offer good work rounds. 

How compatible? well you really should perform your own tests in all honesty. Maybe I am jaded but I have seen some insane code over the last few years that much of the time makes me say "Surely that cannot work, it's obviously wrong" but turns out it works, despite my eyes bleeding. 

That ain't no lie!

 

How many behaviors will have been modified based on what "should be", vs, what is?

It's a tough decision, do we do a great product or just a compatible one? I would prefer a great product that makes sense for all concerned would be first and compatibility second, but that is *MY* personal opinion. 

And you are by no means alone! I've said it before, and I'll say it again. Railo doing things right ***is the only reason*** I'm still even remotely involved with CFML anymore. If Railo changes that, then I have no incentive to "hang around". And while it's (quite?) possible that many would be perfectly fine with me not hanging around, I know for a fact that I am nowhere close to alone in feeling that way about Railo.

 

I would say much of your code should work as expected but there *ARE* edge cases that crop up from time to time. 

I am sorry if I cannot be more precise, but with something like 130 tags and 597 functions, keeping track on how compatible they are with 4 different versions of another product is fairly *hard*. 

I'm going to go one step farther: Railo 4+ shouldn't care about being compatible with ANY version of ACF or BD/OBD. Railo is the clear leader in the development of the language, and should not be concerned with it.

 

Let's say that CF 7/8/9 code should be PRETTY compatible. 

Exactly.
 

And by the way, I never knew you could do ListToArray("bladadada", "") and get an array of the characters, I would have never done that personally and aside from this thread, never have come across it (and I see a lot of code) (but I know you will all prove me wrong by saying you do this day in day out and I am an idiot for not knowing such a handy function) 

You're definitely not wrong. As I stated early on in this thread, I cannot imagine a world in which I'd even think of such a thing, let alone try it. And I'm clearly not alone there, either. ;-)