YAML, numeric keys in LinkedHashMap structure

[Ari Fernelius]

I banged my head to the wall for a day, trying to find out the most convenient way to read YAML files. 

In short: Working with SnakeYAML, I get a structure, from where I can examine the structKeyList, but can't access any of the keys.

Example:

file: graphicIDs.yaml

10:
  description: Moon
  graphicFile: res:/dx9/model/worldobject/planet/moon.red
38:
  description: Caldari Frigate Bantam
  graphicFile: res:/dx9/model/ship/caldari/Frigate/CF1/CF1_T1.red
  sofFactionName: caldaribase
  sofHullName: cf1_t1
  sofRaceName: caldari

code:

<cfscript>
yamlPath = 'c:/databases/';
yamlGraphicIDs = parseYaml(yamlPath & 'graphicIDs.yaml');


function parseYaml(file)
{
Yaml = createObject('java','org.yaml.snakeyaml.Yaml','c:/lucee/lib/snakeyaml-1.15.jar');
yamlFileData = FileRead(arguments.file);
return Yaml.load(yamlFileData);
}
</cfscript>

Now, everything seems to work nicely...  #structKeyList(yamlGraphicIDs)# lists the keys.

But then I hit the brick wall. structKeyExists function does not find any of the keys. Neither does any bracket notation of the structure. CFDUMP shows nothing.

Looking at getMetaData() of the returned structure shows it's a LinkedHashMap. A normal Lucee structure is a type of StructImpl. Now, I'm not much of a Java person, so all this is a bit cryptic to me. 

I do some Google-fu, and luckily find an old ColdFusion ORM related solution: 

Let's javacast.  #yamlGraphicIds.get(javacast("int",10)).description# works nicely!

Question: Would there be a way to convert the LinkedHashMap structure to a format that can be cfdumped and manipulated normally?

If I create a new structure with StructNew(), I surely can address numeric key values without problems, because the structure is of a different type, native to Lucee.

Cheers,

-Ari

[Desmond Miles]

Sorry I replied somewhere else...here : https://groups.google.com/forum/?hl=en#!topic/lucee/bwQ__npnYdo

Don't remember how I did that...really sorry...

[Igal @ Lucee.org]

so you called StructKeyExists(yamlGraphicIds, "someKey") and it didn't work?

what do you see when you do dump(yamlGraphicIds) ?

Igal Sapir
Lucee Core Developer
Lucee.org

--
You received this message because you are subscribed to the Google Groups "Lucee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lucee+un...@googlegroups.com.
To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/82b0198c-99ea-4c6a-91a8-02e762b120d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[One Lookup]

Calling structKeyExists() will always returned FALSE if :

• The underlying struct is a not a CFML Struct (say LinkedHashMap)
• AND if its keys are not Strings e.g LinkedHashMap<Integer,Object>.
  

The reason if because Railo and Lucee converts keys to String and this is will return FALSE with a Map<Integer,Object> for example.

StructKeyExists(javaMap,stringKey) only work if in the example it was a LinkedHashMap<String,Object> instead of LinkedHashMap<Integer,Object>

If interested, below are links to the corresponding stuff in Lucee ;-)

• The structKeyExists() invocation : https://bitbucket.org/lucee/lucee/src/b74116ad69bd3616fb4ee85f5f26dd42490ee485/lucee-java/lucee-core/src/lucee/runtime/functions/struct/StructKeyExists.java?at=master#cl-53
  
• Which lead to the java.util.Map.containsKey() call : https://bitbucket.org/lucee/lucee/src/b74116ad69bd3616fb4ee85f5f26dd42490ee485/lucee-java/lucee-core/src/lucee/runtime/type/wrap/MapAsStruct.java?at=master#cl-206

---------

Don't expect I'm "Desmond Miles". It's just a pseudonym ;-)

--
You received this message because you are subscribed to a topic in the Google Groups "Lucee" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lucee/WSne0EeMFkY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lucee+un...@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/551AC0B8.4060206%40lucee.org.

[One Lookup]

IMHO what we see in cfdump is a visual representation i.e the keys in string format (this doesn't mean they are of type java.lang.String in the underlying Java Map)

---------

Don't expect I'm "Desmond Miles". It's just a pseudonym ;-)

[Igal @ Lucee.org]

not sure where you get your information from, but the keys are not of type String.  they're of type lucee.runtime.type.Collection.Key

Igal Sapir
Lucee Core Developer
Lucee.org

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAMT%2B4CASOuy3z2qmN_XvhZGF2HxnhJ%3DUsWdFE_qNXcTWmVnUPQ%40mail.gmail.com.

[One Lookup]

Sure, I've looked in Lucee source code on BitBucket (master branch).

In the Java class lucee.runtime.functions.struct.StructKeyExists at line 53 there's the following :

return struct.containsKey(key) && struct.get(key,null)!=null

But the above struct.containsKey() will call something like in the Java class lucee.runtime.type.wrap.MapAsStruct (which is called by Caster)

boolean contains = map.containsKey(key.getString());

And here key is the Collection.Key class which getString() method return a java.lang.String...

I concluded that it was the workflow because the struct provided to structKeyExists() is not a StructImpl but a java.util.LinkedHashMap

---------

Don't expect I'm "Desmond Miles". It's just a pseudonym ;-)

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/551AC431.20201%40lucee.org.

[Igal @ Lucee.org]

Collection.Key is an optimized class that is used as the key for all the structs, scopes, etc.

Calling structKeyExists() will always returned FALSE if :

• The underlying struct is a not a CFML Struct (say LinkedHashMap)
• AND if its keys are not Strings e.g LinkedHashMap<Integer,Object>.

did you test that or did you conclude it by looking at the source code?

one thing to note is that for non-Structs the default implementation is CaSe sensitivity in the keys, so the key must be in the Correct CaSe if the Map object is not a Struct and does not use a case-insensitive comparator.

Igal Sapir
Lucee Core Developer
Lucee.org

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAMT%2B4CC1M39E_%2BrnaaGehjK_wkzgfd4WOpW%2BEVV_ePgZArO51Q%40mail.gmail.com.

[One Lookup]

I'll post a cfml example (I think I'm currently using Lucee 4.5.1.000). I'll do it later because I'm currently in a train to get home (not sure I'll have internet connection...)

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/551AC954.3060608%40lucee.org.

[Desmond Miles]

Here's a simple test (Lucee 4.5.1.000).

The output will show false returned by structKeyExists() and true for containsKey(). Indeed javaCast("int",key) is not a String.

Can you test it and tell what it produces then ?

<cfset list=createObject("java","java.util.LinkedHashMap").init()>
<cfset list.put(javaCast("int",10),{description="First value"})>
<cfset list.put(javaCast("int",20),{description="Second value"})>
<cfset list.put(javaCast("int",30),{description="Third value"})>
<cfset list.put(javaCast("int",40),javaCast("null",0))>

<cfset keys=structKeyList(list)>
<cfoutput>
 <cfloop list="#keys#" index="key">
 Key #key# : Exists in CFML ? #structKeyExists(list,javaCast("int",key))# - Exists in Java ? #list.containsKey(javaCast("int",key))#<br>
 </cfloop>
</cfoutput>

To unsubscribe from this group and stop receiving emails from it, send an email to lucee+unsubscribe@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/82b0198c-99ea-4c6a-91a8-02e762b120d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Lucee" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lucee/WSne0EeMFkY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to lucee+unsubscribe@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/551AC0B8.4060206%40lucee.org.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lucee" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lucee+unsubscribe@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAMT%2B4CASOuy3z2qmN_XvhZGF2HxnhJ%3DUsWdFE_qNXcTWmVnUPQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Lucee" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lucee/WSne0EeMFkY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to lucee+unsubscribe@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/551AC431.20201%40lucee.org.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Lucee" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lucee+unsubscribe@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/CAMT%2B4CC1M39E_%2BrnaaGehjK_wkzgfd4WOpW%2BEVV_ePgZArO51Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Lucee" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lucee/WSne0EeMFkY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to lucee+unsubscribe@googlegroups.com.

[Desmond Miles]

Here's an example showing the structKeyExists() behaviour.

<cfset list=createObject("java","java.util.LinkedHashMap").init()>
<cfset list.put(javaCast("int",10),{description="First value"})>
<cfset list.put(javaCast("int",20),{description="Second value"})>
<cfset list.put(javaCast("int",30),{description="Third value"})>
<cfset list.put(javaCast("int",40),javaCast("null",0))>


<cfset keys=structKeyList(list)>
<cfoutput>
 <cfloop list="#keys#" index="key">
 Key #key# : Exists in CFML ? #structKeyExists(list,javaCast("int",key))# - Exists in Java ? #list.containsKey(javaCast("int",key))#<br>
 </cfloop>
</cfoutput>

I run it with Lucee 4.5.1.000 (Java 1.7 / Tomcat 8 / Ubuntu 14.04) and this should output the following :

Key 10 : Exists in CFML ? false - Exists in Java ? true
Key 20 : Exists in CFML ? false - Exists in Java ? true
Key 30 : Exists in CFML ? false - Exists in Java ? true
Key 40 : Exists in CFML ? false - Exists in Java ? true

Can you try it and tell if you have the same behaviour then ?

[Ari Fernelius]

Igal, Desmond Miles actually explained it pretty well, and everyone is well on track.

StructKeyExists() results FALSE for any key in the structure. StructKeyList does list the keys OK. The only way to access the keys further is by using javacast.

I am quite happy this subject got plenty of attention, especially from the people who actually understand the issue. :)

Thanks!

It's enough for me that people realized this issue exists. It's a bit of a nuisance at least. Whether it's an issue that should be fixed in some way, is another subject, as I'm not competent enough to suggest a workaround for this by changing the Lucee source code. Maybe an additional parameter in Lucee, for Duplicate() or StructCopy() that would translate a LinkedHashMap keys from integer to string? Or some other automatic behaviour that wouldn't hurt the standard performance? Just throwing some end-user-point-of-view ideas in the air... :-)

-Ari

[One Lookup]

I agree with you, this is a bit of nuissance (sorry if I've wrote a lot about this function). Sometimes "over-analyzing" things already studied, may be useful for thechies an developers IMHO (no hate)

I think this shouldn't be considered as an issue as long as structKeyExists() is supposed to check the existence of a "String" key in an Object which will be considered "as a CFML Struct" (that is with String keys).

In fact in Lucee everyone have access to the Java layer and is supposed to do so when they don't find their needs e.g People did java.lang.Thread.sleep() long before any CFML sleep() exists and that wasn't an issue at all.

The most used solution for the original question, consist of loop over the Java Map and populate a CFML struct.

Another one could be using the toCFML() of the Lucee Java class : lucee.runtime.java.JavaProxy

You may use it as this :

<!---
The Javadoc doesn't tell much on what it does exactly...but it seems working
In fact given a Java Map it will recursively convert any keys to String using the Java toString() result.
It will recursively convert values to a CFML object only for a limited predefined Java types.
Otherwise it leave them as-is
--->
<cfset cfmlValue=createObject("java","lucee.runtime.java.JavaProxy").toCFML(mapWithIntegerKeys)>
<!--- This will return TRUE --->
<cfset result=structKeyExists(cfmlValue,"existingKeyAsString")>

PS : In ACF10 I get the same structKeyExists() behaviour. Though I don't wait much from their closed source....

---------

Don't expect I'm "Desmond Miles". It's just a pseudonym ;-)

--
You received this message because you are subscribed to a topic in the Google Groups "Lucee" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/lucee/WSne0EeMFkY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to lucee+un...@googlegroups.com.

To post to this group, send email to lu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/79ad982d-8794-47ce-bc0c-fa1779bcc920%40googlegroups.com.

[Ari Fernelius]

I can totally agree in this case with your resolution. Manually looping would have been of course an obvious solution, but I hated the idea of the performance loss - even though that doesn't matter so much when I only load the yaml data into an application variable at application start.

But wow, I didn't know about the toCFML(). It works perfectly. Thanks!

[Michael Offner]

Lucee can deal well with all kind of Maps, including LinkedHashMap.

problem here is not that Lucee cannot handle Maps, the problem is with the function structKeyExists.

this function has the following pattern:

structKeyExists(struct struct,string key):boolean

the second argument must be a string mean your "javaCast("int",key)" get converted to a string before the function is called.

so the function structKeyExists can only check for string keys...

Micha

To view this discussion on the web visit https://groups.google.com/d/msgid/lucee/85860bf1-92df-4650-8c89-2c788998db85%40googlegroups.com.