xml to json in Scala

[Maia R.]

Hello everyone,

I have *.xml file for my landmarks. I am working with IntelliJ and have a sbt file. 

Could someone tell me how to convert xml file into *.json with Scala ?  

I found some advices but as I am also new to Scala and Java, I couldn't manage to use  import.scala.xml._ in my scala file.

 I tried to put this line in sbt file:

libraryDependencies += "org.scala-lang.modules" %% "scala-xml" % "1.0.6" 

Unfortunately, it does not compile and I have an error saying:

object xml is not a member package scala.

 

Thank you,

Maia

[Marcel Luethi]

Hi Maia,

I have not done any xml processing in Scala, but I just run a small test to see how it works. Adding the line you mentioned to build.sbt (and refreshing the project in Intellij from the sbt window)  works for me. I could successfully run the following program:

object XMLTest {

  def main(args : Array[String]) : Unit = {
    val foo = <foo>
      <bar type="greet">hi</bar>
      <bar type="count">1</bar>
      <bar type="color">yellow</bar>
    </foo>
    println(foo.child.map(c => println(c.text)))
  }

Once you have read the xml, there are various options available for converting it to json. Scalismo already comes with a json library as a dependency, which you can directly use. It is called spray-json (https://github.com/spray/spray-json). There are many other options available, the simplest is maybe upickle http://www.lihaoyi.com/upickle/.

Li Haoyi has just published a new Scala book called "hands-on scala", which explains how many of the common tasks such as data processing, building web-services, parsing files, working with structured data, etc can be done in scala. (https://www.handsonscala.com/)

Best regards,

Marcel

--
You received this message because you are subscribed to the Google Groups "scalismo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalismo+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/53b6c9a7-1eac-4523-8d61-fd0ee58a55a5n%40googlegroups.com.

[Maia R.]

Hi Marcel,

What do you mean by : 'Scalismo already comes with a json library as a dependency, which you can directly use'.

I check the Scalismo APU and there are only few methods with LandMarkIO:

http://unibas-gravis.github.io/scalismo/latest/api/scalismo/io/LandmarkIO$.html .

As my json landmark file (which I managed to convert from xml to json with python xmltodict !) doesn't have the same convention name as for LandmarkIO, I need to convert it to be used in Scalismo (read and rename id, etc.).

As far as I understand, I need to add the library dependencies in sbt as :

"com.lihaoyi" %% "upickle" % "0.7.1"

https://www.lihaoyi.com/post/HowtoworkwithJSONinScala.html

and then parse each entry from my json file into Scalismo landmark file ?

Thank you,

Best regards

Maia

[Marcel Luethi]

Hi Maia

I meant that Scalismo internally uses spray.json. This means, when you have a project that uses Scalismo, you will automatically also have spray.json available as a library (it is a transitive dependency). Hence you won't need to add any new dependency to build.sbt to use spray.json.

In case you want to use upickle instead, then you will need to add it to build.sbt as you suggested.

Best regards,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/5a8c63eb-89f3-44e6-9268-6ef4427c989bn%40googlegroups.com.

[Maia R.]

Thank you Marcel.

My new question is more Scala-relative: I added first

=======

"com.lihaoyi" %%% "upickle" % "0.9.5" // SBT  

=======

an then in my sbt file, I add: 

=======

import upickle.default._ 

=======

in my Scala file

But I still have error saying: "uPickle is not found"

What step did I miss ?

Thank you

[Marcel Luethi]

Hi Maia

You will need to reload the project, such that sbt (the build system) can fetch the dependency and make them available for the IDE. You can do this by pressing reload button in the sbt tool window (which you find usually on the right side of the IntelliJ window).

Best regards,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/adf58c19-a413-4db0-9ba1-d6e48e455a35n%40googlegroups.com.

[Maia R.]

Thank you very much, it is working.

[Maia R.]

Thank you very much, It is working.

[Maia R.]

Dear Marcel,

I have a *.json file which contains my custom landmarks. I am able to read this file with:

====

import spray.json._

import DefaultJsonProtocol._

import upickle.default._

import org.json4s._

import org.json4s.native.JsonMethods._

val jsonLandmarkFile = new File(referenceDir, "myLM.json")

val json_content = scala.io.Source.fromFile(jsonLandmarkFile).mkString.parseJson

println("JSON string read:")

val json = json_content.prettyPrint

println(json)

I get this:

{

  "APPLICATION": {

    "@applicationVersion": "APPLICATION-1.0.0",

    "ApplicationInfo": {

      "value": {

        "@type": "Type",

        "@version": "1.1",

        "value": [{

          "#text": "date",

          "@key": "date",

          "@type": "String"

        }, {

          "#text": "Info",

          "@key": "xxx",

          "@type": "xxx"

        }

        }]

      }

    }

    "Landmarks": {

      "value": {

        "@type": "Type",

        "@version": "1.1",

        "value": [{

          "#text": "-4.36 -21.5 -54.6",

          "@key": "LM1",

          "@type": "Type"

        }, {

          "#text": "-4.1 -19.2 47.2",

          "@key": "LM2",

          "@type": "Type"

        }, {

          "#text": "-3.9 6.16 253.6",

          "@key": "LM3",

          "@type": "Type"

        }

        }]

      }   

}

I'd like to convert those landmarks to Scalismo landmarks from json_content. But I am stuck on how to do this in Scala (sorry, I am new to Scala and json as well). I had a look in LandmarkIO.scala and tried this: 

val landmarks = json_content.asJsObject.getFields("Landmarks")  

but I wasn't able to get something as json_content is a very nested json file ...

Could you give me some hints, please. 

Thank you,

Best regards

Maia

Le mardi 18 août 2020 à 03:03:06 UTC-4, marcel...@gmail.com a écrit :

[Marcel Luethi]

Dear Maia

I cannot explain you in all the details how to do this on this mailing list, as it is a bit off-topic and also not easy to explain in a few sentences only.

However, I will try to sketch the approach: First, you need to model your data using case classes. The classes should correspond to the structure of your json format.

For example, for the landmarks part it could be something like this

case class Landmarks(values : Seq[Landmark])

case class Landmark(point: Point[_3D], key: String, type : String)

Now you need a piece of code that can convert to and from the Json Representation to your case classes. How this is done is explained here:

https://github.com/spray/spray-json#providing-jsonformats-for-case-classes

The approach that spray json is using is nice for larger applications and leads to well structured code. However, it is maybe not the most beginner friendly way. As you say you are new to Scala, it might be easier to use the aforementioned library upickle. This is deliberately kept very simple, and mimics some of the popular python json libraries. As mentioned before, the book "Hands on Scala" would explain in detail how this can be done. It is definitely worth the investment.

In case you don't want to dig too deeply into it, we can also offer you consulting services via our company shapemeans (www.shapemeans.com).

Best regards,

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/9261f56d-554d-4f0d-b1a1-7e1636d6671cn%40googlegroups.com.

[V R]

Dear Marcel,

Thsnk you very much for the explanations.

Best regards,

Maia

[V R]

Dear Marcel,

Again, a big thank for your awesome kindness to reply every question here.  God bless you !

I will do my best with the book and your kind advices. 

Best regards,

Maia

[Maia R.]

Hi Marcel and dear community,

I managed to convert my landmark xml file to json as below:

========

val bufLMs = createLandmarks(LandmarksBuf)  // here I used ArrayBuffer to append all my landmarks from my xml file
val meshLandmarks = bufLMs.toSeq 

LandmarkIO.writeLandmarksJson[_3D](meshLandmarks, new java.io.File(lmsDir, "LM.json"))

=========

I checked and everything is okay, I mean, I get the same landmarks from both xml and json files. I get the list of landmarks from another software.

Unfortunately, my landmarks are shifted wrt to the mesh when displaying with Scalismo UI: do I need to apply transformation matrix as it seems that the mesh and the landmarks do not have the same 'reference' ?

Thank you

[Marcel Luethi]

Dear Maia,

Great to hear that you succeeded with the landmarks.

When you clicked your landmark in another program, it might happen that they are shifted or flipped. This usually happens when the software uses internally a different coordinate system than Scalismo (see e.g. this discussion https://www.slicer.org/wiki/Coordinate_systems for more information). Scalismo uses the LPS coordinate system.

An easy way to find out if this is the case is to click the same landmark once in scalismo (using scalismo-ui) and once in your software and to compare the coordinates. A usual pattern is that one or two coordinate axis are flipped (the sign is reversed), which is then simple to recover.

Best regard,s

Marcel

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/995131b4-4765-4bef-9d48-d423ae9d8908n%40googlegroups.com.