Transform set of repeating data blocks using one query

[Dmitry]

Hi Adrian,

I would say thank you again for your efforts in development of facinating ObjectPath, I really like it. Here is my problem:

I have JSON with repeated blocks of data series - daily weather forecast. Each block, lets say, represents observations for 1 day. At the end I would like to get the same JSON structure (repeated blocks), but data in each block must be transformed: some attributes are thrown away, values of other attributes used to create new attributes and so on. As an example:

source = [{'dt': 1, 'rain':3, 'useless':0}, {'dt':2, 'snow':5, 'useless':0}]  
dest   = [{'time':1, 'precip_type':'rain', 'precip_intensity':3}, {'time':2, 'precip_type':'snow', 'precip_intensity':5}]    

Using excellent Python library I could process them separately using one query to get list of repeating blocks and then transform each block using dedicated query for each attribute. 

But I'm curious whether it is possible to get resulting data set using one ObjectPath query?

Thanks in advance!

Dmitry.

[Adrian Kalbarczyk]

Hi,

Thanks for the kudos!

I would do it in Python:

r=[]
for i in dataset:
  r.append({
    "time":i.get("dt"),
    "precip_type": i.get("rain") and "rain" or i.get("snow") and "snow" or "",
    "precip_intensity": i.get("rain") and "rain" or i.get("snow") or 0
})
# the result is in r

ObjectPath is currently not well suited for this kind of data transformation (it's too slow), but rather for querying and doing analytics on complex nested structures. For transformation of data using ObjectPath you can check out the other project of mine called ACR, which is essentially what you are looking for but running in web environment ACR. The flow there is: validation -> processing/transformation -> storing in DB or passing the data further. Your transformation would look there as follows:

<?xml version="1.0" encoding="UTF-8"?>
<view xmlns="http://asyncode.com/View"
            xmlns:op="http://asyncode.com/Interpreter">

    <post>
        <param name="data" type="json"/>
    </post>

    <node name="transform"
        command="op:exec"
        path="$.data">
        {
            time:!.dt,
            precip_type: !.rain and "rain" or !.snow and "snow" or "",
            precip_intensity: !.rain and "rain" or !.snow or 0
        }
    </node>
</view>

The language will undergo major refreshment over the next few months and documentation is to be written, but if you'd like to try it, feel free to do so.

I hope this helps.