Working with elasticsearch JSON format for tree
830 views
Skip to first unread message
kofiba...@gmail.com
Mar 14, 2018, 1:59:06 AM3/14/18
to vega-js
Hi, I’m trying to make a tree diagram but I’m having some trouble with getting my data in a way that things work nicely and some guidance would be appreciated. I’m just starting out so bear with me!
I’ve got a data set in a JSON that pre aggregates the data in already a "nest" structure (I’m using elasticsearch/kibana) and it looks like the "nodes" and "leaves" are having trouble getting the proper data because the Nest transform isn’t getting back with children that they can use. The datums have no children for some reason! I’m thinking it’s because the ES search is aggregating things and thus the tree is getting a different data format than it's used to. Is there some other transformation I can use for when I have JSON in the below format so that the tree diagram will work properly? Any guidance will be greatly appreciated!
My data is inputted into Vega in this form:
{
"aggregations": {
"InfluencerName": {
"buckets": [
{
"key": "Aubrey Graham (Drake)",
"doc_count": 4474,
"brand": {
"buckets": [
{
"key": "Nike",
"doc_count": 1250
},
{
"key": "Moncler",
"doc_count": 1120
},
{
"key": "OVO",
"doc_count": 762
}
]
}
},
{
"key": "Ariana Grande",
"doc_count": 3958,
"brand": {
"buckets": [
{
"key": "La Perla",
"doc_count": 485
},
{
"key": "Alex Woo",
"doc_count": 428
},
{
"key": "Levi's",
"doc_count": 375
}
]
}
}
]
}
My js is:
{
"$schema": "https://vega.github.io/schema/vega/v3.json",
"width": 960,
"height": 500,
"padding": 2.5,
"autosize": "none",
"signals": [
{
"name": "layout", "value": "squarify",
"bind": {
"input": "select",
"options": [
"squarify",
"binary",
"slicedice"
]
}
},
{
"name": "aspectRatio", "value": 1.6,
"bind": {"input": "range", "min": 0.2, "max": 5, "step": 0.1}
}
],
"data": [
{
"name": "tree",
"url": {
// %context%: true
// %timefield%: @timestamp
index: production
body : {
size:0,
aggs: {
InfluencerName: {
terms: {
field: personname.keyword
},
aggs: {
brand: {
terms: {
field: brandname.keyword,
size: 1
}
}
}
}
}
}
}
# We only need a specific array of values from the response
format: {property: "aggregations.InfluencerName.buckets" }
"transform": [
// {
// "type": "stratify",
// "key": "brand.bucket",
// "parentKey": "key"
// },
{
"type": "nest",
"key": " aggregations.InfluencerName.buckets.key",
"keys": ["aggregations.InfluencerName.buckets.key","brand.buckets.key"],
// "generate": true
},
{
"type": "treemap",
"field": " aggregations.InfluencerName.buckets.doc_count",
"sort": {"field": " aggregations.InfluencerName.buckets.doc_count"},
"round": true,
"method": {"signal": "layout"},
"ratio": {"signal": "aspectRatio"},
"size": [{"signal": "width"}, {"signal": "height"}]
}
]
},
{
"name": "nodes",
"source": "tree"
"transform": [{ "type": "filter", "expr": "datum.children" }]
},
{
"name": "leaves",
"source": "tree",
"transform": [{ "type": "filter", "expr": "!datum.children" }]
}
],
"scales": [
{
"name": "color",
"type": "ordinal",
"range": [
"#3182bd", "#6baed6", "#9ecae1", "#c6dbef", "#e6550d",
"#fd8d3c", "#fdae6b", "#fdd0a2", "#31a354", "#74c476",
"#a1d99b", "#c7e9c0", "#756bb1", "#9e9ac8", "#bcbddc",
"#dadaeb", "#636363", "#969696", "#bdbdbd", "#d9d9d9"
]
},
{
"name": "size",
"type": "ordinal",
"domain": [0, 1, 2, 3],
"range": [256, 28, 20, 14]
},
{
"name": "opacity",
"type": "ordinal",
"domain": [0, 1, 2, 3],
"range": [0.15, 0.5, 0.8, 1.0]
}
],
"marks": [
{
"type": "rect",
"from": {"data": "nodes"},
"interactive": false,
"encode": {
"enter": {
"fill": {"scale": "color", "field": "key"}
},
"update": {
"x": {"field": "x0"},
"y": {"field": "y0"},
"x2": {"field": "x1"},
"y2": {"field": "y1"}
}
}
},
{
"type": "rect",
"from": {"data": "leaves"},
"encode": {
"enter": {
"stroke": {"value": "#fff"}
},
"update": {
"x": {"field": "x0"},
"y": {"field": "y0"},
"x2": {"field": "x1"},
"y2": {"field": "y1"},
"fill": {"value": "transparent"}
},
"hover": {
"fill": {"value": "green"}
}
}
},
{
"type": "text",
"from": {"data": "nodes"},
"interactive": false,
"encode": {
"enter": {
"font": {"value": "Helvetica Neue, Arial"},
"align": {"value": "center"},
"baseline": {"value": "middle"},
"fill": {"value": "#000"},
"text": {"field": " aggregations.InfluencerName.buckets.key"},
"fontSize": {"scale": "size", "field": "depth"},
"fillOpacity": {"scale": "opacity", "field": "depth"}
},
"update": {
"x": {"signal": "0.5 * (datum.x0 + datum.x1)"},
"y": {"signal": "0.5 * (datum.y0 + datum.y1)"}
}
}
}
]
}
Message has been deleted
Roy I
Mar 14, 2018, 12:02:39 PM3/14/18
to vega-js
For Elastic Search, see this post:
and Vega visualization plugin for Kibana -- by nyurik (Yuri Astrakhan)
kofiba...@gmail.com
Mar 14, 2018, 7:18:20 PM3/14/18
to vega-js
I checked out the post, but if you had data in the structure above, how would you transform it to create a tree diagram with it?
Nest is the right way to go so far right?
Nest is the right way to go so far right?
Yuri Astrakhan
Mar 14, 2018, 9:40:29 PM3/14/18
to kofiba...@gmail.com, vega-js
copying my reply from https://github.com/nyurik/kibana-vega-vis/issues/50#issuecomment-373231300 :
I would recommend using ES composite aggregation instead of sub-aggregations. It makes things much easier to process in Vega, or at least until the plugin/core Kibana is updated to include the new Vega's flatten transform. Composite agg allows you to get a flat list of objects, making it much easier to manipulate.
I would recommend using ES composite aggregation instead of sub-aggregations. It makes things much easier to process in Vega, or at least until the plugin/core Kibana is updated to include the new Vega's flatten transform. Composite agg allows you to get a flat list of objects, making it much easier to manipulate.
--
You received this message because you are subscribed to the Google Groups "vega-js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vega-js+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages