Hierarchical data in pxWeb

[Erika]

Hi everyone,

I work at the Swedish Higher Education Authority (UKÄ), and we are currently exploring pxWeb as a potential solution for our statistical database. Part of our evaluation involves assessing whether the px format is compatible with our data structure.

 

We have some hierarchical data organized across multiple levels with various subgroups. Currently, our users can download files in long format and create pivot tables as needed. However, with the px format’s matrix/cube structure, all possible combinations between each level are generated, including those that aren’t valid in our context.

 

I have seen the suggestion to combine columns to reduce the dimension of the cube. I have also seen that there are .vs and .agg files, and the keyword HIERARCHIES.  

 

What are the potential approaches for managing this type of hierarchical data in pxWeb? Additionally, if there are any relevant examples, I would greatly appreciate any insights.

 

Thanks in advance!

Best regards,

Erika

[Hans Baumgartner]

Hi,

Would be nice to see some of your problematic data “tables”

PxWeb is mainly meant for dissemination of AGGREGATED statistical data (tables).
Remember I am talking about the px-file base PxWeb here.

> I have also seen that there are .vs and .agg files.

Example of vs and aggregarion files usage can be seen in this table:
http://tkpxhopea01p.valtion-ext.fi/pxweb
All the subtables are aggregated on the fly from the original px-table by selecting from the pull down menus “area” and “age”

Example of table with 104 series in one table (Variable (selection box)  “Information” has 104 statistical series)
https://pxweb2.stat.fi/PxWeb/pxweb/en/Postinumeroalueittainen_avoin_tieto/Postinumeroalueittainen_avoin_tieto__uusin/paavo_pxt_12f8.px/
Or 107 series by postal code
https://pxweb2.stat.fi/PxWeb/pxweb/en/Postinumeroalueittainen_avoin_tieto/Postinumeroalueittainen_avoin_tieto__uusin/paavo_pxt_12f7.px/  

In some cases we just put the whole hierarchical variable in one long variable (selection box)
https://pxweb2.stat.fi/PxWeb/pxweb/en/Postinumeroalueittainen_avoin_tieto/Postinumeroalueittainen_avoin_tieto__uusin/paavo_pxt_12f8.px/
Look at the variable (selection box) Area

Or possibly like this with CODES added also in the text field:
https://pxweb2.stat.fi/PxWeb/pxweb/en/StatFin/StatFin__alyr/statfin_alyr_pxt_13ww.px/
look at the variable (selection box) “Industry (Tol 2008)”

>and the keyword HIERARCHIES.  

We do not use the keyword HIERARCHIES at all. Only the user interface has a rudimentary support for it.

 

> I have seen the suggestion to combine columns to reduce the dimension of the cube

Remember that there is a big possibility to produce enormous px-tables if you have a lot of variables and combinations in one table.
Combining variable is a good idea if the table is huge and the cube format forces all the combinations including those that have no real data.

We have one px-database where we have combined variables (px-variables (selection boxes)), but do you really need it?

I would keep the px-filesizes well under 1-10 million cells if possible, we go to over 50 million, but it is too much.

Feel free to roam in our px-file based Statistical databases:
https://pxweb2.stat.fi/PxWeb/pxweb/en/StatFin/      This is our main Open Data database!
https://pxweb2.stat.fi/PxWeb/pxweb/en/
https://tieliikenneonnettomuudet.stat.fi/PXWeb/pxweb/en/Tieliikenneonnettomuudet/

We  also produce and maintain PxWeb services for other organizations too:
https://vero2.stat.fi/PXWeb/pxweb/en/Vero/
https://visitfinland.stat.fi/PXWeb/pxweb/en/VisitFinland/
https://trafi2.stat.fi/PXWeb/pxweb/en/TraFi/
https://kototietokanta.stat.fi/PXWeb/pxweb/en/Kototietokanta/

We also have a lot of chargeable PxWeb databases …

Hans Baumgartner

 

Lähettäjä: pca...@googlegroups.com <pca...@googlegroups.com> Puolesta Erika
Lähetetty: perjantai 15. marraskuuta 2024 13.59
Vastaanottaja: pcaxis <pca...@googlegroups.com>
Aihe: Hierarchical data in pxWeb

--
You received this message because you are subscribed to the Google Groups "pcaxis" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pcaxis+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pcaxis/8023cd90-ca70-4b88-9ea4-bfdbe884d74cn%40googlegroups.com.