Visualisation of data using the Highcharts charting library

[Alan Major]

Hi,

Now that I have over a year's worth of data generated by the combination of a Davis Vantage PRO2 weather station and WeeWX software running on a Raspberry Pi I would like to present comparatives between the current and previous year's statistical data in a more visual way. To do this I have decided to use the Highcharts charting library.

My work flow involves pre-processing the data needed for a chart by specifying a template file in the 'SummaryByYear' section of skin.conf to create a file containing an array of the required data. Unfortunately what I found is that by doing this the year is being duplicated in the list used by the drop down box for selecting the NOAA reports. See here for my attempts so far and the problem with the '-Select Year-' drop down.

Looking through the code in cheetahgenerator.py it seems that a fix could be achieved by adding a test to see if the year already exists in 'SummaryByYear' before appending the year. However, I am only an enthusiastic dabbler when it comes to all this stuff and have only got this far by a lot of 'trial and error' and I do not have a clue as to how to write the code which needs to inserted into the cheetahgenerator.py file to perform this test.

Would someone be able to help me or alternatively suggest a better way of preventing the years being duplicated in the SummaryByYear list when multiple templates are being used to generate SummaryByYear files.

Very many thanks.

Alan Major
Birdham, UK

[mwall]

On Tuesday, January 14, 2014 7:02:18 AM UTC-5, Alan Major wrote:

Looking through the code in cheetahgenerator.py it seems that a fix could be achieved by adding a test to see if the year already exists in 'SummaryByYear' before appending the year. However, I am only an enthusiastic dabbler when it comes to all this stuff and have only got this far by a lot of 'trial and error' and I do not have a clue as to how to write the code which needs to inserted into the cheetahgenerator.py file to perform this test.

alan,

nice graphical summaries of multi-year data!

could you post the relevant sections of the skin.conf and/or weewx.conf?

m

[Alan Major]

Matthew

This is the relevant section of the skin.conf in which I request the data files to be created:

###############################################################################

[CheetahGenerator]
    # This section is used by the generator CheetahGenerator, and specifies
    # which files are to be generated from which template.

    # Possible encodings are 'html_entities', 'utf8', or 'strict_ascii'
    encoding = html_entities

    search_list_extensions = examples.xsearch.MyXSearch, user.forecast.ForecastVariables

    [[SummaryByMonth]]
        # Reports that summarize "by month"
        [[[NOAA_month]]]
            encoding = strict_ascii
            template = NOAA/NOAA-YYYY-MM.txt.tmpl

    [[SummaryByYear]]
        # Reports that summarize "by year"
        [[[NOAA_year]]]
            encoding = strict_ascii
            template = NOAA/NOAA-YYYY.txt.tmpl
           
        [[[Data_year]]]
            encoding = utf8
            template = Data/Data-YYYY.json.tmpl
       
        [[[RainTraceDays]]]
            encoding = utf8
            template = Data/RainTraceDays-YYYY.json.tmpl
       
        [[[RainDays]]]
            encoding = utf8
            template = Data/RainDays-YYYY.json.tmpl
       
        [[[RainSoakDays]]]
            encoding = utf8
            template = Data/RainSoakDays-YYYY.json.tmpl
           
        [[[Rain]]]
            encoding = utf8
            template = Data/Rain-YYYY.json.tmpl

As a result of calling the 6 different templates a year appears 6 times in the list of years!

Thanks

Alan Major
Birdham, UK

[mwall]

On Tuesday, January 14, 2014 10:22:32 AM UTC-5, Alan Major wrote:

As a result of calling the 6 different templates a year appears 6 times in the list of years!

alan,

please modify cheetahgenerator.py, changing this (around line 229):

                    if period == 'SummaryByYear':
                        self.outputted_dict['SummaryByYear'].append(_yr_str)

to this:
                    if period == 'SummaryByYear' and not _yr_str in self.outputted_dict['SummaryByYear']:
                        self.outputted_dict['SummaryByYear'].append(_yr_str)

[Alan Major]

Matthew

Thank you for your help. Code modified and after a restart all working correctly. Just need to remember that I made this change when it comes time to upgrade. :-)

Alan

[mwall]

On Tuesday, January 14, 2014 12:21:37 PM UTC-5, Alan Major wrote:

Matthew

Thank you for your help. Code modified and after a restart all working correctly. Just need to remember that I made this change when it comes time to upgrade. :-)

the fix has been applied at r1892.  it will be included in the next weewx release.

m

[Alan Major]

What service - it is the support available through this group and people like you that makes an already brilliant piece of software stand head and shoulders above the rest. Thank you.

[vds]

On Tuesday, January 14, 2014 4:02:18 AM UTC-8, Alan Major wrote:

Now that I have over a year's worth of data generated by the combination of a Davis Vantage PRO2 weather station and WeeWX software running on a Raspberry Pi I would like to present comparatives between the current and previous year's statistical data in a more visual way. To do this I have decided to use the Highcharts charting library.

My work flow involves pre-processing the data needed for a chart by specifying a template file in the 'SummaryByYear' section of skin.conf to create a file containing an array of the required data. 

I really like the temperature charts and the wind roses.  Would it be possible to pass along more details on how you preprocess things and what scripts you cooked up to generate the images ?   I have data here going back to Nov 2006 and I'd love to start cooking up data showing historical highs/lows versus current highs/lows

example - a chart with the highs and lows per day of the year, with this year's highs/lows overlaid onto it - the Seattle Times does this every month plus it does a previous-year chart usually on January 2nd -  a typical monthly link is http://seattletimes.nwsource.com/ABPub/2011/03/01/2014368241.pdf

 

[Alan Major]

Vince

The temperature chart uses the 'Box plot' chart type from the Highcharts portfolio. The attached tmpl file creates the data array used by the javascript shown below which draws the chart and is embedded in the html tmpl file for the page (don't forget to add the 'Chart1' id to the container in the html code where to want the chart to appear and to adjust the location of the src files to suit your installation).

<script src="js/jquery-1.8.2.min.js"></script>
<script src="js/highcharts.js"></script>
<script src="js/highcharts-more.js"></script>
<script src="js/themes/grid.js"></script>
<script>
    $(document).ready(function() {

        var options = {
            chart: {
                renderTo: 'Chart1',
                type: 'boxplot'
            },
           
            title: {
                text: '2013/14 Monthly Temperature Statistics'
            },
           
            subtitle: {
                text: '(Max, Avg Max, Mean, Avg Min, Min)'
            },
           
            legend: {
                enabled: true
            },
       
            xAxis: {
                categories: ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec']
            },
           
            yAxis: {
                minorTickInterval: 'auto',
                minorTickLength: 0,
                title:  {
                    text: 'Degrees C'
                }
            },
           
            plotOptions: {
                boxplot: {
                    fillColor: 'LightSteelBlue',
                    groupPadding: 0.2,
                    lineWidth: 3,
                    medianWidth: 3,
                    stemWidth: 3,
                    whiskerLength: '75%',
                    whiskerWidth: 3
                }
            },

            tooltip: {
                enabled: false
            },
            series: [{
                name: '2013',
                pointPadding: 0.25
            }, {
                name: '2014',
                pointPadding: 0.25
            }]
        };

        $.getJSON('Data/Data-2013.json', function(data) {
            options.series[0].data = data;
            $.getJSON('Data/Data-2014.json', function(data) {
            options.series[1].data = data;
            var chart = new Highcharts.Chart(options);
            });
        });

    });    
    </script>

Sadly the wind roses are nowhere near as automated - they are static images which are manually hand cranked using an Excel spreadsheet to extract the data from the archive database and massage it into shape before plotting in Excel using the Radar chart type.

HTH

Alan Major
Birdham, UK