Scaling the Y axis in plots

[Derek Sedunary]

I have added Matt's cmon service to weewx but some of the plots (as in the attached image) have really large Y values the do not fit within the image boundaries.

I could attempt to scale the original value but would rather just scale the axis (similar to what you can do in MS Excel plots) so that the sample plot would have a Y scale of 100K to 900K or some thing similar.

Any suggestion, the fine manual gives examples of range and interval but not scale.

Derek...

[Thomas Keffer]

I'm not sure what you're asking. It's possible to set ymin and ymax. Why not set them to 100k and 900k, respectively? Or, is it you want to specify the total range  (800k, in this case) and let it slide up and down as needed? If so, that's kind of an ill-defined concept.

-tk

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

--
You received this message because you are subscribed to the Google Groups "Weewx user's group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Derek Sedunary]

Why not set them to 100k and 900k, respectively?

I had not realised that I could.

So instead of the plot displaying 900000 on the Y scale it will display 900k?

Tried it and it did not work.
Oct 27 11:51:07 debian weewx[2878]: reportengine: Caught unrecoverable exception in generator weewx.imagegenerator.ImageGenerator
Oct 27 11:51:07 debian weewx[2878]:         ****  invalid literal for float(): 100k
Oct 27 11:51:07 debian weewx[2878]:         ****  Traceback (most recent call last):
Oct 27 11:51:07 debian weewx[2878]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 131, in run
Oct 27 11:51:07 debian weewx[2878]:         ****      obj.start()
Oct 27 11:51:07 debian weewx[2878]:         ****    File "/home/weewx/bin/weewx/reportengine.py", line 294, in start
Oct 27 11:51:07 debian weewx[2878]:         ****      self.run()
Oct 27 11:51:07 debian weewx[2878]:         ****    File "/home/weewx/bin/weewx/imagegenerator.py", line 38, in run
Oct 27 11:51:07 debian weewx[2878]:         ****      self.genImages(self.gen_ts)
Oct 27 11:51:07 debian weewx[2878]:         ****    File "/home/weewx/bin/weewx/imagegenerator.py", line 103, in genImages
Oct 27 11:51:07 debian weewx[2878]:         ****      plot.setYScaling(weeutil.weeutil.convertToFloat(plot_options.get('yscale')))
Oct 27 11:51:07 debian weewx[2878]:         ****    File "/home/weewx/bin/weeutil/weeutil.py", line 30, in convertToFloat
Oct 27 11:51:07 debian weewx[2878]:         ****      res = [None if s in ('None', 'none') else float(s) for s in seq]
Oct 27 11:51:07 debian weewx[2878]:         ****  ValueError: invalid literal for float(): 100k
Oct 27 11:51:07 debian weewx[2878]:         ****  Generator terminated...

Derek...

[Derek Sedunary]

I'm not sure what you're asking.

The numbers on the Y scale are too long (too many zeros) and I would like to covert the scale into engineering units to remove the zeros so I can read what the number is.

At the moment I can see 5 zeros but not the leading digit.  I would like to be able to convert 500000 to 500k without having to edit cmon.py to scale the values stored in the db.

Derek..

[Thomas Keffer]

Oh! Now I get it! I'm sorry, I didn't realize you meant the string "100k" literally. 

You can't do that, but you can request an exponent form by specifying option "y_label_format". It would look something like:

[[[daybyte_plot]]]

  y_label_format = "%e"

    [[[[obs_type1]]]]

    [[[[obs_type2]]]]

-tk

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

--

[Derek Sedunary]

  y_label_format = "%e"

Thanks for that.

The Y axis now shows exponential notation. The only problem is that exponential notation takes 5 characters to replace 5 zeros, so I have not actually gained anything other than learning something new with programming which is always a bonus.

I have managed to scale the disk usage plot so it now shows megabytes rather than kilobytes by changing some values in cmon.py. I will try altering some of the other plots the same way.

Thanks.

Derek...

[Thomas Keffer]

It's a reasonable request. There might be a way to add a scaling function easily. I'll think about it!

-tk

Tom Keffer

kef...@threefools.org
+1 541-386-8891 (h)
+1 541-490-9507 (c)

Skype: tkeffer

--

[Steve]

Hi Derek, I hope you're still watching this.

Do you mind sharing what values you changed in cmon.py to get displayable scales? I have the same issue.

Regards,

Steve. 

[Derek Sedunary]

I will have a look at it to night and post some instructions of what I did to get around the problem. May be Matt will come up with a better solution.

Derek...

[mwall]

On Thursday, December 26, 2013 3:54:03 PM UTC-5, Derek Sedunary wrote:

I will have a look at it to night and post some instructions of what I did to get around the problem. May be Matt will come up with a better solution.

here is a rather invasive approach, but it works.  a cmon plot is attached.

i patched units.py so that it will recognize the data labels from cmon, and so that it will understand bytes, kilobytes, megabytes, gigabytes, and terabytes.  these are in a new group called group_data.  technically it is only scaling from one to another, not units conversions.  but weewx does not (yet) distinguish between units conversions and scaling.

i consider this a hack job.  or perhaps proof-of-concept to explore our options.  long-term we should make units extensible instead of hard-coding new types into the group and conversion definitions.

here is how to do it:

1) install the attached units.py

mv bin/weewx/units.py bin/weewx/units.py-orig
cp /var/tmp/units.py bin/weewx/units.py

2) add a few entries to the Units section of your skin.conf

[Units]
    [[Groups]]
        ...
        group_data = gigabyte

    [[StringFormats]]
        ...
        byte              = %.0f
        kilobyte          = %.2f
        megabyte          = %.2f
        gigabyte          = %.2f
        terabyte          = %.2f

    [[Labels]]
        ...
        byte              = " B"
        kilobyte          = " KB"
        megabyte          = " MB"
        gigabyte          = " GB"
        terabyte          = " TB"

3) restart weewx

sudo /etc/init.d/weewx stop
sudo /etc/init.d/weewx start

m

[Steve]

Thanks for the replies Matthew & Derek,

I'm using the new units.py and added the group, string formats & labels to my skin.conf but for some reason it looks like it has interfered with my day charts, but not my weekly charts, I didn't make any other changes to my skin.conf at the time, Any clues? http://wotid.dyndns.org/weather/cpu.html 

(daymem & week mem attached).

Regards,

Steve.

[Steve]

A bit more of a play and it's working.

In my skin.conf I changed the group_data from gigabyte to megabyte:

group_data = megabyte # added for cmon values

Then my day cpu memory values appeared again.

It might have also had something to do with me not having and aggregate_type and aggregate_interval in my day_cpu_images? I copied and pasted the existing day_images and week_images defaults into my new xxx_cpu_images, that I then altered for different colours etc. I've not played to confirm.

week_cpu_images

    [[week_cpu_images]]
 image_background_color = 0xffffff
 chart_background_color = 0xffffff
 show_daynight = false
 image_width = 640
 image_height = 210
 x_label_format = %d
 bottom_label_format = %d/%m/%y %H:%M
 time_length = 604800    # == 7 days
 aggregate_type = avg
 aggregate_interval = 360

New images attached.

Regards,

Steve.

[vds]

Is it possible to set different units for the different plots ?

Issue I'm having here is that I have a small memory system (128MB) with a mess'o'disk space (90GB) so I'd like to get different units for each graph, using MB for memory and GB for disk. Currently I set group_data to megabytes as I want that one most accurate normally (the disk runs only 1% used currently).

Here's a record from my computer.sdb in case that'll help...

PRAGMA foreign_keys=OFF;

BEGIN TRANSACTION;

CREATE TABLE archive (`dateTime` INTEGER NOT NULL PRIMARY KEY, `usUnits` INTEGER, `mem_total` INTEGER, `mem_free` INTEGER, `mem_used` INTEGER, `swap_total` INTEGER, `swap_free` INTEGER, `swap_used` INTEGER, `cpu_user` INTEGER, `cpu_nice` INTEGER, `cpu_system` INTEGER, `cpu_idle` INTEGER, `cpu_iowait` INTEGER, `cpu_irq` INTEGER, `cpu_softirq` INTEGER, `load1` REAL, `load5` REAL, `load15` REAL, `proc_active` INTEGER, `proc_total` INTEGER, `net_eth0_rbytes` INTEGER, `net_eth0_rpackets` INTEGER, `net_eth0_rerrs` INTEGER, `net_eth0_rdrop` INTEGER, `net_eth0_tbytes` INTEGER, `net_eth0_tpackets` INTEGER, `net_eth0_terrs` INTEGER, `net_eth0_tdrop` INTEGER, `disk_root_total` INTEGER, `disk_root_free` INTEGER, `disk_root_used` INTEGER);

INSERT INTO "archive" VALUES(1388204690,1,123328,19772,103556,14466528,14450968,15560,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0.31,0.18,0.22,3,119,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,96132940,89851088,1398496);

[Derek Sedunary]

Steve

This is a much better solution than my method of editing cmon.py and adding multiple "divide by 1024's" to get the scaling more suitable for graphing.

I am trying Matt's solution on my system now.

It was certainly a lot easier to implement than my way.

Derek...

[Derek Sedunary]

Looking in units.py this section has been added:

                  "mem_total"          : "group_data",
                  "mem_free"           : "group_data",
                  "mem_used"           : "group_data",
                  "swap_total"         : "group_data",
                  "swap_free"          : "group_data",
                  "swap_used"          : "group_data",
                  "net_eth0_rbytes"    : "group_data",
                  "net_eth0_tbytes"    : "group_data",
                  "disk_root_total"    : "group_data",
                  "disk_root_used"     : "group_data",

Maybe you could edit     "disk_root_total" : "group_data" and  "disk_root_used" : "group_data" to some thing like "disk_root_total" : "group_datadisk" and  "disk_root_used" : "group_datadisk" and then add a line in skin.conf in the[Units]/[[Groups]]section that says group_datadisk = gigabyte, then you could have a group_data in megabytes and group_datadisk in gigabytes (this is just a guess, I have not tried it).

Derek...

[Derek Sedunary]

There is a bit more to it than my suggestion above, so don't bother trying that, it doesn't work.

Derek..

[Derek Sedunary]

I think you also need to add the line:
               "group_datadisk"        : "kilobyte",
to the US customary unit system: section and to the metric unit system:

I have tried it but for some reason my cmon plots only update every 10 minutes (all the the others are 5 minutes) so I need to wait another 5 minutes to see if it works.

Derek...

[Derek Sedunary]

Yep, that worked.

Drive plot in gigabytes and memory plot in megabytes

Derek...

[Derek Sedunary]

To sum it all.

1. Edit Matt's units.py file above and find these three sections:

# This data structure maps observation types to a "unit group"
obs_group_dict = {"altitude"           : "group_altitude",
                 .....

                  "mem_total"          : "group_data",
                  "mem_free"           : "group_data",
                  "mem_used"           : "group_data",
                  "swap_total"         : "group_data",
                  "swap_free"          : "group_data",
                  "swap_used"          : "group_data",
                  "net_eth0_rbytes"    : "group_data",
                  "net_eth0_tbytes"    : "group_data",
                  "disk_root_total"    : "group_data",
                  "disk_root_used"     : "group_data",
                  }

# This dictionary maps unit groups to a standard unit type in the
# US customary unit system:
USUnits     = {"group_altitude"    : "foot",
              .....
               "group_volt"        : "volt",
               "group_data"        : "kilobyte",
               }

# This dictionary maps unit groups to a standard unit type in the
# metric unit system:
MetricUnits = {"group_altitude"    : "meter",
              .....
               "group_volt"        : "volt",
               "group_data"        : "kilobyte",
               }

2. Change the highlighted bits to match this:

# This data structure maps observation types to a "unit group"
obs_group_dict = {"altitude"           : "group_altitude",
                 .....

                  "mem_total"          : "group_data",
                  "mem_free"           : "group_data",
                  "mem_used"           : "group_data",
                  "swap_total"         : "group_data",
                  "swap_free"          : "group_data",
                  "swap_used"          : "group_data",
                  "net_eth0_rbytes"    : "group_data",
                  "net_eth0_tbytes"    : "group_data",

                  "disk_root_total"    : "group_datadisk",
                  "disk_root_used"     : "group_datadisk",               
                  }


# This dictionary maps unit groups to a standard unit type in the
# US customary unit system:
USUnits     = {"group_altitude"    : "foot",
              .....
               "group_volt"        : "volt",
               "group_data"        : "kilobyte", 
               "group_datadisk"    : "kilobyte",
               }

# This dictionary maps unit groups to a standard unit type in the
# metric unit system:
MetricUnits = {"group_altitude"    : "meter",
              .....
               "group_volt"        : "volt",
               "group_data"        : "kilobyte",
               "group_datadisk"    : "kilobyte",
              }

3. Edit skin.conf to add the new group:

[Units]
    [[Groups]]
        ...
        group_data = megabyte
        group_datadisk = gigabyte

4. Restart weewx:

sudo /etc/init.d/weewx stop
sudo /etc/init.d/weewx start

5. Wait for the plots to be generated, the disk_root plot should be in gigabytes and the remaining memory/data plots in megabytes.

Well, it worked for me, try it at your own risk, you can always go back to the original files and restart weewx.

Derek...

[Steve]

Worked like a charm for me thanks Derek.

On Sunday, 29 December 2013 11:19:06 UTC+11, Derek Sedunary wrote:

To sum it all.

[vds]

On Saturday, December 28, 2013 6:47:52 PM UTC-8, Steve wrote:

Worked like a charm for me thanks Derek.

Me too :-)

I have mine set to MB for memory and network, and GB for disk.

In units.py:

                  "mem_total"          : "group_datamem",

                  "mem_free"           : "group_datamem",

                  "mem_used"           : "group_datamem",

                  "swap_total"         : "group_datamem",

                  "swap_free"          : "group_datamem",

                  "swap_used"          : "group_datamem",

                  "net_eth0_rbytes"    : "group_datanet",

                  "net_eth0_tbytes"    : "group_datanet",

                  "disk_root_total"    : "group_datadisk",

                  "disk_root_used"     : "group_datadisk",

               "group_data"        : "kilobyte",

               "group_datadisk"    : "kilobyte",

               "group_datanet"     : "kilobyte",

               "group_datamem"     : "kilobyte",

               "group_data"        : "kilobyte",

               "group_datadisk"    : "kilobyte",

               "group_datanet"     : "kilobyte",

                "group_datamem"       : "kilobyte",

In skin.conf:

group_data       = kilobyte

group_datadisk   = gigabyte

group_datanet    = megabyte

group_datamem    = megabyte

 

[Andrej B.]

Happy New Year!

Maybe someone can suggest - approach to scaling Y axis units in cmon explained in this thread will work witch the newest weewx 3.8.2 ?

Thanks!

Andrej

[Andrej B.]

Versti pranešimą į lietuvių  

On Thursday, December 26, 2013 3:54:03 PM UTC-5, Derek Sedunary wrote:

I will have a look at it to night and post some instructions of what I did to get around the problem. May be Matt will come up with a better solution.

here is a rather invasive approach, but it works.  a cmon plot is attached.

i patched units.py so that it will recognize the data labels from cmon, and so that it will understand bytes, kilobytes, megabytes, gigabytes, and terabytes.  these are in a new group called group_data.  technically it is only scaling from one to another, not units conversions.  but weewx does not (yet) distinguish between units conversions and scaling.

i consider this a hack job.  or perhaps proof-of-concept to explore our options.  long-term we should make units extensible instead of hard-coding new types into the group and conversion definitions.

here is how to do it:

1) install the attached units.py

mv bin/weewx/units.py bin/weewx/units.py-orig
cp /var/tmp/units.py bin/weewx/units.py

mwall Is possible to update units.py file to meet weewx 3.8.2 syntax (old attached file does not work) Thanks Andrej

2013-12-27

[gjr80]

This one has been around for a while, in fact it is issue #1 on github. I think we have been around the buoy a few times on this, I remember various discussions about kB, MB, GB etc but I don't think it went anywhere, perhaps that is indicative of the age of issue #1 :)

Maybe something to live (all our individual fixes) with until/if the weeWX plot engine is changed/updated/rewritten ?

Gary