Skewed Coordinate System?

[Patrick Marsh]

In Meteorology we use a diagram known as a Skew-T/Log-P. In short, the x-coordinate is temperature and the y-coordinate is pressure. The temperature lines (x-coordinate) are skewed such that "vertical lines" (in reality, lines of constant X) are skewed to the right by some angle. Pressure (y-coordinate) is plotted on a log scale. I've attached a sample image of a Skew-T/Log-P Diagram.

A friend of mine and I worked this past winter to add the ability to plot Skew-T/Log-P diagrams in Matplotlib. (It's currently sitting in a pull request that I need to finish.) However, MPL is too slow for a lot of what I want to do down the road, so I decided to give pyqtgraph a go. Unfortunately, after two days, I cannot get the skewed x-axis to work.

My question: is it even possible to generate a skewed coordinate system?

Below is a sample script I put together to try to even get something skewed. It appears that temperature (red line) is different, but I'm not sure how/why, nor how to recreate the figure attached.

Thanks for any help/advice.

import sys

import numpy as np

import pyqtgraph as pg

from pyqtgraph.Qt import QtGui, QtCore

app = QtGui.QApplication(sys.argv)

mainWindow = QtGui.QMainWindow()

mainWindow.resize(800,600)

mainWindow.show()

title = 'Sounding and Hodograph Analysis and Research Program in Python'

mainWindow.setWindowTitle(title)

mainWindow.setStyleSheet("background-color: rgb(128, 128, 128);")

centralWidget = QtGui.QWidget()

mainWindow.setCentralWidget(centralWidget)

centralWidget.setObjectName("centralwidget")

layout = QtGui.QVBoxLayout()

centralWidget.setLayout(layout)

pw1 = pg.PlotWidget(name='Plot1')

layout.addWidget(pw1)

pres = [969., 925., 850., 700., 500., 400., 300., 250., 200., 150., 100.]

tmpc = [34.2, 30., 22., 15.6, -6.3, -13.3, -31.3, -40.9, -53.1, -67.9, -68.3]

pres = np.asarray(pres)

tmpc = np.asarray(tmpc)

p1 = pw1.plot()

p1.setPen('r')

p1.setData(x=tmpc, y=pres)

p1.getViewBox().invertY()

pw1.addLine(0, pen={'color': (200, 200, 255), 'style': QtCore.Qt.DotLine})

skewTransform = QtGui.QTransform()

skewTransform.shear(np.deg2rad(30.), 0)

p1.setTransform(skewTransform)

mainWindow.show()

app.exec_()

[Luke Campagnola]

On Fri, Jul 26, 2013 at 4:03 PM, Patrick Marsh <patrick...@gmail.com> wrote:

In Meteorology we use a diagram known as a Skew-T/Log-P. In short, the x-coordinate is temperature and the y-coordinate is pressure. The temperature lines (x-coordinate) are skewed such that "vertical lines" (in reality, lines of constant X) are skewed to the right by some angle. Pressure (y-coordinate) is plotted on a log scale. I've attached a sample image of a Skew-T/Log-P Diagram.

A friend of mine and I worked this past winter to add the ability to plot Skew-T/Log-P diagrams in Matplotlib. (It's currently sitting in a pull request that I need to finish.) However, MPL is too slow for a lot of what I want to do down the road, so I decided to give pyqtgraph a go. Unfortunately, after two days, I cannot get the skewed x-axis to work.

My question: is it even possible to generate a skewed coordinate system?

Below is a sample script I put together to try to even get something skewed. It appears that temperature (red line) is different, but I'm not sure how/why, nor how to recreate the figure attached.

Thanks for any help/advice.

The short answer is that pretty much anything is possible, since we are ultimately just drawing with Qt's basic primitives. The question is how much effort will this take, and will it be worth your while. I think that the approach you have chosen--adding skew to the transform of a plot line--is a perfectly valid approach. You can see that it is working by plotting the same data in a separate color but without the skew applied. 

If you wanted to transform multiple objects in the same way (say, several plots + vertical grid lines + circular grid lines), then it might make more sense to put all of these objects inside a QGraphicsItemGroup, then apply the skew transform to the group. This would ensure that all items are sheared in the same way

Another approach would be to write the plot values into numpy arrays, and then transform those values before plotting them (this function is helpful: http://pyqtgraph.org/documentation/functions.html#pyqtgraph.transformCoordinates ). Internally, this is how the log transformation is applied since QTransform only supports affine transformations.

I think in order to make better suggestions on the best approach, I'll need a better idea of how you want the plot to behave:

- Will the user be able to scale / pan the plot interactively?

- How are the curved grid lines defined?

- Can you send a test script that would serve to demonstrate whether the transformation is working correctly? (perhaps by showing a matplotlib and pyqtgraph plot together?)

As an example, I have attached a modification of your script that does the following:

- Logarithmic scaling on y-axis

- Horizontal grid lines provided by the plot

- Static set of vertical grid lines

- Grid lines and t/p plot are both children of a group with a shear transformation

Note that the values on the x-axis do not take into account the shear transformation; a little more effort would be required to make this correction. Also, the shear value that goes into the group transformation is arbitrary, since we would have to take into account the x/y scale factors in order to comput the correct factor for a 30 degree skew.

 Luke

[Patrick Marsh]

Hi, Luke,

Thanks for the prompt/detailed reply. Hopefully my response provides additional insights into what I hope to do. I preface everything with the statement I am a novice when it comes to using Qt and/or pyqtgraph.

In late 2010 I embarked to create the application I previously described. I started with Tkinter because it shipped with Python. Unfortunately, Tkinter was so primitive that I didn't see it as a long term solution. You can see what I accomplished in the following. Ultimately, this is the initial functionality I wish to recreate, before moving toward adding the additional information shown in the image I attached in my initial post.

The data crunching program: https://github.com/pmarshwx/SHARPpy-orig (it is pure Python so it should install trivially)

The script to run the program: https://gist.github.com/pmarshwx/1668504  (to run this script do: 'python run_sharppy.py OUN')

You'll create images such as those in this blog post: http://www.pmarshwx.com/blog/2012/01/11/sharppy-preview-ams-presentation/.

I had to manually create the transformation for plotting everything; and manually call the transformation on every plot object and on all the cursor readout information.

At SCIPY 2012 I was finally convinced to give up my pursuit with Tkinter and utilize Matplotlib. Instead of trying to recreate the manual transformation object, Ryan May and I embarked on adding a skew method to the underlying Maplotlib Affine transformation class. The thought here was instead of trying to transform each individual object (and keep track of all of them), let's just transform the underlying coordinate system such that we get all the cursor readout, vertical lines, plotting, etc for "free". You can see the fruits of our labor in this Matplotlib pull request (which hasn't been committed): https://github.com/matplotlib/matplotlib/pull/1664. A static image of the result can be found here: https://f.cloud.github.com/assets/221526/70340/607ed056-5f98-11e2-9117-0e349fae22bf.png. And the code to create that image (including overriding some classes) can be found here: https://gist.github.com/dopplershift/3149192.

One issue I've had with Maptlotlib is that it is very slow at rendering and, to be honest, the plots look a bit cartoonish in some respects. Additionally, I was going to have to embed Matplotlib into something as I need to add multiple widgets (including text/tables). All of this had me start looking for something else…when I was shown pyqtgraph. I loved the pyqtgraph examples for their speed, their relative simplicity, and their crispness. And, to be honest, I liked the fact I was closer to the underlying graphics engine. So, early in the week, I started over for a third (and hopefully) last time. I realize that I will go through some growing pains on the front end, but I think in the long run, this is a much better solution…assuming I can pick it up.

Now, to address some of your specific questions:

- Will the user be able to scale / pan the plot interactively?

Eventually I would like this. I've noticed that the pyqtgraph objects have all of this hooked up (or am I mistaken?) on the front end, which is one reason why I thought about staying at the pyqtgraph level rather than just staying down in the Qt primitives.

 

- How are the curved grid lines defined?

I am not entirely sure which curved lines you are talking about? The ones that start from bottom right and go to the upper left? These are actually straight lines (lines of constant potential temperature [known as dry adiabats in Skew-T lingo]). They appear curved as a result of the log scaling. There are actually curved lines that go from the bottom right to the upper left (not shown in the original image). These are known as moist adiabatic, and they are computed through a helper function that iterates "up through the atmosphere" to calculate what that curve is. Ultimately, I'm just plotting pres and temp for all the curves.

 

- Can you send a test script that would serve to demonstrate whether the transformation is working correctly? (perhaps by showing a matplotlib and pyqtgraph plot together?)

Hopefully some of the information linked at the top here will help answer this for you.

 

As an example, I have attached a modification of your script that does the following:

- Logarithmic scaling on y-axis

- Horizontal grid lines provided by the plot

- Static set of vertical grid lines

- Grid lines and t/p plot are both children of a group with a shear transformation

Thanks! This looks great, at least for the "grid lines". The temperature curve still doesn't look like it is skewed properly. When the temperature curve is skewed correctly, it should be more vertical than horizontal. 

 

Note that the values on the x-axis do not take into account the shear transformation; a little more effort would be required to make this correction. Also, the shear value that goes into the group transformation is arbitrary, since we would have to take into account the x/y scale factors in order to comput the correct factor for a 30 degree skew.

This is great to know! In previous versions I tried to emulate the coordinate transformations we applied to the Matplotlib code. However, with Qt/pyqtgraph I could not figure out what the units in the shear needed to be.

I guess ultimately the first thing I need to decide is to try and wrap this transformation into the underlying QT architecture (similar to what we've done with Matplotlib) or write a stand-alone mapping function that sits outside the plotting function and is just called manually on everything (a la the Tkinter example).

Thanks again for any insight you may have. I appreciate that you are most likely busy and that you don't necessarily have any obligation to me!

[Luke Campagnola]

On Sat, Jul 27, 2013 at 1:13 PM, Patrick Marsh <patrick...@gmail.com> wrote:

Now, to address some of your specific questions:

- Will the user be able to scale / pan the plot interactively?

Eventually I would like this. I've noticed that the pyqtgraph objects have all of this hooked up (or am I mistaken?) on the front end, which is one reason why I thought about staying at the pyqtgraph level rather than just staying down in the Qt primitives.

I'm not sure what "hooked up on the front end means", but I can explain how it works in pyqtgraph: Each ViewBox instance contains a QGraphicsItemGroup which is the parent of all items displayed inside the view. When the user scales or pans the view, all we do internally is update the transformation for the group item and Qt takes care of transforming all of its children to match. This means that I never need to transform the actual data coordinates manually as long as the transformation is supported by Qt.

Allowing user-interactive scaling would be interesting (probably not too difficult) since as you scale either axis, the shear factor needed to maintain a 30-degree angle would change. You might also want to dynamically adjust the spacing and extent of the vertical grid lines.

  

- How are the curved grid lines defined?

I am not entirely sure which curved lines you are talking about? The ones that start from bottom right and go to the upper left? These are actually straight lines (lines of constant potential temperature [known as dry adiabats in Skew-T lingo]). They appear curved as a result of the log scaling. There are actually curved lines that go from the bottom right to the upper left (not shown in the original image). These are known as moist adiabatic, and they are computed through a helper function that iterates "up through the atmosphere" to calculate what that curve is. Ultimately, I'm just plotting pres and temp for all the curves.

Ok, in that case these could be drawn easily by plotting many points along a straight line, then letting pyqtgraph handle the log and shear transformations.

   

As an example, I have attached a modification of your script that does the following:

- Logarithmic scaling on y-axis

- Horizontal grid lines provided by the plot

- Static set of vertical grid lines

- Grid lines and t/p plot are both children of a group with a shear transformation

Thanks! This looks great, at least for the "grid lines". The temperature curve still doesn't look like it is skewed properly. When the temperature curve is skewed correctly, it should be more vertical than horizontal. 

You can of course increase the shear as much as you like, since it is arbitrary. A value of -100 makes it much closer to vertical. Here's the reason: In the data points given, the range of x values is roughly 100, while the range of y values (after log10 scaling) is roughly 1. Thus to vertically align the first and last points requires a shear factor about 100:1.

  

Note that the values on the x-axis do not take into account the shear transformation; a little more effort would be required to make this correction. Also, the shear value that goes into the group transformation is arbitrary, since we would have to take into account the x/y scale factors in order to comput the correct factor for a 30 degree skew.

This is great to know! In previous versions I tried to emulate the coordinate transformations we applied to the Matplotlib code. However, with Qt/pyqtgraph I could not figure out what the units in the shear needed to be.

I guess ultimately the first thing I need to decide is to try and wrap this transformation into the underlying QT architecture (similar to what we've done with Matplotlib) or write a stand-alone mapping function that sits outside the plotting function and is just called manually on everything (a la the Tkinter example).

As far as I know, there is no simple way to get inside QGraphicsView to generate your own transformations. It natively supports affine + projection transformations, but not logarithmic. Pyqtgraph only supports log scaling by transforming the plot data before it is sent to Qt to be rendered, so the basic limitation is that if you want log (or any other non-affine) transformations, your options are to use pyqtgraph's plot curves (which handle the log transformation for you) or to transform the data manually before generating your graphics. I hope this is clear.. if not, I'm happy to try explaining it another way.

 

I've attached the script I sent previously with the shear factor increased and some faked adiabat-like curves added in. If you can confirm that this actually generates quantitatively-correct output, then I think this is a good solution. 

Luke