Picking a point in 3D
1,468 views
Skip to first unread message
Bi Ge
Jun 18, 2015, 5:35:18 PM6/18/15
to pyqt...@googlegroups.com
Hi folks,
I would like to pick a point in a 3D scatter plot. I read from Luke's post here:
"
This version also has methods for determining the projection and view matrices, which allows you to determine the relationship between 3D and 2D manually:
m = glView.projectionMatrix() * glView.viewMatrix()# Map 3D location to normalized device coordaintes: (-1,-1) and (1,1) are opposite corners of the viewpt = m.map(QtGui.QVector3D(x,y,z))# Inverse mapping:pt = m.inverted()[0].map(QtGui.QVector3D(x,y,z))
"
My questions are:
1. Does inverse mapping meaning from 2D to 3D? If so, why does the map take a 3D vector when the point on screen is only 2D:
2. After apply inverse mapping on a point on the screen, we get a point(vector) in 3D and from here I just need to cast a ray and find where it hits the point I want to find correct?
Bi
Mathew Schwartz
Jun 18, 2015, 8:57:37 PM6/18/15
to pyqt...@googlegroups.com
Im not exactly sure on your questions so I wont try to answer them, but if you are trying to pick a point, there is a method available for that already, itemsAt http://www.pyqtgraph.org/documentation/3dgraphics/glviewwidget.html
In that case, you dont need the 3d vector (its most likely a vector because it needs to point into the screen) just the 2d xy coordinates
--
You received this message because you are subscribed to the Google Groups "pyqtgraph" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyqtgraph+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/c2fc8530-f073-407f-9539-dc82f8fbb875%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Bi Ge
Jun 18, 2015, 9:02:18 PM6/18/15
to pyqt...@googlegroups.com
I've tried itemsAt but it only returns the GLScatterPlotItem object instead of a point.
Bi
Mathew Schwartz
Jun 20, 2015, 9:30:31 PM6/20/15
to pyqt...@googlegroups.com
Can you explain a bit more of what you want? If you dont want to select a point already in the plot, how do you pick a 3d point on a 2d screen, there is a third axis that you need to define how deep you want to select.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/ae710fb0-771f-464b-9c03-cf184e37382d%40googlegroups.com.
Bi Ge
Jun 20, 2015, 10:23:16 PM6/20/15
to pyqt...@googlegroups.com
I am trying to select a point already in the plot. It is true that I only have the relative x and y axes but from my minimal computer graphics experience I assume I can either a) construct a ray based on the view and camera position and do a poor-man's collision detection to find out which point is closest to that ray. b) convert all points in the scene to 2D based on view and camera position and select the point closed to the mouse.
Of course we will run into situations where two points in 3D overlay in 2D but we are always going to select the point closer to the camera.
Hopefully this cleared my question a bit.
Bi
Bi
Bi Ge
Jun 21, 2015, 1:12:45 AM6/21/15
to pyqt...@googlegroups.com
Found out why my selected point is jumping all around the screen.
When applying as Luke pointed out the range for x and y is [-1, 1] but unlike the mouse position returned by GLViewWidget, (-1, -1) is actually the right top corner.
pt = m.map(QtGui.QVector3D(x,y,z)) Thanks for all the help.
Bi
B. Oytun Peksel
May 12, 2016, 6:14:23 AM5/12/16
to pyqtgraph
BiGe, can you please post your solution? I am struggling with the same thing!
Sean Ge
May 12, 2016, 10:41:47 AM5/12/16
to pyqt...@googlegroups.com
"
`
# w = WorldView()
__WIDTH = 512
__HEIGHT = 424
m = w.projectionMatrix() * w.viewMatrix()
projected_array = np.zeros((__WIDTH * __HEIGHT, 2))
view_w = w.width()
view_h = w.height()
mouse_x = w.mousePos.x()
mouse_y = w.mousePos.y()
# b array contains the raw coordinates of all the points on screen
for i in xrange(0, __WIDTH, step):
for j in xrange(0, __HEIGHT, step):
pt = m.map(QtGui.QVector3D(b[j*__WIDTH+i, 0],
b[j*__WIDTH+i, 1],
b[j*__WIDTH+i, 2]))
# origin range [-1, 1]
projected_array[j*__WIDTH+i, 0] = (pt.x() + 1)/2
projected_array[j*__WIDTH+i, 1] = (- pt.y() + 1)/2
projected_array[:, 0] = (projected_array[:, 0] -
(mouse_x/view_w))
projected_array[:, 1] = (projected_array[:, 1] -
(mouse_y/view_h))
distance_array = np.power(np.power(projected_array[:, 0], 2) +
np.power(projected_array[:, 1], 2), 0.5)
min_index = np.nanargmin(distance_array)
# mark the selected point on screen with a big sphere
sp2.setData(pos=b[min_index, :], size=100)
`
"
Should be self-explanatory, we are basically projecting every point onto the screen to see which one is the closest to the mouse (slow; pretty sure OpenGL has built-in way of doing this) let me know if you have any questions.
Bi
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/53d1d9d3-6d9c-4b16-941a-358295c54995%40googlegroups.com.--
You received this message because you are subscribed to a topic in the Google Groups "pyqtgraph" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pyqtgraph/mZiiLO8hS70/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pyqtgraph+...@googlegroups.com.
B. Oytun Peksel
May 16, 2016, 3:51:35 AM5/16/16
to pyqtgraph
Thanks for the response! I have one question. I don't understand what b is. What do you mean by raw coordinates? Can you explain what you are doing inside the for loop?
Sean Ge
May 16, 2016, 5:25:06 AM5/16/16
to pyqt...@googlegroups.com
b contains the coordinates (positions) of the points you want to render, i.e. when you call:
w = WorldView()
sp1 = gl.GLScatterPlotItem(pos=b, size=1)
w.addItem(sp1)
Inside the for loop I just iterate through every point in b, calculate their corresponding positions on the screen (stored in projected_array) using the projection matrix and choose the "selected point" based on finding the smallest distance between projected point on the screen and the pixel clicked by the mouse, thus:
# find distance between mouse and points on screen
projected_array[:, 0] = (projected_array[:, 0] -
(mouse_x/view_w))
projected_array[:, 1] = (projected_array[:, 1] -
(mouse_y/view_h))
distance_array = np.power(np.power(projected_array[:, 0], 2) +
np.power(projected_array[:, 1], 2), 0.5)
min_index = np.nanargmin(distance_array)
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/bfd11fe1-f617-47d6-bcca-d9848ad04bf6%40googlegroups.com.
B. Oytun Peksel
May 16, 2016, 7:42:45 AM5/16/16
to pyqtgraph
Ok! I understand what you have done now. Apart from one thing.
w = WorldView()
Don't know what this is and could not find anything googling. When I get projectionMatrix() and ViewMatrix from pyqtgraph.opengl.GLViewWidget
I get the wrong result. It always gives me the same indice wherever I click. projected_array contains values greater than 1 which I assume should not happen.
So the key question is what is WorldView?
Sean Ge
May 16, 2016, 5:51:22 PM5/16/16
to pyqt...@googlegroups.com
Oops, WorldView is an extended class of opelgl.GLViewWidget, the definition is put below at the end of this thread.
" It always gives me the same indice wherever I click. " What exactly do you mean the same indice? The projection matrix and view matrix simply reflects the current position/view of your view point, it has nothing to do with where you click. Yes the projected_array shouldn't contain any value greater than 1.
Note I don't own any rights to the WorldView code:
class WorldView(gl.GLViewWidget):
def __init__(self, parent=None):
global ShareWidget
if ShareWidget is None:
# create a dummy widget to allow sharing objects
# (textures, shaders, etc) between views
ShareWidget = QtOpenGL.QGLWidget()
QtOpenGL.QGLWidget.__init__(self, parent, ShareWidget)
self.setFocusPolicy(QtCore.Qt.ClickFocus)
self.opts = {
# will always appear at the center of the widget
'center': Vector(0, 0, 0),
# distance of camera from center
'distance': 10.0,
# horizontal field of view in degrees
'fov': 60,
# camera's angle of elevation in degrees
'elevation': 30,
# camera's azimuthal angle in degrees
# (rotation around z-axis 0 points along x-axis)
'azimuth': 45,
# glViewport params; None == whole widget
'viewport': None,
}
self.setBackgroundColor('k')
self.items = []
self.noRepeatKeys = [QtCore.Qt.Key_Right, QtCore.Qt.Key_Left,
QtCore.Qt.Key_Up, QtCore.Qt.Key_Down,
QtCore.Qt.Key_PageUp, QtCore.Qt.Key_PageDown]
self.keysPressed = {}
self.keyTimer = QtCore.QTimer()
self.keyTimer.timeout.connect(self.evalKeyState)
self.makeCurrent()
self.ray = QtGui.QVector3D(0, 0, 0)
self.select = False
def mousePressEvent(self, event):
print ("Pressed button", event.button(), "at", event.pos())
self.mousePos = event.pos()
if event.button() == 2:
self.select = True
else:
self.select = False
print (self.itemsAt((self.mousePos.x(), self.mousePos.y(), 3, 3)))
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/ee0ce5da-a424-4b63-b02e-1444e8d56672%40googlegroups.com.
B. Oytun Peksel
May 19, 2016, 3:00:29 AM5/19/16
to pyqtgraph
First of all I would like to thank Bi Ge for his/her help. I could not make it work so I tried to find a different approach to the problem. I think it is important we have alternatives cause this will be a important function for pyqtgraph development.
So instead of projecting all points to the 2D viewport space I took the approach of ray casting and used this page as reference; http://antongerdelan.net/opengl/raycasting.html
My questions are embedded in the comments in the code as Q1..Q5 in the method call mPosition. Please scroll down to see it. it is the last method.
from PySide import QtCore, QtGuiimport pyqtgraph.opengl as glimport numpy
class PlotObject(gl.GLViewWidget): """ Override GLViewWidget with enhanced behavior
""" #: Fired in update() method to synchronize listeners. #sigUpdate = QtCore.Signal(float, float) App = None
def __init__(self, app=None):
if self.App is None: if app is not None: self.App = app else: self.App = QtGui.QApplication([]) super(PlotObject,self).__init__() self.Gridxy = gl.GLGridItem() self.Gridyz = gl.GLGridItem() self.Gridxz = gl.GLGridItem() self.Axes = gl.GLAxisItem() self.Gridxy.setSize(6000,2000,3) self.Gridxy.setSpacing(200,200,0) self.Gridxy.translate(3000, 0, 0)
self.Gridyz.setSize(1800,2000,3) self.Gridyz.setSpacing(200,200,0) self.Gridyz.translate(900, 0, 0) self.Gridyz.rotate(-90, 0, 1, 0)
self.Gridxz.setSize(6000,2000,3) self.Gridxz.setSpacing(200,200,0) self.Gridxz.translate(3000, -1000, 0) self.Gridxz.rotate(-90, 1, 0, 0) self.Poss = []
self.Plot = gl.GLScatterPlotItem()
self.addItem(self.Plot) self.addItem(self.Gridxy) self.addItem(self.Gridyz) self.addItem(self.Gridxz) self.addItem(self.Axes) self._downpos = []
#self.sigUpdate.connect(self.rayCast) self.setWindowTitle('Center of Gravity of Parts')
def mousePressEvent(self, ev): """ Store the position of the mouse press for later use.
""" super(PlotObject, self).mousePressEvent(ev) self._downpos = self.mousePos
def mouseReleaseEvent(self, ev): """ Allow for single click to move and right click for context menu.
Also emits a sigUpdate to refresh listeners. """ super(PlotObject, self).mouseReleaseEvent(ev) if self._downpos == ev.pos(): x = ev.pos().x() y = ev.pos().y() if ev.button() == 2 : self.mPosition() elif ev.button() == 1: x = x - self.width() / 2 y = y - self.height() / 2 #self.pan(-x, -y, 0, relative=True) print self.opts['center'] print x,y self._prev_zoom_pos = None self._prev_pan_pos = None
def mouseMoveEvent(self, ev): """ Allow Shift to Move and Ctrl to Pan.
""" shift = ev.modifiers() & QtCore.Qt.ShiftModifier ctrl = ev.modifiers() & QtCore.Qt.ControlModifier if shift: y = ev.pos().y() if not hasattr(self, '_prev_zoom_pos') or not self._prev_zoom_pos: self._prev_zoom_pos = y return dy = y - self._prev_zoom_pos def delta(): return -dy * 5 ev.delta = delta self._prev_zoom_pos = y self.wheelEvent(ev) elif ctrl: pos = ev.pos().x(), ev.pos().y() if not hasattr(self, '_prev_pan_pos') or not self._prev_pan_pos: self._prev_pan_pos = pos return dx = pos[0] - self._prev_pan_pos[0] dy = pos[1] - self._prev_pan_pos[1] self.pan(dx, dy, 0, relative=True) self._prev_pan_pos = pos else: super(PlotObject, self).mouseMoveEvent(ev) def plotGLPlot(self, objs): poss = numpy.array([0, 0, 0]) self.Poss = [] self.GlobalInds = [] weights = numpy.array(0, dtype=float) def pswc (x) : return 10 * x**0.25 #pseudoweight calculation with exponential scaling for obj in objs: for i,cogs in enumerate(obj.CoG): for cog in cogs: #cog[1] = 0 if obj.PieceWeight[i]: poss = numpy.vstack([poss,numpy.asarray(cog.T)]) self.Poss.append(numpy.matrix(cog.T)) # for picking stuff self.GlobalInds.append(obj.Index[i]) pw = pswc(obj.PieceWeight[i]) weights = numpy.append(weights, pw)
maxw = max(weights) threshold = numpy.mean(weights) self.Colors = numpy.empty([len(weights),4]) for i, pw in enumerate(weights): if pw <= threshold: c = pw / maxw self.Colors[i] = numpy.array([c,1,0,0.7]) else: c = 1 - pw / maxw self.Colors[i] = numpy.array([1,c,0,0.7])
self.removeItem(self.Plot) self.Plot = gl.GLScatterPlotItem() self.Plot.setData(pos=poss, size=weights, color=self.Colors, pxMode=False) self.Sizes = weights self.addItem(self.Plot) self.show()
def mPosition(self): #This function is called by a mouse event ## Get mouse coordinates saved when the mouse is clicked( incase dragging) mx = self._downpos.x() my = self._downpos.y() self.Candidates = [] #Initiate a list for storing indices of picked points #Get height and width of 2D Viewport space view_w = self.width() view_h = self.height() #Convert pixel values to normalized coordinates x = 2.0 * mx / view_w - 1.0 y = 1.0 - (2.0 * my / view_h) # Convert projection and view matrix to numpy types and inverse them PM = numpy.matrix([self.projectionMatrix().data()[0:4], self.projectionMatrix().data()[4:8], self.projectionMatrix().data()[8:12], self.projectionMatrix().data()[12:16]]) PMi = numpy.linalg.inv(PM) VM = numpy.matrix([self.viewMatrix().data()[0:4], self.viewMatrix().data()[4:8], self.viewMatrix().data()[8:12], self.viewMatrix().data()[12:16]]) VMi = numpy.linalg.inv(VM) #Move to clip coordinates by chosing z= -1 and w 1 (Dont understand this part) # Q1: Why are we picking arbitrary -1 for z? ray_clip = numpy.matrix([x, y, -1.0, 1.0]).T # get transpose for matrix multiplication # Q2 = Clip space should clip some of the scene depending on the zoom. How is it done? Is it implicit # in the transformation matrices? # Convert to eye space by view matrix ray_eye = PMi * ray_clip ray_eye[2] = -1 ray_eye[3] = 0 #Convert to world coordinates ray_world = VMi * ray_eye ray_world = ray_world[0:3].T # get transpose for matrix multiplication ray_world = ray_world / numpy.linalg.norm(ray_world) # normalize to get the ray # Q3: Since we normalize this vector, does it mean the values are a b c values of a ray definition in # linear algebra such as z = ax+by+c # Now I 'll use the ray intersection with spheres. I assume every point is a sphere with a radius #Please see http://antongerdelan.net/opengl/raycasting.html scroll down to spehere intersection O = numpy.matrix(self.cameraPosition()) # camera position should be starting point of the ray # Q4: Is this approach correct? Is starting point really the camera coordinates obtained like this? print O, ray_world for i, C in enumerate(self.Poss): # Iterate over all points OC = O - C b = numpy.inner(ray_world, OC) b = b.item(0) c = numpy.inner(OC, OC) #Q5: When the plot function is called with pxMode = False the sizes should reflect the size of point #dots in world coordinates. So I assumed they were the diameter of the spheres. Is this correct? Otherwise how do I reach the #diameter of spheres in terms of world coordinates? c = c.item(0) - numpy.square((self.Sizes[i] / 2 )) bsqr = numpy.square(b) if (bsqr - c) >= 0: # means intersection self.Candidates.append(self.GlobalInds[i])
print self.Candidates Message has been deleted
B. Oytun Peksel
May 19, 2016, 6:05:14 AM5/19/16
to pyqtgraph
Well I realize now even posting a question makes your understanding boost.I solved the problem couple of hours later. It works very good so far
def mPosition(self): #This function is called by a mouse event ## Get mouse coordinates saved when the mouse is clicked( incase dragging) mx = self._downpos.x() my = self._downpos.y() self.Candidates = [] #Initiate a list for storing indices of picked points #Get height and width of 2D Viewport space view_w = self.width() view_h = self.height() #Convert pixel values to normalized coordinates x = 2.0 * mx / view_w - 1.0 y = 1.0 - (2.0 * my / view_h) # Convert projection and view matrix to numpy types and inverse them PMi = self.projectionMatrix().inverted()[0] # PMi = numpy.matrix([PMi[0:4], # PMi[4:8], # PMi[8:12], # PMi[12:16]]) VMi = self.viewMatrix().inverted()[0] # VMi = numpy.matrix([VMi[0:4], # VMi[4:8], # VMi[8:12], # VMi[12:16]]) #Move to clip coordinates by chosing z= -1 and w 1 (Dont understand this part) # Q1: Why are we picking arbitrary -1 for z? ray_clip = QtGui.QVector4D(x, y, -1.0, 1.0) # get transpose for matrix multiplication # Q2 = Clip space should clip some of the scene depending on the zoom. How is it done? Is it implicit # in the transformation matrices? # Convert to eye space by view matrix ray_eye = PMi * ray_clip ray_eye.setZ(-1) ray_eye.setW(0) #Convert to world coordinates ray_world = VMi * ray_eye ray_world = QtGui.QVector3D(ray_world.x(), ray_world.y(), ray_world.z()) # get transpose for matrix multiplication ray_world.normalize() #ray_world = ray_world / numpy.linalg.norm(ray_world) # normalize to get the ray # Q3: Since we normalize this vector, does it mean the values are a b c values of a ray definition in # linear algebra such as z = ax+by+c # Now I 'll use the ray intersection with spheres. I assume every point is a sphere with a radius #Please see http://antongerdelan.net/opengl/raycasting.html scroll down to spehere intersection O = numpy.matrix(self.cameraPosition()) # camera position should be starting point of the ray ray_world = numpy.matrix([ray_world.x(), ray_world.y(), ray_world.z()]) # Q4: Is this approach correct? Is starting point really the camera coordinates obtained like this? print O, ray_world for i, C in enumerate(self.Poss): # Iterate over all points OC = O - C b = numpy.inner(ray_world, OC) b = b.item(0) c = numpy.inner(OC, OC) #Q5: When the plot function is called with pxMode = False the sizes should reflect the size of point #dots in world coordinates. So I assumed they were the diameter of the spheres. Is this correct? Otherwise how do I reach the #diameter of spheres in terms of world coordinates? c = c.item(0) - (self.Sizes[i]/2)**2 #numpy.square((self.Sizes[i])) bsqr = numpy.square(b) if (bsqr - c) >= 0: # means intersection self.Candidates.append(self.GlobalInds[i])
print self.Candidates
musicallyeternal
Feb 9, 2018, 8:08:17 AM2/9/18
to pyqtgraph
Thank you so much for posting this. i was struggling with this for a while and your solution really helped me! I hope this gets implemented into some version of pyqtgraph - it works quite well!
m.e.
m.e.
Landry Kotto
Jul 27, 2020, 9:05:17 AM7/27/20
to pyqtgraph
what is objs in the method that follow? I have tried to set GLScatterPlotItem, GLviewWiget, but i gotan error : objet is not iterable , and i don't know why !
What value objs can take ?
Thanks
the method is : def plotGLPlot(self, objs): # new commentaire : add item -> plot 3D point
poss = numpy.array([0, 0, 0])
self.Poss = []
self.GlobalInds = []
weights = numpy.array(0, dtype=float)
def pswc (x) : return 10 * x**0.25 #pseudoweight calculation with exponential scaling
for obj in objs:
for i,cogs in enumerate(obj.CoG):
for cog in cogs:
#cog[1] = 0
if obj.PieceWeight[i]:
poss = numpy.vstack([poss,numpy.asarray(cog.T)])
self.Poss.append(numpy.matrix(cog.T)) # for picking stuff
self.GlobalInds.append(obj.Index[i])
pw = pswc(obj.PieceWeight[i])
print("PieceWeight : "+str(pw))
weights = numpy.append(weights, pw)
maxw = max(weights)
threshold = numpy.mean(weights)
self.Colors = numpy.empty([len(weights),4])
for i, pw in enumerate(weights):
if pw <= threshold:
c = pw / maxw
self.Colors[i] = numpy.array([c,1,0,0.7])
else:
c = 1 - pw / maxw
self.Colors[i] = numpy.array([1,c,0,0.7])
self.removeItem(self.Plot)
#....self.Plot = gl.GLScatterPlotItem()
#.....self.Plot.setData(pos=poss, size=weights, color=self.Colors, pxMode=False)
self.Sizes = weights
#....self.addItem(self.Plot)
self.show()
#.....self.Plot.setData(pos=poss, size=weights, color=self.Colors, pxMode=False)
self.Sizes = weights
#....self.addItem(self.Plot)
self.show()
Landry Kotto
Aug 2, 2020, 8:29:01 AM8/2/20
to pyqtgraph
Hello,
I didn't have some new ?
Thanks
Reply all
Reply to author
Forward
0 new messages