Important: Please read the Qt Code of Conduct  https://forum.qt.io/topic/113070/qtcodeofconduct
Wrong values with openGL perspective divide and viewport()

Hi,
currently I need a function with Vector3D vertices as input and the corresponding viewport xy coordinates as outputs. This means I have to calculated Model matrix
 View Matrix (QMatrix4x4::lookAt())
 Perspective matrix (QMatrix4x4::perspective())
 Perspective divide (divide through w, QVector3D::toVector3DAffine())
 Viewport transformation (QMatrix4x4::viewport())
Following short illustrative example. The camera is translated (1 to positive z) and also rotated = 90 deg (case with wrong values). The camera shows to the origin (0,0,0).
Camera: (0, 0, 1)
CameraUp: (0, 1, 0)
CameraDir: (0, 0, 0)Vertex(9.9, 9.9, 9); // 9.9 instead of 10 to see it in the openGL viewport
The AoV vert/hori are both 90deg with aspRat=1.
Without camera rotation the vertex is correctly in the bottom/right position (distance from camera 10, 10, 10). So far everthing ok, also verified in an parallel openGL window.
The problems occur if the rotation angle is <> 0 and/or the near plane is < 1. Then the w values from xyzw are wrong (> 1).QMatrix4x4 mdlM; mdlM.setToIdentity(); QMatrix4x4 viewM; viewM.setToIdentity(); QMatrix4x4 camT; camT.setToIdentity(); camT.rotate(90, 1, 0, 0); // camera rotation QVector3D camXYZ = QVector3D(0, 0, 1); QVector3D camUD = camT * QVector3D(0, 1, 0); QVector3D camLA = camT * QVector3D(0, 0, 1) + camXYZ; viewM.lookAt(camXYZ, camLA, camUD); float aovV = 90; QMatrix4x4 persM; persM.perspective(aovV, 1.0, 1, 10000.0); QMatrix4x4 mvpM = persM * viewM * mdlM; QVector4D vertex = QVector4D(9.9, 9.9, 9, 1); QVector4D xyzw = vertex * mvpM; //wrong w values QVector3D xyzwNTC = xyzw.toVector3DAffine(); QMatrix4x4 viewpM; viewpM.viewport(0, 0, 400, 400, 1, 10000.0); QVector4D xy = viewpM * xyzwNTC;
Correct results (rotation angle = 0):
camXYZ QVector3D(0, 0, 1) camUD QVector3D(0, 1, 0) camLA QVector3D(0, 0, 0) aovV 90 vertex QVector4D(9.9, 9.9, 9, 1) xyzw QVector4D(9.9, 9.9, 8.0018, 10) xyzwNDC QVector3D(0.99, 0.99, 0.80018) xy QVector4D(398, 398, 9001, 0)
Wrong results (rotation angle = 90). In the openGL viewport the point correctly moves from bottom/right to top/right. But in the xyzwNDC vector x value is greater 1 and thus outside the viewing volume. y is also wrong.
camXYZ QVector3D(0, 0, 1) camUD QVector3D(0, 0, 1) camLA QVector3D(0, 1, 1) aovV 90 vertex QVector4D(9.9, 9.9, 9, 1) xyzw QVector4D(9.9, 8.0018, 9.9, 8.1018) xyzwNDC QVector3D(1.22195, 0.987657, 1.22195) xy QVector4D(444.39, 2.4686, 11109.6, 0)
So unfortunately I don't know what I'm doing wrong. Has anyone an idea?

The problem was QVector4D xyzw = vertex * mvpM;
Should be QVector4D xyzw = mvpM * vertex;