Playing with Qt Scene Graph

I’ve planned to get familiar with Qt Scene Graph for a several months now, but Gunnar Sletta’s presentation at the Qt Developer Day 2010 in Munich kicked me to actually do something. The goal of my Qt Scene Graph research project can be listed as follows:

  • How to build and use Qt Scene Graph
  • Get the whole picture of scene graph architecture
  • And check out the code i.e how it works
  • Try to create something of your own by using Qt Scene Graph

Before going any further, I go directly to the last bullet point and show a video about small QML demo which uses Qt Scene Graph. I borrowed the particle parts from Qt Scene Graph’s testing code and created few vertex and fragment shaders for rotation, scaling and manipulating colors.

Building Qt Scene Graph

First thing that you need do in order to play with Qt Scene Graph is to build it. The scene-graph source tree contains good instructions how to build it but I encountered some problems when tried to build it. The first problem was that for some reason in OS X, the command “qmake -r” didn’t work in Qt Scene Graph source tree. So here are the steps that I did:

  • Clone Qt master (git clone git://gitorious.org/qt/qt.git)
  • Build Qt with -developer-build configure option and DON’T run “make install” for it. Qt Scene Graph requires private headers and if you do “make install” the private headers are not copied to the installation directory.
  • Make sure that your “qmake” is in your PATH or just execute it directly from you Qt source tree’s “bin” directory.
  • set QTDIR to point your Qt Master (export QTDIR=/path/to/your/Qt/Master)
  • Clone Qt Scene Graph (git clone git://gitorious.org/qt-labs/scene-graph.git)
  • Go the the scnene-graph dir and run “qmake -r”. If it doesn’t work, then run:
    • qmake launcher.pro library.pro qmltime/qmltime.pro qt-scene-graph.pro
  • run make
  • after this you will have scene-graph/bin/qmlscene binary and you can run qml app e.g. by running “qmlscene helloworld.qml”

Qt Scene Graph Architecture

I try not to go too deep into details, but I will go through some parts of the scene graph because:

  1. I want to understand how it works
  2. I’m not very familiar with OpenGL (ES 2) and this is a good opportunity to start learning it
  3. I want to see where Qt will go in the future

So here is a high level picture of how scene graph changes the current architecture of Qt. This is mostly copied from Gunnar’s presentation at the Qt Developer Days 2010.

Architecture impact in high level

One of the problems with the current architecture in Qt is that when it comes to painting, it is not as optimal as it could be. One reason for this is the legacy, the OpenGL support has been added there afterwards and QPainter doesn’t work  in a way that it would gain the most out of the performance. Gunnar gave pretty good example in his presentation. Let’s think of the the picture above. If QPainter should draw that image, here is what it does:

  • Drawing works in bottom-up manner. First it draws the white background
  • Then it draws the yellow (opaque) rectangles with fills
  • Then it draws the blue (opaque) rectangle with fills
  • and then it draws the text

The problem is that QPainter does extra work for example the yellow rectangles are mostly covered by blue opaque rectangles and now what QPainter does, is that it draws and fills the areas that will end up to be “under” the other opaque objects. You can easily figure 0ut that this is not as optimal as it could be. The following part is borrowed from Gunnar Sletta’s Qt Labs post:

“To get optimal performance from hardware rendering, we must minimize the number of state changes; that is, don’t change textures, matrices, colours, render targets, shaders, vertex and index buffers etc. unless you really need to. Big chunks of data should also be uploaded to video memory once and stay there for as long as the data are needed, rather
than being sent from system memory each frame. To achieve this, we need to know as much as possible about what is going to be rendered.”

How Qt Scene Graph works then?

At this point I suggest you to read Gunnar Sletta’s post from Qt Labs blog, if you haven’t done it yet. In his post he explains the basics, but in order to get better understanding how it really works, I needed to take a look at the source code. Here’s what happens under the hood.

UML class diagram of Qt Scene Graph

Note that the UML class diagram is bit rough and doesn’t tell you all the details. You can imagine that drawing a perfect class diagram is a huge job so the idea is to describe basic case. Btw, I didn’t include the TransformationNode there.

DISCLAIMER: The following description should give a high level description how classes interact and it should give you some kind of understanding how scene graph works. There are probably errors and I may have misunderstood something.  Please feel to comment, if you notice any errors or you disagree what I have written in this article.

QxGraphicsView is a new entry point if you want to use scene graph in your QML application. Basically you use it as you would use QDeclarativeView and pass an url of QML file to it via QxGraphicsView::setSource(const QUrl &). This class calls qt_scenegraph_register_type() function from qml_scene_global.cpp, which actually registers all the “custom” Qx-classes to QML engine. This needs to be done in order to provide scene graph specific implementations of QML elements to QML system. QxGraphicsView inherits QGLWidget and calls QmlRenderer when ever QML canvas needs to be drawn.

QxItem is a base class for all scene graph based QML items. It provides focus, key and input method event handling and implements callback methods inherited from QDeclarativeParserStatus. According docs QDeclarativeParserStatus is usually used to avoid performing any expensive operations until after all the properties have been set on an object.

QxGraphicsObject provides properties for all items such as position (x, y, z), width, height opacity, scale, rotation and clip. QxGraphicsObject also inherits QObject in order to have Qt MetaObject system and enabling the signal-slot mechanism. Event handling for mouse events happens here, but these are basically empty implementations atm. QxGraphicsObject also stores the paint node of each object and makes sure that it’s in the right position in a tree. Node is something that QmlRenderer will draw in the end. QxGraphicsObject is also responsible of handing geometry changes and setting clipping.

QxRectangle is here as an example of scene graph based Rectangle item. All the items in scene graph based QML system must inherit QxItem.  It also has a Node member, which is an instance of DefaultRectangleNode. DefaultRectangleNode implements the RectangleNodeInterface and the instance is created by createRectangle() function implemented in qt_adaptation_layer.cpp. Basically the DefaultRectangleNode is a GeometryNode which sets e.g. the materials such as pen color. At least this is how I understood it to work. The RectangleNodeInterface is responsible of defining the interfaces for settings rectangle size. If you change the size of the rectangle it calls RectangleNodeInterface::setRect(QRectF) which then is implemented by DefaultRectangleNode.

This is pretty much how the class infrastructure works if you want to construct a scene graph based QML item.  But how all this will be rendered is then a different matter. I try not to go too deep into details, but here it is how I see what happens under the hood.

Node provides a interfaces to organize nodes in a tree. It also defines what the type of the Node object is. There are BasicNodeType, RootNodeType, GeometryNodeType, TransformNodeType and UserNodeType. There are also subnode types, but I skip them here. When QmlRenderer traverses through the nodes, it checks e.g. if the type of the Node is the GeometryNodeType and decides the render order based on that. The GeometryNode provides the visual appearance for items and uses shaders for rendering.

So in the high level, the QxGraphicsView (QGLWidget) contains an instance of QmlRenderer. When QmlRenderer notifies the view about changes in the scene graph, it will update the view. In the QxGraphicsView::paintEvent(..) method, the view makes QmlRenderer to render scene. When rendering the scene, it initializes gl functions which calls QGLFunctions::initializeGLFunctions(), it makes sure that all the functions operates on current QGLContext and switches rendering to specified QGLFrameBufferObject. Then it calls the QmlRenderer::render() method which does all the magic.

And as I stated earlier, I’m pretty sure that I have missed something that might be important or I may have also misunderstood something. Comments are welcome :)

What you can do with the scene graph?

I guess it’s always good to ask questions and try to find answers for them. I think the reason number one for Qt Scene Graph project is the performance. So basically you can use Qt Scene Graph based QML as you would use the official QML. The difference is that it works closer to HW.

The other reason is that if you have worked with Qt Graphics View you might have noticed that it’s not as optimal as it could be when it comes to performance. The official version of QML is built  on top of Qt Graphics View and for example the effects for QML objects (drop shadow, blur) are not working so well. With Qt Scene Graph you can quite easily create your own effects and at some point I think Qt Components or plain QML will provide a set of QML effect items taking an advantage of shaders.

As an example there is ShaderEffectItem which provides nice QML interface for you to create a custom effect. You can define vertex and fragment shaders that are applied to other QML item e.g. to Image. Here’s an example:

ShaderEffectItem {
    property real scaleFactor: 1
    vertexShader:   "uniform highp mat4 qt_Matrix;              \n" +
                    "attribute highp vec2 qt_MultiTexCoord0;    \n" +
                    "attribute highp vec4 qt_Vertex;            \n" +
                    "varying highp vec2 qt_TexCoord;            \n" +
                    "uniform highp float scaleFactor;           \n" +
                    "void main() {                              \n" +
                    "   vec4 a = qt_Vertex;                     \n" +
                    "   a.x = a.x * scaleFactor;                \n" +
                    "   a.y = a.y * scaleFactor;                \n" +
                    "   gl_Position =  qt_Matrix * a;           \n" +
                    "   qt_TexCoord = qt_MultiTexCoord0;        \n" +
                    "}\n" ;

    fragmentShader: "varying highp vec2 qt_TexCoord;                    \n" +
                    "uniform lowp sampler2D source;                     \n" +
                    "void main(void){                                   \n" +
                    "    gl_FragColor = texture2D(source, qt_TexCoord);       \n" +
                    "}\n";

    active: true
    blending: true
    smooth: true
    source: myItem
    anchors.fill: myItem
}
// Image item
Image{
    id: myItem
    source: "foo.png"
    ..
}

To connect your item to the ShaderEffectItem you need pass the source item id “myItem” to the ShaderEffectItem’s source property and make sure that ShaderEffectItem anchor.fill uses your item. The neat thing here is that you can define a property like the “scaleFactor” in the example above. Then you can change the value of the property e.g. with NumberAnimation and it actually effects on your vertex or fragment shader as shown in the example.

I also noticed that if you have more than one effect, you can’t “connect” them directly to the same object. What you need to do is that you define the source property of the first ShaderEffectItem to refer to your item. Then the second ShaderEffectItem source takes a reference to the first ShaderEffectItem. Otherwise you just can’t connect more than one effect to your item.

DISCLAIMER: I have  never done OpenGL programming so these shaders in the example are results of the “late night studies” after my son and wife went to sleep. So please comment, if you feel that they are completely wrong.

Even though, I don’t have any experience of implementing  or designing any applications on top of OpenGL (ES 2) directly, I think that what Gunnar and other trolls have created here, looks really promising. Nothing forces you to create vertex or fragment shaders in order to be able to use Qt Scene Graph. It is just an extra stuff that you will get for free with the package. I believe that, when Qt Scene Graph will be released, it doesn’t take a long time before Qt starts to get really cool looking effects.

The Future of Qt Scene Graph?

Qt Scene Graph is not a small deal when it comes to the QML architecture. For example it will remove the current way of extending QML on top of Qt Graphics View. I guess, all the classes with Qx prefix will be replaced at some point with something else. Therefore, if you have extended QML with your own objects you might end up to have some find-and-replace workout at some point. Still, I think that it’s worth to do that exercise to gain all the performance improvements.

According to Gunnar Sletta’s presentation at the Qt Developer Days in Munich, Qt Scene Graph is still a Qt labs project. You never know when they decide to integrate it to the Qt master, but my guess is that it won’t happen before Qt  5.0. This is just a guess and it’s hard to predict what happens in Nokia.

Thanks for reading my blog!

Tags: , , , , , ,

7 Responses to “Playing with Qt Scene Graph”

  1. andy.fillebrown says:

    Hi,

    I’m having trouble building scene-graph with mingw.

    qxclipnode.cpp can’t find qglfunctions.h or QtGui/qglyphs.h

    … and I can’t find them either. Is mingw not supported? Do I have the wrong version of Qt? I built from git source configured with -developer-build but I do not find either of those headers anywhere.

    Cheers,
    ~ andy.f

  2. catherine says:

    That looks intriguing.
    Would you mind sharing the source code for the demo shown in the video?

  3. sq says:

    QGraphicsView is designed to work perfectly well with huge scenes, i.e. millions of items.
    Is this also true for QtQuick 1 (since it’s build on top of QGraphicsView)?

    And how will that be with QtQuick 2?
    In other words, will the new Scene Graph e.g. allow for techniques like spatial partitioning?

    • zchydem says:

      @sq QGraphicsView was originally designed to handle at least thousands of items (not sure about millions), but because of Qt legacy, QPainter based drawing didn’t handle painting as optimized as it should have done that. With QtQuick 2.0 and Qt 5 the new scene graph handles drawing more optimized. I.e. it’s much faster as QtQuick 1.0.

      I haven’t got time to follow what’s happening in Qt5 in depth lately, but if you are interested in about details, you should check the source code and the documentation: http://doc.qt.nokia.com/5.0-snapshot/qtquick.html

      All the classes starting with “QSG” are scene graph classes.

Leave a Reply

You must be logged in to post a comment.