QQuick + OpenSceneGraph flickering - qtquick2

I am using QQuickControls 2 (SwipeView) with OpenSceneGraph for the 3d rendering. Using QQuickFramebufferObject for the integration.
Since I have introduced a SwipeView, I observe some flickering of my GUI.
I have looked for ages in the doc (it probably sums literally up to weeks) and have absolutely no idea why I have this flickering.
Here is a video of the faulty behavior (flickering starts at ~20s).
And here is my rendering code:
class OsgRenderer : public QQuickFramebufferObject::Renderer
{
public:
explicit OsgRenderer();
QOpenGLFramebufferObject *createFramebufferObject(const QSize &size) LC_OVERRIDE;
void synchronize(QQuickFramebufferObject* item) LC_OVERRIDE;
void render() LC_OVERRIDE;
protected:
friend class OsgWindow;
OsgItem* m_osgItem;
};
void OsgRenderer::render()
{
assert(m_osgItem);
if ( !m_osgItem->getViewer() )
return;
// Without this line the model is not displayed in the second
// and subsequent frames.
QOpenGLContext::currentContext()->functions()->glUseProgram(0);
// Ask OSG to render.
m_osgItem->getViewer()->frame(); // WARNING: non-blocking (executed in a thread of its own - in a thread-safe way).
// Reset OpenGl state for QtQuick.
m_osgItem->window()->resetOpenGLState();
}
Any idea of where that can come from?

In general it's not a good idea to have QtQuick and OSG both render to the same OpenGL context.
OSG it's keeping its GL state internally between frames, but Qt might modify it from "outside" without notifying osg, and this might cause rendering issues.
A more solid approach is have them use separate (and shared) GL context, and copy the context of the fbo osg redners to in a texture used by Qt.
I've successfully implemented this approach here: https://github.com/rickyviking/qmlosg
but haven't tested it with recent versions of QtQuick.
A more up-to-date integration can be found here: https://github.com/podsvirov/osgqtquick

Related

How to detect if InputSystem.EnhancedTouch.Finger is over UI Toolkit Element?

I'm using InputSystem.EnhancedTouch for my game and I have a few buttons that are available through UI Elements. When I click on the UI Elements buttons it activates the InputSystem thus creating a bad experience. I want to filter out all the InputSystem.EnhancedTouch events that come through UI.
TL;DR - I want UI Elements to block InputSystem.EnhancedTouch events from triggering
I have found quite some resources but nothing really works. Unity and some other people say to use EventSystem.current.IsPointerOverGameObject but it doesn't work and throws the following warning(I think this is meant to work with normal input only, not enhanced one)
I have tried a few other solutions such as UIDocument.rootVisualElement.panel.Pick or EventSystem.current.RaycastAll but nothing seems to work, or return any consistent data that can be used.
InputSystem.EnhancedTouch binding
private void Awake()
{
EnhancedTouchSupport.Enable();
Touch.onFingerDown += OnFingerDown;
Touch.onFingerMove += OnFingerMove;
Touch.onFingerUp += OnFingerUp;
}

Unity A* graph won't scale with canvas

I am using the A* graph package that I found here.
https://arongranberg.com/astar/download
it all works well in scene view and I was able to set the graph to treat walls like obstacles.
However once I start the game the canvas scales and the graph's nodes no longer align with the walls.
this is really messing up my path finding. If anyone has any ideas how to fix this that would be much appreciated. I tried parenting the graph to the canvas but it still doesn't scale.
Kind regards
For anyone struggling with this what we did is edited the script of the A* code, in the update function we just got it to rescan once. This means that once the game started and all the scaling had taken place the graph re adjusted its bounds. This is probably not the most proper way but it only took four lines and worked for us.
private bool scanAgain = true;
private void Update () {
// This class uses the [ExecuteInEditMode] attribute
// So Update is called even when not playing
// Don't do anything when not in play mode
if (!Application.isPlaying) return;
navmeshUpdates.Update();
// Execute blocking actions such as graph updates
// when not scanning
if (!isScanning) {
PerformBlockingActions();
}
// Calculates paths when not using multithreading
pathProcessor.TickNonMultithreaded();
// Return calculated paths
pathReturnQueue.ReturnPaths(true);
if (scanAgain)
{
Scan();
scanAgain = false;
}

MRTK V2 - Enable/Disable Spatial Mapping at Runtime

I know that this question has already been asked here twice, but the answers did not fix my problem. I need to enable spatial mapping on runtime. After scanning my environment I want to disable it, or hide at least the visualization of polygons, so I can save some fps. But by disabling spatial mapping I still want to have the colliders of my environment.
What I tried:
1. This example from this post did nothing.
if (disable){
// disable
MixedRealityToolkit.SpatialAwarenessSystem.Disable();
}
else
{
// enable
MixedRealityToolkit.SpatialAwarenessSystem.Enable()
}
2. Trying to disable the visualization gives me every time a nullreference. I guess GetObservers is giving null back or maybe meshOserver is null:
foreach(var observer in MixedRealityToolkit.SpatialAwarenessSystem.GetObservers())
{
var meshObserver = observer as IMixedRealitySpatialAwarenessMeshObserver;
if (meshObserver != null)
{
meshObserver.DisplayOption = SpatialAwarenessMeshDisplayOptions.None;
}
}
3. The example given by mrtk in there SpatialAwarenessMeshDemo scene, shows how to start and stop the observer. By starting everything starts fine but after suspending and clearing the observers the whole spatial map disappears, so my cursor does not align to my environment. So this is not what I need.
SpatialAwarenessSystem.ResumeObservers(); //start
SpatialAwarenessSystem.SuspendObservers();//stop
SpatialAwarenessSystem.ClearObservations();
What I have right now:
My Spatial Awareness Profile looks like this:
My code starts the spatial mapping with ResumeObservers, the foreach-loop gives me a nullreference and SuspendObserver is comment out, because it disables the whole spatial map thing:
if (_isObserverRunning)
{
foreach (var observer in SpatialAwarenessSystem.GetObservers())
{
var meshObserver = observer as IMixedRealitySpatialAwarenessMeshObserver;
if (meshObserver != null)
{
meshObserver.DisplayOption = SpatialAwarenessMeshDisplayOptions.None;
}
}
//SpatialAwarenessSystem.SuspendObservers();
//SpatialAwarenessSystem.ClearObservations();
_isObserverRunning = false;
}
else
{
SpatialAwarenessSystem.ResumeObservers();
_isObserverRunning = true;
}
Question: How do I start and stop spatial mapping the right way, so that I can save some performance and still have the colliders of the spatial map to interact with.
My specs:
MRTK v2.0.0
Unity 2019.2.0f1
Visual Studio 2017
!--Edit--inlcuding-Solution--!
1. With option #1 I was wrong. It does what its meant for, but I used it the wrong way. If you disable for example SpatialAwarenessSystem while running the spatial mapping process, it disables the whole process including the created spatial map. So after that you cant interact with the invironment.
2. What worked for me was using for the start ResumeObservers() in combination with setting display option to visible and for stopping spatial mapping the method SuspendObservers() in combination with display option none.
3. The Nullreference if fixed by rewritting and casting to IMixedRealityDataProviderAccess:
if (CoreServices.SpatialAwarenessSystem is IMixedRealityDataProviderAccess provider)
{
foreach (var observer in provider.GetDataProviders())
{
if (observer is IMixedRealitySpatialAwarenessMeshObserver meshObs)
{
meshObs.DisplayOption = option;
}
}
}
4. Performance: To get your fps back after starting an observer, you really need to disable the system via MixedRealityToolkit.SpatialAwarenessSystem.Disable();, but this will of course disable also the spatial map, so you cant interactive with it anymore.
#Perazim,
The recommendation is based on your option #3. Call ResumeObservers() to start and SuspendObservers() to stop. There is no need to call ClearObservations() unless you wish to have them removed from your scene.
The example calls ClearObservations() to illustrate what was, at the time, a new feature added to the Spatial Awareness system.
Please file an issue on GitHub (https://github.com/microsoft/MixedRealityToolkit-Unity/issues) for #1 (failure of Enable() and Disable() to impact the system). Those methods should behave as advertised.
Thank you!
David

QQuickFramebufferObject: where is the custom FBO used ? How can this cause flickering?

I am currently integrating a custom OpenGL viewport within QML via QQuickFramebufferObject. I observe some flickering of the whole app window (not only of the QQuickFramebufferObject window) and I am wondering why.
My custom QQuickFramebufferObject::Renderer does create a separate FPO:
QOpenGLFramebufferObject* OsgRenderer::createFramebufferObject(const QSize &size)
{
QOpenGLFramebufferObjectFormat format;
format.setAttachment(QOpenGLFramebufferObject::CombinedDepthStencil);
return new QOpenGLFramebufferObject(size, format);
}
So I understand that Qt defines its own FBO type (QOpenGLFramebufferObject) so it can later use the FBO to mix it with its own GL rendering, is that right?
I then do all my rendering
void OsgRenderer::render()
{
assert(m_osgItem);
if ( !m_osgItem->getViewer() )
return;
// Without this line the model is not displayed in the second
// and subsequent frames.
QOpenGLContext::currentContext()->functions()->glUseProgram(0);
// Ask OSG to render.
m_osgItem->getViewer()->frame(); // WARNING: non-blocking (executed in a thread of its own - in a thread-safe way).
// Reset OpenGl state for QtQuick.
m_osgItem->window()->resetOpenGLState();
}
I expect QtQuick to actually gather my custom rendered FBO with its own rendering of the window after the call to render(), is that right?
My rendering is actually achieved in a separate context. In this context, how can it cause flickering? Any idea?

Gstreamer 1.0 Pause signal

I need to detect when the current playing audio/video is paused. I cannot find anything for 1.0. My app is a bit complex but here is condensed code
/* This function is called when the pipeline changes states. We use it to
* keep track of the current state. */
static void state_changed_cb(GstBus *bus, GstMessage *msg, CustomData *data)
{
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed(msg, &old_state, &new_state, &pending_state);
if(GST_MESSAGE_SRC(msg) == GST_OBJECT(data->playbin))
{
g_print("State set to %s\n", gst_element_state_get_name(new_state));
}
}
gst_init(&wxTheApp->argc, &argv);
m_playbin = gst_element_factory_make("playbin", "playbin");
if(!m_playbin)
{
g_printerr("Not all elements could be created.\n");
exit(1);
}
CustomData* data = new CustomData(xid, m_playbin);
GstBus *bus = gst_element_get_bus(m_playbin);
gst_bus_set_sync_handler(bus, (GstBusSyncHandler) create_window, data, NULL);//here I do video overly stuffs
g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, &data);
What do I do wrong? I cannot find working example on connecting such events on Gstreamer 1.0 and 0.x seems a bit different than 1.0 so the vast exaples there don't help
UPDATE
I have found a way to get signals. I run wxWidgets timer with 500ms time span and each time timer fires I call
GstMessage* msg = gst_bus_pop(m_bus);
if(msg!=NULL)
{
g_print ("New Message -- %s\n", gst_message_type_get_name(msg->type));
}
Now I get a lot of 'state-change' messages. Still I want to know if that message is for Pause or Stop or Play or End of Media (I mean way to differentiate which message is this) so that I can notify the UI.
So while I get signals now, the basic problem, to get specific signals, remains unsolved.
You have to call gst_bus_add_signal_watch() (like in 0.10) to enable emission of the signals. Without that you can only use the other ways to get notified about GstMessages on that bus.
Also just to be sure, you need a running GLib main loop on the default main context for this to work. Otherwise you need to do things a bit different.
For the updated question:
Check the documentation: gst_message_parse_state_changed() can be used to parse the old, new and pending state from the message. This is also still the same as in 0.10. From the application point of view, and conceptionally nothing much has changed really between 0.10 and 1.0
Also you shouldn't do this timeout-waiting as it will block your wxwidget main loop. Easiest solution would be to use a sync bus handler (which you already have) and dispatch all messages from there to some callback on the wxwidget main loop.