Very slow frame rate and ImGui::MouseDoubleClicked

Flix01 commented

2014-11-07 13:27:46 +00:00

(Migrated from github.com)

Hi everybody. Many compliments for the project! That's what I was looking for...
I'm new to ImGui (and to Github thanx to ImGui), so please be patient with me...
Now the issue:

I've found out that when I clamp the frame rate of my application to save CPU/GPU usage, double clicks stop working.
That's probably caused to the fact that ImGui::NewFrame() is called less frequently and the ImGui::MouseDoubleClickTime is too small to catch double clicks in these cases.

I've been able to work around the issue by using my own variables, catching double click events myself and refeeding the ImGui::MouseDoubleClicked array with them soon after ImGui::NewFrame().

This works, but maybe we could add a variable to tell ImGui we're going to set the ImGui::MouseDoubleClicked array directly ourself and ImGui should skip updating it inside ImGui::NewFrame(), can't we?

Another think I would like to know is whether or not ImGui can be used at zero frame per seconds.
I mean in 3D editor-like applications, where a postUpdate() should be called only as a response to some action by the user (e.g. dragging a window and stuff like that). That's the main reason why I've tried clamping the frame rate in the first place (currently I've tested it at 10 fps: ImGui works well with the double click fix, it's only not very smooth when dragging a window).

P.S. I'm using openGL on Ubuntu Linux 64 bit, but that shouldn't make any difference.

P.S.2. Actually another minor issue happened when working at low frame rates with my fix: in the demo project I wasn't able to minimize the "Debug" window (double clicking its title bar). All the other windows work as expected. And if I add the ImGui::Begin()/End() when creating it PLUS a name different from "Debug" it starts working again.

Hi everybody. Many compliments for the project! That's what I was looking for... I'm new to ImGui (and to Github thanx to ImGui), so please be patient with me... Now the issue: I've found out that when I clamp the frame rate of my application to save CPU/GPU usage, double clicks stop working. That's probably caused to the fact that ImGui::NewFrame() is called less frequently and the ImGui::MouseDoubleClickTime is too small to catch double clicks in these cases. I've been able to work around the issue by using my own variables, catching double click events myself and refeeding the ImGui::MouseDoubleClicked array with them soon after ImGui::NewFrame(). This works, but maybe we could add a variable to tell ImGui we're going to set the ImGui::MouseDoubleClicked array directly ourself and ImGui should skip updating it inside ImGui::NewFrame(), can't we? Another think I would like to know is whether or not ImGui can be used at zero frame per seconds. I mean in 3D editor-like applications, where a postUpdate() should be called only as a response to some action by the user (e.g. dragging a window and stuff like that). That's the main reason why I've tried clamping the frame rate in the first place (currently I've tested it at 10 fps: ImGui works well with the double click fix, it's only not very smooth when dragging a window). P.S. I'm using openGL on Ubuntu Linux 64 bit, but that shouldn't make any difference. P.S.2. Actually another minor issue happened when working at low frame rates with my fix: in the demo project I wasn't able to minimize the "Debug" window (double clicking its title bar). All the other windows work as expected. And if I add the ImGui::Begin()/End() when creating it PLUS a name different from "Debug" it starts working again.

ocornut commented

2014-11-10 01:48:02 +00:00

(Migrated from github.com)

(A)
Input polling / aliasing is a gross mistake I made at the time I was trying to "simplify" the setup procedure prior to release. The current input method is not suited to low frame-rate. I will undergo a series of changes to takes events as input. For now your workaround poking within ImGui::MouseDoubleClicked[] is ok.

The change itself to easy but I'm concerned about breaking the API for people who update to newer version of the library. I may bundle a couple of API breaking changes all-together. The problem will breaking the API is that even if the "fix" is trivial it makes people less inclined to stay up to date so there's a balance to find here

In particular it seems that some gestures on MacOS creates mouse clicks that are very short (probably sending a click event and a release event in a short amount of time). The OpenGL example added those lines as a workaround to cope with it:
io.MouseDown[0] = mousePressed[0] || glfwGetMouseButton(window, GLFW_MOUSE_BUTTON_LEFT) != 0; // If a mouse press event came, always pass it as "mouse held this frame", so we don't miss click-release events that are shorter than 1 frame.

But the system should be revamped to not miss events.

(B)
Working at zero frame rate is not supported but your application could probably adjust the framerate dynamically somehow. I have never considered it because I always used ImGui in highly dynamic scenario displaying "live" values so I wanted to have it running at maximum framerate. Is your concern that updating ImGui is taking too much of your frame? I'd be interested in numbers and examples, in my experience on windows you can get pretty elaborate gui running under 1 ms.

(A) Input polling / aliasing is a gross mistake I made at the time I was trying to "simplify" the setup procedure prior to release. The current input method is not suited to low frame-rate. I will undergo a series of changes to takes events as input. For now your workaround poking within ImGui::MouseDoubleClicked[] is ok. The change itself to easy but I'm concerned about breaking the API for people who update to newer version of the library. I may bundle a couple of API breaking changes all-together. The problem will breaking the API is that even if the "fix" is trivial it makes people less inclined to stay up to date so there's a balance to find here In particular it seems that some gestures on MacOS creates mouse clicks that are very short (probably sending a click event and a release event in a short amount of time). The OpenGL example added those lines as a workaround to cope with it: io.MouseDown[0] = mousePressed[0] || glfwGetMouseButton(window, GLFW_MOUSE_BUTTON_LEFT) != 0; // If a mouse press event came, always pass it as "mouse held this frame", so we don't miss click-release events that are shorter than 1 frame. But the system should be revamped to not miss events. (B) Working at zero frame rate is not supported but your application could probably adjust the framerate dynamically somehow. I have never considered it because I always used ImGui in highly dynamic scenario displaying "live" values so I wanted to have it running at maximum framerate. Is your concern that updating ImGui is taking too much of your frame? I'd be interested in numbers and examples, in my experience on windows you can get pretty elaborate gui running under 1 ms.

UUSim commented

2014-11-10 19:50:23 +00:00

(Migrated from github.com)

A different approach is to render the application's contents to a separate buffer, at 0fps (or at any desirable fps).
Then render the Gui and any other overlays at a different framerate (e.g. whenever keyboard or mouse state changes), and blend the gui buffer, and the application buffer.

A different approach is to render the application's contents to a separate buffer, at 0fps (or at any desirable fps). Then render the Gui and any other overlays at a different framerate (e.g. whenever keyboard or mouse state changes), and blend the gui buffer, and the application buffer.

ocornut commented

2015-05-09 09:20:55 +00:00

(Migrated from github.com)

@Pagghiu seems to have it working at 0 fps here
https://github.com/ocornut/imgui/issues/207

@Pagghiu seems to have it working at 0 fps here https://github.com/ocornut/imgui/issues/207

Pagghiu commented

2015-05-12 10:56:54 +00:00

(Migrated from github.com)

Exactly!
I confirm that is totally feasible to have IMGUI working like a retained mode UI, redrawing only when necessary.

I have done it in windows and osx, integrating with MFC (or pure WinAPI) and Cocoa (with CVDisplayLink for the ones interested).

The key is to trigger a redraw operation only when "something happens". For example on windows I'm subclassing the WndProc and simply triggering a redraw every time I receive a message that might change the visual state (Mouse Events, Window Resize Events, KeyDown/Up/Char Events, background erase etc.). I also trigger anytime I receive an IO message from application async io loop (may not apply to you case if you're using Sync IO).
You will have to handle threading properly if not everything runs on the same thread, like on OSX where the CVDisplayLink thread is handled by the OS to keep the 60 FPS without tearing/vsync problems while Mouse etc. messages are on the main application thread.

Exactly! I confirm that is totally feasible to have IMGUI working like a retained mode UI, redrawing only when necessary. I have done it in windows and osx, integrating with MFC (or pure WinAPI) and Cocoa (with CVDisplayLink for the ones interested). The key is to trigger a redraw operation only when "something happens". For example on windows I'm subclassing the WndProc and simply triggering a redraw every time I receive a message that might change the visual state (Mouse Events, Window Resize Events, KeyDown/Up/Char Events, background erase etc.). I also trigger anytime I receive an IO message from application async io loop (may not apply to you case if you're using Sync IO). You will have to handle threading properly if not everything runs on the same thread, like on OSX where the CVDisplayLink thread is handled by the OS to keep the 60 FPS without tearing/vsync problems while Mouse etc. messages are on the main application thread.

ocornut commented

2015-05-12 11:06:18 +00:00

(Migrated from github.com)

I wouldn't call it "working like a retained UI", it is still quite different. It's probably cool if you have the sort of application that requires idling. There is the possibility that past a certain update frequency from the network it'll be always running at max speed and that may be problematic for your use case. Perhaps local inputs events vs network inputs can be treated differently.

I am working on menus at the moment and I need to add a timer to facilitate moving between menus (when menus appears/disappears immediately it is easy while dragging from a hovered item to a submenu to hover something else). I suppose what you want to do when you get user inputs is to keep the app running at stable frame-rate for 1 or 2 seconds past each input, so interactive elements that are based on timer will work. If your only aim if you have idle application when not being used that could be a viable workaround.

I wouldn't call it "working like a retained UI", it is still quite different. It's probably cool if you have the sort of application that requires idling. There is the possibility that past a certain update frequency from the network it'll be always running at max speed and that may be problematic for your use case. Perhaps local inputs events vs network inputs can be treated differently. I am working on menus at the moment and I need to add a timer to facilitate moving between menus (when menus appears/disappears immediately it is easy while dragging from a hovered item to a submenu to hover something else). I suppose what you want to do when you get user inputs is to keep the app running at stable frame-rate for 1 or 2 seconds past each input, so interactive elements that are based on timer will work. If your only aim if you have idle application when not being used that could be a viable workaround.

Pagghiu commented

2015-05-12 11:25:33 +00:00

(Migrated from github.com)

Oh, I know that retained mode ui only redraw specific section of a window, with dirty rects etc. while here we're always redrawing everything when we need.
Probably yes, a better description of what I'm doing is that I'm not rendering anything during idle time when nothing happens and I'm running almost full speed when user is interacting (mouse move etc.) or io happens.
If you will start to have some kind of "animations" or anything that will happen some seconds in the future, like those menus, I would suggest to keep a publicly readable flag that could tell "I need to re-render in x milliseconds" or at least "I need to re-render in a while".
It would just be the minimum of all the timers setup in a frame in case, you could easily calculate them in the End() function and let anyone that wants to optimise idle time do the right thing.

Oh and regarding the update frequency from the network, I've already experienced this problem and already solved by delaying the first re-render with a timer that starts after the first IO event is arrived.
As an example if I'm on idle and I receive 100 io/events in 200 milliseconds (it happens) I'm not rendering anything, letting the network going full speed, then I update the UI. The lag in screen update is not noticeable at all. If network is continuing to be active, it's fine, it will refresh screen at 5 FPS. If needed I could write some logic to ramp up the FPS in the case of a continued network output but in my particular case I don't care, because If user is interacting with the software, he will move mouse and do things that will trigger more frequent updates anyway.

Oh, I know that retained mode ui only redraw specific section of a window, with dirty rects etc. while here we're always redrawing everything when we need. Probably yes, a better description of what I'm doing is that I'm not rendering anything during idle time when nothing happens and I'm running almost full speed when user is interacting (mouse move etc.) or io happens. If you will start to have some kind of "animations" or anything that will happen some seconds in the future, like those menus, I would suggest to keep a publicly readable flag that could tell "I need to re-render in x milliseconds" or at least "I need to re-render in a while". It would just be the minimum of all the timers setup in a frame in case, you could easily calculate them in the End() function and let anyone that wants to optimise idle time do the right thing. Oh and regarding the update frequency from the network, I've already experienced this problem and already solved by delaying the first re-render with a timer that starts after the first IO event is arrived. As an example if I'm on idle and I receive 100 io/events in 200 milliseconds (it happens) I'm not rendering anything, letting the network going full speed, then I update the UI. The lag in screen update is not noticeable at all. If network is continuing to be active, it's fine, it will refresh screen at 5 FPS. If needed I could write some logic to ramp up the FPS in the case of a continued network output but in my particular case I don't care, because If user is interacting with the software, he will move mouse and do things that will trigger more frequent updates anyway.

ocornut commented

2015-05-26 08:43:00 +00:00

(Migrated from github.com)

You asked (here https://github.com/ocornut/imgui/issues/126#issuecomment-105418646 )

"Please expose the "next timer due" value somewhere in the api, so it's easy to schedule a refresh in idle/event driven re-drawing scenarios. If you will take the time to do that, also include the text cursor timer as well (one just needs the min of the two)."

About including the text cursor, that would mean that whenever a text field is active your application would stay awake. So if you clicked a text field, etc. and alt-tabbed somewhere else it would stay awake, probably not what you want here? We need to differentiate animations that have timers that have an effect interactions with those which don't.

You asked (here https://github.com/ocornut/imgui/issues/126#issuecomment-105418646 ) "Please expose the "next timer due" value somewhere in the api, so it's easy to schedule a refresh in idle/event driven re-drawing scenarios. If you will take the time to do that, also include the text cursor timer as well (one just needs the min of the two)." About including the text cursor, that would mean that whenever a text field is active your application would stay awake. So if you clicked a text field, etc. and alt-tabbed somewhere else it would stay awake, probably not what you want here? We need to differentiate animations that have timers that have an effect interactions with those which don't.

Pagghiu commented

2015-05-26 09:26:36 +00:00

(Migrated from github.com)

I would say that if you clicked a text field and alt-tabed somewhere else, the correct behaviour for the app is to stay awake.
It's not a big dial for a number of reasons: the refreshes for the blinking cursors would be just a couple in a second, it's not distinguish-able from a 100% sleep state (cpu will still stay 0% all the time).
Second, one could intercept application "deactivated" values (WM_ACTIVE in windows and similar on other os) and decide to pause the entire app (rendering) after a while (that's what I currently do).

Maybe I'm missing something, but I don't get why it could be useful to differentiate animations that effect interactions and those which don't, and I don't want the beautiful ImGui api to became too complex! :)

I would say that if you clicked a text field and alt-tabed somewhere else, the correct behaviour for the app is to stay awake. It's not a big dial for a number of reasons: the refreshes for the blinking cursors would be just a couple in a second, it's not distinguish-able from a 100% sleep state (cpu will still stay 0% all the time). Second, one could intercept application "deactivated" values (WM_ACTIVE in windows and similar on other os) and decide to pause the entire app (rendering) after a while (that's what I currently do). Maybe I'm missing something, but I don't get why it could be useful to differentiate animations that effect interactions and those which don't, and I don't want the beautiful ImGui api to became too complex! :)

wizzard0 commented

2016-11-11 14:19:13 +00:00

(Migrated from github.com)

Maybe we could simplify this by creating a 2-event queue, like storing "XWasRecentlyPressed" + "XWasRecentlyReleased" instead of "XIsPressed"?

Flix01 commented

2017-02-08 11:05:53 +00:00

(Migrated from github.com)

I think it's better to close this very old issue I started many years ago...

😆 1

ocornut commented

2022-01-17 17:30:12 +00:00

(Migrated from github.com)

Better late than never, Dear ImGui 1.87 WIP now uses a new IO api (e.g. io.AddKeyEvent() io.AddMouseButtonEvent() etc.) which implement an input trickling queue, normally solving this problem correctly and everywhere for all types of inputs. We'll monitor feedback and expect bugs to have creeped as we reoverhauled that system.

See e.g.

new IO api b8e56dce83
input queue 7374b96f5c
There are many other commits but those should provide a good general sense of what changed.

Also see #4858 for a general overview (we'll improve this topic soon).

Better late than never, Dear ImGui 1.87 WIP now uses a new IO api (e.g. `io.AddKeyEvent()` `io.AddMouseButtonEvent()` etc.) which implement an input trickling queue, normally solving this problem correctly and everywhere for all types of inputs. We'll monitor feedback and expect bugs to have creeped as we reoverhauled that system. See e.g. - new IO api b8e56dce83985a49e2e236900d80980510b173f3 - input queue 7374b96f5cdf9adb1a1da033623ce2a4fd7f4934 There are many other commits but those should provide a good general sense of what changed. Also see #4858 for a general overview (we'll improve this topic soon).

🎉 1

Rows
Columns

Very slow frame rate and ImGui::MouseDoubleClicked #71