ImGui overlay with UpdateLayeredWindow function - overlay

As title, I'm trying to create a partially transparent in-game overlay using ImGui that's clickable on the UI but click-through otherwise, i.e. you can click on the ImGUI elements but outside the elements you can interact with the game.
I was able to do it using
https://github.com/ocornut/imgui/blob/master/examples/example_win32_directx11/main.cpp
by making the window style as WS_EX_TOPMOST | WS_EX_LAYERED, and using
SetLayeredWindowAttributes(hwnd, RGB(0, 0, 0), 0, ULW_COLORKEY);
However this would impact the rendering performance of the underlying game.
So, I decided to try the UpdateLayeredWindow function.
Here is one of the templates I referenced :
https://github.com/riley-x/TransparentWindow
This does not impact the performance and worked perfectly, and now I have to integrate ImGUI rendering to replace the green ellipse.
However, ImGUI uses the following code :
g_pd3dDeviceContext->OMSetRenderTargets(1, &g_mainRenderTargetView, NULL);
g_pd3dDeviceContext->ClearRenderTargetView(g_mainRenderTargetView, clear_color_with_alpha);
But in order to use the
BOOL UpdateLayeredWindow(
HWND hWnd,
HDC hdcDst,
POINT *pptDst,
SIZE *psize,
HDC hdcSrc,
POINT *pptSrc,
COLORREF crKey,
BLENDFUNCTION *pblend,
DWORD dwFlags
);
function, I have to tell ImGUI to render to an HDC.
I've thought of some possible solutions,
I'm not sure how g_mainRenderTargetView can possibly bind to an HDC, unlike ID2D1Factory::CreateDCRenderTarget which can actually bind to an HDC with BindDC.
Or I could let the ImGUI render as usual but retrieve the back buffer as a bitmap, and then do the following
HDC hdcWnd = GetDC(hwnd);
HDC hdcMem = CreateCompatibleDC(hdcWnd);
HBITMAP memBitmap = CreateCompatibleBitmap(hdcWnd, rect.right - rect.left, rect.bottom - rect.top);
SelectObject(hdcMem, memBitmap);
m_pRenderTarget->BindDC(hdcMem, &rect);
m_pRenderTarget->BeginDraw();
m_pRenderTarget->Clear({ 0 });
m_pRenderTarget->DrawBitmap(bitmap.Get()); // ???
m_pRenderTarget->EndDraw();
POINT pt0 = { 0 };
SIZE sz = { rect.right - rect.left, rect.bottom - rect.top };
BLENDFUNCTION bfunc = { 0 };
bfunc.AlphaFormat = AC_SRC_ALPHA;
bfunc.BlendFlags = 0;
bfunc.BlendOp = AC_SRC_OVER;
bfunc.SourceConstantAlpha = 255;
UpdateLayeredWindow(hwnd, hdcWnd, &pt0, &sz, hdcMem, &pt0, 0, &bfunc, ULW_COLORKEY);
DeleteObject(memBitmap);
ReleaseDC(hwnd, hdcWnd);
ReleaseDC(0, hdcMem);
where m_pRenderTarget is made from :
D2D1_RENDER_TARGET_PROPERTIES props = D2D1::RenderTargetProperties(
D2D1_RENDER_TARGET_TYPE_DEFAULT,
D2D1::PixelFormat(
DXGI_FORMAT_B8G8R8A8_UNORM,
D2D1_ALPHA_MODE_PREMULTIPLIED),
0,
0,
D2D1_RENDER_TARGET_USAGE_NONE,
D2D1_FEATURE_LEVEL_10
);
d2Factory->CreateDCRenderTarget(&props, m_pRenderTarget.GetAddressOf());
The problem is when I tried to retrieve the backbuffer bitmap, it always failed.
Or maybe there is some other way to make the overlay work without using UpdateLayeredWindow?

Related

How to capture the window where flutter widget's area is located

hi guys im making my own flutter desktop program
but when trying to make a fucntion that capturing the window where flutter widget's area is located i have troubled..
first, i tried to transparent the program's background and capture the widget(by using flutter_acrylic and RenderRepaintBoundary).transparent image
when i caputred widget the image wasnt include transparent window.. but only tranparent color was. because caputre algorithm see only the widget. im blocked in this problem.widget capture image
anyone who have idea for this problem please give me wisdom...
+Now I tying to get information about the win32 API. Any ideas on win32 capture API would be much appreciated.
i did!! anyone want to know see these links https://learn.microsoft.com/ko-kr/windows/win32/gdi/capturing-an-image + https://pub.dev/packages/win32
output image
About win32 capture API, you can use GDI BitBlt function. Here is a MSDN sample Capturing an Image.
You need to find the flutter desktop program window's handle with FindWindow, then get the rect range, and capture the window from DC.
HDC hWndDC = GetWindowDC(hwnd);
RECT capture_rect{ 0,0,0,0 };
RECT wnd_rect;
RECT real_rect;
GetWindowRect(hwnd, &wnd_rect);
DwmGetWindowAttribute(hwnd, DWMWINDOWATTRIBUTE::DWMWA_EXTENDED_FRAME_BOUNDS, &real_rect, sizeof(RECT));
int offset_left = real_rect.left - wnd_rect.left;
int offset_top = real_rect.top - wnd_rect.top;
capture_rect = RECT{ offset_left,offset_top,real_rect.right - real_rect.left + offset_left,real_rect.bottom - real_rect.top + offset_top };
//capture_rect ?? wnd_rect (You can calculate the capture_rect based on the size of your window)
int width = capture_rect.right - capture_rect.left;
int height = capture_rect.bottom - capture_rect.top;
HDC hMemDC = CreateCompatibleDC(hWndDC);
HBITMAP hBitmap = CreateCompatibleBitmap(hWndDC, width, height);
SelectObject(hMemDC, hBitmap);
BitmapPtr bitmap;
bool ok = BitBlt(hMemDC, 0, 0, width, height, hWndDC, capture_rect.left, capture_rect.top, SRCCOPY);
DeleteDC(hWndDC);
DeleteDC(hMemDC);
DeleteObject(hBitmap);

I want to implement the automatic capture function that shows the game view in Unity

I'm using this as a capture.
ScreenCapture.CaptureScreenshot($"Screenshot{_shotIndex}.png", size);
But the CpatureScreenshot() is too slow.
So I tried different scripts but failed. One of them is that I succeeded in capturing the scene view.
public void TakeTransparentScreenshot(Camera cam, int width, int height, string savePath)
{
// Depending on your render pipeline, this may not work.
var bak_cam_targetTexture = cam.targetTexture;
var bak_cam_clearFlags = cam.clearFlags;
var bak_RenderTexture_active = RenderTexture.active;
var tex_transparent = new Texture2D(width, height, TextureFormat.ARGB32, false);
// Must use 24-bit depth buffer to be able to fill background.
var render_texture = RenderTexture.GetTemporary(width, height, 24, RenderTextureFormat.ARGB32);
var grab_area = new Rect(0, 0, width, height);
RenderTexture.active = render_texture;
cam.targetTexture = render_texture;
cam.clearFlags = CameraClearFlags.SolidColor;
// Simple: use a clear background
cam.backgroundColor = Color.clear;
cam.Render();
tex_transparent.ReadPixels(grab_area, 0, 0);
tex_transparent.Apply();
// Encode the resulting output texture to a byte array then write to the file
byte[] pngShot = ImageConversion.EncodeToPNG(tex_transparent);
File.WriteAllBytes(savePath, pngShot);
cam.clearFlags = bak_cam_clearFlags;
cam.targetTexture = bak_cam_targetTexture;
RenderTexture.active = bak_RenderTexture_active;
RenderTexture.ReleaseTemporary(render_texture);
Texture2D.Destroy(tex_transparent);
}
How do I capture a screenshot in Unity3d with a transparent background?
I need to capture the game view, but this script does the scene view.
Inevitably I tried to implement a feature that was automatically captured using CaptureScreenshot() and Coroutine. it was too slow to work.

How can I get the image of a PictureBox generated with OpenGL?

I have associated the PictureBox to OpenGL through its handle
hDC = GetDC((HWND)picBoxA->Handle.ToInt64());
This works perfectly and the graphics are drawn in the PictureBox.
The problem is that I've been looking for all day for a way to get the data to be copied to another PictureBox or a bitmap, but I can not find the way.
I have tried it in different ways and I think the logic solution should be copy the device context using BitBlt, copying from the HDC, but this just copies the background color (PicBoxA->backColor).
HDC hdcA = GetDC((HWND)picBoxA->Handle.ToInt64());
HDC hdcB = GetDC((HWND)picBoxB->Handle.ToInt64());
BitBlt(hdcB, 0, 0, 100, 100, hdcA, 0, 0, SRCCOPY);
Where are these image data? How can I access them? Is it not possible to capture the inside of the PictureBox?

Gtk - draw event fired for wrong widget, and widget not redrawn

I'm trying to create a custom scrollable text area. I created a DrawingArea and a ScrollBar inside a Grid. I have attached the draw event of DrawingArea to this.on_draw method which simply looks at ScrollBar's value and moves the Cairo.Context appropriately before drawing the Pango.Layout.
The first problem is that this.on_draw is getting invoked whenever the ScrollBar is touched even though I have not registered any events with ScrollBar. How do I prevent this, or check this?
The second problem is that even though this.on_draw is invoked, the changes made to the Context is not displayed unless the ScrollBar value is near 0 or 100 (100 is the upper value of Adjustment). Why is this happening?
I did find out that if I connect the value_changed event of ScrollBar to a method that calls queue_redraw of DrawingArea, it would invoke this.on_draw and display it properly after it. But due to the second problem, I think this.on_draw is getting invoked too many times unnecessarily. So, what is the "proper" way of accomplishing this?
using Cairo;
using Gdk;
using Gtk;
using Pango;
public class Texter : Gtk.Window {
private Gtk.DrawingArea darea;
private Gtk.Scrollbar scroll;
private string text = "Hello\nWorld!";
public Texter () {
GLib.Object (type: Gtk.WindowType.TOPLEVEL);
Gtk.Grid grid = new Gtk.Grid();
this.add (grid);
var drawing_area = new Gtk.DrawingArea ();
drawing_area.set_size_request (200, 200);
drawing_area.expand = true;
drawing_area.draw.connect (this.on_draw);
grid.attach (drawing_area, 0, 0);
var scrollbar = new Gtk.Scrollbar (Gtk.Orientation.VERTICAL,
new Gtk.Adjustment(0, 0, 100, 0, 0, 1));
grid.attach (scrollbar, 1, 0);
this.darea = drawing_area;
this.scroll = scrollbar;
this.destroy.connect (Gtk.main_quit);
}
private bool on_draw (Gtk.Widget sender, Cairo.Context ctx) {
ctx.set_source_rgb (0.9, 0.9, 0.9);
ctx.paint ();
var y_offset = this.scroll.get_value();
stdout.printf("%f\n", y_offset);
ctx.set_source_rgb (0.25, 0.25, 0.25);
ctx.move_to(0, 100 - y_offset);
var layout = Pango.cairo_create_layout(ctx);
layout.set_font_description(Pango.FontDescription.from_string("Sans 12"));
layout.set_auto_dir(false);
layout.set_text(this.text, this.text.length);
Pango.cairo_show_layout(ctx, layout);
return false;
}
static int main (string[] args) {
Gtk.init (ref args);
var window = new Texter ();
window.show_all ();
Gtk.main ();
return 0;
}
}
Also, please point out any (possibly unrelated) mistake if you find one in the above code.
The part that you are missing is that a draw signal does not mean "redraw everything". Instead, GTK+ sets the clip region of the cairo context to the part that needs to be redrawn, so everything else you do doesn't have any effect. The cairo function cairo_clip_extents() will tell you what that region is. The queue_draw_area() method on GtkWidget will allow you to explicitly mark a certain area for drawing, instead of the entire widget.
But your approach to scrollbars is wrong anyway: you're trying to build the entire infrastructure from scratch! Consider using a GtkScrolledWindow instead. This automatically takes care of all the details of scrolling for you, and will give you the overlay scrollbars I mentioned. All you need to do is set the size of the GtkDrawingArea to the size you want it to be, and GtkScrolledWindow will do the rest. The best way to do this is to subclass GtkDrawingArea and override the get_preferred_height() and/or get_preferred_width() virtual functions (being sure to set both minimum and natural sizes to the sizes you want for that particular dimension). If you ever need to change this size later, call the queue_resize() method of GtkWidget. (You probably could get away with just using set_size_request(), but what I described is the preferred way of doing this.) Doing this also gives you the advantage of not having to worry about transforming your cairo coordinates; GtkScrolledWindow does this for you.

Animated GIF in SWT table/tree viewer cell

http://www.java2s.com/Code/Java/SWT-JFace-Eclipse/DisplayananimatedGIF.htm describes how to display an animated GIF in SWT - in general. While the code works and is easily comprehensible I'm facing serious issues displaying an animated GIF in a SWT/JFace table/tree viewer cell with that technique. -> all code below
Essentially, I implemented my own OwnerDrawLabelProvider which creates an ImageLoader in paint(Event, Object) and starts an animation thread. The problem seems to be that this animation thread is not the UI thread and I don't know which GC or Display instance to use in its run() method.
I tried creating a separate GC instance in the thread's constructor - derived from event.gc - but the thread fails writing to that GC as soon as I step out of the debugger...
Sat Jan 9 22:11:57 192.168.1.6.local.home java[25387] : CGContextConcatCTM: invalid context 0x0
2010-01-09 22:12:18.356 java[25387:17b03] It does not make sense to draw an image when [NSGraphicsContext currentContext] is nil. This is a programming error. Break on _NSWarnForDrawingImageWithNoCurrentContext to debug. This will be logged only once. This may break in the future.
Sat Jan 9 22:12:41 192.168.1.6.local.home java[25387] : CGContextConcatCTM: invalid context 0x0
How do I need to handle this situation?
Below are the relevant code sections:
/* Called by paint(Event, Object). */
private void paintAnimated(final Event event, final ImageLoader imageLoader) {
if (imageLoader == null || ArrayUtils.isEmpty(imageLoader.data)) {
return;
}
final Thread animateThread = new AnimationThread(event, imageLoader);
animateThread.setDaemon(true);
animateThread.start();
}
private class AnimationThread extends Thread {
private Display display;
private GC gc;
private ImageLoader imageLoader;
private Color background;
public AnimationThread(final Event event, final ImageLoader imageLoader) {
super("Animation");
this.display = event.display;
/*
* If we were to simply reference event.gc it would be reset/empty by the time it's being used
* in run().
*/
this.gc = new GC(event.gc.getDevice());
this.imageLoader = imageLoader;
this.background = getBackground(event.item, event.index);
}
#Override
public void run() {
/*
* Create an off-screen image to draw on, and fill it with the shell background.
*/
final Image offScreenImage =
new Image(this.display, this.imageLoader.logicalScreenWidth,
this.imageLoader.logicalScreenHeight);
final GC offScreenImageGC = new GC(offScreenImage);
offScreenImageGC.setBackground(this.background);
offScreenImageGC.fillRectangle(0, 0, this.imageLoader.logicalScreenWidth,
this.imageLoader.logicalScreenHeight);
Image image = null;
try {
/* Create the first image and draw it on the off-screen image. */
int imageDataIndex = 0;
ImageData imageData = this.imageLoader.data[imageDataIndex];
image = new Image(this.display, imageData);
offScreenImageGC.drawImage(image, 0, 0, imageData.width, imageData.height, imageData.x,
imageData.y, imageData.width, imageData.height);
/*
* Now loop through the images, creating and drawing each one on the off-screen image before
* drawing it on the shell.
*/
int repeatCount = this.imageLoader.repeatCount;
while (this.imageLoader.repeatCount == 0 || repeatCount > 0) {
switch (imageData.disposalMethod) {
case SWT.DM_FILL_BACKGROUND:
/* Fill with the background color before drawing. */
offScreenImageGC.setBackground(this.background);
offScreenImageGC.fillRectangle(imageData.x, imageData.y, imageData.width,
imageData.height);
break;
case SWT.DM_FILL_PREVIOUS:
// Restore the previous image before drawing.
offScreenImageGC.drawImage(image, 0, 0, imageData.width, imageData.height,
imageData.x, imageData.y, imageData.width, imageData.height);
break;
}
imageDataIndex = (imageDataIndex + 1) % this.imageLoader.data.length;
imageData = this.imageLoader.data[imageDataIndex];
image.dispose();
image = new Image(this.display, imageData);
offScreenImageGC.drawImage(image, 0, 0, imageData.width, imageData.height, imageData.x,
imageData.y, imageData.width, imageData.height);
// Draw the off-screen image.
this.gc.drawImage(offScreenImage, 0, 0);
/*
* Sleeps for the specified delay time (adding commonly-used slow-down fudge factors).
*/
try {
int ms = imageData.delayTime * 10;
if (ms
I posted the same problem to the SWT newsgroup http://www.eclipse.org/forums/index.php?t=tree&th=160398
After many hours of frustrating trial-and-error a co-worker came up with a feasible solution. My initial approaches to have this implemented in a totally self-contained LabelProvider failed miserably.
One approach that didn't work was to override LabelProvider#update() and to call timerExec(100, new Runnable() {...viewer.update()... from within that method. The "life"-cycle of that is hard to control and it uses too many CPU cycles (10% on my MacBook).
One of the colleague's ideas was to implement a custom TableEditor: a label with an image (one frame of the animated GIF) but no text. Each TableEditor instance would start its own thread in which it updates the label's image. This works quite well, but there's a separate "animation" thread for each animated icon. Also, this was a performance killer, consumed 25% CPU on my MacBook.
The final approach has three building blocks
an OwnerDrawLabelProvider which paints either a static image or the frame of an animated GIF
an animation thread (the pace maker), it calls redraw() for the column which contains the animated GIFs and it also calls update()
and the viewer's content provider that controls the animation thread.
Details in my blog http://www.frightanic.com/2010/02/09/animated-gif-in-swt-tabletree-viewer-cell/.
Can't you let a LabelProvider return different images and then call viewer.update(...) on the elements you want to animate. You can use Display.timerExec to get a callback instead of having a separate thread.
See my answer here for how you can change colors. You should be able to do something similar with images.