How to create a screenshot of a transparant Unity window + OS - unity3d

So for an app we are trying to create we would like to turn the Unity window of a standalone app transparant(everything besides a couple of buttons) and have the user take a screenshot of their view/OS + the unity layer together.
So example:
The user opens our application, clicks on a button and the whole unity window except for a couple of buttons turn transparant. The user can then use it's OS like normal, while the buttons mentioned earlier stay on top. The user can then click on of the buttons to create a screenshot of their OS, which we will then be saved to their system. This way we can for example show anything from within Unity(3D model, images) on top of the users OS, via a screenshot.
Currently, we can turn the window transparant with a simular setup like this: https://alastaira.wordpress.com/2015/06/15/creating-windowless-unity-applications/
That works fine, and so does clicking in windows etc. However, we would now like to create a screenshot and save it somewhere. For this we tried multiple things, and a couple of years back before we put this project aside, we got it working through a custom dll that uses the "using System.Drawing" code that we called from inside unity. See an example of this dll and code below.
using System.Drawing;
namespace ScreenShotDll
{
public class ScreenShotClass
{
public static void TakeScreenShotRect(int srcX, int srcY, int dstX, int dstY) //both fullscreen screenshot and cropped rectangle screenshot
{
int width = Math.Abs(srcX - dstX);
int height = Math.Abs(srcY - dstY);
Bitmap memoryImage;
memoryImage = new Bitmap(width, height);
Size s = new Size(memoryImage.Width, memoryImage.Height);
Graphics memoryGraphics = Graphics.FromImage(memoryImage);
memoryGraphics.CopyFromScreen(srcX, srcY, 0, 0, s);
string str = "";
try
{
str = string.Format(AppDomain.CurrentDomain.BaseDirectory + #"Screenshot.png");
}
catch (Exception er)
{
Console.WriteLine("Sorry, there was an error: " + er.Message);
Console.WriteLine();
}
memoryImage.Save(str);
}
However this does not seem to work anymore. We are on the IL2CPP backend in Unity and get the error: NotSupportedException: System.Drawing.Bitmap
We are also trying to use the user32.dll from within Unity and using the GetPixel, ReleaseDC, and GetActiveWindow functions of this, as posted on a couple of forums, but all we get there is a white image.
Any ways to adjust our custom dll or any other way to do this would be highly appreciated. Please let me know if you need more information.

After a couple of days, I managed to resolve this from within Unity.
I used the following code, to make a screenshot of the window, and save it to a bitmap. Then save it on my disk to a .png.
[DllImport("user32.dll", SetLastError = true)] static extern int GetSystemMetrics(int smIndex);
[DllImport("user32.dll", SetLastError = false)] static extern IntPtr GetDesktopWindow();
[DllImport("gdi32.dll", EntryPoint = "CreateCompatibleDC", SetLastError=true)] static extern IntPtr CreateCompatibleDC([In] IntPtr hdc);
[DllImport("gdi32.dll", EntryPoint = "DeleteDC")] public static extern bool DeleteDC([In] IntPtr hdc);
[DllImport("gdi32.dll", EntryPoint = "DeleteObject")] public static extern bool DeleteObject([In] IntPtr hObject);
[DllImport("gdi32.dll", EntryPoint = "CreateCompatibleBitmap")] static extern IntPtr CreateCompatibleBitmap([In] IntPtr hdc, int nWidth, int nHeight);
[DllImport("gdi32.dll", EntryPoint = "SelectObject")] public static extern IntPtr SelectObject([In] IntPtr hdc, [In] IntPtr hgdiobj);
[DllImport("gdi32.dll", EntryPoint = "BitBlt", SetLastError = true)] static extern bool BitBlt([In] IntPtr hdc, int nXDest, int nYDest, int nWidth, int nHeight, [In] IntPtr hdcSrc, int nXSrc, int nYSrc, uint dwRop);
private void screenShot()
{
Debug.Log("In screenShot!");
int nScreenWidth = GetSystemMetrics(0);
int nScreenHeight = GetSystemMetrics(1);
IntPtr hDesktopWnd = GetDesktopWindow();
IntPtr hDesktopDC = GetDC(hDesktopWnd);
IntPtr hCaptureDC = CreateCompatibleDC(hDesktopDC);
IntPtr hCaptureBitmap = CreateCompatibleBitmap(hDesktopDC, nScreenWidth, nScreenHeight);
SelectObject(hCaptureDC, hCaptureBitmap);
BitBlt(hCaptureDC, 0, 0, nScreenWidth, nScreenHeight, hDesktopDC, 0, 0, 0x00CC0020 | 0x40000000);
Bitmap bmp = Image.FromHbitmap(hCaptureBitmap);
ImageConverter converter = new ImageConverter();
byte[] bytes = (byte[])converter.ConvertTo(bmp, typeof(byte[]));
string path = Application.dataPath;
if (Application.platform == RuntimePlatform.OSXPlayer) {
path += "/../../";
}
else if (Application.platform == RuntimePlatform.WindowsPlayer) {
path += "/../";
}
File.WriteAllBytes(path + "Screenshot" + ".png", bytes);
ReleaseDC(hDesktopWnd, hDesktopDC);
DeleteDC(hCaptureDC);
DeleteObject(hCaptureBitmap);
}

I was able to get screenshots working by creating a class library. I setup my project as a .Net 4.7.2 library so this might not work for you (IL2CPP). I set my unity project scripting version to .Net 4.x I'm not sure if that was necessary. I just copy my libraries dll file into the assets folder.
These are the using statements that I needed, and CapturePng takes in a memory stream from Unity.
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
public static void CapturePng(MemoryStream memoryStream, int width, int height, int resolutionX, int resolutionY)
{
//Create a new bitmap.
using (var bmpScreenshot = new Bitmap(
width,
height,
PixelFormat.Format32bppArgb))
{
// Create a graphics object from the bitmap.
using (var gfxScreenshot = Graphics.FromImage(bmpScreenshot))
{
// Take the screenshot from the upper left corner to the right bottom corner.
gfxScreenshot.CopyFromScreen(
new Point(0, 0),
new Point(0, 0),
new Size(width, height),
CopyPixelOperation.SourceCopy);
// Save the screenshot to the specified path that the user has chosen.
new Bitmap(bmpScreenshot, new Size(resolutionX, resolutionY)).Save(memoryStream, ImageFormat.Png);
}
}
}
This is what my Unity code looks like.
Texture2D tex = new Texture2D(200, 300, TextureFormat.RGB24, false);
MemoryStream ms = new MemoryStream();
Screenshot.CapturePng(ms, Screen.currentResolution.width, Screen.currentResolution.height);
ms.Seek(0, SeekOrigin.Begin);
tex.LoadImage(ms.ToArray());
MeshRenderer.material.mainTexture = tex;
I hope this helps you some how, at least it doesn't require any "user32.dll" imports or "gdi32.dll" imports.

Related

How to convert DIB to Bitmap in .NET Core 3.1?

How do I properly retrieve a Device Independent Bitmap(DIB) pointer as a Bitmap in C# ASP.NET Core 3.1?
Issue: When I attempt to save a Bitmap from a Dib pointer, I receive System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory"
I'm using the following constructor
Bitmap(Int32, Int32, Int32, PixelFormat, IntPtr) with the assumption that the final parameter can use the DIB pointer.
using System.Drawing.Common.dll
Bitmap(Int32, Int32, Int32, PixelFormat, IntPtr)
Using the following syntax:
public class Example
{
[DllImport("kernel32.dll", CharSet = CharSet.Auto, ExactSpelling = true)]
public static extern IntPtr GlobalLock(int handle);
//https://stackoverflow.com/questions/2185944/why-must-stride-in-the-system-drawing-bitmap-constructor-be-a-multiple-of-4
public void GetStride(int width, PixelFormat format, ref int stride, ref int bytesPerPixel)
{
int bitsPerPixel = System.Drawing.Image.GetPixelFormatSize(format);
bytesPerPixel = (bitsPerPixel + 7) / 8;
stride = 4 * ((width * bytesPerPixel + 3) / 4);
}
public Bitmap ConvertDibToBitmap(IntPtr dib)
{
IntPtr dibPtr = GlobalLock(dib);
BITMAPINFOHEADER bmi = (BITMAPINFOHEADER)Marshal.PtrToStructure(dibPtr, typeof(BITMAPINFOHEADER));
/*
biBitCount: 24
biClrImportant: 0
biClrUsed: 0
biCompression: 0
biHeight: 2219
biPlanes: 1
biSize: 40
biSizeImage: 12058624
biWidth: 1704
biXPelsPerMeter: 7874
biYPelsPerMeter: 7874
*/
//We're able to initalize the object here:
Bitmap img = new Bitmap(bmi.biWidth, bmi.biHeight, stride, PixelFormat.Format32bppArgb, dibPtr);
img.Save("OutputTest.bmp");
/*This line throws the exception "System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory"*/
}
}
I'm assuming the issue is in the calculation of stride and/or the pixel format. I'm unsure of how to determine PixelFormat from BITMAPINFOHEADER

Unity code works in the editor but does not work on Build (EXE File) C#

My code load external images and build from them a SkyBox for unity3d.
All the files of the SkyBox are on the right paths (Copied manually).
The code loads 6 external images and then builds a SkyBox And I think there's a problem in the loading when it's in build stat. (But not sure)
Or maybe Unity prevents me from doing it?
And i have no error or warning code. It really drives me crazy!!!
using System.IO;
using UnityEngine;
public class ChangeSkyBox : MonoBehaviour
{
public static Texture2D LoadPNG(string filePath)
{
Texture2D tex = null;
byte[] fileData;
if (File.Exists(filePath))
{
fileData = File.ReadAllBytes(filePath);
tex = new Texture2D(1024, 1024);
tex.LoadImage(fileData);
}
tex.wrapMode = TextureWrapMode.Clamp;
return tex;
}
public static Material CreateSkyboxMaterial(SkyboxManifest manifest)
{
Material result = new Material(Shader.Find("RenderFX/Skybox"));
result.SetTexture("_FrontTex", manifest.textures[0]);
result.SetTexture("_BackTex", manifest.textures[1]);
result.SetTexture("_LeftTex", manifest.textures[2]);
result.SetTexture("_RightTex", manifest.textures[3]);
result.SetTexture("_UpTex", manifest.textures[4]);
result.SetTexture("_DownTex", manifest.textures[5]);
return result;
}
private Texture2D[] textures;
private void Start()
{
Texture2D xt1 = LoadPNG(Directory.GetCurrentDirectory() + "\\Assets\\SkyBox\\Front.png");
Texture2D xt2 = LoadPNG(Directory.GetCurrentDirectory() + "\\Assets\\SkyBox\\Back.png");
Texture2D xt3 = LoadPNG(Directory.GetCurrentDirectory() + "\\Assets\\SkyBox\\Left.png");
Texture2D xt4 = LoadPNG(Directory.GetCurrentDirectory() + "\\Assets\\SkyBox\\Right.png");
Texture2D xt5 = LoadPNG(Directory.GetCurrentDirectory() + "\\Assets\\SkyBox\\Top.png");
Texture2D xt6 = LoadPNG(Directory.GetCurrentDirectory() + "\\Assets\\SkyBox\\Bottom.png");
SkyboxManifest manifest = new SkyboxManifest(xt1, xt2, xt3, xt4, xt5, xt6);
Material newMat = new Material(Shader.Find("RenderFX/Skybox"));
newMat = CreateSkyboxMaterial(manifest);
RenderSettings.skybox = newMat;
DynamicGI.UpdateEnvironment();
}
}
public struct SkyboxManifest
{
public Texture2D[] textures;
public SkyboxManifest(Texture2D front, Texture2D back, Texture2D left, Texture2D right, Texture2D up, Texture2D down)
{
textures = new Texture2D[6]
{
front,
back,
left,
right,
up,
down
};
}
}
I believe your problem lies on this Line.
Material result = new Material(Shader.Find("RenderFX/Skybox"));
Unity cannot find it at runtime. To fix it, make the "base" Material by hand in Unity and attach it to your script throurgh the Inspector.

Powershell - capture mouse click event inside a powershell console

I'm trying to do some trick with Powershell. I want to write a script and that script can listen to the mouse events (click, move, etc) inside the powershell console.
For example, while my script is active, when I click the mouse inside the powershell console, the console can output the position of my cursor.
Is it possible? If possible, how?
Thanks in advance.
I've found that it's possible to do this using the following:
[System.Windows.Forms.UserControl]::MouseButtons
Which returns a string of the currently pressed mouse buttons. We poll this and [System.Windows.Forms.Cursor]::Position as per WorWin to keep track of where the mouse is and what buttons are pressed.
Keeping track of where the mouse is inside of the console window is a bit tricky, though. I personally use a custom class for this.
Add-Type -ReferencedAssemblies System.Drawing #"
using System;
using System.Drawing;
using System.Runtime.InteropServices;
public class Window
{
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern bool GetWindowRect(IntPtr hWnd, out RECT lpRect);
[DllImport("user32.dll")]
public static extern IntPtr GetForegroundWindow();
public RECT bounds;
public bool isForeground;
private IntPtr hWnd;
public int Width;
public int Height;
public Window(IntPtr handle)
{
hWnd = handle;
Update();
}
public void Update()
{
if(GetWindowRect(hWnd, out bounds))
{
Width = bounds.Right - bounds.Left;
Height = bounds.Bottom - bounds.Top;
if(GetForegroundWindow() == hWnd){isForeground = true;}
else{isForeground = false;}
}
}
public bool Contains(Point pt)
{
return bounds.Contains(pt);
}
}
public struct RECT
{
public int Left;
public int Top;
public int Right;
public int Bottom;
public Boolean Contains(Point pt)
{
if( (pt.X >= Left) && (pt.X < Right) && (pt.Y >= Top) && (pt.Y < Bottom) ){ return true;}
return false;
}
}
public struct POINT
{
public int X;
public int Y;
public static implicit operator Point(POINT point)
{
return new Point(point.X, point.Y);
}
}
"#
To get the Powershell console window:
$ourID = (get-process -id (Get-WmiObject -class win32_process -Filter ("ProcessID = $pid")).parentprocessid).mainwindowhandle;
$win = New-Object Window($ourID);
Now, $win.bounds gives us the outer bounds of the console window, so if we want to find which buffercell the cursor is in we're going to have to do some work on our bounds.
$innerOffsets = new-object RECT;
$innerOffsets.Top = [Windows.Forms.SystemInformation]::CaptionHeight + [Windows.Forms.SystemInformation]::HorizontalResizeBorderThickness + [Windows.Forms.SystemInformation]::Border3DSize.Height;
$innerOffsets.Bottom = -([Windows.Forms.SystemInformation]::HorizontalResizeBorderThickness + [Windows.Forms.SystemInformation]::Border3DSize.Height);
$inneroffsets.Left = [Windows.Forms.SystemInformation]::VerticalResizeBorderThickness + [Windows.Forms.SystemInformation]::Border3DSize.Width;
$inneroffsets.Right = -([Windows.Forms.SystemInformation]::VerticalResizeBorderThickness + [Windows.Forms.SystemInformation]::Border3DSize.Width);
if([console]::bufferheight -gt [console]::windowheight)
{
$inneroffsets.right -= [Windows.Forms.SystemInformation]::HorizontalScrollBarThumbWidth;
}
if([console]::bufferwidth -gt [console]::windowwidth)
{
$inneroffsets.bottom -= [Windows.Forms.SystemInformation]::VerticalScrollBarThumbHeight;
}
This gives us the offsets we need to get the inner bounds of the window. From here we can get our relative location on the window.
$mp = [Windows.Forms.Cursor]::Position;
$mp.x -= ($win.bounds.left + $inneroffsets.left);
$mp.y -= ($win.bounds.top + $inneroffsets.top);
Now we can convert our mouse position to buffer cell position.
$innerWidth = ($win.bounds.right + $inneroffsets.right) - ($win.bounds.left + $inneroffsets.left);
$innerheight = ($win.bounds.bottom + $inneroffsets.bottom) - ($win.bounds.top + $inneroffsets.top);
$mp.x = [Math]::Floor($mp.x / ( $innerwidth / [console]::windowWidth));
$mp.y = [Math]::Floor($mp.y / ( $innerheight / [console]::windowheight));
You'll have to use $win.update() every time you check. You can also use $win.isforeground and [Windows.Forms.UserControl]::MouseButtons -match "Left" to only check when the console window is clicked as well.
Not sure how all of it is going to piece together.
Maybe this will help.
<# Gets the Mouse Position #>
[System.Windows.Forms.Cursor]::Position

Modify WebcamTexture from unity native plugin

I just want to pass a WebcamTexture to a native plugin (using GetNativeTexturePtr() for performance), draw something on it and render results into a plane on the screen.
I guess that some Unity thread would be updating the WebcamTexture contents every frame so writing to that texture may produce some flickering effects as there would be 2 threads updating its contents. Because of that I'm using another texture "drawTexture" to render the result.
The algorithm is easy, on every render event:
Read camTexture
Copy camTexture contents to drawTexture
Draw things on drawTexture
Render drawTexture (OpenGL bindTexture)
I'm following the NativeRenderingPlugin example, but every time I try to copy contents from camTexture to drawTexture (even only 1 pixel) the main thread freezes.
I think that may be happening because I'm trying to read camTexture while it is being modified by an external thread.
Here it is some code:
C# plugin source
[DllImport ("plugin")]
private static extern void setTexture (IntPtr cam, IntPtr draw, int width, int height);
[DllImport ("plugin")]
private static extern IntPtr GetRenderEventFunc();
WebCamTexture webcamTexture;
Texture2D drawTexture;
GameObject plane;
bool initialized = false;
IEnumerator Start () {
// PlaneTexture
// Create a texture
drawTexture = new Texture2D(1280,720,TextureFormat.BGRA32,true);
drawTexture.filterMode = FilterMode.Point;
drawTexture.Apply ();
plane = GameObject.Find("Plane");
plane.GetComponent<MeshRenderer> ().material.mainTexture = drawTexture;
// Webcam texture
webcamTexture = new WebCamTexture();
webcamTexture.filterMode = FilterMode.Point;
webcamTexture.Play();
yield return StartCoroutine("CallPluginAtEndOfFrames");
}
private IEnumerator CallPluginAtEndOfFrames()
{
while (true) {
// Wait until all frame rendering is done
yield return new WaitForEndOfFrame();
// wait for webcam and initialize
if (!initialized && webcamTexture.width > 16) {
setTexture (webcamTexture.GetNativeTexturePtr(), drawTexture.GetNativeTexturePtr(), webcamTexture.width, webcamTexture.height);
initialized = true;
} else if (initialized) {
// Issue a plugin event with arbitrary integer identifier.
// The plugin can distinguish between different
// things it needs to do based on this ID.
// For our simple plugin, it does not matter which ID we pass here.
GL.IssuePluginEvent (GetRenderEventFunc (), 1);
}
}
}
C++ plugin (draw texture method)
static void ModifyTexturePixels()
{
void* textureHandle = drawTexturePointer;
int width = textureWidth;
int height = textureHeight;
if (!textureHandle)
return;
int textureRowPitch;
void* newEmptyTexturePointer = (unsigned char*)s_CurrentAPI->BeginModifyTexture(textureHandle, width, height, &textureRowPitch);
memcpy(newEmptyTexturePointer, webcamTexturePointer, width * height * 4);
s_CurrentAPI->EndModifyTexture(textureHandle, width, height, textureRowPitch, newEmptyTexturePointer);
}
Any help would be appreciated.

Is there a command in Powershell to flash window?

I have a powershell script that takes a long time to run and I often don't know when it's finished unless I check.
Is there a command that I can stick to the end of my powershell script that can notify me when the scripts are run, kind of like the Communicator chat window turns yellow on the menu bar when someone messages me notifying me to go check?
You can import FlashWindowEx from user32.dll as shown in this article http://learn-powershell.net/2013/08/26/make-a-window-flash-in-taskbar-using-powershell-and-pinvoke/
Add-Type -TypeDefinition #"
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Runtime.InteropServices;
public class Window
{
[StructLayout(LayoutKind.Sequential)]
public struct FLASHWINFO
{
public UInt32 cbSize;
public IntPtr hwnd;
public UInt32 dwFlags;
public UInt32 uCount;
public UInt32 dwTimeout;
}
//Stop flashing. The system restores the window to its original state.
const UInt32 FLASHW_STOP = 0;
//Flash the window caption.
const UInt32 FLASHW_CAPTION = 1;
//Flash the taskbar button.
const UInt32 FLASHW_TRAY = 2;
//Flash both the window caption and taskbar button.
//This is equivalent to setting the FLASHW_CAPTION | FLASHW_TRAY flags.
const UInt32 FLASHW_ALL = 3;
//Flash continuously, until the FLASHW_STOP flag is set.
const UInt32 FLASHW_TIMER = 4;
//Flash continuously until the window comes to the foreground.
const UInt32 FLASHW_TIMERNOFG = 12;
[DllImport("user32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool FlashWindowEx(ref FLASHWINFO pwfi);
public static bool FlashWindow(IntPtr handle, UInt32 timeout, UInt32 count)
{
IntPtr hWnd = handle;
FLASHWINFO fInfo = new FLASHWINFO();
fInfo.cbSize = Convert.ToUInt32(Marshal.SizeOf(fInfo));
fInfo.hwnd = hWnd;
fInfo.dwFlags = FLASHW_ALL | FLASHW_TIMERNOFG;
fInfo.uCount = count;
fInfo.dwTimeout = timeout;
return FlashWindowEx(ref fInfo);
}
}
"#