Unity and VR: offset using "ReadPixels()" function - unity3d

I've ran into a small issue developing a picture taker for a VR project. I need to take a screenshot of a specific zone, which is a rectangle with variable width and height. To do that, I have a transform anchored to the top right corner of the bounding box that represents where the picture is going to be taken, and one anchored to the lower right corner.
Here's what it should look like. I've added little red circles to show the transforms's position.
Here's what a screencap using the left eye looks like. It's the same result if I use "both eyes" as a target in the Camera settings.
Here's what a screencap using the right eye looks like. So not only is it too far left or right, it's also a tad too high.
Here's the code that creates the Rect, and here's the code that reads the pixels.
When the Main Camera targets the left eye, there's almost half of the Rect's with as an offset to the left, when it targets the right eye, there's that same offset to the right, when it targets both, there's a softer offset to the left, and all of these have a slight vertical offset upwards.
Any help is appreciated. I'll keep this thread updated if I find anything!
public void SubmitPicture()
{
Vector2 upperLeftPosition = mainCamera.WorldToScreenPoint(upperLeftTransform.position);
Vector2 lowerRightPosition = mainCamera.WorldToScreenPoint(lowerRightTransform.position);
pictureBoxRect.x = upperLeftPosition.x;
pictureBoxRect.y = mainCamera.scaledPixelHeight - upperLeftPosition.y;
pictureBoxRect.width = lowerRightPosition.x - upperLeftPosition.x;
pictureBoxRect.height = lowerRightPosition.y - upperLeftPosition.y;
pictureSnapper.OnInput(AbsoluteRect(pictureBoxRect));
}
public void OnInput(Rect pictureBox)
{
if ((int)pictureBox.width > 0 && (int)pictureBox.height > 0)
{
videoPlayer.Stop();
Texture2D videoTexture = new Texture2D((int)pictureBox.width, (int)pictureBox.height);
videoTexture.ReadPixels(pictureBox, 0, 0);
videoTexture.Apply();
byte[] imageData = videoTexture.GetRawTextureData();
if (debug)
{
byte[] imagePng = videoTexture.EncodeToPNG();
File.WriteAllBytes(Application.dataPath + "/" + savename + ".png", imagePng);
}
}
}
private Rect AbsoluteRect(Rect rect)
{
if (rect.width < 0)
{
rect.x -= rect.width;
rect.width = Mathf.Abs(rect.width);
}
if (rect.height < 0)
{
rect.y += rect.height / 2;
rect.height = Mathf.Abs(rect.height);
}
return rect;
}
Updated to add the picture references.

Related

How do i fix the placement tool in my level editor?

So i'm currently making a game, and i've recently added a level editor, but the placing tool does not work how i wanted it to.
https://youtu.be/MuUvnVTL6eg
If you've watched this video, you've probably realized that the block placing works pretty much how placing rectangles in ms pain with alt does, and i want it to work like placing rectangles in ms pain without alt xd.
I'm using this code to place the block:
if (Input.GetKeyDown(KeyCode.Mouse0)){
startDrawPos = Camera.main.ScreenToWorldPoint(Input.mousePosition);
tmpObj = spawnObject(blocks[selected].gameObject, startDrawPos);
drawing = true;
}
if (Input.GetKey(KeyCode.Mouse0)){
Vector2 mPos = Camera.main.ScreenToWorldPoint(Input.mousePosition);
Vector2 tmpScale = new Vector2(startDrawPos.x - mPos.x, startDrawPos.y - mPos.y);
tmpObj.transform.localScale = tmpScale;
}
if (Input.GetKeyUp(KeyCode.Mouse0))
{
drawing = false;
var scale = tmpObj.transform.localScale;
//Code below destroys the object if it's too small to avoid accidental placements
if (scale.x <= 0.1 && scale.x > -0.1 || scale.y <= 0.1 && scale.y > -0.1)
{
Destroy(tmpObj);
}
}
(All of this code is in the Update() function)
(spawnObject function just instantiates the object prefab)
There is a bit more code but it has nothing to do with the position of the block, it just detect which block is selected and decides if it can be resized or not.
I solved this problem. But because your complete script is not in question, I rebuilt the code with IEnumerator, Here, by pressing the left mouse button, IEnumerator is activated and all commands are grouped in one method to make the code more efficient.
private void Update()
{
if (Input.GetKeyDown(KeyCode.Mouse0)) StartCoroutine(DrawRect());
}
How does the Desktop Rect formula work?
By running IEnumerator, the code first records the starting point of the mouse. It also makes a simple cube because I do not have access to your objects. Now until the mouse is pressed. Resize Rect to the difference between current and recorded points. The only thing is that to avoid ALT control, you have to place it between the current and initial points. The reason for adding the camera forward is to be seen in the camera.
cubeObject.transform.position = (startDrawPos + currentDrawPos) / 2;
The final structure of the DrawRect is as follows:
public IEnumerator DrawRect()
{
drawing = true;
var scale = Vector2.zero;
var startDrawPos = Camera.main.ScreenToWorldPoint(Input.mousePosition);
var cubeObject = GameObject.CreatePrimitive(PrimitiveType.Cube);
while (Input.GetKey(KeyCode.Mouse0))
{
var currentDrawPos = Camera.main.ScreenToWorldPoint(Input.mousePosition);
cubeObject.transform.position = (startDrawPos + currentDrawPos) / 2 + Camera.main.transform.forward * 10;
scale = new Vector2(startDrawPos.x - currentDrawPos.x, startDrawPos.y - currentDrawPos.y);
cubeObject.transform.localScale = scale;
yield return new WaitForEndOfFrame();
}
if (scale.x <= 0.1 && scale.x > -0.1 || scale.y <= 0.1 && scale.y > -0.1) Destroy(cubeObject);
drawing = false;
}

Angular9 Signature Pad issue: Signature drawn offset from pen touch

I implemented a signature pad using https://github.com/ramsatt/Angular9SignaturePad/tree/master/src/app/_componets/signature-pad and it works fine on smaller devices but on iPad or bigger devices like 7" upwards, it doesn't work properly.
When drawing on the screen, the resulting line has an offset from where the user touched (Signature drawn doesn't appear directly under the pen as user draws).
please how can I fix this.
So I fixed it by adding the below code and calling it in ngOnInit
resizeCanvas() {
var width = this.signaturePadElement.nativeElement.width;
var height = this.signaturePadElement.nativeElement.height;
var ratio = Math.max(window.devicePixelRatio || 1, 1);
if (ratio <= 2) {
this.signaturePadElement.nativeElement.width = width * ratio;
this.signaturePadElement.nativeElement.height = height * ratio;
this.signaturePadElement.nativeElement
.getContext("2d")
.scale(ratio, ratio);
}
then do
ngOnInit(){
this.resizeCanvas()
}
this.signaturePadElement is your Element gotten using ViewChild()

Direction Vector around Mesh's Surface

How do I best describe the direction vector to move an object left (or right/up/down) from its current position, factoring the current view perspective into the equation?
For example, imagine the box depicted in the following picture, placed at origin (0,0,0). If I wanted to move the point on top of the box to the left, I'd have to make a step in (-1,0,0) direction (i.e. currentPos = currentPos + Vector3.left).
If I looked at the box from behind, I'd have to make a step in (1,0,0) direction to continue moving in the same direction.
i.e. When I'm looking at this box, placed at origin, and press an input button LEFT, I need for the point to move left and continue moving in that direction for as long as I keep pressing the button. It ultimately wraps around the surface, i.e. if I keep pressing just LEFT the point will wrap around the cube and re-appear so to speak.
To indefinitly trace a decal around a cube:
private Transform Cube;
private Transform Decal;
void Update() {
Vector3 moveLocal = Vector3.zero;
// Set moveLocal from User Input
moveLocal += Vector3.left * Input.GetAxis("Left"); // etc
if (moveLocal.sqrMagnitude > 0) {
var decalDirection = (Decal.position - Cube.position).normalized;
var angle = Vector3.SignedAngle(Cube.forward, decalDirection, Cube.up);
// Align decal to correct cube face
if (angle < -135 || angle > 135) { // Rear face
Decal.forward = -Cube.forward;
}
else if (angle < -45) { // Left face
Decal.forward = -Cube.right;
}
else if (angle > 45) { // Right face
Decal.forward = Cube.right;
}
else { // Front Face
Decal.forward = Cube.forward;
}
// Now move decal in it's local space:
Decal.Translate(moveLocal, Space.Self);
}
}
Admittedly, this only rotates Left/Right, but I hope you get the idea =)
As i mentioned you in my opinion you can achieve this with surface normals. I cracked some code which is not an exact solution because i used collision normals and on corners that gives different results. In any case i share it so that it gives you the idea. You can go left and right with this code relative to camera(even though it does not use anything related to camera) because it depends on surface normals.
private Vector3 normal;
private Vector3 contact;
private Vector3 direction;
private GameObject obj;
void Start () {
}
void Update () {
if (obj == null)
return;
if( Input.GetKey(KeyCode.LeftArrow))
{
obj.transform.position += direction * Time.deltaTime;
}
if (Input.GetKey(KeyCode.RightArrow))
{
obj.transform.position -= direction * Time.deltaTime;
}
}
void OnCollisionEnter(Collision collision)
{
Debug.Log("Collided");
Debug.Log("If works");
obj = collision.gameObject;
normal = collision.contacts[0].normal;
contact = collision.contacts[0].point;
//Cross product of normal and up gives direction Right.
//This up can be cube's up as well in cases it is tilted
direction = (-Vector3.Cross(Vector3.up, normal));
}
You can also see how it works from images below:

Pixel perfect 2d camera in unity

I have a very simple script that updates my orthographic camera on a given resolution so that it accurately scales the view to be pixel perfect.
Here is some relevant code:
OrthographicSetting get_override(int size)
{
return Overrides.FirstOrDefault(x => x.OrthographicSize == size);
}
void update_ortho()
{
m_last_size = Screen.height;
float ref_size = (OrthographicSize / PixelsPerUnit) * 0.5f;
OrthographicSetting or = get_override(m_last_size);
float ppu = or != null ? or.PixelsPerUnit : PixelsPerUnit;
float ortho_size = (m_last_size / ppu) * 0.5f;
float multiplier = Mathf.Max(1, Mathf.Round(ortho_size / ref_size));
ortho_size /= multiplier;
this.GetComponent<Camera>().orthographicSize = ortho_size;
Debug.Log(m_last_size + " " + ortho_size + " " + multiplier + " " + ppu);
}
[System.Serializable]
public class OrthographicSetting
{
public int OrthographicSize;
public float PixelsPerUnit;
}
OrthographicSetting get_override(int size)
{
return Overrides.FirstOrDefault(x => x.OrthographicSize == size);
}
With this, i can specify a set of overrides for every resolution.
My current setup is using 100 pixels per unity unit. All of my sprites use point filtering with no compression. Yet i still get strange results. 90% of the sprites render fine, but some seem to be rendering incorrectly.
Here's a screenshot to illustrate:
I may have solved the problem. I was using a sprite shader with pixel snap turned on. if i turn off pixel snap the problem seems to more or less go away. I still get the occasional problem with game objects that aren't in "nice" positions, i dunno how to avoid that problem though...

cocos2dx: Sprite3D rotating, culling error

Hi I'm trying to have 2 sprites with different z in 3d world and a camera that rotates around the center of the screen and points at the center of the screen.
Even if the sprites has different z (and zorder, I don't know if this is necessary) the sprites are always visualized while I'm expecting to have the second sprite hided from the other...
This is helloworld layer init
auto sp3d = Sprite3D::create();
sp3d->setPosition(visibleSize.width/2, visibleSize.height/2);
addChild(sp3d);
auto sprite = Sprite::create("JP9_table.png");
auto spritePos = Vec3(0,0,0);
sprite->setScale(0.3);
sprite->setPosition3D(spritePos);
sp3d->addChild(sprite,0);
auto sprite2 = Sprite::create("JP9_logo_yc.png");
auto spritePos2 = Vec3(0,0,10);
sprite2->setPosition3D(spritePos2);
sp3d->addChild(sprite2,10);
sp3d->setCullFace(GL_BACK);
sp3d->setCullFaceEnabled(true);
this->setCameraMask((unsigned short)CameraFlag::USER2, true);
camera = Camera::createPerspective(60, (float)visibleSize.width/visibleSize.height, 1.0, 1000);
camera->setCameraFlag(CameraFlag::USER2);
camera->setPosition3D(spritePos + Vec3(-200,0,800));
camera->lookAt(spritePos, Vec3(0.0,1.0,0.0));
this->addChild(camera);
this->scheduleUpdate();
angle=0;
and this is update:
void TestScene::update(float dt)
{
angle+=0.1;
Size visibleSize = Director::getInstance()->getVisibleSize();
Vec2 origin = Director::getInstance()->getVisibleOrigin();
Vec3 spritePos=Vec3(visibleSize.width/2,visibleSize.height/2,0);
camera->setPosition3D(Vec3(visibleSize.width/2,visibleSize.height/2,0) + Vec3(800*cos(angle),0,800*sin(angle)));
camera->lookAt(spritePos, Vec3(0.0,1.0,0.0));
}
I have tryed something simplier:
auto sp3d = Sprite3D::create();
sp3d->setPosition(visibleSize.width/2, visibleSize.height/2);
addChild(sp3d);
auto sprite = Sprite::create("JP9_table.png");
auto spritePos = Vec3(0,0,0);
sprite->setScale(0.3);
sprite->setPosition3D(spritePos);
sp3d->addChild(sprite,0);
auto sprite2 = Sprite::create("JP9_logo_yc.png");
auto spritePos2 = Vec3(0,0,10);
sprite2->setPosition3D(spritePos2);
sp3d->addChild(sprite2,10);
sp3d->setCullFace(GL_BACK);
sp3d->setCullFaceEnabled(true);
even with sp3d->runAction(RotateTo::create(20,vec3(0,3000,0))) same error.
Is it a cocos2dx bug?
the sprite with z=10 disappear before it is covered by the other sprite...
remain hidden for a while, and when it should be hidden completely reappear!!!
Do I have forgot something?
thanks
Maybe you should check this.
_camControlNode = Node::create();
_camControlNode->setNormalizedPosition(Vec2(.5,.5));
addChild(_camControlNode);
_camNode = Node::create();
_camNode->setPositionZ(Camera::getDefaultCamera()->getPosition3D().z);
_camControlNode->addChild(_camNode);
auto sp3d = Sprite3D::create();
sp3d->setPosition(s.width/2, s.height/2);
addChild(sp3d);
auto lship = Label::create();
lship->setString("Ship");
lship->setPosition(0, 20);
sp3d->addChild(lship);
and
_lis->onTouchMoved = [this](Touch* t, Event* e) {
float dx = t->getDelta().x;
Vec3 rot = _camControlNode->getRotation3D();
rot.y += dx;
_camControlNode->setRotation3D(rot);
Vec3 worldPos;
_camNode->getNodeToWorldTransform().getTranslation(&worldPos);
Camera::getDefaultCamera()->setPosition3D(worldPos);
Camera::getDefaultCamera()->lookAt(_camControlNode->getPosition3D());
};