I'm trying to use a cloud TTS within my Unity game.
With the newer versions (I am using 2019.1), they have deprecated WWW in favour of UnityWebRequest(s) within the API.
I have tried the Unity Documentation, but that didn't work for me.
I have also tried other threads and they use WWW which is deprecated for my Unity version.
void Start()
{
StartCoroutine(PlayTTS());
}
IEnumerator PlayTTS()
{
using (UnityWebRequestMultimedia wr = new UnityWebRequestMultimedia.GetAudioClip(
"https://example.com/tts?text=Sample%20Text&voice=Male",
AudioType.OGGVORBIS)
)
{
yield return wr.Send();
if (wr.isNetworkError)
{
Debug.LogWarning(wr.error);
}
else
{
//AudioClip ttsClip = DownloadHandlerAudioClip.GetContent(wr);
}
}
}
The URL in a browser (I used Firefox) successfully loaded the audio clip allowing me to play it.
What I want it to do is play the TTS when something happens in the game, it has been done within the "void Start" for testing purposes.
Where am I going wrong?
Thanks in advance
Josh
UnityWebRequestMultimedia.GetAudioClip automatically adds a default DownloadHandlerAudioClip which has a property streamAudio.
Set this to true and add a check for UnityWebRequest.downloadedBytes in order to delay the playback before starting.
Something like
public AudioSource source;
IEnumerator PlayTTS()
{
using (var wr = new UnityWebRequestMultimedia.GetAudioClip(
"https://example.com/tts?text=Sample%20Text&voice=Male",
AudioType.OGGVORBIS)
)
{
((DownloadHandlerAudioClip)wr.downloadHandler).streamAudio = true;
wr.Send();
while(!wr.isNetworkError && wr.downloadedBytes <= someThreshold)
{
yield return null;
}
if (wr.isNetworkError)
{
Debug.LogWarning(wr.error);
}
else
{
// Here I'm not sure if you would use
source.PlayOneShot(DownloadHandlerAudioClip.GetContent(wr));
// or rather
source.PlayOneShot((DownloadHandlerAudioClip)wr.downloadHandler).audioClip);
}
}
}
Typed on smartphone so no warranty but I hope you get the idea
Related
Working on a Unity hybrid VR (cardboard) /2D app. The cardboard side of it works fine. I am having trouble with the 2D/VR switching.
When I am in 2D mode, reticle does not move, although screen taps register. So the app seems unaware of the gyro.
I feel like I am missing something fundamental here. I have a GvrEventSystem prefab that has both an EventSystem and GvrPointerInputModule components.
What obvious thing am I over-looking?
ETA:
I have been asked to add relevant code. Here is the code for 2D-VR switching on-the-fly. This code executes w/out error, and the app switches between VR and 2D mode every 3 seconds:
readonly string NONE_STRING = "";
readonly string CARDBOARD_STRING = "cardboard";
void Start()
{
Invoke("GoPhone", 3.0f);
}
void GoPhone()
{
SetVREnabled(false);
Invoke("GoVR", 3.0f);
}
void GoVR()
{
SetVREnabled(true);
Invoke("GoPhone", 3.0f);
}
void SetVREnabled(bool isEnabled)
{
if (isEnabled)
{
StartCoroutine(LoadDevice(CARDBOARD_STRING));
}
else
{
StartCoroutine(LoadDevice(NONE_STRING));
}
}
IEnumerator LoadDevice(string newDevice)
{
if (String.Compare(XRSettings.loadedDeviceName, newDevice, true) != 0)
{
XRSettings.LoadDeviceByName(newDevice);
yield return null;
if (!XRSettings.loadedDeviceName.Equals(NONE_STRING))
XRSettings.enabled = true;
}
}
Although I feel like my problem is a configuration problem, and not a code problem. In the editor, which does not support VR mode, the app behaves in 2D mode as expected.
Also ETA:
JIC
User error! I did not follow the "Magic Window" instructions as detailed at https://developers.google.com/vr/develop/unity/guides/magic-window... let my folly be a warning to future generations!
I have 2 issues when using PhotoCapture on the Hololens 2, which are probably connected. They did not occur on the Hololens 1.
The only obtainable resolution is 3904x2196
Getting the CameraToWorldMatrix always fails
As can be seen in the docs, this resolution has no framerate associated with it.
My presumption is that the CameraToWorldMatrix is only obtainable for camera profiles with a lower resolution.
How can i change the resolution and get the matrix within Unity?
Minimal Reproducible Example
I am using Unity 2019.2.19f1 and Visual Studio 2019 Community (16.4.5)
Create a new Unity Project following the steps here, except using IL2CPP as the scripting backend instead of .NET
In Player Settings: Capabilities enable InternetClient, InternetClientServer, PrivateNetworkClientServer, Webcam, Microphone and SpatialPerception, in Player Settings: Supported Device Families select Holographic
Create a new empty gameobject and add the following script:
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using UnityEngine;
using UnityEngine.Windows.WebCam;
public class PhotoCaptureController : MonoBehaviour
{
PhotoCapture photoCaptureObject = null;
bool isRunning = false;
void Start()
{
StartCoroutine(StartCameraCapture());
}
private IEnumerator StartCameraCapture()
{
if (!Application.HasUserAuthorization(UserAuthorization.WebCam))
{
yield return Application.RequestUserAuthorization(UserAuthorization.WebCam);
}
if (Application.HasUserAuthorization(UserAuthorization.WebCam))
{
Debug.Log("Creating PhotoCapture");
PhotoCapture.CreateAsync(false, OnPhotoCaptureCreated);
}
else
{
Debug.Log("Webcam Permission not granted");
}
}
private void Update()
{
if (isRunning)
{
photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
}
}
void OnPhotoCaptureCreated(PhotoCapture captureObject)
{
photoCaptureObject = captureObject;
IEnumerable<Resolution> availableResolutions = PhotoCapture.SupportedResolutions;
foreach (var res in availableResolutions)
{
Debug.Log("PhotoCapture Resolution: " + res.width + "x" + res.height);
}
Resolution cameraResolution = availableResolutions.OrderByDescending((res) => res.width * res.height).First();
CameraParameters c = new CameraParameters();
c.hologramOpacity = 0.0f;
c.cameraResolutionWidth = cameraResolution.width;
c.cameraResolutionHeight = cameraResolution.height;
c.pixelFormat = CapturePixelFormat.BGRA32;
captureObject.StartPhotoModeAsync(c, OnPhotoModeStarted);
}
private void OnPhotoModeStarted(PhotoCapture.PhotoCaptureResult result)
{
if (result.success)
{
isRunning = true;
photoCaptureObject.TakePhotoAsync(OnCapturedPhotoToMemory);
}
}
void OnCapturedPhotoToMemory(PhotoCapture.PhotoCaptureResult result, PhotoCaptureFrame frame)
{
if (result.success)
{
if (frame.TryGetCameraToWorldMatrix(out Matrix4x4 cameraToWorldMatrix))
{
Debug.Log("Successfully obtained CameraToWorldMatrix: " + cameraToWorldMatrix.ToString());
}
else
{
Debug.Log("Failed to obtain CameraToWorldMatrix");
}
}
frame.Dispose();
}
}
Change Build Settings:
Build, open VS solution, set build target to ARM64, Debug and deploy to Device
This CameraCapture plugin may work for you. It's a native plugin for Unity. The readme.txt contains information on building it. Basically, you need to build the native plugin variants before trying to open up the Unity sample. If you haven’t built C++/WinRT components, you may need to go through the setup steps here. The SpatialCameraTracker.cs file shows how to receive the camera matrices.
There is minimal support on this but all the code is in the repo
I'm working on a game in Unity, using Firebase for user and game data.
Everything is fine saving the game to Unity, but in Multiplayer I have the challenge to pair an open game to a unique player 2.
The flow is:
1. You look for a game with a state of "open"
2. If no game is found within X seconds, a new one is created with the state of "open"
I have a query that returns open games (those with state "open")
This query is added a Firebase eventhandler
public void OnButtonLookForOpenMPGames()
{
this.OpenFirebaseGamesAsHost = this.LiveGamereference.OrderByChild("State").EqualTo(WYMSettings.MP_GAME_STATE_OPEN_TO_PLAYER2)
.LimitToFirst(3);
this.OpenFirebaseGamesAsHost.ChildAdded += this.OnOpenMPGameFound;
}
This fires fine in the handler:
private void OnOpenMPGameFound(object sender, ChildChangedEventArgs args)
{
Debug.Log("Looking for open games");
if (args.DatabaseError != null)
{
// handle errors
}
else
{
Debug.Log("Open game found (it may be my own)");
// remove event listener because we got a hit
// it will fire as many times as it's true!
this.OpenFirebaseGamesAsHost.ChildAdded -= this.OnOpenMPGameFound;
if (args.Snapshot.Child("HostId").Value.ToString() != FirebaseAuth.DefaultInstance.CurrentUser.UserId)
{
Debug.Log("Other game than mine found open");
// this.AddDelayedUpdateAction(() =>
// {
// this.MPGameLockInTransaction(args.Snapshot.Reference);
// });
this.MPGameLockInTransaction(args.Snapshot.Reference);
}
}
}
The problem: The handler passes this on to the transaction function, but it fails with an inner exception (Internal Task Faulted) - I've tried many different variations, but can't figure this one out from the official Firebase example.
Problem code:
What I want to achieve is to lock only that game to those two players, the host and the players 2 in this example querying for open games. The State is an int, but here a string for clarity.
private void MPGameLockInTransaction(DatabaseReference mpreference)
{
mpreference.RunTransaction(mutableData =>
{
MultiPlayerGame transactionMPG = mutableData.Value as MultiPlayerGame;
if (transactionMPG == null)
{
return TransactionResult.Abort();
}
if (transactionMPG.State != "open")
{
// game is taken, abort
Debug.Log("transaction aborted");
return TransactionResult.Abort();
}
transactionMPG.State = "game started";
mutableData.Value = transactionMPG;
return TransactionResult.Success(mutableData);
}).ContinueWith(task =>
{
if (task.Exception != null)
{
Debug.Log("Transactionlock to game failed" + task.Exception.ToString());
// Look over again
// not implemented yet
}
else
{
Debug.Log("starting game immediately");
this.AddDelayedUpdateAction(() => this.StartMPGameImmediatelyFromSearch());
}
});
}
This is just a guess but it might be related to threading.
In newer .Net version ContinueWith might end on a thread where most of the Unity API may not be called (including Debug.Log).
Instead you could try and use ContinueWithOnMainThread from the extensions instead which was created exactly for this reason.
With the .NET 4.x runtime, continuations may often execute in the background, whilst this extension method pushes them onto the main thread.
The .NET 3.x runtime didn't have this issue as continuations were contained within the Parse library, which did this implicitly.
I am trying to make a unityWebRequest to an API and everything runs smoothly on a normal unity app, but when I transfer the application to my oculus quest as soon as the method "checkCode()" is called triggering the GetAuthRequest() method that does the unityWebRequest the oculus application crashes.
I've tried rewriting the code in a couple of different ways and nothing seems to be fixing the issue. My Quest has internet access and I believe I have Android SDK installed.
public void CheckCode()
{
StartCoroutine(GetAuthRequest("http://myURL/", attemptedCode));
}
IEnumerator GetAuthRequest(string uri, string attemptedCode)
{
setValue("IN_REQ");
using (UnityWebRequest webRequest = UnityWebRequest.Get(uri + attemptedCode))
{
//// Request and wait for the desired page.
yield return webRequest.SendWebRequest();
if (webRequest.isNetworkError)
{
setValue("NET_ERROR");
}
else
{
if (webRequest.downloadHandler.text.Equals("true"))
{
ClearCode();
setValue("Authenticated!");
}
else
{
ClearCode();
setValue("Error...");
}
}
}
}
try using "https://myURL/" instead of "http://myURL/"
Question asked just to give solution :)
Main idea is to recognize QR-code on Unity without ANY additional action like tapping on the screen or sth like this.
(For me it's not necessary that "vuforia free" have watermark, so here is my solution)
(Also Vuforia works with camera much faster and no need to realize manually autofocus)
Continious QR code recognition using Vuforia as webcam source and ZXing Library as QR recognizer
using UnityEngine;
using Vuforia;
using ZXing;
public class QRCodeReader : MonoBehaviour {
private bool _isFrameFormatSet;
IBarcodeReader _barcodeReader = new BarcodeReader();
void Start () {
InvokeRepeating("Autofocus", 2f, 2f);
}
void Autofocus () {
CameraDevice.Instance.SetFocusMode(CameraDevice.FocusMode.FOCUS_MODE_TRIGGERAUTO);
RegognizeQR();
}
private Vuforia.Image GetCurrFrame()
{
return CameraDevice.Instance.GetCameraImage(Vuforia.Image.PIXEL_FORMAT.GRAYSCALE);
}
void RegognizeQR()
{
if (!_isFrameFormatSet == _isFrameFormatSet)
{
_isFrameFormatSet = CameraDevice.Instance.SetFrameFormat(Vuforia.Image.PIXEL_FORMAT.GRAYSCALE, true);
}
var currFrame = GetCurrFrame();
if (currFrame == null)
{
Debug.Log("Camera image capture failure;");
}
else
{
var imgSource = new RGBLuminanceSource(currFrame.Pixels, currFrame.BufferWidth, currFrame.BufferHeight, true);
var result = _barcodeReader.Decode(imgSource);
if (result != null)
{
Debug.Log("RECOGNIZED: " + result.Text);
}
}
}
}
Its possible to realize also without Vuforia, ofc. Unity provides the possibility to get a camera and show it’s input on a webcamtexture. It's possible to find more documentation here.
ZXing lib you can find here, or build it by your own hands using sourse code located on github.
Both libs is cross-platform, so must be no issues on different devices.