I am trying to implement voice commands into my unity project to eventually be used in the HoloLens. At the moment, I am simply trying to make a cube change colors using the speech input handler script and speech input source script. I have the handler script but I can't find the source script anywhere. How do I obtain the source script? Why do I not have it? I am using Unity 2018.4.12f1 and I am using the Mixed Reality Toolkit. If you need additional info to help me please ask!
In versions after MRTK2, SpeechInputSource is no longer needed. Instead, keyword service (e.g., Windows Speech Input Manager) must be added to the input system's data providers. Please check out the SpeechInputExample scene to understand how to use speech input.
The guide you are reading may be outdated, please read the official documentation to learn how to use Speech function in the latest version of MRTK.
Related
I am using unity + MRTK to develop an application for HoloLens 2. I am trying to use "speech styles" for MRTK TextToSpeech.SpeakSsml method (MRTK API Reference). Text to speech works; however, I am unable to employ speech styles.
Example ssml:
<speak version=""1.0"" xmlns=""http://www.w3.org/2001/10/synthesis"" xmlns:mstts=""https://www.w3.org/2001/mstts"" xml:lang=""en-US"">
<mstts:express-as style=""cheerful"">
Cheerful hello!
</mstts:express-as>
<break time=""1s"" />
<mstts:express-as style=""angry"">
Angry goodbye!
</mstts:express-as>
</speak>
My guess is that the default voice does not support speech styles. But, if I add a voice element to use another voice (there are four available voices listed in the documentation), TextToSpeech won't work at all. So, I am facing two problems:
When using the SpeakSsml method instead of StartSpeaking, the selected voice (TextToSpeech.Voice) is disregarded and I am unable to change it using the voice element.
I couldn't find documentation for supported SSML elements for available voices in MRTK TextToSpeech Class.
Any ideas or useful links?
Thank you!
The TextToSpeech provided by MRTK depends on Windows 10 SpeechSynthesizer class, so it works offline and does not support adjust speaking styles. And the mstts:express-as element is only available in the Azure Speech Service, for more information please refer to this documentation: Improve synthesis with Speech Synthesis Markup Language (SSML)
Using Apache Beam Python SDK version 2.9.0, is it possible to get a renderable pipeline graph representation similar to Google’s dataflow instead of running it?
I have difficulties to assemble complex pipelines and I would be happy to see an assembled pipeline before trying to execute it using DirectRunner.
Have a look at this unit test. This should give you an example of how this works with Python SDK. TextRenderer simply returns the dot representation in text format.
There is also an explanation for how to do it and how to handle DOT files, but it is written for Java SDK (link). However, it should give you an idea how this can implemented, and the part about DOT files there is language agnostic.
You might also want to have a look at PyDot package to see how to handle the output programmatically in Python.
Helo Guys,
I've ran into a very deep trouble within a very long time of googling havent found answers. so here it is.
i'm trying to make Augmented Reality without any commercial plugins i have to pay for (vuforia) in unity3d, this program i will send to android.
i've messing around with vuforia, kudan, and other, but i want more power with my apps i can control myself.
i want to make nft marker based augmented reality, i've tried artoolkit, but the explanation and samples only get into generating .fset .iset .fset3 file but it stopped there, no info about generating the .patt file needed nor info about where to put those dataset files.
i've tried using openspace3d, but also only generating fset iset file, no .patt file.
my question is : how to generate .patt file, and if i have the dataset file (iset fset fset3) how do i proceed on creating my own custom nft marker augmented reality in unity3d?
I'm afraid you have a slight misconception of marker files for Artoolkit.
Artoolkit knows two types of markers. Patterns and NFT markers.
The pattern (.patt) files are for the so-called framemarkers.
The fset(.iset,.fset,.fset3) files are for the NFT markers. You don't need a pattern file for NFT markers.
For a comprehensive list with examples of AR-Marker you can look here.
So in your case, you only need the fset files.
Training:
For training, you can use the description and the tools from this site and this site. But be warned. Finding good values is a bit of a trial and error.
Usage:
If you haven't already done so, download the Unity package from this page. If you're using windows you can also download the windows tool on the same page.
That package comes with several example scenes. Including TwoNFTMarker. ( under Assets/Artoolkit5-unity/Example Scenes). Open that and look at the ARToolkit Object. In the inspector, you will find two ARMarker scripts with two NFT Markers already configured. To replace the standard NFT Marker with your own you can put your own fset files under Assets/StreamingAssets and restart unity. After that, you should be able to choose your files in the marker script.
That being said. NFT markers work considerably worse than in vuforia and kudan. There is a reason why this a "paid" frameworks.
I hope this helps.
I am doing an app in which I require a business card reader I googled alot but BBY is the only solution which I was able to find out. Can anybody help me out with some opensource library which can be tweaked or used directly as a business card reader.
Please enlighten me on this.
you can look into the Tesseract open source engine... its pretty good for image processing.. i mean it will extract the text out of the image but then you will have to process it to extract name ,phone numbers and other details.
this guy has explained how to use it in iOS .. http://tinsuke.wordpress.com/2011/11/01/how-to-compile-and-use-tesseract-3-01-on-ios-sdk-5/
We started an open source project to build a Javascript library (based on the OCR engine tesseract.js for the OCR part) that exctract the relevant data from a business card based on heuristic criteria.
The library (BCR Library, available on github) is usable in any html project (included mobile cordova, phone gap or ionic projects) just including it via script tag.
The library doesn't have any external api call and fully works offline.
I think that you should give a try to Covve Bussiness Card Scan API. The quality of the result is great in various languages. You can check a comparison analysis of similar services here.
[Disclosure] I'm part of the team developing the service.
I have been fighting a losing battle against loading fonts from an embedded file for use with DirectWrite. I am writing a simple puzzle game that has a C#/XAML interface but also uses SurfaceImageSource to add some DirectX content.
I have written a WinRT component that handles all of the DirectX code, and it works quite nicely. Some of my DirectX content is text drawn using the DirectWrite API. I can draw all the text I like so long as I'm loading an installed font from the system using IDWriteFactory::GetSystemFontCollection(), etc. But, I cannot seem to find a way to load a custom font from an embedded file.
From what I can tell Metro apps are not allowed to load files from the filesystem in the same way as a traditional app. So, the IDWriteFactory::CreateFontFileReference() method that takes a normal file path is worthless to me, right? I need to load my file from an ms-appx URL.
So, I wrote a custom font loader in my WinRT component that implements the IDWriteFontCollectionLoader interface (which is a ton of work if you've never done it before btw) that loads the font from an ms-appx URL using the new StorageFile API. Now, I can load my IDWriteFontFile and I can get a IDWriteFontFace, but if I try to call any of the truly useful methods on the font face it returns E_UNEXPECTED. I can get the number of glyphs and the glyph indices, but if I try to call something like GetGlyphRunOutline() or GetDesignGlyphMetrics(), it fails with E_UNEXPECTED. Using the same drawing code that generates an ID2D1PathGeometry using GetGlyphRunOutline() works great as long as I install the font file and get the IDWriteFontFace through the series of calls starting with IDWriteFactory::GetSystemFontCollection(). I am working with a normal true type font.
So, how do I load a custom font from an embedded file into DirectWrite in a Metro app? I'm probably just missing something easy, because I am certain that other people will want to be able to load custom fonts in this way.
I have a sample project (or could prepare one easily) for anyone who can help me identify my problem.
I have loaded the two IDWriteFontFace objects side by side, and I tried to figure out what is different between the one that works and the one that breaks. What I need to see in order to find out why it is failing is opaque to me hidden behind inside the IDWriteFontFace interface. HELP PLEASE!
Question also posted here: Building Metro style games with DirectX Forum
Well, the answer is... don't write a IDWriteFontCollectionLoader! You can use IDWriteFactory::CreateFontFileReference() with the StorageFile API. I was under the impression from all of the Microsoft Conference talks I had attended that in Metro you would be unable to access the native file system directly; the way forward would be to use the StorageFile API which references resources, etc. using ms-appx URLs. I understood that this was done for concurrency and to allow the OS to insulate itself from Metro apps that would be downloaded from the store by creating a file system sandbox. I think that is accurate. But, I feel like I was led to believe that we would never be able to get native file system paths. That is NOT true. IStorageFile provides a way. Just use IStorageFile.Path. I never looked at it because I just assumed the Path property would hold the ms-appx URL that I used to create the object. Microsoft probably provided this for exactly the purpose in my problem above: calling legacy COM interfaces that require a native path.
I haven't done any testing to determine whether the WinRT framework actually sandboxes you if you try to access a native file system path outside your own app package. I'm betting that it does...