I have integrated my dialogflow agent with google assistant. There is a welcome intent that will ask you to choose any of the option
Choose any of the sports
1. NBA
2. NHL
3. FIH
It reads the response with ever individual words(as an abbreviation). But when I produce the same in response from webhook, it is not reading the response with individual words(or not considering the response as abbreviation) and reads together. How can I achieve this? Am I missing something in the response?
You likely want to make sure you're sending back SSML in your response, rather than sending back text and letting it convert it to speech, and specifically marking the abbreviations using the <say-as> tag and telling it to interpret the contents as characters.
So you might send it back as something like:
<speak>
Are you interested in learning more about
the <say-as interpret-as="characters">NBA</say-as>,
the <say-as interpret-as="characters">NHL</say-as>
or the <say-as interpret-as="characters">FIH</say-as>?
</speak>
The little pronunciation differences with and without SSML are serious problems. I stick in a speak /speak for everything. Also a unique number I like and a test hook to have speech 'count' or not so there is a way to test things. Also a hook so an intent is triggered for 'repeat that please' :
Point is to use sayUsual for everything ordinary.
// Mostly SSML start char kit as globals
const startSp = "<speak>", endSp = "</speak>";
// Handle "Can you repeat that ?" well
var vfSpokenByMe = "";
// VF near globals what was said, etc
var repeatPossible = {}; repeatPossible.vf = ""; repeatPossible.n = 0;
// An answer from this app to the human in text
function absorbMachineVf( intentNumber, aKind, aStatement )
{
// Numbers reserved for 'repeats'
if( intentNumber > 9000 ) { return; }
// Machine to say this, a number for intents too
repeatPossible.vf = aStatement; repeatPossible.n = intentNumber;
}
// Usual way to say a thing
function sayUsual( n, speechAgent, somethingToSay )
{
// Work with an answer of any sort
absorbMachineVf( n, 'usual', somethingToSay );
// Sometimes we are just pretending, so
if( !testingNow )
{ speechAgent.add( startSp + somethingToSay + endSp ); }
// Make what we said as an answer available 'for sure' to rest of code
vfSpokenByMe = somethingToSay; // Even in simulation
}
Related
I am looking for something that allows me from a mail PDF attachment to get a data in a google sheet.
We all often get PDF attachments in our email and it will be great if we get the entire data in a google sheet.
DO let me know if there is anything like this
Explanation:
Your question is very broad and it is impossible to give a specific answer because that answer would depend on the pdf but also on the data you want to fetch from that, besides all the other details you skipped to mention.
Here I will provide a general code snippet which you can use to get the pdf from the gmail attachments and then convert it to text (string). For this text you can use some regular expressions (which have to be very specific on your use case) to get the desired information and then put it in your sheet.
Code snippet:
The main code will this one. You should only modify this code:
function myFunction() {
const queryString = "label:unread from example#gmail.com" // use your email query
const threads = GmailApp.search(queryString);
const threadsMessages = GmailApp.getMessagesForThreads(threads);
const ss = SpreadsheetApp.getActive();
for (thr = 0, thrs = threads.length; thr < thrs; ++thr) {
let messages = threads[thr].getMessages();
for (msg = 0, msgs = messages.length; msg < msgs; ++msg) {
let attachments = messages[msg].getAttachments();
for (att = 0, atts = attachments.length; att < atts; ++att) {
let attachment_Name = attachments[att].getName();
let filetext = pdfToText( attachments[att], {keepTextfile: false} );
Logger.log(filetext)
// do something with filetext
// build some regular expression that fetches the desired data from filetext
// put this data to the sheet
}}}
}
and pdfToText is a function implemented by Mogsdad which you can find here. Just copy paste the code snippet provided in that link together with myFunction I posted in this answer. Also you have some options which you can use that are very well explained in the link I provided. Important thing to note, to use this library you need to enable the Drive API from the resources.
This will get you started and if you face any issues down the road which you can't find the solution for, you should create a new post here with the specific details of the problem.
It is possible to differentiate among speakers/users with the Watson-Unity-SDK, as it seems to be able to return an array that identifies which words were spoken by which speakers in a multi-person exchange, but I cannot figure out how to execute it, particularly in the case where I am sending different utterances (spoken by different people) to the Assistant service to get a response accordingly.
The code snippets for parsing Assistant's json output/response as well as OnRecognize and OnRecognizeSpeaker and SpeechRecognitionResult and SpeakerLabelsResult are there, but how do I get Watson to return this from the server when an utterance is recognized and its intent is extracted?
Both OnRecognize and OnRecognizeSpeaker are used only once in the Active property, so they are both called, but only OnRecognize does the Speech-to-Text (transcription) and OnRecognizeSpeaker is never fired...
public bool Active
{
get
{
return _service.IsListening;
}
set
{
if (value && !_service.IsListening)
{
_service.RecognizeModel = (string.IsNullOrEmpty(_recognizeModel) ? "en-US_BroadbandModel" : _recognizeModel);
_service.DetectSilence = true;
_service.EnableWordConfidence = true;
_service.EnableTimestamps = true;
_service.SilenceThreshold = 0.01f;
_service.MaxAlternatives = 0;
_service.EnableInterimResults = true;
_service.OnError = OnError;
_service.InactivityTimeout = -1;
_service.ProfanityFilter = false;
_service.SmartFormatting = true;
_service.SpeakerLabels = false;
_service.WordAlternativesThreshold = null;
_service.StartListening(OnRecognize, OnRecognizeSpeaker);
}
else if (!value && _service.IsListening)
{
_service.StopListening();
}
}
}
Typically, the output of Assistant (i.e. its result) is something like the following:
Response: {"intents":[{"intent":"General_Greetings","confidence":0.9962662220001222}],"entities":[],"input":{"text":"hello eva"},"output":{"generic":[{"response_type":"text","text":"Hey!"}],"text":["Hey!"],"nodes_visited":["node_1_1545671354384"],"log_messages":[]},"context":{"conversation_id":"f922f2f0-0c71-4188-9331-09975f82255a","system":{"initialized":true,"dialog_stack":[{"dialog_node":"root"}],"dialog_turn_counter":1,"dialog_request_counter":1,"_node_output_map":{"node_1_1545671354384":{"0":[0,0,1]}},"branch_exited":true,"branch_exited_reason":"completed"}}}
I have set up intents and entities, and this list is returned by the Assistant service, but I am not sure how to get it to also consider my entities or how to get it to respond accordingly when the STT recognizes different speakers.
I would appreciate some help, particularly how to do this via Unity scripting.
I had the exact same question about dealing with the Assistant's messages, so I looked at the Assistant.OnMessage() method that returns a string like “Response: {0}”, customData[“json”].ToString() plus the JSON output that will be something like this:
[Assistant.OnMessage()][DEBUG] Response: {“intents”:[{“intent”:”General_Greetings”,”confidence”:1}],”entities”:[],”input”:{“text”:”hello”},”output”:{“text”:[“good evening”],”nodes_visited”: etc...}
I personally parse the JSON in order to extract the content from messageResponse.Entities. In the above example, you can see that that the array is empty, but if you are populating it, then that’s where you need to extract the values from and then in your code you can do what you want.
Regarding the different speaker recognition, in the Active property whose code you have included, the _service.StartListening(OnRecognize, OnRecognizeSpeaker) line takes care of both, so perhaps put some Debug.Log statements inside their code blocks to see if they are called or not.
Please set SpeakerLabels to True
_service.SpeakerLabels = true;
I am having a java program send data to me over a specific socket to my node.js application. I want to be able to obtain all of the data, which is information from a SQlite database, and send it off to something else.
I've found something like the following can work but it seems to be unreliable as data is missing and sometimes it doesn't even show up.
stream.addListener('data', function(data){
buffer.write(data.toString());
});
on a side note, I need the socket to stay open so I can't call the "end" event.
I really don't have any attachment to stream.addListener so i can use something else if it works how i want. Basically what i'm asking is, What is the most effective way to obtain data from a socket using node.js?
P.S. thank you for your time
The data event is not guaranteed to have all the data sent to it in one go. You'll need to build up a buffer over multiple events and watch for delimiters of some kind (newlines, null characters, whatever you feel). Here's an example from a project where I'm parsing data from IRC (converted from CoffeeScript); parseData is the event handler for the data event (e.g. socket.on('data', this.parseData);):
IrcConnection.prototype.parseData = function(data) {
var line, lines, i;
data = data.replace("\r\n", "\n");
this.buffer += data;
lines = this.buffer.split("\n");
this.buffer = "";
/* Put the last line back in the buffer if it was incomplete */
if (lines[lines.length - 1] !== '') {
this.buffer = lines[lines.length - 1];
}
/* Remove the final \n or incomplete line from the array */
lines = lines.splice(0, lines.length - 1);
for (i = 0; i < lines.length; i++) {
line = lines[i];
this.emit('raw', line);
}
};
From what I understood here, "V8 has a generational garbage collector. Moves objects aound randomly. Node can’t get a pointer to raw string data to write to socket." so I shouldn't store data that comes from a TCP stream in a string, specially if that string becomes bigger than Math.pow(2,16) bytes. (hope I'm right till now..)
What is then the best way to handle all the data that's comming from a TCP socket ? So far I've been trying to use _:_:_ as a delimiter because I think it's somehow unique and won't mess around other things.
A sample of the data that would come would be something_:_:_maybe a large text_:_:_ maybe tons of lines_:_:_more and more data
This is what I tried to do:
net = require('net');
var server = net.createServer(function (socket) {
socket.on('connect',function() {
console.log('someone connected');
buf = new Buffer(Math.pow(2,16)); //new buffer with size 2^16
socket.on('data',function(data) {
if (data.toString().search('_:_:_') === -1) { // If there's no separator in the data that just arrived...
buf.write(data.toString()); // ... write it on the buffer. it's part of another message that will come.
} else { // if there is a separator in the data that arrived
parts = data.toString().split('_:_:_'); // the first part is the end of a previous message, the last part is the start of a message to be completed in the future. Parts between separators are independent messages
if (parts.length == 2) {
msg = buf.toString('utf-8',0,4) + parts[0];
console.log('MSG: '+ msg);
buf = (new Buffer(Math.pow(2,16))).write(parts[1]);
} else {
msg = buf.toString() + parts[0];
for (var i = 1; i <= parts.length -1; i++) {
if (i !== parts.length-1) {
msg = parts[i];
console.log('MSG: '+msg);
} else {
buf.write(parts[i]);
}
}
}
}
});
});
});
server.listen(9999);
Whenever I try to console.log('MSG' + msg), it will print out the whole buffer, so it's useless to see if something worked.
How can I handle this data the proper way ? Would the lazy module work, even if this data is not line oriented ? Is there some other module to handle streams that are not line oriented ?
It has indeed been said that there's extra work going on because Node has to take that buffer and then push it into v8/cast it to a string. However, doing a toString() on the buffer isn't any better. There's no good solution to this right now, as far as I know, especially if your end goal is to get a string and fool around with it. Its one of the things Ryan mentioned # nodeconf as an area where work needs to be done.
As for delimiter, you can choose whatever you want. A lot of binary protocols choose to include a fixed header, such that you can put things in a normal structure, which a lot of times includes a length. In this way, you slice apart a known header and get information about the rest of the data without having to iterate over the entire buffer. With a scheme like that, one can use a tool like:
node-buffer - https://github.com/substack/node-binary
node-ctype - https://github.com/rmustacc/node-ctype
As an aside, buffers can be accessed via array syntax, and they can also be sliced apart with .slice().
Lastly, check here: https://github.com/joyent/node/wiki/modules -- find a module that parses a simple tcp protocol and seems to do it well, and read some code.
You should use the new stream2 api. http://nodejs.org/api/stream.html
Here are some very useful examples: https://github.com/substack/stream-handbook
https://github.com/lvgithub/stick
I want to write a word addin that does some computations and updates some ui whenever the user types something or moves the current insertion point. From looking at the MSDN docs, I don't see any obvious way such as an TextTyped event on the document or application objects.
Does anyone know if this is possible without polling the document?
Actually there is a way to run some code when a word has been typed, you can use SmartTags, and override the Recognize method, this method will be called whenever a word is type, which means whenever the user typed some text and hit the space, tab, or enter keys.
one problem with this however is that if you change the text using "Range.Text" it will detect it as a word change and call the function so it can cause infinite loops.
Here is some code I used to achieve this:
public class AutoBrandSmartTag : SmartTag
{
Microsoft.Office.Interop.Word.Document cDoc;
Microsoft.Office.Tools.Word.Action act = new Microsoft.Office.Tools.Word.Action("Test Action");
public AutoBrandSmartTag(AutoBrandEngine.AutoBrandEngine _engine, Microsoft.Office.Interop.Word.Document _doc)
: base("AutoBrandTool.com/SmartTag#AutoBrandSmartTag", "AutoBrand SmartTag")
{
this.cDoc = _doc;
this.Actions = new Microsoft.Office.Tools.Word.Action[] { act };
}
protected override void Recognize(string text, Microsoft.Office.Interop.SmartTag.ISmartTagRecognizerSite site,
Microsoft.Office.Interop.SmartTag.ISmartTagTokenList tokenList)
{
if (tokenList.Count < 1)
return;
int start = 0;
int length = 0;
int index = tokenList.Count > 1 ? tokenList.Count - 1 : 1;
ISmartTagToken token = tokenList.get_Item(index);
start = token.Start;
length = token.Length;
}
}
As you've probably discovered, Word has events, but they're for really coarse actions like a document open or a switch to another document. I'm guessing MS did this intentionally to prevent a crappy macro from slowing down typing.
In short, there's no great way to do what you want. A Word MVP confirms that in this thread.