I have a lot of existing data that I would like to use as training data for a wit.ai chatbot. The data is stored in a csv file where each row has a statement/question and a response to that statement/question.
I know that wit.ai requires you to assign intents to comments made and so I'm wondering if there is a way to simply send over the data I have and have the chatbot start learning intents on its own.
Thanks!
Thanks for posting. We know this is not perfect yet but we release an import/export feature a few days ago. Looking at the structure of the json export, one can probably easily feed with existing data. It would require creating one story per statement/question and a response. More info here:
https://wit.ai/docs/recipes#copyexportversion-my-app
"Teaching" Wit.Ai is not exactly what some might think it is.
You will have to create stories for your User says column. The replies are irrelevant to be honest. You can't "teach" wit.ai to reply. Replies are defined in the story or in your code.
What wit.ai might need from your data are keywords and key-phrases which make the entity recognition better for wit.ai.
Here is the simplest example:
Entity color is recognized based on keywords listed. So if you have a lot of data as an example of user input - you can try to break it down first into "which entities which user input should produce" and then keywords from those input.
Using your data for "teaching" - would be a little difficult since it will require you to create a lot of Stories in wit.ai to cover possible user input and entity identification. But you can still do it like this:
(rough example)
Make one story about user asking the time for example
Mark in the user input which entities should be derived from that input:
Sort your list you have to get all possible way of asking for the time:
How late is it?
Can you tell me the time?
I wonder what's the time now?
Use a script (Python) to "shoot" all these user inputs at your story.
Once done - go to Understanding time of wit.ai and go through all input correcting\adding the entities you defined.
This process will "teach" entities if they are keywords based or some other algorithm.
That's the best I can think of about how to use your existing data. Wit.Ai is different from other language processing tool-sets and "teaching" it with existing data is somewhat "puzzling" :)
Related
First time posting, so feel free to give me feedback if I could improve something about this post... Now on to my question.
I am currently developing a Google Action, the Action will allow the user to define important events, such as Bob's Birthday or Fred's Graduation, and save data about said events. Later, the user will be able to ask for info about the event and get it returned back to them.
I am using the Dialogflow API with "Inline Editor" fulfillment to keep it as simple as possible for right now. The problem I am running into is this, the event has an entity type of #sys.any, so anything the user says is excepted as valid input. I would like some way to bias towards events I already have stored for the user however, so that they are more likely to find the event they are looking for.
I found another answer on here discussing speech biasing (What is meant by speech bias and how to use speechBiasHints in google-actions appResponse) which defined speech biasing as the ability to"influence the speech to text recognition," which is exactly what I believe I want. While that answer provided sample code, it was for the Actions SDK, not the Dialogflow SDK, which I am using.
Can anyone provide an example of how to fill the "speechBiasingHints" section of the ExpectedInput response of the Conversation Webhook using the DialogFlow Webkook?
Note: This is for a student project, and I'm new to developing Google Actions and still very much learning about everything that is capable with Google Actions. Any feedback or suggestions are very welcome.
The question you link to does quite a few things differently than the approach you're taking. The Action SDK provides more low-level control, but doesn't have much Natural Language Processing (NLP) capabilities, which Dialogflow provides.
Dialogflow handles biasing a little differently through the use of Entities, so you don't need to control the speech biasing directly, Dialogflow can handle that for you, to some extent.
Since each user may have a different event name, you'll probably want to use a User Entity, which is an entity you define and then populate on a user-by-user basis through Dialogflow's API. In your sample phrases, you can then use this entity name instead of #sys:any, or create another set of phrases that use this entity in addition.
I created a dataset using 2 extractors: a "many rows" extractor which is then called by a "just one row" extractor via the "URLs from another API / Chain APIs" option. This has given me the data that I need and I have saved it as a dataset. Although the dataset is working I'm lost as to how to use the REST API to retrieve it?
I'm aware that there used to be an "integrate" button on the dataset page which would outline how to use an external client library to get the data but in it's absence now I don't know which of the APIs to use and how to use them?
I've attempted at using the "Query Methods" GET call "/store/connector/{id}/_query" but it requires an "id" which I don't know where to find from? I attempted to use the "_connectorVersionGuid" value when I saved the dataset as JSON but that didn't work.
Any help and advice would be much appreciated.
Thanks,
AJ
AJ,
Francesco here, from import.io.
First of all, thanks for formulating the question so clearly.
I have bad news and good news.
The bad news is at the moment Bulk and Chain are only available as client side feature, so it's not really possible to call a chain with a single REST call.
The good news is that we are actually working on it :)
Bulk as an API is actually in beta testing, and I hope to have a Chain as an API as well.
A workaround I sometimes use myself is to use an external integration as a service platform, like Node-RED (http://nodered.org/) or built.io (https://www.built.io/)
Call me crazy, but I'm looking for a service that will deliver autocomplete functionality similar to Google, Twitter, etc. After searching around for 20 min I thought to ask the geniuses here. Ideas?
I don't mind paying, but it would great if free.. Also is there a top notch NLP service that I can submit strings to and get back states, cities, currencies, company names, establishments, etc. Basically I need to take unstructured data (generic search string) and pull out key information with relevant meta-data.
Big challenge, I know.
Sharing solutions I found after further research.
https://github.com/haochi/jquery.googleSuggest
http://shreyaschand.com/blog/2013/01/03/google-autocomplete-api/
If you dont want to implement it yourself, you can use this service called 'Autocomplete as a Service' which is specifically written for these purposes. You can access it here - www.aaas.io.
you can add metadata with each record and it returns metadata along with the matching results. Do check out demo put up on the home page. It has got a very simple API specifically written for autocomplete search
It does support large datasets and you can apply filters as well while searching.
Its usage is simple - Add your data and use the API URL as autocomplete data source.
Disclaimer: I am founder of it. I will be happy to provide this service to you.
I'm currently in the process of trying to define a dynamic form in Play (Scala). What I mean by a "dynamic" form is one which presents different form elements (and POSTs different data) depending on the state of some runtime (specifically database-side) data.
A brief illustrative example: on a to-do list application, how could one create a form which creates a "delete" check box next to each of the list elements? I realize this could be done with GET links or with AJAX/javascript, but I want this to be a standard POST form with a "submit" button.
I'm aware of the repeated mapping form functionality, but is it possible to define an even more dynamically generated form, where the structure is less strict than just single+repeated elements?
Thanks for any suggestions you might have. I realize this might just be more than POST is cut out for.
The closest thing that I have to an answer for this is that POST and any sane web framework supports "array" inputs. Hence, if you can get your data into any regularized format, you can serialize/deserialize via that. You may be able to support semiexotic data patterns by using this method as well.
I am working on an application which will have users.. who create posts.. and other users can like/comment on any post.
I am trying to figure out a best way to design db tables for this. I have read the anypics tutorial on parse.com site. They keep all comments and likes in a table called "Activity". (which makes sense) being able to query any type of activity (like/comment) from a separate table without having to touch "posts" table.
My question is- in this scenario how do I fetch all posts that current user created along with likes and comments on each those posts?
Anypic app by parse makes a separate request to fetch number of likes on each post (which I think is not ideal.) I am new to nosql data stores.. so if someone could help me out with suggestion on how to structure data that would be great.
Also, how bad is it to store all likes/comments as an array in the post itself? I think this won't scale but I might be wrong.
Thanks
In terms of Parse, I would use an afterSave Cloud Function to update the Post anytime a like/comment is added.
Have a look at the documentation here, in the most simple case you just create an afterSave for the Activity class that increments the like/comment count.
In a more advanced scenario you might want to handle update/delete too. E.g. if someone can change their 'like' to 'not like' you would need to look at the before/after value and increase/decrease the counter as needed.
I am a fan of storing extra 'redundant' data, and no-sql/document-db systems work well for that. This is based on the idea that writes are done infrequently compared to the number of reads, so doing some extra work during/after the write has less impact and makes the app work more smoothly.