I need to develop a chatbot with these properties:
Platform - Skype for Business On-Premise
Function - Replies to user queries by looking in various knowledgebases (Multiple Platforms - Databases, Web APIs, etc.)
Basic textual conversation to begin with and will gradually evolve to send attachments
No calls/videos, just chat
Will be hosted on an external server with organisation VPN
A simple sip will be created for the chatbot which can be pinged by any user. I should be able to get this through to our IT dept.
Limited time for development
Scalability is an essential requirement but the organisation is fairly new to this, so they might be patient and allow me to make mistakes
My research has led me to these possible approaches:
SfB SDK - I have rejected this approach because it requires the client to be running at all times and doesn't seem to be scalable
UCMA with Application Endpoint - Haven't rejected this approach, but seems like I'll not go ahead with this because creation of Application Platforms seems tedious and requires me to make a lot of SfB server related IT requests
UCMA with User Endpoint - Great affinity towards this. I have experimented Tom Morgan's (thoughtstuff.co.uk) stuff and this seems like something I can start off right away
UCWA with Application Endpoint - Rejected this approach, because UCWA (from my research) appears unsuitable for On-premise and the setup also seems time consuming
UCWA with User Endpoint - Haven't rejected this approach, but I'm not sure if the Web API way is really a good approach for On-premise platform
I'd like to ask how am I doing so far, but that seems too vague
What would you suggest is a good way to achieve this?
Also, can someone be patient enough to reply the drawbacks and advantages of each approach for my use case. I'd like to make an informed decision and not reject any approach, just because of a misunderstood overhead
I have been asking around in my organisation and other circles.
And since I am not receiving any quick responses, I'll keep adding what I have learnt.
This way a person in the dev community will have a log of how I went with this.
UCWA is better suited for S4B online (compared to on premise) and is generally used by people who are comfortable with RESTful and have low familiarity with .NET development
UCMA is apparently THE WAY to go and for any on-premise bot requirements, preferably with an application endpoint.
So for our development, we are starting with UCMA user endpoint so that we can deliver a basic start as a version-one
And meanwhile we shall also get in touch with the IT department and Lync administrators for creation of Application Endpoints
Once we have this the same functionality that we had with the user endpoint will be copied over to the Application Endpoint version
Keep watching this space for further updates
Related
Background:
I'm building an API service app. The app is just like any other, you send an HTTP request and receive a response. This seems simple up until I start thinking about user registration, payments, authentication, logging and so on.
Application:
tl;dr simple app diagram
Endpoints listening for HTTP requests and doing all the request related work. This is the core of the service, what the service user would use this app for. Directly not accessible to the end user (unless somehow it knows the url). Python flask server, deployed on google cloud RUN.
API gateway acting like a proxy and a single access point forwarding the requests to the endpoints. This is the service access point for the end users. This part will also be responsible for authentication, limitations, logging and tracking the use of the API endpoints. Python flask server, deployed on google cloud RUN.
Website including documentation, demo and show off of API calls through API gateway, registration, payment (thinking of Stripe) etc. VueJS app on NodeJS server on google cloud compute VM.
Database storing credentials of registered users, payment information and auth keys. Not implemented yet.
Problems:
Is this architecture proper? What could be done differently or improved? How could I further simplify all the interactions between separate parts of the app? Am I not missing any essential parts?
Haven't yet implemented the database part and I'm not sure what should I
use? There are plenty of options on google cloud. Also I could go with something simple and just install a DB with http/JSON interface on google cloud compute VM. How do I chose the DB? Given such an app, what would be the best choice?
Please recommend literature/blogs/other sources of info on similar app
architecture for new developers not familiar with it?
This is pretty open ended, but here are some general comments:
Think about how your UI will work. Are you setting up a static app served directly from cloud storage or do you need something rendered on the server? Personally I prefer separating UI from API when I can but you need to be aware of things like search engine optimization. Even if you need to render some content dynamically your site can still be static. Take a look at static site generators like Gatsby. I haven't had to implement a server rendered UI in years and that makes me happy.
API gateway might be fine, but you don't really need it for anything. It might be simpler to start without it and concentrate on what actually matters. If your APIs are being called by an external client you can't trust the calls anyways and any API key you might be using will be exposed. I'd say don't worry about it for a single app. That being said, if you definitely want to use a GW then use one, just be aware that it is mostly a glorified proxy and not some core part of your architecture.
Make sure your API implementations don't store any local state so you can rely on Cloud Run scaling your services up and down. Definitely don't ever store state directly inside your containers. If you need state on the server it needs to be in some external data store.
Use JWTs or an external IDM (that will generate JWTs) for authentication. Keep session data on the client side as much as possible and pass the JWT in every API call to authenticate the caller. If you are implementing login on your own the only APIs you need to expose without tokens are for auth and password recovery, which you can separate into their own service.
Database selection depends on how well you understand your processes, how transactional your services are and your existing skillset. Overall I would use what you are comfortable with, you can probably succeed with a lot of things. Certain NoSQL flavors can seem simple on the surface but if you don't have a clear understanding on the types of queries you need to run they can get tedious to work with. Generally you should stick to relational databases for OLAP style implementations and consider NoSQL for OLTP. Personally I like MongoDB and it is very popular, probably because it sort of sits in the middle of the pack which makes it fit a lot of applications. Using MongoDB also makes you cloud agnostic since it is available on every platform. Using platform specific database flavors can lock you down to a specific vendor.
Whatever you do, don't start installing things on VMs. You can be almost 100% sure you are doing it wrong if this comes up. Remember, the services you consume don't all have to be managed by Google or even run on GCP. You can get MongoDB capacity directly from MongoDB who manage it on your behalf on all of the Big3 cloud vendors.
At least think about the long term, even if you don't necessarily need to have it impact your architecture right now. If you are expecting your app to be up for years try to make it more platform agnostic than less. This might mean sticking away from some really platform specific serverless features that will force you to jump a couple of extra hoops. If you are using Cloud Run you are using containers which already makes your app pretty portable, don't lock it to one platform by using a lot of platform specific features. That being said, don't stay away from them either. You should always go for the low hanging fruit, so don't try to avoid using things like secrets manager etc. If your app has a short lifespan and you need really fast time to market then don't worry about it.
Just my 2c, what you are doing is very generic and can be done in a lot of different ways.
In our experience with other APIs there is usually a test/private mode and then after the app is approved it can go into live/public mode.
After we create an application based on Microsoft Graph / Outlook REST, what is the process to get the application live/public available for use by our customers and how long does it take? Or is it already live?
Your question isn't really specific, so I'll just answer the general question.
Everybody can create an application for the Microsoft cloud. There isn't a test environment, so every application is live the moment you create it (and switch on multi-tenant). It is always up to the user (or tenant admin) to grant your application access!
Microsoft does however offer various ways to get your application under the attention of a much larger audience. And to get your application in such a marketplace they have various review/test/... processes in place.
We are developing some REST api's for internal use. To test these microservices we are toying with the idea that every service has a sandbox mode so we can do integration tests that are as close as possible to the real deal.
To see if this path is worth trying we are looking for documentation / best practices on how to manage this sandbox and how to implement this internally. When we look for the keywords Sandbox, REST API and Best Practices we only find how to implement as consumer of existing sandboxes.
So does anyone have some documentation / links in how to tackle this problem and what the pro's and con's are of the different ways?
Kr,
Thomas
I'd say there are two ways to proceed:
Basic: keep a separate sandbox instance of a service. You always deploy a new code to this instance first and run automated/manual tests to verify if everything works fine. A datastore could be a snapshot from the production data or artificial testing data. I would rather we have a "Snapshot" but it depends whether it is applicable in your particular case (privacy etc.)
Advanced: I spied this technique on Facebook Marketing API. This API provides an interface to set up and launch advertising campaigns. They didn't provide a sandbox api for testing purposes (at least last year when the system I was working on had been integrating with Facebook). However if you use a keyword "test" in a name of a campaign or an adset (key entities in the ad world) they would never launch and spend your money. You can try extend this concept on your particular domain and run tests on (or very close to) your production
Hope this helps
I'm writing a mobile application, and this mobile application will be talking to a server in Azure to get data, and possibly push some too.
Now, I always had it in mind that I was going to write a simple RESTful web service that was going to deal with all this, and host it on Azure in a website - however, I've since discovered their Mobile Services, and I'm not entirely sure I understand what it's for.
Looking at it, it seems that it's about extracting away the code running on the server from you, and what you push to it, is what you get back? So it wouldn't be much use for me, as my web service is going to be returning data dependant on logic etc. However the ability for the Mobile Services to deal with push notifications directly is nice (how hard is it to use Azure Websites to send push notifications through mobile services?)
However, I wonder if I'm missing something? I'd rather not have to go re-invent the wheel as such, is Mobile Services will do what I need, especially as I'll be using Xamarin and there is a nice component to talk to Mobile Services...
Thanks
I am facing the same question: Should I use Azure Mobile Services or roll my own REST service. Below are some discoveries I have made so far.
With Azure Mobile Services there is a backend running a set of assemblies, which your code ultimately depends on. This is fine until you start using versions of other assemblies, which are not compatible with the versions Microsoft hosts. To me this is a major pain. This problem alone makes me question whether I want to use Azure Mobile Services. If you make your own REST service, I do not expect you to face these problems. See this link for details.
You should note that an Azure Mobile Services is tied to a single Azure region: If your Azure region fails, you need to manually migrate to another Azure region. This might not be a problem for your project. If you want to run in more than one region, I think it will be easier to roll your own REST service.
Regarding push notification, you could do that without using Mobile Service. See these links:
http://msdn.microsoft.com/library/azure/jj927170.aspx
http://weblogs.asp.net/scottgu/broadcast-push-notifications-to-millions-of-mobile-devices-using-windows-azure-notification-hubs
Auto-scaling is supported out-of-the-box in Azure Mobile Services. You would need to make your own scaling logic if you make your own REST service. You can scale VMs using Azure’s REST API, but it will be more work than using Mobile Services.
Authentication is supported in Azure Mobile Services. You would need to look into a separate framework to do this if you make your own REST service.
If you use Mobile Services you will probably create ASP.NET Web APIs through the Visual Studio wizards. There are examples and tutorials, which is great. However, even if you roll your own service, you could build on other frameworks. One example is ServiceStack, which I would strongly consider if you want to make your own REST service. I think ServiceStack could also help with authentication.
If you use Mobile Services you will tie yourself stronger to Azure. By rolling your own REST service it will probably be easier to move to another infrastructure later.
I have been looking for a way to protect my RESTful APIs. This appeared simple, but it seems to not be so simple. First off, I am writing an iOS app connecting to a Play Framework server. None of this has anything to do with Google, Facebook, Twitter, or LinkedIn (shocking I know). Oh, and my current plans do not require custom apps to use my APIs, its just my apps for the time being.
Basic Authentication
What appeared to be simple was a basic user/pass on a /auth method managing a cookie session. That may draw some groans as being too simple or weak but mostly it moved identity to a session key quickly verified. My initial setup was to expire the sessions every day, but that lead to the iOS app forcing a login daily proving to be an annoyance.
OAuth
I posted a question on an iOS board and received a blunt direction towards OAuth. My research of OAuth began but holy sh*t is that complicated and there does not seem to be any server side examples... just plenty of people complaining about how frustrating it is. All the client examples show connecting to Google, Facebook, Twitter, and LinkedIn. Oh Joy!
After watching Eran Hammer's rant about OAuth1 and OAuth2, it seemed fruitless to continue and his OZ idea (which looks really clean) is only at the early stages in node.js.
Question
So, my question to the broad StackOverflow community is... what do you do for securing your REST APIs?
I'd suggest to consider approach used by biggest players i.e. Amazon Web Services or Windows Azure - HMAC. Although it isn't comfortable in implementation, as you can see it's trusted technique.
The general idea is to sign the request's parts (i.e. headers) in the iOS with secret key and try to recalculate it on the Play app to verify that request is authentic and not manipulated. If it won't fail, you can be (almost) sure, that was sent from somebody, who uses valid secret key.
Take a look into Windows' document to get the concept (I think that for common task, you can use the less number of elements used for signing).
There is also other interesting post (based on AWS authentication) which describes whole process even better.
Edit
Of course you should realize that authentication in iOS and securing API requests are different things, even if you'll expire your session every 15 minutes, you can't be sure that somebody won't overhear it and then will be able to send a fake request from the outside. Signing every request should minimize the risk.
On the other hand, if you'll prepare clear rules for signing the requests and will write short doc (which I recommend even for yourself), you can deliver it to the other developer and he'll be able to implement it in (almost) any platform supporting SHA256, so you will have API ready for using from 3-rd party apps - if you'll decide to publish it in the future.
Since Play Framework is in Java, you could use Apache Shiro
I haven't used it yet.. (I am planning to though) So I don't know if it's the best option.
Just do something simple, send the authorization code / password in a custom header over HTTPS .
So the only problem with the Basic Authentication approach was that the user has to login every day? Why not offer the user an option to save his username/password on the device? That way he can choose between security and convenience.