I am looking for the best way to test this scenario using automation. Before you mention or ask, yes, i have throughly searched to make sure or even get info or knowledge on this topic.
I have a web service that provides various pricing for health insurance. The pricing is based on different class category of race, age, location etc. All that is inputed into the web service to get the fee for the client.
Right now everything is done locally with SoupUI. As you know SoupUI does not have a way for you to randomly change the data in the body, but i need a way to automate this so random age, race, location, etc can be inputed in and verify the result against a CSV file with the information.
Is this possible or is there a better way to test this web service?
-
Per #lloyd request, because he is King here and other known methods i know:
UI Option: We build a UI to interface with the API and use Selenium to run the countless scenarios we have, but they do not want to do this. They are more incline on testing the API itself.
Ruby: Use Ruby to connect to the API, using Rspec and Airborne (https://github.com/brooklynDev/airborne), the downsize of this is also the same as using SoupUI. Everything is entered in manually, and the code just runs through the various variations we have to get the return and match it against what it should be.
I also found another tool, https://assertible.com/ but the downsize of that is we cannot limit the random generator between 2 selected numbers, plus no way to do the comparison.
Then there is Postman, which is also similar to SoupUI so not a real automation since we cannot change the values in the body. http://blog.getpostman.com/2015/09/03/how-to-write-powerful-automated-api-tests-with-postman-newman-and-jenkins/
Related
I am trying to figure out how to run an A/B Test for a change on a Page Step for a Single Page. The idea is we have a payment flow with several page steps each containing a form. We'd like to swap out forms and test how our users react. We are trying to avoid changing the URL.
I looked into tools such as Google Analytics, but that requires a different URL to run the A/B test. The hesitation about creating a new URL is because our users are known to bookmark them, and we don't want to keep a backlog of redirects from invalid URLs, also we'd like to avoid constantly deploying new URLs for our tests.
I cannot seem to find any tool to do this, so I've tried to think of a few solutions but I'm not having a lot of luck.
My best idea is to build both a and b forms into the page, and when a user accesses the flow, the session randomly(based on a preset%) stores a value that dictates whether the user is in test a or b. Then when they step into that form, the server will serve the proper form to them. If they abandon their session, we'd track that, and if they complete the action, we'd track that.
I feel like there should be a better solution, but I just cannot come up with one.
My results online were either blogs showing how to approach it from a high level, and all of them used different URLs, I have found almost no developer resources.
Thanks.
We're using ExtJS 4.2.2, and .NET as our server.
Whenever you need the server to be involved, you need server-side instrumentation. No free tools offer that, but you could consider Optimizely "full-stack" (has support for C#) or Variant (does not yet).
I have a rest service which receives a customer's ID and an input, acording to this the services response something different, it's basically a menu, but the reponse could be different (depends on the customers input), so I need to stress this service to see how many request the server can handle and try to determine the max TPS, but since is a flow I don't know how I can simulate this, any idea or page that can be useful?
Thanks in advance for your help
What you mostly need is an understanding of how to handle dynamic parameters. Imagine you are simulating a user which goes on a blog, views a random blog article and then posts a comment about it.
It's all about designing a user which has a dynamic behavior, which changes depending on variables or server output. JMeter supports this kind of simulation very well by providing dozen of useful components like:
CSV Datasets,
Regexp Variable extractors,
and more.
We have written an article which explains how to simulate users with a dynamic behavior. It's very similar to what you would do in JMeter since OctoPerf is based on JMeter, with a Web UI built on top.
For our Web api project we use the following url versioning system:
https://{fqdn}/{apiVersion}/{apiResourceName}/{resourcePath}?{parameters}
for instance we can have something like the following:
https://myapi.mysite.com/v1/customer/2
Now considering above, let say you want to release two versions (live,test) to the customer. One live version (working with live data) and the other one is the test (working with test data for customer development test).
For live one I could easily use the one I mentioned : https://myapi.mysite.com/v1/customer/2 .
How do you name the test version of the above api? what is the test version of a api url version v1? Can specify the test api url?
Also what are best practices for fully qualified domain name of the API {fqdn} when using url versioning?
There are really several ways to do this.
One way, for instance, is to simply use attribute routing to give it a different path. Create a separate method, give it a path of /vtest/customer/2 for example and if users access this /vtest/ version (or v2 or 3 or whatever) then return the test data/new version. See an example in this question
Another way is to host your "test data" API in a different application in your server and have your web.config point to test versions of your database/source data. Using IIS, you'd configure two different applications (one for test, other for live) and the base URL would differ e.g.: https://myapi.mysite.com/appname1/v1/customer/2 vs https://myapi.mysite.com/appname2/v1/customer/2 and your appname could be something like live vs test. Have a look at this simple example
You could also just host them in different servers altogether, which would result in your {fqdn} changing between test and live versions (e.g. server.com/v1/customer/2 vs testserver.com/v1/customer/2) - this is what I do at my current job, and I find it very effective as it isolates live/test data (and API versions) avoiding confusion between them.
I also found this blog post detailing how to do this with namespaces
In other words there isn't just one best/correct way to do what you want, it all boils down to how you (or your company/boss/team) wants to structure and control test vs live data in your APIs. Take a look at these options to see which one is best in your case, hope I was able to help.
I think the title of your question is misleading. The problem you are trying to solve is not versioning (because your client is connecting to the same version of your application: v1). It is about having multiple environments: one for live data and one (or more) for test data.
At my company we solve this issue through the host name. At https://live.mysite.com/api/v1 we host v1 of the API connected to live data. At https://nodex.mysite.com/api/v1 we host v1 of the API connected to test data. Our clients can request new nodes as necessary (e.g. client1-devnode.mysite.com/api/v1 to develop against, and client1-testnode.mysite.com/api/v1 to test against. Each node get it's own set of test data.
Most of the live projects different server for different environments.
Instead of using different version of API endpoints, You should use different servers for different environment like this :
For Prod/live : https://myapi.mysite.com/v1/customer/2
For Test : https://myapi.mysitetest.com/v1/customer/2
For Dev : https://myapi.mysitedev.com/v1/customer/2
You need to configure environment specific properties for different backend endpoints you are hitting. Like : test.properties/dev.properties/live.properties
With my experience in API developing i found that there are 2 way of making server (test/developer)/live
I will show an example with your link type
https://{fqdn}/{apiVersion}/{apiResourceName}/{resourcePath}?{parameters}
In your case you can use or settings based and Link based testing type
What is settings based?
Settings based is that your server for example https://rest.mysite.com/v1/customer/2
will acting as test if you or your customer will set in he's settings server status to test and if as live - status to live.
This method is good in some cases but in order to test and to have live in same time,- this type not recommended.
What is link|URL|URI based?
This method have 2 types of identifying request is test or live
One way is to set test as a parameter https://api.mysite.com/test/v1/customer/2 and without test it goes to live
Second way is to set api to testApi or apiTest for example https://testapi.mysite.com/v1/customer/2 or https://apitest.mysite.com/v1/customer/2 . This way customer have both test and live and he can do testing and having live project too.
And don`t forget for security always check customer and verify before giving live api access.
As an option you may use custom defined header. If request contains custom header -> redirect request to test version of API.
I am very confused about the correct or recommended mechanism to use for accessing google fusion tables APIs in app scripts. There seem to be two methods with examples but no discussion about which is preferred or why. Is one of these interfaces newer and preferred while the other is dying? Is one obsolete or more restricted in what it can do?
Method 1 is the REST API described here
https://developers.google.com/fusiontables/docs/v2/sql-reference#Select
Method 2 is a set of library functions sort of described here under the Apps Script/Google Advanced Services:
https://developers.google.com/apps-script/advanced/fusion-tables
For example, using the REST api to do a dql query, we end up with something like this:
function runSQL(sql){
var getDataURL = 'https://www.googleapis.com/fusiontables/v1/query?sql='+sql;
var dataResponse = UrlFetchApp.fetch(getDataURL,getUrlFetchOptions()).getContentText();
return dataResponse;
}
And using the advanced API we use something like this:
result = FusionTables.Query.sql(sql, { hdrs: false });
The REST API seems much harder to use, requireing complex oAuth and developer keys to be configured in advance and coded into the application while the Advanced Services API harvests all this behind the scenes and makes for simple API calls like I show here.
I have seen numerous examples using each of the above with no hint as to why one author chose her mechanism instead of the other.
Your help is greatly appreciated.
The service within app-script is a work in progress, so the full functionality of the API might not be fully supported at the moment. As you mentioned though, the big advantage of the service over the REST API is that you do not have to handle the OAuth flow, as you only need to enable it on your script (as stated here).
The Apps Script "advanced service" implementation still lacks some advanced functionality (like alt=media format queries or multipart / resumable uploads) -- if it actually has those features, it lacks extremely basic documentation of them, to the point that the Apps Script editor autocomplete is unaware of them. The tradeoff of these functionality gaps is that you don't need to handle keys, request building, etc.
So, if you're doing simple sql select / importRows work, the Advanced Service should be able to cover almost all your needs. If you need to delete from your FusionTables, you might want to consider setting up the REST API - because deleting is 1 record per query, the better way to delete is to instead "download what you want to keep, then re-upload it back via replaceRows."
(This worked for me for a while, but eventually what I was keeping outgrew the Apps Script service's limitations and I began receiving Empty Response errors from the call to replaceRows. My remedy was to perform my record maintenance tasks via the REST API, where I can specify resumable uploads, timeouts, etc., while more "normal" interactions are done through the Advanced Service.)
So I am in need to learn how to use soapUI pretty quick. I'm finding it pretty tedious to start so I was hoping I might be able to get some help here. Here's what I need to do.
Lets say we have Company A and Company B which is a subset of Company B. Now Company A offers a webservice accessible by Company B such that Company B can gather daily aggregated data from Company A's database. Now Company B wants to take this data and publish it on their website.
What I'd like is a very basic overview of what I need to do to extract and parse the data onto a website. Just the outline of the process so I can get started.
What languages should I be using at what stages and what not.
Any help would be highly appreciated.
This isn't a SoapUI question, rather, you need to get started with SOA/SOAP. That's fine, and SoapUI is a good way to figure it out.
The SOAP webservice published by company A will have "methods" that allow access to data, or let you add/update data. Suppose they're a used car registry. They may have methods such as GetCarList, GetCarData, SubmitBid, etc.. You call those via SOAP, and get a response. Both the request and response are XML.
CompanyA will publish a "wsdl" that describes the interface. Plug that into SoapUI, and you should be able to make sample requests. There will probably be empty fields, with ? in them, which are placeholders for data (carID, VIN, MaxRecords, etc..). See if you can modify the request enough to satisfy the server, press the GO button, and see what you get back. If you're lucky, you'll get data back, and you're on your way.
When you get down to building your app, you can pretty much use the language of your choice. .Net, PHP, Java, native C, even Delphi... The SOAP is designed to be platform/language agnostic. If you can talk via HTTP/HTTPS, you should be in business.