...just wanted to confirm that since the latest soundcloud api does not provide a data interface, we are left with parsing the results from an http request.
my concern is that the resulting structure could change at anytime and thus render my parsing schema invalid. is anyone else doing something similar? or better?
This is correct. All SoundCloud API responses will be serialized as JSON or XML. We take backwards compatibility pretty seriously, so you can rely on the format and data returned.
Most languages have at least one library capable of automatically parsing JSON into an appropriate data type (i.e. an array of hashes). You can always check to ensure a key exists before you try to access it, for example in Python:
import json
import urllib
url = 'https://api.soundcloud.com/tracks.json'
fp = urllib.urlopen('%s?%s' % (url, urllib.urlencode({
'client_id': 'YOUR_CLIENT_ID',
'limit': 2
})))
data = fp.read()
tracks = json.loads(data)
for track in tracks:
print track.get('title', 'No title available')
Does that help answer your question?
Related
I have a simple API that return Something for a given ID and it must be used without any kind of authentication, the URL should be permanent and yet I want to avoid as much as possible it to be botted.
The Url is something like this:
https://url/{SomeId}/doSomething
The problem is that this is very predicable and a bot could easily try all the ID and get everything associated to it.
I'm looking for a way to make the URL non predictable like for example:
https://url/{SomeId}/doSomething?Key=SomeVeryLongCryptographicKey
In this way except if you run a supercalculator you shouldn't be able to deduce all the URLs
I know that there is probably a lot of ways to do that, like using a DB which I want to avoid.
I guess I'm looking for a kind a JWT associated to the ID without expiration but maybe there is better solution.
Edit: to give a better example i'm looking to do a bit like did Zoom for permanent invitation Links. They had predictable room Ids and they added a password making it unpredictable lie so:
https://us05web.zoom.us/j/132465789?pwd=SUxIU0pLankyhTRtWmlGbFlNZ21Ndz08
What would be the best/light/"secure" way to achieve that ?
Also I'm using .Net if there is a library doing that, it would be great.
I think your idea of using a JWT makes the most sense. Better to use something standard from a cryptographic point of view, and the JSON format allows for encoding whatever you need to provide to the receiving endpoint (user names, entity names, entity IDs, other things).
There are standard Microsoft libraries for building and validating JWTs, but I prefer the library Jwt.Net (https://www.nuget.org/packages/JWT). It lets you do something like this quite easily:
var token = JwtBuilder()
.WithAlgorithm(new RS256Algorithm(publicKey,privateKey))
.AddClaim("uri", String.Format("https://example.com/api/{0}/{1}", entityName, entityId))
.Encode();
Just add whatever claims you like, and the JWT will then contain what you want to transfer (I've used an example of the URI that you want to give to the entity) and a signature with your private key. You could even just give a URL like https://example.com/from_token/eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1cmkiOiJodHRwczovL2V4YW1wbGUuY29tL2FwaS90ZXN0RW50aXR5LzEyMzQifQ.K2P4wSF6g1Kt-IHMzlklWTV09_MIkoiNHQztSIYOohmOWe7aBfFSQLIKSbdTECj9XPjNNG_AjH9fdjFglkPuYfr2G9rtl2eh5vTjwdM-Uc0X6RkBu0Z2j7KyMKjkaI3zfkIwhtL1mH873xEBtNOGOW18fuBpgnm8zhMAj1oD3PlDW8-fYBrfLb6VK97DGh_DyxapbksgUeHst7cAGg3Nz97InDPtYcWDi6lDuVQsj3t4iaJBRL8IM785Q8xjlHHhzdfcX3xU4IhflyNHHXxP56_8ahNNMOZKWdwgbTSIxEEB98b3naY3XknC-ea7Nc1y4_5fszrYdy3LaQWV43jpaA
and have the handler decode the entity name and ID you want to retrieve directly from the URI while verifying the signature. Decoding with the library is just as easy:
var json = JwtBuilder.Create()
.WithAlgorithm(new RS256Algorithm(_key))
.MustVerifySignature()
.Decode(token);
I'm currently working on ng-admin.
I'm having a problem retrieving user data from my REST API (connected to a MongoDB) and displaying it.
I have identified the problem as the following:
When I enter http://localhost:3000/users into my browser, I get a list of all users in my database.
When I enter http://localhost:3000/users?_page=1&_perPage=30&_sortDir=DESC&_sortField=id,
I get [] as a result.
I am quite new to this, I used both my browser and the POSTMAN Chrome extension to test this and get the same result.
http://localhost:3000/users_end=30&_order=DESC&_sort=id&_start=0
This (/users_end) is a different request than /users.
It should be:
http://localhost:3000/users?end=30&_order=DESC&_sort=id&_start=0
Or, by looking at the other parameters:
http://localhost:3000/users?_end=30&_order=DESC&_sort=id&_start=0
with end or _end being the first parameter (mark the ?).
Update (it is ? and before the _, I have edited.):
If adding parameters to the request returns an empty list, try adding only one at a time to narrow down the problem (there's probably an error in the usage of those parameters - are you sure you need those underscores?).
Your REST API must have a way to handle pagination, sorting, and filtering. But ng-admin cannot determine exactly how, because REST is a style and not a standard. So ng-admin makes assumptions about how your API does that by default, that's why it adds these _end and _sort query parameters.
In order to transform these parameters into those that your API understands, you'll have to add an interceptor. This is all thoroughly explained in the ng-admin documentation: http://ng-admin-book.marmelab.com/doc/API-mapping.html
I am building a REST API and facing this issue: How can REST API pass very large JSON?
Basically, I want to connect to Database and return the training data. The problem is in Database I have 400,000 data. If I wrap them into a JSON file and pass through GET method, the server would throw Heap overflow exception.
What methods we can use to solve this problem?
DBTraining trainingdata = new DBTraining();
#GET
#Produces("application/json")
#Path("/{cat_id}")
public Response getAllDataById(#PathParam("cat_id") String cat_id) {
List<TrainingData> list = new ArrayList<TrainingData>();
try {
list = trainingdata.getAllDataById(cat_id);
Gson gson = new Gson();
Type dataListType = new TypeToken<List<TrainingData>>() {
}.getType();
String jsonString = gson.toJson(list, dataListType);
return Response.ok().entity(jsonString).header("Access-Control-Allow-Origin", "*").header("Access-Control-Allow-Methods", "GET").build();
} catch (SQLException e) {
logger.warn(e.getMessage());
}
return null;
}
The RESTful way of doing this is to create a paginated API. First, add query parameters to set page size, page number, and maximum number of items per page. Use sensible defaults if any of these are not provided or unrealistic values are provided. Second, modify the database query to retrieve only a subset of the data. Convert that to JSON and use that as the payload of your response. Finally, in following HATEOAS principles, provide links to the next page (provided you're not on the last page) and previous page (provided you're not on the first page). For bonus points, provide links to the first page and last page as well.
By designing your endpoint this way, you get very consistent performance characteristics and can handle data sets that continue to grow.
The GitHub API provides a good example of this.
My suggestion is no to pass the data as a JSON but as a file using multipart/form-data. In your file, each line could be a JSON representing a data record. Then, it would be easy to use a FileOutputStream to receive te file. Then, you can process the file line by line to avoid memory problems.
A Grails example:
if(params.myFile){
if(params.myFile instanceof org.springframework.web.multipart.commons.CommonsMultipartFile){
def fileName = "/tmp/myReceivedFile.txt"
new FileOutputStream(fileName).leftShift(params.myFile.getInputStream())
}
else
//print or signal error
}
You can use curl to pass your file:
curl -F "myFile=#/mySendigFile.txt" http://acme.com/my-service
More details on a similar solution on https://stackoverflow.com/a/13076550/2476435
HTTP has the notion of chunked encoding that allows you send a HTTP response body in smaller pieces to prevent the server from having to hold the entire response in memory. You need to find out how your server framework supports chunked encoding.
Quick problem here that I don't know how to solve and thought you guys could give me a heads-up on which way to go. I have successfully pulled all my connection data using the LinkedIn REST API both in XML and JSON and dumped them (the former using the cPickle plugin). The issue is that I need to refer to a single field within the data and so decided to use XML as it seemed to be the easiest one to use by far. When I refer to the specific field in my .pickle file, it gives me the following error:
AttributeError: 'str' object has no attribute 'location'
However, opening the pickle file with notepad, I can see that all my connections do have their location field stored in the XML format. It's very strange!
Here's my referral code:
import cPickle
connections_data = 'linkedin_connections.pickle'
response = cPickle.load(open(connections_data))
print response
locations = [ec.location for ec in response]
I have a print function set up to show what's in my file and all of the data appears as a normal XML output using the people call of the REST API. The XML data appears as follows:
<person>
<id>ID_number</id>
<first-name>blah</first-name>
<last-name>blah</last-name>
<headline>Business Development and Sales Executive at Computaris</headline>
<picture-url>picture_url</picture-url>
<api-standard-profile-request>
<url>profile_request</url>
<headers total="1">
<http-header>
<name>x-li-auth-token</name>
<value>name</value>
</http-header>
</headers>
</api-standard-profile-request>
<site-standard-profile-request>
<url>request_url</url>
</site-standard-profile-request>
<location>
<name>location</name>
<country>
<code>country_code</code>
</country>
</location>
<industry>industry</industry>
Any help will be much appreciated.
I am trying to use HTTP to POST a file to an outside API from within a grails service. I've installed the rest plugin and I'm using code like the following:
def theFile = new File("/tmp/blah.txt")
def postBody = [myFile: theFile, foo:'bar']
withHttp(uri: "http://picard:8080/breeze/project/acceptFile") {
def html = post(body: postBody, requestContentType: URLENC)
}
The post works, however, the 'myFile' param appears to be a string rather than an actual file. I have not had any success trying to google for things like "how to post a file in grails" since most of the results end up dealing with handling an uploaded file from a form.
I think I'm using the right requestContentType, but I might have missed something in the documentation.
POSTing a file is not as simple as what you have included in your question (sadly). Also, it depends on what the API you are calling is expecting, e.g. some API expect files as base64 encoded text, while others accept them as mime-multipart.
Since you are using the rest plugin, as far as I can recall it uses the Apache HttpClient, I think this link should provide enough info to get you started (assuming you are dealing with mime-multipart). It shouldn't be too hard to change it around to work with your API and perhaps make it a bit 'groovy-ier'