I am trying to use HTTP to POST a file to an outside API from within a grails service. I've installed the rest plugin and I'm using code like the following:
def theFile = new File("/tmp/blah.txt")
def postBody = [myFile: theFile, foo:'bar']
withHttp(uri: "http://picard:8080/breeze/project/acceptFile") {
def html = post(body: postBody, requestContentType: URLENC)
}
The post works, however, the 'myFile' param appears to be a string rather than an actual file. I have not had any success trying to google for things like "how to post a file in grails" since most of the results end up dealing with handling an uploaded file from a form.
I think I'm using the right requestContentType, but I might have missed something in the documentation.
POSTing a file is not as simple as what you have included in your question (sadly). Also, it depends on what the API you are calling is expecting, e.g. some API expect files as base64 encoded text, while others accept them as mime-multipart.
Since you are using the rest plugin, as far as I can recall it uses the Apache HttpClient, I think this link should provide enough info to get you started (assuming you are dealing with mime-multipart). It shouldn't be too hard to change it around to work with your API and perhaps make it a bit 'groovy-ier'
Related
I'm using Grails 2.3.2 working on a REST API using the built in Grails REST support. I'm having trouble getting rid of the "class" element in the JSON response. Based on a tutorial by Bobby Warner, I have found adding the following to the resources.groovy file:
meterRenderer(JsonRenderer, Meter) {
excludes = ['class']
}
This works fine for show, but for the index controller function, I'm responding with a list of Meters. In this, the "class" doesn't go away. What does it take to get rid of this in the list response?
Edit: To clarify, I am looking for a way to leverage the Content Negotiation feature of Grails new respond functionality without locking myself down to render as JSON implementions.
I guess if you switch to using GSON (github) instead of JSON then you need not worry about that particular exclusion.
That is driven by a config setting provided by the plugin as grails.converters.gson.domain.include.class (default is false).
nickdos' SO link had the answer. I added the following to my BootStrap.groovy:
grails.converters.JSON.registerObjectMarshaller(Meter) {
return it.properties.findAll {k,v -> k != 'class'}
}
And the respond call results in no "class" item. Oddly enough, I lost the "id" item in the process, but I'll save that for another SO question. :)
I have been trying to set up a sample AngularJS app with webMethods Integration Server on the backend. Using $resource, I can easily pull normal JSON files and manipulate the data within the file. However, the goal is that I want to create services in webMethods Designer and call them from AngularJS using $resource to display the data in my app. The problem is that from AngularJS I cannot extract the data I need from the service that I'm creating in Designer. In Designer I can use (in WMPublic) documentToJSONString, and output something like:
jsonString {"id":"1", "name":"Dan", "quantity":"3"}
But I cannot extract the data because this is not a pure JSON string. Does anyone know how to (1) extract the JSON string output data using AnularJS or (2) output a JSON document from Designer? I am calling a REST service; something to the effect of
http://localhost:2222/rest/Get/getOrderData
from my services.js file in AngularJS.
Here is my services.js file:
/* Services */
var orderServices = angular.module('orderServices', ['ngResource']);
orderServices.factory('Order', ['$resource',
function($resource){
return $resource('http://localhost:2222/rest/REST/getOrderData', {}, {
query: {method:'GET', isArray:true}
});
}]);
Then, in my app, I want to use an ng-repeat to call things like {{order.id}}, {{order.name}} etc. Is anyone good with webMethods and Angular or done this before?
To force the response that you want, I would have used the service
pub.flow:setResponse mapping the jsonString to it's string parameter and probably hardcoded (eww!) the contentType parameter to 'application/json'
You may also need to use the service pub.flow:setResponseCode to set the response code.
They would be the last services in getOrderData
I would have invoked it using the below (where namespace is the folder structure in designer)
http://localhost:2222/invoke/namespace:getOrderData
The above applies to Integration Server V8 and it looks like you're using V9 since some of the services that you mention didn't exist in V8. This would also apply to a normal flow service, not a specific REST one (assuming they exist in V9).
I'm testing out the new API, but having no luck downloading a test image file. The file exists, is accessible through the web UI, and is retrievable using the v1.0 API.
I'm able to access the metadata ("https://api.box.com/2.0/files/{fileid}") using both commandline curl and pycurl. However, calls to "https://api.box.com/2.0/files/{fileid}/data" bring back nothing. An earlier post (5/1) received the answer that the download feature had a bug and that "https://www.box.com" should be used as the base URL in the interim. That, however, just provokes a 404.
Please advise.
You should be able to download via http://api.box.com/2.0/files/<fildID>/content ... Looks like we have a bug somewhere in the backend. Hope to have it fixed soon.
Update 11/13/2012 -- This got fixed at least a month ago. Just updated the URL to our newer format
For me it works when its /content instead of /data... python code below
import requests
fileid = str(get_file_id(filenumber))
url = https://api.box.com/2.0/files/1790744170/content
r = requests.get(url=url, headers=<HEADERS>, proxies=<PROXIES>)
infoprint("Downloading...")
filerecieved = r.content
filename = uni_get_id(fileid, "name", "file")
f = open(filename, 'w+')
infoprint("Writing...")
f.write(filerecieved)
f.close()
Does anyone have a definitive way to post to a user's wall, using nothing but the .NET Framework, or Silverlight?
Problems deriving from people's attempts have been asked here on SO, but I cannot find a full, clear explanation of the Graph API spec and a simple example using WebClient or some similar class from System.Net.
Do I have to send all feed item properties as parameters in the query string? Can I construct a JSON object to represent the feed item and send that (with the access token as the only parameter)?
I expect its no more than a 5 line code snippet, else, point me at the spec in the FB docs.
Thanks for your help,
Luke
This is taken from how we post to a user's wall. We place the data for the post in the request body (I think we found this to be more reliable than including all the parameters in the query part of the request), it has the same format as a URL encoded query string.
I agree that the documentation is rather poor at explaining how to interact with a lot of resources. Typically I look at the documentation for information on fields and connections, then work with the Graph API Explorer to understand how the request needs to be constructed. Once I've got that down it's pretty easy to implement in C# or whatever. The only SDK I use is Facebook's Javascript SDK. I've found the others (especially 3rd party) are more complicated, buggy, or broken than rolling my own.
private void PostStatus (string accessToken, string userId)
{
UriBuilder address = new UriBuilder ();
address.Scheme = "https";
address.Host = "graph.facebook.com";
address.Path = userId + "/feed";
address.Query = "access_token=" + accessToken;
StringBuilder data = new StringBuilder ();
data.Append ("caption=" + HttpUtility.UrlEncodeUnicode ("Set by app to describe the app."));
data.Append ("&link=" + HttpUtility.UrlEncodeUnicode ("http://example.com/some_resource_to_go_to_when_clicked"));
data.Append ("&description=" + HttpUtility.UrlEncodeUnicode ("Message set by user."));
data.Append ("&name=" + HttpUtility.UrlEncodeUnicode ("App. name"));
data.Append ("&picture=" + HttpUtility.UrlEncodeUnicode ("http://example.com/image.jpg"));
WebClient client = new WebClient ();
string response = client.UploadString (address.ToString (), data.ToString ());
}
I don't know much about .net or silverlight, but the facebook api works with simple http requests.
All the different sdks (with the exception of the javascript one) are mainly just wrappers for the http requests with the "feature" of adding the access token to all requests.
Not in all requests the parameters are sent as querystring, in some POST requests you need to send them in the request body (application/x-www-form-urlencoded), and you can not send the data as json.
If the C# sdk is not to your liking, you can simply create one for your exact needs.
As I wrote, you just need to wrap the requests, and you can of course have a method that will get a json as parameter and will break it to the different parameters to be sent along with the request.
I would point you to the facebook documentation but you haven't asked anything specific so there's nothing to point you to except for the landing page.
What I'm Trying To Do
I'm trying to create a solution of any kind that will run nightly on a Windows server, authenticate to a website, check a web page on the site for new links indicating a new version of a zip file, use new links (if present) to download a zip file, unzip the downloaded file to an existing folder on the server, use the unzipped contents (sql scripts, etc.) to build an instance of a database, and log everything that happens to a text file.
Forms App: The Part That Sorta Works
I created a Windows Forms app that uses a couple of WebBrowser controls, a couple of threads, and a few timers to do all that except the running nightly. It works great as a Form when I'm logged in and run it, but I need to get it (or something like it) to run on it's own like a Service or scheduled task.
My Service Attempt
So, I created a Windows Service that ticks every hour and, if the System.DateTime.Now.Hour >= 22, attempts to launch the Windows Forms app to do it's thing. When the Service attempts to launch the Form, this error occurs:
ActiveX control '8856f961-340a-11d0-a96b-00c04fd705a2' cannot be instantiated because the current thread is not in a single-threaded apartment.
which I researched and tried to resolve by either placing the [STAThread] attribute on the Main method of the Service's Program class or using some code like this in a few places including the Form constructor:
webBrowseThread = new Thread(new ThreadStart(InitializeComponent));
webBrowseThread.SetApartmentState(ApartmentState.STA);
webBrowseThread.Start();
I couldn't get either approach to work. In the latter approach, the controls on the Form (which would get initialized inside IntializeComponent) don't get initialized and I get null reference exceptions.
My Scheduled Task Attempt
So, I tried creating a nightly scheduled task using my own credentials to run the Form locally on my dev machine (just testing). It gets farther than the Service did, but gets hung up at the File Download Dialog.
Related Note: To send the key sequences to get through the File Download and File Save As dialogs, my Form actually runs a couple of vbscript files that use WScript.Shell.SendKeys. Ok, that's embarassing to admit, but I tried a few different things including SendMessage in Win32 API and referencing IWshRuntimeLibrary to use SendKeys inside my C# code. When I was researching how to get through the dialogs, the Win32 API seemed to be the recommended way to go, but I couldn't figure it out. The vbscript files was the only thing I could get to work, but I'm worried now that this may be the reason why a scheduled task won't work.
Regarding My Choice of WebBrowser Control
I have read about the System.WebClient class as an alternative to the WebBrowser control, but at a glance, it doesn't look like it has what I need to get this done. For example, I needed (or I think I needed) the WebBrowser's DocumentCompleted and FileDownload events to handle the delays in pages loading, files downloading, etc. Is there more to WebClient that I'm not seeing? Is there another class besides WebBrowser that is more Service-friendly and would do the trick?
In Summary
Geez, this is long. Sorry! It would help to even have a high level recommendation for a better way to do what I'm trying to do, because nothing I've tried has worked.
Update 10/22/09
Well, I think I'm closer, but I'm stuck again. I should end up with a decent-sized zip file with several files in it, but the zip file resulting from my code is empty. Here's my code:
// build post request
string targetHref = "http://wwwcf.nlm.nih.gov/umlslicense/kss/login.cfm";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(targetHref);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
// encoding to use
Encoding enc = Encoding.GetEncoding(1252);
// build post string containing authentication information and add to post request
string poststring = "returnUrl=" + fixCharacters(targetDownloadFileUrl);
poststring += getUsernameAndPasswordString();
poststring += "&login2.x=0&login2.y=0";
// convert to required byte array
byte[] postBytes = enc.GetBytes(poststring);
request.ContentLength = postBytes.Length;
// write post to request
Stream postStream = request.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length);
postStream.Close();
// get response as stream
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
// writes stream to zip file
FileStream writeStream = new FileStream(fullZipFileName, FileMode.Create, FileAccess.Write);
ReadWriteStream(responseStream, writeStream);
response.Close();
responseStream.Close();
The code for ReadWriteStream looks like this.
private void ReadWriteStream(Stream readStream, Stream writeStream)
{
// taken verbatum from http://www.developerfusion.com/code/4669/save-a-stream-to-a-file/
int Length = 256;
Byte[] buffer = new Byte[Length];
int bytesRead = readStream.Read(buffer, 0, Length);
// write the required bytes
while (bytesRead > 0)
{
writeStream.Write(buffer, 0, bytesRead);
bytesRead = readStream.Read(buffer, 0, Length);
}
readStream.Close();
writeStream.Close();
}
The building of the post string is taken from my previous forms app that works. I compared the resulting values in poststring for both sets of code (my working forms app and this one) and they're identical.
I'm not even sure how to troubleshoot this further. Anyone see anything obvious as to why this isn't working?
Conclusion 10/23/09
I finally have this working. A couple of important hurdles I had to get over. I had some problems with the ReadWriteStream method code that I got online. I don't know why, but it wasn't working for me. A guy named JB in Claudio Lassala's Virtual Brown Bag meeting helped me to come up with this code which worked much better for my purposes:
private void WriteResponseStreamToFile(Stream responseStreamToRead, string zipFileFullName)
{
// responseStreamToRead will contain a zip file, write it to a file in
// the target location at zipFileFullName
FileStream fileStreamToWrite = new FileStream(zipFileFullName, FileMode.Create);
int readByte = responseStreamToRead.ReadByte();
while (readByte != -1)
{
fileStreamToWrite.WriteByte((byte)readByte);
readByte = responseStreamToRead.ReadByte();
}
fileStreamToWrite.Flush();
fileStreamToWrite.Close();
}
As Will suggested below, I did have trouble with the authentication. The following code is what worked to get around that issue. A few comments inserted addressing key issues I ran into.
string targetHref = "http://wwwcf.nlm.nih.gov/umlslicense/kss/login.cfm";
HttpWebRequest firstRequest = (HttpWebRequest)WebRequest.Create(targetHref);
firstRequest.AllowAutoRedirect = false; // this is critical, without this, NLM redirects and the whole thing breaks
// firstRequest.Proxy = new WebProxy("127.0.0.1", 8888); // not needed for production, but this helped in order to debug the http traffic using Fiddler
firstRequest.Method = "POST";
firstRequest.ContentType = "application/x-www-form-urlencoded";
// build post string containing authentication information and add to post request
StringBuilder poststring = new StringBuilder("returnUrl=" + fixCharacters(targetDownloadFileUrl));
poststring.Append(getUsernameAndPasswordString());
poststring.Append("&login2.x=0&login2.y=0");
// convert to required byte array
byte[] postBytes = Encoding.UTF8.GetBytes(poststring.ToString());
firstRequest.ContentLength = postBytes.Length;
// write post to request
Stream postStream = firstRequest.GetRequestStream();
postStream.Write(postBytes, 0, postBytes.Length); // Fiddler shows that post and response happen on this line
postStream.Close();
// get response as stream
HttpWebResponse firstResponse = (HttpWebResponse)firstRequest.GetResponse();
// create new request for new location and cookies
HttpWebRequest secondRequest = (HttpWebRequest)WebRequest.Create(firstResponse.GetResponseHeader("location"));
secondRequest.AllowAutoRedirect = false;
secondRequest.Headers.Add(HttpRequestHeader.Cookie, firstResponse.GetResponseHeader("Set-Cookie"));
// get response to second request
HttpWebResponse secondResponse = (HttpWebResponse)secondRequest.GetResponse();
// write stream to zip file
Stream responseStreamToRead = secondResponse.GetResponseStream();
WriteResponseStreamToFile(responseStreamToRead, fullZipFileName);
responseStreamToRead.Close();
sl.logScriptActivity("Downloading update.");
firstResponse.Close();
I want to underscore that setting AllowAutoRedirect to false on the first HttpWebRequest instance was critical to the whole thing working. Fiddler showed two additional requests that occurred when this was not set, and it broke the rest of the script.
You're trying to use UI controls to do something in a windows service. This will never work.
What you need to do is just use the WebRequest and WebResponse classes to download the contents of the webpage.
var request = WebRequest.Create("http://www.google.com");
var response = request.GetResponse();
var stream = response.GetResponseStream();
You can dump the contents of the stream, parse the text looking for updates, and then construct a new request for the URL of the file you want to download. That response stream will then have the file, which you can dump on the filesystem and etc etc.
Before you wonder, GetResponse will block until the response returns, and the stream will block as data is being received, so you don't need to worry about events firing when everything has been downloaded.
You definitely need to re-think your approach (as you've already begun to do) to eliminate the Forms-based application approach. The service you're describing needs to operate with no UI at all.
I'm not familiar with the details of System.WebClient, but since it
provides common methods for sending
data to and receiving data from a
resource identified by a URI,
it will probably be your answer.
At first glance, WebClient.DownloadFile(...) or WebClient.DownloadFileAsync(...) will do what you need.
The only thing I can add is that once you have scraped your screen and have the fully qualified name of the file you want to download, you could pass it along to the Windows/DOS command 'get' which will fetch files via HTTP. You can also script a command-line FTP client if desired. It's been a long time since I tried something like this in Windows, but I think you're almost there. Once you have fetched the correct file, building a batch file to do everything else should be pretty easy. If you are more comfortable with Unix, google "unix services for windows" just keep an eye on the services they start running (DHCP, etc). There are some nice utilities which will let your treat dos as a unix-like shell (ls -l, grep, etc) Finally, you could try another language like Perl or Python but I don't think that's the kind of advice you were looking for. :)