I am trying to use celery to create a task management system, where the tasks are written in phantomjs. Basically the task is to crawl a URL and return a JSON object (that I'll want to use once the task is complete). There will be a list of URLs which will each have it's own crawl task.
I looked at this http://docs.celeryproject.org/en/latest/userguide/remote-tasks.html#calling-webhook-tasks to implement tasks in another language. However celery does not provide very good documentation at all on how the external webhook script should be written, what dependencies are required, and how it is run.
Should the task simply be stored on a server, which I will call with the HTTPCallback? How is it run in the PhantomJS framework. How are results from here stored or returned and how do I keep track of it? I have read a lot of docs and I still cannot find the exact interface or API for the other side of the webhook API.
I realize my question may not be too clear, so let me know, otherwise some good examples of this HTTPCallback API workflow would be helpful for a basic understanding, and how the webhook script in a different language is structured.
Any insight would be greatly appreciated. Thanks a lot!
Related
I have created a web application that will have to operate under a heavy client load, so I was planning to test it out with automated services, yet I don't seem to be able to find any. In essence, I am looking for a way to imitate real requests from various locations to my server. WebPageTest seems to do the job, but I can only run it once. Is there a way I can automate it to run every so many seconds?
TIA
There is a WebPageTest API (https://github.com/WPO-Foundation/webpagetest-docs/blob/master/dev/api.md) which you could use to automate these requests using a CRON job.
There are also off-the-shelf solutions that do this, such as SpeedCurve (paid service).
When doing async things (like creating a Project) with the Resource Manager API, the initial call returns an Operation struct, which you can then poll using a get API to check if the operation completed and if there was an error. However while many other APIs, and it's part of the standard long running operations prototype, the Resource Manager API doesn't seem to offer this? This is almost certainly just a case of "vendor didn't think anyone would need it", but throwing up a question in case someone knows a tricky way to list all the pending RM operations.
I am slightly confused because it seems like the web console for Google Cloud does show a list of operations, but I can't quite figure out where those come from. Maybe there is non-public API for it hidden away somewhere?
Got a response from Google that an official API is on their list but no ETA, and there is no workaround.
Also confirmed via network trace that the Cloud Console does indeed use its own magic API that can read all Operations types from every API at the same time, including project creation. But it's very specific to the needs of the console.
I am looking for a way to discover a USB zstick "/dev/ttyACM0" using OpenHab command line interface as I am not going to be using a GUI and I need to create a script to automate few things.
In their website here and here OpenHab have some documentation, and that's what I found for discovery:
GET /discovery
POST /discovery/bindings/{bindingId}/scan
however I was unable to make that into a useful curl command. any idea on where to start?
I would suggest to try out the REST Documentation, for finding the right endpoint/expression.
You can install it through paper ui:
It will then be available through <YOUR_OH_IP>:<PORT>/doc/index.html.
It provides some ui with explanations and examples for the endpoints and you can test them live through it.
This should help you find the correct rest call to use with curl afterwards.
I don't know if adding devies through REST is possible. If so you can find it in the REST Documentation.
If it doesn't work though the REST API you might want to write a small bash script which adds the things you want. Then you can call your script with
executeCommandLine in a rule
I am very confused about the correct or recommended mechanism to use for accessing google fusion tables APIs in app scripts. There seem to be two methods with examples but no discussion about which is preferred or why. Is one of these interfaces newer and preferred while the other is dying? Is one obsolete or more restricted in what it can do?
Method 1 is the REST API described here
https://developers.google.com/fusiontables/docs/v2/sql-reference#Select
Method 2 is a set of library functions sort of described here under the Apps Script/Google Advanced Services:
https://developers.google.com/apps-script/advanced/fusion-tables
For example, using the REST api to do a dql query, we end up with something like this:
function runSQL(sql){
var getDataURL = 'https://www.googleapis.com/fusiontables/v1/query?sql='+sql;
var dataResponse = UrlFetchApp.fetch(getDataURL,getUrlFetchOptions()).getContentText();
return dataResponse;
}
And using the advanced API we use something like this:
result = FusionTables.Query.sql(sql, { hdrs: false });
The REST API seems much harder to use, requireing complex oAuth and developer keys to be configured in advance and coded into the application while the Advanced Services API harvests all this behind the scenes and makes for simple API calls like I show here.
I have seen numerous examples using each of the above with no hint as to why one author chose her mechanism instead of the other.
Your help is greatly appreciated.
The service within app-script is a work in progress, so the full functionality of the API might not be fully supported at the moment. As you mentioned though, the big advantage of the service over the REST API is that you do not have to handle the OAuth flow, as you only need to enable it on your script (as stated here).
The Apps Script "advanced service" implementation still lacks some advanced functionality (like alt=media format queries or multipart / resumable uploads) -- if it actually has those features, it lacks extremely basic documentation of them, to the point that the Apps Script editor autocomplete is unaware of them. The tradeoff of these functionality gaps is that you don't need to handle keys, request building, etc.
So, if you're doing simple sql select / importRows work, the Advanced Service should be able to cover almost all your needs. If you need to delete from your FusionTables, you might want to consider setting up the REST API - because deleting is 1 record per query, the better way to delete is to instead "download what you want to keep, then re-upload it back via replaceRows."
(This worked for me for a while, but eventually what I was keeping outgrew the Apps Script service's limitations and I began receiving Empty Response errors from the call to replaceRows. My remedy was to perform my record maintenance tasks via the REST API, where I can specify resumable uploads, timeouts, etc., while more "normal" interactions are done through the Advanced Service.)
I want to generate simple document approval system in workflow foundation. We have already available document approval process example on MSDN but it is quite complex to understand as it involves WCf and all. I just want a startup with workflow foundation so it's better if I follow step by step guide tutorial. Thanks in advance.
I wrote a blog post on how to implement a simple approval workflow using emails and ASP.Net MVC.
http://blog.petegoo.com/index.php/2010/08/22/workflow-4-email-approval-sample/
I think http://visualstudiogallery.msdn.microsoft.com/7fe6f504-a58d-456e-8f55-e64bddc81a41 is a good tutorial to start with. It is better if you try this with state machine type of workflow because you need several user inputs from outside the workflow in several stages.