New to TCL and having an issue with using the ::rest::simple url query ?config? ?body? command - specifically getting basic authentication to work. The example given here (https://core.tcl-lang.org/tcllib/doc/tcllib-1-18/embedded/www/tcllib/files/modules/rest/rest.html#section4) is as follows:
set url http://twitter.com/statuses/update.json
set query [list status $text]
set res [rest::simple $url $query {
method post
auth {basic user password}
format json
}]
So my attempt is:
package require rest
package require json
set url http://0.0.0.0:5000/api/id
set response [rest::simple $url {
method get
auth {basic user password}
format json
}]
puts $response
However, I keep getting a 401 error when I try and run the above against a mock API endpoint for GET:
"GET /api/id?auth=basic%20user%20password&method=get&format=json HTTP/1.1" 401 -
I can make a curl request against that same endpoint using basic auth (with Python as well), and if I disable basic auth on the endpoint this works just fine in TCL:
set url http://0.0.0.0:5000/api/id
set response [rest::simple $url {
method get
format json
}]
puts $response
So it's something to do with the basic auth credentials in the TCL rest module.
Thanks to Shawn's comment pointing out I was misreading the meaning of ? in TCL docs. Parameters surrounded by question marks are optional, rather than parameters followed by question marks. I was interpreting ::rest::simple url query ?config? ?body? as meaning the query param was optional. If there is no query, you can use an empty query as the required parameter. This ended up working:
set response [rest::simple $url {} {
method get
auth {basic user password}
format json
}]
Related
I need to extract data from an external REST API inside an Airflow DAG. The API is protected so I need to first authenticate/login to the API as a user, and then extract data by passing an access_token in the API call. Need some help in implementing this functionality. If anyone has ever done anything similar or some example would really help. Thanks
It will be better that you do it in python code as far as calling the rest api. This python code can take argument such as a filename with timestamp and then dump the data to that file. So this can be tested independently outside of airflow. It would be testing just like a regular python code.
Then call this python code/file with a BashOperator. In DAG, it will look something like this. Note, filename is hard-coded but that can be replaced with Jinja templates
that can return a filename with timestamp.
task1 = BashOperator(task_id='get_data',
bash_command="python ~/airflow/dags/src/rest_api_call.py data_20220701_010101.txt")
task2 = PythonOperator(
task_id='load_data',
provide_context=True,
python_callable=load_data_fn,
op_kwargs={ 'filename': 'data_20220701_010101.txt'},
dag=dag)
...
def load_data_fn(**kwargs):
print(kwargs.get("filename"))
...
Create http connection (in the UI under admin->connections)
In your DAG you should create 2 SimpleHttpOperator. The first task will login and read the access-token, the second one would send a request to get the data and in the header you should put the access-token from the login task.
Here an example of login and data tasks assuming the login is basic_auth and the access-token is bearer auth
userAndPass = b64encode(b"user:password").decode("ascii")
headers = {'Authorization': 'Basic %s' % userAndPass}
login = SimpleHttpOperator(
task_id="login",
http_conn_id="conn_id",
endpoint="/login",
method="POST",
headers=headers,
response_check=lambda response: response.json()["access_token"])
get_data = SimpleHttpOperator(
task_id="get_data",
http_conn_id="conn_id",
endpoint="/data",
method="POST",
headers={"Authorization": "Bearer {{ ti.xcom_pull(task_ids='login', key='return_value') }}",
"Accept": "application/json"},
response_check=lambda response: response.json())
(login >> get_data)
I'm using query string parameters in a Graph API request to fetch users from Azure AD, however none appear to be working when I execute it through Powershell.
Here's the documentation on the API: https://learn.microsoft.com/en-us/graph/query-parameters
Here's some details about the scenario:
The request: https://graph.microsoft.com/v1.0/users?$top=999&$select=userPrincipalName
Executed via Powershell using the Invoke-RestMethod command
It seems to work when I use the Graph Explorer: https://developer.microsoft.com/en-us/graph/graph-explorer
I assume your app registration is set up properly and you have given the appropriate permissions etc. got the token and make the restmethod call, and the call returns full user profiles and not just the upn. It would be helpful if you provided some code. Assuming the above, from the little additional info, probably powershell is replacing your $top and $select with blank or something, because it will likely treat those as powershell variables instead of literally put them in the request url. meaning you probably did something like invoke-restmethod -Uri "graphurl?$top=etcetc" change that to single quote to take the literal string eg. invoke-restmethod -Uri 'graphurl?$top=etcetc'
I'm trying to build a gRPC client for Google's Firestore API in Elixir.
Before starting to write some code , I thought that it would be wise to first start with BloomRPC in order to debug the issue.
As base url, I'm using https://firestore.googleapis.com where I pinned the rot certificate.
As auth I'm using an access_token obtained using oauth with the following 2 scopes: "https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/datastore"
being passed as a Authorization header:
{
"Authorization": "Bearer 29ya.a0AfH6SMBDYZPsVgQv0PMqyGRXypc3DfWF_2jbvAJKMquTaryCvxEB7X1Rbprfk1Ebrwid2bXbcR3Aw-d1Tytlo_lThkyqRDRIbnSz5-nQ3xWklkmjyFMAuQtFCoz01hk3vbgBwd2gdbFNNWiOU_8NqPC_vElyz2-cQ34"
}
And I get back:
{
"error": "3 INVALID_ARGUMENT: Missing required project ID."
}
So clearly I should be passing the project ID somehow but I can't find it anywhere in the docs. Anybody any clues?
I just figured out what I was doing wrong.
Basically the Bearer token I was using is correct (obtained via the OAuth Playground).
The trick was to specify the PROJECT_ID in the parent parameter of the request:
{
"parent": "projects/[project-id]/databases/(default)/documents",
"page_size": 10
}
I should have just read the docs properly :)
I am using postman application to do a SugarCRM POST request, following is the request
https://{site_url}/rest/v11/{module_name}/filter?filter=[{"$is_null"="logoutdttime"}]&fields=name,username,logoutdttime&order_by=date_entered&max_num=10
I am getting the error
{
"error": "invalid_parameter",
"error_message": "Unexpected filter type string."
}
but when I remove the filter, I get the response
https://{site_url}/rest/v11/{module_name}/filter?fields=name,username,logoutdttime&order_by=date_entered&max_num=10
what am I doing wrong
The filter is defined as array via the key and in an assoc array style rather than json. That's weird, but that's how it works in Sugar Query-String params.
Try:
https://{site_url}/rest/v11/{module_name}/filter?filter[0][$is_null]=logoutdttime&fields=name,username,logoutdttime&order_by=date_entered&max_num=10
Alternatively you can use the POST /filter endpoint instead of GET.
Then you can pass all params in the request body as json.
PS: Also are you sure you mean $is_null and not $empty?
try this:
filter[0][name][$starts]=any text
Document:
https://support.sugarcrm.com/Documentation/Sugar_Developer/Sugar_Developer_Guide_12.0/Integration/Web_Services/REST_API/Endpoints/Accountsrecordlinklink_namefilter_GET/
In my server side Meteor.js method, I'm trying to correctly make a request to Domino Data Lab's (DDL) Rest API.
DDL provides a platform for makes it possible to call a data science model via a REST API. Their documentation on this API is here:
http://support.dominodatalab.com/hc/en-us/articles/204173149-API-Endpoints-Model-Deployment
But, I doubt the documentation is helpful because I think an experienced Meteor developer will see the request examples in CURL or Python and know how to get the params correctly into the JSON format that DDL is looking for.
Domino Datalab provides the instructions for 4 methods, but not for Meteor.js. I'll post the examples for Curl and Python:
Examples
CURL Request
curl -v -X POST \
https://app.dominodatalab.com/MYURL \
-H 'Content-Type: application/json' \
-H 'X-Domino-Api-Key: YOUR_API_KEY' \
-d '{"parameters": [ "FOO", "BAR", "ETC"]}'
Python Request
import requests
response =
requests.post("https://app.dominodatalab.com/MYURL",
headers = {
"X-Domino-Api-Key": "YOUR_API_KEY",
"Content-Type": "application/json"
},
json = {
"parameters": ["FOO", "BAR", "ETC"]
}
)
print(response.status_code)
print(response.headers)
print(response.json())
I've tried a few different ways (using both the dataand paramsoptions) based on the documentation for Meteor, but here is my best try:
Meteor.methods({
score_app: function(){
var test = HTTP.call("POST", "https://app.dominodatalab.com/MYURL",
{ headers: {
"Content-Type": "application/json",
"X-Domino-Api-Key": "YOUR_API_KEY"
},
// This is where the problem is. Have tried multiple syntax versions and tried using the `params`options for the HTTP call instead of `data`
data: {'params': [143]
}
},
function (error, result) {
// The syntax below should be if not an error, log the result (for testing etc, otherwise, log "http post error". I may have incorrectly switched this around, but the original version I got from an online example had it the console.log statements in the reverse order.
if (!error) {
console.log(result);
} else{
console.log("http post error");
};
});
}
});
I've been using this entry in the Meteor documentation to send the parameters as a JSON object correctly:
http://docs.meteor.com/api/http.html
The connection to Data Domino Lab (DDL) is made correctly, but it doesn't recognize the parameters correctly because the request is not sending the parameters in the JSON format that DDL wants.
result: 'You must provide a JSON object in your request body
with a parameters key containing an array of parameters.' } }
I'm on the DDL free plan, but I will email a link to this question to their tech support. This is a niche issue, but it could be important to Meteor.js developers in the future wishing to link to a data science model in DDL.
I'm one of the engineers at Domino who has worked on the API Endpoints feature recently. The error
message you're getting means that the JSON object you're sending to our server doesn't contain the
key "parameters". I'm not an expert in Meteor, but it looks like you're using "params" where you
should use "parameters" in your JSON payload.
Around line 9 can you change...
{'data': {'params': [143]}}
to
{'data': {'parameters': [143]}}
If my understanding of your code is correct, that'll work correctly.
Cheers!