I have a google storage bucket with Audit Logs enabled. Every one\two days I getting logs about PERMISSION DENIED. The log is specifying what kind of access the requestor is asking for. But, not give me enough information to answer the question - who is requesting?
This is the log message:
{
"insertId": "rr6wsd...",
"logName": "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Fdata_access",
"protoPayload": {
"#type": "type.googleapis.com/google.cloud.audit.AuditLog",
"authenticationInfo": {},
"authorizationInfo": [
{
"permission": "storage.buckets.get",
"resource": "projects//buckets/BUCKET_NAME",
"resourceAttributes": {}
}
],
"methodName": "storage.buckets.get",
"requestMetadata": {
"callerSuppliedUserAgent": "Blob/1 (cr/340918833)",
"destinationAttributes": {},
"requestAttributes": {
"auth": {},
"reason": "8uSywAZKWkhOZWVkZWQg...",
"time": "2021-01-20T03:43:38.405230045Z"
}
},
"resourceLocation": {
"currentLocations": [
"us-central1"
]
},
"resourceName": "projects//buckets/BUCKET_NAME",
"serviceName": "storage.googleapis.com",
"status": {
"code": 7,
"message": "PERMISSION_DENIED"
}
},
"receiveTimestamp": "2021-01-20T03:43:38.488787956Z",
"resource": {
"labels": {
"bucket_name": "BUCKET_NAME",
"location": "us-central1",
"project_id": "PROJECT_ID"
},
"type": "gcs_bucket"
},
"severity": "ERROR",
"timestamp": "2021-01-20T03:43:38.399417759Z"
}
As you can see, the only information who talking about "who is trying to access" is
"callerSuppliedUserAgent": "Blob/1 (cr/340918833)",
But what that means? mean nothing to me.
How I can understand who is trying to access this permission?
The callerSuppliedUserAgent can be anything the client application puts in their request headers - Ignore it as this header can be faked. Only legitimate applications put anything meaningful in the header.
This is an unauthenticated request. There is no identity to record. Most likely a troll scanning the Internet looking for open buckets.
Notice that the auth key is empty. No authorization was provided in the request.
"requestAttributes": {
"auth": {},
"reason": "8uSywAZKWkhOZWVkZWQg...",
"time": "2021-01-20T03:43:38.405230045Z"
}
It's not easy to use the official LinkedIn API and I cannot find a valid documentation.
Following the official documentation I created a new application in order to obtain the Client ID and Client Secret
When I now make a POST call through Postman to https://www.linkedin.com/oauth/v2/accessToken this is what I obtain:
{
"error": "invalid_grant_type",
"error_description": "The passed in grant_type is invalid"
}
Where am I wrong?
EDIT AFTER HELP FROM #Amit Singh
Thanks to #AmitSingh I was able to create 2 different applications, the test with the Client Credentials flow gave me as a result an error retrieving the token:
{
"error": "access_denied",
"error_description": "This application is not allowed to create application tokens"
}
When I try to use the LinkedIn 3-legged workflow I receive Unauthorized
EDIT 3: GETTING THERE THROUGH POSTMAN
I now see that I can ask Postman to do the job, however when I press on Get New Access Token it opens an error page. I believe the error might be in these 4 elements:
Token name: maybe I have to give a special token name?
Auth URL: I set https://www.getpostman.com/oauth2/callback as explained here but maybe I have to set something else?
Access Token URL: I left it blank, maybe I have to put something here?
State: I set a random string like Hello123Boy but maybe I have to put something else. Maybe is too long. Maybe is too short. Maybe it has to contain symbols, etc... ?
...Also, in the guide you linked it says that the applicatoin needs to have:
r_liteprofile
rw_company_admin
w_member_social
mine has nothing:
Being recently created is still under review. It says it can take up to 90 days. Is that true?
4th EDIT: I WANT TO BELIEVE!
Here we are, at least now I'm getting the error: Bummer, something went wrong. The redirect_uri does not match the registered value. This is amazing: finally an error that says where the problem is!
On the app the, on the Products tab, I choose Sign In with LinkedIn. As
Authorized redirect URLs for your app I set https://www.getpostman.com/oauth2/callback
In Postman I setup Auth URL and Access Token URL as you said:
LinkedIn Credential workflows
LinkedIn offers 2 different credential workflows.
LinkedIn 3-legged workflow - When you want to use an API that will access LinkedIn member's data. Authorization Code grant type needed.
LinkedIn Client Credentials flow - When you want to use an API that will access non-member resources. Client credentials grant needed.
What are grant types?
"Grant type" refers to how you have acquired an access token in an OAuth workflow.
Several grant types are supported. Some of them are:
Client Credentials - Used when you want to access your own resources and not any other users
Authorization Code - Used when an app wants to access a client's data
Refresh token - Exchange an expired access token for a valid access token, used to avoid repeated user involvement
Password - Used when there is high trust between the app and the user e.g. LinkedIn mobile app, you provide your username and password
Client Credentials flow
What you need to know
Grant type used here is Client credentials - client_credentials.
Remember to set your Content-Type to application/x-www-form-urlencoded for all POST requests in OAuth.
Steps
Create an App and get your Client ID and Client Secret. Steps are shown in the respective docs linked above. Let's say they have values - <client_id> and <client_secret>.
Send a POST required to https://www.linkedin.com/oauth/v2/accessToken with following information.
Parameters
grant_type : client_credentials
client_id : <client_id>
client_secret : <client_secret>
NOTE : client_credentials is the literal text to be entered for grant_type.
Response will return a JSON Object containing your access token and its expiry duration in seconds.
Response
{
"access_token" : <access_token>,
"expires_in" : "1800"
}
Use the <access_token> obtained in Step 2 make API requests.
Example
Request URL: https://www.linkedin.com/v2/jobs
Request type: GET
Parameters
Authorization: Bearer <access_token>
LinkedIn 3-legged workflow
What you need to know
Grant type will be Authorization code - code, since you want to access a user's data.
Your Content-Type should be application/x-www-form-urlencoded for all POST requests in OAuth.
Redirect URLs are URLs where you OAuth server will redirect the user after successful authorization.
These are verified against your provided redirect URLs to ensure that it's not fraudulent.
These should be absolute URLs.
URL arguments are ignored and cannot include a #.
Steps
Create app and provide the Redirect URLs, if not already provided. Check docs for information regarding how to do this.
Get your Client ID and Client Secret. Let's say the values are <client_id> and <client_secret>.
Generate a random, hard to guess string. Let's say it's <random-string>.
Choose one of the redirect URLs provided in Step 1, where you want user to be redirected after authorization. Let's say it is <redirect_uri>.
Let's suppose you want to:
r_emailaddress - Get his email address
w_member_social - Post, comment and like posts on behalf of the user.
These are referred as "permission scopes", as in what permissions is the user authenticating you for. When sending these scopes in your request, they should be URL-encoded and space-delimited. In this particular instance, our scope will be scope: r_emailaddress%20w_member_social. We have URL-encoded the scopes mentioned above.
Adding more information regarding scopes from the Microsoft docs:
The scopes available to your app depend on which Products or Partner Programs your app has access to. Your app's Auth tab will show current scopes available. You can apply for new Products under the Products tab. If approved, your app will have access to new scopes.
Send a POST request to https://www.linkedin.com/oauth/v2/authorization with following information.
Parameters
response_type : code
client_id : <client_id>
redirect_uri : <redirect_uri>
state : <random_string>
scope : r_emailaddress%20w_member_social
After the request, the user will be presented with LinkedIn's Auth screen and asked to approve the request.
After user approves the request and the <redirect_uri> has been verified, user will be redirected to provided <redirect_uri> along with the access code <access_code> and a value in state argument. Let's say in the state argument is <state_value>.
Verify that the <state_value> is equal to the <random_string> before working with the <access_code> to get access token, for security purposes. Also, use the <access_code> within 30 minutes of being issued, for security reasons.
Next, send a POST request to https://www.linkedin.com/oauth/v2/accessToken with following information to get the access token.
Parameters
grant_type : authorization_code
client_id : <client_id>
client_secret : <client_secret>
redirect_uri : <redirect_uri>
code : <access_code>
NOTE : authorization_code is the literal text to be passed in grant_type.
You should get a similar response as in the Client Credentials workflow containing your access token and expiry duration.
Response
{
"access_token" : <access_token>,
"expires_in" : "1800"
}
Use the <access_token> obtained in Step 9 make API requests.
Example
Request URL: `https://www.linkedin.com/v2/me`
Request type: GET
Parameters:
Authorization: Bearer <access_token>
How to do this in Postman?
Create a new Collection.
Right click, select edit collection and move to authorization tab.
In "Type", select "OAuth2.0", click on "Get New Access Token".
You will see a screen where all the familiar terms mentioned above are there. Fill those, check the "Authorize via Browser" checkbox for authorization.
Now you have the access token and can proceed to make your API calls.
Postman has been designed to make such operations easier, but you have to know how to do it. For more details, you can read their official docs.
Thanks to #timur and #AmitSingh I finally arrived to authenticate to LinkedIn API.
A brief step by step solution in pictures:
Authorized redirect URLs for your app = https://oauth.pstmn.io/v1/callback
OAuth 2.0 scopes = must have r_emailaddress and r_liteprofile
In the Products Tab set Sign In with LinkedIn
Now open Postman > Collections > New Collection > Authorization and set the parameters as in picture:
TYPE = OAUTH 2.0
Token Name = put whatever you want
Callback URL = https://oauth.pstmn.io/v1/callback (should be greyed out once you tick Authorize using browser)
Tick Authorize using browser
Auth URL = https://www.linkedin.com/oauth/v2/authorization
Access Token URL = https://www.linkedin.com/oauth/v2/accessToken
Client ID = The Client ID you find on your LinkedIn App
Client Secret = The Client Secret you find on your LinkedIn App
Scope = r_liteprofile r_emailaddress
State = put whatever you like
Now click on Get New Access Token, a page will open on your browser and you will be able to login using your LinkedIn account. Once done you are authenticated.
Now use the code provided by #timur and on Postman go to Import > Upload File and import that .JSON file. You will now have the 4 queries and you can drag and drop them in your collection.
Assuming you've created your app, added correct redirect URL and enabled "Sign In with LinkedIn" product for your app, the issue you are having is probably that the first call returns a login page where your users are supposed to authenticate.
submit the request to https://www.linkedin.com/oauth/v2/authorization (you seem to have done that)
parse response of step 1 and extract all form values, add username and password to simulate user login
make POST request and use values from previous step as x-www-form-urlencoded data
manually follow the redirect header from step 3
make note of second redirect header but do not follow it - instead extract the code
POST code from previous step to https://www.linkedin.com/oauth/v2/accessToken and get access_token in response
From here, I was able to successfully transition to the auth code by following the steps.
I am not sure if you use the online Postman, but here's my complete collection export file for reference:
{
"info": {
"_postman_id": "397761c9-4287-43f2-860a-3c34cb710d50",
"name": "Linkedin oAuth",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"item": [
{
"name": "01 request Login form",
"event": [
{
"listen": "test",
"script": {
"exec": [
"const $ = cheerio.load(pm.response.text());\r",
"var inputs = $('form').serializeArray();\r",
"var payload = '';\r",
"inputs.forEach(i => {\r",
" payload += encodeURIComponent(i.name)+ '=' + encodeURIComponent(i.value) + '&';\r",
"})\r",
"payload += 'session_key='+ encodeURIComponent(pm.collectionVariables.get('username')) + '&'\r",
"payload += 'session_password='+ encodeURIComponent(pm.collectionVariables.get('password'))\r",
"\r",
"pm.collectionVariables.set(\"form_data\", payload);"
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "https://www.linkedin.com/oauth/v2/authorization?response_type=code&client_id={{client_id}}&redirect_uri={{redirect_uri}}&scope=r_liteprofile&state={{$guid}}",
"protocol": "https",
"host": [
"www",
"linkedin",
"com"
],
"path": [
"oauth",
"v2",
"authorization"
],
"query": [
{
"key": "response_type",
"value": "code"
},
{
"key": "client_id",
"value": "{{client_id}}"
},
{
"key": "redirect_uri",
"value": "{{redirect_uri}}"
},
{
"key": "scope",
"value": "r_liteprofile"
},
{
"key": "state",
"value": "{{$guid}}"
}
]
}
},
"response": []
},
{
"name": "02 Submit login form",
"event": [
{
"listen": "prerequest",
"script": {
"exec": [
""
],
"type": "text/javascript"
}
},
{
"listen": "test",
"script": {
"exec": [
"var url = 'https://www.linkedin.com'+ pm.response.headers.get(\"Location\");\r",
"pm.collectionVariables.set('first_redirect', url);\r",
"//console.log(pm.collectionVariables.get('first_redirect'));"
],
"type": "text/javascript"
}
}
],
"protocolProfileBehavior": {
"followRedirects": false
},
"request": {
"method": "POST",
"header": [
{
"key": "Content-Type",
"value": "application/x-www-form-urlencoded",
"type": "text"
}
],
"body": {
"mode": "raw",
"raw": "{{form_data}}",
"options": {
"raw": {
"language": "text"
}
}
},
"url": {
"raw": "https://www.linkedin.com/checkpoint/lg/login-submit",
"protocol": "https",
"host": [
"www",
"linkedin",
"com"
],
"path": [
"checkpoint",
"lg",
"login-submit"
]
}
},
"response": []
},
{
"name": "03 handle login-success redirect",
"event": [
{
"listen": "test",
"script": {
"exec": [
"var sdk = require('postman-collection');\r",
"var redirect = new sdk.Url(pm.response.headers.get(\"Location\"));\r",
"pm.collectionVariables.set('code', redirect.query.filter(q => q.key === 'code').map(k => k.value)[0]);\r",
"//console.log(pm.collectionVariables.get('code'));"
],
"type": "text/javascript"
}
},
{
"listen": "prerequest",
"script": {
"exec": [
"console.log(pm.variables.get('first_redirect'));\r",
"pm.request.url.update(pm.variables.get('first_redirect'));"
],
"type": "text/javascript"
}
}
],
"protocolProfileBehavior": {
"followRedirects": false
},
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "{{first_redirect}}",
"host": [
"{{first_redirect}}"
]
}
},
"response": []
},
{
"name": "04 Get Auth Code",
"request": {
"method": "POST",
"header": [],
"url": {
"raw": "https://www.linkedin.com/oauth/v2/accessToken?grant_type=authorization_code&code={{code}}&redirect_uri={{redirect_uri}}&client_id={{client_id}}&client_secret={{client_secret}}",
"protocol": "https",
"host": [
"www",
"linkedin",
"com"
],
"path": [
"oauth",
"v2",
"accessToken"
],
"query": [
{
"key": "grant_type",
"value": "authorization_code"
},
{
"key": "code",
"value": "{{code}}"
},
{
"key": "redirect_uri",
"value": "{{redirect_uri}}"
},
{
"key": "client_id",
"value": "{{client_id}}"
},
{
"key": "client_secret",
"value": "{{client_secret}}"
}
]
}
},
"response": []
}
],
"event": [
{
"listen": "prerequest",
"script": {
"type": "text/javascript",
"exec": [
""
]
}
},
{
"listen": "test",
"script": {
"type": "text/javascript",
"exec": [
""
]
}
}
],
"variable": [
{
"key": "client_id",
"value": "your app id"
},
{
"key": "client_secret",
"value": "your app secret"
},
{
"key": "redirect_uri",
"value": "your urlencoded redirect uri such as https%3A%2F%2Flocalhost%3A8080"
},
{
"key": "username",
"value": "user login"
},
{
"key": "password",
"value": "user password"
}
]
}
I am trying to send messages to an Event Hub through Data Factory's Web activity.
My Data Factory is set up with a system assigned managed identity (MSI), and that identity has been given the "Azure Event Hubs Data Sender" role on the Event Hubs Namespace (RBAC / IAM).
The Web activity is set up as follows:
{
"name": "Send to Event Hub",
"type": "WebActivity",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"url": "https://<eventhubnamespace>.servicebus.windows.net/<eventhub>/messages?timeout=60&api-version=2014-01",
"method": "POST",
"body": {
"DeviceId": "dev-01",
"Temperature": "37.0"
},
"authentication": {
"type": "MSI",
"resource": "https://servicebus.azure.net"
}
}
}
Unfortunately, it seems that the request is always unauthorized. The web activity fails with the following message:
{
"errorCode": "2108",
"message": "Invoking Web Activity failed with HttpStatusCode - 'Unauthorized'.",
"failureType": "UserError",
"target": "Event Hub",
"details": []
}
I also tried specifying the Content-Type header according to the send event REST API docs, but it made no difference.
According to this article, it should be possible to authenticate against Event Hub using a Service Principal with OAuth2.0 Bearer token. My understanding is that ADF's MSI is no different than a Service Principal, except that I don't have to manage the secrets and tokens on my own. What am I missing?
I'm deploying some REST apis using API Gateway and Lambda Functions. Because of some architectural restrictions, the API must be available only by REST endpoints. On top of the API's I need to implement a GraphQL interface to allow part of our users to query this data. To deploy the GraphQL endpoints I'm using AWS AppSync. Based on that restrictions, I created the AppSync HTTP DataSource pointing to API Gateway stage url (https://api-gateway-api-id.execute-api.eu-central-1.amazonaws.com). It worked fine. Then I secured the API Gateway REST endpoint to use AWS_IAM, created a role for the datasource with permissions to invoke-api on the selected api inovocation arn and configured the HTTP Datasource using aws cli.
For example, here is my Role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "appsync.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
And here is the policy attached to this role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "execute-api:Invoke",
"Resource": "arn:aws:execute-api:eu-central-1:9999999999:api-gateway-api-id/*/*/*"
}
]
}
And after all of that I updated my data source from aws cli with the following config:
{
"dataSource": {
"dataSourceArn": "arn:aws:appsync:eu-central-1:99999999999:apis/appsync-pi-id/datasources/Echo",
"name": "Echo",
"type": "HTTP",
"serviceRoleArn": "arn:aws:iam::99999999999:role/roleName",
"httpConfig": {
"endpoint": "https://api-gateway-api-id.execute-api.eu-central-1.amazonaws.com",
"authorizationConfig": {
"authorizationType": "AWS_IAM",
"awsIamConfig": {
"signingRegion": "eu-central-1",
"signingServiceName": "appsync"
}
}
}
}
}
Now when I try to make a query, I get the following error:
Credential should be scoped to correct service: 'execute-api'
As I understand, the correct service to be used to formulate the signature is the execute-api. I have some experience creating AWSV4 Signatures and knows that for this case it would be this one.
Somebody knows where I'm making a mistake?
With help from Ionut Trestian I found the error. I changed the configuration to use a different signatureService, like the following:
{
"dataSource": {
"dataSourceArn": "arn:aws:appsync:eu-central-1:99999999999:apis/appsync-pi-id/datasources/Echo",
"name": "Echo",
"type": "HTTP",
"serviceRoleArn": "arn:aws:iam::99999999999:role/roleName",
"httpConfig": {
"endpoint": "https://api-gateway-api-id.execute-api.eu-central-1.amazonaws.com",
"authorizationConfig": {
"authorizationType": "AWS_IAM",
"awsIamConfig": {
"signingRegion": "eu-central-1",
"signingServiceName": "execute-api"
}
}
}
}
}
Apparently I didn't understand correctly the configuration values. In my defense, I didn't found any documentation regarding this options, only a few examples scattered through the web. :-)
In case anyone else ends up here as I did wondering what else can be placed as a signingServiceName (I was looking for s3 specifically), I found this helpful blog post https://blog.iamjkahn.com/2019/12/invoking-even-more-aws-services-directly-from-aws-appsync.html
I am having trouble setting up an admin Role within the loopback project with mongodb. I've set up a simple model, added acls, created a user and associated the user with the admin Role but I can't access the model through the API Explorer - it always comes back with "Authorization Required".
Here are the steps I took:
1) I added model test through slc:
slc loopback:model
? Enter the model name: test
? Select the data-source to attach test to: moti_db (mongodb)
? Select model's base class: PersistedModel
? Expose test via the REST API? Yes
? Custom plural form (used to build REST URL):
Let's add some test properties now.
Enter an empty property name when done.
? Property name: name
? Property type: string
? Required? No
2) create user through Sign Up page: id = 1
3) create admin Role through API: id = 1
4) changed Role and RoleMapping to public=true so I could set them through the rest api
Here's the test data model:
{
"name": "test",
"base": "PersistedModel",
"idInjection": true,
"properties": {
"name": {
"type": "string"
}
},
"validations": [],
"relations": {},
"acls": [
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "$everyone",
"permission": "DENY"
},
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "admin",
"permission": "ALLOW"
}
],
"methods": []
}
Here's the Role
{
"id": 1,
"name": "admin",
"created": "2015-05-22T19:45:23.887Z",
"modified": "2015-05-22T19:45:23.887Z"
}
Here's the RoleMapping:
{
"id": 1,
"principalType": "USER",
"principalId": "1",
"roleId": 1
}
Did you set an Authorization Token at the top of the Explorer? You need to call User.login() with an email and password, then set the Auth Token to the returning auth ID value. Then this token is sent with all subsequent calls (as an authorization: header value and allows you to make calls that require auth.
Another helpful thing is to run the API with ACL debugging, you can do
DEBUG=loopback:security slc run
and it will show you how it determines access. Maybe the output will be helpful (paste it here). Otherwise it's just a guess as your code looks good and assuming you've previously logged in and are using a token.