Accessing Concourse REST API from resource - concourse

I am trying to write a custom Concourse resource (in Python) that accesses the Concourse instance's REST API for information. I'm stuck at obtaining the bearer token at login. The issue is that when I follow the gist of this shell script
#!/bin/bash
## Variables required #need to update these to take inputs for getting token per team and target.
CONCOURSE_URL="http://localhost:8080"
CONCOURSE_USER="test"
CONCOURSE_PASSWORD="test"
CONCOURSE_TEAM="test"
CONCOURSE_TARGET="my-concourse"
function get_token() {
## Create a file named token that will be used to read and write tokens
touch token
## extract the LDAP authentication url and write to token file
LOCAL_AUTH_URL=$CONCOURSE_URL$(curl -b token -c token -L "$CONCOURSE_URL/sky/login" -s | grep "/sky/issuer/auth/local" | awk -F'"' '{print $4}')
echo "url is $LOCAL_AUTH_URL"
# login using username and password while writing to the token file
curl -s -o /dev/null -b token -c token -L --data-urlencode "login=$CONCOURSE_USER" --data-urlencode "password=$CONCOURSE_PASSWORD" "$LOCAL_AUTH_URL"
ATC_BEARER_TOKEN=`grep 'Bearer' token | cut -d\ -f2 | sed 's/"$//'`
echo $ATC_BEARER_TOKEN
}
there are many redirects involved, and at least some of them refer to the concourse instance as being at http://localhost:8080, which does not work from inside the docker container of the resource.
So I wanted to parametrize the external base url and explicitly give it in resource config. Manually handling the redirects and rewriting the local IP into the URL fails at the last "approval" step with a code 400, probably because it looks like some kind of a cross-domain attack.
The environment variable ATC_EXTERNAL_URL is always localhost:8080 and I suspect that this is also used when forming out the redirect urls. Can this be set somewhere?
I'm bad at golang, but it seems to me that https://github.com/concourse/concourse-pipeline-resource calls the fly binary to achieve some kind of login from inside a resource. Can't say I can get what it does and how.
All help appreciated...

The env var $ATC_EXTERNAL_URL most likely corresponds to the external url specified when you start Concourse, so yes, it can (and if you're using external auth like Github or OAuth, must) be changed. You're correct in assuming that it's used to construct callback URLs.
Also, I don't want to be That Guy(TM), but the Concourse REST API is not public and is subject to change at any time. What are you trying to do that you can't get from the fly CLI? Your resource could call the ATC_EXTERNAL_URL to get the fly CLI when it's needed then execute commands that way.

Related

How do I curl against a git.io URL generated from a private repo?

I have a script that I'd like to be able to access via a curl command against its https://raw.githubusercontent.com/... location. Using git.io, it's really easy to shorten this URL to something like https://git.io/ABCDE.
But there's an issue related to the fact that my script exists in a private repository. If I directly curl against the githubusercontent URL, I get 404: Not Found. I'm able to bypass this by passing an authorization header with the request, e.g.
$ curl -H "Authorization: token <My Github Personal Access Token>" \
https://raw.githubusercontent.com/...
> !#/bin/bash
... # rest of script
However, when I use my shortened URL, I don't get anything back. Not even a 404.
$ curl -H "Authorization: token <My Github Personal Access Token>" \
https://git.io/ABCDE
$
Anyone know what's going on here?
The way a URL shortener works is that it issues some sort of 3xx-series HTTP status code that redirects you to the new location, and then you make your request against that new location. However, by default, curl does not follow redirects, so all you see when you make your request is the output from git.io, which in this case is nothing.
If you want to follow redirects, then you should use the -L option to curl, which will make it follow redirects. Note that this can be insecure in many cases when passing credentials, since any credentials passed with -H will be passed to any remote server that the data is redirected to. In this case, that's what you want, but it can be a security problem in other cases if the credentials were only intended for the original server.

Jenkins API Crumb 403

I am using Curl to make requests to the Jenkins API - which is being done over https.
Jenkins is setup in IIS using a reverse proxy as advised by Jenkins.
The Jenkins site has windows authentication.
I need to pass a Crumb with my request - 403 error.
However, when I request for the Crumb, it states that I need to provide a Crumb?
I need this to work via the command prompt/batch file.
Thanks
To be able to do API calls to Jenkins, you need to generate a token for a given user in Jenkins. For example, let's do it with user Foo. You'll need to sign in with Foo user and then in the web UI: Foo (right upper corner) > Configure > API token > Add new token.
When you have your token saved somewhere, you can retrieve the breadcrumb with this command:
curl --user Foo:<token> 'https://www.mysuperduperjenkins.com/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)'
What this is doing basically is fetching the XML from Jenkins that contains the breadcrumb and getting the specific element where the breadcrumb is. The result of the command should be something similar to:
Jenkins-Crumb:<someLongLongLongToken>
You can save the breadcrumb into a variable, for example name it breadcrumb and then:
curl -u 'Foo:<token>' -X POST --data '<parameters>' -H '$breadcrumb' www.mysuperduperjenkins.com
In this curl we're using the breadcrumb as a header, obtained in previous step. If the crumb is valid you should be able to perform the request without 403/401 errors.

What is the format of the request for authentication to python-eve api

I, basically, have two related questions.
In my case, I have a global Mongo authentication as admin with the password pass. I implemented my BCryptAuth as described in this answer, so I can create users without any authentication like this:
curl -d 'username="barack"' -d 'password="obama"' http://0.0.0.0:5000/users/?pretty
Then, I can access my api using my Mongo authentication like this:
curl -u admin:pass http://0.0.0.0:5000/users/?pretty
But I cannot use any of my new user's credentials to get any data. So, the request
curl -u barack:obama http://0.0.0.0:5000/users/?pretty
does not work. I always get 401:
{
"_status": "ERR",
"_error": {
"code": 401,
"message": "Please provide proper credentials"
}
}
In this example, after implementing basic authentication, the request is the following:
curl -H "Authorization: Basic YWRtaW46c2VjcmV0" -i http://example.com
It is not clear for me what the token after the Basic keyword is. I also tried to use this format in my case, but nothing works.
So, the first question is, what must be the format of the request as the barack user with the password obama for the BasicAuth?
Also, I am trying to implement the TokenAuth. Then the second question is, what is the format of the request with the token?
I could not find any clear information about this.
UPDATE
Thanks to #gcw, I figured out how to make the request using the Authorization header.
Couple of remarks:
The Base64 decoding on linux is done with echo '<base64encodedstring>' | base64 -d. The -D is the macos option.
The Base64 encoding is done with: echo -n '<string>' | base64. The -n is important - it drops the new line character, which is by default included in the <string>.
But I still was not able to use my newly created users to login to the api.
The Authorization header must have the admin:pass information in base64 format. If you check the contents of YWRtaW46c2VjcmV0 by decoding it with echo "YWRtaW46c2VjcmV0" | base64 -D you can see that it is admin:secret. Try to use your credentials in base64 format to see if that works. Also, you can generate the Authorization header using Postman Basic Auth tab.
About using the TokenAuth, you can use Bearer instead of Basic to pass your token in base64 format. This answer here explains some details about different HTTP authentication methods.

Where do I find the REST urls and json structs corresponding to each vmc command?

I am trying to use CloudFOundry REST calls to create app, bind service etc instead of using vmc or eclipse plugin. I know we can use vcap-java-client for this but I also wanted to test calling the REST services directly such as by using cUrl utility. Where do I find the REST urls and json structs corresponding to each vmc command ?
The best way for you to see the actual rest calls to the Cloud Controller is executing the vmc commands and add the -t. So for example:
vmc apps -t
So this vmc command will show you a list of your apps and so what you get out of the additional -t is a more verbose information which shows the Cloud Controller service you are calling. In this case it would be /apps
The same command on the bottom will give you the exact output you would get from the above vmc command:
curl -H "Authorization: <Your-Oauth-token>" https://api.cloudfoundry.com/apps
The Authorization token can also be found in the -t generated output. you need to copy the entire Authorization "bearer"
Sudipta,
When you login and run vmc apps -t, you should see the following output:
Getting applications. >>>
REQUEST: GET /apps
REQUEST_HEADERS:
accept : application/json
user-agent : Ruby
authorization : bearer <token>
content-length : 0
The authorization in REQUEST_HEADERS is what you would need to use.

Github v3 API - create a REPO

I’m trying to use the Github v3 API - I already implemented the required OAuth flow and it works well.
Now I’m trying some of the Repos API endpoints (http://developer.github.com/v3/repos/).
So far, I’m able to get a List of my repos using: GET /user/repos
However, when I try to create a repo using POST /user/repos, I get a 404.
Any thoughts what I might be doing wrong?
Joubert
Can you please tell us how exactly you did the HTTP request? The 404 sounds like you were using a wrong path, probably. But to give a reliable answer instead a wild guess, we need to see your request, including how you are sending your token, just mask it with 'xxx' or something.
I'll show you in the meantime an example request, that is working:
curl -XPOST -H 'Authorization: token S3CR3T' https://api.github.com/user/repos -d '{"name":"my-new-repo","description":"my new repo description"}'
You would need to replace the OAuth token of course: S3CR3T
I had the same issue. The reason why you are getting a 404 with your oauth access token is that when you authorize to github you need to also additionally pass the scopes you want. For example, in the header you should see "X-OAuth-Scopes: repo, user", which means this user has read/write access to his profile and repositories. Once you have set the correct scopes you should be able to do POST/PUT requests just fine.
To see whether or not you have the correct permissions. You can do something like the following. Substitute the XXXXXXX with your access token.
curl -I https://api.github.com/user?access_token=XXXXXXXX
For creating repositories as a user you can use an personal access token and basic auth, which can be much simpler when you are fluffing around on the command line and have 2FA enabled.
curl -d '{"name":"test"}' -u githubuser:personaccesstoken https://api.github.com/user/repos
Create a personal access token here https://github.com/settings/tokens and make sure it has the 'repo' scope.
This script lets you read in in the token and project name as variables so you can use it in a script
#!/usr/bin/env bash -u
#
TOKEN=`cat token_file`
PROJECT=myproject
curl -X POST -H 'Content-Type: application/x-www-form-urlencoded' -d '{"name": "'"$PROJECT"'"}' https://api.github.com/user/repos?access_token=$TOKEN