How can we use tsd reinstall with authentication to install typescript definitions? - github

We have a build system on our network that frequently hits the github api limit for our company's IP address. This, of course, also blocks local developers.
The readme indicates that we should be able to authenticate for more requests, but I can't see how.
The Github API has a 60-requests-per-hour rate-limit for
non-authenticated use. You'll likely never hit this as TSD uses local
caching and the definition files are downloaded from Github RAW urls.
If you need some more then a scope-limited Github OAuth token can be
used to boost the limit to 5000.

From the tsd page on npm:
.tsdrc
This is a optional JSON encoded file to define global settings. TSD looks for it in the user's home director (eg: %USERPROFILE% on Windows, $HOME / ~ on Linux), and in the current working directory.
"proxy" - Use a http proxy
Any standard http-proxy as supported by the request package.
{
"proxy": "http://proxy.example.com:88"
}
"token" - Github OAuth token:
The OAuth token can be used to boost the Github API rate-limit from 60 to 5000 (non-cached) requests per hour. The is token needs just 'read-only access to public information' so no additional OAuth scopes are necessary.
{
"token": "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"
}
You can create this token on Github.com:
Go to https://github.com/settings/tokens/new
Deselect all scopes to create a token with just basic authentication.
(verify you really deselected all scopes)
(wonder why these presets were set??)
Enter a identifying name, something like "TSD Turbo 5000"
Create the token.
Copy the hex-string to the token element in the .tsdrc file.
Verify enhanced rate-limit using $ tsd rate
Change or revoke the token at any time on https://github.com/settings/applications
Note: keep in mind the .tsdrc file is not secured. Don't use a token with additional scope unless you know what you are doing.
The bare 'no scope' token is relatively harmless as it gives 'read-only access to public information', same as any non-autenticated access. But it does identify any requests done with it as being yours, so it is still your responsibility to keep the token private.

Related

How to download google CDN images?

In google, images are hosted in CDN URL type and I tried to download as an image from that CDN but it throws an error in C#. Used this c# code attached below.
using (var webClient = new WebClient())
{
byte[] imageBytes = webClient.DownloadData(imageUUl);
System.IO.File.WriteAllBytes(#"E:\Temp\img2.jpeg", imageBytes);
}
URL: https://lh6.googleusercontent.com/vpsleVfq12ZnALrwbIUqCTa0Fpqa5C8IUViGkESOSqvHshQpKCyOq4wsRfTcadG2WYgcW3m0yq_6M2l_IrSM3qr35spIML9iyIHEULwRu4mWw4CUjCwpVfiWnd5MXPImMw=w1280
Thanks in advance.
In GCP Cloud CDN, you can use a signed URL with authentication or signed cookies to authorize users and provide them with a time-limited token for accessing your protected content, so a Cloud CDN does not block requests without a signature query parameter or Cloud-CDN-Cookie HTTP cookie. It rejects requests with invalid (or otherwise malformed) request parameters, due to this I suggest reviewing your browser client security settings; how the authentication is managed to your CDN URL. Some clients store cookies by default if the security policy is allowed, also review how your CDN URL security ingress is configured because when you are using a CDN URLs with signed cookies the responses to signed and unsigned requests are cached separately, so a successful response to a valid signed request is never used to serve an unsigned request.
In another hand if you are using a signed CDN URL to limited secure access to file for a limited amount of time, there are some steps that you need to follow first:
Ensure that Cloud CDN is enabled.
If necessary, update to the latest version of the Google Cloud CLI:
`gcloud components update`
Creating keys for your signed URLs
To create keys, follow these steps.
1.In the Google Cloud Console, go to the Cloud CDN page.
2.Click Add origin.
3.Select an HTTP(S) load balancer as the origin.
4.Select backend services or backend buckets. For each one
-Click Configure, and then click Add signing key.
-Under Name, give the new signing key a name.
-Under the Key creation method, select Automatically generate or Let me enter.
- If you're entering your own key, type the key into the text field.
- Click Done.
- Under Cache entry maximum age, provide a value, and select a Unit of time from the drop-down list. You can choose among second, minute, hour, and day. The maximum amount of time is three (3) days.
5. Click Save.
6. Click Add.
Configuring Cloud Storage permissions.
Before you run the following command, add at least one key to a backend bucket in your project; otherwise, the command fails with an error because the Cloud CDN cache fill service account is not created until you add one or more keys for the project. Replace PROJECT_NUM with your project number and BUCKET with your storage bucket.
gsutil iam ch \ serviceAccount:service-PROJECT_NUM#cloud-cdn-fill.iam.gserviceaccount.com:objectViewer \ gs://BUCKET
List the keys on a backend service or backend bucket, run one of the
following commands:
gcloud compute backend-services describe BACKEND_NAME
gcloud compute backend-buckets describe BACKEND_NAME
Sign URLs and distribute them.
You can sign URLs by using the gcloud compute sign-url command or by using code that you write yourself. If you need many signed URLs, custom code provides better performance.
This command reads and decodes the base64url encoded key value from KEY_FILE_NAME, and then outputs a signed URL that you can use for GET
or HEAD requests for the given URL.
gcloud compute sign-url \
"https://example.com/media/video.mp4" \
--key-name my-test-key \
--expires-in 30m \
--key-file sign-url-key-file
In this link, you can find more info related to signed URLs and signed cookies.
You can't download an image in that way, since you need to provide an OAuth token. and you need to have the profile scope enabled
var GoogleAuth; // Google Auth object.
function initClient() {
gapi.client.init({
'apiKey': 'YOUR_API_KEY',
'clientId': 'YOUR_CLIENT_ID',
'scope': 'https://www.googleapis.com/auth/userinfo.profile',
'discoveryDocs': ['https://discovery.googleapis.com/discovery/v1/apis']
}).then(function () {
GoogleAuth = gapi.auth2.getAuthInstance();
// Listen for sign-in state changes.
GoogleAuth.isSignedIn.listen(updateSigninStatus);
});
}

Keycloak authentication: how can a external user get an token without exposing client secret

I have a query about how keycloak is supposed to be working with client without GUI access.
Basically I have:
A keycloak server configured with a realm, clients(Access type confidential) and Users
A server application with a GUI that also provide API, secure with keycloak (client, user, blablabla)
This is kind of working already as I am able to log on the GUI, have the redirect, etc..
Even accessing the APIs works well, when I have access to a GUI: I log on my UI, follow the redirect and get my UI to display the token. The the human (to differentiate the user from an application), can use the token in any API client.
In this context the user never sees the client secret, which is instinctively the right way. (note that I am very opened to people telling me my instinct is wrong!)
What I am NOT able to do so far is to find the way a server application (without GUI) can get a valid token?
The authorization_endpoint, as far as I understand it, requires both the client id and the client secret) to get a token, which I would rather avoid: I don't think giving my client secret to all my "customers" is the proper way to do it.
Alternatively I could create an API on my client that woudl ask for user credential and ask for the token in its behalf, but that would expose the clients credentials to my application, which is against the whole concept!
I tried setting my client Access type as public, but when I use the API call below I also get a error:
POST /auth/realms/realmname/protocol/openid-connect/tokenAPI
'grant_type=client_credentials'
'client_id=client_id'
'username=username'
'password=password'
{
"error": "unauthorized_client",
"error_description": "Public client not allowed to retrieve service account"
}
Would anyone know how this is supposed to be done ?
Thanks in advance.
Max
(...) A server application (without GUI) can get a valid token... typically using the Client Credentials flow.
But we would define in this case a dedicated Client for your server (client?) application to authenticate against. The returned token (not bound to a specific user) will serve for authorizations on allowed applications (i.e. your classic GUI or API clients).
So, basically you should (in very short):
define a specific confidential Client in your Keycloak
add the desired applications (or other Clients) to the Client Scope(s). Those you want to authorize transitively from this Client.
authenticate against this Client with Client Credentials flow (given the token endpoint, client id, credentials, scope)
ensure that you are authenticating through TLS and that parameters are included in request body (and not in headers - for enhanced privacy)
further harden security of your Client(s)
When you do not want anymore this particular server (client?) application to access your applications, you can change the corresponding "authentication" Client's secret/credentials or simply delete it.
"I don't think giving my client secret to all my "customers" is the proper way to do it."
You are right and the proposed method above strictly avoids that. Each customer would have its own credentials.
EDIT
(adding more details)
By performing as above, you would end up with the following scheme:
Flow Keycloak Server
C/S app. or Customer X <--- Client Creds ---> Auth. Client X
--- Access Token ---> Appl. Client <--> Appl. Server
C/S app. or Customer Y <--- Client Creds ---> Auth. Client Y
--- Access Token ---> Appl. Client <--> Appl. Server
Browser users <--- Standard ------> Appl. Client <--> Appl. Server
Note: this is not a detailed flow chart. Arrows mostly show relationships here.
Finally, please note that the terminology may differ a little here, but the proposed method is basically the same that Google uses. So you may aswell take some inpiration from there:
https://developers.google.com/identity/protocols/oauth2
I just had the same problem some weeks ago
In my case, I have a backend API and a frontend application that the users can use.
Eventually, I can't share the client_secret to the frontend application.
So here is my solution:
On keycloak, create a client (ex front_end_client) with grant type public
This client is going to be used by the frontend application to authenticate users using implicit flow (with PKCE will be more secure)
On keycloak, create a second client (On the same REALM as the first client) with grant type confidential, this client is going to be used by the backend API
Now, this is how it works:
Frontend app authenticate users and get the access token (Using the font_end_client)
The frontend app sends this token for every request to the backend
Backend app verify this token, and can retrieve permissions from it

How can i validate links within a private github repository

Background
We are writing some documentation for our support team.
We want to include links to files that are stored in private GitHub repositories.
We do not want the documentation to become stale if somebody refactors the code in the private GitHub repositories, so instead I am setting up a CI job that parses the documentation (with jsoup if you are interested) and finds all the links.
Once we have all the links we start checking them.
NOTE: we have written a custom link checker, because one of the critical set of links we have is for our monitoring solution, and sadly (also understandably) the SaaS we are using returns 404's for any unauthenticated requests on the URLs of the alerts.
The SaaS itself uses a 2FA to access the Web UI, so what we have ended up doing is parsing the URLs and then constructing an equivalent call to the SaaS API to validate the link.
For the monitoring system we use, this is easy: all the URLs are the same format.
Question
Can we validate a random GitHub URL as valid (ideally using only curl - I can translate to my chosen HTTP client from there, and curl gives a more generic answer) using a Personal Access Token? And if so, how?
The URLs could be:
simple direct to repo URLs: https://github.com/<org>/<repo>
direct to branch URLs: https://github.com/<org>/<repo>/tree/<branch>
file URLs: https://github.com/<org>/<repo>/blob/<path/to/file>
diff URLs: https://github.com/<org>/<repo>/compare/[<branch>...]<branch>
other URLs that are based on the presence of the repo and do not vary in child path, e.g. https://github.com/<org>/<repo>/pulls, https://github.com/<org>/<repo>/settings/collaboration, etc
plus who knows what other URLs people will add within the docs...
Things I have tried that didn't work
HTTP Basic authentication with the Personal Access Token as the password, e.g.
curl -I -u stephenc:2....token.redacted....b https://github.com/stephenc/<repo-name>
HTTP/1.1 404 Not Found
HTTP Bearer authentication, e.g.
curl -I -H "Authorization: bearer 2....token.redacted....b" https://github.com/stephenc/<repo-name>
HTTP/1.1 404 Not Found
It looks like it works for some URLs (no idea which ones).
I can access curl -u agentgonzo:$TOKEN https://raw.githubusercontent.com/agentgonzo/repo/path/to/file using the API Token as my username, but the same doesn't work on https://github.com URLs. Not sure if this will help you or not.
I got an answer from GitHub Support: No
Since a personal access token won't work for GitHub web UI URLs, no, there isn't a way to verify all possible GitHub private repo URLs without making API calls in some cases.

Disable anonymous access to buildbot web application

I've deployed buildbot in cloud vms, docker, and such. I've been able to setup authentication, but could not disable anonymous access.
It so happens that, I really can't allow anonymous access since it is a private owned resource, worst of all in many logs from build steps, passwords and other sensitive information show up.
buildbot version: 0.9.8
Documentation is scarse/nonexistant on this subject.
Thanks in advance.
Buildbot itself only allows to disable access to REST API. So anonymous users will see 'empty' web interface with no builds, logs etc. Access to the web interface can be disabled only by external web server settings.
Example authz config:
c['www']['authz'] = util.Authz(
allowRules=[
util.AnyEndpointMatcher(role='admins', defaultDeny=False),
util.AnyControlEndpointMatcher(role='admins', defaultDeny=False),
util.AnyEndpointMatcher(role='anonymous')
],
2.5.12.5. Authorization rules
One can implement the default deny policy by putting an AnyEndpointMatcher with nonexistent role in the end of the list. Please note that this will deny all REST apis, and most of the UI do not implement proper access denied message in case of such error.

Grunt required github token during docker build

Grunt requires a github authentication token while it is executed during a docker build for an image. How do I provide grunt with the required token? Interactive passing doesn't work (as I already exacted).
The message of grunt is:
Running "volo:add:-nostamp:twbs/bootstrap/3.3.2:packages/bootstrap/" (volo) task
GitHub auth required for api.github.com/repos/twbs/bootstrap/tags: {"message":"API rate limit exceeded for XXX.XXX.XXX.XXX. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://developer.github.com/v3/#rate-limiting"}
GitHub access token required to complete action. In a web browser, go to:
https://github.com/settings/tokens
and generate a new "Personal access token" for volo. Only "repo" and "public_repo" permissions are needed. Paste the generated token below.
GitHub Personal Access Token:
You can change the URL of the repository to:
https://<token>:x-oauth-basic#github.com/MyApp/myapp.git
More informations at: https://devops.profitbricks.com/tutorials/configure-a-docker-container-to-automatically-pull-from-github-using-oauth/