EDIT: See comments on Schwern's answer for what I'm looking for in general. It doesn't have to be exactly what I'm asking for in the question
I have a web app which is an editor. I would like users to be able to give me a GitHub file URL, my app would automatically load in the file from GitHub, and then I make a process for them which is as easy as possible to submit that change to GitHub again. Ideally the user wouldn't need to save / upload a file or do any copy/pasting.
GitHub has a URL scheme where you can go to an "Edit" page for a file, make your changes, and then create a PR or create a commit (depending on what you would like to do and your permissions). This is an example:
https://github.com/rails/rails/edit/main/README.md
Looking at the HTML for the form I see that some of the fields have names associated. Using those names I can auto-fill the commit title and description:
https://github.com/rails/rails/edit/main/README.md?message=foo&description=bar
But I can't find a way to automatically fill in/replace the actual contents of the file. Is there a way?
I realize that for some browsers URLs can only be so long (maybe that's not true anymore?), so maybe this isn't perfect. I'd also be open to other suggestions on how to accomplish what I'm looking for.
Don't try to do this via web scraping, it's fragile and slow. Use the Github API.
Specifically, you'd get access via OAuth, get the file, let the user edit it, and then send the edited version.
There is no way to do exactly what you want. The ideal tool for this job is an OAuth App. However, creating one with the GitHub API requires that you store a client ID and a client secret, and there is no secure way to store the client secret in a frontend-only app.
Therefore, you'll need to create a backend to create the OAuth app so that you can issue credentials necessary to use the API on behalf of the user or to push data into the repository via the standard protocols.
As Schwern mentioned, you should not try to do this by driving the GitHub web interface. That isn't a stable interface and may break at any time.
Related
Does anyone know if there is any information about how the raw tokens are created?
TLDR is that I want to create links to files (specifically images) from private repos with the raw token attached. I need this to happen automatically, I do not want to "click the raw button" to get the token, that being said I do have access to the logged in users personal access token. Can I use this access token in order to automatically create a raw link with the raw token attached?
Further info:
GHE is a bit broken, and it doesn't seem top of the list from the github developers to fix it. Trying to access images from a different domain results in CORB issues. I can get the files I need using octokit, as mentioned above the users do need to login to GHE, so I have access to their access token.
What I want to do is to show markdown information, I get the markdown file through octokit, but in markdown you can of course link to images. These images will often be stored along with the markdown file in github, resulting in either relative or direct urls in the markdown file. I want to render this markdown file along with whatever images that is specified in the markdown file, but as I mentioned earlier rendering it directly will result in CORB issues.
The idea I had was that I instead could swap these GHE urls to urls with the raw token attached, using a url like that for an image would definitely work, and it does not matter that it isnt a permanent url. On the contrary it is more secure with a temporary token, and the urls would be recreated every time the user hits the page anyway, so no need for permanent links.
If I could use the users auth token to create a link to a raw image it would solve my issues, is this possible? If not, do you have any suggestions on an alternative way to do this?
The only other way I can think of is to create a proxy, that authenticates and fetches the files through octokit and returns them. This would however need to use a service account instead of the currently logged in user, which opens up a security hole where users who shouldn't have access to certain files suddenly can use the proxy instead.
Am I missing something?
Thankful for any help!
No, personal access tokens and other similar tokens can't be used there. If you want to use a personal access token, you have two options:
Use the /repos/OWNER/NAME/contents/ endpoints with Accept: application/vnd.github.raw and pass the token in the Authorization header. This will return the raw file, but it won't use the correct content type, so it probably won't render in the browser, but it can be programmatically downloaded.
Use the same endpoint without that Accept header but with the Authorization header and then you'll get a JSON response with download_url, which contains the correct token for that URL.
Note that all tokens in raw file URLs for private repositories are temporary and expire after a while, or when the user changes their password.
I will recommend that for your purpose, you probably want to deploy these documents and images to some sort of static server on a periodic basis (say, with your CI system) and host them there. That's going to be a lot easier than trying to write a proxy.
At my company, we have a CLI which allows our customers to upload data to our backend solution. The CLI runs on PR changes within a job. Let's assume the uploaded data looks like this:
{
name: "John",
age: 20,
}
Once the upload is completed, I'd like to create a comment on the PR with to following body:
John is 20 years old.
I've found the following ways to do this:
GitHub App (a bot)
GitHub OAuth App
Personal Access Token
GITHUB_TOKEN
GitHub App
The GitHub App needs to do the following things:
Fetch the data via a user-specific API key
Create a comment
I already created a comment via a bot, but I have no clue how to fetch the data.
As far as my understanding goes, I'd like other users to be able to install this GitHub app from the marketplace to work out of the box. From the ProBot Docs I understand that the bot operates on a webhook basis. Meaning I need to subscribe to a 'job completed' event (not sure if that's the correct name but I think you get the idea) and then fetch the data via a user-specific API we are providing on our platform. However, I see no way for the user of our App to configure an API key (or any form of secret) so the bot can make authenticated requests to our endpoints.
I'd prefer to use GitHub App because the comment coming from the bot would have our company branding and also an indicator that this comment has been created by the integration.
OAuth App
I already tested this by using Postman, however, the comment looks like it's coming from a specific user. Therefore, it has no company branding and it's not clear that an integration created the comment. However, the great part is that we could integrate this with our application, so our backend could create the comment once the data is received.
What I like about this approach is that we also need to implement such a feature for GitLab, Azure, etc, and using OAuth likely scales well with the other providers in comparison to the GitHub app, which is GitHub-specific.
Personal Access Token
Works pretty much like the OAuth App, but instead of our backend creating the comment, the comment is created by the CLI (and the access token is passed into the CLI). However, I think this approach is a bit sketchy.
GITHUB_TOKEN
While I haven't tried this yet, I assume that the token has limited but sufficient permissions to create a comment. As of now, I don't know what the comment will look like, but I think we can rather safely pass this into the CLI to create the comment from there. Since the permissions are limited and the token is invalidated after the workflow I see limited risk for the user of our CLI (and services).
Edit: The comment is coming from the github-actions bot, which is not the branding we are looking for, but it's clear that the comment has been created by the integration.
Questions
What's the best way to accomplish what I am trying?
Is there any way I can make this work with GitHub Apps (aka bots)?
So I have a platform that works like this: Users can create accounts by logging in with their Google (I USE AUTH0) and then they can create "Projects" which contain lots of other unimportant stuff regarding my current problem (like todo lists, ability to upload files etc; they can also Edit the project by changing some of it's attributes like name, description, theme and so on). There is a home page where everyone can see each other's projects and access them (but not upload files, change the tasks in the to do lists; this is possible only by the person that owns it).
By using a tool like Burp, people can see the request made from frontend to backend, for example when accessing one of the projects, and modify it on the fly.
This is what it looks like inside Burp when they access one of the projects:
As you can see there is a Get request to /projects/idOfTheProject; they can replace the GET with DELETE for example and they will successfully delete it; they can also see what is sent to the backend when a project is edited (name changed, description, thumbnail picture etc) and change anything they want about it.
How should I prevent this?
What I've looked at so far:
a. JWT - Probably the best fitting for my situation, but required the most work to be done (as I already have my platform almost finished with no such a security measure implemented yet, so I may need to rewrite a lot of things in both backend and frontend)
b. Sending the user's id that initiated the action as well to the backend and verify if it has the necessary privileges - the worst solution as users can access each other's profile and see the id, then just change another field in the request's JSON
c. Have a sort of token for each user and send that instead of the user's id - in this way somebody can't get your token by just looking at the communication between frontend and backend (only if it is using YOUR account). That token should be taken maybe somewhere from the auth0 when they create their account? If they provide something like that; or I can just create it myself and store it alongside the other user variables. You would still see the requests in plain text but even if you modified something you would still have to "guess" the owner's token, which will be impossible.
For frontend I use NextJS and for backend Flask.
Thank you in advance!
The TL;DR is that you don’t. A determined user will always be able to see what requests are being sent out by the code running on their computer and over their network. What you are describing when asking how to prevent people from “sniffing” these requests is security through obscurity, which isn’t actually secure at all.
What you should do instead is have an authorization system on your backend which will check if the current user can perform a given action on a given resource. For example, verifying that a user is an administrator before allowing them to delete a blog post, or making sure that the current user is on the same account as another user before allowing the current user to see details about the other user.
I'm using the "github page" to create my personal page, but I'm going to need a hosting service because it will require some queries in the database. How can I use my GitHub Page url as a domain?
GitHub pages is not really designed for this kind of function. It's there to be a static page, where all content on the page is 'hardcoded' (meaning no dynamically generated data). What you're asking falls along the lines of a web application.
But if you're looking to be a maverick, there might be some options out there for you.
I personally haven't done something like this, but found a couple DB services you might want to check out.
Firebase by Google
RdbHost
The above recommendations may be useful if you're expecting data entry from visitors to your page. But if your data is static as well...you might be better off using s JSON file or some alternative where the data can live right in your repo.
How can I view a raw text file on GitHub?
I created an unlisted REPO and inside the repo is a text file.
When I view it in raw format I get exactly what I want. And if I use C# to Download the text of that page, I get what I want.
My only concern is that the link to the github raw file is
https://raw.githubusercontent.com/USERNAME/STUFF/master/version.txt?token=THETOKEN
Does the THETOKEN part ever change? Like can I use this link for the rest of my life to access that raw text file? Or does the link change?
The token part is an auth token so it will work as long as that set of credentials is valid (until you revoke it). However, you should still manage that token as a secret.
If you plan to share this C# program with others you should not share your token with them, since this will give them access to other parts of your GitHub account.
If you want to share this file publicly you should publish it to a public CDN or another service that provides access control. For example you can use Amazon S3 with signed URLs for a few cents a month.