Can I host a satis repo simply as a github repo? - github

I've set up a satis repository on github to share some company internal packages across projects.
Now when I try to "depend" on the new repository, I tried this:
"repositories": [ {
"type": "composer",
"url": "https://raw.githubusercontent.com/[organisation]/satis/master/web/packages.json?token=[token-copied-from-url]"
} ]
and it works far enough that composer finds package.json, however, then it fails with:
[Composer\Downloader\TransportException]
The "https://raw.githubusercontent.com/[organization]/satis/master/web/packages.json?token=[token-copied-from-url]/include/all$[some-json-file].json" file could not be downloaded (HTTP/1.1 404 Not Found)
which isn't surprising as the ?token part appears to generate an invalid url.
I can work around this by manually moving the contents of the included file into packages.json directly, but this is less than ideal, especially if satis decides to generate multiple files.
Another problem I assume this will cause is that I don't know much about the validity of the token. Perhaps it doesn't have a long lifetime and then satis will need to be regenerated regularly.
Is there a way I can get away with hosting my satis repo as "just" a github repo?

You can store your static Satis repository in a private GitHub repo, and then use GitHub's raw.githubusercontent.com domain to serve it over HTTPS. The slightly hacky part is ensuring that composer properly authenticates against the GitHub repo.
Push Satis repository to GitHub
Generate your Satis repository and push it to your private GitHub repo, lets's say https://github.com/your-org/your-satis-repo inside the output/ directory.
Prepare your projects' composer.json files
In your projects' composer.json files, add your Satis repo to the "repositories" section:
{
"type": "composer",
"url": "https://raw.githubusercontent.com/your-org/your-satis-repo/master/output"
}
Set up HTTP basic authentication
Finally, to make composer authenticate via HTTP basic auth against raw.githubusercontent.com you will need do add a new entry to the "http-basic" section in your local composer's auth.json:
{
"http-basic": {
"raw.githubusercontent.com": {
"username": "GITHUB_USERNAME",
"password": "GITHUB_TOKEN"
}
}
}
Caveats
We've found that raw.githubusercontent.com is cached, so it can take a few minutes before changes to your Satis repository are visible.

Initial testing suggests it can be done.
I think you need to remove packages.json from your repository URL and I suspect the ?token parameter. In theory you can pass the token via a header:
https://developer.github.com/v3/#authentication
I have not tested this however.
You can see a working test without authentication here:
https://github.com/markchalloner/satishostcomposer
https://github.com/markchalloner/satishost
To try it out:
git clone git#github.com:markchalloner/satishostcomposer.git
cd satishostcomposer
composer install
Should install vendor/markchalloner/satishostdemo
Example satis.json:
{
"name": "Satis Host",
"homepage": "https://raw.githubusercontent.com/markchalloner/satishost/master/web/",
"archive": {
"directory": "dist",
"format": "tar",
"prefix-url": "https://raw.githubusercontent.com/markchalloner/satishost/master/web/",
"skip-dev": true
},
"repositories": [
{
"_comment": "Demo packages",
"type": "package",
"package": {
"name": "markchalloner/satishostdemo",
"version": "0.1.0",
"dist": {
"type": "zip",
"url": "dist/demo.zip"
}
}
}
],
"require-all": true
}
Example composer.json (in your project):
{
"repositories": [
{
"type": "composer",
"url": "https://raw.githubusercontent.com/markchalloner/satishost/master/web",
"options": {
"http": {
"header": [
"API-TOKEN: YOUR-API-TOKEN"
]
}
}
}
],
"require": {
"markchalloner/satishostdemo": "0.1.0"
},
"minimum-stability": "dev"
}
Thanks

Related

How to trigger the PR Lint Github app on any PR

I have installed the PRLint Github app (https://github.com/ewolfe/prlint) on my GitHub repository
As required, I have some PRLint configuration in .github/prlint.json
prlint.json
{
"title": [
{
"pattern": "^(build|ci|docs|feat|fix|perf|refactor|style|test):\\s",
"message": "PR title needs to be prefixed with a topic"
}
],}
However when I'm creating any PR, this is not triggering the PRLint to check if the PR is meeting the requirements or not
Under the repository "branch protection rules" I opted to pass all status checks to pass.
Please note I tried to look for PRLint in search box but nothing was found.
Well I found an alternate app for it that served the purpose
PRLint Reloaded
https://github.com/maor-rosenfeld/prlint-reloaded
My ~/.github/prlint.json
{
"title": [
{
"pattern": "^(build|ci|docs|feat|fix|perf|refactor|style|test):\\s",
"message": "PR title needs to be prefixed with a topic"
}
]
}
Repository Settings
That's it! :)

Need to report on data from Retrospectives - Azure Dev Ops

We need a way to access data through an automated way (either Rest API or some SDK) that is contained within the Retrospective Azure Dev Ops extension. Currently, there is an option to export CSV but the process is manual and limited to each Retrospective. Any ideas/thoughts?
You can try like as the following steps:
Run the API to get the information of project teams in a project.
Request URL
POST https://dev.azure.com/{organization_Name}/_apis/Contribution/HierarchyQuery?api-version=5.0-preview.1
Request Body
{
"contributionIds": ["ms.vss-admin-web.org-admin-groups-data-provider"],
"dataProviderContext": {
"properties": {
"teamsFlag": true,
"sourcePage": {
"url": "https://dev.azure.com/{organization_Name}/{project_Name}/_settings/teams",
"routeId": "ms.vss-admin-web.project-admin-hub-route",
"routeValues": {
"project": "{project_Name}",
"adminPivot": "teams",
"controller": "ContributedPage",
"action": "Execute",
"serviceHost": "{organization_Id} ({organization_Name})"
}
}
}
}
}
Run the API to list the retrospectives for a specified project team in the project.
GET https://extmgmt.dev.azure.com/{organization_Name}/_apis/ExtensionManagement/InstalledExtensions/ms-devlabs/team-retrospectives/Data/Scopes/Default/Current/Collections/{projectTeam_identityId}/Documents?api-version=3.1-preview.1
Run the API to get more details about a specified retrospective.
GET https://extmgmt.dev.azure.com/{organization_Name}/_apis/ExtensionManagement/InstalledExtensions/ms-devlabs/team-retrospectives/Data/Scopes/Default/Current/Collections/{retrospective_Id}?api-version=3.1-preview.1
However, we have not any available interface (API or CLI) to Export CSV content.

Azure DevOps API to access Multi Repo Sources of a Build

I have a YAML Build Definition that uses multiple repos.
When looking at the Build Results, there is a "Sources" Card that lists the Repos and the commit ID that was used for this particular build.
I tried looking through all the api documentation for this info, but I can't seem to find it easily. I took a look at the Get Build call, but that just gives the Main Repo used.
Does anyone know the API well enough to point me to the api call to access that set of Data that contains the repos/resources and their commit ids ?
Thanks!
You can use the following REST API:
GET https://dev.azure.com/{Organization}/{Project}/_build/results?buildId={Build ID}&__rt=fps&__ver=2
Its response body is quite massive, the information about multiple repositories is in:
fps -> dataProviders -> data -> ms.vss-build-web.run-details-data-provider -> repositoryResources
It shows all source repositories including their names, ids, versions, and so on.
This REST API is not documented and I get it from Developer Tools. Most of the information contained in it is web page information.
It looks that this is not available over REST API (yet). However, this is really strange, because if you hit get build endpoint:
https://dev.azure.com/{{organization}}/{{project}}/_apis/build/builds/826?api-version=6.0-preview.6&expand=all
in link section you will find sourceVersionDisplayUri
"_links": {
"self": {
"href": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_apis/build/Builds/826"
},
"web": {
"href": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_build/results?buildId=826"
},
"sourceVersionDisplayUri": {
"href": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_apis/build/builds/826/sources"
},
"timeline": {
"href": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_apis/build/builds/826/Timeline"
},
"badge": {
"href": "https://dev.azure.com/thecodemanual/4fa6b279-3db9-4cb0-aab8-e06c2ad550b2/_apis/build/status/43"
}
},
but the URL redirects you to commit which actually trigerred a build.
It still possible to get commits and repos from the logs
https://dev.azure.com/{{organization}}/{{project}}/_apis/build/builds/826/logs/6?api-version=6.0-preview.2
But for that you need to get trough them and parse responses:
2020-05-08T11:13:04.3354990Z Syncing repository: kmadof/devops-templates (github)
.
.
.
.
2020-05-08T11:13:03.3810662Z ##[command]git checkout --progress --force 24602dc40710a53502d306d1d41d3bca3c9a9b80
2020-05-08T11:13:03.3816465Z Note: switching to '24602dc40710a53502d306d1d41d3bca3c9a9b80'.
2020-05-08T11:13:05.1455197Z ##[command]git checkout --progress --force 0d79e869239bbf0087492ac2cda3a59e9ef11a39
2020-05-08T11:13:05.1458382Z Note: switching to '0d79e869239bbf0087492ac2cda3a59e9ef11a39'.

Github action bot pushing to gh-pages causes page build to fail

I have a github repo where I have set up github actions to build the documentation automatically and push it to the gh-pages branch.
Here is the full script, and everything goes find including pushing the update, which is done by the following 2 lines:
git -c user.email=<hidden from question> -c user.name="${GITHUB_ACTOR}" commit -m "Github Action-built docs update"
git push "https://${GITHUB_ACTOR}:${{ secrets.GITHUB_TOKEN }}#github.com/$GITHUB_REPOSITORY.git" HEAD:gh-pages
The branch is correctly updated and I see a notification in my github activity feed that says that the github-actions bot pushed to the gh-pages branch.
However, the github.io pages do not get updated and in my repo’s configuration I see the message:
Your site is having problems building: Page build failed.
I can not manage to find out any more information or a more useful error message. Checking the branch show it correctly contains all the expected files, including the .nojekyll. The github token seems to have all the necessary permissions, since it allows to push and has read/write or deployments.
If I check out the branch and push a different commit with identical contents the build then works:
git clone -b gh-pages https://github.com/Cimbali/pympress ./pages
cd pages
git commit --amend --no-edit
git push -f
So why does this happen? And how can I fix it?
I have managed to get the page builds from the github API. Here is the failed build:
{
"url": "https://api.github.com/repos/Cimbali/pympress/pages/builds/152994257",
"status": "errored",
"error": {
"message": "Page build failed."
},
"pusher": {
"login": "github-actions[bot]",
"id": <some integer>,
"node_id": "<some base 64 value>",
"avatar_url": "https://avatars2.githubusercontent.com/in/15368?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/github-actions%5Bbot%5D",
<[...]>
"received_events_url": "https://api.github.com/users/github-actions%5Bbot%5D/received_events",
"type": "Bot",
"site_admin": false
},
"commit": null,
"duration": 29,
"created_at": "2019-11-17T16:55:03Z",
"updated_at": "2019-11-17T16:55:03Z"
}
And here is for comparison the (manually triggered) successful build that followed:
{
"url": "https://api.github.com/repos/Cimbali/pympress/pages/builds/152998876",
"status": "built",
"error": {
"message": null
},
"pusher": {
"login": "Cimbali",
"id": <some integer>,
"node_id": "<a different base 64 value>",
"avatar_url": "https://avatars2.githubusercontent.com/u/6126377?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/Cimbali",
<[...]>
"received_events_url": "https://api.github.com/users/Cimbali/received_events",
"type": "User",
"site_admin": false
},
"commit": "b87d18729c945002566bae920361b2bd6bc43be6",
"duration": 10306,
"created_at": "2019-11-17T17:46:09Z",
"updated_at": "2019-11-17T17:46:20Z"
}
The suggestion I made in the comments that seems to have worked is to use a repo scoped Personal Access Token instead of the default GITHUB_TOKEN.
An email from GitHub support confirmed that it is intentional behaviour that the GITHUB_TOKEN doesn't work.
GitHub Pages site builds aren’t able to be triggered by bot users or integrations that push to your repository. This is to prevent excessive resource usage and abuse of GitHub Pages systems.

Github delete branch even after made commits to it?

I have a github repo and I am working on the site, I don't know how to do command line stuff.
So I have a branch I made commits too. And I see it can be deleted. This is kind of nice. I always do small unit tests and just want to delete it. So if I do delete a branch is all traces and history of it gone?
If you don't have a local repo anymore (meaning you cannot get your deleted branch through the local git reflog and push it back to GitHub), you still can retrieve (for a limited time, 90 days by default), your deleted branch from the GitHub repo itself.
See the GitHub Events API.
Those events are limited in time, so "all traces and history of it gone?": yes, eventually.
curl https://api.github.com/repos/<user>/<repo>/events
As described in this blog post, you will get a JSON payload which will include any push event you ever made, including your old branch:
"id": "1970551769",
"type": "PushEvent",
"actor": {
"id": 563541,
"login": "<user>",
"gravatar_id": "<id>",
"url": "https://api.github.com/users/<user>",
"avatar_url": "<url>"
},
"repo": {
"id": 9652839,
"name": "<user>/<repo>",
"url": "https://api.github.com/repos/<user>/<repo>"
},
"payload": {
"push_id": 303837533,
"size": 1,
"distinct_size": 1,
"ref": "refs/heads/<branch>", <============================
"head": "a973ddd28d599c9ba128de56182f8769d2b9843b",
"before": "4ef3d74316c04c892d17250f0ba251b328274e5f",
"commits": [
{
"sha": "384f275933d5b762cdb27175aeff1263a8a7b7f7",
"author": {
"email": "<email>",
"name": "<author>"
},
"message": "<commit message>",
"distinct": true,
"url": "https://api.github.com/repos/<user>/<repo>/commits/384f275933d5b762cdb27175aeff1263a8a7b7f7"
}
]
},
"public": false,
"created_at": "2014-02-06T14:05:17Z"
}
You can then fetch, using that specific commit 'a973ddd28d599c9ba128de56182f8769d2b9843b', make a branch locally and push it back to GitHub!
git fetch origin a973ddd28d599c9ba128de56182f8769d2b9843b:refs/remotes/origin/yourOldBranch
git checkout -b yourOldBranch
git push