I have 5-6 github python repos. These repos has python test cases as well, which ran using pytest and github action. I want see the results of test cases running in all the repos at a single place/tool.
please suggest
Related
In my code hosted on GitHub, we perform some tests and quite a bit of post-processing using GitHub Actions. Now, we would like to (or, actually, have to) use Gitlab runners hosted by a supercomputing center to do some further testing and benchmarking. This cannot be done with self-hosted GitHub runners, because I cannot influence their decision. We do not want to move the whole workflow and community over to some Gitlab instance either. So here's my (general) question: Is there a way to use Gitlab runners from within GitHub Actions?
What I have tried and what kind of works is to mirror the repository over to the Gitlab instance and let the runners do their magic there. Using this neat approach, the GitHub Action will wait for the results of the runners and integrate them into its own results. However, this does not work if contributors fork the repository and make pull requests.
In principle, it looks like this could be doable if the contributors also have accounts and corresponding permissions at the Gitlab instance. This is fine for now, because the community is small and the Gitlab instance is accessible to external contributors. Note that manual action from the maintainers of the code (i.e., me) is required before contributors can execute code with the runners for the first time, so we should be fine concerning security.
However, I cannot get this to work for pull requests, because I fail to mirror them. As said, direct pushes are fine, but nothing else works. This leads me to the more specific questions: How can I mirror a pull request from GitHub to a Gitlab repository? How can I enable this for both pull request and pushes (and do I need even more cases)?
Any help is appreciated! I'm really no expert on GitHub Actions, Gitlab runners or even git itself (beyond the basics). If there's a better way to achieve this, I'm happy to hear about it!
I can think of several workarounds:
1. Change what triggers your pipelines
Since you cannot mirror pull requests, but you can mirror branches, adapt the pipeline triggers in Gitlab so the pipelines are launched whenever there is a new commit, instead of a new PR.
You can always use a staging branch if you want to limit the pipeline executions.
2. Use webhooks
If the Gitlab instance is available on the internet, create a GitHub action that triggers a Gitlab pipeline execution whenever there is a PR on Github, or even open a PR directly in Gitlab. It is well documented:
Trigger a pipeline using curl
API to create merge request
Just completed a GitHub workflow using more of them are actions, but also with one bash script.
When writing the workflow, it seems much quicker use bash script than actions.(since some actions are just do one thing. ) Why are the some reasons that we just need GitHub actions rather than bash script or python script trigger?
Or we are just supposed to use script languages for most part, then use GitHub actions for small portion of the whole workflow?
Interesting but not easy to answer with more information about what your goal is. The right answer might depend on your use case.
I have not used GitHub actions yet. Let me try to explain it anyway, starting pretty high level. Unfortunately, there's no option to add a table of contents ;) Please let me know if this helps.
1. What are GitHub Actions for?
From this "What is GitHub Actions? Benefits and examples" PDF file
GitHub Actions is a CI/CD tool for the GitHub flow. You can use it to integrate and deploy code changes to a third-party cloud application platform as well as test, track, and manage code changes. GitHub Actions also supports third-party CI/CD tools, the container platform Docker, and other automation platforms.
From docs.github.com
GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. You can create workflows that build and test every pull request to your repository or deploy merged pull requests to production. [...]
GitHub Actions goes beyond just DevOps and lets you run workflows when other events happen in your repository.
2. Continuous Integration/Continuous Deployment (CI/CD)
Usually, people run CI/CD tools to build, deploy, test, and run other tasks while doing that. We use another 3rd party CI/CD pipeline using Rake to build, test, and check links. Our pipeline invokes these small scripts you mention.
3. GitHub actions and scripts
From Essential features of GitHub Actions
If your job generates files that you want to share with another job in the same workflow, or if you want to save the files for later reference, you can store them in GitHub as artifacts. Artifacts are the files created when you build and test your code. For example, artifacts might include binary or package files, test results, screenshots, or log files. Artifacts are associated with the workflow run where they were created and can be used by another job. All actions and workflows called within a run have write access to that run's artifacts.
Here's the key point, I guess. You can really do a lot of crazy stuff within a workflow. All is related/specific to GitHub. Workflows are event-driven, meaning that you can run a series of commands after a specified event has occurred. For example, every time someone creates a pull request, you can automatically run a command that executes a test or other script.
4. GitHub action workflow and scripts
You can include different scripts in your workflow, e.g. using
Javascript: https://github.com/actions/github-script
Python: https://github.com/marketplace/actions/run-python-script
5. (Complex) Examples
You can check out the repository for docs.github.com for some more complex examples, see action-scripts and workflow folders. GitHub themselves seems to use it pretty heavily.
6. Advantages/Disadvantages of GitHub actions
OR: Differences to other CI tools
It took some time to find something not marketing-ish. Key points are:
beginner-friendly using YAML config files
no need to set up your own CI pipeline
You can check out this SO post from 2019 for a list of what's good and bad about GitHub actions.
In short - for readability and the DRY ("Don't repeat yourself") principle.
It's more or less the same as using functions in programming.
I can agree that some trivial actions are useless.
But "actions/checkout" for example is priceless!
Currently, we commit Chef cookbooks to individual repos within our GitHub organization. We are configuring a Jenkins job per repo / cookbook which will execute cookstyle first when a commit occurs, and if cookstyle passes with no issues, it will execute Test Kitchen. We have a template Jenkins job we copy and configure for each cookbook we create.
Does anyone know if it's possible to have GitHub hooks in Jenkins to listen for commit events across the entire organization, and then execute cookstyle on a repo where a commit occurred that contain a Chef cookbook? I'd like to have one central job handling the lint testing for our organization.
You wouldn't generally use a single job for this. You would use the GitHub Organization support to scan for all repositories and build them when they change. You can use webhooks to ping individual repos.
You can create github web hooks for your github organization and configure your jenkins with the help of github plugin. Refer this documentation to see how to configure
The only similar question I found is the following: Trigger travis ci builds if another git repository updates
Keep in mind that our E2E tests are full stack: we have a live server running, and our tests are run against the UI which hits the server. Nothing is stubbed or mocked.
Now, I can trigger in that way a travis rebuild, however problem arises when I have interdependency between branch and a different repository branch.
So let's say I have 3 repositories: backend, frontend and e2etests. If I create the new branch frontend/foo which requires also the backend/foo, there is no way e2e tests will ever pass because they will run one time with frontend/foo and backend/master and another time with frontend/master and backend/foo.
Did anyone face this problem previously? How did you deal with it?
After a lot of digging it was clear that there is no solution and everyone has their own built-in.
What worked for us is a git pre-push hook, environment variables pointing to our repositories and when you push it asks if you want to use a different branch for the UI project (if you are on the UI, asks if you want to use a different branch for the server project) and by default it uses the current branch for the project you are on. So it's quite fast on push, usually you'll just press ENTER an additional time, the two branches are basically the required dependencies and a travis build will be triggered with an API endpoint.
Works fine so far.
I am trying to create the following workflow for continuous integration on my project
Developer commits code change to Github
Github sends entire codebase to Travis CI
Travis CI does the following
Runs tests
Builds distribution worthy package, saves in filesystem location local to Travis CI
Deploys distribution worthy package to Nodejitsu
Nodejitsu restarts server with the distribution worthy package
Is this possible? What are the steps? Via Github webhooks, I do have Travis CI running tests, and Nodejitsu deploying (untested and distribution worthy) Github codebase, but I do not have the workflow described above. Am I expecting something that is not possible?
Thank you!
This is possible, and how we deploy many of our systems internally.
For a quick getting started guide please try:
https://www.nodejitsu.com/getting-started-with-github
Detailed documentation can be found here:
https://www.nodejitsu.com/documentation/features/webhooks/
If you have any other questions, feel free to reach out to Nodejitsu support.