is it possible to use static ip when using github actions - github

Now I am using github actions as my project CI, I want to do some unit test when using github actions build my project. When I using unit test, the project must using database, now my database have a white list and only IP in white list could connect my database, but now when I run unit test in GitHub Actions, I did not know the GitHub Actions IP address. Is it possible to use a static ip or any other way to solve the problem? I am not want to any IP could connect my database, it may have a security problem. any suggestion?

This is currently only possible with a self-hosted runner on a VM you can control the IP address of.
See also:
About self-hosted runners.
Alternatively, your GitHub action workflow may be able to adjust the firewall settings as part of the run.
Or you could use something like SQL Server LocalDB or SQLLite to connect to the database locally on the runner. Or spin up a temporary DB in a cloud environment, open it up to the runner and throw it away afterwards.
Or you could use a VPN client to connect to actions runner to your environment. You can install anything you want on the runner.

You can dynamically retrieve the GitHub Actions runner's IP address during your workflow using the public-ip action and update your RDS instance's security group ingress rules before and after your unit test steps.
This will allow you to use GitHub's hosted runners with your workflow instead of hosting your own.
Note: You will need to also set AWS credentials on your runner with permissions to update the associated security group. Also, you need to make sure the RDS instance is in a public subnet with an Internet Gateway attached and security group attached to it.
Your workflow should look something like this:
deploy:
name: deploy
runs-on: ubuntu-latest
env:
AWS_INSTANCE_SG_ID: <your-rds-subnet-sg-id>
steps:
- name: configure aws credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: <your-ci-aws-access-key>
aws-secret-access-key: <your-ci-aws-secret-key>
aws-region: <your-rds-aws-region>
- name: get runner ip addresses
id: ip
uses: haythem/public-ip#v1.2
- name: whitelist runner ip address
run: |
aws ec2 authorize-security-group-ingress \
--group-id $AWS_INSTANCE_SG_ID \
--protocol tcp \
--port 22 \
--cidr ${{ steps.ip.outputs.ipv4 }}/32
- name: connect to your rds instance and run tests
run: |
...run tests...
- name: revoke runner ip address
run: |
aws ec2 revoke-security-group-ingress \
--group-id $AWS_INSTANCE_SG_ID \
--protocol tcp \
--port 22 \
--cidr ${{ steps.ip.outputs.ipv4 }}/32
Ideally though you would run your integration tests in an EC2 within the same VPC as your RDS instance to avoid publicly exposing your RDS instance.

This is in beta (September 1, 2022) but it is possible to assign static IP address to runners:
Fixed IP ranges to provide access to runners via allow list services
Setup a fixed IP range for your machines by simply ticking a check box, this provides an IP range that can be allow listed in internal systems and in GitHub’s allow list to keep using Actions while making your GitHub environment more secure.
More details here

If your database happens to be Redis or PostgreSQL, GitHub Actions includes a built-in feature called Service Containers to spin up an ephemeral database in CI for testing purposes.
These databases are short-lived: after your job that uses it completes, the service container hosting the database is destroyed. You can either run the database in a container or directly on the virtual machine if desired.
For more info, see Creating PostgreSQL service containers in the GitHub Actions docs.
If you happen to be using another database, you can install do some more manual legwork to install and run it yourself.

Related

ADO container jobs using private container

I have an ADO pipeline I'm trying to run as a containerized job. The yaml is setup with the following line:
container: myDockerHub/myRepo:myTag
Where that actually points to a tag in a private repo on DockerHub. The job errors with a message that access to the repo is denied and may require a login. Which is perfectly true. It's a private repo that does require a login. But how do I tell ADO to login to the repo?
I have a service connection setup to DockerHub, and I use docker login successfully in other non-containerized jobs where a script is spinning up a docker image. But since this is using the container global option, I don't see any way to "preface" it with a login instruction. What do I need to get it to work here?
I don't see anything about authentication on the Microsoft documentation on container jobs
You can use your DockerHub service connection with the endpoint property:
container:
image: registry:myimage
endpoint: private_dockerhub_connection

GitHub CI/CD cannot deploy to Azure SQL as it cannot add firewall rule due to "deny public network access" being set to Yes

I have an Azure SQL server where I wish to deploy my database via dacpac using GitHub CI/CD. I am using the Azure SQL Deploy action with Service Principal for Azure login action
Due to policy restrictions on the client side, the "Deny Public Network Access" is always enabled and therefore while deploying even though the service principal login works, the GitHub action is unable to add the IP Address to the firewall rule.
We are using Self-Hosted GitHub runners. Is there any workaround to deploying the database via CI/CD under such circumstances where we cannot add the firewall rule to whitelist the agent/runners IP Address?
The solution was to do away with Azure Login action and add self-hosted runner virtual network in the Azure SQL Firewall settings:
The Azure Login action attempts to add IP Address of the runner to the Azure SQL Firewall. Hence, this action must not be used. I removed this action and relied on the second step for seamlessly accessing Azure SQL Database instead.
The Azure SQL Deploy action requires either above login step or "allow access to azure services" to be turned ON. Both of which were not an option for me. I decided to go with an action that runs sqlpackage.exe. https://learn.microsoft.com/en-us/sql/tools/sqlpackage/sqlpackage-publish?view=sql-server-ver15
I am using self-hosted runners which are hosted within the virtual network configured within Azure. However, the point that I had missed was to add the virtual network to the Firewall settings in Azure SQL. Once, I did that, I did not require an explicit login or whitelisting runner ip addresses.

How can I find the right inbound rule for my Github action to deploy on my AWS EC2 server?

I just created the action on my project and configured everything over there, but unfortunately I'm getting a message like this into the 'deploy file' section> ssh: connect to host ec2-MYIP.us-east-2.compute.amazonaws.com port 22: Operation timed out
Good thing is that I know what's happening. I have to allow as an Inbound Rule the following:
Type: SSH / Protocol: TCP / Post range: 22 / Source: ::/0;
As you can see here, it works fine without limiting the source IP >
But obviously I don't want to do that for security reasons, so I need to find out the source I need to put there.
I've tried a lot of Github IP addresses already, but all of them were unsuccessful.
Does anyone here know what's the right source for it to work in a protected way or how can I find it?
Action I am using > https://github.com/wlixcc/SFTP-Deploy-Action
The IP addresses of GitHub hosted runners are documented here: https://docs.github.com/en/free-pro-team#latest/actions/reference/specifications-for-github-hosted-runners#ip-addresses
Windows and Ubuntu runners are hosted in Azure and have the same IP address ranges as Azure Data centers.
[...]
Microsoft updates the Azure IP address ranges weekly in a JSON file that you can download from the Azure IP Ranges and Service Tags - Public Cloud website. You can use this range of IP addresses if you require an allow-list to prevent unauthorized access to your internal resources.
An improved answer over riQQ's: Dynamically retrieve the Github Action runner's IP address during your workflow using the public-ip action and
update your EC2 server's security group ingress rules before and after your SSH steps.
Your EC2 instance will never be exposed to public IP addresses on your SSH port.
Note: You will need to also set AWS credentials on your runner with permissions to update the associated EC2 security group.
Your workflow should look something like this:
deploy:
name: deploy
runs-on: ubuntu-latest
env:
AWS_INSTANCE_SG_ID: <your-ec2-security-group-id>
steps:
- name: configure aws credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: <your-ci-aws-access-key>
aws-secret-access-key: <your-ci-aws-secret-key>
aws-region: <your-ec2-aws-region>
- name: get runner ip address
id: ip
uses: haythem/public-ip#v1.2
- name: whitelist runner ip address
run: |
aws ec2 authorize-security-group-ingress \
--group-id $AWS_INSTANCE_SG_ID \
--protocol tcp \
--port 22 \
--cidr ${{ steps.ip.outputs.ipv4 }}/32
- name: ssh into your ec2 and do whatever
run: |
...do whatever you need to do...
- name: revoke runner ip address
run: |
aws ec2 revoke-security-group-ingress \
--group-id $AWS_INSTANCE_SG_ID \
--protocol tcp \
--port 22 \
--cidr ${{ steps.ip.outputs.ipv4 }}/32

How can I deploy content to a static website in Azure Storage that has IP restrictions enabled?

I'm getting an error in my release pipeline (Azure DevOps) when I deploy content to a static website in Azure Storage with IP restrictions enabled.
Error parsing destination location "https://MYSITE.blob.core.windows.net/$web": Failed to validate destination. The remote server returned an error: (403) Forbidden.
The release was working fine until I added IP Restrictions to the storage account to keep the content private. Today, we use IP restrictions to control access. Soon, we will remove the IP restrictions in favor of vpn and vnets. However, my expectation is that I will have the same problem.
My assumption is that Azure DevOps cannot access the storage account because it is not whitelisted in the IP Address list. My release pipeline uses the AzureBlob File Copy task.
steps:
- task: AzureFileCopy#2
displayName: 'AzureBlob File Copy'
inputs:
SourcePath: '$(System.DefaultWorkingDirectory)/_XXXXX/_site'
azureSubscription: 'XXXX'
Destination: AzureBlob
storage: XXXX
ContainerName: '$web'
I have already enabled "trusted Microsoft services" but that doesn't
help.
Whitelisting the IP Addresses for Azure DevOps is not a good option because there are TONS of them and they change regularly.
I've seen suggestions to remove the IP restrictions and re-enable them after the publish step. This is risky because if something were to fail after the IP restrictions are removed, my site would be publicly accessible.
I'm hoping someone has other ideas! Thanks.
You can add a step to whitelist the agent IP address, then remove it from the whitelist at the end of the deployment. You can get the IP address by making a REST call to something like ipify.
I have done that for similar scenarios and it works well.
I would recommend a different approach: running an Azure DevOps agent with a static IP and/or inside the private VNet.
Why I consider this a better choice:
audit logs will be filled with addition and removal of rules, making harder analysis in case of attack
the Azure connection must be more powerful than needed, specifically to change Rules in Security Groups or Firewall or Application Gateway or else, while it only needs deploy permissions
it opens traffic from outside, while temporarily, while a private agent needs always initiate from inside
No solution is perfect, so it is important to chose the best for your specific scenario.

Kubernetes only allow pull from specific private registry

I have a private registry and I want to allow worker nodes (running on Azure Kubernetes Services) to be able to pull images only from this registry.
Is there a way to allow worker nodes to only pull images from a specific private registry?
I would be surprised if the only way to achieve that is through complex firewall rules.
As far as I know Kubernetes does not have a feature which you are referring to.
You can read about Pull an Image from a Private Registry, which describes how to create a secret which holds the authorization token and how to use it.
On the other hand, I was able to find something in Docker called Content trust.
Content trust allows operations with a remote Docker registry to enforce client-side signing and verification of image tags. Content trust provides the ability to use digital signatures for data sent to and received from remote Docker registries. These signatures allow client-side verification of the integrity and publisher of specific image tags.
Currently, content trust is disabled by default. To enable it, set the DOCKER_CONTENT_TRUST environment variable to 1
Link to the documentation is available here, also you can read about it inside Docker blog A secure supply chain for Kubernetes, Part 2
If you have an private registry your docker tags of images inside it must have prefixed by the registry hostname. If not present, the command uses Docker’s public registry located at registry-1.docker.io by default. As example your registry hostname is docker.mycompany.net the tag of the image should be docker.mycompany.net/{image-name}:{image-version}. Documentation on docker tag can be found here.
So you just have to use that full tag of the image with the hostname prefix in your container specs as example to above scenario it should be like this.
containers:
- name: my-container
image: docker.mycompany.net/{image-name}:{image-version}
ports:
- containerPort: 80