gitlab ci does not update code at remote server - deployment

I want to deploy ma test app from local repo to gitlab repo and with gitlab ci push it to my remote server. SSH connection is working, gitlab CI shows that job is passed, but code on remote server is not updated.
I made bare repo in: /home/repos/testDeploy.git
And folder for files is in: /home/example.com/web/testDeploy
I added
My .gitlab-ci.yml file
stages:
- deploy
deployment:
stage: deploy
environment:
name: production
url: http://www.example.com/testDeploy
only:
- master
before_script:
- 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
- eval $(ssh-agent -s)
- chmod 600 ~/.ssh/id_rsa_gitlab && chmod 700 ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
script:
- git remote add production ssh://user#server:port/home/repos/testDeploy.git
- git push -f production master
- echo "Deployed to production!"
Also, i have post-receive hook:
#!/bin/sh
git --git-dir=/home/repos/testDeploy.git --work-tree=/home/example.com/web/testDeploy checkout -f
I make changes in my local repo, commit and push to origin master to gitlab. Job is passed, but as I mention above, file on remote server is not update.
Output from gitlab job is:
Fetching changes...
HEAD is now at 595db67 as
Checking out 595db67b as master...
Skipping Git submodules setup
$ which ssh-agent || ( apt-get update -y && apt-get install openssh- client -y )
/usr/bin/ssh-agent
$ eval $(ssh-agent -s)
Agent pid 40589
$ chmod 600 ~/.ssh/id_rsa_gitlab && chmod 700 ~/.ssh
$ [[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
$ git branch
* (HEAD detached at 595db67)
master
production
$ git push -f production master
Everything up-to-date
$ echo "Deployed to production!"
Deployed to production!
Job succeeded
What I am doing wrong? Can you someone help me please to figure it out? Thank you for all your answers.

Related

Gitlab - deploy changes only using FTP

We’re running an older gen project and we need to deploy pushes to our main branch using LFTP from Gitlab.
The problem we’re having is that each push uploads all of the files instead of only changes. Currently our pipeline looks like this:
image: ubuntu:18.04
before_script:
- apt-get update -qy
- apt-get install -y lftp
build:
script:
# Sync to FTP
- lftp -e "set ftp:ssl-allow no;open $FTP_IP; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose -n localDir/ remoteDir/; bye"
I’ve googled what to do, but didn’t find a clear answer. Can anyone help me with this situation?
Thanks
The problem is that the mirror command is doing a full sync instead of just changes. To solve this, you can use the --only-newer option with the mirror command.
So the only-newer option didn't work due to the timestamps not being correct. There's a tool in GIT that restore timestamps so I just chucked that in and it now works as it should...
Looks like this now:
build:
image: ubuntu:latest
stage: build
script:
- apt update
- apt install git-restore-mtime -y
# - ls -la
# This command restores the modified timestamps from commits
- /usr/lib/git-core/git-restore-mtime
# - ls -la
- apt-get update -qy
- apt-get install -y lftp
# Sync to FTP
- lftp -e "set ftp:ssl-allow no;open $FTP_IP; user $FTP_USERNAME $FTP_PASSWORD; mirror -X .* -X .*/ --reverse --verbose --only-newer -n localDir/ remoteDir/; bye"

How to run Jmeter script on Github

Could somebody help me run my Jmeter script to our Github? FYI the Jmeter I'm using different plugins. Your response is highly appreciated. Thank you so much
This is how I install my Jmeter machine on linux box/playground
sudo apt-get update
sudo apt install curl -y
sudo apt install -y default-jdk
sudo curl -O https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.3.tgz
sudo tar -xvf apache-jmeter-5.3.tgz
cd apache-jmeter-5.3/lib
sudo curl -O https://repo1.maven.org/maven2/kg/apc/cmdrunner/2.2.1/cmdrunner-2.2.1.jar
cd ext/
sudo curl -O https://repo1.maven.org/maven2/kg/apc/jmeter-plugins-manager/1.6/jmeter-plugins-manager-1.6.jar
cd ..
sudo java -jar cmdrunner-2.2.1.jar --tool org.jmeterplugins.repository.PluginManagerCMD install-all-except jpgc-hadoop,jpgc-oauth,ulp-jmeter-autocorrelator-plugin,ulp-jmeter-videostreaming-plugin,ulp-jmeter-gwt-plugin,tilln-iso8583
Output: Jmeter script able to run on Github.
What do you mean by "Jmeter script able to run on Github"? Github is one (of many) implementations of a Git repository, it only stores files and their version history, you cannot "run" anything there.
If you're talking about Github Actions then just use run keyword and put your commands there.
Example workflow definition would be something like:
name: CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: setup-jmeter
run: |
sudo apt-get update
sudo apt install curl -y
sudo apt install -y default-jdk
sudo curl -O https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-5.3.tgz
sudo tar -xvf apache-jmeter-5.3.tgz
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib && sudo curl -O https://repo1.maven.org/maven2/kg/apc/cmdrunner/2.2.1/cmdrunner-2.2.1.jar
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib/ext && sudo curl -O https://repo1.maven.org/maven2/kg/apc/jmeter-plugins-manager/1.6/jmeter-plugins-manager-1.6.jar
cd $GITHUB_WORKSPACE/apache-jmeter-5.3/lib && sudo java -jar cmdrunner-2.2.1.jar --tool org.jmeterplugins.repository.PluginManagerCMD install-all-except jpgc-hadoop,jpgc-oauth,ulp-jmeter-autocorrelator-plugin,ulp-jmeter-videostreaming-plugin,ulp-jmeter-gwt-plugin,tilln-iso8583
- name: run-jmeter-test
run: |
$GITHUB_WORKSPACE/apache-jmeter-5.3/bin/./jmeter.sh -n -t test.jmx -l result.jtl
Also be informed that according to JMeter Best Practices you should be using the latest version of JMeter so consider upgrading to JMeter 5.5 or whatever is the latest stable version which is available at JMeter Downloads page

fatal: Commit found, use 'git ftp push' to sync

I am getting this error in my circleci yaml file when uploading to my ftp site via my github, there is definitely a change for it to commit but i'm getting this error:
HEAD is now at 76a63d8 Updated config.yml
fatal: Commit found, use 'git ftp push' to sync. Exiting.
..
Even though I have that in my yaml file:
version: 2
jobs:
deploy:
docker:
- image: circleci/node:8-browsers
working_directory: ~/repo
steps:
- checkout
- run: npm install
- run:
name: Deploy Master Branch
command: |
sudo apt-get update
sudo apt-get -qq install git-ftp
echo "Deploying project ..."
git ftp init --user ${username} --passwd ${password} ${ftp_location}
- git reset --hard
- git ftp push
workflows:
version: 2
master-deploy:
jobs:
- deploy:
filters:
branches:
only: main
yes i am also getting the same error and i fixed it with this code
sudo apt-get update
sudo apt-get -qq install git-ftp
git config git-ftp.user ${username}
git config git-ftp.url ${ftp_location}
git config git-ftp.password ${password}
git ftp push
Please follow this code

How to deploy with .gitlab-ci.yml in runner used by docker?

I installed docker and gitlab + a runner using this tutorial: https://frenchco.de/article/Add-un-Runner-Gitlab-CE-Docker
The problem is that when I try to modify the .gitlab-ci.yml to make a deployment on my host machine I can not do it.
My .yml :
stages:
- deploy
deploy_develop:
stage: deploy
before_script:
- apk update && apk add bash && apk add openssh && apk add rsync
- apk add --no-cache bash
script:
- mkdir -p ~/.ssh
- ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
- cat ~/.ssh/id_rsa.pub
- rsync -hrvz ~/ root#172.16.1.97:~/web_dev/www/test/
environment:
name: develop
And the problem is that in ssh or rsync I always have the same error message in my job:
$ rsync -hrvz ~/ root#172.16.1.97:~/web_dev/www/test/
Host key verification failed.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.3]
I tried to copy the ssh id_rsa and id_rsa.pub in the host, it's the same.
Surely a problem because my runner is in a docker can be? It is strange because I manage to ping my host (172.16.1.97) since the execution of the .yml. An idea has my problem?
Looks like you did not add the public key into your authorized_keys on the host server for the deploy-user?
For example, I use gitlab-ci to deploy my webapp, and therefore I added the user gitlab on my host machine, and added the public key to authorized_keys and then I can connect with ssh gitlab#IP -i PRIVATE_KEY to that server.
My gitlab-ci.yml looks like this:
deploy-app:
stage: deploy
image: ubuntu
before_script:
- apt-get update -qq
- 'which ssh-agent || ( apt-get install -qq openssh-client )'
- eval $(ssh-agent -s)
- ssh-add <(cat "$DEPLOY_SERVER_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
- chmod 755 ./deploy.sh
script:
- ./deploy.sh
where I added the private key's content as a variable to my gitlab-instance. (see https://docs.gitlab.com/ee/ci/variables/)
The deploy.sh looks like this:
#!/bin/bash
set -eo pipefail
scp app/docker-compose.yml gitlab#"${DEPLOY_SERVER_IP}":~/apps/${NGINX_SERVER_NAME}/
ssh gitlab#$DEPLOY_SERVER_IP "apps/${NGINX_SERVER_NAME}/app.sh update" # this is just doing docker-compose pull && docker-compose up in the app's directory.
Maybe this helps? It's working fine for me and scp/ssh are giving more intuitive error messages than what you got with rsync in this particular case.

Pushing on a Git repository with Travis-CI

I want to execute a script every time I push into the master, this script will create some files that I want to commit and push. The log of the Travis build seems to be adding the files, committing and pushing, but nothing happens.
My .travis.yml is this one:
before_install:
- openssl aes-256-cbc -K $encrypted_290c3b2c061d_key -iv $encrypted_290c3b2c061d_iv -in id_rsa.enc -out /tmp/id_rsa -d
- eval "$(ssh-agent -s)" # Start the ssh agent
- chmod 600 /tmp/id_rsa
- ssh-add /tmp/id_rsa
install:
- wget --user $docencia_ac_username --password $docencia_ac_password http://docencia.ac.upc.edu/FIB/grau/PEC/Protegido/Documentacion/eines-sisa-64BITS.tgz
- tar -xf eines-sisa-64BITS.tgz
- export PATH=$PATH:$(pwd)/eines-sisa/bin
script:
- chmod +x ./compile_to_hex.sh
- "./compile_to_hex.sh"
after_success:
- rm -rf $TRAVIS_BUILD_DIR/tmp/
- git config --local user.name "Marc43"
- git config --local user.email "my mail"
- git add $TRAVIS_BUILD_DIR/hex/*
- git commit -m "At $(date) hex files builded - travis-ci [skip ci]"
- git push git#github.com:Marc43/sisa_hexbuilder.git master > /dev/null 2>&1
before_deploy:
- rm -rf eines-sisa*
in the git user.email it really goes my email but I decided to delete it for the question.
And the travis log for the build is:
$ git add $TRAVIS_BUILD_DIR/hex/*
$ git commit -m "At $(date) hex files builded - travis-ci [skip ci]"
[detached HEAD 10e7e48] At Sun Apr 15 08:06:17 UTC 2018 hex files builded - travis-ci [skip ci]
4 files changed, 21 insertions(+)
create mode 100644 hex/exemple.hex
create mode 100644 hex/joc_io.hex
create mode 100644 hex/tb_branch.hex
create mode 100644 hex/tb_sum.hex
I know that there is another way to do this via GitHub tokens or something like that, anyway I don't know how to do it one way or the other. I've tried to do it with the deploy too but it never uploaded my files, just tagged the same commit I did. Any ideas?
Thank you,
Marc
I run several repositories that need documentation compiling to distributable formats (e.g. AsciiDoc to HTML, MD to PDF), rather than having to build and commit every time I want to update the distributable, I’d like to automate this process. This is where I use TravisCI as a build server.
before_install:
- sudo apt-get install pandoc
- gem install asciidoctor
script:
- make
after_success:
- .travis/push.sh
env:
global:
secure: hZJlqgOzA2zIUJSWIka0PylqNaTkfHq+kS48RrHmocrK0vLyCW7ECWrzez2f2RVdTNzPi0b+yJq2uCbFfWjImZqg+XY1I75/CVVdSYMk7PJkYZ/iBDixMYY8CAkRRd5yZft9uZAdZzR4KLCPN18n7qfISv/M9VA8989NKcVyiEU=
push.sh
#!/bin/sh
setup_git() {
git config --global user.email "travis#travis-ci.org"
git config --global user.name "Travis CI"
}
commit_website_files() {
git checkout -b gh-pages
git add . *.html
git commit --message "Travis build: $TRAVIS_BUILD_NUMBER"
}
upload_files() {
git remote add origin-pages https://${GH_TOKEN}#github.com/MVSE-outreach/resources.git > /dev/null 2>&1
git push --quiet --set-upstream origin-pages gh-pages
}
setup_git
commit_website_files
upload_files
Reference - https://gist.github.com/willprice/e07efd73fb7f13f917ea