I have a Gradle project that consists of a master project and 2 others that included using includeFlat directive. Each of these 3 projects has its own repo on GitHub. To build it I checkout all 3 projects into a common top folder then cd into the master project and run gradle build. And it works great!
Now I need to deploy the resulting app to AWS EB (Elastic Beanstalk) which is also works great when I produce the artifact locally and then deploy it manually. I want to automate the process so I'm trying to set it up using CodePipelines + Jenkins as described in this document adjusted for Gradle.
The problem is that if I specify 3 Sources in the pipe I end up with my projects extracted on top of each other creating a mess in Jenkins workspace. I need to somehow configure each project to be output to its own directory within Jenkins workspace and I just don't see a way to do it (at least in UI)
Then, of course even if I achieve what I want I need somehow to cd into the master directory to run gradle build and again I'm not sure how to do that
P.S. Great suggestions from #Phil but unfortunately is seems that CodePipeline does not currently support Git submodules or subtrees
I would start common build, when changes happened on any of 3 repos. With say 5 minutes delay, to have single build, even if changes are introduced to more then one repo.
I can't see good way to deal with deployment in other way than using eb deploy... old way... Please install aws tools at your jenkins machine. Create deployment job triggered on successful build. And put bash script doing deployment there. Please put more details about your deployment, that way I can help with deployment script.
Related
I'm having a hard time figuring out how to correctly deploy to different environments with TeamCity (in terms of cross BuildConfiguration dependencies) and hope to get some input as to how to configure my SubProjects/BuildConfigurations properly. Lets start based on a concrete example: I made this test "TeamcityConfigurationTests" to better learn how TeamCity handled dependencies, and the current state shows the result i am looking for:
I have 3 subProjects, Dev, Test and Prod - and all associated tasks for those "environments" as seperate build configurations within that subProject. This is to more clearly visualize what is going on, and if anything breaks, to be able to see immediately what is broken (separate Build, UnitTest and DeployToDev BuildConfigurations, rather than 3 different steps in one single Build Configuration).
Ideally, i only want to build my application once in the Dev.Build step, and let the Dev.UnitTest and Dev.DeployToDev steps grab that artifact and run tests and deploy. That i got going for me, by having snapshot and artifact dependencies. But i am having trouble getting the correct artifact when i want to deploy from Dev -> Test or Test -> Prod.
My issue is to correctly reference the latest successfully DEV deployed artifact when running Test.DeployToTest - and the same for getting the latest successfully TEST deployed artifact when running Prod.DeployToProd. (Essentially i want to promote the artifact to the next environment).
Now, my issue is, if I in the Test.DeployToTest have a SnapshotDependency to Dev.DeployToDev and an artifact dependency to Dev.Build, and the VCS source has changed since Deploy to Dev has run, it triggeres running all the DEV steps again. This is not the worst part, the same happens when i run Prod.DeployToProd if the VCS source changed since the initial build on dev (because of all the snapshot dependencies). Meaning, that rather than promoting Test -> Prod, I Build and deploy whatever is currently on VCS to Dev, Test AND Prod.
How am i supposed to set this up correctly?
The only other option i am aware of, is letting Dev.DeployToDev also publish the same artifact, and only have an (LatestSuccessful) ArtifactDependency in Test.DeployToTest. I would also have to publish the artifact again in Test.DeployToTest, for letting Prod.DeployToProd only have a (LatestSuccessfull) artifact dependency to Test.DeployToTest. (This would be to get rid of the SnapshotDependencies causing previous environments to run build/deploy again in case of VCS changes). But then i am publishing the artifact 3 times, rather than just the one time when the application is originally built in DEV - which i would like to avoid. Also, i have cases where no artifact is needed for deploying to Test and Prod, so there is no artifact to depend on (essentially i only need the BuildNumber from the "Dependent" environment i want to promote from).
I hope for some input. Thank you
Regards
Frederik
For anyone wondering, i made a JetBrains support ticket, and got the following response:
Basically, there are options to resolve your case:
Option 1: use "Promote" action form the build's Actions top-right menu
(or change the type of the Deploy* configurations to deployment and
use the action from the block on the build results. This is the
preferred way: before deploying you select the build to deploy and
"promote" it to the next environment. There is also an experimental
hidden feature to hide the "Run" button: add
"teamcity.ui.runButton.caption" configuration parameter in the build
configurations to empty value.
Option 2: do not use snapshot dependency, use only artifact dependency
on the latest successful build. However, when you run the build you
cannot be sure that the last successful build you see will be
deployed: while the build is standing int he queue, another
Dev.DeployToDev can finish and then be deployed as the last
successful.
We went with option 1
I am using VSTS for my OPA5 Tests, so all works for one project. For this I created a Build for these Projects i wanted to test.
But if I want to test all projects, do I need to create a build for all Project or is there a solution to build all projects with one build definition?
The build should do always the same things, saved in a YAML File.
I have seen thats is possible to do builds with difficult branches but not with difficult repositories.
So has anyone a solution for this or is it impossible at the moment?
Yes, it's possible.
You just need to clone another git repositories at the beginning of the build.
So you can add a PowerShell task as the first task and execute git clone command.
And If you are using YAML file, just add the script to execute the PowerShell task.
Besides, you can also refer the post VSTS build from multiple repositories.
I am working on a project with multiple people, a website application which requires webpack to be built, uglified, concatenated into a few files e.g. app.min.js, style.min.css etc. - As a result of this, in an effort to prevent merge conflicts we recently added the build folder to .gitignore, under the assumption that we would be able to build during deployment.
When pushing to the Master branch, we automatically "deploy" through Semaphore CI (similar to Travis) which runs composer install, npm install, and finally "npm run build" which triggers the webpack build. This is all built and then tested on the CI side of things, and then Semaphore automatically deploys to Amazon's Elastic Beanstalk where our application is hosted.
The problem with this is, it seems Semaphore doesn't upload the build it's just tested, but rather the Master branch itself which has no built JS or CSS. I'm wondering if there's a way to push these built files to deployment as well, or if running the entire build process AGAIN on Elastic Beanstalk is the only route. It seems unnecessary to have to do that process essentially 3 times, locally, CI, and then deployment. Every time a step like this is needed on EB the actual re-instantiation time gets longer, which I'd like to keep as short as possible.
Obviously if building it a 3rd time on EB is the only way to go about this then I'll have to, just wondering if there are better solutions for this whole workflow.
I haven't worked with Semaphore CI, but you might be able to use an .ebignore file.
If you create one, the cli will use that instead of your .gitignore file.
I find in some deployment situations you want the inverse of your .gitignore (all compiled, no src). It essentially lets you pick the files from your project directory that you want to deploy, in the same way as the .gitignore file.
Edit: I just noticed the documentation on aws is lacking. It only mentions file exclusion, but you can include files too.
Edit 2: I don't think Semaphore supports the use of .ebignore, so right now this solution isn't of any use. :(
I just had a great first experience with https://deploybot.com/. The can deploy directly to elastic beanstalk. It might be interesting or you.
I am trying to deploy a Bamboo Deployment Project. On manual deploy the project only gets queued and never starts.
I have three Tasks:
Clean working directory
Dowload war artifact
Deploy to server with a script
I also tried my deployment project it with one simple echo script task for debugging purposes. This also never executes and no logs appear.
Can anyone help or knows a possible reason for this problem?
This may happen in two cases
The agent with which you are trying to build may not have the necessary capabilities.
The agent might be dedicated to some other projects.
The solution is to add necessary capabilities in the first case or to dedicate the agent to deployment project also.
I've been researching this for a while but can't find an answer.
I use Bamboo 5.3 with Artifactory plugin 1.6.2. I have a build project that generates a .war and two .zips. I also have a Bamboo Deployment project that creates releases with these three files and deploys to DEV, QA and so on.
For a build project I am able to use the artifactory plugin, that's fine. The problem is that I end up with a lot of artifacts if I publish all the builds. I would like to publish to Artifactory only the files from the releases, so that is happens less often, and that the people would see only the 3-4 releases tries, not the 150 builds.
My issue is that when creating my Deployment tasks (like download, copy, call ssh script...) there is no 'Artifactory Generic Deploy', like in the build project tasks.
I see there is a new Bamboo 5.4 with some improvement around the deployment process, maybe this could help?
Support for deployment task from Bamboo to Artifactory will be available starting with version 1.8.0 of the Artifactory plugin.
Here is the Jira issue.
I faced a similar issue. Hopefully the next release of the artifactory plugin will integrate with deployment projects.
If you are willing to use Maven to broker the deployment, deploy-file can get the job done.
In the deployment project, after your artifact download task add a Maven 3.x task for each artifact you want to send.
You'll need to specify a build JDK and for environment variables I'm using MAVEN_OPTS="-DskipTests=true -XX:MaxPermSize=4096m"
For the actual maven command:
deploy:deploy-file
-Durl=http://${bamboo.artifactory_username}:${bamboo.artifactory_password}#${bamboo.artifactory_url}/artifactory/${bamboo.destinationRepo}
-DrepositoryId=localhost
-Dfile=${bamboo.pathToArtifact}/${bamboo.arftifactName}-${bamboo.majorVersion}.${bamboo.minorVersion}.${bamboo.arftifactExtension}
-DgroupId=${bamboo.arftifactGroup}
-DartifactId=${bamboo.arftifactName}
-Dversion=${bamboo.majorVersion}.${bamboo.minorVersion}
-Dpackaging=${bamboo.arftifactExtension}
-DgeneratePom=true
Hope this helps!
The artifactory API is pretty usable for this purpose. You can deploy directly using curl in a shell script.
See https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API for the details.