I'm deploying my app to ElasticBeanstalk. I'm using an .ebignore file because there are files that I do not want to check into git, but I do want deployed with the app(like application secrets, config vars, etc). The issue I'm facing is that when using an .ebignore, the deploy takes FOREVER. I've used the --verbose flag, and I can see that it is recursing my entire node_modules directory and skipping each file individually. When I deploy by using .gitignore, it becomes very fast.
Has anyone else experienced this? How do I speed up this process?
Related
I have a Github repo that includes some large graphical assets. These assets result in failed deploys to Netlify due to Netlify's size restrictions. Is there any way I can keep these files within the Github repo but exclude them from Netlify deploys, in the same way I could use a .slugignore file when deploying to heroku.
Netlify doesn't really have explicit size restrictions, though uploads of files >20MB may fail. Are your files bigger than that? If so, hosting them on Netlify's CDN also doesn't make sense as the CDN edge cache will ignore them and they'll load slowly for browsers anyway.
To not deploy them, the most straightforward way is to remove them after your build, something like this:
npm run build && rm dist/hugefile1.jpg dist/subdir/hugefile2.pdf
You can get fancier and use a file to list them or just look for everything huge. Warning - something huge that it DOES make sense to host is your sourcemap (if you use one), so watch out for what this might catch!
npm run build && find dist -type f -size +20M
Effectively - you can do anything you could do in a shell script. NB: You need to make sure that your build pipeline fails if any necessary step fails - this is why the examples show && to chain commands rather than ; (build could fail, find succeed, and we publish an empty site!).
More details here: https://www.netlify.com/blog/2016/10/18/how-our-build-bots-build-sites/ and you can test out your scripts using the methodology described here: https://github.com/netlify/build-image#testing-locally
I am deploying to AWS EB from Codeship. Codeship does an npm install to run the tests. It them bundles everything and sends it to AWS, where another npm install happens.
How do I prevent Codeship from bundling my node_modules folder?
The integrated Elastic Beanstalk deployment is based on copying the files over to AWS, so if you want to "ignore" a folder, add a script based deployment before the Elastic Beanstalk deployment and remove the folders you don't want to copy over.
See https://github.com/codeship/scripts/blob/master/deployments/elastic_beanstalk.sh for a script that is very similar (though not quite identical) to the commands run for the integrated deployment.
And see https://documentation.codeship.com/basic/builds-and-configuration/deployment-pipelines/#multi-step-deployment-pipelines for a bit more information on deployment pipelines containing multiple individual steps.
I am working on a project with multiple people, a website application which requires webpack to be built, uglified, concatenated into a few files e.g. app.min.js, style.min.css etc. - As a result of this, in an effort to prevent merge conflicts we recently added the build folder to .gitignore, under the assumption that we would be able to build during deployment.
When pushing to the Master branch, we automatically "deploy" through Semaphore CI (similar to Travis) which runs composer install, npm install, and finally "npm run build" which triggers the webpack build. This is all built and then tested on the CI side of things, and then Semaphore automatically deploys to Amazon's Elastic Beanstalk where our application is hosted.
The problem with this is, it seems Semaphore doesn't upload the build it's just tested, but rather the Master branch itself which has no built JS or CSS. I'm wondering if there's a way to push these built files to deployment as well, or if running the entire build process AGAIN on Elastic Beanstalk is the only route. It seems unnecessary to have to do that process essentially 3 times, locally, CI, and then deployment. Every time a step like this is needed on EB the actual re-instantiation time gets longer, which I'd like to keep as short as possible.
Obviously if building it a 3rd time on EB is the only way to go about this then I'll have to, just wondering if there are better solutions for this whole workflow.
I haven't worked with Semaphore CI, but you might be able to use an .ebignore file.
If you create one, the cli will use that instead of your .gitignore file.
I find in some deployment situations you want the inverse of your .gitignore (all compiled, no src). It essentially lets you pick the files from your project directory that you want to deploy, in the same way as the .gitignore file.
Edit: I just noticed the documentation on aws is lacking. It only mentions file exclusion, but you can include files too.
Edit 2: I don't think Semaphore supports the use of .ebignore, so right now this solution isn't of any use. :(
I just had a great first experience with https://deploybot.com/. The can deploy directly to elastic beanstalk. It might be interesting or you.
I'm trying to figure out the best way to ignore some folders and files from my Codeship deployment process. At the moment it compiles all my assets as part of the deployment process but I don't really want it uploading node_modules to the server.
Is there a way to ignore the folder or remove the folder before deployment?
I tried deleting it after I ran grunt but I think it gets cached as it didn't work.
The method I used was to rm -r YOUR_PATH/node_modules in the test pipeline after Grunt has been run.
This however seemed to have some issues when running on FTP deploy. For any of the SSH deploys it seems to work fine.
As part of a Capistrano deployment I want to sync my CSS, JS and images to a remote location (Amazon S3). However, I'm worried that if the Capistrano deployment fails, this will leave me with updated CSS, JS and images.. but the main application code will still be on the previous release.
I'm wondering if there is a way with Capistrano to only trigger a task if the deployment is marked as complete and current folder is pointing at the newly deployed release folder? This would at least allow me to only update the static files once I know the main source has been updated..
Of course, I would still have the problem of - what if the sync with S3 fails? But I think this way would have the better balance in terms of points of failure.
Capistrano has a bunch of hooks you can use during the deploy.
http://capistranorb.com/documentation/getting-started/flow/
You probably want to use the 'deploy:finished' hook for your use case, for example.
after 'deploy:finished', 'deploy:sync_assets_to_S3'
Then create a cap task to perform your file uploads.