I hope the question is somehow explaining, what I want to do. If not, I'll try to give a more detailed explanation:
I am currently migrating a big project from SVN to Git. In this project, all the depencencies where stored in different folders and where commited to the SVN repo. I already learned, that you should not put the vendors folder in your repo. But how would I than update the depenencies? In another ressource I read, that you should only versioning the composer.lock file. How about the composer.json file?
Here is the strategy, I had in mind:
Excluding the vendor folder in the .gitignore file
Adding composer.json and composer.lock file to the repo
Adding a composer_update.php file to use it in a post-receive hook
When we have to update some dependencies, we have than to do the following:
Updating the composer.json file
Running composer update locally to update the composer.lock file (or the composer_update.php script in your local dev instance)
Pushing the changes to Stash/Bitbuckt/Github, which will than execute the composer_update.php script through the post-receive hook
Would you recommend something like this, or is there a better way to do it? I am sorry, but I am really new to composer.
You are supposed to commit both composer.json and composer.lock, as these two files are essential for Composer operation. I bet you misread on what not to commit - from all the things a run with Composer creates, you ONLY commit the lock file, and nothing else.
Commit both the composer.json and composer.lock file.
To update dependencies while development just run composer update in a terminal.
To update dependencies on production run composer install in a terminal. You may add this command to the post-receive hook.
Related
I'm using a module for my angular app called angular-translate. However, I've had to make a few small modifications to the source code to get everything working the way I'd like, and now I want to persist those changes on npm install. A colleague suggested that I fork the repo of the source code and point to my forked repo as a dependency, which I've tried in these ways, e.g.
npm install https://github.com/myRepo/angular-translate
npm install https://github.com/myRepo/angular-translate/archive/master.tar.gz
The first gives me a directory like this with no build. Just a package.json, .npmignore, and some markdown files
-angular-translate
.npmignore
.nvmrc
CHANGELOG.md
package.json
etc
The second npm install gives me the full repo, but again I don't get a build like when I use the command npm install angular-translate. I've seen some dicussion of running the prepublish script, but I'm not sure how to do this when installing all the modules. I've also tried publishing the fork as my own module to the npm registry, but again I get no build, and I'm not sure that's the right thing to do...
I apologise for my ignorance on the topic. I don't have a huge amount of experience with npm. Would love to get some feedback on this issue. It seems like it could be a common enough issue when modifications need to be made to a package's source code? Maybe there's a better solution?
Try npm install <ghusername>/<repoName>, where <ghUsername> is your GitHub username (without the #) and <repoName> is the name of the repository. That should correctly install it. You will most likely want to use the --save or --save-dev flag with the install command to save dependency in your package.json.
If that isn't working correctly, check the contents of your .npmignore file.
Don't panic if the install command takes a long time; installing from a git repository is slower than installing from the npm registry.
Edit:
Your problem is that in your case, dist/ is not committed to the repo (since it is in the .gitignore). That is where the actual code lives. dist/ is built from the files in src/ before the package is published to the npm registry, but dist/ is never committed to the repo.
It's ugly, but in this case you will have to remove dist/ from the .gitignore and then run:
npm run build
git add .
git commit
git push
(Ensure that you have run npm install first)
You then should be able to install from github.
There might be another way to do this using a prepare script, but I'm not sure if that's possible; I've never tried it. Edit: Cameron Tacklind has written an excellent answer detailing how to do this: https://stackoverflow.com/a/57829251/7127751
TL;DR use a prepare script
and don't forget package.json#files or .npmignore
Code published to npmjs.com is often not what's in the repository for the package. It is common to "compile" JavaScript source files into versions meant for general consumption in libraries. That's what's usually published to npmjs.com.
It is so common that it's a feature of npm to automatically run a "build" step before publishing (npm publish). This was originally called prepublish. It seems that Npm thought it would be handy to also run the prepublish script on an npm install since that was the standard way to initialize a development environment.
This ended up leading to some major confusion in the community. There are very long issues on Github about this.
In the end, in an effort to not change old behavior, they decided to add two more automatic scripts: prepublishOnly and prepare.
prepublishOnly does what you expect. It does not run on npm install. Many package maintainers just blindly switched to this.
But there was also this problem that people wanted to not depend on npmjs.com to distribute versions of packages. Git repositories were the natural choice. However it's common practice to not commit "compiled" files to git. That's what prepare was added to handle...
prepare is the correct way
If you have a repository with source files but a "build" step is necessary to use it,
prepare does exactly what you want in all cases (as of npm 4).
prepare: Run both BEFORE the package is packed and published, on local npm install without any arguments, and when installing git dependencies.
You can even put your build dependencies into devDependencies and they will be installed before prepare is executed.
Here is an example of a package of mine that uses this method.
Problems with .gitignore
There is one issue with this option that gets many people.
When preparing a dependency, Npm and Yarn will keep only the files that are listed in the files section of package.json.
One might see that files defaults to all files being included and think they're done.
What is easily missed is that .npmignore mostly overrides the files directive and, if .npmignore does not exist, .gitignore is used instead.
So, if you have your built files listed in .gitignore, like a sane person, and don't do anything else, prepare will seem broken.
If you fix files to only include the built files or add an empty .npmignore, you're all set.
My recommendation
Set files (or, by inversion, .npmignore) such that the only files actually published are those needed by users of the published package. Imho, there is no need to include uncompiled sources in published packages.
Original answer: https://stackoverflow.com/a/57503862/4612476
Update for those using npm 5:
As of npm#5, prepublish scripts are deprecated.
Use prepare for build steps and prepublishOnly for upload-only.
I found adding a "prepare": "npm run build" to scripts fixed all my problems.
Just use the command npm install git+https://git#github.com/myRepo/angular-translate.git. Thanks.
To piggyback off of #RyanZim's excellent answer, postinstall is definitely a valid option for this.
Either do one of the following:
Update the package.json in your forked repo to add a postinstall element to scripts. In here, run whatever you need to get the compiled output (Preferred).
Update your package.json, and add a postinstall that updates the necessary directory in node_modules.
If you've forked another persons repository, then it might be worth raising an issue to illustrate the issue that installing their package through GitHub does not work as it does not provide the necessary means to build the script. From there, they can either accept a PR to resolve this with a postinstall, or they can reject it and you can do #2.
If you are using yarn like me. Imagine that you want to use a package like this:
yarn add ghasemikasra39/gridfs-easy --save where the ghasemikasra39 is the username and gridfs-easy is the name of the repo
I've got Composer installed and I've used it to download HTMLPurifier locally. Now I'd like to push that download to my OpenShift Git repo. So, in a Git Bash window, I run the following...
git add -A :/
git commit -a -m "Uploading HTML Purifier"
git push origin master
At this point Git reports that the push was successful but when I ls the directory through SSH, it shows that the HTMLPurifier directory is empty. Why is that? How do I get Git to push those files?
Additional Info: I noticed that the HTMLPurifier directory is indeed a Git repo itself and contains a .gitignore file in its root directory. I tried deleting it and re-running the above commands but to no avail...
You should try to avoid pushing downloaded dependencies into a repository. It is recommended to add the vendor directory into the .gitignore file at top level. But what you must do instead is commit and push both composer.json and composer.lock.
Here's why: The vendor directory is managed by Composer. Running Composer will probably do minor things during an update, but may also be doing heavy stuff if the Composer team decides to optimize things.
Also, if you require a branch of a package, and Composer knows the repository of that package, it will default to cloning a Git repo or do a SVN checkout instead of trying to grab a ZIP package of that branch (often there is no way to get such a package for branches, and even tagged versions in a plain Git repository do not have such download ability. Composer knows that Github offers such downloads, and detects Github by looking at the repo URL.)
So you can assume that Composer will put a lot of repository meta data into the vendor file, and if you blindly commit these, things will get ugly. First of all, you are committing way too many files, increasing your repository by an unnecessary amount, which will slow down things. Then, if cloning Git repositories, these will be treated as submodules, and that has another bunch of nastiness I am told. If you are just learning Git, it probably isn't a good idea to start with these. And if you are sufficiently known to the tools (Git and Composer), you probably won't need them either.
There really is only one reason why you'd try to commit a modified version of the vendor directory: If your release process is completely depending on all files being present in your one repository, without any way to run a composer install during the release to make these files appear on the target server.
In such cases, you'd install or update the packages with Composer, and then go through all created directories and delete any .git and .svn (and probably also .hg for Mercurial) folders you encounter. Only then you'd be able to commit the files into your own repository.
But note that this step might be a tedious step to do manually - you probably want to create an update script that does all that work for you. You also might run into issues when updating dependencies because Composer expects files to simply go away when deleted, and not be in the way when being written. I cannot tell you exactly what you'd be experiencing because it depends on how you'd do stuff, but I expect you stumbling upon random puzzling issues.
Bottom line: Avoid committing the dependencies into your own repository if possible.
Try using the -force option, you will also most likely need to delete the .git directory inside the HTMLPurifier directory too.
When we do "yo webapp" (assuming webapp generator is installed), it scaffold projects which contains file relevant to bower, grunt and then there is app folder, which we all know what's it about.
My question is, out of this structure what are the files that needs to be maintained in SCM, Should it be only app directory or should it whole structure ?(assuming there are no additional grunt task or any build file changes from earlier scaffolding)
Yeoman webapp generator will produce a .gitignore file which includes files that should not be committed to a SCM. This file includes the following directories:
node_modules
dist
.tmp
.sass-cache
bower_components
test/bower_components
It is clear that .tmp and .sass-cache have no reason to be in the repo as they both are only temporary.
There is however a discussion whether bower (and rarely node) dependencies should be checked in. For most projects I recommend not to.
Please note that in either case one should never change the packages directly in the bower_components or node_modules folder as any change will be lost at next bower install or npm install. A fork of the original project (either as a independent repo or to folder in the project - e.g. lib) is a better idea - a follow up pull request would then add a lot of karma :)
The dist folder with the build of the application may be committed depending on your deployment method. There is a very good guide on deployment on Yeoman site.
As a start, you should put everything into SCM with the exception of app/bower_components, test/bower_components and node_modules. All files under these directories come from public repo, either node or bower repo.
In this setup, whenever another developer checkout from SCM, he needs to run 2 commands: npm install and bower install. What I typically do is I create a file called install.sh (install.bat on Windows) and have these 2 commands inside this script file. In this way, when you find that you need to run more commands for initialization, you can easily add to this script file and new developers can just checkout and run install.sh.
In some cases, I found that I need to perform small modification to a public library. In this case, I will check this library inside bower_components into SCM as well. This is not common, but it happens.
I have a Laravel 4 project, and I would like to know which files should be ignored when using a version control software such as Git, Mercury or SVN?
The structure of my project looks like the following screen capture.
I'm pretty new to Composer so I'm not very clear about what goes to a repo what not. If someone can post their .gitignore file or their SVN ignore property, it could be handy.
For reference, that .gitignore file can be found here:
/bootstrap/compiled.php
/vendor
composer.phar
composer.lock # Remove this one after you create a project
.env.*.php
.env.php
.DS_Store
Thumbs.db
As noted in the below comment, you probably want to commit composer.lock in your project. Laravel ignores it by default so the authors of the laravel/laravel package don't accidently impose packages on you.
Your project should include the composer.lock file so you can install packages of stable versions (via composer install instead of composer update) properly in your production environments.
Note that the config file:
app/config/app.php
Has a cryptographic key in it that wouldn't be great to commit to a repository. Or, at least, the file needs to be overwritten in production.
You might also want to see the Laravel docs here and here. This discusses how to setup different Laravel configurations for different environments and protect sensitive information. All your .env.local.php type files should not be included in version control. Note that the .env.*.php and .env.php is added in the default Laravel .gitignore file. You can see it here
Laravel has posted their .gitignore on GitHub, which can be found here.
As of today, it looks like this:
/bootstrap/compiled.php
/vendor
composer.phar
composer.lock
.env.*.php
.env.php
.DS_Store
Thumbs.db
GitHub has a repository of suggested .gitignore files for almost all kinds of projects at: http://github.com/github/gitignore
Alternatively, you can search it for your project using this handy and extremely useful online tool: http://www.gitignore.io
I have a Symfony project, which was always held in an svn project.
Now I am trying to make the move to git (on github).
The problem is that
git status
always tells me that there are modified and new files/folders, which I cannot seem to commit...
When I
git add
these files/folders, then commit will just tell me I have nothing to commit, and that the files/folders are still untracked?
All the other stuff in my Symfony project is committed and pushed file, its just certain stuff in the vendors folder which I can't seem to get going.
How can I fix this?
ps: I am also having trouble getting composer to update my vendors.
My best practice is to completely ignore the vendors directory, and install all the vendors bundle with composer on every development machine using composer.json and composer.lock (thus, you will need to add these files to your repo!)
i guess it has to do with the fact that the vendors also have .git folders inside of them
I confirm a nested git repo won't respond well to git command, if that nested repo already consider said files committed.
Or, at the very least, you should add those files while being within that nested repo (not from a parent directory).
However, a git status done in a parent directory should ignore the content of that nested repo. See How to make top-level git to track all the files under another sub-directory git .