How to config the _data folder to another git repository? The whole folder is clone from github and want to continue due to easy update. But the _data folder need to sync with another data repository. How can I config it?
Actually just one yaml file : _data/authorlist.yaml
What to put into package.js if want a script to sync the yaml file with github?
Thank you.
You can use Git submodules to keep a shared repository of data files that you can use in multiple projects. This will allow you to keep a reference to the data repository in your projects, and pull updates to this repository with a single command. The great thing is that submodules are a built-in feature of Git, so it's independent of any NPM scripts, environments (like a bash script would be) or frameworks. See the link above for documentation on how to set up and work with submodules.
Related
To set expectations, I'm new to build tooling. We're currently using a hosted agent but we're open to other options.
We've got a local application that kicks off a build using the VSTS API. The hosted build tasks involve the Get sources step from a GitHub repo to the local file system in VSO. The next step we need to copy over a large number of files (upwards of about 10000 files), building the solution, and running the tests.
The problem is that the cloned GitHub repo is in the file system in Visual Studio Online, and my 10000 input files are on a local machine. That seems like a bit much, especially since we plan on doing CI and may have many builds being kicked off per day.
What is the best way to move the input files into the cloned repo so that we can build it? Should we be using a hosted agent for this? Or is it best to do this on our local system? I've looked in the VSO docs but haven't found an answer there. I'm not sure if I asking the right questions here.
There are some ways to handle the situation, you can follow the way which is closest to your situations.
Option 1. Add the large files to the github repo
If the local files are only related to the code of the github repo, you should add the files into the same repo so that all the required files will be cloned in Get Sources step, then you can build directly without copy files step.
Option 2. Manage the large files in another git repo, and then add the git repo as submodule for the github repo
If the local large files are also used for other code, you can manage the large files in a separate repo, and treat it as submodule for github repo by git submodule add <URL for the separate repo>. And in your VSTS build definition, select Checkout submodules in Get sources step. Then the large files can be used directly when you build the github code.
Option 3. Use private agent on your local machine
If you don’t want add the large files in the github repo or a separate git repo for some reasons, you can use a private agent instead. But the build run time may not improve obviously, because the changed run time is only the different between copying local files to server and copying local files to the same local machine.
I've got Composer installed and I've used it to download HTMLPurifier locally. Now I'd like to push that download to my OpenShift Git repo. So, in a Git Bash window, I run the following...
git add -A :/
git commit -a -m "Uploading HTML Purifier"
git push origin master
At this point Git reports that the push was successful but when I ls the directory through SSH, it shows that the HTMLPurifier directory is empty. Why is that? How do I get Git to push those files?
Additional Info: I noticed that the HTMLPurifier directory is indeed a Git repo itself and contains a .gitignore file in its root directory. I tried deleting it and re-running the above commands but to no avail...
You should try to avoid pushing downloaded dependencies into a repository. It is recommended to add the vendor directory into the .gitignore file at top level. But what you must do instead is commit and push both composer.json and composer.lock.
Here's why: The vendor directory is managed by Composer. Running Composer will probably do minor things during an update, but may also be doing heavy stuff if the Composer team decides to optimize things.
Also, if you require a branch of a package, and Composer knows the repository of that package, it will default to cloning a Git repo or do a SVN checkout instead of trying to grab a ZIP package of that branch (often there is no way to get such a package for branches, and even tagged versions in a plain Git repository do not have such download ability. Composer knows that Github offers such downloads, and detects Github by looking at the repo URL.)
So you can assume that Composer will put a lot of repository meta data into the vendor file, and if you blindly commit these, things will get ugly. First of all, you are committing way too many files, increasing your repository by an unnecessary amount, which will slow down things. Then, if cloning Git repositories, these will be treated as submodules, and that has another bunch of nastiness I am told. If you are just learning Git, it probably isn't a good idea to start with these. And if you are sufficiently known to the tools (Git and Composer), you probably won't need them either.
There really is only one reason why you'd try to commit a modified version of the vendor directory: If your release process is completely depending on all files being present in your one repository, without any way to run a composer install during the release to make these files appear on the target server.
In such cases, you'd install or update the packages with Composer, and then go through all created directories and delete any .git and .svn (and probably also .hg for Mercurial) folders you encounter. Only then you'd be able to commit the files into your own repository.
But note that this step might be a tedious step to do manually - you probably want to create an update script that does all that work for you. You also might run into issues when updating dependencies because Composer expects files to simply go away when deleted, and not be in the way when being written. I cannot tell you exactly what you'd be experiencing because it depends on how you'd do stuff, but I expect you stumbling upon random puzzling issues.
Bottom line: Avoid committing the dependencies into your own repository if possible.
Try using the -force option, you will also most likely need to delete the .git directory inside the HTMLPurifier directory too.
I've repos with various configuration files, like .vimrc, _vimrc (for windows), or .bashrc. I'd like to create one repo that holds all of them, so I can easily set up my environment on clean machines. I'd like linux / windows specific files to be in different branches (?), so that cloning on linux would not download windows files.
How do I do that, assuming all existing repos are on github and the resulting repo should be on github too?
Is there a way to force git to just download files from a branch and save them in a directory, without creating a local repo? For .bashrc it'd be faster to just download and save to ~, instead of cloning and then copying that file to home dir.
Maybe something like dropbox would be better for this?
I am trying to make make a task list for one of my projects. The project is in a git repository and I work on it from various different computers. What I want is to store all my tasks in my git repo, so that wherever I use Eclipse, I can see my tasks.
The Mylyn task list doesn't seem to offer a seamless solution to read and write my tasks in my git repo. Have you got any suggestions?
You can change the path where Mylyn saves its files to something inside your workspace. Then you should be able to manage the files with git just like everything else. I don't know about the multi user support of this approach though.
git can track only files so you need to import/export your tasks and store it in file in your repo.
I've got a project in a git repository that uses some custom (and so far unversioned) setup scripts for the build environment etc. I'd like to put these under version control (hopefully git) but keep them versioned separate from the project itself, while still living in the base directory of the project - I've considered options like local branches but these seem to have the problem that switch back to master (or any other "real" branch) will throw away the working copies of the setup scripts.
I'm on Windows using msysgit so I've got a few tools to play with; does anyone have a recommendation or solution?
If you really need them separate from your main git repo while still living directly within it, you could try:
creating a new repo with those script within it
and:
adding that new repo as a submodule to your repo. Except:
a/ those scripts won't live directly in the base directory, but in a subfolder representing the submodule
b/ you need of course to not publish (push) that new repo, in order for other cloning your main repo to not get those setup files
or:
merging that new repo into your main repo (with the subtree project), but:
you need to split back your project to get rid of those files
for a project with a large history, and with frequent push, that step (the split) can be long and cumbersome.
I would consider a simpler solution, involving some evolution to your current setup files:
a private repo (as in "not pushed") with those setup files
environment variables with the path of your main git repo in order for your setup files (which would not be directly within the base directory of said main repo) to do their job in the right directory (like beginning for instance with a 'cd right_main_git_repo_dir').
I want to share an additional solution and some samples from which to start.
I've has a similar problem in attempting to build Mozilla Firefox with Buildbot -- I need to have some files in the root folder (namely the .mozconfig file and some helper scripts) and I wanted to version them separately.
My solution is as follow:
checkout the Firefox code from the Mercurial repository;
checkout an additional repository with the additional file I need;
before starting the build, I copy these file to the folder with the Firefox code.
This approach is implemented in the following repositories:
buildconfig-mozilla-central: it contains the Buildbot configuration, which
pulls both repositories
copies the files from the scripts repository
and start the build;
buildscripts-mozilla-central: the repository with the build configuration and helper scripts.
Please note that the code might not be well factored (for example the paths) but it should be a good starting point.
This procedure is tailored for Firefox, but it can be applied to any repository.