I would like some guidance when working projects with Moovweb. I'd like to understand what are some of the best practices at managing my Moovweb projects on Github. Specifically how to move them and maintain them from development to staging to production.
Really appreciate the information.
J
I'd check out some of the advice here that walks you through how to setup a professional development environment:
https://console.moovweb.com/learn/training/building_a_site/project_setup
The main idea is to setup your project so that it has 3 git remotes. A git remote is simply a URL where your code is being hosted.
origin: This remote will be where you and your developers host your code base. You’ll coordinate changes to your code here and manage different branches and bug fixes during development.
stage: This remote is where you’ll deploy your code base for testing. This will be a Moovweb Cloud URL that actually builds your code live so you can QA it before you put it on your production site.
production: This remote is where you’ll deploy your final production site that has been tested and approved for public use.
Then the release process would be as follows:
Test your changes locally in the browser
Push those changes to staging to QA them and make sure they work properly in a production environment.
Push those changes to Github so that other developers working on the project will have the latest code.
Push to production.
Related
I am making a small web project which I am doing on the LAMP stack using the Symfony 1.4 MVC framework. I am interested in documenting everything, right from the database to the code and I want to push all my code onto GitHub. I know how to setup a repository on GitHub and push changes etc live.
What I am unable to wrap around my head is this:
I have the development server, which after testing it out I push it onto my GitHub repository, after this I want to naturally have to push it onto my production server. Typically, this being a one man project, I can easily push the project live to my production server using rsync. And this is how I would typically configure it in my YAML files and then symfony will automatically push the changes to my production server.
Naturally, I want GitHub as the intermediary where the code to my project is saved and from there I should be able to deploy it to my production server.
But how exactly I should achieve this, I have no clue.
Can anybody suggest the missing pieces to this puzzle?
You can use Capifony to deploy your symfony project.
I've never worked with any version control systems before. Now I'm trying to learn Mercurial, but I'm confused (I've already read about 10-15 articles + hginit.com). I don't know how to organize the workflow.
I have a testing server and a production server. I work from my office computer and from my home laptop. I make changes directly on the testing server, and every week or so copy new code to my production server. I also need wiki/issues/etc. pretty much everything bitbucket.org has. I know that's a bad way of doing things.
Is there any tutorial or articles on how to organize the workflow? I'd also appreciate any schemes/sketches describing the process.
Thank you!
[Edit: Changed based on comments]
Using Bitbucket
Once you have created an account.
You should be able to create a repo with an appropriate url. Then you can clone it to create a local repository.
Check out getting started.
See the following to push the updates to BitBucket.
BitBucket comes with very extensive documentation.
Also there are, other useful tools to work with BitBucket:
BitbucketExtension that allows you to use command line for a number of operations.
Using Mercurial Queues and bitbucket.org
Organizing workflow
You will have to evolve a workflow that suits you. In your case, it looks like you have a testing server and production server.
So , you can setup two repositories, one for the testing server ad one for production. You can make push to testing server automatic so that you can test out the changes immediately. You can tag releases that are then pushed to production server.
Your local repo can be used to publish changes to testing server.
You can push the approved changes, tagged to BitBucket repository.
I am using mercurial for website development. I "think" I'm using it correctly.
I develop on my development machine, commit fairly regularly. I will somewhat regularly push my commits to my hosted site-dev repository.
If things are set up how I want them for the live site, I push from my dev machine to the hosted site-live repository. Then I pull down from that repository onto the live server.
However, there are some changes that need to be made (changing directories from localhost to www.example.com, changing the DB connection stuff, etc.).
What I did was made these changes on my live machine, then pushed them back up to the site-live repository. I don't know why I did that, really, but at least there's a changeset sitting there with the necessary config changes.
What I don't know how to do is manage this process. I'm a little lost beyond committing, pushing and pulling with hg. I'm a single developer and haven't even done a merge yet.
Is there some way to keep that particular changeset identified, and just apply it, hopefully even BEFORE I pull from the repo down to the live server?
I think you can tell from my question that I'm in a little over my head with hg and workflow at the moment ;)
This is my understanding:
What essentially you are trying to do is have a development, staging and deployment environment. You do your development using 'development' repository, test it on a staging environment and then once satisfied, pull those changes into deployment repository.
And when you pull from staging to deployment, you need to change your environment / configuration data.
My take is you should not be changing the configuration at all.
You should have configuration files such that you have a
basic configuration file
basic.conf
Environment specific overrides
basic.dev.conf, basic.staging.conf and basic. deployment.conf
Use environment variable:
The overrides to the basic configuration data should be defined via an environment
specific variable : APP_ENV : dev or staging or deploy
This way you should be able to override the configuration based on the environment without changing the configuration information.
It is not a good idea to rely on making changes to config files each time you pull your code from development to staging to deployment.
I would keep the live server outside the version control. Meaning that I would have a small "install" script that pulls updates from the repository, removes any unnecessary development files, and applies the correct configuration files. Both development and production configuration files should be in version control.
I have finished developing the core of a web application I have been working on. Since I was the only developer I just developed locally (lamp stack) without using version control (probably stupid but anyway..). Now that it is getting close to production ready, I have a couple other developers working with me so I set up a repository for my code.
This is my question: I still want to be able to test any changes locally first before posting to production. How do I manage this with a repository without having to maintain 2 versions of my code (that I have to synch up manually)? For one, the production code has a few differences here in there (such as database constants etc.). I'd like to be able to change my code in my local repository, test it on my local apache server, then check the code directly into production (is this even possible using eclipse)?
I am using eclipse and subversion (php code). I know I asked many questions but hopefully you get the idea of what I am trying to do...and I assume its rather common. Thanks.
In addition to the excellent answers you've gotten already, I'd like to emphasize that if there are differences between your dev and production code, you're adding risk. You should be using the same, well-tested code in both locations; any difference between the environments should be expressed in configuration files. Any configuration files in source control should be samples only; your deployment script should not push new configuration files to production.
This, in combination with tagged releases and a staging environment that mimics production, should help you promote your code smoothly to the production environment.
I would suggest a few things
Use tags/branches in SVN. When the code is production ready, tag it with a unique name.
Set up a staging area for integration testing. After a release is tagged for staging, yank it from your vcs and copy it into the staging area. This can be as simple as a different directory tree or a second install of your server.
Put constants into separate files that can copied/merged over into the staging and deployment directories
Test the staged version against dev to insure everything works as it did in your dev environment. I would point staging to production databases when I am sure it is working and ready to be promoted. Test that it also works against prod.
Once everything works in staging, update the production copy. I would suggest you create a clean deployment directory then copy that entire deployment over to the production server after copying/merging config settings.
This was my approach is dealing with perl/cgi many years ago and it worked pretty well. SVN handles tags/branching much better so it should be easier to deal with. We had very few production problems once we started staging the files before pushing to prod.
It sounds like you haven't created any branches or tags, and probably have a "trunk" that isn't labeled as such. Best practices would dictate that you have a trunk for the current stable code, branches that you develop against, and tags that are actually used on the production site. There is a short description and diagram on Wikipedia.
Of course, that's just best practice. Your project sounds small enough that you could get away with splitting your code into a development/ directory and a production/ directory in your code repository. Checkin code to the development directory, and once a change is fully tested, merge it into the production directory.
Whether you do it the right way or the easy way, it's important to do something to separate your development code from your production code. As you add more developers, it will be increasingly unlikely that the development code base is stable because people are checking in code that hasn't been fully tested, isn't complete, whatever. Spending a little extra time on managing two branches of code will save you a lot of headaches later on.
Describe the process you use to develop web applications at a not-so-high level, focusing on VC, bug tracking, QA, unit testing, deployment and anything else similar (minus the planning/client communication side of things).
I'm new in this area, so my rough example (read: haven't used this process) is no doubt abit off, so to speak - point out it's flaws so I can learn.
Eg.
Create project repository on local SVN server.
Create batch/shell scripts for DNS mappings.
Check out project, begin work on local working copy.
Develop features as branches.
Track bugs with Mantis (link commits to bugs through it's SVN integration (no idea if that exists)).
Document as you go.
Do QA on branch.
Merge to trunk when stable.
Unit Testing?
Commit to repository when feature is implemented and stable.
Copy releases to tags in repository. Eg. /project/tags/rel-123/
Use Phing to upload to staging server. (Could someone please clarify exactly what a staging server is used for beyond 'testing'?)
Use Phing to prep live site for update, set up DB/deploy, etc.
Create/checkout HEAD version ("main branch")
Develop code and sync with the main branch -at least- daily
After development is done, write and run unit tests
Go through code review and submit code/changes to the main branch
Let continuous builder run all unit tests and system/integration tests on main branch
When ready, cherry pick revisions and integrate them to the QA branch
Run system and integration tests, fix reported bugs, or rollback as necessary; this repeats steps 4-7
After QA signoff, integrate QA change to release branch
Run unit tests, system/integration tests on release branch
Deploy to production and run sanity tests.
A staging server is a copy of your production environment that is as up-to-date as possible. On my current project, we're able to keep each release independent from each other, so our "staging server" is our production server, just accessed from a different url.
Notes and discreprencies:
All of the steps have some variation depending on the size of your project. The larger your project, the better the benefit from cherry picking and environment separation. In smaller projects, these can just be time sinks and are often ignored or bypassed.
To clarify, there is a Development stack, QA stack, and Staging stack. Depending on your project size, QA might be staging, Production might be staging, or some combination thereof. The separation of the Dev and QA stacks is the most important one.
In the steps above, I'm assuming both code and relevant data is versioned or tracked. Having a release and build process that takes control data into account makes a release very, very easy.
In a small-medium sized project, there may or may not be a release branch; it depends on the frequency of code change. Also, depending on the frequency of code change and size of your project, you may integrate the full QA branch to the Release branch, or cherry pick specific revisions to integrate to the release branch.
FWIW, I've found "migration scripts" to be of little value. They're always a one-off script with little reuse and make rollbacks a pain in the ass. Its much easier, I would argue better, to have the application backwards-compatible. After a few releases (when a rollback is a laughable), data cleanup should be done, if necessary.
Very roughly:
Create repository in SVN
Checking local working copy to developer environment
Update/commit changes frequently
Deploy to stage from SVN trunk using custom deploy script
QA tests on stage, reports bugs in Mantis
Developers fix bugs, mark as resolved
Re-deploy to stage
QA tests bugs, closes if fixed
QA is finished, do regression testing
Deploy to production using custom deploy script
Do a little dance
We also create branches for future versions or features. These eventually get merged into the trunk.
We keep our db structures synchronized with a custom db comparison tool that is executed during the deploys.
Old post, but interesting question !
At my company now :
Create a new Github repo
Configure Jenkins
Clone locally
Start a branch
Develop and add tests (server, client and e2e)
Commit for every step, and fetch + rebase to keep the branch in sync
When ready, push the branch to the server : a pre-commit check lint and tests and block if not ok
Create a pull request for the branch
Here, jenkins automatically runs tests on the branch and flag it as "green" or "broken tests" directly in the pull request
Have at last 2 colleagues review the pull request and fix their findings (back to step 5)
When everything green and 2 colleagues have agreed, the last one merges the pull request
Delete the branch on server
When ready, push a new version
Latest version get immediately deployed on a testing platform
QA validate the corrections and features introduced (back to 5 if problem)
(TODO) deploy to a pre-prod with identical parameters than the production
Deploy to production
Go apologize to users for the bugs introduced ;) and report them in the issue manager
get feature requests and report them in the issue manager
restart cycle at step 2