I create websites for people. I have given them the ability to edit certain areas of their published pages using CushyCMS. That works fine, and everyone is happy with it.
When I go to publish some of my more extensive changes, I first need to pull down the latest version that they have produced. Then I make my changes, and upload everything to production.
I would like to use some sort of version control in this process. This should be a classic update-edit-commit-publish workflow, but I'm not sure how to go about this. Basically I want to avoid pulling down everything locally and doing the commits. I only want to pull down what has changed.
I use filezilla, and it doesn't do a good job of identifying changed files. I can't rely on the filesize, because sometimes it stays the same. I can't rely on timestamps because the server time is different than my machine, and it never seems to work correctly.
How can I get around my problem? I use Notepad++, Subversion and FileZilla, but I'm willing to try other tools if they would make this process easier.
It comes down to CushyCMS's decision to edit files directly and not put the user-provided content in a database like WordPress, DotNetDuke, Drupal, etc. So the real answer is you can't get there from here and should look into migrating to a database backed CMS. Thats not what you want to hear though.
Version control will get you part of the way to concurrency but there is always the possibility of a user updating a page between your pull down and publishing the revised copy since your users wouldn't be checking into the version control system directly. That would require them to learn the version control system and negate the ease that CushyCMS (or any CMS really) provides. You'll want to try and find a system that allows your live site to be the Master to which you compare and check-out files from. I do not know of any mainstream systems that currently work that way.
I found that it was easiest to use a tool like Beyond Compare to handle the synchronization.
Related
I am working on a website of 3,000+ pages that is updated on a daily basis. It's already built on an open source CMS. However, we cannot simply continue to apply hot fixes on a regular basis. We need to replace the entire system and I anticipate the need to replace the entire system on a 1-2 year basis. We don't have the staff to work on a replacement system while the other is being worked on, as it results in duplicate effort. We also cannot have a "code freeze" while we work on the new site.
So, this amounts to changing the tire while driving. Or fixing the wings while flying. Or all sorts of analogies.
This brings me to a concept called "continuous migration." I read this article here: https://www.acquia.com/blog/dont-wait-migrate-drupal-continuous-migration
The writer's suggestion is to use a CDN like Fastly. The idea is that a CDN allows you to switch between a legacy system and a new system on a URL basis. This idea, in theory, sounds like a great idea that would work. This article claims that you can do this with Varnish but Fastly makes the job easier. I don't work much with Varnish, so I can't really verify its claims.
I also don't know if this is a good idea or if there are better alternatives. I looked at Fastly's pricing scheme, and I simply cannot translate what it means to a specific price point. I don't understand these cryptic cloud-service pricing plans, they don't make sense to me. I don't know what kind of bandwidth the website uses. Another agency manages the website's servers.
Can someone help me understand whether or not using an online CDN would be better over using something like Varnish? Is there free or cheaper solutions? Can someone tell me what this amounts to, approximately, on a monthly or annual basis? Any other, better ways to roll out a new website on a phased basis for a large website?
Thanks!
I think I do not have the exact answers to your question but may be my answer helps a little bit.
I don't think that the CDN gives you an advantage. It is that you have more than one system.
Changes to the code
In professional environments I'm used to have three different CMS installations. The fist is the development system, usually on my PC. That system is used to develop the extensions, fix bugs and so on supported by unit-tests. The code is committed to a revision control system (like SVN, CVS or Git). A continuous integration system checks the commits to the RCS. When feature is implemented (or some bugs are fixed) a named tag will be created. Then this tagged version is installed on a test-system where developers, customers and users can test the implementation. After a successful test exactly this tagged version will be installed on the production system.
A first sight this looks time consuming. But it isn't because most of the steps can be automated. And the biggest advantage is that the customer can test the change on a test system. And it is very unlikely that an error occurs only on your production system. (A precondition is that your systems are build on a similar/equal environment. )
Changes to the content
If your code changes the way your content is processed it is an advantage when your
CMS has strong workflow support. Than you can easily add a step to your workflow
which desides if the content is old and has to be migrated for the current document.
This way you have a continuous migration of the content.
HTH
Varnish is a cache rather than a CDN. It intercepts page requests and delivers a cached version if one exists.
A CDN will serve up contents (images, JS, other resources etc) from an off-server location, typically in the cloud.
The cloud-based solutions pricing is often very cryptic as it's quite complicated technology.
I would be careful with continuous migration. I've done both methods in the past (continuous and full migrations) and I have to say, continuous is a pain. It means double the admin time for everything, and assumes your requirements are the same at all points in time.
Unfortunately, I would say you're better with a proper rebuilt on a 1-2 year basis than a continuous migration, but obviously you know best about that.
I would suggest you maybe also consider a hybrid approach? Build yourself an export tool to keep all of your content in a transferrable state like CSV/XML/JSON so you can just import into a new system when ready. This means you can incorporate new build requests when you need them in a new system (what's the point in a new system if it does exactly the same as the old one) and you get to keep all your content. Plus you don't need to build and maintain two CMS' all the time.
My company has 3 developers. Me, another guy, and a VP dev. I really want to implement source control, especially since our code seems to randomly change on it's own. We tend to develop on the server, live, etc.
I'm fine with having a copy of our database on my machine to work against, if necessary, as is the other guy. The VP dev doesn't want it. How can I work with him to change his mind, or make it work for him?
You have to make him think it's his idea.
Point out that with source control you not only have a built-in backup of everything, but you also have the previous versions - let him realize how much of a good thing that is.
Install SVN and tell the one that opposes it that "everybody does it" :)
And seriously - source control is a MUST even for a single developer, let alone for three.
As for the DB server - you can use one development server (it can be a regular machine). It is of course no problem if you use each a local copy, but you must have a strong database schema generation/synchronization tools.
You should have source control. There isn't much excuse for not having it. Source control will protect you against changes that will cause problems in your code. I would recommend putting the db schema and data (sample set) in your version control. This will allow independent changes to the db without screwing up what your users see live on the site.
Note that you're not really asking about source control here, but about where your development dataset resides. Local databases per-developer are best, if possible, but failing that, a reasonable alternative is to just have a virtual machine containing your source control server and a development database.
Putting things under source control is really easy - literally, 10 minutes from now you could have your source under source control. Rather than try and persuade him the benefits I would just go ahead and do it anyway.
Start simply by putting a copy of your source under source control - even if he doesn't use it just merge the changes from live into your source control repository on a regular basis. At least that way you have a revision history (and if you are him are the only people changing the source, it means that any changes you didnt make, he must have made)
With luck, slowly over time he will begin to see the benefits (him: oh no - everything just broke! you: Don't worry, I'll just look and see what has changed since the last working copy...)
It sounds like you need to convince him that it is
Necessary to solve a problem,
an appropriate solution (does exactly what you need) and
easy to use.
It sounds like you have the information to demonstrate #1: the last time the code, "changed on its own," on the server and you lost someone's work or mixed results poorly. Bam, there's your "problem." #3 is the next more difficult: you need to pick an SCM with a good set of tools and do a demo. The TortoiseX line of products (TortoiseHg, TortoiseSVN) are great for this, because they make it non-scary.
Item 2 is the hardest: to demonstrate that this is the appropriate solution. Perhaps, to convince him of this, you might refer to anecdotes of other programmers or by looking at Github, where you can look back at previous versions of a product. I'm clutching at straws, here, because I feel like his argument will be, "Ach, and that's when it's a huge headache, is when things break. It won't be worth it."
Obviously there are a large number of ways to deal with people (and for the most part you have a "people") problem.
The first thing I'd do is find out why he's so against source control. Often times people who don't like source control either don't like:
The extra work of committing
Don't always work next to an internet
See no extra value in it
There are different solutions to each of these problems. Obviously the third one is tricky, so I'll handle it last.
If they don't like the extra work of committing, some chron scripts will help them (or windows scheduler). Something that regularly commits in the background, or recursively goes through his files and adds them for the next commit. This will mean you'll do a little more work on your end to clean up extra files and deal with broken builds, but its a step. Alternatively if he's emailing you the code, a script that commits the emails works as well.
If he's not always working next to internet access, consider a system like GIT. The advantage of GIT (over something like SVN) is that it utilizes a pull model instead of a push. As a result you pull updates from other GIT users instead of pushing commits. If you are working on a plane and don't have internet access, this is a valuable feature.
Finally, demonstrating the importance of the system is tough. The best example is almost always: "My machine burned down." I suppose you could nuke his box, but for the moment let's look at ways that don't piss off your boss.
A good way to demonstrate the importance of a repository is a Daily Build. Having a daily build means you can readily integrate features and find bugs faster. Setting up a repository with a daily build will significantly improve your work conditions, and its likely to make a good impression.
These are just a few of the reasons that people don't like source control, but the key idea is finding what his reason is and adapting to it.
We have this huge application that has 18 projects in our source control (VSS).
Whenever we are working on small changes everything is fine because each developer has a set of a few files checked out to himself, and hopefully no one is going to need them until they are checked in (in about 4 to 8 hours).
But when we want to work on big changes a developer keeps so many files checked out for some days and make it hard for others to do their assigned tasks.
Here's a scenario for example:
Last week we wanted to implement a feature that will fetch every list in our application using a paging mechanism. Therefore we should change the UI, business and data access layers.
There's a developer assigned to this task, she's checked out a lot of files, and she's blocking other tasks.
How should we plan to develop these kind of features?
Switch to a better Version Control System. VSS suffers from design issues and its hard lock principles. Subversion is available for free and can be used for large scale application development. Branching and tagging are cheap operations and there is no hard lock.
Your company will definitively live better with Subversion. Try it out!
Server Easy to setup on what system ever (Windows/Linux)
TortoiseSVN Client for Windows that integrates within Windows Explorer
SVN Manual Read at least the first few chapters
There are many other alternatives but VSS is a pain in large scale development. As there is a better free solution available, why stick to a vendor?
If upgrading to a real VCS isn't an option right now, have the person doing the major feature download a local copy of everything and then make his changes outside of version control. Merge it all at once (over a weekend or something in case it gets complicated).
This doesn't help the developer who will need version control while making the "big changes"
Well, you do what you gotta do with the tools you've got. He could always install a modern VCS like git which works locally. Just check the entire baseline into git (minus history) and go.
VSS sucks, migrate to a real SCM, microsoft will probably help you seamlessly upgrade to TFS which doesn't have this problem. Or migrate to any one of FOSS SCMs like subversion, but the transition will probably be harder (but may be cheaper).
Have you considered sharing and branching? Also, you can allow multiple checkouts with users who have experience. In your case of making a large application change, I suggest labeling then creating a branch. If something happens to "big changes", you will not the production version. You can make your quick fixes in the released code then merge them into "big changes" once it is ready. Check the help topic "Sharing and Branching".
Use a version control system that works in merge mode (optimistic), not in lock mode.
Merge mode is optimistic is that it assumes changes will usually not be made in the same place. If it happens in the same place then it is usally easy to resolve.
An example of a version control system that can work in merge mode is CVS. It is outdated now, but others exist.
SVN is the answer to your problems. I have used it and its a breeze to learn/work with it. But there are a few new kids on the block. Try GIT. I have been hearing a lot about it althought havent had a chance to try it
VSS is just an old story, we use Subversion (server) and TortoiseSVN (client) now. (That's just based on our preference)
By the way, migrating to other version control / source control - only - will not solve your team issue. The problem is about communication. If she can't communicate with the others and stay with her habit (working with a lot of files without checking them in), she will put your team down, you must let her know how to work with team using version control. If not, she will put you into "Merge" problems when using Subversion. ^^
You already got the advice about changing to a usable VCS.
Above that, you and the developers should train to break the big changes into smaller ones. I'd consider about 10 commits per day and full time developer a normal rate. It makes the locks much less painful.
The principle should be: make the smallest possible change that brings you toward your desired state and works (as in the software compiles and passes all tests).
In the case you sketched out. Adding a parameter to one layer, and changing all calls to that layer (possibly with a dummy value) would be one change. Actually using that value, would be another change.
This should result in much less files locked for a much shorter period.
Long-term, you'll want to migrate to another VCS. As others have mentioned, consider either open source or TFS if you want to stick with MS. (We use TFS, but I'm not going to sing its praises - it's OK.) As AMissico mentioned, branching will help with any VCS that supports it. Learning to use branching effectively is not trivial, and will require study and/or training.
Continuous Integration will also help. TeamCity is what we use, and it's relatively simple to set up. See FeatureBranch.
I've been tasked with setting up a version control for our web developers. The software, which was chosen for me because we already have other non-web developers using it, is Serena PVCS.
I'm having a hard time trying to decide how to set it up so I'm going to describe how development happens in our system, and hopefully it will generate some discussion on how best to do it.
We have 3 servers, Development, UAT/Staging, and Production. The web developers only have access to write and test their code on the Development server. Once they write the code, they must go through a certification process to get the code moved to UAT/Staging, then after the code is tested thoroughly there, it gets moved to Production.
It seems like making the Developers use version control for their code on Development which they are constantly changing and testing would be an annoyance. Normally only one developer works on a module at a time so there isn't much, if any, risk of over-writing other people's work.
My thought was to have them only use version control when they are ready to go to UAT/Staging. This allows them to develop and test without constantly checking in their code.
The certification group could then use the version control to help see what changes had been made to the module and to make sure they were always getting the latest revision from the developer to put up on UAT/Staging (now we rely on the developer zip'ing up their changed files and uploading them via a web request system).
This would take care of the file side of development, but leaves the whole database side out of version control. That's something else that I need to consider...
Any thoughts or ideas would be greatly appreciated. Thanks.
I would not treat source control as annoyance. See Nicks answer for the reasons.
If I were You, I would not decide this on my own, because it is not a
matter of setting up a version control software on some server but
a matter of changing and improving development procedures.
In Your case, it might be worth explaining and discussing release branches
with Your developers and with quality assurance.
This means that Your developers decide which feature to include into a release
and while the staging crew is busy on testing the "staging" branch of the source,
Your developers can already work on the next release without interfering with the staging team.
You can also think about feature branches, which means that there is a new branch for every specific new feature of the web site. Those branches are merged back, if the feature is implemented.
But again: Make sure, that Your teams agreed to the new development process. Otherwise, You waste Your time by setting up a version control system.
The process should at least include:
When to commit.
When to branch/merge.
What/When to tag.
The overall work flow.
I have used Serena, and it is indeed an annoyance. In addition to the unpleasantness of the workflow overhead Serena puts on top of the check in-check out process, it is a real pain with regard to doing anything besides the simplest of tasks.
In Serena ChangeMan, all code on local machines is managed through a central server. This is a really bad design. This means a lot of day-to-day branch maintenance work that would ordinarily be done by developers has to go through whomever has administrator privileges, making that person 1) a bottleneck and 2) embittered because they have a soul-sucking job.
The centralized management also strictly limits what developers are able to do with the code on their own machine. For example, if you want to create a second copy of the code locally on your box, just to do a quick test or whatever, you have to get the administrator to set up a second repository on your box. When you limit developers like this, you limit the productivity and creativity of your team.
Also, the tools are bad and the user interface is horrendous. And you will never be able to find developers who are already trained to use it, because its too obscure.
So, if another team says you have to use Serena, push back. That product is terrible.
Using source control isn't any annoyance, it's a tool. Having the benefits of branching and tagging is invaluable when working with new APIs and libraries.
And just a side note, a couple of months back one of the dev's machine's failed and lost all his newest source, we asked when the last time he committed code to the source control and it was 2 months. Sometimes just having it to back up stuff when you reach milestones is nice.
I usually commit to source control a couple of times a week, depending if I've hit a good stopping point and I'm about to move on to something different or bigger.
Following on from the last two good points I would also ask your other non-web developers what developmet process they are using so you won't have to create a new one. They would also have encountered many of he problems that occur in your environment, both technical using the same OS and setup and managerial.
Is there any form of version control for Linden Scripting Language?
I can't see it being worth putting all the effort into programming something in Second Life if when a database goes down over there I lose all of my hard work.
Unfortunately there is no source control in-world. I would agree with giggy. I am currently moving my projects over to a Subversion (SVN) system to get them under control. Really should have done this a while ago.
There are many free & paid SVN services available on the net.
Just two free examples:
http://www.sourceforge.net
http://code.google.com
You also have the option to set one up locally so you have more control over it.
Do a search on here for 'subversion' or 'svn' to learn more about how to set one up.
[edit 5/18/09]
You added in a comment you want to backup entire objects. There are various programs to do that. One I came across in a quick Google search was: Second Inventory
I cannot recommend this or any other program as I have not used them. But that should give you a start.
[/edit]
-cb
You can use Meerkat viewer to backupt complete objects. or use some of the test programas of libopenmetaverse to backup in a text environment. I think you can backup scripts from the inventory with them.
Jon Brouchoud, an architect working in SL, developed an in-world collaborative versioning system called Wikitree. It's a visual SVN without the delta-differencing that occurs in typical source code control systems. He announced that it was being open sourced in http://archvirtual.com/2009/10/28/wiki-tree-goes-open-source/#.VQRqDeEyhzM
Check out the video in the blog post to see how it's used.
Can you save it to a file? If so then you can use just about anything, SVN, Git, VSS...
There is no good source control in game. I keep meticulous version information on the names of my scripts and I have a pile of old versions of things in folders.
I keep my source out of game for the most part and use SVN. LSLEditor is a decent app for working with the scripts and if you create a solution with objects, it can emulate alot of the in game environment. (Giving Objects, reading notecards etc.) link text
I personally keep any code snippets that I feel are worth keeping around on github.com (http://github.com/cylence/slscripts).
Git is a very good source code manager for LSL since its commits work line-by-line, unlike other SCM's such as Subversion or CVS. The reason this is so crucial is due to the fact that most Second Life scripts live in ONE FILE (since they can't call each other... grrr). So having the comparison done on the file level is not nearly as effective. Comparing line by line is perfect for LSL. With that said, it also (alike SourceForge and Google Code) allows you to make your code publicly viewable (if you so choose) and available for download in a compressed file for easier distribution.
Late reply, I know, but some things have changed in SecondLife, and some things, well, have not. Since the Third Party Viewer policy still keeps a hard wall up against saving and loading objects between viewer and system, I was thinking about another possibility so far completely overlooked: Bots!
Scripted agents, AKA Bots, have all usual avatar actions available to them. Although I have never seen one used as an object repository, there is no reason you couldn't create one. Logged in as a separate account the agent can be wherever you want automatically or by command, then collect any or all objects you are working on at set intervals or by command, and anything they have collected may be given to you or collaborators.
I won't say it's easy to script an agent, and couldn't even speak for making an extension to a scripted agent myself, but if you don't want to start from scratch there is an extensive open source framework to build on, Corrade. Other bot services don't seem to list 'object repository' among their abilities either but any that support CasperVend must already provide the ability to receive items on request.
Of course the lo-fi route, just regularly taking a copy and sending the objects to a backup avatar, may still be a simple backup solution for one user. Although that does necessitate logging in as the other account either in parallel or once every 20 or-so items to be sure they are being received and not capped by the server. This process cannot rename the items or sort them automatically like a bot may. Identically named items are listed in inventory as most recent at the top but this is a mess when working with multiples of various items.
Finally, there is a Coalesce feature for managing several items as one in inventory. This is currently not supported for sending or receiving objects, but in the absence of a bot, can make it easier to keep track of projects you don't wish to actually link as one item. (Caveat; don't rezz 'no-copy' coalesced items near 'no-build' land parcels, any that cannot be rezzed are completely lost)