We use gradle as our build tool and use the idea plugin to be able to generate the project/module files. The process for a new developer on the project would look like this:
pull from source control.
run 'gradle idea'.
open idea and be able to develop without any further setup.
This all works nicely, but generally only gets exercised when a new developer joins or someone gets a new machine. I would really like to automate the testing of this more frequently in the same way we automate our unit/integration tests as part of our continuous integration process.
Does anyone know if this is possible and if there is any libraries for doing this kind of thing?
You can also substitue idea for eclipse as we have a similar process for those that prefer using eclipse.
The second step (with or without step one) is easy to smoke test (just execute the task as part of a CI build), the third one less so. However, if you are following best practices and regenerate IDEA files rather than committing them to source control, developers will likely perform both steps more or less regularly (e.g. every time a dependency changes).
As Peter noted, the real challenge is step #3. The first 2 ones are solved by your SCM plugin and gradle task. You could try automating the last task by doing something like this
identify the proper command line option, on your platform, that opens a specified intellij project from the command line
find a simple good enough scenario that could validate that the generated project is working as it should. E.g. make a clean then build. Make sure you can reproduce these steps using keyboard shortcuts only. Validation could be made by validating either produced artifacts or test result reports, etc
use an external library, like Robot, to program the starting of intellij and the running of your keyboards. Here's a simple example with Robot. Use a dynamic language with inbuilt console instead of pure Java for that, it will speed your scripting a lot...
Another idea would be to include a daemon plugin in intellij to pass back the commands from external CLI. Otherwise take contact with the intellij team, they may have something to ease your work here.
Notes:
beware of false negatives: any failure could be caused by external issues, like project instability. Try to make sure you only build from a validated working project...
beware of false positives: any assumption / unchecked result code could hide issues. Make sure you clean properly the workspace, installation, to have a repeatable state and standard scenario matching first use.
Final thoughts: while interesting from a theoretical angle, this automation exercise may not bring all the required results, i.e. the validation of the platform. Still it's an interesting learning experience and could serve as a material for a nice short talk, especially if you find out interesting stuff. Make it a beer challenger with your team when you have a few idle hours to try to see who can implement the fastest a working solution ;) Good luck!
Related
In a Java project I'm working (alongside a team of 8 devs), we have a large backlog of features that lack automated tests. We are covering this backlog and we need to keep track of who wrote a JUnit test and when, plus we have to measure how many test we wrote as a team in a week/month/semester (as you may have figured out already, this information is for management purposes). We figured we'd do this by marking the tests with the information we need (author, creation date) and let Eclipse do the processing work, showing us tests we wrote, who wrote'em and how far we were from reaching our goals. How would you smart people go about this? What plugins would you use?
I tried to use Eclipse Custom Tags for this, but it's not the purpose of the feature, and the results I got were kind of brittle. I created a TEST tag that was supposed to mark a test method. It looks like this: (date is mm-dd-yyyy)
//TEST my.name 08-06-2011
Since Eclipse processes tag description by substringing (contains/doesn't contain), it's, as I said, very brittle. I can timestamp the tag, but it's just a string. Eclipse can't process it as a date, compare dates, filter by date interval, stuff like that.
I searched for plugins, but no dice.
My B-plan is to create an Annotation with the information we need and process our test packages using Eclipse Annotation Processing Tool. I haven't tried anything on this front yet, though, just an idea. Anyone knows a good plugin for this kind of Annotation processing? Or any starter tips for dealing with Eclipse APT.
Thanks a bunch, folks
I would not use Eclipse for this.
Your team should be checking the tests into a version control system such as Subversion, Git, Team Foundation Server, etc. From there it should a fairly straightforward matter to determine the owner and check-in time. You can and should do this sort of metrics calculation during every build. Better yet, be sure that your build script actually runs your tests and uses a tool like EMMA to instrument the code and determine the actual coverage.
As a fallback for measuring coverage, if you choose a naming convention then you may even be able to correlate the test classes by file name back to the feature under test.
Many modern build systems, such as CruiseControl, have integration for doing these sorts of things quite nicely.
I'm currently working on a project that's been going on for several years straight. The development-team is small (less than 5 programmers), and source-control is virtually non-existent, and the deployment-process as is is just based on manually moving files from one server to another. The project is in classic ASP, so building isn't an issue, as both deployment and testing is just about getting the files to where they need to be and directing the browser at the correct location. Currently all development is done on a network-drive which is also the test-server. The test-server is only available when inside the the local network (can be accessed trough vpn), and is available on the address 'site.test' in the browser (requires editing to the hosts-file on all the clients, but since there are so few of us that hasn't proven to be any problem at all). All development is done in visual studio. Whenever a file is change the developer that changed the file is required to write the file he changed into a word-document and include a small description as of what was changed and why. Then, whenever there's supposed to be a version-bump (deployment), our lead-developer goes trough the word-document and copies every file (file by file) that has changed over to the production-server. Now, I don't think I need to tell you that this method is very error prone (a developer might for instance forget to add that he changed some dependency, and that might cause problems when deployed), and there's a lot of work involved with deploying.
And here comes the main question. I've been asked by the lead developer to use some time and see if I can come up with a simple solution that can simplify and automate the "version-control" and the deployment. Now, the important thing is that it's as easy as posible to use for the developers. Two of the existing developers have worked with computers for a long time, and are pretty stuck up in their routines, so for instance changing it into something like git bash wouldn't work at all. Don't get me wrong, I love git, but the first time one of them got a merge-conflict they wouldn't know what to do at all. Also, it would be ideal to change to a more distributed development-process where the developers wouldn't need to be logged into vpn (or need internet at all) to develop, and the changes they made offline could be synced up when they were done with them. Now, I've looked at Teem Development Server from Microsoft because of it's strong integration with Visual Studio. As far as I've tested it seems possible to make Visual Studio prompt the user if they want to check in changes whenever the user closes Visual Studio. Now, using TFS for source-control would probably eliminate most of the problems with the development, but how about deployment? Not to mention versioning? As far as I've understood (I've only looked briefly at TFS), TFS has a running number for every check-in, but is it possible to tell TFS that this check-in should be version 2.0.1 of the system (for example), and then have it deploy it to the web-server? And another problem, the whole solution consists of about 10 directories with hundreds of files in, though the system itself (without images and such) is only 5 directories, and only these 5 should be deployed to the server, is this possible to automate?
I know there's a lot of questions here, but what is most important is that I want to automate the development process (not the coding, but the managing of the code), and the deployment process, and I want to make it as simple as possible to use. I don't care if the setup is a bit of work, cause I got enough time at hand to setup whatever system that fits our needs, but the other devs should not have to do a lot of setup. If all of the machines that should use the system needs to be setup once, that's no problem at all, cause I can do that, but there shouldn't bee any need to do config and setups as we go.
Now, do any of you have any suggestions to what systems to use/how to use them, in order to simplify the described processes above? I've worked with several types of scm-systems before (GIT, HG and SubVersion), but I don't have any experience with build-systems at all (if that is needed). Articles, and discussion on how to efficiently setup systems like this would be greatly appreciated. In advance, thanks.
This is pretty subjective territory, but I think you need to get some easy wins first. The developers who are "stuck up in there ways" are the main roadblock here. They are going to see change as disruptive and not worth it. You need to slowly and carefully go for the easy wins.
First, TFS is probably not going to be a good choice. It's expensive, heavy, and the source control in TFS is pretty lousy. Go for Subversion: it's easy to setup and easy to use, and it's free. Get that in place first, and get the devs using it. Much easier said than done.
Later (possibly much later), once the devs are using it and couldn't imagine life without a VCS, then you could switch to Hg or Git if you need first class branching and all those other nice features.
Once you have Subversion in place, you can use something like JetBrains TeamCity or Jenkins, both of which are free and easy to use. However, I'm just assuming you don't have a lot of tests and build scripts that the CI server is really going to be running, so it's far more important that you get VCS first. In all things: keep it as simple as possible. Baby steps. Get some wins, build trust, repeat.
I can't even begin to think where to begin with this! Intending no offense directed at you, apart from the mention of git and HG, this post could have been written 10 years ago.
1) Source control - How can a team of developers possibly work effectively without some form of source control? Hell, even if it's Visual Source Safe (* shudder *) at least it would be something. You have to insist that the team implement source control. You know what's available so I won't get into preaching about that. (However, Subversion with TortoiseSVN has worked quite well for me.)
2)
"write the file he changed into a
word-document and include a small
description as of what was changed and
why"
You have got to be kidding... What happens if two developers change the same file? Does the lead then have to manually merge two changes that s/he extracts from the word doc? Please see #1 and explain to them how commit comments work.
Since your don't really need to "build" (i.e. compiled, etc.) anything, you should be able to solve most of your problems with some simple tools. First and foremost you need to use a source control solution. Yes, the developers would have to learn how to use another tool (EEEK!). You could do the initial leg work of getting the code into the repository. If you have file access to the other developers machines, you could even copy a checked-out working copy to their machines so they wouldn't have to do the checkout themselves (not really that hard). You could then use all the creamy goodness of version control to create version branches when each deployment needs to be done. You could write simple scripts using the command line SVN tools to check out said branches and automatically copy the files to the target server(s). Using a tool like BeyondCompare, the copy process could be restricted to only the files that are different (plus BC can handle an FTP target if that is an issue). By enforcing commit comments on the SVN repo, you'll guarantee that the developers provide comments, and for each set of changes between releases you could very easily generate a list of all those comments using the CSM log retrieval features.
If I want to enable a new piece of functionality to a subset of known users first, is there any automated system of framework that exists to do this?
Perhaps not directly with version control - you might be interested to read how flickr goes about selectively deploying functionality: http://code.flickr.com/blog/page/2/
And this guy talks about implementing something similar in a rails app: http://www.alandelevie.com/2010/05/19/feature-flippers-with-rails/
Most programming languages have if statements.
I don't know what "switching between them at runtime" means. You usually don't check executable code into an SCM system. There's a separate process to check out, build, package, and deploy. That's the province of continuous integration and automated builds in agile techniques.
SCM systems like Subversion allow you to have tags and branches for parallel development. You're always free to build, package, and deploy those as you see fit.
As far as I know no...
If you wanted a revision control system that had multiple versions that you could switch between. Find a SCM you like and lookup branching.
But, it sounds like you want it to me able to switch versions in the SCM programmatically during runtime. The problem with that is, for a revision control system to be able to do that it would have to be aware of the language and how it's implemented.
It would have to know how load and run the next version. For example, if it was C code it would have to dynamically compile and run it on the fly. If it was PHP it would have to magically load the script in a sandbox http server that has PHP support. Etc... In which case, it isn't possible.
You can write an app to change the version in the scm by using the command line.
To do it during runtime, that functionality has to be part of the application itself.
The best (only) way I can think of doing it is to have one common piece of code that acts like a 'bootloader', which uses a system call to checkout the correct branch based on whatever your requirements are. It then (if necessary) compiles that code, and runs it.
It's not technically 'at runtime', but it appears that way if it works.
Your first other option is something that dynamically loads code, but that's very language-dependent, and you'd need to specify.
The other is to permanently have both in the working codebase (which doubles your size if it's a full duplication), and switch at runtime. You can save a good bit of space by using objects that are shared between both branches, and things like conditional compilation to use the same source files for both targets.
I was always wondering why it is a big deal having version control support inside an IDE.
I always preferred to use a command-line/standalone version of the version control of choice and never found IDE integration helpful.
I know it can be helpful sometimes, for example to automatically keep track of renames, but I was bitten by version control plugins a couple of times (especially the ClearCase Eclipse plugin) that I'm now finding it counter productive compared to the command-line version, where I have better control.
What is your opinion?
Integrated Source Control also helps to only keep the important files under Source control. For example, when I add a new File in Visual Studio, the Plugin (visualSVN) will allow me to add it easily without me having to remember to go outside of my IDE and run the command to add it to the repository. On the other hand, it will automatically ignore temporary files, like the obj/ and bin/ Folders.
Essentially: Integrated Version Control that actually works is a great way to keep the repository clean and complete.
I like how some IDE's implement this. Ankh-SVN for Visual Studio is not that great and is a bit buggy, however Subeclipse I find to work exceedingly well when I'm using Eclipse.
I think it really depends on the IDE you're using and the quality of that plug-in. It's going to work well for some setups and terrible for others.
That's why I like Subversion with Tortoise SVN so much. I can choose to use the IDE integration when and where it makes sense, otherwise, just like you said, I can simply use the command line or in my case, the windows explorer based client!
Integration of the IDE with version control and, in particular, software change management (SCM) helps bringing together the philosophies of the IDE and the source control system.
One example is temporary files and binaries, that should not be checked-in and, e.g. in Visual Studio, end up within the source directory if you're not carefully creating new project and solution templates with a non-default directory configuration.
Another could be tracking of work items and complex bug fixes.
Also it saves some ceremony and context-switching when editing files.
Advanced integrations may also allow to push the change management system's concept of "configuration" ("branch", "tag", "view") into the IDE.
ClearCase integration, however, is clearly not "advanced".
A lot of it is simply the preference and comfort level of the user. Some folks are comfortable with the command line. Some prefer a GUI.
I wouldn't make generalized assumptions that all version control within the IDE is bad or buggy based on experiences with a particular plugin which had issues.
Why even have an IDE? Why not just do everything with a command line? ;)
The answer is that having it integrated with the IDE is "better".
My #1 reason:
You can visually see if a file is checked out or not, and if you need to edit a file, you can take the action right there where you are working.
There are more, but that is the big one.
It's depend on your IDE and the way you work with VCS.
Me and my team using VSS plugin-ins inside Delphi IDE, it gives a lot of flexibable feature when working together for example, All our forms are check-in when you start to write a letter or move components it asked if you want to check-out the code file or form.
also when some one change any code in other forms, it pop up and telling you it's already update by someone else and asking you to update current files in your H.D.
and you just get everything while you are in the IDE, you don't need to move to other external file, or command prompt to do a simple task.
I find most people who like to deal with command prompt working mostly in code without GUI IDE or may I be wrong.
Nearly all of my subversion needs can be handled by the IDE interface. It's a lot faster to do 2 quick clicks than pop up a command line, cd to the right place, issue the command, etc.
Command line has it's place, but with the current crop of IDEs, that place continues to shrink.
I have battle scars from using a buggy implementation of an IDE/VCS integration. In all honesty, if it was not buggy it would have been great. As long as there are great tools like TortoiseSVN, I don't see a need for IDE/VCS integration. I'd rather have more tools that do their job well than a few buggy tools.
Version control support in an IDE generally gives you a better view. The IDE actually knows what type of file you are looking at when doing a diff, which means it can do context highlighting and help you do merges more effectively.
I also think it saves setup time. In stead of installing all kinds of tools, a developer can download the IDE, do a checkout an be on it's way. If every developer on a project uses the same IDE, they can help eachother.
"Counterproductive" is a large word. If you have serious CVS/SVN problems maybe once a month, it's still way to few to have complicated clients installed on all your dev machines.
I have both systems where there is an integrated IDE (Microsoft FrontPage against an IIS Development Web site with Visual Source Safe on all of the web content) and where there is not (java command-line development, Visual Studio Express Editions). An intermediate case that I use is jEdit 4.x with VSS integration via plug-in.
I think the integrated case is valuable for the reason it always is -- you don't have to leave your application to interact with source-control functions and you don't have to worry about remembering to add new files and to check out files before editing them. The ability to have a smooth work process and to minimize the risk of oversights is powerful, as far as I am concerned. Even when the IDE-plugin integration is less than perfect (the jEdit 4.x case), I still prefer it over not having it.
I also agree that having explorer integration on Windows, the case for Tortoise SVN, is also a great capability, even when IDE integration is available. This allows convenient operation without having to launch the IDE while also being able to launch from the explorer window into the IDE (depending on file type) or editor or make or whatever while operating in Windows Explorer.
And yes, the command-line interfaces remain valuable, especially for scripting of recuring-operation patterns.
I operate in many contexts. Having low barriers and fluidity of operation in all of them is to be prized.
I'm not sure I understand the question. IDEs by definition are integrated, meaning that they're supposed to help you avoid the need to get out of the environment for anything project-related. Version control obviously fits the bill.
If you're looking for more practical reasons, one is that IDEs can offer you awareness by the nature of their graphical presentation. Eclipse, for example, will present files and directories that have changed. With additional plugins or suites, you can ever get real-time awareness as soon as another user is editing the same file, helping you predict a merge conflict before it occurs. I'm not familiar with a commandline based mechanism.
I use intellij integrated with cvs on a regular basis and by far the best feature of the integration of version control inside the IDE is line-by-line indications of what is added, edited, or deleted along with easy access (mouse hover/tool tip) to the pre-edit changes.
This is all within the source code in a non-obtrusive way.
For the nuts and bolts of version control (checkin/checkout/update/etc) I sometimes use the IDE and sometimes use the command line.
The number 1 reason for an SCM integrated with the IDE is that it makes it more effortless to use it and eliminates the need to REMEMBER to check things out. Through experience I have seen that steps that developers construe as extraneous, which often encompases anything other than writing code, don't get done. Making them do extra steps increases the odds that developers won't bother with it and will work around the source control system
I've used the MIT Continuous testing plugin in the past, but it has long since passed out of date and is no longer compatible with anything approaching a modern release of Eclipse.
Does anyone have a good replacement? Free, naturally, is preferred.
I found that Infinitest now has an Eclipse plugin that seems to work pretty well.
There is a list in this Ben Rady article at Object Mentor: Continuous Testing Explained. Unfortunately the only Eclipse tool appears to be CT-Eclipse which is not currently maintained either.
There is also Fireworks for IntelliJ and Infinitest which is not IDE specific but also has some IntelliJ integration.
My experience is that continuous testing within the IDE can become unwieldy and distracting, so I prefer to use something like CruiseControl to do this kind of testing. One tool I have found very useful is EclEmma, which gives you a very fast coverage turnaround for your units, helping you to decide when you have finished testing a particular area of the code.
Infinitest decides what tests it wants to run. Often it runs the wrong ones. Green bar sometimes good, sometimes meaningless.
I've had good experience with infinitest on a small and simple project. I've not run into any issues with it and find it fast and helpful.
I also use Infinitest (and voted for one of its answers), but I wanted to add another approach, which relies on the build server. Whenever you want to implement something, create a branch in your VCS, do your changes, commit to your branch. If you have a build server configured, which runs unit tests on every checkin, your unit tests are then run on the build server without actually having polluted the trunk (or HEAD, whatever you call it) and without you waiting for the test run to finish.
I admit that this is not really continuous unit testing in the sense you asked the question, but for large projects or large test suites even a "normal" continuous test runner may slow you down way to much.
For small projects I also recommend Infinitest or CT Eclipse.