A bit of context: I am practicing with the former editions of the Google Code Jam, and trying to solve a lot of these puzzles in Java. For each puzzle I create a specific project in Eclipse.
I also built a little "Sample" solver project containing usual operations on input/output files, handling of test cases, script files to run the program on a file quickly, and so on. Now I am using this framework on every puzzle, simply modifying a core "Solver" class which contain the main algorithm. All other files stay the same on every project.
My problem is, I am versioning my work but clearly the only relevant source code to version for each project is this Solver class (and some input/output files). All the rest is duplicated and I would like it to be easily updated when I modify something in the sample project.
On the other hand, I want to be able to easily checkout a project and get it fully working.
I was thinking of using SVN externals to do this but external definitions apply only to subdirectories - and my relevant files are in the same folders as the duplicated ones.
Also SVN ignore does not fulfill my purposes because I would still have to manually replicate any change to my sample project throughout each project.
Do you know of a good way to handle this? Thanks!
Code reuse is typically not accomplished using the version control system, but using polymorphism or libraries. One disadvantage of using the version control system is that you have to do a svn update to pull the new externals from the repository, which strikes me as awkward if you have many projects checked out. Another thing to consider is the development workflow when modifying the reused code. To test your changes, you will probably want to run them with a particular solver, but to do that, you need to svn update - and I am pretty sure you will forget to every once in while, and wonder why your bugfix has no effect ... Therefore I recommend one of the following two approaches:
Polymorphism
Put all your solvers in the same project, making reuse rather trivial. To invoke the right solver, you could do something like:
interface Solver {
// your methods
}
class Ex1Solver implements Solver {
// your solution
}
public static void main(String[] args) throws Exception {
Solver solver = (Solver) Class.forName(args[0]).newInstance();
// work with solver
}
Library
Define an eclipse project for the reused test harness, and a project for each solution. Declare the reused project as dependency of the solution project (In eclipse, right click on the project -> build path -> configure build path -> projects -> add). The test harness would create the solver in the same way as in the polymorphism solution.
You can use svn:externals with files (starting with 1.6) as well, but i would think about a solution on base of a library, cause it's sound like your "framework" is such a kind of thing.
Related
When writing a contract for an API i found myself repeating the same things over and over. For example regex validations for complex json object need to be copy pasted.
Thats tedious and not very DRY.
I'm using Kotlin DSL and Maven
I tried to extract the common parts in another file to reuse it. (Kotlin extension functions ftw.)
After trying multiple things to reuse that file I gave up.
The common parts should be as close to the actual contracts as possible. I don't want to export them in another project, and build them seperat, as they are an vital part of the contracts.
I tried it the following ways:
just put the file in the same directory and importing the functions, hoping it would be resolved as it would in java - did not work at all (my expectations were low, was worth a shot)
putting it in another maven module, and add that as dependency to the spring-cloud-contract-maven-plugin. that worked, as long as the dependant module was built and installed in the local maven repo. if no built version was available maven could not resolve it.
experimenting with kotlin script #file:Import() and #file:DependOn to tackle my issue, no luck.
Is there another way, that I missed? Is there a prefered way doing this?
This must be a common issue, right?
Using Matlab for development and Mercurial for version-control, how do I properly version all code for each of my projects, when they share some common classes and functions?
My current scheme addresses this imperfectly; I have a repository for each project and a repository for the common library. This necessitates a manual manipulation protocol, including:
Manually referencing the project name/version in the commit description for the library
Manually updating the changeset for both the project and library, if reverting to a previous state
This has worked reasonably well so far, but does run the risk of human error in following the protocol and unintended consequences of a library modification on another project. The latter can be addressed with hg update -r on the library, but is error-prone since I have to remember to go back, as I move between projects.
Searching here (and elsewhere), I thought I had found salvation in sub-repository branches, only to discover the practice is basically frowned upon and considered a feature of last resort.
I then found that some folks eschew direct versioning with the project in favor of treating the library as a package for the build software to manage. Taking the library off the Matlab path, creating version clones and telling the builder which one to use, for any particular project, is a brilliant idea - except that I also use the Matlab interpreter to run/debug my code, as well as use the library in various scripts, so I need the library on the Matlab path - which means the builder will automatically pick up the library version that's on the path.
The only other scheme I can think of is to copy the library dependencies into the Project folder for revision control by the project repository. A change protocol would have to include copying the affected library class/function back to the library folder and typing an appropriate commit message. The trick here would be in manually updating project copies of library files, unless there's a Mercurial command to selectively pull from a foreign repo.
Does anybody have a better, more robust way to manage shared library code among projects in both interpreter and build environments?
Thanks to everyone who commented and to those who took the time to read my question. I am loathe to ask questions, since I never think my queries are so novel as to be previously un-asked. But in this case, I was finding it hard to come up with the right search terms/phrase; hence the less-than concise phrasing of my question.
I still don't know if there's a standard approach to managing software configuration for a project, when it includes non-project-specific dependencies, but the scheme I've decided to adopt is outlined below. I should say that the development framework I'm using is Matlab, which may well be argued isn't a terribly good framework for developing a GUI application, but it's the only one I have for now. Should I move to .Net, or some other framework, then maybe some of the issues I'm having would be much more readily resolved.
I decided the ability to version a project in its entirety took precedence and so I copied all of the project-agnostic dependencies (that is, functions and classes that I've developed) from a central library repo to a folder within the project repo.
It just means I have to be disciplined in managing the Matlab search path, as well as the protocol for copying changes made to these dependencies back to the central library - and for polling the library for any changes that originated from another project.
This doesn't seem elegant, but it does make me think more carefully about the functions and interfaces that I put into the library, which should be a good thing.
I have seen, read and thought of different ways of using Workspaces (per project, per application (multi-asseted or not), per program language, per target (web-development, plugins,..), and so on) and I am still doubting what the best approach is.
Can anyone give a detailed, but not a page long insight into this?
This involves a lot of sub-questions, so to speak, and I don't know all the specific sub-questions I should ask, as I am sure I don't know all aspects of Eclipse (and Workspaces), but I'll try to give an example of what I am looking for:
What for?
What did the Eclipse development team expect it to be used for?
What do other/most people think?
What do you think?
... ?
Why?
Are there configuration conflicts vs. sharing merits?
Any filespace reasons?
Performance?
... ?
I am speaking of the minimum use-case for a developer that uses different languages and protocols, not necessarily all of them in one project (E.g. Php, Javascript and XML for some projects, C# for others, Java and SQL for still others, etc..)
Edit 2012-11-27: Don't get me wrong. I don't doubt the use of
Workspaces, I just want to use it as it is meant to be or otherwise if
anyone would think it better. So "what for?" means: What's the best use? And
"why?" actually targets on the "what for?", in other words: tell me the reasons
for your answer.
I'll provide you with my vision of somebody who feels very uncomfortable in the Java world, which I assume is also your case.
What it is
A workspace is a concept of grouping together:
a set of (somehow) related projects
some configuration pertaining to all these projects
some settings for Eclipse itself
This happens by creating a directory and putting inside it (you don't have to do it, it's done for you) files that manage to tell Eclipse these information. All you have to do explicitly is to select the folder where these files will be placed. And this folder doesn't need to be the same where you put your source code - preferentially it won't be.
Exploring each item above:
a set of (somehow) related projects
Eclipse seems to always be opened in association with a particular workspace, i.e., if you are in a workspace A and decide to switch to workspace B (File > Switch Workspaces), Eclipse will close itself and reopen. All projects that were associated with workspace A (and were appearing in the Project Explorer) won't appear anymore and projects associated with workspace B will now appear. So it seems that a project, to be open in Eclipse, MUST be associated to a workspace.
Notice that this doesn't mean that the project source code must be inside the workspace. The workspace will, somehow, have a relation to the physical path of your projects in your disk (anybody knows how? I've looked inside the workspace searching for some file pointing to the projects paths, without success).
This way, a project can be inside more than 1 workspace at a time. So it seems good to keep your workspace and your source code separated.
some configuration pertaining to all these projects
I heard that something, like the Java compiler version (like 1.7, e.g - I don't know if 'version' is the word here), is a workspace-level configuration. If you have several projects inside your workspace, and compile them inside of Eclipse, all of them will be compiled with the same Java compiler.
some settings for Eclipse itself
Some things like your key bindings are stored at a workspace-level, also. So, if you define that ctrl+tab will switch tabs in a smart way (not stacking them), this will only be bound to your current workspace. If you want to use the same key binding in another workspace (and I think you want!), it seems that you have to export/import them between workspaces (if that's true, this IDE was built over some really strange premises). Here is a link on this.
It also seems that workspaces are not necessarily compatible between different Eclipse versions. This article suggests that you name your workspaces containing the name of the Eclipse version.
And, more important, once you pick a folder to be your workspace, don't touch any file inside there or you are in for some trouble.
How I think is a good way to use it
(actually, as I'm writing this, I don't know how to use this in a good way, that's why I was looking for an answer – that I'm trying to assemble here)
Create a folder for your projects:
/projects
Create a folder for each project and group the projects' sub-projects inside of it:
/projects/proj1/subproj1_1
/projects/proj1/subproj1_2
/projects/proj2/subproj2_1
Create a separate folder for your workspaces:
/eclipse-workspaces
Create workspaces for your projects:
/eclipse-workspaces/proj1
/eclipse-workspaces/proj2
The whole point of a workspace is to group a set of related projects together that usually make up an application. The workspace framework comes down to the eclipse.core.resources plugin and it naturally by design makes sense.
Projects have natures, builders are attached to specific projects and as you change resources in one project you can see in real time compile or other issues in projects that are in the same workspace. So the strategy I suggest is have different workspaces for different projects you work on but without a workspace in eclipse there would be no concept of a collection of projects and configurations and after all it's an IDE tool.
If that does not make sense ask how Net Beans or Visual Studio addresses this? It's the same theme. Maven is a good example, checking out a group of related maven projects into a workspace lets you develop and see errors in real time. If not a workspace what else would you suggest? An RCP application can be a different beast depending on what its used for but in the true IDE sense I don't know what would be a better solution than a workspace or context of projects. Just my thoughts. - Duncan
Basically the scope of workspace(s) is divided in two points.
First point (and primary) is the eclipse it self and is related with the settings and metadata configurations (plugin ctr). Each time you create a project, eclipse collects all the configurations and stores them on that workspace and if somehow in the same workspace a conflicting project is present you might loose some functionality or even stability of eclipse it self.
And second (secondary) the point of development strategy one can adopt.
Once the primary scope is met (and mastered) and there's need for further adjustments regarding project relations (as libraries, perspectives ctr) then initiate separate workspace(s) could be appropriate based on development habits or possible language/frameworks "behaviors".
DLTK for examples is a beast that should be contained in a separate cage.
Lots of complains at forums for it stopped working (properly or not at all) and suggested solution was to clean the settings of the equivalent plugin from the current workspace.
Personally, I found myself lean more to language distinction when it comes to separate workspaces which is relevant to known issues that comes with the current state of the plugins are used. Preferably I keep them in the minimum numbers as this is leads to less frustration when the projects are become... plenty and version control is not the only version you keep your projects.
Finally, loading speed and performance is an issue that might come up if lots of (unnecessary) plugins are loaded due to presents of irrelevant projects.
Bottom line; there is no one solution to every one, no master blue print that solves the issue. It's something that grows with experience,
Less is more though!
Although I've used Eclipse for years, this "answer" is only conjecture (which I'm going to try tonight). If it gets down-voted out of existence, then obviously I'm wrong.
Oracle relies on CMake to generate a Visual Studio "Solution" for their MySQL Connector C source code. Within the Solution are "Projects" that can be compiled individually or collectively (by the Solution). Each Project has its own makefile, compiling its portion of the Solution with settings that are different than the other Projects.
Similarly, I'm hoping an Eclipse Workspace can hold my related makefile Projects (Eclipse), with a master Project whose dependencies compile the various unique-makefile Projects as pre-requesites to building its "Solution". (My folder structure would be as #Rafael describes).
So I'm hoping a good way to use Workspaces is to emulate Visual Studio's ability to combine dissimilar Projects into a Solution.
It's just a feature for structuring projects.
Obviously Eclipse designers tried to avoid having global settings for Eclipse and decided to put them into workspace.
Each Eclipse app depends on each workspace settings.
Is it a good decision? I think it's not so.
It lacks flexibility. It was naive to expect that global settings can be avoided.
It doesn't allow you to have single projects (it can be a surprise for Eclipse designers but it happens quite often).
But it still works.
Many people use it. Sometimes they suffer but more frequently everything is ok.
Situation:
I need several swf/exe output files compiled in FlashDevelop from several projects. More than 60% of ActionScript 3.0 source is common for all project, rest are project-specific. How can I organize that in FlashDevelop? I want to have "one-click-to-build all" setting without duplicating common codebase (so when I need to fix something I do not need to copy-paste solution into several files).
All sources are under develeopment and will change very often.
A straightforward solution is to make an external classpath, for instance:
c:\dev\shared_src\
c:\dev\project1\
c:\dev\project2\
Then configure each project:
Project Properties > Classpath
Add Classpath > select '../shared_src'
PS: of course you should keep everything under source control.
Using svn:externals you could structure your repository in such a way that the commom parts are stored just once in the source control system, so changes made can be synchronised with just a single commit and update cycle.
For example, imagine that you have ^/ProjectA and ^/ProjectB, each of with require ^/Common as a sub directory.
Using svn:externals, pull ^/Common into both projects.
The exact nature of doing this will depend on the version of svn you use, and any client you use (such as TortoiseSvn). Refer to the relevant edition of the svn book for specifics.
The ease of implementing this will depend quite a lot on how separate the common code currently is in your application; and pulling in directories as directories is much more practical than trying to pull them into an existing directory; and unfortunately wildcards for filepaths are not supported.
However, based on your description of your aim; this is the most straight-forward solution I can imagine.
Hope this helps.
Almost any IDE creates lots of files that have nothing to do with the application being developed, they are generated and mantained by the IDE so he knows how to build the application, where the version control repository is and so on.
Should those files be kept under version control along with the files that really have something to do with the aplication (source code, application's configuration files, ...)?
The things is: on some IDEs if you create a new project and then import it into the version-control repository using the version-control client/commands embedded in the IDE, then all those files are sent to the respitory. And I'm not sure that's right: what is two different developers working on the same project want to use two different IDEs?
I want to keep this question agnostic avoiding references to any particular IDE, programming language or version control system. So this question is not exactly the same as these:
SVN and binaries - but this talks about binaries and SVN
Do you keep your build tools in version control? - but this talks about build tools (e.g. putting the jdk under version control)
What project files shouldn’t be checked into SVN - but this talks about SVN and dll's
Do you keep your project files under version control? - very similar (haven't found it before), thanks VonC
Rules of thumb:
Include everything which has an influence on the build result (compiler options, file encodings, ASCII/binary settings, etc.)
Include everything to make it possible to open the project from a clean checkout and being able to compile/run/test/debug/deploy it without any further manual intervention
Don't include files which contain absolute paths
Avoid including personal preferences (tab size, colors, window positions)
Follow the rules in this order.
[Update] There is always the question what should happen with generated code. As a rule of thumb, I always put those under version control. As always, take this rule with a grain of salt.
My reasons:
Versioning generated code seems like a waste of time. It's generated right? I can get it back at a push of a button!
Really?
If you had to bite the bullet and generate the exact same version of some previous release without fail, how much effort would it be? When generating code, you not only have to get all the input files right, you also have to turn back time for the code generator itself. Can you do that? Always? As easy as it would be to check out a certain version of the generated code if you had put it under version control?
And even if you could, could you ever be sure that didn't miss something?
So on one hand, putting generated code under version control make sense since it makes it dead easy to do what VCS are meant for: Go back in time.
Also it makes it easy to see the differences. Code generators are buggy, too. If I fix a bug and have 150'000 files generated, it helps a lot when I can compare them to the previous version to see that a) the bug is gone and b) nothing else changed unexpectedly. It's the unexpected part which you should worry about. If you don't, let me know and I'll make sure you never work for my company ever :-)
The major pain point of code generators is stability. It doesn't do when your code generator just spits out a random mess of bytes every time you run (well, unless you don't care about quality). Code generators need to be stable and deterministic. You run them twice with the same input and the output must be identical down to least significant bit.
So if you can't check in generated code because every run of the generator creates differences that aren't there, then your code generator has a bug. Fix it. Sort the code when you have to. Use hash maps that preserve order. Do everything necessary to make the output non-random. Just like you do everywhere else in your code.
Generated code that I might not put under version control would be documentation. Documentation is somewhat of a soft target. It doesn't matter as much when I regenerate the wrong version of the docs (say, it has a few typos more or less). But for releases, I might do that anyway so I can see the differences between releases. Might be useful, for example, to make sure the release notes are complete.
I also don't check in JAR files. As I do have full control over the whole build and full confidence that I can get back any version of the sources in a minute plus I know that I have everything necessary to build it without any further manual intervention, why would I need the executables for? Again, it might make sense to put them into a special release repo but then, better keep a copy of the last three years on your company's web server to download. Think: Comparing binaries is hard and doesn't tell you much.
I think it's best to put anything under version control that helps developers to get started quickly, ignoring anything that may be auto-generated by an IDE or build tools (e.g. Maven's eclipse plugin generates .project and .classpath - no need to check these in). Especially avoid files that change often, that contain nothing but user preferences, or that conflict between IDEs (e.g. another IDE that uses .project just like eclipse does).
For eclipse users, I find it especially handy to add code style (.settings/org.eclipse.jdt.core.prefs - auto formatting on save turned on) to get consistently formatted code.
Everything that can be automatically generated from the source+configuration files should not be under the version control! It only causes problems and limitations (like the one you stated - using 2 different project files by different programmers).
Its true not only for IDE "junk files" but also for intermediate files (like .pyc in python, .o in c etc).
This is where build automation and build files come in.
For example, you can still build the project (the two developers will need the same build software obviously) but they then could in turn use two different IDE's.
As for the 'junk' that gets generated, I tend to ignore most if it. I know this is meant to be language agnostic but consider Visual Studio. It generates user files (user settings etc..) this should not be under source control.
On the other hand, project files (used by the build process) most certainly should. I should add that if you are on a team and have all agreed on an IDE, then checking in IDE specific files is fine providing they are global and not user specific and/or not needed.
Those other questions do a good job of explaining what should and shouldn't be checked into source control so I wont repeat them.
In my opinion it depends on the project and environment. In a company environment where everybody is using the same IDE it can make sense to add the IDE files to the repository. While this depends a bit on the IDE, as some include absolute paths to things.
For a project which is developed in different environments it doesn't make sense and will be pain in the long run as the project files aren't maintained by all developers and make it harder to find "relevant" things.
Anything that would be devastating if it were lost, should be under version control.
In my opinion, anything needed to build the project (code, make files, media, databases with required program info, etc) should be in repositories. I realise that especially for media/database files this is contriversial, but to me if you can't branch and then hit build the source control's not doing it's job. This goes double for distributed systems with cheap branch creation/merging.
Anything else? Store it somewhere different. Developers should choose their own working environment as much as possible.
From what I have been looking at with version control, it seems that most things should go into it - e.g. source code and so on. However, the problem that many VCS's run into is when trying to handle large files, typically binaries, and at times things like audio and graphic files. Therefore, my personal way to do it is to put the source code under version control, along with general small sized graphics, and leave any binaries to other systems of management. If it is a binary that I created myself using the build system of the IDE, then that can definitily be ignored, because it is going to be regenerated every build. For dependancy libraries, well this is where dependancy package managers come in.
As for IDE generated files (I am assuming these are ones that aren't generated during the build process, such as the solution files for Visual Studio) - well, I think it would depend on whether or not you are working alone. If you are working alone, then go ahead and add them - they will allow you to revert settings in the solution or whatever you make. Same goes for other non-solution like files as well. However, if you are collaborating, then my recomendation is no - most IDE generated files tend to be, well, user specific - aka they work on your machine, but not neccesarily on others. Hence, you may be better of not including IDE generated files in that case.
tl;dr you should put most things that relate to your program into version control, excluding dependencies (things like libraries, graphics and audio should be handled by some other dependancy management system). As for things directly generated by the IDE - well, it would depend on if you are working alone or with other people.