What is a good strategy for dealing with changing product and feature names in source code. Here's the situation I find myself in over and over again (most of you can relate?)...
Product name starts off as "DaBomb"
Major features are "Exploder", "Lantern" and "Flag".
Time passes, and the Feature names are changed to "Boom", "Lighthouse" and "MarkMan"
Time passes, and the product name changes to "DaChronic"
...
...
Blah, blah, blah...over and over and over
And now we have a large code base with 50 different names sprinkled around the directory tree and source files, most of which are obsolete. Only the veterans remember what each name means, the full etimologic history, etc.
What is the solution to this mess?
Clarification: I don't mean the names that customers see, I mean the names of directories, source files, classes, variables, etc. that the developers see where the changing product and feature names get woven into.
Given your clarification that you "don't mean the names that customers see, [you] mean the names of directories, source files, classes, variables, etc. that the developers see", yeah, this can be an annoying problem.
The way teams I've been on have coped with best when we've had a policy of always using only one name for each thing in the code base. If the name changes later on we either stay with the old name in the code, or we migrate all instances of the old name to the new name. The important thing is to never start using the new name in the code unless all instance of the old name have been migrated. That way you only ever have to keep 2 names for something in your head: the "old name", used in the code, and the name everyone else uses.
We've also often chosen a very generic/descriptive name for things when starting out if we know the "brand name" is likely to change.
I consider renaming to better naming conventions just another form of refactoring. Create a branch, perform the renames, run unit/integration tests, commit, merge, repeat. It's all about process control to keep consistency in the project.
The solution to the mess is to not create it in the first place. Once a code path is named, there's rarely a good reason to change it and never a good reason to use a new name alongside the old one. When "Exploder" becomes "Boom", you have two choices: Either keep using Exploder exclusively, and never mention Boom anywhere, or change all instances of Exploder to Boom and then continue on using Boom exclusively and never mention Exploder again.
If you're using both Exploder and Boom in the same code base, you're doing it wrong.
Also, I know you clarified that you're not talking about the user-visible names, but, if you start out working with your own internal names which are relevant to what the code does and completely independent of what marketing wants to call the product/feature, then this is much less likely to become an issue. If you're already referring to Exploder internally as TNT, then what difference does it make if Exploder gets changed to Boom?
How do you deal with Localization? Same thing; same method.
We use an internal and and external name. It could be as simple as a static variable definition like
public static final String EXPLODER = "Boom";
And in code you'll always use the reference to EXPLODER. Same for path names and the like - hard coding those paths at different places is a no-go anyway. If some guys starts digging through internal stuff (like JS sources or ini files or whatever), who cares if they discover Exploder?
Just use internal names, and ignore changes to marketing/official names: https://softwareengineering.stackexchange.com/a/208578/55472.
Related
In SugarCRM, you can create your custom modules (e.g. MyModule) and they are kept in /modules just like stock objects, with any default metadata, views, language files, etc. For a custom module MyModule, you might have something like:
/modules/MyModule/MyModule.class.php
/modules/MyModule/MyModule.php
/modules/MyModule/language/
/modules/MyModule/metadata
And so on, so that everything is nicely defined and all the modules are kept together. The module becomes registered with the system by a file such as /custom/Extension/application/Include/MyModule.php with contents something like:
<?php
$beanList['MyModule'] = 'MyModule';
$beanFiles['MyModule'] = 'modules/MyModule/MyModule.php';
$moduleList[] = 'MyModule';
Obviously, the $beanFiles array references where we can find the base module's class, usually an extension of the SugarBean object. Recently I was advised that we can adjust that file's location for the sake of customization, and it makes sense to a degree. Setting it like $beanFiles['MyModule'] = 'custom/modules/MyModule/MyModule.php'; would allow us to access the base class via Module Loader even if the security scan tool prevents core file changes, and this would also allow us to not exactly extend, but replace stock modules like Accounts or Calls, without modifying core files and having system upgrades to wipe out the changes.
So here's my question: what is the best practice here? I've been working with SugarCRM pretty intensely for several years and this is the first time I've ever been tempted to modify the $beanFiles array. My concern is that I'm deviating from best practice here, and also that somehow both files modules/MyModule/MyModule.php and custom/modules/MyModule/MyModule.php could be loaded which would cause a class name conflict in PHP (i.e. because both classes are named MyModule...). Obviously, any references to the class would need to be updated (e.g. an entryPoint that works with this module), but am I missing any potential ramifications?
Technically it should be fine, but I can see how it could be possible that both the core version and your version could conflict if both are referenced. It all depends on the scenario, but I prefer to extend the core bean and find somewhere in the stack where I can have my custom version used in place of the core bean. I wrote up an example a couple of years ago here: https://www.sugaroutfitters.com/blog/safely-customizing-a-core-bean-in-sugarcrm
For most use-cases, there's a way to hijack Sugar to use your bean at a given point.
If you can't get around it you can always grep to see where the core module is explicitly being included to ensure that there won't be conflict down the road.
EDIT: I completely re-wrote the question since it seems like I was not clear enough in my first two versions. Thanks for the suggestions so far.
I would like to internationalize the source code for a tutorial project (please notice, not the runtime application). Here is an example (in Java):
/** A comment */
public String doSomething() {
System.out.println("Something was done successfully");
}
in English , and then have the French version be something like:
/** Un commentaire */
public String faitQuelqueChose() {
System.out.println("Quelque chose a été fait avec succès.");
}
and so on. And then have something like a properties file somewhere to edit these translations with usual tools, such as:
com.foo.class.comment1=A comment
com.foo.class.method1=doSomething
com.foo.class.string1=Something was done successfully
and for other languages:
com.foo.class.comment1=Un commentaire
com.foo.class.method1=faitQuelqueChose
com.foo.class.string1=Quelque chose a été fait avec succès.
I am trying to find the easiest, most efficient and unobtrusive way to do this with the least amount of manual grunt work (other than obviously translating the actual text). Preferably working under Eclipse. For example, the original code would be written in English, then externalized (to properties, preferably leaving the original source untouched), translated (humanly) and then re-generated (as a separate source file / project).
Some trails I have found (other than what AlexS suggested):
AntLR, a language parser / generator. There seems to be a supporting Eclipse plugin
Using Eclipse's AST (Abstract Syntax Tree) and I guess building some kind of plugin.
I am just surprised there isn't a tool out there that does this already.
I'd use unique strings as methodnames (or anything you want to be replaced by localized versions.
public String m37hod_1() {
System.out.println(m355a6e_1);
}
then I'd define a propertyfile for each language like this:
m37hod_1=doSomething
m355a6e_1="Something was done successfully"
And then I'd write a small program parsing the sourcefiles and replacing the strings. So everything just outside eclipse.
Or I'd use the ant task Replace and propertyfiles as well, instead of a standalone translation program.
Something like that:
<replace
file="${src}/*.*"
value="defaultvalue"
propertyFile="${language}.properties">
<replacefilter
token="m37hod_1"
property="m37hod_1"/>
<replacefilter
token="m355a6e_1"
property="m355a6e_1"/>
</replace>
Using one of these methods you won't have to explain anything about localization in your tutorials (except you want to), but can concentrate on your real topic.
What you want is a massive code change engine.
ANTLR won't do the trick; ASTs are necessary but not sufficient. See my essay on Life After Parsing. Eclipse's "AST" may be better, if the Eclipse package provides some support for name and type resolution; otherwise you'll never be able to figure out how to replace each "doSomething" (might be overloaded or local), unless you are willing to replace them all identically (and you likely can't do that, because some symbols refer to Java library elements).
Our DMS Software Reengineering Toolkit could be used to accomplish your task. DMS can parse Java to ASTs (including comment capture), traverse the ASTs in arbitrary ways, analyze/change ASTs, and the export modified ASTs as valid source code (including the comments).
Basically you want to enumerate all comments, strings, and declarations of identifiers, export them to an external "database" to be mapped (manually? by Google Translate?) to an equivalent. In each case you want to note not only the item of interest, but its precise location (source file, line, even column) because items that are spelled identically in the original text may need different spellings in the modified text.
Enumeration of strings is pretty easy if you have the AST; simply crawl the tree and look for tree nodes containing string literals. (ANTLR and Eclipse can surely do this, too).
Enumeration of comments is also straightforward if the parser you have captures comments. DMS does. I'm not quite sure if ANTLR's Java grammar does, or the Eclipse AST engine; I suspect they are both capable.
Enumeration of declarations (classes, methods, fields, locals) is relatively straightforward; there's rather more cases to worry about (e.g., anonymous classes containing extensions to base classes). You can code a procedure to walk the AST and match the tree structures, but here's the place that DMS starts to make a difference: you can write surface-syntax patterns that look like the source code you want to match. For instance:
pattern local_for_loop_index(i: IDENTIFIER, t: type, e: expression, e2: expression, e3:expression): for_loop_header
= "for (\t \i = \e,\e2,\e3)"
will match declarations of local for loop variables, and return subtrees for the IDENTIFIER, the type, and the various expressions; you'd want to capture just the identifier (and its location, easily done by taking if from the source position information that DMS stamps on every tree node). You'd probably need 10-20 such patterns to cover the cases of all the different kinds of identifiers.
Capture step completed, something needs to translate all the captured entities to your target language. I'll leave that to you; what's left is to put the translated entities back.
The key to this is the precise source location. A line number isn't good enough in practice; you may have several translated entities in the same line, in the worst case, some with different scopes (imagine nested for loops for example). The replacement process for comments, strings and the declarations are straightforward; rescan the tree for nodes that match any of the identified locations, and replace the entity found there with its translation. (You can do this with DMS and ANTLR. I think Eclipse ADT requires you generate a "patch" but I guess that would work.).
The fun part comes in replacing the identifier uses. For this, you need to know two things:
for any use of an identifier, what is the declaration is uses; if you know this, you can replace it with the new name for the declaration; DMS provides full name and type resolution as well as a usage list, making this pretty easy, and
Do renamed identifiers shadow one another in scopes differently than the originals? This is harder to do in general. However, for the Java language, we have a "shadowing" check, so you can at least decide after renaming that you have an issues. (There's even a renaming procedure that can be used to resolve such shadowing conflicts
After patching the trees, you simply rewrite the patched tree back out as a source file using DMS's built-in prettyprinter. I think Eclipse AST can write out its tree plus patches. I'm not sure ANTLR provides any facilities for regenerating source code from ASTs, although somebody may have coded one for the Java grammar. This is harder to do than it sounds, because of all the picky detail. YMMV.
Given your goal, I'm a little surprised that you don't want a sourcefile "foo.java" containing "class foo { ... }" to get renamed to .java. This would require not only writing the transformed tree to the translated file name (pretty easy) but perhaps even reconstructing the directory tree (DMS provides facilities for doing directory construction and file copies, too).
If you want to do this for many languages, you'd need to run the process once per language. If you wanted to do this just for strings (the classic internationalization case), you'd replace each string (that needs changing, not all of them do) by a call on a resource access with a unique resource id; a runtime table would hold the various strings.
One approach would be to finish the code in one language, then translate to others.
You could use Eclipse to help you.
Copy the finished code to language-specific projects.
Then:
Identifiers: In the Outline view (Window>Show View>Outline), select each item and Refactor>Rename (Alt+Shift+R). This takes care of renaming the identifier wherever it's used.
Comments: Use Search>File to find all instances of "/*" or "//". Click on each and modify.
Strings:
Use Source>Externalize strings to find all of the literal strings.
Search>File for "Messages.getString()".
Click on each result and modify.
On each file, ''Edit>Find/Replace'', replacing "//\$NON-NLS-.*\$" with empty string.
for the printed/logged string, java possess some internatization functionnalities, aka ResourceBundle. There is a tutorial about this on oracle site
Eclipse also possess a funtionnality for this ("Externalize String", as i recall).
for the function name, i don't think there anything out, since this will require you to maintain the code source on many version...
regards
Use .properties file, like:
Locale locale = new Locale(language, country);
ResourceBundle captions= ResourceBundle.getBundle("Messages",locale);
This way, Java picks the Messages.properties file according to the current local (which is acquired from the operating system or Java locale settings)
The file should be on the classpath, called Messages.properties (the default one), or Messages_de.properties for German, etc.
See this for a complete tutorial:
http://docs.oracle.com/javase/tutorial/i18n/intro/steps.html
As far as the source code goes, I'd strongly recommend staying with English. Method names like getUnternehmen() are worse to the average developer then plain English ones.
If you need to familiarize foreign developers to your code, write a proper developer documentation in their language.
If you'd like to have Javadoc in both English and other languages, see this SO thread.
You could write your code using freemarker templates (or another templating language such as velocity).
doSomething.tml
/** ${lang['doSomething.comment']} */
public String ${lang['doSomething.methodName']}() {
System.out.println("${lang['doSomething.message']}");
}
lang_en.prop
doSomething.comment=A comment
doSomething.methodName=doSomething
doSomething.message=Something was done successfully
And then merge the template with each language prop file during your build (using Ant / Gradle / Maven etc.)
What does this file hold and how important is it to keep it "correct"?
I've seen people write scripts to just merge any change dealing with it and I've heard others merging it manually every time.
What is the correct way to handle it and why?
The project.pbxproj contains all of the metadata about your project that Xcode uses to build it; the settings, the file references, configuration, targeted platforms, etc...
I.e. it is a critically important.
There really isn't a great answer for this. Typically, teams will avoid conflict by limiting edits to the project to one team member at a time.
The Xcode team has put a lot of effort into making the file merge-friendly. In managing several large projects via svn, I've generally found that the merges are automatic and painless.
Until they aren't. And when they aren't, revert, merge changes by hand (i.e. make the changes in the project that conflicted), and move on.
Try my script xUnique.
What it does:
convert project.pbxproj to JSON format
Iterate all objects in JSON and give every UUID an absolute path, and create a new UUID using MD5 hex digest of the path
All elements in this json object is actually connected as a tree
We give a path attribute to every node of the tree using its unique attribute; this path is the absolute path to the root node,
Apply MD5 hex digest to the path for the node
Replace all old UUIDs with the MD5 hex digest and also remove unused UUIDs that are not in the current node tree and UUIDs in wrong format
Sort the project file inlcuding children, files, PBXFileReference and PBXBuildFile list and remove all duplicated entries in these lists
see sort_pbxproj method in xUnique.py if you want to know the implementation;
It's ported from my modified sort-Xcode-project-file, with some differences in ordering PBXFileReference and PBXBuildFile
With different options, you can use xUnique with more flexibility
...and why has the package this misleading name (I assumed it had something to do with JavaME or mobile/smart phones)?
I found no references on the internet about scala.mobile.Code or scala.mobile.Location at all nor did I manage to do anything with those classes except getting ClassCastExcetions or NoSuchMethodErrors.
Actually there is not even a single test against scala.mobile in the Scala's test tree which could help understanding that code.
The classes really smell like they were forgotten in the source tree a long time ago and got accidentally released since that.
Maybe I just missed something about them?
Update:
scala.mobile was removed in Scala 2.9.
I just checked the source code.
When Scala changed the name mangling of class files a few years ago and it seems people forgot to update these classes accordingly.
So my answer would be:
At least Location has no purpose, because it is not possible to get anything sensible out of it (except exceptions) and Code without Location is severely limited. It works though if you pass the class literal to Code directly:
import scala.mobile._
val c = new Code(classOf[scala.collection.mutable.StringBuilder])
c.apply[StringBuilder, String]("append")("Foo")
c.apply[String]("toString")() // returns "Foo"
c.apply[Int]("length")() // returns 3
Looks like yet-another implementation in the standard library of reflection-slightly-nicer.
The description of Location pretty much explains what that is about:
The class Location provides a create method to instantiate objects
from a network location by specifying the URL address of the jar/class file.
It might be used by remote actors. Maybe.
As for why it has this misleading name? Well, back in 2004 smart phones had really low penetration, so maybe the association wasn't all that strong.
Problem
Your organization has many separate applications, some of which interact with each other (to form "systems"). You need to deploy these applications to separate environments to facilitate staged testing (for example, DEV, QA, UAT, PROD). A given application needs to be configured slightly differently in each environment (each environment has a separate database, for example). You want this re-configuration to be handled by some sort of automated mechanism so that your release managers don't have to manually configure each application every time it is deployed to a different environment.
Desired Features
I would like to design an organization-wide configuration solution with the following properties (ideally):
Supports "one click" deployments (only the environment needs to be specified, and no manual re-configuration during/after deployment should be necessary).
There should be a single "system of record" where a shared environment-dependent property is specified (such as a database connection string that is shared by many applications).
Supports re-configuration of deployed applications (in the event that an environment-specific property needs to change), ideally without requiring a re-deployment of the application.
Allows an application to be run on the same machine, but in different environments (run a PROD instance and a DEV instance simultaneously).
Possible Solutions
I see two basic directions in which a solution could go:
Make all applications "environment aware". You would pass the environment name (DEV, QA, etc) at the command line to the app, and then the app is "smart" enough to figure out the environment-specific configuration values at run-time. The app could fetch the values from flat files deployed along with the app, or from a central configuration service.
Applications are not "smart" as they are in #1, and simply fetch configuration by property name from config files deployed with the app. The values of these properties are injected into the config files at deploy-time by the install program/script. That install script takes the environment name and fetches all relevant configuration values from a central configuration service.
Question
How would/have you achieved a configuration solution that solves these problems and supports these desired features? Am I on target with the two possible solutions? Do you have a preference between those solutions? Also, please feel free to tell me that I'm thinking about the problem all wrong. Any feedback would be greatly appreciated.
We've all run into these kinds of things, particularly in large organizations. I think it's most important to manage your own expectations first, and also ask whether it's really necessary to tell every system and subsystem on a given box to "change to DEV mode" or "change to PROD mode". My personal recommendation is as follows:
Make individual boxes responsible for a different stage - i.e. "this is a DEV box", and "this is a PROD box".
Collect as much of the configuration that differs from box to box in one location, even if it requires soft links or scripts that collect the information to then print out.
A. This way, you can easily "dump this box's configuration" in two places and see what differs, for example after a new deployment.
B. You can also make configuration changes separate from software changes, at least to some degree, which is a good way to root out bugs that happen at release time.
Then have everything base its configuration on something/somewhere that is not baked-in or hard-coded - just make sure to collect and document it in that one location. It almost doesn't matter what the mechanism is, which is a good thing, because some systems just don't want to be forced to use some mechanisms or others.
Sorry if this is too general an answer - the question was very general. I've worked in several large software-based organizations before, and this seemed to be the best approach. Using a standalone server as "one unit of deployment" is the most realistic scenario (though sometimes its expensive), since applications affect each other, and no matter how careful you are, you destabilize a whole system when you move any given gear or cog.
The alternative gets very complex very quickly. You need to start rewriting the applications that you have control over in order to have them accept a "DEV" switch, and you end up adding layers of kludge to the ones you don't have control over. Usually, the ones you don't have control over at least base their properties on something defined on a system-wide level, unless they are "calling the mothership for instructions".
It's easier to redirect people to a remote location and have them "use DEV" vs "use PROD" than it is to "make this machine run like DEV" vs "make this machine run like PROD". And if you're mixing things up, like having a DEV task run together on the same box as a PROD task, then that's not a realistic scenario anyways: I guarantee that eventually you will be granting illegal DEV-only access to somebody on PROD, and you'll have a DEV task wipe out a PROD database.
Hope this helps. Let me know if you'd like to discuss more specifics involved.
I personally prefer solution 2 (the app should know itself, by its configuration, what environment it is running in). With solution 1 (pass the environment name as a startup parameter) the danger of using the wrong environment specifier is much too high. Accessing the TEST database from PROD code and vice versa may cause mayhem, if the two installed code bases are not of the same version, as is often the case.
My current project uses solution 1, but I don't like that. A previous project I worked on used a variation of solution 2: The build process generated one setup file for every environment, making sure that they contained the same code base but appropriate configuration paramters. That worked like a charm, but I know it contradicts the paradigm that the "exact same build files must be deployed everywhere".
I think I have asked a related, self-answered, question, before I read this one : How to organize code so that we can move and update it without having to edit the location of the configuration file? . So, on that basis, I provide an answer here. I don't like the idea of "smart" application (solution 1 here) for such a simple task as finding environment settings. It seems a complicated framework for something that should be simple. The idea of an install script (solution 2 here) is powerful, but it is useful to allow the user to change the content of the config file, but would it allow to change the location of this config file? What is this "central configuration service", where is it located? My answer is that I would go with option 2, if the goal is to set the content of the configuration file, but I feel that the issue of the location of this configuration file remains unanswered here.
If you're using JSON to store/transmit configuration (or can use JSON in your pre-deploy process to output to some other format) you can annotate key/property names for environment/context-specific values with arbitrary or environment-specific suffixes, and then dynamically prefer/discriminate them at build/deploy/run/render -time, while leaving un-annotated properties alone.
We have used this to avoid duplicating entire configuration files (with the associated problems well known) AND to reduce repetition. The technique is also perfect for internationalization (i18n) -- even within the same file, if desired.
Example, snippet of pre-processed JSON config:
var config = {
'ver': '1.0',
'help': {
'BLURB': 'This pre-production environment is not supported. Contact Development Team with questions.',
'PHONE': '808-867-5309',
'EMAIL': 'coder.jen#lostnumber.com'
},
'help#www.productionwebsite.com': {
'BLURB': 'Please contact Customer Service Center',
'BLURB#fr': 'S\'il vous plaît communiquer avec notre Centre de service à la clientèle',
'BLURB#de': 'Bitte kontaktieren Sie unseren Kundendienst!!1!',
'PHONE': '1-800-CUS-TOMR',
'EMAIL': 'customer.service#productionwebsite.com'
},
}
... and post-processed (in this case, at render time) given dynamic, browser-environment-known location.hostname='www.productionwebsite.com' and navigator.language of 'de'):
prefer(config,['www.productionwebsite.com','de']); // prefer(obj,string|Array<string>)
JSON.stringify(config); // {
'ver': '1.0',
'help': {
'BLURB': 'Bitte kontaktieren Sie unseren Kundendienst!!1!',
'PHONE': '1-800-CUS-TOMR',
'EMAIL': 'customer.service#productionwebsite.com'
}
}
If a non-annotated ('base') property has no competing annotated property, it is left alone (presumably global across environments) otherwise its value is replaced by an annotated value, if the suffix matches one of the inputs to the preference/discrimination function. Annotated properties that do not match are dropped entirely.
You can mix and match this behaviour to annotate configuration to achieve distinctions of global, default, specific that are (assuming you're sensible) readable with zero/minimal duplication.
The single, recursive prefer() function (as we're calling it, lacking the need or desire to make an entire project/framework out of it) we've developed so far (see jsFiddle, with inline docs) goes a bit further than this simple example, and (explained in greater detail here) handles deeply-nested configuration objects, as well as preferential ordering and (if you need to stay flat) combination of suffixes.
The function relies on JS ability to reference object properties as strings, dynamically, and tolerate # and & delimiters in property names which are not valid in dot-notation syntax but consequently (help) prevent developers from breaking this technique by accidentally referring to pre-processed/annotated attributes in code (unless they, non-conventionally don't prefer to use dot-notation.)
We have yet to have this break anything for us, nor have we been schooled on any fundamental flaws of this technique, beyond irresponsible/unintended usage or investment/fondness for existing frameworks/techniques that pre-exist. We have also not profiled it for performance (we only tend to run this once per build/session, etc.) so in your own usage, YMMV.
Most configurations transmitted client-side of course would not want to contain sensitive pre-production values, so one could (should!) use the same function to generate a production-only version (with no annotations) in pre-deploy, while still enjoying a SINGLE configuration file upstream in your process.
Further, if you're doing this for i18n, you may not want the entire wad going over the wire, so could process it server-side (cached or live, etc.) or pre-process it in build/deploy by splitting into separate files, but STILL enjoying a single source of truth as early in your workflow as possible.
We have not explored implementing the same function in Java (or C#, PERL, etc.) assuming it's even possible (with some exotic reflection maybe?) but a build environment that includes NodeJS could farm that step out easily.
Well if it suits your needs and you have no problem of storing the connection strings in the source control repository, you could create files like:
appsettings.dev.json
appsettings.qa.json
appsettings.staging.json
And choose the right one in the deployment script and rename it to the actual appsettings.json, which is then read by your app.