My current project involves using LDAP (Active Directory) and I'm using issue tracking for all of my projects, so the idea of combining both of them crossed my mind. In order to fit the requirements of StackOverflow I'll try to formulate this as question but I admit, this is more about just getting some opinions, please forgive me :):
I think that issue-tracking and SCM (software configuration management) in general would be a good application for LDAP because of the following reasons:
Easy to integrate into existing infrastructure (no need for additional user management)
Fine-grained access control for projects/issues etc.
Ready-To-Use hierarchical, property-oriented storage (which is typically needed for SCM/issue trackers)
Standard-API with bindings for almost all languages/technologies
Searching/Indexing, Backup/replication functionality already present in most LDAP solutions
Extensible schema already part of the LDAP technology (it would be easy to add properties to issues/projects etc.)
So my questions are:
Are you aware of any existing attempts to define a (standard) schema for issue-tracking resp. SCM (i.e. class definitions for issues, projects, versions, releases, revisions etc)
LDAP usually manage relatively slowly-changing data. How well would current implementations (OpenLDAP, ActiveDirectory) handle data (mainly in terms of performance and amount of data) that typically changes very frequently?
Are there any other drawbacks of such a solution you can think of?
and of course
Who would like to try to start such a project :) ...
The OP precises:
The question is not about using an existing issue tracker with LDAP authentication (redmine can do this for example),
but about storing tickets/issues/etc. directly within the LDAP tree...
Currently, each issue tracker has it's own API for accessing data, having all data accesible via LDAP could make writing tools (e.g. integration into IDEs etc.) much easier
To which the answer is easy.
Don't.
LDAP is not (repeat, not) made for that, and there is much more to an SCM or an Issue Tracker than just a bunch of hierarchical data.
An SCM has to come up with a way to store/reference efficiently deltas, entire tree, branches, labels.
an Issue Tracker is all about multiple relationship between one item and several other (several parents/children, related, duplicated, ...), plus has to manage somehow a tight reference with the code (or rather the changeset, set of version modified)
While it is true than by adding a all lot of new objectClass types, you could end up with a similar structure, you would essentially take what it is a Lightweight Directory (ie optimized for reading only) and transform it into a huge referential (with lots of read/write operations and complex data structures).
If you are looking about an unifying API, one generic one (not just for SCM or Bug Tracking) is OSLC (Open Services for Lifecycle Collaboration), an open-sourced protocol currently used for Change Management by RTC (Rational Team Concert).
Related
Im building a toolbox for a certain branch of biology. One of the reasons Julia was chosen is its simplicity, as biologists wont be assumed to be able to write complex C-code
What I'd like to add is a way for users to share their own custom methods for others to review/verify/use, both to promote collaboration and to add a bit of sense of community
What Im sure of is that this specific demography of (mostly) biologist wont be able or have patience to fork a github project or anything that could be considered remotely complex, especially when it wont benefit them explicitly to do so
So, what I'd like to do is provide the simplest of interfaces, with the add/view options to either add a routine or view routines (along with descriptions, ratings etc)
I can only think of two ways to accomplish storing the scripts pushed by users, by having them on a server, or, more simply, using SQL
tl;dr can postgresql store scripts or is that a terrible idea
I ask, mainly because there will be 'raw data' available on a postgresql server, and I'd like to be able to keep that and the 'community methods' both in the same place for convenience sake
To summarize the discussion in the comments to this question:
Version control is an excellent solution to sharing control, but from a scientist's perspective, it can be difficult and complicated. Luckily, GitHub now offers a GUI that is easy to learn and yet retains a lot of the power of Git. For instance, GitHub allows one to edit files directly from the web UI.
Introduction
Drools Guvnor has it's own versioning system, that in production use allows the users of an application to modify the rules and decision tables in order to adapt to change in their business. Yet, the same assets continue to live on the development version control system, where new features to the app are developed.
This post is for looking insight/ideas/experience on rule development and deployment when working with Drools rules and Guvnor.
Below are some key concepts I've been puzzling about.
Deployment to Guvnor
First of all, what is the best way to deploy the drl files and decision tables to production environment? Just simply put them on a zip package and then unzip to Web-Dav folder? What I have navigated around Drools, I haven't found a way to import more than one file at a time. The fact model can be added as a jar archive, though. Guvnor seems to have a REST API of some sort, but using that would require custom deployment scripts.
Change management
Secondly, once the application is in production, the users will likely want to change the values in decision tables in order to set the discount percentages to higher for premium clients etc. This is all fine and dandy, until comes the time to start development of version 2.0 of the app.
Now what we have at this point is
drl files and decision tables in version controlling system
drl files and decision tables in production environment with user modifications, versioned by Guvnor
Now we are in the point of getting the rules and decision tables back from the Guvnor. And again is the Web-Dav folder the best for this, what other options there are?
Merge tools today can even handle Excel file diffs, but sounds like a merge hell to me on a big scale projects.
Keeping the fact model backwards compatible
Yet another topic is fact model integrity. For the assumed version 2.0, developers always want to make refactoring and tear the whole fact model upside down. Still, it must remain backwards compatible with the previous versions as there may be user modified rules that depend on that. Any tips on this? Just keep the fact model simple and clean? Plan ahead / suggest what the users could want to change?
Summary
I'm certain I'm not the first, and surely not the last, to consider options on deployment and change management with Drools and Guvnor. So, what I'd like to hear is comment, discussion, tips etc. on some best (and also the worst in order to avoid them) practices to handle these situations.
Thanks.
The best way to do things depends very much on your specific application and the environment you work in. However the following are pointers from my own experience. Note that I'll add just a few points for now. I'll probably come back to this answer when things come to me.
After the initial go-live, keep releases incremental and small
For your first release you have the opportunity to try things out. Take advantage of this opportunity, and do as much refactoring as possible, because...
Your application has gone live and your business users are maintaining rules in decision tables. You have made great gains in what folks in the industry like to call "business agility". Unfortunately, this tends to be at the expense of application development agility. All of your guided editor rules and decision table rules are tied to your fact model, so any changes existing properties of that fact model will break your decision tables. Unlike in most IDEs these days, you can't just right-click on a fact's getX() method, rename it, and expect all code which relies on that property to be updated.
Decision tables and guided rules are hard to refactor. If a fact has been renamed, then in many (all?) versions of Guvnor, that rule/table will no longer open. You need to get at the underlying XML file via WebDav and do some text searching and replacing. That may be very difficult, considering that to do this, you need to download the file from production to a test environment, make changes, test them, deploy them to a test environment. When you're happy with your changes you need to push them back up to the 'production' Guvnor. Unfortunately, while you were doing that, the users have updated a number of decision tables and you need to either start again, or re-apply the past couple of days' changes. In an ideal world, your business users will agree to make no rule changes for a period of time. But the only way to make that feasible is to keep the changes small so that you can make them in a couple of hours or a day depending on how generous they feel.
To mitigate this:
Keep facts used within Guvnor separate from your application domain classes. Your application developers can then refactor the internal application model to their hearts content, but such changes will not affect the business model. Maintain mappings between the two and ensure there are tests covering those mappings.
Avoid changes such as renaming facts or their properties. Make sure that facts you create and their properties have names which suit the domain and agree these with the business. On the other hand, adding a new property is relatively painless. It is well worth prompting the users to give you an eye on their future plans.
Keep facts as simple as possible. Don't go more complex than name-value pairs unless you really need to. For one thing, your business users will find it much easier to maintain rules. Your priority with anything managed within Guvnor should always be about making it easy for business users to maintain.
Keep external dependencies out of your facts. A developer may think it's a good idea to annotate a fact as a JPA #Entity, for easy persistence. Unfortunately, that adds dependencies which need to be added to Guvnor's classpath, requiring a restart.
Tips & tricks
My personal technique for making cross-environment changes is to connect Eclipse to two Guvnor WebDav directories, and checkout the rules into local directories, where each local directory maps to an environment. I then use the Eclipse diff tooling.
When building a Guvnor-managed knowledge base, I create a separate Maven project containing only the facts, and with no dependencies on anything else. It makes it a lot easier to keep them clean this way. Also, when I really do need to add a dependency (i.e. I use JodaTime where possible), then the build can have a step to generate a shaded JAR containing all the dependencies. That way you only ever deploy one JAR to Guvnor, which is guaranteed to contain the correct versions of your dependencies.
I'm sure there will be more that I think of. I'll try to remember to come back to this...
We're an SME with SAP implemented. We're trying to use the transactional data in SAP to build another system in PHP for our trucking division for graphical reports, etc. This is because we don't have in-house expertise ABAP development and any SAP modifications are expensive.
Presently, I've managed to achieve our objectives with read-only access to our Quality DB2 server and any writes go to another DB2 server. We've found the CPU usage on the SELECT statements to be acceptable and the user is granted access only to specific tables/views.
SAP's Quality DB2 -> PHP -> Different DB2 client
Would like your opinion on whether it is safe to read from production the same way? Implementing all of this again via the RFC connector seems very painful. Master-Slave config is an option for us but again will involve external consultancy.
EDIT
Forgot to mention that our SAP guys don't want to build even reports for another 6-months - they want to leave the system intact. Which is why we're building this in PHP on the top.
If you don't have ABAP expertise, get it - it's not that hard, and you'll get a lot of stuff "for granted" (as in "provided by the platform") that you'll have to implement manually otherwise - like user authentication and authority management and software logistics (moving stuff from the development to the production repository). See these articles for a short (although biased) introduction. If you still need an external PHP application, fine - but you really should give ABAP a try first. For web applications, you might want to look into Web Dynpro ABAP. Using the IGS built'in chart engine with the BusinessGraphics element, you'll get a ton of the most custom chart types for free. You can also integrate PDF forms created with Adobe Livecycle Designer.
Second, while "any SAP modifications are expensive" might be a good approach, what you're suggesting isn't a modification. That's add-on development, and it's neither expensive nor more complex than any other programming language and/or environment out there. If you can't or don't want to implement your own application entirely using the existing infrastructure, at least use a decent interface - web services, RFC, whatever. From an ABAP point of view, RFC is always the easiest option, but you can use SOAP or REST as well, although you'll have to implement the latter manually. It's not that hard either.
NEVER EVER access the SAP database directly. Just don't. You'll have to implement all the constraints like client dependency or checks for validity dates and cancellation flags for yourself - that's hardly less complex than writing a decent interface, and it's prone to break every time the structure is changed. And if at some point you need to read some of the more complex contents like long texts, you're screwed - period. Not to mention that most internal or external auditors (if that happens to be an issue with your company and/or legal requirements) don't like direct database access to a system as critical as this one, which again can cause lots of trouble from people you really don't want to mess with. It's just not worth it.
we are creating hospital information system software. The project will be different hospital to hospital and contain different use cases. But lots of parts will be the same. So we will use branching mechanism of the source control. If we find a bug in one hospital, how can we know the other branches have the same bug or not.
IMAGE http://img85.imageshack.us/img85/5074/version.png
The numbers in the picture which we attached show the each hospital software.
Do you have a solution about this problem ?
Which source control(SVN,Git,Hg) we will be suitable about this problem ?
Thank you.!
Ok, this is not really a VCS question, this is first and foremost and architectural problem - how do you structure and build your application(s) in such a way that you can deliver the specific use cases to each hospital as required whilst being able, as you suggest, to fix bugs in commom code.
I think one can state with some certainty that if you follow the model suggested in your image you aren't going to be able to do so consistently and effectively. What you will end up with is a number of different, discrete, applications that have to be maintained separately even though they have at some point come from a common set of code.
Its hard to make more than good generalisations but the way I would think about this would be something along the following lines:
Firstly you need a core application (or set of applications or set of application libraries) these will form the basis of any delivered system and therefore a single maintained set of code (although this core may itself include external libraries).
You then have a number of options for your customised applications (the per hospital instance) you can define the available functionality a number of means:
At one extreme, by configuration - having one application containing all the code and effectively switching things on and off on a per instance basis.
At the other extreme by having an application per hospital that substantially comprises the core code with customisation.
However the likelyhood is that whilst the sum of the use cases for each hospital is different individual use cases will be common across a number of instances so you need to aim for a modular system i.e. something that starts with the common core and that can be extended and configured by composition as much as by any other means.
This means that you are probably want to make extensive use of Inversion of Control and Dependency Injection to give you flexibility within a common framework. You want to look at extensibility frameworks (I do .NET so I'd be looking at the Managed Extensibility Framework - MEF) that allow you to go some way towards "assembling" an application at runtime rather than at compile time.
You also need to pay attention to how you're going to deploy - especially how you're going to update - your applications and at this point you're right you're going to need to have both your version control and you build environment right.
Once you know how you're going build your application then you can look at your version control system - #VonC is spot on when he says that the key feature is to be able to include code from shared projects into multiple deliverable projects.
If it were me, now, I'd probably have a core (which will probably itself be multiple projects/solutions) and then one project/solution per hospital but I would be aiming to have as little code as possible in the per hospital projects - ideally just enough framework to define the instance specific configuration and UI customisation
As to which to use... if you're a Microsoft shop then take a good long hard look at TFS, the productivity gains from having a well integrated environment can be considerable.
Otherwise (and in any case), DVCS (Mercurial, Git, Bazaar, etc) seem to me to be gaining an edge on the more traditional systems as they mature. I think SVN is an excellent tool (I use it and it works), and I think that you need a central repository for this kind of development - not least because you need somewhere that triggers your Continuous Integration Server - however you can achieve the same thing with DVCS and the ability to do frequent local, incremental, commits without "breaking the build" and the flexibility that DVCS gives you means that if you have a choice now then that is almost certainly the way to go (but you do need to ensure that you establish good practices in ensuring that code is pushed to your core repositories early)
I think there is still a lot to address purely from the VCS question - but you can't get to that in useful detail 'til you know how you're going to structure your delivered solution.
All of those VCS (Version Control System) you mention are compatible with the notion of "shared component", which allows you to define a common shared and deployed code base, plus some specialization in each branch:
CVCS (Centralized)
Subversion externals
DVCS (Distributed)
Git submodules (see true nature of submodules)
Hg SubRepos
Considering the distributed aspect of the release management process, a DVCS would be more appropriate.
If the bug is located in the common code base, you can quickly see in the other branches if:
what exact version of the common component they are referring to.
they refer the same or older version of that common component than the one in which the bug has been found (in which case chances are they also do have the bug)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
We have written a software package for a particular niche industry. This package has been pretty successful, to the extent that we have signed up several different clients in the industry, who use us as a hosted solution provider, and many others are knocking on our doors. If we achieve the kind of success that we're aiming for, we will have literally hundreds of clients, each with their own web site hosted on our servers.
Trouble is, each client comes in with their own little customizations and tweaks that they need for their own local circumstances and conditions, often (but not always) based on local state or even county legislation or bureaucracy. So while probably 90-95% of the system is the same across all clients, we're going to have to build and support these little customizations.
Moreover, the system is still very much a work in progress. There are enhancements and bug fixes happening continually on the core system that need to be applied across all clients.
We are writing code in .NET (ASP, C#), MS-SQL 2005 is our DB server, and we're using SourceGear Vault as our source control system. I have worked with branching in Vault before, and it's great if you only need to keep 2 or 3 branches synchronized - but we're looking at maintaining hundreds of branches, which is just unthinkable.
My question is: How do you recommend we manage all this?
I expect answers will be addressing things like object architecture, web server architecture, source control management, developer teams etc. I have a few ideas of my own, but I have no real experience in managing something like this, and I'd really appreciate hearing from people who have done this sort of thing before.
Thanks!
I would recommend against maintaining separate code branches per customer. This is a nightmare to maintain working code against your Core.
I do recommend you do implement the Strategy Pattern and cover your "customer customizations" with automated tests (e.g. Unit & Functional) whenever you are changing your Core.
UPDATE:
I recommend that before you get too many customers, you need to establish a system of creating and updating each of their websites. How involved you get is going to be balanced by your current revenue stream of course, but you should have an end in mind.
For example, when you just signed up Customer X (hopefully all via the web), their website will be created in XX minutes and send the customer an email stating it's ready.
You definitely want to setup a Continuous Integration (CI) environment. TeamCity is a great tool, and free.
With this in place, you'll be able to check your updates in a staging environment and can then apply those patches across your production instances.
Bottom Line: Once you get over a handful of customers, you need to start thinking about automating your operations and your deployment as yet another application to itself.
UPDATE: This post highlights the negative effects of branching per customer.
Our software has very similar requirements and I've picked up a few things over the years.
First of all, such customizations will cost you both in the short and long-term. If you have control over it, place some checks and balances such that sales & marketing do not over-zealously sell customizations.
I agree with the other posters that say NOT to use source control to manage this. It should be built into the project architecture wherever possible. When I first began working for my current employer, source control was being used for this and it quickly became a nightmare.
We use a separate database for each client, mainly because for many of our clients, the law or the client themselves require it due to privacy concerns, etc...
I would say that the business logic differences have probably been the least difficult part of the experience for us (your mileage may vary depending on the nature of the customizations required). For us, most variations in business logic can be broken down into a set of configuration values which we store in an xml file that is modified upon deployment (if machine specific) or stored in a client-specific folder and kept in source control (explained below). The business logic obtains these values at runtime and adjusts its execution appropriately. You can use this in concert with various strategy and factory patterns as well -- config fields can contain names of strategies etc... . Also, unit testing can be used to verify that you haven't broken things for other clients when you make changes. Currently, adding most new clients to the system involves simply mixing/matching the appropriate config values (as far as business logic is concerned).
More of a problem for us is managing the content of the site itself including the pages/style sheets/text strings/images, all of which our clients often want customized. The current approach that I've taken for this is to create a folder tree for each client that mirrors the main site - this tree is rooted at a folder named "custom" that is located in the main site folder and deployed with the site. Content placed in the client-specific set of folders either overrides or merges with the default content (depending on file type). At runtime the correct file is chosen based on the current context (user, language, etc...). The site can be made to serve multiple clients this way. Efficiency may also be a concern - you can use caching, etc... to make it faster (I use a custom VirtualPathProvider). The largest problem we run into is the burden of visually testing all of these pages when we need to make changes. Basically, to be 100% sure you haven't broken something in a client's custom setup when you have changed a shared stylesheet, image, etc... you would have to visually inspect every single page after any significant design change. I've developed some "feel" over time as to what changes can be comfortably made without breaking things, but it's still not a foolproof system by any means.
In my case I also have no control other than offering my opinion over which visual/code customizations are sold so MANY more of them than I would like have been sold and implemented.
This is not something that you want to solve with source control management, but within the architecture of your application.
I would come up with some sort of plugin like architecture. Which plugins to use for which website would then become a configuration issue and not a source control issue.
This allows you to use branches, etc. for the stuff that they are intended for: parallel development of code between (or maybe even over) releases. Each plugin becomes a seperate project (or subproject) within your source code system. This also allows you to combine all plugins and your main application into one visual studio solution to help with dependency analisys etc.
Loosely coupling the various components in your application is the best way to go.
As mention before, source control does not sound like a good solution for your problem. To me it sounds that is better yo have a single code base using a multi-tenant architecture. This way you get a lot of benefits in terms of managing your application, load on the service, scalability, etc.
Our product using this approach and what we have is some (a lot) of core functionality that is the same for all clients, custom modules that are used by one or more clients and at the core a the "customization" is a simple workflow engine that uses different workflows for different clients, so each clients gets the core functionality, its own workflow(s) and some extended set of modules that are either client specific or generalized for more that one client.
Here's something to get you started on multi-tenancy architecture:
Multi-Tenant Data Architecture
SaaS database tenancy patterns
Without more info, such as types of client specific customization, one can only guess how deep or superficial the changes are. Some simple/standard approaches to consider:
If you can keep a central config specifying the uniqueness from client to client
If you can centralize the business rules to one class or group of classes
If you can store the business rules in the database and pull out based on client
If the business rules can all be DB/SQL based (each client having their own DB
Overall hard coding differences based on client name/id is very problematic, keeping different code bases per client is costly (think of the complete testing/retesting time required for the 90% that doesn't change)...I think more info is required to properly answer (give some specifics)
Layer the application. One of those layers contains customizations and should be able to be pulled out at any time without affect on the rest of the system. Application- and DB-level "triggers" (quoted because they may or many not employ actual DB triggers) that call customer-specific code or are parametrized with customer keys) are very helpful.
Core should never be customized, but you must layer it in somewhere, even if it is simplistic web filtering.
What we have is a a core datbase that has the functionality that all clients get. Then each client has a separate database that contains the customizations for that client. This is expensive in terms of maintenance. The other problem is that when two clients ask for a simliar functionality, it is often done differnetly by the two separate teams. There is currently little done to share custiomizations between clients and make common ones become part of the core application. Each client has their own application portal, so we don't have the worry about a change to one client affecting some other client.
Right now we are looking at changing to a process using a rules engine, but there is some concern that the perfomance won't be there for the number of records we need to be able to process. However, in your circumstances, this might be a viable alternative.
I've used some applications that offered the following customizations:
Web pages were configurable - we could drag fields out of view, position them where we wanted with our own name for the field label.
Add our own views or stored procedures and use them in: data grids (along with an update proc) and reports. Each client would need their own database.
Custom mapping of Excel files to import data into system.
Add our own calculated fields.
Ability to run custom scripts on forms during various events.
Identify our own custom fields.
If you clients are larger companies, you're almost going to need your own SDK, API's, etc.