Is there an Integration platform which is open source and free even for a lot of data, for unlimited runs and flows? - rest

From my understanding, Informatica Cloud, Boomi, Talend, JitterBit are all integration tools which have "Connectors" to connect to servers (and I believe these Connectors in turn call APIs to access the required data). I saw many others but none of them are free although some are open source.
Are there any tools that help you visualize the integration process for free? If not, why not?
Tools like Informatica, Boomi provide drag and drop which show the entire flow.

Talend Open Studio is one such tool. not completely visual but almost there.
There are not many free tools as data storage technologies are constantly changing. It would be expensive for developers to keep up with constantly changing technologies without a source of income.

Developing data integration tools are resource intensive. Why would anybody(any enterprise company) spend so much of their effort and give away for free. Also the provider of the tools have to provide support for enterprise level . P1 means 4 hour response which means building the capability of the support team on par with the developer. All of these cost money and time. The only way to recoup is to sell the finished product and provide services.

Related

MongoDB for commercial use

As I am pretty horrible in reading English legal documents I hoped one of you could answer this question.
In about a month I need to do an internship at a company for my bachelor. They would like me to develop a system for internal use (will not be sold) that requires a database.
They are allowing me a free hand (from what I understood) in selecting a database. As (as far as I understand atm) the data that needs to be stored does not contain a lot of relations (1 or 2) and is not heavily queried, I was thinking of using mongoDB as a back-end server.
Can mongoDB community be used freely in this type of an application under the new license? Most I find using Google involves the old license.
First of all, it's important to know why MongoDB adopted a new license for the product Community Server. This change was made as a response to a increasing number of cloud providers that are offering MongoDB database as a paid service to their users without playing by the open-source rules. Indeed, it's pretty unfair to have companies reselling the free version of a product you spent a lot of money to develop without contributing anything back.
As you can read in MongoDB new license's FAQ What specifically is the difference between the GPL and the SSPL:
A company that offers a publicly available MongoDB as a service must release the software it uses to offer such service under the terms of the SSPL, including the management software, user interfaces, application program interfaces, automation software, monitoring software, backup software, storage software and hosting software, all such that a user could run an instance of the service using the source code made available.
That means that a company that offers MongoDB Community Version as a service to their users, must open the source code of the softwares developed to make that service work, like: monitoring tools, user interfaces, etc.
What changes to you: nothing.
Be the software you are developing for internal or external use, your company is just using MongoDB as a component of the project, not as the final product. So you are free to keep using it.

When to use a workflow engine - use and misuse

THE CONTEXT:
I should develop a software to calculate billing for a lot of customers
The software should be used by different local administrations, each one with its own rules to calculate the billing to its citizens.
At first i've thought to a workflow engine in order to "design" different calculation flows and apply then to the customers.
In the past i had a little experience with a workflow manager product (i worked a little with IBM BPM) and i had a lot of difficult to debug what happens when something went wrong and i found a lot of performance issue (respect to a simple OOP software).
Maybe these difficulties went caused by my poor knowledge of the tool, or maybe IBM BPM is not as good as IBM says.
Anyway, respect to my objective (produce a custom billing, and make it as flexible as possible in therm of configuration and process) is a workflow engine a suitable product?
Any suggestion about tools, frameworks and above all how to approach the problem.
My initial idea of the architecture is to develop a main software in c# (where i'm more confident) and using a workflow engine (like JBpm) as a black box, invoking previously configured flows into the bpm.
I would recommend using Cadence Workflow for your use case. It records all events that are related to your workflow in an execution history. It makes troubleshooting of production issues very straightforward.
As workflow is essentially a program in Java (or Go) you have unlimited flexibility on implementation. The performance is also not an issue as Cadence was built from ground up for high scalability. It was tested to over hundred million open workflows and tens of thousands of events per second.
See the presentation that goes over Cadence programming model.

What are the top 5 deployment automation tools?

I am using IBM UrbanCode udeploy, great tool but afraid IBM will kill it sooner or later with their slow response in supporting. What are the other choices?
UCD isn't going anywhere. Its in high demand these days, and is gaining traction in z/OS shops as well.
However, there other popular tools are Chef and Puppet. Just be prepared for manual data entry and less friendly interface. They work, but are more work to use.

Software to consolidate information flows into a company

At our company, we are looking at replacing a number of legacy systems that handle information from our customers into our company. Typical systems allow the user to drop an ftp file somewhere. This file will then be transformed by a number of programs and eventually end up in some kind of database. In total we have +30 different "systems" or applications that does this. And, it is more or less a mess.
We believe we lack a common system to manage these flows: triggered by upload or possible another event, register the data, create some sort of "job" (or process) from it, pass it through the variuos services/transformation programs it needs to go through, provide feedback to the customer, provide information about progress, etc to us, handle failures and so on. Sort of like Jenkins (/Hudson/CruiseControl/similar) but for information transformation jobs, rather than build jobs, and with a job beeing more of a "process instance" of a job, then the job itself (e.g. different data should trigger the job several times, running concurrently).
We are cabable of writing such software ourselves, but surely software as this exists(?) I have been googling around, and found that what we need ma possibly be "job scheduling" software or "business process management" software. However, these are all new domains for us, and I am quite uncertain to as what kind of software would fit our needs. It appears one could invest quite a deal of ressources into this type of software before
So, what I am looking for is pointers to what kind of software or systems that could solve the kind of needs we have. Preferably Open Source, Java based, running in a Java EE container or similar, but really, at this point, almost any pointer/hint will be welcomed :-)
Thanks in advance
P.S. I realise I may be out of scope for Stackexchange, but I have been unable to locate another forum where this kind of question might be answered, so I hope it is OK.
I know of the following products:
Redwood Cronacle (I worked with it 1994-1997 and it still runs). Purchase product. Oracle and C based. Strong in multiple server platforms. Embeddable.
Oracle E-business suite core. Purchase product. Oracle based. Strong for integration with the same ERP system. Weak for multiple server platforms.
Invantive Vision (I developed it :-). Purchase product. Oracle and Java based. Strong in integration with ETL (Pentaho open source). Weak for multiple server platforms. Embeddable.
Quartz Scheduler. Apache license. Java based. Worked with in 2004 or so. Strong focus on embedding.
Hi I don’t know if you will find that solution in open source or Java. It sounds like bespoke or custom software to me. I would advise you to search for a project management software developer with high level of IT and Data warehousing. Ask for bespoke and customized installations with a real time database. I think you will solve your problem with this.

Is automatic upgrades a realistic feature to expect from enterprise Web applications?

Most of the work I do is with what could be considered enterprise Web applications. These projects have large budgets, longer timelines (from 3-12 months), and heavy customizations. Because as developers we have been touting the idea of the Web as the next desktop OS, customers are coming to expect the software running on this "new OS" to react the same as on the desktop. That includes easy to manage automatic upgrades. In other words, "An update is available. Do you want to upgrade?" Is this even a realistic expectation? Can anyone speak from experience on trying to implement this feature?
At my company we have enterprise installations ranging into the thousands of seats. If we implemented an auto-upgrade, our customers would mutiny!
Large installations have peculiar issues that don't apply to small ones. For example, with 2000 users (not all of whom are, let us say, the most sophisticated of tool users), tool-training is a big deal: training time, internal demos, internal process documents, etc.. They cannot unleash a new feature or UI change without a chance to understand how it fits in their process and therefore what their internal best practices are and how to communicate that to their users.
Also when applications fail, it's the internal IT team who are responsible. Therefore, they want time to install a new version in a test area, beat it up, and deploy on a Saturday only when they're good and ready.
I can see the value in making minor patches more easy to install, particularly when the patch is just for a bug-fix and not for anything that would require retraining, and if the admins still get final say over when it's installed. But even then, I don't believe anyone has ever asked for this! Whether because they don't want it or they are trained to not expect it, it doesn't seem worth it.
Well, it really depends on your business model but for a lot of applications the SaaS model can end up biting you. It's great for a lot of things but for some larger applications the users are not investing as significant amount up front and could possibly move to something else before you've made any money.
See
http://news.zdnet.com/2424-9595_22-218408.html
and here
http://www.25hoursaday.com/weblog/2008/07/21/SoftwareAsAServiceWhenYourBusinessModelBecomesAParadox.aspx
for more information
One of the primary reasons to implement an application as a web application is that you get automatic upgrades for free. Why would users be getting prompted for upgrades on a web app?
For Windows applications, the "update is available, do you want to upgrade?" functionality is provided by Microsoft using ClickOnce, which I have used in an enterprise environment successfully -- there are a few gotchas but for the most part it is a good way to manage automatic deployment and upgrade of Windows apps.
For mobile apps, you can also implement auto-upgrades, although it is a little trickier.
In any case, to answer your question in a broad sense, I don't know if it is expected that all enterprise apps should make upgrading easy, but it certainly is worth the money from an IT support standpoint to architect them to allow for easy upgrading.
If you're providing a hosted solution, I wouldn't bother. Let the upgrade happen silently (perhaps with a notice that you did it). If you're selling an application that's hosted on their servers, let the upgrade decision be made by a single owner, not every user of the app.