As of Dec 2020 in the Collibra Marketplace many tools are showing an alert as to Connect... what would cause this and what impact does it have for Collibra developers? Can't seem to find further details as to the reasons, only signposts to documentation describing how to use the preferred API-based methods:
We have made the decision to transition away from Collibra Connect so
that we can better serve you and ensure you can use future product
functionality without re-instrumenting or rebuilding integrations. For
more information, please reach out to your Customer Success Manager.
Learn more about different methods to build integrations in Collibra
Developer Portal.
This example for Talend Data Lineage, just one of the many Connect tools:
Collibra's long-term direction is to transition away from Collibra Connect, which is an OEM of Mule ESB. Collibra provides native connectors to numerous source systems that vastly simplify integration and ingestion. The connectors not only ingest metadata but also provide additional value-added features such as data profiling, sampling and data classification, with more features to come in the future. Customers can also bring their ESB of choice, including Mulesoft, and work with Collibra's APIs to build integrations. For customers who choose to keep and use Mulesoft as their ESB, they will need to license directly with Mulesoft and can continue to use any integrations that they developed with Collibra Connect. Additionally, we are providing the Collibra Mulesoft Connector free of charge, so customers can continue to use it to build and maintain Mulesoft integrations going forward.
Related
Is it possible to scale Axon Framework without Axon Server Enterprise? I'm interested in creating a prototype CQRS app with Axon, but the final, deployable system has to be be free from licensing fees. If Axon Framework can't be scaled to half a dozen nodes using free software, then I should probably look elsewhere.
If Axon Framework turn out not to be a good choice for the system, what would you recommend? Would building something around Apache Pulsar be a sensible alternative?
I think I have good news for you then.
You can utilize Axon Framework perfectly fine without Axon Server Enterprise.
Firstly, you can use the Axon Server Standard edition, which is completely free and you can check out the code too if you want.
If you prefer to get infrastructure back in your own hands, you can also select different approaches to distributing the CommandBus and the EventBus/EventStore.
For the CommandBus the framework provides the DistributedCommandBus for which two implementation are in place, being:
JGroups
Spring Cloud
I'd argue option 2 is the most ideal for distributing your commands, as it gives you the freedom to choose whichever Spring Cloud Discovery Service implementation you desire. That should give you the handles to work "free of licenses" in that area.
For distributing your Events, you can broadly look at two approaches:
Share the database, aka your EventStore, among all instances
Use a event message bus to distribute your event messages
If you want instances of your nodes to be able to Event Source the Command Model, you are inclined to use option 1. This is required as Axon Framework requires a dedicated EventStore to be able to source the Command Models from history.
When you just want to connect other applications to your Event Stream, option 2 would be a good fit. Again, the framework has two options in this area:
AMQP
Kafka
The only thing I'd like to point out on this part additionally is that the Kafka extension is still in a release candidate state. It is being worked on actively at the moment though.
All these extensions and their options should be clearly stated in the Reference Guide, so I'd definitely check this documentation out if you are gonna start an application.
There is a sole thing you cannot distribute though, which is the QueryBus.
There is an outstanding issue to resolve this, for which someone put the effort in to provide a PR.
After seeing how Axon Server Standard edition does it though, he intentionally closed the PR (with the following comment) as it didn't seem feasible to him to maintain such a tool at this stage.
So, can you use Axon Framework without Axon Server Enterprise?
Well, I think you can. :-)
Mind you though, although you'd be winning on not having a license fee if you don't use Axon Server Enterprise, it doesn't mean your production is gonna be free.
You'd be introducing back quite some infrastructure set up and production time to get this going.
Hope this gives you plenty of feedback #ahoffer!
From my understanding, Informatica Cloud, Boomi, Talend, JitterBit are all integration tools which have "Connectors" to connect to servers (and I believe these Connectors in turn call APIs to access the required data). I saw many others but none of them are free although some are open source.
Are there any tools that help you visualize the integration process for free? If not, why not?
Tools like Informatica, Boomi provide drag and drop which show the entire flow.
Talend Open Studio is one such tool. not completely visual but almost there.
There are not many free tools as data storage technologies are constantly changing. It would be expensive for developers to keep up with constantly changing technologies without a source of income.
Developing data integration tools are resource intensive. Why would anybody(any enterprise company) spend so much of their effort and give away for free. Also the provider of the tools have to provide support for enterprise level . P1 means 4 hour response which means building the capability of the support team on par with the developer. All of these cost money and time. The only way to recoup is to sell the finished product and provide services.
My situation is that the Microsoft IIS app server and code in C# already exists.
The Web Services and contracts have been done in the .NET framework. My question is what open source Enterprise Service Bus is available to register the endpoint for sending messages to/from the services on the IIS app server? Can I have a Java-based ESB when my endpoint is written in a different language, C#?
I'm looking for an open-source ESB where I can deploy existing WSDL to register the Microsoft server endpoint and wondering if a Java-based ESB will work? What kind of issues would creep up? Is it better to match endpoint vendor type with esb vendor type?
Warewolf ESB is a GPL Licensed Open Source ESB based on the Microsoft Stack. You can download an installer from http://warewolf.io, or the source code from Github at https://github.com/Warewolf-ESB/Warewolf-ESB
Its very easy to work with, and is highly extensible. The development team are also very friendly and are quick to respond to the communities queries.
Since nobody else has chimed in, I guess I'll give it a go.
You'll see that most of the open-source service buses in .NET (NServiceBus, MassTransit, etc) stay pretty far away from WSDL - instead preferring to take a more message-centric approach.
Full disclosure, I'm deeply involved with NServiceBus.
Java-based ESBs tend to be more open to integrating more web-service-centric approaches.
The tradeoffs associated with introducing non-Microsoft technology in your production environment center around your operations team's familiarity with them. There's also the development side of things, but in my experience, that tends to be more minor in comparison.
Hope that helps in some small way.
We manage a multi-locale, multi-language Pivotal CRM System with developers spread across UK and India. We do not have any Source Control System to manage the development work. Is there any source control system that we can integrate with Pivotal CRM.
Note: "Please check with the providers CDC Software" is not a valid answer :-)
I worked on Pivotal CRM for 6 years, and stopped asking CDC for source control. Their last response was that source control was a highly requested feature, but that they have no intent in making Pivotal CRM development work with any source control system.
I don't want to sound too jaded, but if you treat Pivotal CRM as a vertical solution that needs only the minimum of changes you will do well. If you are treating it as a framework to house lots of customizations/programming to implement unique business requirements, you will run into nothing but roadblocks and un-necessary challenges. Use SugarCRM, Microsoft Dynamics, or Salesforce.com instead.
we (a team of about 150) are considering moving our ALM solution from Bugzilla/CVS to Jira/svn/Confluence/Bamboo/Fisheye. SO has a lot of good info on those, but I would be interested to learn about another tool from Atlassian - a Single Sign On (SSO) Crowd, I am considering adding it to the mix for an LDAP integration with our Novell id's.
has someone had any experience with Crowd?
how does it handle 100/200/500 (after recession, that is) users?
any tips/tricks?
would you choose different, open source SSO solutions?
thanks
EDIT:
a year has passed...
We got Crowd and went with ActiveDirectory integration along with internal Crowd directory (for short-term contractors, etc.). So far the solution works just great.
EDIT2:
Another year: still going strong (We have 1K users now). Nested groups is a killer feature, thankfully it is working fine after last point release.
EDIT3:
mid-2012 - 7.5K users - going strong. with a little automation for onboarding (Confluence pages with Ajaxified forms + a little Crowd plugin)
Major disclosure: I'm the Crowd Product Manager. So, apply as much NaCl as you think wise.
I'd be very surprised if you had any issues with 500 users. Especially since Novell seems to be one of the better directory servers in terms of performance. The only time I'd expect to see problems is if your Crowd server and Novell directory server are on opposite sides of the world. Don't do that unless you have to :-)
We have plenty of users connecting thousands of users to JIRA, Confluence, and the Dev Tools with Crowd.
Any issues - drop us a line (sales#atlassian.com or http://support.atlassian.com) and we'll help out.
Cheers,
Dave.
ps: I hope that didn't come off as a sales pitch or "we make magic products that are perfect in every possible way, now give us your money!"
We're using Crowd with about 80 users and expect that number to climb into the hundred when we roll it out for client access. Crowd is important to us because it allows us to integrate Jira and Confluence (the Atlassian wiki) with SSO, which is critical.
Crowd works well for us but it does have some quirks. We are using it to draw authentications from Active Directory. There are some things that are a little inelegant. We need to do some more digging to troubleshoot those.
But that aside, Crowd is a big win for us, for these two reasons:
SSO across Atlassian apps
Ability to have our internal users drawn from Active Directory, and add clients directly to Crowd and not bog down AD
We're very happy with all the Atlassian tools.
I haven't had experience with Crowd on such a large set of users as yours, but I did find it very easy to set up and manage our JIRA, Confluence and SVN instances with Crowd (we only have 25 users). It will handle Apache authentication as well, so I'm planning to switch our various authenticated Web sites to Crowd as well.
According to Atlassian's site, Crowd should easily be able to handle 500 users; there are some useful case studies and Webinar recordings on the site that will tell you more.
I do have few installations of Crowd with over 16000 users, most comming from LDAP/Active Directory and I would say that the performance would not be a problem but there are other problems which Atlassian did considered solving in years:
There is no auto account creation/registration in crowd
None of the Atlassian products allows people to register accounts with an email validation
There is no way to prevent people from creating several accounts with the same email address.
SSO works only if you have only one domain.
If you do no have many users you can configure Confluence to coonect to Jira directly instead of using Crowd. Atlassian products do already have an interal crowd instance in them, but its performance is limited to about 200 users or so (it's more about the number of authentications made, not the total number of users).
Considering the above limitations, I would summarize that Crowd is far overpriced for what it delivers, unless you are getting a free license if you are eligible.
We have also Crowd installed and connected within the Atlassian product family. It is backed by a corporate LDAP (M$ AD). So far it is great and works pretty well.
BUT currently we're struggling with integration of so called custom applications. We have e.g. Prometheus for monitoring data which doesn't have any authentication built in. So we have an Apache 2.4 in front as SSL endpoint. To add authentication we considered integrating it with Crowd. There is a Apache Crowd connector that is no longer supported (which would be fine by me). There are only the sources available, but built on Apache 2.2. We have to use Apache 2.4 (corporate policy) where some of the required API has been removed.
So either we invest considerable amount of time to migrate the Connector to current Apache API or we do something else (like using a generic LDAP connector towards AD). Which makes the whole Crowd idea a bit a two sided sword for us. (We wanted to centralize user management within our project into a single tool like Crowd to get rid of corporate processes and regulations on the central LDAP).
UPDATE: We now use https://github.com/fgimian/cwdapache connector for Apache 2.4 (with slight adaptions it can be built for Ubuntu 16.04). This adds support for Apache Basic Auth with Crowd groups/users.
UDAPTE2: Bitbucket, Jira, Confluence, Crucible work out of the box of course. User migration is a bit cumbersome though (renaming old users and then integrate with Crowd or use unsupported SQLs).
Jenkins 2 and Nexus 3 seem to work fine.
FURTHER DOWN THE ROAD:
Right now I am considering Crowd as a centralized tool for identity and access management for Atlassian products. There it works fine and does what it should. Integrating numerous other applications just sucks since available integrations are not supported/updated.
Example: if you want to have Crowd authentication with nginx there is nothing usable available. There is a OpenId Connect module available, but Crowd lacks support for that (they only support outdated OpenId v2.0). Not even talking about OAuth. There is a Atlassian OAuth library, but Crowd doesn't have it yet (or will ever). Even the Google Apps support will vanish, since Google dropped support: https://developers.google.com/identity/protocols/OpenID2Migration