We are trying to choose an approach we should follow in integrating on-promise Dynamic CRM with other companies and third-party software and systems, we use in our company.
Concerning this msdn page: https://msdn.microsoft.com/en-us/library/dn932127.aspx we have some options. But my team members want to implement integration through service calls to external systems inside plugins.
I am talking about Data Integration and Data Synchronization for a total solution that Dynamics CRM is a part of it. Considering the limitations in plugin and the options written in the mentioned msdn page, I think bringing the integration code and logic out of the CRM stuffs and encapsulate it in into a separate connector solution would be a better solution. The connector of course
should support real-time and offline or on-demand data synchronization.
I want to know what are the disadvantages/advantages in plugins approach in integration with external systems, and what are advantages/disadvantages of using a connector solution?
Thanks
Implementing integration to third party system within plugins can be a good approach. You should be aware of how plugins work and how the result of your integration affects the success or failure of the database action that triggers the plugin. Some important items to consider:
Is the database action dependent upon a result from the integration. For instance if your integration is verifying/modifying your data, you may want the result of the integration to affect what is written to the database. Such a plugin would probably be registered on a pre plugin.
Should the database action go through if the integration fails? Many integration are done in a post plugin outside of the transaction so that if the integration fails it does not cause the database action to fail.
This really sounds like something you might want to solve using tooling outside of CRM. For example Scribe, SSIS (with Kingswaysoft) or some ESB type application.
Especially the first two provide no-code solutions and would probably have a lower TCO than writing your own plugins.
Related
How is temporal.io related to cadenceworkflow.io? What should be used if starting a new project depending on the cadence workflow service?
Disclaimer: I'm the original co-founder and tech lead of the Cadence project and currently co-founder/CEO of the Temporal Technologies.
temporal.io is the fork of the Cadence project by the original founders and tech leads of the Cadence project Maxim Fateev and Samar Abbas. The fork is fully open source under the same MIT (with some SDKs under Apache 2.0) license as Cadence. We started Temporal Technologies and received VC funding as we believe that the programming model that we pioneered through AWS Simple Workflow, Durable Task Framework and the Cadence project has potential which goes far beyond a single company. Having a commercial entity to drive the project forward is essential for the longevity of the project.
The temporal.io fork has all the features of Cadence as it constantly merges from it. It also implemented multiple new features.
Here are some of the technical differences between Cadence and Temporal as of initial release of the Temporal fork.
All thrift structures are replaced by protobuf ones
All public APIs of Cadence rely on Thrift. Thrift object are also stored in DB in serialized form.
Temporal converted all these structures to Protocol Buffers. This includes objects stored in the DB.
Communication protocol switched from TChannel to gRPC
Cadence relies on TChannel which was TCP based multiplexing protocol which was developed at Uber. TChannel has a lot of limitations like not supporting any security and having a very limited number of language bindings. It is essentially deprecated even at Uber.
Temporal uses gRPC for all interprocess communication.
TLS Support
Cadence doesn't support any communication security as it is a limitation of TChannel.
Temporal has support for mutual TLS and is going to support more advanced authentication and authorization features in the future.
Simplified configuration
Temporal has reworked the service configuration. Some of the most confusing parts of it are removed. For example, the need to configure membership seeds is eliminated. In temporal each host upon startup registers itself with the database and uses the list from the database as the seed list.
Release pipelines
Cadence doesn't test any publicly released artifacts including docker images as its internal release pipeline is ensuring the quality of the internally built artifacts only. It also doesn't perform any release testing for dependencies that are not used within Uber. For example, MySQL integration is not tested beyond rather incomplete unit tests. The same applies to the CLI and other components.
Temporal is making heavy investment into the release process. All the artifacts including a full supported matrix of dependencies are going to be subjected through a full release pipeline which is going to include multi-day stress runs.
The other important part of the release process is the ability to generate patches for production issues. The ability to ensure quality of such patches and produce all the necessary artifacts in a timely manner is important for anyone running Temporal in production.
Payload Metadata
Cadence stores activity inputs and outputs and other payloads as binary blobs without any associated metadata.
Temporal allows associating metadata with every payload. It enables features like dynamically pluggable serialization mechanisms, seamless compression, and encryption.
Failure Propagation
In Cadence activity and workflow failures are modeled as a single binary payload and a string reason field. Only Java client supports chaining exceptions across workflow and activity boundaries. But this chaining relies on fragile GSON serialization and doesn't work with other languages.
Temporal activity and workflow failures are modeled as protobufs and can be chained across components implemented in different SDKs. For example, a single failure trace can contain a chain that is caused by an exception that originates in activity written in Python, propagated through Go child workflow up to Java workflow, and later to the client.
Go SDK
Temporal implemented the following improvements over Cadence Go client:
Protobuf & gRPC
No global registration of activity and workflow types
Ability to register activity structure instance with the worker. It greatly simplifies passing external dependencies to the activities.
Workflow and activity interceptors which allow implementing features like configuring timeouts through external config files.
Activity and workflow type names do not include package names. This makes code refactoring without breaking changes much simpler.
Most of the timeouts which were required by Cadence are optional now.
workflow.Await method
Java SDK
Temporal implemented the following improvements over Cadence Java client:
Workflow and activity annotations to allow activity and workflow implementation objects to implement non-workflow and activity interfaces. This is important to play nice with AOP frameworks like Spring.
Polymorphic workflow and activity interfaces. This allows having a common interface among multiple activity and workflow types.
Dynamic registration of signal and query handlers.
Workflow and activity interceptors which allow implementing features like configuring timeouts through external config files.
Activity and workflow type name generation improved
SDKs not supported by Cadence
Typescript SDK, Python SDK, PHP SDK
SDKs under active development
.NET SDK, Ruby SDK
Temporal Cloud
Temporal Technologies monetizes the project by providing a hosted version of the Temporal service. There are dozens of companies (including SNAP) already using it in production.
Other
We have a lot of other features and client SDKs for other languages planned. You can find us at Temporal Community Forum.
Overview
Using iWF will let you switch between Cadence & Temporal easily.
In addition, iWF will provides a nice abstraction on top of the both and makes your life a lot better.
The fact is both Cadence & Temporal are under active development. You can see they have some different focuses if looking at their road maps. The two projects share the same vision to let everyone rethink about programming models of long-running business.
Tasks across domain+clusters
If you have multiple Cadence clusters,
this allows starthing childWF across different clusters and domains.
Support Both Thrift&gRPC
gRPC support is completely done on the server side. Internal traffic is all using gRPC and we are working on letting users migrating from Thrift to gRPC.
Authorization
The permission is based on domain but can be extended. Different from Temporal, the permission policy can be stored within Cadence domain data storage so that you don't have to build another service/storage to manage them.
Note that the whole proposal is developed by community member.
Workflow Shadower
Workflow Shadower is built on top of Workflow Replayer to address this problem. The basic idea of shadowing is: scan workflows based on the filters you defined, fetch history for each workflow in the scan result from Cadence server and run the replay test. It can be run either as a test to serve local development purpose or as a workflow in your worker to continuously replay production workflows.
Graceful domain failover
This allows XDC(multiple clusters) mode to reduce the pain of rerun some tasks during failover.
NoSQL plugin model
This allows implementing different NoSQL persistence in a minimum way. By the time writing this post, Temporal haven't started working on it.
MongoDB support
On top of the NoSQL interfaces, MongoDB support is WIP.
Using multiple SQL instances as sharded SQL
This allows user to have a Cadence cluster with a much larger scale. (then using XDC to add more DB instances)
Configuration Storage for Dynamic config
This enables us changing the dynamic configuration(like for ratelimiting) without making any deployment. Just a CLI command can control the behavior of the system.
It's in experiment and still WIP for production ready.
Workflow notification
A WIP eco system project to allow getting notification from Cadence. This is the benefits of Cadence using Kafka to deliver visibility messages. Temporal doesn't use Kafka which will be super difficult to support this feature.
Periodical Healthchecker(Canary) and Benchmark tool and benchmark setup docs
More Documentation
Seamless Cluster Migration guidance
Dashboard/Monitoring
...
...
Other small improvements that Temporal is missing
TerminateIfRunning IDReusePolicy
All domain API forwarding policy
Better & cleaner XDC configuration
Tooling to deserialize database blob data
...
...
I'm from the Cadence team at Uber, and I wanted to let you know that Cadence continues to be developed actively by our team. Below is a section of the update that we shared with the Cadence community recently:
We want to reinforce that Uber's Cadence team is committed to the
growth and open source development of the Cadence project. Today,
Cadence powers 100+ different use cases within Uber and that number
grows quickly. Collectively, there are 50M+ ongoing executions at any
moment on average and our customers finish 3B+ executions per month.
Outside of Uber, we also know that many engineering teams at various
companies have already adopted Cadence for their business-critical
workflows. We are excited to continue evolving Cadence as an
open-source project in a backward-compatible way with an increased
focus on reliability, scalability, and maintainability in the near
term.
It's probably too early to compare Cadence and Temporal. Still, I have a few ideas around how we can systematically shed light on Cadence's roadmap to ensure all the necessary information is out there to enable such comparisons going forward. I'll update this post with links when we create a page with information about the roadmap.
In the meantime, please let me know if you need further information about Cadence that would be helpful in this context.
Temporal.io is a company that has forked cadence project and are now building on top of it - naming it temporal.
It is founded by the authors of cadence.
I would suggest using temporal.io as it is under active development
I want to develop and application with the Oracle ADF, but I want a RESTful web service that will allow for other clients (a desktop app, a mobile app, etc...). TO help me determine the best way to do this I have two questions:
Does the ADF have some easy way to create a REST service built into the application?
If not, can I create a JAX-RS web service and hook it up in the same container as the ADF application?
I am thinking that because of what the ADF is, it may be best not to try to create a service with it. If so, please alert me of this.
Now with the release of jDeveloper 12.2.1 ADF comes with inbuilt REST Freamework which will solve all kinds of requirements(CRUD Operations, Custom method invocation, Paging, Data Consistency Check etc.)
The link for getting started with this
https://docs.oracle.com/middleware/1221/adf/develop/GUID-8F85F6FA-1A13-4111-BBDB-1195445CB630.htm#ADFFD589
will help a lot in getting started with REST and ADF.
For getting into more advanced features please follow this link
https://docs.oracle.com/middleware/1221/adf/develop/GUID-589F3905-5A8D-402D-B2D2-3BEEB2D7DDD4.htm#ADFFD54082
this will help in implementing all the other features in the ADF way.
Yes you can build and consume REST from ADF Faces, please see below a step-by-step tutorial for ADF 12c:
http://docs.oracle.com/cd/E53569_01/tutorials/tut_rest_ws/tut_rest_ws.html
This will work well for simple use cases, where there is not much complexity behind data entry screens.
However, you must keep in mind that ADF is most powerful when using Business Components and Bindings. Any other combination will decrease your productivity.
Yes, you can build REST service on top of ADF components, here's a fresh tutorial showing how to proceed with many Java technologies : http://multikoop.blogspot.com/2014/06/create-restful-services-on-top-of-adf.html and also http://waslleysouza.com.br/en/2014/10/expose-adf-bc-restful-web-service/
With JDeveloper 12.2.1 release, an easy way to create REST services has been provided. Using this feature, you can expose the entity objects of your choice as REST resources and then consume them easily via the DC palette.
Take a look at this blog which provides some insight into the same:
http://stick2code.blogspot.in/2015/11/creating-adfbc-rest-services-new.html
I am involved with a team that uses Xcelsius for their dashboard presentation design. I know of options provided by SAP to connect to the database. My team used flynet but lisence issues are pushing us to think of custom webservice. Can design a custom webservice (C# or Java) and use it with Xcelsius. I will realy like to know if its worth it or there are better options for my team. thanks
I was recently involved with an engagement that built out custom web services to both write new and alter existing records in the database. This was the only option for us as SAP does not provide any viable options to write back out of the box. Also, I don't think they (SAP) should but this was a unique requirement that necessitated the workflow.
The web services were in .net
We are currently trying to decide how to implement workflow into our enterprise application. We must support very complex workflow scenarios and we must also handle approx 10000 concurrent users. The application domain is healthcare. The client is a WPF application talking to an IIS backend using WCF. The question is: Does anybody have any experience with Windows workflow foundation in such a large enterprise application? From the requirements is seems like WWF fits the bill, but I am worried about performance and scalability. Should I continue to investigate into WWF or is WWF just not suitable for this kind of applications? We also need to let our consultants have the ability to alter workflows and re-hosting the WWF designer certainly sounds tempting. What do you think. Is WWF the way to go?
You can create a workflow that is directly hosted by IIS. You can use the "Receive" activity to setup the WF to handle requests. You can then attach a "Send" activity to reply back to the client with a response, and even have the workflow continue processing in the background that activity. Hope that provides some direction to get you started. You can achieve this with either the .NET Framework 3.5 or 4.x versions, but .NET 4.0 is much easier to setup.
The performance will rely on the hardware.
Anyway, workflow services are perfectly scalables in IIS. You will be able to set up a server farm that will serve all the request.
Regarding the requirement of altering workflows, always is possible to alter the workflow in a way that future requests will create instances of the new altered workflow. Old and already initiated workflows can not take any variation. So,if I have not understood wrongly, WWF does not fit this requirement.
We currently have an application that is based on an in-house developed workflow engine with YAML based DSL. We are looking to move parts of it to Java.
I have discovered a number of java solutions like Intalio, JBPM, Drools Expert, Drools Flow etc.
They appear to be aimed at businesses where the business analyst creates the workflows using a graphical editor and submits them to the workflow engine. They seem geared towards ease of use for non-technical people rather than for developers with a focus on human interaction.
The workflows tend to look like.
Discover-a-file -\
-> join -> process-file -> move-file -> register-file
Discover-some-metadata -/
If any step fails we need to retry it X times. We also need to be able to stop the system and be able to restart it and have it continue from where it was (durable).
Some of our workflows can be defined by a set of goals we need to achieve so Jess's backwards rule chaining sounds interesting but it is not open source.
It might be that what we are after is a Finite State Machine engine or just an Enterprise Service Bus and do everything as JMS queues.
Is there a good open source workflow engine that is both standards-based but also geared towards developers. We don't particular want to use a graphical workflow designer or write reams of XML and it should ideally be in Java or language agnostic (makes REST/Soap calls to external services).
Thanks,
Tom
Both Activiti and Bonita are open source and standard based (BPMN2). See for example this blog post.
Ruote is not standard based but seems close to your DSL approach and runs on a JVM thanks to JRuby.
Intaloi an open source BPM engine it offers a BPMN-support Designer and a BPEL engine. it's written in Java.
Camunda BPM is a developer-friendly Open Source workflow engine that is based on the open standards BPMN 2.0, DMN 1.1 and CMMN 1.1.
While it does come with a comfortable graphical workflow designer it also ships with a fluent API to build workflows programmatically. Camunda is written in Java, but can also be invoked from other languages via its REST API and it can make REST/Soap calls to external services.
jBPM 5 (open source, ASL, BPMN2) is just released and it's the best of Drools Flow and jBPM 4. It's lightweight but it can also integrate deeply with the Drools rule engine to make decisions.
For anyone looking for Python based enterprise grade solution.
Zengine, is GPL3 BPMN workflow based framework with Tornado, Rabbit AMQP, advanced permissions, extensible scaffolding features and more.
Built on top of following major components;
SpiffWorkflow: Powerful workflow engine with BPMN 2.0 support.
Tornado: Tornado is a Python web framework and asynchronous networking library.
Pyoko: Django esque ORM for Riak KV store.
RabbitMQ: Fast, ultrasharp AMQP server written with legendary Erlang lang.