we are making some loging issue, where we need write the logentries in the DB. But the process run in a transaction and by rollback are our new logentries also deleted. can I make a write in DB out of the transaction? something like write in temptable with NO-UNDO option...? that the new logentries still remain in DB...?
Another possibility would be to use an app server. Transactions on app server sessions are independent from transactions in the original session (that's what the optional and redundant "DISTINCT TRANSACTION" syntax is all about).
Another option would be to use a simple messaging system. One very easy to setup and use option is STOMP. It is platform neutral and very easy to get going with.
Julian Lyndon-Smith posted the following on PEG about a month ago, and it really is as easy to setup and use as he says (I've tried it, I used ApacheMQ which is also very easy to setup and use):
Following on from presentations in Boston and Finland, dot.r is
pleased to announce the open source Stomp project, available
immediately.
Download from either http://www.dotr.com or
https://bitbucket.org/jmls/stomp , the dot.r stomp programs allow you
to connect your progress session to any other application or service
that is connected to the same message broker.
Open source, free message brokers that support Stomp are:
Fuse
(http://fusesource.com/products/fuse-mq-enterprise/) [a Progress company now owned by Red Hat inc]
Fuse MQ Enterprise is a standards-based, open source messaging platform that deploys with a very small footprint. The lack of license
fees combined with high-performance, reliable messaging that can be
used with any development environment provides a solution that
supports integration everywhere
ActiveMQ
Apache ActiveMQ (tm) (http://activemq.apache.org/)is the most popular
and powerful open source messaging and Integration Patterns server. Apache
ActiveMQ is fast, supports many Cross Language Clients and Protocols, comes
with easy to use Enterprise Integration Patterns and many advanced features
while fully supporting JMS 1.1 and J2EE 1.4.
Apache ActiveMQ is released under the Apache 2.0 License.
RabbitMQ
RabbitMQ is a message broker. The principal idea is pretty simple: it
accepts and forwards messages. You can think about it as a post
office: when you send mail to the post box you're pretty sure that Mr.
Postman will eventually deliver the mail to your recipient. Using this
metaphor RabbitMQ is a post box, a post office and a postman.
The major difference between RabbitMQ and the post office is the fact
that it doesn't deal with paper, instead it accepts, stores and
forwards binary blobs of data - messages.
Please feel free to log any issues on the
https://bitbucket.org/jmls/stomp issue system, and fork the project in
order to commit back all those new features that you are going to add
...
dot.r Stomp uses the permissive MIT licence
(http://en.wikipedia.org/wiki/MIT_License)
Have fun, enjoy !
Julian
Every change to the database must be part of a transaction. If you do not explicitly start one it will be implicitly started for you and scoped to the next outer block with transaction capabilities.
However and although I would not recommend you to, work with sub-transactions. You can invoke a sub transaction by explicitly specifying a DO TRANSACTION within the transaction scope. Although the database will never know about it, the client can roll back the sub transaction while the database can commit the transaction.
But in order to implement something like this you must master the concepts of transaction scope, block behavior and error handling.
RealHeavyDude.
Write your log entries to a no-undo temp-table.
When the code will commit a transaction, or transactions aren't active (transactionID = ?) have your code write the log entries out.
I don't think there is any way to do this in ABL as you planned either efficiently (sprinkling temp-table flushes or other tidbits all over the place is gross) or reliably (what if the application crashes with an un-flushed temp-table?), as others have mentioned. I would suggest making your complicated logging less coupled to your app by making the database writes asynchronous, occurring outside of your application if possible.
Since you're on Windows, you could change your logging to use the .NET log4net library instead of ABL constructs. log4net has a few appenders that would be useful:
AdoNetAppender which lets you log directly to a database
RemoteSyslogAppender which uses the syslog protocol, letting you log to an external Unix syslog or rsyslog daemon (rsyslog supports writing log messages to databases)
UDPAppender which sends the log messages via UDP packets somewhere else to be handled (e.g. a logFaces server, which supports writing to databases)
If you must do it in ABL then you could use a named output stream specifically for your log messages (OUTPUT TO STREAM) which writes to a specific location where an external process is listening to handle it. This file could be a pipe created by something like mkfifo or just a regular text file that is monitored for changes with inotify (not sure what the Windows equivalents of these are). This external process would handle parsing the messages and writing them to the database (basically re-inventing rsyslog).
I like the no-undo temp-table idea, just be sure to put the database write part in a "FINALLY" block in case of unhandled exceptions.
Related
I'm willing to use ejabberd / mongooseIm in a microservice network. XMPP should be our chat protocol aside from a REST API network. I want to send messages incoming at the xmpp server downstream to worker services. Has anybody done this or could lead me into the right direction?
My first thoughts are using RabbitMQ for sending the new incoming messages to the workers.
There are basically two choices to giving your workers access to the messages routed by ejabberd / MongooseIM. I'll focus on MongooseIM, since I know it better (DISCLAIMER: I'm in the dev team).
The first is to scan the message archive in an async / polling fashion. The Message Archive Management describes XMPP level protocol for accessing it, but for your use case the important part is message persistence - so just making sure the relevant module (mod_mam) is enabled in server config and the messages will hit the database. The databases supported for MAM are PostgreSQL and Riak, though there was also some work on a Cassandra backend (YMMV). This doesn't require tinkering with the server / in Erlang for as long as there's a DB driver for your language of choice available. Since PR#657 it's possible to store the messages in raw XML or even some custom format if you're willing to write the serialization module.
The second option is to use the server mechanism of hooks and handlers (also available in ejabberd), which can trigger a server action on events like "user sent a message", "user logged in", "user logged out", ... This, however, requires a server side extension written in Erlang. In the simplest case the extension could forward any interesting event (with message content and metadata) via AMQP or just call some external HTTP/REST API - that way the real work is carried out by the workers giving you the freedom with regard to implementation language. This options also doesn't require to enable mod_mam or set up a database for message persistency (which you could still have with a persistent message queue...).
In general, the idea is perfectly feasible.
Generally, the most common XMPP extension use to build messaging systems for machines-to-machines, internet of things, microservices, etc is PubSub, as defined in XEP-0060.
This is a module you can enable in ejabberd. It is API based, so you can even customize the behaviour of that module to your application specific.
Pubsub basically allows to decouple senders and receivers and is especially designed for that use case.
I would like to send all data received from fuse, in a specific Topic, to a Business Process in BPM Studio. Is there any way?
Example:
I send a value to 'testTopic' in Fuse. Then Fuse send this value to a Business Process (or the Business Process retrieve it), then the Business Process do things based on the value recevied, like sending another value to another topic
Is somithing of this kind possible?
Yes it most definitely is possible, although you would need to route from the 'testTopic' to one of the JMS Queues that jBPM can listen on and transform the message to reflect a valid jBPM command. The generic principle is described in the documentation at http://docs.jboss.org/jbpm/v6.0/userguide/jBPMRemoteAPI.html#d0e12149. The real power becomes clear when you look at all the jBPM commands you can send in the packages
org.drools.core.command.runtime.process (Maven: org.drools:drools-core)
and
org.jbpm.services.task.commands (Maven: org.jbpm:jbpm-human-task-core).
When talking from the outside world, it would typically be necessary to identify a correlationKey in the process which is basically the "Business Key" that can be used to identify a process uniquely e.g. as 'ApplicationNumber' for an application process. This can be used to then identify which process you may want to signal/abort/etc.
Since you are working in Fuse you should probably also consider routing that message to the jBPM Rest API described at http://docs.jboss.org/jbpm/v6.0/userguide/jBPMRemoteAPI.html#d0e10088. This may simplify your code a bit because it is a more synchronous API. The drawback however is the REST over HTTP invocation typically does not respect the local transaction.
Does JavaMail support notification of new emails through server-push?
If yes, where is the documentation for that?
If no, is there a library that can do it?
You should be using IMAPFolder's idle function to issue the idle command to the server. That will then listen for events, such as a new mail or deleted mail. (See the IMAP spec to see what the messages look like). And you should be using a MessageCountListener to execute code when a number of emails in the mailbox change.
IMAP's idle function is exactly meant to imitate "push" functionality.
http://java.sun.com/products/javamail/javadocs/javax/mail/event/MessageCountListener.html
http://java.sun.com/products/javamail/javadocs/com/sun/mail/imap/IMAPFolder.html
Sorry I didn't post any code that shows how this is used. I didn't want to waste my time since there are many readily available examples on the internet if you search for this stuff.
But be forewarned, this method won't work for more than one IMAP account since the idle command blocks. Unless you want them all on different threads (bad idea).
A Store event listens for notifications issued by your backend store:
http://java.sun.com/products/javamail/javadocs/javax/mail/event/StoreEvent.html
But in my experience the java mail docs are so thin in places, that the best way of finding out what is going on, is to debug through the process yourself.
This is a great allround resource as well; the JavaMail FAQ :
http://www.oracle.com/technetwork/java/faq-135477.html
I'm working on a messaging/notification system for our products. Basic requirements are:
Fire and forget
Persistent set of messages, possibly updating, to stay there until the sender says to remove them
The libraries will be written in C#. Spring.NET just released a milestone build with lots of nice messaging abstraction, which is great - I plan on using it extensively. My basic question comes down to the question of message brokers. My architecture will look something like app -> message broker queue -> server app that listens, dispatches all messages to where they need to go, and handles the life cycle of those long-lived messages -> message broker queue or topic -> listening apps.
Finally, the question: Which message broker should I use? I am biased towards ActiveMQ - We used it on our last project and loved it. I can't really think of a single strike against it, except that it's Java, and will require java to be installed on a server somewhere, and that might be a hard sell to some of the people that will be using this service. The other option I've been looking at is MSMQ. I am biased against it for some unknown reason, and it also doesn't seem to have great multicast support.
Has anyone used MSMQ for something like this? Any pros or cons, stuff that might sway the vote one way or the other?
One last thing, we are using .NET 2.0.
I'm kinda biased as I work on ActiveMQ but pretty much all of benefits listed for MSMQ above also apply to ActiveMQ really.
Some more benefits of ActiveMQ include
great support for cross language client access and multi protocol support
excellent support for enterprise integration patterns
a ton of advanced features like exclusive queues and message groups
The main downside you mention is that the ActiveMQ broker is written in Java; but you can run it on IKVM as a .net assembly if you really want - or run it as a windows service, or compile it to a DLL/EXE via GCJ. MSMQ may or may not be written in .NET - but it doesn't really matter much how its implemented right?
Irrespective of whether you choose MSMQ or ActiveMQ I'd recommend at least considering using the NMS API which as you say is integrated great into Spring.NET. There is an MSMQ implementation of this API as well as implementations for TibCo, ActiveMQ and STOMP which will support any other JMS provider via StompConnect.
So by choosing NMS as your API you will avoid lockin to any proprietary technology - and you can then easily switch messaging providers at any point in time; rather than locking your code all into a proprietary API
Pros for MSMQ.
It is built into Windows
It supports transactions, it also supports queues with no transactions
It is really easy to setup
AD Integration
It is fast, but you would need to compare ActiveMQ and MSMQ for your traffic to know which is faster.
.NET supports it nativity
Supports fire and forget
You can peek at the queue, if you have readers that just look. not sure if you can edit a message in the queue.
Cons:
4MB message size limit
2GB Queue size limit
Queue items are held on disk
Not a mainstream MS product, docs are a bit iffy, or were it has been a few years since I used it.
Here is a good blog for MSMQ
Take a look at zeromq. It's one of the fastest message queues around.
I suggest you have a look at TIBCO Enterprise Messaging Service - EMS, which is a high performance messaging product that supports multicasting, routing, supports JMS specification and provides enterprise wide features including your requirements suchas fire-forget and message persistence using file/database using shared state.
As a reference, FEDEX runs on TIBCO EMS
as its messaging infrastructure.
http://www.tibco.com/software/messaging/enterprise_messaging_service/default.jsp
There are lot other references if i provide, you'd really be surprised.
There are so many options in that arena...
Free: MantaRay a peer to peer fully JMS compliant system. The interesting part of Mantaray is that you only need to define where the message goes and MantaRay routes it anyways that will get your message to it's detination - so it is more resistant to failures of individual nodes in your messaging fabric.
Paid: At my day job I administer an IBM WebSphere MQ messaging system with several hundred nodes and have found it to be very good. We also recently purchased Tibco EMS and it seems that it will be pretty nice to use as well.
We are using EntLib Logging Application Block. And also it turned out that we should use msmq for logging because of performance.
Now we are trying to use Msmqdistributor service to log those messages in the queue.
What are pros and cons of Msmqdistributor service of Enterprise Library?
Please share your experience.
The main drawback is going to be the Microsoft Message Queue (MSMQ) itself. MSMQ has been around for awhile and it is a pretty cool tool. It does however lack utilities. Because of the way that data is stored in the queue, most people end up needing to write some helper utilities for debugging and manually manipulating the queue. Some other things to consider:
Queue size - if too many items get put in the queue, and aren't removed in a timely manner the server can stall.
Purpose - MSMQ is designed for multi-step transactions (such as billing), you mention you are going to use it for logging. If the log is just for debugging, Then a DB table or a flat file or sending errors to a bug tracker will serve you better. If you need complicated logging and are using MSMQ to send the information to a different copmuter, then you will find MSMQ more useful.