My Goal:
Then database table was externally changed, I want to send WebSocket notification to clients.
Question:
Is there a "native" Sails.js way to track changes in database table populated via Model?
I only dabble in sails but I'm not aware of a way. You might make a "model-listener" service that utilizes your adapter of choice's socket/channel capabilities. You'll have to start the listeners at some point via a hook or in the bootstrap file.
The problem you're going to run into is determining if the event(create, update, drop/delete) was external or sails. I'm more familiar with PGSQL and know you can provide an application name to your connection and could include it in your publish message so your listener/subscribe handler can ignore non-sails related events.
PGSQL trigger/notify/listen
Event Triggers
Notify
Listen
MongoDB
Capped Collections
Tailable Cursors
Of course waterline supports more adapters than the two I've listed here but I tried to pick the two I assume are the most popular. I know this might be the answer you had hoped for but it might give you some ideas to try.
Sorry, I'm a new poster so I'll try to provide some links in comments if it will stackoverflow will allow me.
Related
So I have an app taking advantages of Heroku Connect to sync datas between platforms.
I need to find a way to detect when an update has been made by Salesforce (or at least, when the sync has been executed). I'm using sequelize in nodejs, but of course the hooks don't work since heroku connect works directly on the DB and doesn't use the ORM.
So I'm wondering what are my options here.
The solutions that come to my mind (likely there are more):
check out the Heroku Connect system tables like _trigger_log. This table will give you an exact log of the actions HC took (updates/insert/deletes) with information about the record. Yes, you would need to poll it :)
Postgres brings it's own queue-system with LISTEN and NOTIFY. You an write your own database-trigger that will react on change in the salesforce tables, and have a listening/worker-process on the LISTEN queue in PostgreSQL.
I am new in the topic reactive programming and therefore have some questions.
I am developing a small software.
I would like to take the opportunity to get to know reactive programming better.
So I looked at Spring's project-reactor.
I also use R2DBC to reactively access the database.
I would like to know if there is any way that database responds to changes.
Or rather: If a user saves an entry in the database, then servers (for example, RestController) should be notified.
How could I go about doing that?
Enresponding controllers, configuration, entities, etc. I have already implemented.
Sorry for spelling mistakes.
Complement: The updates to the frontend are then made by Server Sent Events.
Basically, what Nick Tsitlakidis mentioned. Let me add a couple of things here.
The typical database query pattern is to query for a number of records. Databases respond with their results and indicate that the query is complete once a server has sent all records to your application. If new records arrive while the query is active or after the query is complete, you do not see these changes immediately because the of isolation and in case the query is complete, then you no longer have a reference to the query.
The feature you're asking is event-driven consumption of data. Databases call this feature continuous queries. Some stores (such as MongoDB with Tailable cursors or Postgres Logical Decoding) come with features that allow keeping a cursor/query open and your client is able to receive continuous updates.
Kafka and JMS also follow the idea of sending (messages) that are consumed typically by listeners or even through a reactive stream.
So it all boils down to the technology that you're using.
My understanding is that reactor can't solve this problem for you on its own. If you want your application to respond (react) on some database change, then you need to identify who's making this change and implement some kind of integration there.
Example, if you have Service1 updating the database, and Service2 needs to respond then Service1 can either call Service2, or, you can emit an event from Service1 and listen for the event from Service2.
The first approach is simpler and easier to implement but it has the disantvantage that is couples the two services. The second is trickier to implement but services are decoupled.
Reactor can help you in both cases :
For events, reactor can give you a way to listen to the events. For example using the reactor-rabbitmq module or the reactor-kafka.
For service-to-service calls, reactor can help you if you use Spring Webflux.
Perhaps you can tell us more about your case so we can provide a more specific solution?
Context
Greetings,
One day I randomly found RethinkDB and I was really fascinated by the whole real-time changes thing. In order to learn how to use this tool I quickly spinned up a container running RethinkDB and i started making a small project. I wanted to make something very simple therefore i thought about creating a service in which speakers can create room and the audience can ask questions. Other users can upvote questions in order to let the speaker know which one are the best. Obviously this project has a lot of realtime needs that i believe are best satisfied by using RethinkDB.
Design
I wanted to use a vary specific set of tools for this. The backend would be made in Laravel Lumen, the frontend in Vue.JS and the database of course would be RethinkDB.
The problem
RethinkDB as it seems is not designed to be exposed to the end user directly despite the fact that no security concern exists.
Assuming that the user only needs to see the questions and the upvoted in real time, no write permissions are needed and if a user changed the room ID nothing bad will happen since the rooms are all publicly accessible.
Therefore something is needed in order to await data updates and push it through a socket to the client (socket.io for example or pusher).
Given the fact that the backend is written in PHP i cannot tell Lumen to stay awake and wait for data updates. From what i have seen from the online tutorials a secondary system should be used that should listen for changes and then push them. (lets say a node.js service for example)
This is understandable however i strongly believe that this way of transferring the data to the user is inefficient and it defeats the purpose of RethinkDB.
If I have to send the action from the client's computer (user asks a question), save it to the database, have a script that listens for changes, then push the changes to socket.io and finally have the client (vue.js) act when a new event arrives, what is the point of having a real-time database in the first place?
I could avoid all this headache simply by having the Lumen app push the event directly to socket.io and user any other database system instead.
I really cant understand the point of all this. I am not experienced with no-sql databases by any means but i really want to experiment with them.
Thank you.
This is understandable however i strongly believe that this way of transferring the data to the user is inefficient and it defeats the purpose of RethinkDB.
RethinkDB has no built in mechanism to transfer data to end-users. It has no access control (in the conventional sense) as well. The common way, like you said, is to spin up one / multiple node instance(s) running socket.io. On each instance you can listen on your RethinkDB change streams and use socket.io's broadcast functionality. This would be a common way, but as RethinkDB's streams are pretty optimized, you could also open a change stream for every incoming socket.io connection.
This question has been asked multiple times, here and here, and the answer to get this working is fairly straight forward: add an environmental variable to your bash_profile and all Meteor instances on your localhost will share that MONGO_URL.
What I've noticed however is that while this may be the case, there's quite a bit of latency in the "reactivity" of Meteor. I've tested this with two very lean Meteor apps, with empty collections. Inserting a document to a collection from one Meteor app, where my second app is querying that same collection and printing out a field from the documents does work, but there's a noticeable lag before it updates. I've ruled out the possibility of the collection insertion being the source of the lag (simple console.log callback on the client of the first app, logging the id of the newly inserted document).
My purpose for having multiple apps (two to be precise) sharing the same MongoDB is to separate an admin panel from a mobile app without going crazy regarding name-spacing and bloat. This configuration works, but I'm not sure it's the "proper" way of accomplishing the task, and it certainly seems to be causing a performance hit.
Any insight into this matter would be appreciated. Thank you!
EDIT: To clarify, the db URL I'm using is on my localhost, and isn't something hosted online.
When you use an external database, by default meteor will use periodic polling (every few seconds) in order to observe any changes. The delay you are experiencing is a result of this polling process. You can remove the delay and reduce your app's CPU usage by taking advantage of meteor's oplog tailing feature. In order to use it you will:
Get access to a mongodb instance with the oplog turned on.
Set the environment variable MONGO_OPLOG_URL so your app(s) can read the oplog.
Personally, I'd recommend compose.io for this. They provide exactly this as part of their basic elastic deployment. See this post for detailed instructions.
For users who wish to connect to the oplog created locally for you, you can obtain the URL via:
MongoInternals.defaultRemoteCollectionDriver().mongo._oplogHandle._oplogUrl
It should end up looking something like mongodb://127.0.0.1:3001/local
I am working on the jabber chatting Applications with the use of XMPP server .
I want to make 2 user friend so I have to add roster with the use of mysql query.
I have make entry in two tables.(1) ofRoster (2)ofRosterGroups.
I make entry in both the table but its not working.
Is there anything where I am missing.
I can do this with the admin panel but i don't want to do that.
I think you are using openfire (those tables in SQL look like the openfire setup). If so, the table you have to edit is "ofGroupUser". To add a user to a group you need to do a sql insert into that table where the group name is the group you want to add the user to, the username is the user you are adding to the group and administrator is the flag of that user's authority (just use 0). An example insert would look like this:
INSERT INTO ofGroupUser VALUES("group name", "user", administrator);
However, as mentioned in the above post this is not a good method for doing this as it will not immediately affect the server. You must restart the server for these changes to take place because openfire (or whatever server you are using) probably only reads the database on start up. Once it caches everything, it will edit the database according to requests (like adding users or groups through the admin console), but will not read from it and your additions will not be seen until a server restart occurs.
Basically, doing manual sql inserts will produce the desired results, and, if you are just testing some functionality, will work just fine as long as you restart the server. If you are using openfire and need to do group administrative work in some way besides the web ui, I would look into using a different server. As far as I know, openfire isn't real great with administration outside of it's web ui. Here is a list of many open source xmpp servers. I'd recommend ejabberd (as mentioned above post) it has a very nice control tool called ejabberdctl with an available expansion module called mod_ctlextra (here is the man page for it which lists commands) that will allow you to do what I assume you are wanting. Then you don't have to worry about sql and restarting, just use their tool which is how it should be.
Also, on a side note, ejabberd is extremely efficient due to the nature of the language used to write it: Erlang. Great stuff.
Hope that helps!
Presumably you are using the odbc modules with ejabberd. The sql schema though defines two tables rostergroups and rosterusers, not the ones you mention in the question. In any case you should not update the tables directly, ejabberd keeps internal state and does not get notified of your changes.
The way to go is by actually having the users send the mutual subscriptions and accept them as per the rfc. Roster Item Exchange might also be useful.