I'm working on an OPC(DA) server that creates a collection of server items and sends them to an OPC client. Each item has a "name" value that determines the order that the items are displayed in. The name of each item is structured as:
Sites.<SiteID>.CurrentValue
So the data might look something like:
Sites.0001.CurrentValue
Sites.0002.CurrentValue
Sites.0003.CurrentValue
Etc.
Or in a tree format:
Sites:
0001:
CurrentValue
0002:
CurrentValue
0003:
CurrentValue
Etc.
Since the items are ordered by name and the only variable part of the item name is the site ID, the items are effectively ordered by site ID. The problem occurs when the OPC client displays the items. The order that they're displayed in is totally different:
Sites:
6219
13501
13502
4000
4001
626262
4002
4003
4004
4005
4006
4007
4008
0030
4009
0200
79791
Etc.
I've been trying to infer some kind of logical ordering system that would give this result, but I'm just not seeing anything. I have tried this with several OPC clients (Matrikon, dOPC, KEP) and they are all consistently presenting items in the above order, which leads me to believe there is some kind of universal OPC ordering system, but I've not been able to find anything.
My hope is that, if I can find out how OPC is ordering these items, I can order the items in the OPC server, such that they will get displayed in a logical order in the OPC client.
My server is Advosol-based (I don't have enough reputation to create a new tag).
Related
In my single page application that uses RESTful services, I want to know what the best practice is for updating one item in an array of items.
Prerequisites
1. Front-end makes a GET request to fetch a list of items
2. Front-end formats the list of items (i.e. converts dates from UTC to local time)
3. Front-end makes a PUT request to the back-end to update the name of an item
Possible solutions
Solution #1
4. Back-end responds with HTTP-200 and the single updated item
5. Front-end reformats the updated item
6. Front-end splices the list of items, finding and replacing the updated item
PRO
- One API request to update the item
CON
- Duplication of data on the front-end, no single source of truth (i.e. the list of items)
Solution #2
4. Back-end responds with HTTP-200 and the updated list of items
5. Front-end reformats the list of items
PRO
- One API request to update the item
CON
- Does not follow the single responsibility principle (i.e. the API for updating the item updates the single item, and returns all items)
Solution #3
4. Back-end responds with HTTP-200 and the single updated item
5. Front-end makes a GET request to fetch all of the items
6. Front-end reformats the list of items
PRO
- More flexible for future implementations, APIs follow the single responsibility principle
CON
- Two API requests to update the item
I want to know what the best practice is for updating one item in an array of items.
An important thing to understand about REST, or HTTP, is that we are designing messages to be consumed by general-purpose components. Which is to say, we are using the readily standardized forms to communicate the semantics.
An HTTP PUT has the semantics of upserting a document into a document store. For your example, where we GET a representation of a list resource, make local edits, and PUT the result, the payload of the PUT is a copy of the complete representation of the resource -- what we are asking is that the server make it's copy look like the client's copy.
Assuming that the server elects to apply the new representation to its copy of the resource, the payload of the response could be a status message ("It worked"), or a copy of the new representation of the resource, or even an empty document (204 No Content) with metadata that describing the new representation of the resource (and the implication that the server accepted the client's representation without modification).
The key idea behind PUT, however, is that the payload is a complete representation of the resource, not merely a description of a change made to it. If the document is very large (in particular, large in comparison to the HTTP headers), and the edit you are making is small, then you may prefer to send a patch-document describing the changes you made to the document, specifying the PATCH method in the request.
Of course, on the web, the most popular document format didn't include hypermedia support for PUT or PATCH, and the most popular clients were browsers, not document editors, so we had to design our change protocols around POST. So it's "fine" to do it that way too, you just need to think about how representations of form data are going to be applied to the resource.
Solution 2 does follow the single responsibility principle, you might be confused by the naming and 'responsibility', but if we consider the true definition of SRP: 'single reason for change' Solution 2 is completely fine and the preferred way if performance is not critical.
https://deviq.com/single-responsibility-principle/
I started a OPC UA project using the Milo project to create a OPC UA Client. I am still very new to OPC UA. Right now I am stuck looking for the best practice to read values from several Nodes after a data change of one specific node.
The information model looks like this:
RfidSensorType
On my server i will have several objects of this RfidSensorType. The client creates a subscription on the CurrentAtTag Node to listen for data changes.
My Question:
When the value of CurrentAtTag is changed a callback function will be called in my client which contains the UaMonitoredItem and the DataValue of the CurrentAtTag.
In my application i need to process (at the same time) also the values of Station, IOLPort and CurrentValue which are changed at that moment too.
How can i access those values within the callback from CurrentAtTag?
My only solution is: Using a synchronous read request within that callback
-> Is that an legit approach?
My Research:
1) TriggeringService
I've seen that a TriggerigService exists, which monitors items will send reports only if one specific node changes it values.
Problem: This will call several callsbacks and noz just one..i need all the informations at the same time to further process them..
2) Event Monitoring
In event monitoring one can select "Event fields" which will be returned for each Event notificaiton. I am not sure if i could select the CurrentAtTag, Station, IOLPort and CurrentValue...
Just like you can subscribe to the server's ServerStatus (nodeid "i=2256"), you should be able to subscribe to the nodeid corresponding to 'RfidSensor_Station1'. The server will send PublishResponse with data of type 'RfidSensorType' encoded as an ExtensionObject. The trick is decoding the ExtensionObject.
As Kevin corrected, because 'RfidSensor_Station1' is not node class 'Variable' then it doesn't have a value attribute and you can not monitor the node for data changes. If you are using a PLC, I might combine all properties of the sensor into a string, or byte array. Then I monitor the new variable, and parse the string in the client.
Or you could make ReadRequest as you describe. That will work just fine.
We have a RESTful web service that returns a collection of tickets. Because it's possible for the collection returned to be too large to be processed in a single gulp, we've added offset and limit query parameters. The idea is that we run the query, then skip the first offset records, then return the next limit records.
The problem is that this can leak tickets.
Suppose, for example, that there are eight tickets that need work, at the time the client first queries:
ID STATUS
00 needs work
01 needs work
02 needs work
03 needs work
04 needs work
05 needs work
06 needs work
07 needs work
If the client requests the tickets that need work, with an offset of 0 and a limit of 4, we'll return:
ID STATUS
00 needs work
01 needs work
02 needs work
03 needs work
If someone else, then does some work, changing some tickets to:
01 doesn't need work
02 doesn't need work
If the client then requests the tickets that need work, with an offset of 4 and a limit of 4, the results of the query will be:
ID STATUS
00 needs work
03 needs work
04 needs work
05 needs work
06 needs work
07 needs work
And after we skip the first four records, we'll return:
06 needs work
07 needs work
And tickets 04 and 05 will have been skipped.
If we go back to the ticket table on every subsequent paging request, we'll leak tickets, if tickets on earlier pages have been changed so that they fall out of the query results.
Part of me is wondering how important this is.
The client is going to request the needs work tickets on some sort of schedule. When there are more tickets than the limit, it will then page through the rest in multiple calls, incrementing offset on each call. If we do nothing, we will sometimes leak needs work tickets, but they will be picked up the next time the client requests new needs work tickets.
That is, the leaked tickets will only be leaked on this pass, they'll show up on the next.
But if it is important that we not leak tickets, I don't see any way of resolving it other than saving the identifiers of all of the needs work tickets during the first call, and then paging through the collection of identifiers, rather than through the tickets themselves.
We could, for example, when the client requests needs work tickets with an offset of zero, populate a second table with the ids of all of the tickets that need work, then return the first limit tickets that are in the second table. The next call, we use offset and limit against the second table, to determine which tickets to return.
The problem with this is we need to deal with multiple clients running simultaneously. So we need a primary key on the second table that we can match against a specific client, based on what is in the request.
I'd like to be able to manage this without putting additional burden on the client programmers. But I don't see how.
Is there any way for me to tell, by examining a request and its headers, that it came from the same client as an earlier request? I've not been able to find one.
We're currently returning paging information in the response headers:
Paging-offset: 0
Paging-limit: 4
Paging-returnedCount: 4
Paging-totalRecordCount: 54
What I'm thinking is that we might return a Paging-collection value, when we're paging, which would provide a key value into the second table. We could then require the client to provide the collection value when they make a request with offset != 0.
Does this seem reasonable? Do you think that this would put too great a burden on the client programmers?
How have other people solved this problem? Or do they just ignore it?
Is there any way for me to tell, by examining a request and its headers, that it came from the same client as an earlier request? I've not been able to find one.
You're not supposed to be able to - stateless protocol. In particular, if you are trying to do REST, you want the request to have all of the necessary information so that a new server can answer the request when the original server is busy.
But possibilities include giving each client its own resource to work with. There are a number of different ways you can match the request to the unique resource.
The problem with this is we need to deal with multiple clients running simultaneously.
As a rule, REST works much better if you can provide multiple clients with a common understanding of resources, rather than trying to tailor your representations to each.
Consider: Alice queries for a pile of work, Bob changes something, then Charlie queries for a pile of work. Can you live with it if Charlie gets a representation of the pile that was cached by Alice's query (ie, before Bob's change)? Cuz that's kind of how the web is designed to work....
(It doesn't have to - you can have each response set a bunch of no-cache headers. But it's something you should be thinking about, because it may be trying to tell you that the REST architectural constraints are not a great fit for your problem.)
How have other people solved this problem? Or do they just ignore it?
Well, it's a concurrent modification that's invalidating your iterator, right? Maybe you just pitch a fit and force the client to start over....
You might look into AtomSyndication, and how some services use it
For your case, I'd probably look at turning the problem around; instead of asking the server for N tickets that have some property, I'd look into asking the server for all tickets in some range that have the property. The client can just keep navigating through ranges until it fills its own bucket.
Another way of describing your problem is that you are trying to page through a mutable collection.
If you drop the paging constraint -- each request always fetches N unworked tickets starting from the first one, that's pretty straight forward.
If you drop the mutable constraint -- paging through an immutable list is straight forward. Making a mutable list immutable may be easy -- instead of asking for the latest version of the list, you ask for the version of the list as of some particular point in time. This is a very happy problem to have when using event sourcing.
One thing we've discussed is having one query return a list of ticket IDs, which would be small enough to return all of them, all of the time, then have a second query to return a single ticket given a query.
Another good answer; that's fundamentally the way web pages work -- (relatively) small payload of html, with hyperlinks to the java script, media, and so on that extends the representation.
I'm implementing a RESTful API which exposes Orders as a resource and supports pagination through the resultset:
GET /orders?start=1&end=30
where the orders to paginate are sorted by ordered_at timestamp, descending. This is basically approach #1 from the SO question Pagination in a REST web application.
If the user requests the second page of orders (GET /orders?start=31&end=60), the server simply re-queries the orders table, sorts by ordered_at DESC again and returns the records in positions 31 to 60.
The problem I have is: what happens if the resultset changes (e.g. a new order is added) while the user is viewing the records? In the case of a new order being added, the user would see the old order #30 in first position on the second page of results (because the same order is now #31). Worse, in the case of a deletion, the user sees the old order #32 in first position on the second page (#31) and wouldn't see the old order #31 (now #30) at all.
I can't see a solution to this without somehow making the RESTful server stateful (urg) or building some pagination intelligence into each client... What are some established techniques for dealing with this?
For completeness: my back-end is implemented in Scala/Spray/Squeryl/Postgres; I'm building two front-end clients, one in backbone.js and the other in Python Django.
The way I'd do it, is to make the indices from old to new. So they never change. And then when querying without any start parameter, return the newest page. Also the response should contain an index indicating what elements are contained, so you can calculate the indices you need to request for the next older page. While this is not exactly what you want, it seems like the easiest and cleanest solution to me.
Initial request: GET /orders?count=30 returns:
{
"start"=1039;
"count"=30;
...//data
}
From this the consumer calculates that he wants to request:
Next requests: GET /orders?start=1009&count=30 which then returns:
{
"start"=1009;
"count"=30;
...//data
}
Instead of raw indices you could also return a link to the next page:
{
"next"="/orders?start=1009&count=30";
}
This approach breaks if items get inserted or deleted in the middle. In that case you should use some auto incrementing persistent value instead of an index.
The sad truth is that all the sites I see have pagination "broken" in that sense, so there must not be an easy way to achieve that.
A quick workaround could be reversing the ordering, so the position of the items is absolute and unchanging with new additions. From your front page you can give the latest indices to ensure consistent navigation from up there.
Pros: same url gives the same results
Cons: there's no evident way to get the latest elements... Maybe you could use negative indices and redirect the result page to the absolute indices.
With a RESTFUL API, Application state should be in the client. Here the application state should some sort of time stamp or version number telling when you started looking at the data. On the server side, you will need some form of audit trail, which is properly server data, as it does not depend on whether there have been clients and what they have done. At the very least, it should know when the data last changed. No contradiction with REST here.
You could add a version parameter to your get. When the client first requires a page, it normally does not send a version. The server replies contains one. For instance, if there are links in the reply to next/other pages, those links contains &version=... The client should send the version when requiring another page.
When the server recieves some request with a version, it should at least know whether the data have changed since the client started looking and, dependending of what sort of audit trail you have, how they have changed. If they have not, it answer normally, transmitting the same version number. If they have, it may at least tell the client. And depending how much it knows on how the data have changed, it may taylor the reply accordingly.
Just as an example, suppose you get a request with start, end, version, and that you know that since version was up to date, 3 rows coming before start have been deleted. You might send a redirect with start-3, end-3, new version.
WebSockets can do this. You can use something like pusher.com to catch realtime changes to your database and pass the changes to the client. You can then bind different pusher events to work with models and collections.
Just Going to throw it out there. Please feel free to tell me if it's completely wrong and why so.
This approach is trying to use a left_off variable to sort through without using offsets.
Consider you need to make your result Ordered by timestamp order_at DESC.
So when I ask for first result set
it's
SELECT * FROM Orders ORDER BY order_at DESC LIMIT 25;
right?
This is the case when you ask for the first page (in terms of URL probably the request that doesn't have any
yoursomething.com/orders?limit=25&left_off=$timestamp
Then When receiving your data set. just grab the timestamp of last viewed item. 2015-12-21 13:00:49
Now to Request next 25 items go to: yoursomething.com/orders?limit=25&left_off=2015-12-21 13:00:49 (to lastly viewed timestamp)
In Sql you would just make the same query and say where timestamp is equal or less than $left_off
SELECT * FROM (SELECT * FROM Orders ORDER BY order_at DESC) as a
WHERE a.order_at < '2015-12-21 13:00:49' LIMIT 25;
You should get a next 25 items from the last seen item.
For those who sees this answer. Please comment if this approach is relevant or even possible in the first place. Thank you.
Where I work, we use Ruby on Rails to create both backend and frontend applications. Usually, these applications interact with the same MySQL database. It works great for a majority of our data, but we have one situation which I would like to move to a NoSQL environment.
We have clients, and our clients have what we call "inventories"--one or more of them. An inventory can have many thousands of items. This is currently done through two relational database tables, inventories and inventory_items.
The problems start when two different inventories have different parameters:
# Inventory item from inventory 1, televisions
{
inventory_id: 1
sku: 12345
name: Samsung LCD 40 inches
model: 582903-4
brand: Samsung
screen_size: 40
type: LCD
price: 999.95
}
# Inventory item from inventory 2, accomodation
{
inventory_id: 2
sku: 48cab23fa
name: New York Hilton
accomodation_type: hotel
star_rating: 5
price_per_night: 395
}
Since we obviously can't use brand or star_rating as the column name in inventory_items, our solution so far has been to use generic column names such as text_a, text_b, float_a, int_a, etc, and introduce a third table, inventory_schemas. The tables now look like this:
# Inventory schema for inventory 1, televisions
{
inventory_id: 1
int_a: sku
text_a: name
text_b: model
text_c: brand
int_b: screen_size
text_d: type
float_a: price
}
# Inventory item from inventory 1, televisions
{
inventory_id: 1
int_a: 12345
text_a: Samsung LCD 40 inches
text_b: 582903-4
text_c: Samsung
int_a: 40
text_d: LCD
float_a: 999.95
}
This has worked well... up to a point. It's clunky, it's unintuitive and it lacks scalability. We have to devote resources to set up inventory schemas. Using separate tables is not an option.
Enter NoSQL. With it, we could let each and every item have their own parameters and still store them together. From the research I've done, it certainly seems like a great alterative for this situation.
Specifically, I've looked at CouchDB and MongoDB. Both look great. However, there are a few other bits and pieces we need to be able to do with our inventory:
We need to be able to select items from only one (or several) inventories.
We need to be able to filter items based on its parameters (eg. get all items from inventory 2 where type is 'hotel').
We need to be able to group items based on parameters (eg. get the lowest price from items in inventory 1 where brand is 'Samsung').
We need to (potentially) be able to retrieve thousands of items at a time.
We need to be able to access the data from multiple applications; both backend (to process data) and frontend (to display data).
Rapid bulk insertion is desired, though not required.
Based on the structure, and the requirements, are either CouchDB or MongoDB suitable for us? If so, which one will be the best fit?
Thanks for reading, and thanks in advance for answers.
EDIT: One of the reasons I like CouchDB is that it would be possible for us in the frontend application to request data via JavaScript directly from the server after page load, and display the results without having to use any backend code whatsoever. This would lead to better page load and less server strain, as the fetching/processing of the data would be done client-side.
I work on MongoDB, so you should take this with a grain of salt, but this looks like a great fit for Mongo.
We need to be able to select items from only one (or several) inventories.
It's easy to ad hoc queries on any fields.
We need to be able to filter items based on its parameters (eg. get all items from inventory 2 where type is 'hotel').
The query for this would be: {"inventory_id" : 2, "type" : "hotel"}.
We need to be able to group items based on parameters (eg. get the lowest price from items in inventory 1 where brand is 'Samsung').
Again, super easy: db.items.find({"brand" : "Samsung"}).sort({"price" : 1})
We need to (potentially) be able to retrieve thousands of items at a time.
No problem.
Rapid bulk insertion is desired, though not required.
MongoDB has much faster bulk inserts than CouchDB.
Also, there's a REST interface for MongoDB: http://github.com/kchodorow/sleepy.mongoose
You might want to read http://chemeo.com/doc/technology, who dealt with the arbitrary property search problem with MongoDB.