Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed yesterday.
Improve this question
We would like to use s3fs with the OCI Object Storage Service to store message files for Tibco Enterprise Message Service (EMS) which has these file system requirements:
Write Order: The storage solution must write data blocks to shared storage in the same order as they occur in the data buffer. (Solutions that write data blocks in any other order (for example, to enhance disk efficiency) do not satisfy this requirement.)
Synchronous Write Persistence: Upon return from a synchronous write call, the storage solution guarantees that all the data have been written to durable, persistent storage.
Distributed File Locking: The EMS servers must be able to request and obtain an exclusive lock on the shared storage. The storage solution must not assign the locks to two servers simultaneously. EMS servers use this lock to determine the primary server.
Unique Write Ownership: The EMS server process that has the file lock must be the only server process that can write to the file. Once the system transfers the lock to another server, pending writes queued by the previous owner must fail.
Does s3fs support these requirements?
Related
In no doubt that we can read files on the disk on Vxworks.
The question is whether it's a real-time API or not.
If the answer is not, one more question arises.
Is it possible to implement a real-time API to read the files on the disk?
Can you elaborate more on the context for the question and what you intend to address?
If you are asking if a read operation on a disk can be blocked, then the answer would be yes but it would depend on how the I/O layer is implemented for the driver being used.
In case you do not want to have a blocking operation on a read, you still be able to use select() to detect if an read operation can be performed on the device.
In addition you might use ioctl() commands to get information on the device or the file.
"real-time" OS means you have defined time-slices and monitoring of time effort, that's it.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
What is difference between Apache Kafka and GCP PubSub? when to use kafka and when to use pubsub.
Since you did not provide your use case, I will state below the main characteristics of each tool.
PubSub:
It is a cloud asynchronous messaging service that decouples senders and receivers provided by Google Cloud. It offers high availability and consistent performance at scale.
No Ops: in PubSub you do not need to worry about partitions and shards.
Scalability: is built-in without any required operation, it handles scalability automatically.
Monitoring: you can monitor your process at a Topic and Subscription level within StackDriver.
Access management: you can configure access at a project, Topic and Subscriber level.
Reliability: it guarantees the the message will be delivered at least once. Although, it does not guarantee ordering (which can be handled in Dataflow).
Message retention in PubSub: the minimum is 10 minutes and the maximum is 7 days.
Kafka:
It is an open-source distributed publish-subscribe messaging ecosystem. It can be used on-prem or deployed in cloud environments.
Scalability: it does not support automatic scalability. Thus, you need to increase partitions, replications etc, manually.
Ordering: it can support ordered messages in the partition level.
Reliability: it guarantees no data loss.
Monitoring: it offers various types of built-in monitoring systems.
Notice that I just shared the main characteristics of each tool. Although there are many others which can be more relevant for your use case. Here are some links where you can find other information about each one's aspects: 1, 2, 3.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
Imagine that we have a computer that does nothing except waiting data from network and does some calculating on it when it is received.
Right now I think that there is no other way to do this besides
periodically checking some processor register that indicates that data is ready and then do computing. In my understanding under the hood network card receives the data, put it to the memory and by itself write to the cpu register that cpu can do computing. After the next
periodically check processor will compute stuff. Can you make explanation of what actually is happening in details (on hardware and in os). This question worries me for ages!
Also will be glad to see any materials about it!
The method described in the question of the processor periodically checking some data register is used and it called polling. Polling is often employed in simpler systems because it's simple to implement, but it has the downside of wasting cpu cycles checking for data that may or may not be there.
In practice though most modern operating systems are going to use a cpu hardware feature called interrupts in conjunction with the OS's scheduler. In the example of getting data from the network, the program will use some blocking read call that will wait for data to be received from the network. Once the read call is made the calling program will stop executing completely and the OS's scheduler will keep the program in the suspended state until data is received.
Often, the way a program signals that it wants to wants to stop executing until something happens (like a packet is received) is with a software construct called a semaphore (or similar mutual exclusion mechanism). However, in the case of networking code the semaphore is not directly accessed by the programmer.
When a packet is received by the network card the network card will store the packet into RAM somewhere. Once this is completed the network card triggers an interrupt to the CPU. The interrupt (generally) causes the CPU to stop executing whatever program may have been currently running and the CPU executes kernel code related to handling network activity. In this case it sends the received packet to the program that was waiting to receive the data.
Finally, the kernel uses the semaphore to signal the packet was received. The kernel then has the scheduler "wake up" the program that was waiting for the data from the network. The program can process the data, and then await the next set of data from the network.
If you want to learn more I'd recommend reading op on interrupts, scheduling and semaphores. These technique though are used for more than just networking. Many other IO devices are accessed in with similar approaches.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
We are looking for a data logger to connect to a PLC through Modbus TCP or RTU. I have found several of these on the market, but I need the ability to post the data back to a web server. Basically we have a website that uses a graph to show the current values from the PLC over time and this data shouldn’t be more than a few seconds old. We have used a raspberry pi, but we are looking for alternatives for a more industrial environment.
Critical features
1. If the connection is lost, then the data that has been logged since the last connection should be sent up.
2. A backup of the logged data should be stored on the device
3. Use some type of frequent update mechanism to send data to the server like a html post.
I have only found one device and I wonder if Im using the wrong search terms/lingo or if COT devices do not exist with these features.
I did a similar project recently using a B&R Plc, and the AsHTTP library. It was able to do http put / get requests directly to a resource on the Web. (I've seen the term rest API used in the Web world) You could write your own code to buffer and store the data locally on flash memory in case it disconnects from the net.
Also B&R lets you use ModbusTCP for free directly through the ethernet port.
I've never used a "data logger" standalone device, but this is one option.
Investigate using an Omron NJ Series PLC with SQL Connection capabilities. This PLC could grab any data on another PLC through EtherNet/IP. From there, it has the ability to log to an SQL Server.
A very useful feature that I have made extensive use of is the spool function. If the connection between the NJ and the SQL Database is broken, the NJ will retain the data that wasn't logged in its own memory and insert it automatically once the connection is reestablished.
https://industrial.omron.ca/en/products/nj5-database-connection
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
Question 1:
When sites are mirrored, the content of their respective servers is synchronized (possibly automatically (live mirrors) or manually). Is this true? Are all servers 'equal', or does a main server exists? which then sends it changes to other 'children servers'? So all changes have to happen on the main server, and children servers are not allowed changes?
Question 2:
Expected advantages:
Global advantage: when a site that is originally hosted in the US is mirrored to a server in London, Europeans will benefit from this. They will have a better response time and because the amount of downloaders is cut down into two pieces (American and European servers) their download speeds can be higher.
Security: When one server crashes or is hacked, the other server can continue to operate normally.
Expected disadvantages:
If live mirroring is not used, some users will have to wait for renewed content.
More servers equals higher upkeep costs.
What other items can be added to these lists?
When sites are mirrored, the content of their respective servers is
synchronized. Is this true?
Yes, mirror sites should always be synchronized with their masters even if, for several reasons (eg. updates propagation times, network failures, etc.) they may not be.
There are several ways to achieve this; for example, a simple method could be using a rsync command in a cron job; a better solution is the "push mirroring" technique, used by the Debian and Ubuntu Linux distributions.
Are all servers 'equal', or does a main server exists, which then
sends it changes to other 'children servers'?
No, not all server are equals; generally the content provider updates one or more master servers which, in turn, provide the updated content to the other mirrors.
For example, in the Fedora infrastructure there are master servers, tier-1 servers (fastest mirrors) and tier-2 servers.
So all changes have to happen on the main server, and children servers
are not allowed changes?
Yes, in a mirrored context the content must be updated only on the master servers (one or more).
Expected advantages
Maybe the most comprehensive list of reasons for mirroring can be found on the Wikipedia:
To preserve a website or page, especially when it is closed or is about to be closed.
To allow faster downloads for users at a specific geographical location.
To counteract censorship and promote freedom of information.
To provide access to otherwise unavailable information.
To preserve historic content.
To balance load.
To counterbalance a sudden, temporary increase in traffic.
To increase a site's ranking in a search engine.
To serve as a method of circumventing firewalls.
Expected disadvantages
Cost: you have to buy additional servers and spend time to operate them.
Inconsistency: when one or more mirrors are not synchronized with the master (and this could happen not only with manual sync, but also with live sync).
As a further reference, since mirroring is a simple form of a Web Distributed System, you could also be interested in this reading.
Also, for files that are popular for downloading, a mirror helps reduce network traffic, ensures better availability of the Web site or files, or enables the site or downloaded files to arrive more quickly for users close to the mirror site. Mirroring is the practice of creating and maintaining mirror sites.
A mirror site is an exact replica of the original site and is usually updated frequently to ensure that it reflects the content of the original site. Mirror sites are used to make access faster when the original site may be geographically distant (for example, a much-used Web site in Germany may arrange to have a mirror site in the United States). In some cases, the original site (for example, on a small university server) may not have a high-speed connection to the Internet and may arrange for a mirror site at a larger site with higher-speed connection and perhaps closer proximity to a large audience.
In addition to mirroring Web sites, you can also mirror files that can be downloaded from an File Transfer Protocol server. Netscape, Microsoft, Sun Microsystems, and other companies have mirror sites from which you can download their browser software.
Mirroring could be considered a static form of content delivery.