Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
Are Solaris kernel stats unix:0:vopstats_zfs:* system (zpools) wide? I mean are those stats for all zpools present on system? I see sometimes huge differences between disks operations vs. ZFS operations and can't judge what makes that difference. For example ZFS write operations shows something between 2-3k operations per second, but disks shows more than 17k operations. On some periods those operations circa do match (if I take also L2ARC into account), but sometime they are totally divergent. I'm doing monitoring on global zone... is it possible, that something is hidden inside zones? It's one kernel so I expect same numbers in global zone and zones. System do have only ZFS, no UFS. No scrub was requested at that time.
That depends on what Solaris release you are using.
From Solaris 11 onward, file system statistics are reported per zone while they are reported globally with Solaris 10 and older.
unix:0:vopstats_zfs:* only report the global zone activity.
unix:xx:vopstats_zfs:* will report the statistics for the zone with id xx.
You do not state if you parse the kstat(1M) command line output or if you use the libkstat API. Note that you can use the -z zone or -Z (all zones) fsstat options through the CLI.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Closed 9 years ago.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Improve this question
I'm working on the architecture of a new system to replace an ancient mainframe app. The mainframe uses IBM IMS and is surprisingly fast with large amounts of data. We've tried 3 DBs so far - MongoDB, SQL Server and Oracle, but they performed poorly under load. We hired an Oracle consultant and a 128 cores server and Oracle still gives us 4x the response time of the old system (same with SQL Server).
Are there any modern hierarchical DBs, that can efficiently support billions of records?
Mainframes have been and remain very fast for certain use cases, so part one is not to assume that mainframe = bad. Having said that, they can be very expensive to maintain, and particularly with legacy apps the skills are starting to evaporate.
If you really wanted a hierarchical database, one valid option would be to modernise your application but retain IMS at the core. IMS is a great hierarchical database, and I don't think IBM are going to EOL IMS any time soon, so is there a real reason to go to a hierarchical database that isn't IMS? A quick visit to their website gave me the impression that they'd discount the product if they thought you were going to migrate to a competing product, so if money is the problem then perhaps the answer is to just ask IBM to discount the product you're already happy with. This white paper (ftp://public.dhe.ibm.com/software/data/ims/pdf/TCG2013015LI.pdf) suggests they're pushing that as an option, and no doubt the later versions of IMS have a bunch of features that might not be available in the version you're running (assuming you've not upgraded to the latest).
I'm surprised you can't get the performance you want out of Oracle though, the system I'm currently working on has a couple of tables at the billion mark and we definitely don't have 128 cores, but we get reasonable performance.
My first question is whether your Oracle consultant really knew their stuff. I've had mixed results, I guess like any skill set people can have variable skills. I often find that when you get performance problems it's because people have over-normalised or over-generalised the database schema - so you've moved from a highly optimised hierarchical structure in IMS that flies to a very abstracted structure in 3NF, and that dies. But sometimes if you put that same hierarchical structure in Oracle, and only allow the same sort of access patterns that were possible in IMS, you'd get all the performance you want.
By that, I mean if in IMS you had clients, clients had orders, and orders had order lines, then I think that means it's pretty hard to do any accesses without starting at the client. It also often means you have large batch processes that process all the clients every day to find out which have orders that you need to do something with.
So, some things here. Firstly, if, in Oracle, you were to build that structure - so I have a client id, the client id is the first element in the primary key of orders, and the client id then the order id are the first two elements in the primary key of order lines, and then I use client id as my clustering key and put client id into every index......probably all my client-based access paths will be really fast. You can also partition by client id, and if needed, run an Oracle RAC cluster with each of those partitions/client ranges effectively running as separate databases on a separate more commodity class machine (say, a dual socket machine = about 20 cores).
Secondly, if I used to have to process all my records once a night to find the orders that needed someone to work on them, then in the new relational world I don't need to do that any more, I just need to find the orders with a status of "pending" or whatever. So maybe Oracle isn't as fast for that batch oriented workload, but if I change my logic and do an indexed query for pending orders, then again I can get all the performance I want. Even more so, perhaps I make order_status into a partitioning key, so my "active" records are all in one partition, and all the older orders are in other partitions - and then I put that partition on an SSD-backed array.
Thirdly, take a look at your storage devices. Performance problems in databases are invariably IO problems - either you're doing too much IO (poorly optimised queries), or your IO subsystem can't keep up with the IO that you need to do. 128 cores is an awful lot of compute, and I've rarely seen a database that is compute bound. Maybe look at a big SSD array, some of them can give you enormous IO throughput. Certainly if you were running Oracle on a RAID 5 spinning disk array your performance is likely to suck.
The last random comment here - a lot of people are getting good results with SAP HANA - a fully in-memory database. That really flies, and is specifically designed for workloads that just won't run fast enough in other databases. I bet SAP would come demo it to you for free if you wanted it.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Is this a perpetually denied feature request or something specific to Postgres?
I've read that Postgre has potentially higher performance than InnoDB but also potentially larger chance of less serialization (I apologize that I don't have a source, and please give that statement a wide berth because of my noobity and memory) and wonder if it might have something to do with that.
Postgres is amazingly functional compared to MySQL, and that's why I've switched. Already I've cut down lines of code & unnecessary replication immensely.
This is just a small annoyance, but I'm curious if it's considered unnecessary because of the UPDATE then INSERT workaround, or if it's very difficult to develop (possibly vs the perceived added value) like boost::lockfree::queue's ability to pass "anything" (, or if it's something else).
PostgreSQL committers are working on a patch to introduce "INSERT ... ON DUPLICATE KEY", which is functionally equivalent to an "upsert". MySQL and Oracle already have functionality (in Oracle it is called "MERGE")
A link to the PostgreSQL archives where the functionality is discussed and a patch introduced: http://www.postgresql.org/message-id/CAM3SWZThwrKtvurf1aWAiH8qThGNMZAfyDcNw8QJu7pqHk5AGQ#mail.gmail.com
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I have a need to collect snmptraps and display them in a web interface. The application already includes
*ruby on rails
*linux
*delayed_job (for queueing)
*postgresql
*A few cron jobs that do snmp queries
Now I need to run something like snmptrapd to collect alarms. Would it be possible for snmptrapd to write its traps to a queue that I can process with a cron job. Something like the built in mqueue of linux would be great. Or even writing it to a postgresql database (I know it supports mysql but no mention of postgres anywhere)
Does anyone know how I can redirect the output of snmptrapd into something I can process with a cron job.
I did something similar in Perl but you can do that with Ruby as well.
First you need to tell snmptrapd who is the default handler for traps. In the snmptrapdconf you can define it as the following:
traphandle default /yourpluginpath/yourplugin
Now every time a trap occours its data will be sent to yourplugin. It's up to him now to handle it. Once you have the packet you can store it to any DB you want, doesn't matter if it is MySQL or PostgreSQL.
The only problem is the library that parses the data that comes from the trap. In Perl I used SNMP::Trapinfo but since I nevery used Ruby I don't know the equivalent, I am sure someone else can point you to the right library. You can even parse it yourself. Actually you can also use a basic shell script if you wish someone else to store the data to the DB.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I want to use a persistent HashTable to provide data storage for my application. Is this possible. A large well supported open-source project would be ideal.
You have 2 options :
a) Seralize your Hashtable to file -- after all the Hashtable class implements Serializable.
b) BerkeleyDB Java Edition -- you can download this for free from Oracle. It is open source. Berkeley DB database is a b-tree. It is fairly straight forward to convert your code from HashTable to Berkeley DB .
Note that if you use simple Hashtable for storing your objects, you will run out out memory when the number of obects in Hashtable increases beyond a certain number. With Berkeley DB, there is no such limitation.
Chronicle Map is an off-heap key-value store for Java, providing ConcurrentMap interface and (optionally) persists data to disk. Under the hood, it's implemented via memory-mapped files.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I have 2 computers which are connected to each other via serial comunication.
The main computer is holding a DB (about 10K words) the computer is working at a 20Hz rate.
I need real-time synchronization of the DB for the other computer - if data is added, deleted, or updated, I want the other computer to see or get the changes in real-time.
If I will transfer whole the DB peirodicly it will take about 5 seconds to update the other side - which is not acceptable.
Spmeone has an idea?
As you said, the other computer has to get the changes (i.e. insert, delete, update) via the serial link.
The easiest way to do this (but maybe impossible, if you can't change certain things) is to extend the database-change methods (or, if thats not possible: every call) to send an insert/delete/update-datagram with all required data over the serial link, which has to be robust against packet-loss (i.e. error detection, retransmission, etc.).
On the other end you have to implement a semantically equivalent database where you replay all the received changes.
Of course you still have to synchronize the databases at startup/initialization or maybe periodically (e.g. once per day).