Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I appreciate there are one or two similar questions on SO but we are a few years on and I know EF's speed and general performance has been enhanced so those may be out of date.
I am writing a new webservice to replace an old one. Replicating the existing functionality it needs to do just a handful of database operations. These are:
Call existing stored procedures to get data (2)
Send SQL to the database to be executed (should be stored procedures I know) (5)
Update records (2)
Insert records (1)
So 10 operations in total. The database is HUGE but I am only dealing with 3 tables directly (stored procedures do some complex JOINs).
When getting the data I build an array of objects (e.g. Employees) which then get returned by the web service.
From my experience with Entity Framework, and because I'm not doing anything clever with the data, I believe EF is not the right tool for my purpose and SqlDataReader is better (I imagine it is going to be lighter and faster).
Entity Framework focuses mostly on developer productivity - easy to use, easy to get things done.
EF does add some abstraction layers on top of "raw" ADO.NET. It's not designed for large-scale, bulk operations, and it will be slower than "raw" ADO.NET.
Using a SqlDataReader will be faster - but it's also a lot more (developer's) work, too.
Pick whichever is more important to you - getting things done quickly and easily (as a developer), or getting top speed by doing it "the hard way".
There's really no good "one single answer" to this "question" ... pick the right tool / the right approach for the job at hand and use it.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I have a data-set that consist of 1500 columns and 6500 rows and I am trying to figure out what the best way is to shape the data for web based user interactive visualizations.
What I am trying to do is make the data more interactive and create an admin console that allows anyone to filter the data visually.
Front-end could potentially be based on Crossfilter, D3 and DC.js and give the user basically end-less filtering possibilities(date, value, country. In addition there will be some pre defined views like top and bottom 10 values.
I have seen and tested some great examples like this one, but after testing it did not really fit for the large amount of columns I had and it was based on a full JSON dump from the MongoDB. This amounted in very long loading times and loss of full interactivity with the data.
So in the end my question is what is the best approach (starting with normalization) in getting the data shaped in the right way so it can be manipulated from a front-end. Changing the amount of columns is a priority.
A quick look at the piece of data that you shared suggests that the dataset is highly denormalized. To allow for querying and visualization from a database backend I would suggest normalizing. This is no small bit software work but in the end you will have relational data which is much easier to deal with.
It's hard to guess where you would start but from the bit of data you showed there would be a country table, an event table of some sort and probably some tables of enumerated values.
In any case you will have a hard time finding a db engine that a lows that many columns. The row count is not a problem. I think in the end you will want a db with dozens of tables.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm trying to design an application in which a user can make multiple operations like add, delete.. Work stations or Applications from database. How do I design it?
To deal this, I've found two solutions that I couldn't be able to choose the best:
1st solution, 2nd solution.
Is this right?
Any brilliant suggestion, please?
Thanks a lot!
To think this out a bit, a user can have one or more operations i.e. a one to many relationship with the operation.
An operation can delete one or more Workstations or Applications, again a one to many relationships.
Therefore, I think your 1st solution captures it nicely.
I think solution 1 is better. Solution 2 requires that you insert 2 records in the 2 many-to-many associative tables, for each single operation. This is more complex, and probably unnecessarily so.
In solution 1 the Operation table becomes the only associative table. 1 operation, 1 insert. You may have to have to make some of the referencing keys nullable, depending on your requirements, but this is manageable. Simpler and sufficient for the needs you expressed.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
Is this a perpetually denied feature request or something specific to Postgres?
I've read that Postgre has potentially higher performance than InnoDB but also potentially larger chance of less serialization (I apologize that I don't have a source, and please give that statement a wide berth because of my noobity and memory) and wonder if it might have something to do with that.
Postgres is amazingly functional compared to MySQL, and that's why I've switched. Already I've cut down lines of code & unnecessary replication immensely.
This is just a small annoyance, but I'm curious if it's considered unnecessary because of the UPDATE then INSERT workaround, or if it's very difficult to develop (possibly vs the perceived added value) like boost::lockfree::queue's ability to pass "anything" (, or if it's something else).
PostgreSQL committers are working on a patch to introduce "INSERT ... ON DUPLICATE KEY", which is functionally equivalent to an "upsert". MySQL and Oracle already have functionality (in Oracle it is called "MERGE")
A link to the PostgreSQL archives where the functionality is discussed and a patch introduced: http://www.postgresql.org/message-id/CAM3SWZThwrKtvurf1aWAiH8qThGNMZAfyDcNw8QJu7pqHk5AGQ#mail.gmail.com
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I would like to add some more tokens such as OPEN into PostgreSQL, what procedure should I follow? I did not find corresponding documents. thanks.
(Assuming you mean "the postgresql server" not "the command line client psql and that by "Token" you mean "SQL command / statement type"):
... yeah, that's not super simple.
If it's a utility command that does not require query planning it isn't super hard. You can take the existing utility commands as guidance on how they work. They're all quite different though. Start with ProcessUtility.
If it's intended to produce a query plan, like SELECT, INSERT, UPDATE, DELETE, CREATE TABLE AS, etc ... well, that tends to be a lot more complicated.
This sort of thing will require some quality time reading the PostgreSQL source code and developer documentation. It's way too complex to give you a step-by-step how-to here, especially since you have not even explained what the command you wish to add is supposed to do.
If at all possible you should develop the functionality you need as a user defined function first. Start with PL/PgSQL, PL/Perl, or whatever, and if you hit the limitations of that develop it as a C extension.
Once you have all the functionality you want as C functions, then think about whether it makes sense to extend the actual SQL syntax.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I'm starting out to design a site that has some requirements that I've never really dealt with. Specifically, the data objects will have similar, but not exact, attributes. Yes, I could probably figure out most of the possible attributes and then just not populate the ones that don't make sense, and therefore keep a traditional "Relational" table and column design, but I'm thinking this might be a really good time to learn NoSQL.
In addition, the user will have 1, and only 1, textbox to search, and I will need to search all data objects and their attributes to find that string.
Ideally, I'd like to have the search return in order of "importance", meaning that if a match for the user's entered string is found in a "name" attribute, it would be returned as a higher confidence match than if the string was matched on a sub-attribute.
Anyone have any experience in this sort of situation? What have you tried that worked or didn't work? Am I wrong in thinking that this project is very well suited to a NoSQL type of database?
Stick with a traditional relational database such as MySQL or Postgresql. I would suggest sorting by relevance in your application code after obtaining the matching results. The size of your result set should impact your design choices, but if you will have less than 1-2k results then just keep it simple and don't worry too much about optimization.
NoSQL is just a dumb key value store, a persistent dictionary that can be shared across multiple application instances. It can solve scalability issues, but introduces new ones since you now just have a dumb data store. Relational databases have had years of performance tuning and do a great job.
I find NoSQL to be much more suited to storing state data, like a users preferences or cache. If you are analyzing the relationship between data then you need a relational database.