In Ormlite, is it possible to make case-insensitive query without writing the actual SQL?
For example, if I am looking for
column name - "AccountName"
and query on the column, I would like to get results for all "Finance", "fInance", "FINANCE", etc. if I do search on "finance".
I was wondering if there's a functional support for this or if I need to write a SQL for it.
Thank you!
I am answering my own question, but the following seems to work.
newDao.query(newDao.queryBuilder().where().like("nameColumn", "finance")
.prepare())
Above seems to return all
"finance", "Finance", "FINANCE" or any other variations of it.
Right now (May 2011) there is no mechanism to do this with ORMLite except for writing the actual SQL and using the queryRaw() and other raw methods.
In many databases, MySQL for example, case insensitivity looks to be the default. But this is not the case with Postgresql nor Oracle.
A quick look around at the various database implementations shows that there is not a very easy and portable way to do this. Am I wrong?
Related
I've had this thought brewing for some time but I can't find anyone online who's discussed this as a possibility.
Currently the recommendations available for making case insensitive searches seem to be either to use "ilike" or "citext".
We're moving away from Microsoft Sql Server to Postgresql and all our code assumes case insensitive comparisons - but our TSQL code base is huge so changing it all to use UCASE() or ilike or citext etc etc isn't really feasible as a commercial development project.
However it must be possible to grab the source of postgresql and change some of the C code so that all string comparisons as case-insensitive, and then make our own compilation of the whole product. I think it would possibly require only a few lines of code to be changed and so upgradeability might not be a huge issue.
So I'm wondering whether anyone on here knows the Postgresql code base well enough to kick around ideas about whether this is feasible and whereabout the code is that does the comparisons just to help us get started. I'm continuing to research this in the meantime, and getting started with just being able to build postgresql on windows, but the hope is to bring others onboard with the idea such that a community project could be started, and as well as case insensitivity there might be other tweaks to allow tsql code to work better thus easing migration projects. My company would contribute to strongly.
Sorry if this is off topic but it seems to strongly lean towards being a developer question and I'm sure many other postgres users would appreciate a case insensitive build in this day and age -thanks
I understand your sentiment, but I believe that you are wrong to assume that this would be a simple change. Otherwise PostgreSQL would probably already have case insensitive collations...
I'd say that your best bet is to use citext throughout. What is the problem you have with that?
You should take this to the hackers list to start a serious discussion, but make sure you read the archives first, because the problem is not a new one.
I'd like to emulate this type of Solr query:
http://wiki.apache.org/solr/MoreLikeThis
with PostgreSQL using its full text search facility.
Is there a way to do something like a "more like this" query with pure postgres?
Not out of the box I am afraid. It might be possible to compare two tsvectors to determine if they are similar enough, or pull the top n similar tsvectors, but there is no out of the box functionality to do this. The good news is that since tsvectors support GIN indexing, the complicated part is done for you.
What I think you'd need to do is create a function in C which determines the intersection of two tsvectors. From there you could create a function which determines if they overlap and an operator which addresses this. From there it shouldn't be too hard to create a ranking based on largest overlap.
Of course I suspect that this will be easiest to do in a language like C but you could probably use other procedural languages as well if you need to.
The wonderful thing about PostgreSQL is that anything is possible. of course the downside is that when you move further from core functionality you get to do a lot of it yourself.
I have a large table with a text field, and want to make queries to this table, to find records that contain a given substring, using ILIKE. It works perfectly on small tables, but in my case it is a rather time-consuming operation, and I need it work fast, because I use it in a live-search field in my website. Any ideas would be appreciated...
Check Waiting for 9.1 – Faster LIKE/ILIKE blog post from depesz for a solution using trigrams.
You'd need to use yet unreleased Postgresql 9.1 for this. And your writes would be much slower then, as trigram indexes are huge.
Full text search suggested by user12861 would help only if you're searching for words, not substrings.
You probably want to look into full text indexing. It's a bit complicated, maybe someone else can give a better description, or you might try some links, like this one for example:
http://wiki.postgresql.org/wiki/Full_Text_Indexing_with_PostgreSQL
I am writing a custom server, which should be very performant.
It has 100.000-600.000 clients connected, and like 10 million records stored.
Database will run on a single server.
The server code is realized via twisted framework (in python).
Now I had it use MySQL, but I think a NoSQL database would be much more efficient (no complex queries, many simple writes / timestamp changes and many simple reads).
Which NoSQL database should I go for? Easy indexing would be a plus, I want the option to search the database from an administration system, create groups from logs containing a specific keyword and stuff like that.
I had a look at Cassandra and MongoDB, MongoDB seemed easier to get in / use for me.
Thanks for the help!
As far as pure learning curve goes, MongoDB has positioned itself to be a very friendly alternative to MySQL. Cassandra is a very different beast and will have a higher learning curve. That said, both have the potential to solve your problem based upon what you describe.
You have pretty simple requirements: easy indexing, arbitrary searches, grouping on keyword, etc -- pretty much every NoSQL system would work. It really comes down to the technologies with which you're comfortable. Like C#? Then go with RavenDB -- it can even automatically add indices as you execute queries. Like Erlang? Then you're a freak, but you should go with CouchDB. Like Javascript and JSON? Go with MongoDB.
Personally I really like Mongo, as it feels like a lovely hybrid of SQL and NoSQL databases. You can index the hell out of it (and get amazing performance!), which makes it almost like a RDBMS. You can also use it like a key/value store, and use it like a "giant hashtable in the sky". Still, YMMV. Play with them and see what works for you.
Cassandra is really designed for multiple server nodes, providing transparent replication. So you won't get the best value out of it with a single server host. Cassandra is also designed primarily for large-scale (and sacrifices indexing and flexible queries as a result). 10 million records isn't really very big, so you can afford to try something more flexible but less scalable.
I want to write some queries which can work in almost all the databases without any SQLExceptions. So, where can I get the ANSI standards to write the queries ?
Not sure that'll help you.
Vendors are touch and go as far as standards implementation and often the standards themselves are imprecise enough such that you could never write a query that would work with all implementors.
For example, SQL 92 defines the concatenation operator as || but neither MySQL nor MSSQL use this (Oracle does). Vendor independent string concatenation is impossible.
Similarly, a standard escape character is not specified so how you handled that might not work in all vendors.
Having said that:
SQL 92:
http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt
Wiki article with links to SQL 99 ISO documents:
http://en.wikipedia.org/wiki/SQL:1999
From wikipedia:
The SQL standard is not freely available. The whole standard may be purchased from the ISO as ISO/IEC 9075(1-4,9-11,13,14):2008.
Nevertheless I would not advise you to follow this strategy because no database engine follows any SQL standard (SQL 99, 2003, etc.) to the letter. All of them take liberties in the way they handle instructions or define variables (for example, when comparing two strings different engines handle case sensitivity differently). A method that is very efficient with one engine can be terrible inefficient for another.
A suggestion would be to develop a standard group of queries and develop different classes that contain the specific implementation of that query for a certain target RDBMS.
Hope this helped
Check out the BNF of the core SQL grammars available at http://savage.net.au/SQL/
This is part of the answer - the rest, as pointed out by Kiranu and MattMitchell, is that different vendors implement the standard differently. No DBMS adheres perfectly to even SQL-92, though most are pretty close.
One observation: the SQL standard says nothing about indexes - so there is no standard syntax for creating an index. It also says nothing about how to create a database; each vendor has their own mechanisms for doing that.
The Sql-92 standard is probably the one you want to target. I believe it's supported most of the major RDBMSs.
Here is a less terse link. Sample content:
PostgreSQL Has views. Breaks standard by not allowing updates to views...
DB2 Conforms to at least SQL-92.
MSSQL Conforms to at least SQL-92.
MySQL Conforms to at least SQL-92.
Oracle Conforms to at least SQL-92.
Informix Conforms to at least SQL-92.
Something else you might consider, if you're using .NET, is to use the factory pattern in System.Data.Common which does a good job of abstracting provider specifics for a number of RDBMSs.
If you are trying to make a product that will work against multiple databases I think trying to only use standard sql is not the way to go, as other answers have indicated, due to the different 'interpretations' of the standard. Instead you should if possible have some kind of data access layer in your application which has different implementations specific for each database. Depending on what you are trying to do, there are tools such as Hibernate which will so a lot of the heavy lifting in regards to this for you.