I've got a Postgres ORDER BY issue with the following table:
em_code name
EM001 AAA
EM999 BBB
EM1000 CCC
To insert a new record to the table,
I select the last record with SELECT * FROM employees ORDER BY em_code DESC
Strip alphabets from em_code usiging reg exp and store in ec_alpha
Cast the remating part to integer ec_num
Increment by one ec_num++
Pad with sufficient zeors and prefix ec_alpha again
When em_code reaches EM1000, the above algorithm fails.
First step will return EM999 instead EM1000 and it will again generate EM1000 as new em_code, breaking the unique key constraint.
Any idea how to select EM1000?
Since Postgres 9.6, it is possible to specify a collation which will sort columns with numbers naturally.
https://www.postgresql.org/docs/10/collation.html
-- First create a collation with numeric sorting
CREATE COLLATION numeric (provider = icu, locale = 'en#colNumeric=yes');
-- Alter table to use the collation
ALTER TABLE "employees" ALTER COLUMN "em_code" type TEXT COLLATE numeric;
Now just query as you would otherwise.
SELECT * FROM employees ORDER BY em_code
On my data, I get results in this order (note that it also sorts foreign numerals):
Value
0
0001
001
1
06
6
13
۱۳
14
One approach you can take is to create a naturalsort function for this. Here's an example, written by Postgres legend RhodiumToad.
create or replace function naturalsort(text)
returns bytea language sql immutable strict as $f$
select string_agg(convert_to(coalesce(r[2], length(length(r[1])::text) || length(r[1])::text || r[1]), 'SQL_ASCII'),'\x00')
from regexp_matches($1, '0*([0-9]+)|([^0-9]+)', 'g') r;
$f$;
Source: http://www.rhodiumtoad.org.uk/junk/naturalsort.sql
To use it simply call the function in your order by:
SELECT * FROM employees ORDER BY naturalsort(em_code) DESC
The reason is that the string sorts alphabetically (instead of numerically like you would want it) and 1 sorts before 9.
You could solve it like this:
SELECT * FROM employees
ORDER BY substring(em_code, 3)::int DESC;
It would be more efficient to drop the redundant 'EM' from your em_code - if you can - and save an integer number to begin with.
Answer to question in comment
To strip any and all non-digits from a string:
SELECT regexp_replace(em_code, E'\\D','','g')
FROM employees;
\D is the regular expression class-shorthand for "non-digits".
'g' as 4th parameter is the "globally" switch to apply the replacement to every occurrence in the string, not just the first.
After replacing every non-digit with the empty string, only digits remain.
This always comes up in questions and in my own development and I finally tired of tricky ways of doing this. I finally broke down and implemented it as a PostgreSQL extension:
https://github.com/Bjond/pg_natural_sort_order
It's free to use, MIT license.
Basically it just normalizes the numerics (zero pre-pending numerics) within strings such that you can create an index column for full-speed sorting au naturel. The readme explains.
The advantage is you can have a trigger do the work and not your application code. It will be calculated at machine-speed on the PostgreSQL server and migrations adding columns become simple and fast.
you can use just this line
"ORDER BY length(substring(em_code FROM '[0-9]+')), em_code"
I wrote about this in detail in this related question:
Humanized or natural number sorting of mixed word-and-number strings
(I'm posting this answer as a useful cross-reference only, so it's community wiki).
I came up with something slightly different.
The basic idea is to create an array of tuples (integer, string) and then order by these. The magic number 2147483647 is int32_max, used so that strings are sorted after numbers.
ORDER BY ARRAY(
SELECT ROW(
CAST(COALESCE(NULLIF(match[1], ''), '2147483647') AS INTEGER),
match[2]
)
FROM REGEXP_MATCHES(col_to_sort_by, '(\d*)|(\D*)', 'g')
AS match
)
I thought about another way of doing this that uses less db storage than padding and saves time than calculating on the fly.
https://stackoverflow.com/a/47522040/935122
I've also put it on GitHub
https://github.com/ccsalway/dbNaturalSort
The following solution is a combination of various ideas presented in another question, as well as some ideas from the classic solution:
create function natsort(s text) returns text immutable language sql as $$
select string_agg(r[1] || E'\x01' || lpad(r[2], 20, '0'), '')
from regexp_matches(s, '(\D*)(\d*)', 'g') r;
$$;
The design goals of this function were simplicity and pure string operations (no custom types and no arrays), so it can easily be used as a drop-in solution, and is trivial to be indexed over.
Note: If you expect numbers with more than 20 digits, you'll have to replace the hard-coded maximum length 20 in the function with a suitable larger length. Note that this will directly affect the length of the resulting strings, so don't make that value larger than needed.
Query:
Select x classifier from tabx x
returns rows with one column named 'classifier' with list of values from the row.
Does anybody know where such features are documented? Is it the only usage of the keyword? I tried to google it but I've found only list of postgresql reserved keywords without explanation.
Example
It works like String_agg with ',' delimiter, but for a row.
It's called a column alias.
For columns it's documented here and for tables it's documented here
The name of the alias is irrelevant and bears no special meaning (including the word "classifier"). It just has to be a valid SQL identifier. Using an alias won't change anything in the resulting data.
If you use the (optional) AS keyword, it might be a bit more obvious:
select some_column as something_else
from some_table as another_name;
The reference x refers to the table alias specified in the FROM clause and refers to the complete row/record. It's not the column alias ("classifier") that does this, it's the reference to the table.
This behaviour is documented here in the manual
Edit after a new query was shown.
select a classifier, a.classifier
from t_attribute a;
The part a classifier still gives an alias to the record column (as explained above). a.classifier simply accesses a column from the table t_attribute.
It could also have been written as:
select a as the_complete_row, a.classifier as classifier
from t_attribute a;
I have a local copy of Postgres running, and I'm working on a C# .Net Core 2.1 app using nHibernate as an ORM.
It's started throwing an exception: PostgresException: 42703: column this_.datasetname does not exist
When I copy the SQL and run in pgAdmin I get a similar error.
This is a short version of the SQL which gives the same error:
SELECT this_.datasetName FROM orders this_
ERROR: column this_.datasetname does not exist LINE 1: SELECT
this_.datasetName FROM orders this_
^ HINT: Perhaps you meant to reference the column "this_.datasetName". SQL state: 42703 Character: 8
If I add quotes around the column name (but not around _this. as suggested) it works, but obviously I can't tell nHibernate to do that.
SELECT this_."datasetName" FROM orders this_
The following also works fine:
SELECT "datasetName" FROM orders
Why would it insist on adding the quotes? It never used to.
That's because this column was created with the surrounding double quotes in the first place - this makes the identifier case-sensitive, while by default it isn't. Since the identifier contains mixed case, you are stucked: the identifier needs to be quoted everywhere you use it.
If you look at the definition of the table, you will see something like:
create table orders (
...,
"datasetName" text,
...
)
I would strongly suggest fixing your schema. Quoted identifiers add no value in general, while on the other hand they make things unnecessary complicated. Camel case does not fill well for database identifiers, snake case is better, since case is not meaningful:
create table orders (
...,
dataset_name text,
...
)
The software I used created tables in postgres with Capitalizations, and I know that postgres and caps are a pain to deal with. I am using a multi-table query but they have Caps and I am not sure how to get the query down correctly to make it work.
I have two databases TBLS and DBS. I want to get the column TBL_NAME where the two DB_ID's are the same.
Here is what I thought might work:
select '"t.TBL_NAME"' from "TBLS" t, "DBS" d where '"t.DB_ID"'='"d.DB_ID"';
Any way I try and place the " or ' I can't seem to get the query to work correctly.
"tablename"."columnname"
See the manual on SQL syntax.
'"X.COL"' is a string literal with content string "X.COL".
"X.COL" is a single unqualified identifier for the object named X.COL. Yes, table, column, etc names can have a . in them.
"X"."COL" is a qualified identifier for COL in object X. Depending on context it can mean "the table COL in schema X", "the column COL in table X", etc. This is the one you want.
I wonder if someone could have an idea what is going wrong with this simple query on a hstore column in PostgreSQL 9.2
The queries are runned in pgAdmin
select attributeValue->"CODE_MUN" from shapefile_feature
returns: « attributevalue » column does not exists
when doing:
select * from shapefile_feature;
all the columns are returned including attributeValue, the hstore column
what is the problem?
PostgreSQL distinguish between "identifiers" and 'literal'. Identifiers are schema, table, column's, .. names, literals are others. A attribute in hstore are not SQL identifiers. So you have to pass their names as literals. Operator "->" is only shortcut for function "fetchval(hstore, text)" with possibility be indexed.
select attributeValue->'CODE_MUN' from shapefile_feature
is internally transformed to (don't do this transformation by self!)
select fetchval(attributeValue, 'CODE_MUN') from shapefile_feature
on buggy example in transformed form, you can better understand to error message:
select fetchval(attributeValue, "CODE_MUN") from shapefile_feature
PostgreSQL tries to find column "CODE_MUN" in shapefile_feature, bacause used double quotes means identifiers (in case sensitive notation).