I'm using postgres 11.5 and try to do the following:
update ccnc set fulltext_tokens = to_tsvector(title || '. ' || description || '. ' || text ) where fulltext_tokens is NULL;
Which results in
ERROR: column " text" does not exist
LINE 1: ...o_tsvector(title || '. ' || description || '. ' || text ) wh...
^
HINT: Perhaps you meant to reference the column "ccnc.text".
However, neither does using ccnc.text help:
felix=# update ccnc set fulltext_tokens = to_tsvector(title || '. ' || description || '. ' || ccnc.text) where fulltext_tokens is NULL;
ERROR: missing FROM-clause entry for table " ccnc"
LINE 1: ...o_tsvector(title || '. ' || description || '. ' || ccnc.text...
... nor is something odd with the respective column (no trailing spaces or the like):
felix=# \d+ ccnc
Table "public.ccnc"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+-----------------------------+-----------+----------+----------------------------------+----------+--------------+-------------
id | integer | | not null | nextval('ccnc_id_seq'::regclass) | plain | |
description | text | | | ''::text | extended | |
text | text | | | | extended | |
title | text | | | | extended | |
Edit:
Also quoting does not help, e.g.:
update ccnc set fulltext = title || '. ' || description || '. ' || "text";
ERROR: syntax error at or near ""text""
LINE 1: ... fulltext = title || '. ' || description || '. ' || "text";
I'd greatly appreciate any help on how to create that new column named fulltext_tokens. Thank you in advance :-)
Despite the very valuable comment by Richard, i.e., not using a column name such as "text" in the first place, one way to avoid the error is to use concat or concat_ws instead of the concat operator ||.
For instance:
update ccnc set fulltext = concat_ws('. ', title, description, text)
Related
Wow, this was a terribly worded query, let me try again.
I'm still learning antlr and trying to understand grammars. I'm using a grammar (not written by me - so I'm trying not to adjust it too much as it's the standard used by many groups, found here).
I'm using it in a Flutter application. When I run it on Linux or Android, it runs without issue. When I try and run it no web, I immediately have issues. The full grammar I'm using is below.
grammar FhirPath;
// Grammar rules [FHIRPath](http://hl7.org/fhirpath/N1) Normative Release
//prog: line (line)*; line: ID ( '(' expr ')') ':' expr '\r'? '\n';
entireExpression: expression EOF;
expression:
term # termExpression
| expression '.' invocation # invocationExpression
| expression '[' expression ']' # indexerExpression
| ('+' | '-') expression # polarityExpression
| expression ('*' | '/' | 'div' | 'mod') expression # multiplicativeExpression
| expression ('+' | '-' | '&') expression # additiveExpression
| expression '|' expression # unionExpression
| expression ('<=' | '<' | '>' | '>=') expression # inequalityExpression
| expression ('is' | 'as') typeSpecifier # typeExpression
| expression ('=' | '~' | '!=' | '!~') expression # equalityExpression
| expression ('in' | 'contains') expression # membershipExpression
| expression 'and' expression # andExpression
| expression ('or' | 'xor') expression # orExpression
| expression 'implies' expression # impliesExpression;
//| (IDENTIFIER)? '=>' expression #lambdaExpression
term:
invocation # invocationTerm
| literal # literalTerm
| externalConstant # externalConstantTerm
| '(' expression ')' # parenthesizedTerm;
literal:
'{' '}' # nullLiteral
| ('true' | 'false') # booleanLiteral
| STRING # stringLiteral
| NUMBER # numberLiteral
| DATE # dateLiteral
| DATETIME # dateTimeLiteral
| TIME # timeLiteral
| quantity # quantityLiteral;
externalConstant: '%' ( identifier | STRING);
invocation: // Terms that can be used after the function/member invocation '.'
identifier # memberInvocation
| function # functionInvocation
| '$this' # thisInvocation
| '$index' # indexInvocation
| '$total' # totalInvocation;
function: identifier '(' paramList? ')';
paramList: expression (',' expression)*;
quantity: NUMBER unit?;
unit:
pluralDateTimePrecision
| dateTimePrecision
| STRING; // UCUM syntax for units of measure
pluralDateTimePrecision:
'years'
| 'months'
| 'weeks'
| 'days'
| 'hours'
| 'minutes'
| 'seconds'
| 'milliseconds';
dateTimePrecision:
'year'
| 'month'
| 'week'
| 'day'
| 'hour'
| 'minute'
| 'second'
| 'millisecond';
typeSpecifier: qualifiedIdentifier;
qualifiedIdentifier: identifier ('.' identifier)*;
identifier:
IDENTIFIER
| DELIMITEDIDENTIFIER
| 'as'
| 'is'
| 'contains'
| 'in'
| 'div';
/****************************************************************
Lexical rules ***************************************************************
*/
/*
NOTE: The goal of these rules in the grammar is to provide a date token to the parser. As such it
is not attempting to validate that the date is a correct date, that task is for the parser or
interpreter.
*/
DATE: '#' DATEFORMAT;
DATETIME:
'#' DATEFORMAT 'T' (TIMEFORMAT TIMEZONEOFFSETFORMAT?)?;
TIME: '#' 'T' TIMEFORMAT;
fragment DATEFORMAT:
[0-9][0-9][0-9][0-9] ('-' [0-9][0-9] ('-' [0-9][0-9])?)?;
fragment TIMEFORMAT:
[0-9][0-9] (':' [0-9][0-9] (':' [0-9][0-9] ('.' [0-9]+)?)?)?;
fragment TIMEZONEOFFSETFORMAT: (
'Z'
| ('+' | '-') [0-9][0-9]':' [0-9][0-9]
);
IDENTIFIER: ([A-Za-z] | '_') ([A-Za-z0-9] | '_')*;
// Added _ to support CQL (FHIR could constrain it out)
DELIMITEDIDENTIFIER: '`' (ESC | ~[\\`])* '`';
STRING: '\'' (ESC | ~['])* '\'';
// Also allows leading zeroes now (just like CQL and XSD)
NUMBER: [0-9]+ ('.' [0-9]+)?;
// Pipe whitespace to the HIDDEN channel to support retrieving source text through the parser.
WS: [ \r\n\t]+ -> channel(HIDDEN);
COMMENT: '/*' .*? '*/' -> channel(HIDDEN);
LINE_COMMENT: '//' ~[\r\n]* -> channel(HIDDEN);
fragment ESC:
'\\' ([`'\\/fnrt] | UNICODE); // allow \`, \', \\, \/, \f, etc. and \uXXX
fragment UNICODE: 'u' HEX HEX HEX HEX;
fragment HEX: [0-9a-fA-F];
I generate the code with the following:
antlr4 -Dlanguage=Dart FhirPath.g4 -visitor -no-listener
Then to test I use the following code:
final input = InputStream.fromString('name');
final lexer = FhirPathLexer(input);
final tokens = CommonTokenStream(lexer);
final parser = FhirPathParser(tokens);
parser.buildParseTree = true;
final tree = parser.expression();
If I run it in a simple dart script, it runs without issue. But if I put it in a Flutter application (again, only on web, otherwise it appears to run without issue), I get this error:
line 1:0 mismatched input 'name' expecting {'as', 'in', 'is', 'contains', 'div', 'mod', IDENTIFIER, DELIMITEDIDENTIFIER}
I assume there's something I don't understand about the grammar, so any insights would be appreciated.
I've concluded this is an error with transpiling to javascript. The antlr4 version works for all of my tests for Android and Linux, then throws a bunch of errors for web. I've gone back to using petitparser instead of antlr4. If anyone else has suggestions feel free to leave them, but for now, I'm going to close this. If you want to compare how the two look, I have both versions here: https://github.com/MayJuun/fhir/tree/main/fhir_path/lib
I am trying to create shortcuts for DBAs/admins in psql.
I have created some functions and queries, which give me information such as schema size in bytes, or a list of tables in order of storage used, etc. These will be used whenever an admin or DBA wants to get some info about the database.
I can run these queries naturally with select, or as a function like get_size(), but I want them to be accessible as shortcuts, similar to the native backslash commands (\dx, \dt, etc).
So I have used psql's \set feature to store queries/functions as variables, which I will put in the psqlrc file:
\set size 'select pg_size_pretty(my_size_function(''public''));'
Then when I type :size in psql I would get the size of the "public" schema.
What I want though, is to be able to dynamically pass a schema name, so I could run things like
:size public, :size schema2 etc.
I tried changing the \set to: \set size 'select pg_size_pretty(my_size_function(:schema));', but I can only call that by executing \set schema '''public''' first.
Since the whole point is to use these universally as shortcuts, having to manually run \set commands each time defeats the purpose.
In Oracle the would be colon a bind variable, which would be read at runtime.
How can I do this with psql?
I use these ways.
Postgres Functions (Inside Postgres CLI)
postgres=# CREATE OR REPLACE FUNCTION getSize(tableName varchar) RETURNS varchar LANGUAGE SQL as
postgres-# $$
postgres$# SELECT pg_size_pretty(pg_relation_size(tableName));
postgres$# $$;
CREATE FUNCTION
postgres=# select getSize('test');
getsize
------------
8192 bytes
(1 row)
From Shell (Outside Postgres CLI)
psqlgettablesize(schema, tablename) - Get table size. Pass all for schema argument to get for all schemas.
$ psqlgettablesize customer events
table_name | total_size | table_size | index_size
--------------------------+---------------------+-------------+---------------
test(complete database) | 19039892127 (18 GB) | |
--------- | --------- | --------- | ---------
customer.events | 24576 (24 kB) | 0 (0 bytes) | 24576 (24 kB)
(3 rows)
psqlgettablecount(schema, tablename) - Get count of rows in a table, in a schema
$ psqlgettablecount customer events
count
----------
51850000
(1 row)
psqlgetvacuumdetails(schema, tablename) - Get vacuum details of a table, in a schema
$ psqlgetvacuumdetails customer events
schemaname | relname | n_live_tup | n_dead_tup | last_analyze | analyze_count | last_autoanalyze | autoanalyze_count | last_vacuum | vacuum_count | last_autovacuum | autovacuum_count
------------+-----------+------------+------------+----------------------------+---------------+----------------------------+-------------------+---------------------------+--------------+-----------------+------------------
customer | events | 0 | 0 | 2019-12-02 18:25:04.887653 | 2 | 2019-11-27 18:49:19.002405 | 1 | 2019-11-29 13:11:15.92002 | 1 | | 0
(1 row)
psqltruncatetable(schema, tablename) - Truncate a table, in a schema, after authorization.
$ psqltruncatetable customer events
Are you sure to truncate table 'customer.events' (y/n)? y
Time: 4.944 ms
psqlsettings(category) - Get settings of Postgres
$ psqlsettings Autovacuum
name | setting | unit | category | short_desc | extra_desc | context | vartype | source | min_val | max_val | enumvals | boot_val | reset_val | sourcefile | sourceline | pending_restart
-------------------------------------+-----------+------+------------+-------------------------------------------------------------------------------------------+------------+------------+---------+---------+---------+------------+----------+-----------+-----------+------------+------------+-----------------
autovacuum | on | | Autovacuum | Starts the autovacuum subprocess. | | sighup | bool | default | | | | on | on | | | f
autovacuum_analyze_scale_factor | 0.1 | | Autovacuum | Number of tuple inserts, updates, or deletes prior to analyze as a fraction of reltuples. | | sighup | real | default | 0 | 100 | | 0.1 | 0.1 | | | f
psqlselectrows(schema, tablename) - Get rows from a table, in a schema
$ psqlselectrows customer events
id | name
----+------
1 | Clicked
2 | Page Loaded
(10 rows)
#Colors
B_BLACK='\033[1;30m'
B_RED='\033[1;31m'
B_GREEN='\033[1;32m'
B_YELLOW='\033[1;33m'
B_BLUE='\033[1;34m'
B_PURPLE='\033[1;35m'
B_CYAN='\033[1;36m'
B_WHITE='\033[1;37m'
RESET='\033[0m'
#Postgres Command With Params
psqlcommand="$POSTGRES_BIN/psql -U postgres test -q -c"
function psqlgettablesize()
{
[ -z "$1" ] && echo -e "${B_RED}Argument 1 missing. Schema name needed. ${B_YELLOW}(Pass 'all' to get details from all schema(s)${RESET}" ||
{
criteria="and table_schema = '$1'"
if [ "$1" == "all" ]; then
criteria=""
fi
if [ "$2" != "" ]; then
criteria+=" and table_name = '$2'"
fi
[ -z "$2" ] && echo -e "${B_YELLOW}Table name not given. ${B_GREEN}Showing size of all tables in $1 schema(s)${RESET}"
query="SELECT
concat(current_database(), '(complete database)') AS table_name, concat(pg_database_size(current_database()), ' (', pg_size_pretty(pg_database_size(current_database())), ')') AS total_size, '' AS table_size, '' AS index_size
UNION ALL SELECT '---------','---------','---------','---------'
UNION ALL SELECT
table_name,
concat(total_table_size, ' (', pg_size_pretty(total_table_size), ')'),
concat(table_size, ' (', pg_size_pretty(table_size), ')'),
concat(index_size, ' (', pg_size_pretty(index_size), ')')
FROM (
SELECT
concat(table_schema, '.', table_name) AS table_name,
pg_total_relation_size(concat(table_schema, '.', table_name)) AS total_table_size,
pg_relation_size(concat(table_schema, '.', table_name)) AS table_size,
pg_indexes_size(concat(table_schema, '.', table_name)) AS index_size
FROM information_schema.tables where table_schema !~ '^pg_' AND table_schema <> 'information_schema' $criteria ORDER BY total_table_size) AS sizes";
$psqlcommand "$query"
}
}
function psqlgettablecount()
{
[ -z "$1" ] && echo -e "${B_RED}Argument 1 missing: Need schema name${RESET}"
[ -z "$2" ] && echo -e "${B_RED}Argument 2 missing: Need table name${RESET}" ||
$psqlcommand "select count(*) from $1.$2;"
}
function psqlgetvacuumdetails()
{
[ -z "$1" ] && echo -e "${B_RED}Argument 1 missing: Need schema name${RESET}" ||
[ -z "$2" ] && echo -e "${B_RED}Argument 2 missing: Need table name${RESET}" ||
$psqlcommand "SELECT schemaname, relname, n_live_tup, n_dead_tup, last_analyze::timestamp, analyze_count, last_autoanalyze::timestamp, autoanalyze_count, last_vacuum::timestamp, vacuum_count, last_autovacuum::timestamp, autovacuum_count FROM pg_stat_user_tables where schemaname = '$1' and relname='$2';"
}
function psqltruncatetable()
{
[ -z "$1" ] && echo -e "${B_RED}Argument 1 missing: Need schema name${RESET}" ||
[ -z "$2" ] && echo -e "${B_RED}Argument 2 missing: Need table name${RESET}" ||
{
read -p "$(echo -e ${B_YELLOW}"Are you sure to truncate table '$1.$2' (y/n)? "${RESET})" choice
case "$choice" in
y|Y ) $psqlcommand "TRUNCATE $1.$2;";;
n|N ) echo -e "${B_GREEN}Table '$1.$2' not truncated${RESET}";;
* ) echo -e "${B_RED}Invalid option${RESET}";;
esac
}
}
function psqlsettings()
{
query="select * from pg_settings"
if [ "$1" != "" ]; then
query="$query where category like '%$1%'"
fi
query="$query ;"
$psqlcommand "$query"
if [ -z "$1" ]; then
echo -e "${B_YELLOW}Passing Category as first argument will filter the related settings.${RESET}"
fi
}
function psqlselectrows()
{
[ -z "$1" ] && echo -e "${B_RED}Argument 1 missing: Need schema name${RESET}" ||
[ -z "$2" ] && echo -e "${B_RED}Argument 2 missing: Need table name${RESET}" ||
$psqlcommand "SELECT * from $1.$2"
}
I couldn't figure out how to dynamically pass variables so I settled for a system of aliases and variables that work together to provide what I was looking for. So I can run :load_schema <schemaname> and then run commands interacting with the "loaded" schema. Loading a schema is simply aliasing a \set command. This way people who are unfamiliar with psql can just press : and tab autocomplete will show them all of my shortcuts.
All my code is here.
This isn't exactly what I was looking for so I won't accept it but it's close enough to post in case anyone else wants to do this.
I'm trying to automate a TSQL select statement on a website. Each year the column names change with the number values at the end of the name increasing by 1 so instead of manually updating the site I'm trying to figure out how to include the dynamic column name in the where clause.
The data looks something like this.
+------+------+------+------+------+
| FY18 | FY19 | FY20 | FY21 | FY22 |
+------+------+------+------+------+
| 1 | 0 | 1 | 0 | 1 |
| 1 | 1 | 0 | 1 | 0 |
| 1 | 0 | 0 | 1 | 0 |
+------+------+------+------+------+
Here is what I've come up with so far... The select statement looks something like this
Select distinct
POS
from TBL_Staff
where [' 'FY'+right(year(dateadd(month,3,getdate()))-1,2) '] = 1
What I'm trying to figure it out is if there is a way to dynamically generate the date and get SQl to recognize name+date as a column
Note: This is fake data so please let me know if something isn't clear
Any help on this is most appreciated.
You could use dynamic SQL
DECLARE #sql NVARCHAR(MAX) = 'Select distinct
POS
from TBL_Staff
where FY' + CAST(right(year(dateadd(month,3,getdate()))-1,2) AS VARCHAR) + ' = 1'
exec sp_executesql #sql
#sql is a variable that is used to construct a SQL statement dynamically. Then the sp_executesql procedure executes it. Beware of using dynamic sql when other alternatives exist. It's harder to read, can be difficult to maintain, and can be a security issue if taking input from users and you're not careful to sanitize input parameters.
I have following content in my csv file(with 3 columns):
141413,"\"'/x=/></script></title><x><x/","Mountain View, CA\"'/x=/></script></title><x><x/"
148443,"CLICK LINK BELOW TO ENTER^^^^^^^^^^^^^^","model\
\
xxx lipsum as it is\
\
100 sometimes unknown\
\
travel evening market\
"
When I import above mentioned csv in mysql using following command, it treats the backslash() as new line; which is the expected behavior.
LOAD DATA INFILE '1.csv' INTO TABLE users FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' LINES TERMINATED BY '\n';
MYSQL Output
But when I try to import to psql using copy command, it treats \ as a normal character.
copy users from '1.csv' WITH (FORMAT csv, DELIMITER ',', ENCODING 'utf8', NULL "\N", QUOTE E'\"', ESCAPE '\');
postgres Output
Try parsing these \ before importing the CSV file, e.g. using perl -pe or sed and the STDIN from psql:
$ cat 1.csv | perl -pe 's/\\\n/\n/g' | psql testdb -c "COPY users FROM STDIN WITH (FORMAT csv, DELIMITER ',', ENCODING 'utf8', NULL "\N", QUOTE E'\"', ESCAPE '\');"
This is how it looks like after the import:
testdb=# select * from users;
id | company | location
--------+-----------------------------------------+-------------------------------------------------
141413 | "'/x=/></script></title><x><x/ | Mountain View, CA"'/x=/></script></title><x><x/
148443 | CLICK LINK BELOW TO ENTER^^^^^^^^^^^^^^ | model +
| | +
| | xxx lipsum as it is +
| | +
| | 100 sometimes unknown +
| | +
| | travel evening market +
| |
(2 Zeilen)
I´m new to Xtext and have a problem.
When I try to create a terminal String without quotes I always get EOF errors.
If I comment out the code for the String without quotes I´dont get an error and everything works fine.
Can someone explain me this?
Or give me some hint how I could better solve this?
Thank you very much
// String without quotes
terminal STRINGWQ: ( ('a'..'z'|'A'..'Z')('a'..'z' | 'A'..'Z' | '_'| '-' | '§' | '?' | '!'| '#'
| '\n' | ':' |'%' | '.' | '*' | '^' | ',' | '&' | '('|')'| '0'..'9'|' ')*);
Rest of Code
grammar org.xtext.example.mydsl.MyDsl with org.eclipse.xtext.common.Terminals
generate myDsl "http://www.xtext.org/example/mydsl/MyDsl"
Model:
(elements += GITest)*
;
GITest:
KWHeader | KWTestCase
;
// KeyWords Header
KWHeader:
'Test' '!''?'
;
KWTestCase:
'testcase' int=INT ':' title = ID |
'Hello' names=ID '!'
;
UPDATE:
Data Type Rule
QSTRING returns ecore::EString: //custom terminal SurveyString
(('a'|'b'|'c'|'d'|'e'|'f'|'g'|'h'|'i'|'j'|'k'|'l'|'m'|'n'|'o'|'p'|'q'|'r'|'s'|'t'|'u'|'v'|'w'|'x'|'y'|'z'|
'A'|'B'|'C'|'D'|'E'|'F'|'G'|'H'|'I'|'J'|'K'|'L'|'M'|'N'|'O'|'P'|'Q'|'R'|'S'|'T'|'U'|'V'|'W'|'X'|'Y'|'Z'|' ')
('a'|'b'|'c'|'d'|'e'|'f'|'g'|'h'|'i'|'j'|'k'|'l'|'m'|'n'|'o'|'p'|'q'|'r'|'s'|'t'|'u'|'v'|'w'|'x'|'y'|'z'|
'A'|'B'|'C'|'D'|'E'|'F'|'G'|'H'|'I'|'J'|'K'|'L'|'M'|'N'|'O'|'P'|'Q'|'R'|'S'|'T'|'U'|'V'|'W'|'X'|'Y'|'Z'|
' '|'_'|'-'|'§'|'?'|'!'|'#'|'%'|'.'|'*'|'^'|','|'&'|'('|')'|'0'|'1'|'2'|'3'|'4'|'5'|'6'|'7'|'8'|'9')*);
UPDATE 2:
Got it working with Data Type Rule und manipulating ID
Code:
STRINGWQ: ((' ')?ID)((ID)?(INT)? ' ' (ID)?);
terminal ID: '^'?('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9'|'/'|';'|','|'#'|'!'|'§'|'$'|'%'|'&'|
'('|')'|'='|'?'|'\\'|'*'|'+'|'.'|'-'|'>'|'<'|'|'|'['|']'|'{'|'}')*;
But now I have the problem that xtext not recognizes when STRINGWQ ends.
So I dont get keyword suggestions in the next line.
For example if i dont use STRINGWQ but INT I get suggestions in the next line.
But with STRINGWQ I dont.
How can I define an end of Data Type Rules?
Thank you
Your STRINGWQ shadows the terminal rule ID and basically all other rules including the keywords. Chances are good that the entire document will by consumed as a single terminal token of type STRINGWQ. You should try to model your string as a datatype rule.
This works for me:
Property:
id=ID | int=INT | prop=PROPERTY_VALUE | spec=SPECIAL;
PROPERTY_VALUE:
(':');
SPECIAL:
(' ' | '/' | ';' | ',' | '!' | '§' | '%' | '&' | '(' | ')' | '?' | '*' | '+' | '.' | '-' | '|' | '[' | ']')
I separated the colon as I need to use PROPERTY_VALUE in another way.
But you also could add it to special.