I have this postgres table:
CREATE TABLE News (
tags text[](10),
contract varchar(40),
...others
);
I need to get all the distinct tags for a given contract. I found the postgresql request which works well:
SELECT array_agg(acc)
FROM (
SELECT DISTINCT unnest(tags::text[])
FROM my_schema.news
WHERE contract = 'acontract'
) AS dt(acc);
As I am using spring data jpa, I try to convert this request into an HSQL request but I can't make it work. Any idea on what the conversion could be?
In addition to SET DATABASE SQL SYNTAX PGS TRUE you need to reference the arrays according to the SQL standard.
CREATE TABLE News (tags text array[10], contract varchar(40))
Then
select array_agg(acc) from (
select distinct acc from news, unnest(tags) un(acc)
WHERE contract = 'acontract'
)
Related
I have a RLS policy violation on a Postgres function. I believe it's because the policy relies on rows created in the function. A SELECT command is run in the function. New rows are not available because they are still in a transaction.
Here is the function:
CREATE FUNCTION public.create_message(organization_id int, content text, tags Int[])
RETURNS setof public.message
AS $$
-- insert message, return PK
WITH moved_rows AS (
INSERT INTO public.message (organization_id, content)
VALUES($1, $2)
RETURNING *
),
-- many to many relation
moved_tags AS (
INSERT INTO public.message_tag (message_id, tag_id)
SELECT moved_rows.id, tagInput.tag_id
FROM moved_rows, UNNEST($3) as tagInput(tag_id)
RETURNING *
)
SELECT moved_rows.* FROM moved_rows LEFT JOIN moved_tags ON moved_rows.id = moved_tags.message_id
$$ LANGUAGE sql VOLATILE STRICT;
Here is the policy:
CREATE POLICY select_if_organization
on message_tag
for select
USING ( message_id IN (
SELECT message.id
FROM message
INNER JOIN organization_user ON (organization_user.organization_id = message.organization_id)
INNER JOIN sessions ON (sessions.user_id = organization_user.user_id)
WHERE sessions.session_token = current_user_id()));
Ideas:
Add a field to the joining table to simplify the policy, but it violates normal form.
Return user input instead of running the SELECT, but input may be escaped and I should be able to run a SELECT command
Split into two functions. Create the message row, then add the message_tag. I'm running postgraphile, so two mutations. I have foreign key relations setup between the two. I don't know if graphile will do that automatically.
Error message:
ERROR: new row violates row-level security policy for table "message_tag"
CONTEXT: SQL function "create_message" statement 1
I receive the error when I run the function. I want the function to run successfully, insert one row in the message table, and turning the input array into rows for the message_tag table with message_tag.message_id=message.id, the last inserted id. I need a policy in place so users from that join relation only see their own organization's message_tag rows.
Here is another policy on the INSERT command. It allows INSERT if a user is logged in:
create policy insert_message_tag_if_author
on message_tag
for insert
with check (EXISTS (SELECT * FROM sessions WHERE sessions.session_token = current_user_id()));
According to the error message, this part of your SQL statement causes the error:
INSERT INTO public.message_tag (message_id, tag_id)
SELECT moved_rows.id, tagInput.tag_id
FROM moved_rows, UNNEST($3) as tagInput(tag_id)
RETURNING *
You need to add another policy FOR INSERT with an appropriate WITH CHECK clause.
I ended up adding a field to the joining table, and creating a policy with that. That way, RLS validation does not require a row which would be created in a middle of a function.
In AWS Athena you can write
SHOW CREATE TABLE my_table_name;
and see a SQL-like query that describes how to build the table's schema. It works for tables whose schema are defined in AWS Glue. This is very useful for creating tables in a regular RDBMS, for loading and exploring data views.
Interacting with Athena in this way is manual, and I would like to automate the process of creating regular RDBMS tables that have the same schema as those in Redshift Spectrum.
How can I do this through a query that can be run via psql? Or is there another way to get this via the aws-cli?
Redshift Spectrum does not support SHOW CREATE TABLE syntax, but there are system tables that can deliver same information. I have to say, it's not as useful as the ready to use sql returned by Athena though.
The tables are
svv_external_schemas - gives you information about glue database mapping and IAM roles bound to it
svv_external_tables - gives you the location information, and also data format and serdes used
svv_external_columns - gives you the column names, types and order information.
Using that data, you could reconstruct the table's DDL.
For example to get the list of columns and their types in the CREATE TABLE format one can do:
select distinct
listagg(columnname || ' ' || external_type, ',\n')
within group ( order by columnnum ) over ()
from svv_external_columns
where tablename = '<YOUR_TABLE_NAME>'
and schemaname = '<YOUR_SCHEM_NAME>'
the query give you the output similar to:
col1 int,
col2 string,
...
*) I am using listagg window function and not the aggregate function, as apparently listagg aggregate function can only be used with user defined tables. Bummer.
I had been doing something similar to #botchniaque's answer in the past, but recently stumbled across a solution in the AWS-Labs' amazon-redshift-utils code package that seems to be more reliable than my hand-spun queries:
amazon-redshift-utils: v_generate_external_tbl_ddl
If you don't have the ability to create a view backed with the ddl listed in that package, you can run it manually by removing the CREATE statement from the start of the query. Assuming you can create it as a view, usage would be:
SELECT ddl
FROM admin.v_generate_external_tbl_ddl
WHERE schemaname = '<external_schema_name>'
-- Optionally include specific table references:
-- AND tablename IN ('<table_name_1>', '<table_name_2>', ..., '<table_name_n>')
ORDER BY tablename, seq
;
They added show external table now.
SHOW EXTERNAL TABLE external_schema.table_name [ PARTITION ]
SHOW EXTERNAL TABLE my_schema.my_table;
https://docs.aws.amazon.com/redshift/latest/dg/r_SHOW_EXTERNAL_TABLE.html
How can I make the table looks like below turn into concatenated form
1844|a
1847|a,b
1848|34,qaz
Good day,
This question was asked with the tag "sql-server-2012", but I can't feel good to give the answer without first show how simple it is to solve these type of questions with new function STRING_AGG that was added to SQL Server 2017.
Therefore, First I will add the code for the sample DDL+DML (create the relevant table and insert sample data), which is something that the person who ask the question must ad to the question, but was not added here
Secondly I will show the solution in SQL Server 2017 and above
And finally I will give the solution for older versions like SQL Server 2012
Have fun, and I recommend to read all
DDL + DML
NOTE! Instead of posting images and stories on your table, it is always better to present us a simple way to reproduce the issue! This means that when you have question related to queries, then you should provide us with queries to create a sample table (including the indexes) and to insert sample data. In addition you should add the requested result, which was provided here.
/*************************** DDL+DML */
drop table if exists T; -- this work only from SQL Server 2016
create table T(id int, txt nvarchar(10))
GO
INSERT T(id,txt)
values (1844,'a'),(1847,'a'),(1847,'b'),(1848,'34'),(1848,'q')
GO
Select * from T
GO
Solution for SQL Server 2017 and above
/*************************** Solution */
-- New from 2016: STRING_SPLIT ( string , separator )
-- New from 2017: STRING_AGG ( expression, separator ) [ <order_clause> ]
SELECT id,STRING_AGG(ISNULL(txt,''), ',') WITHIN GROUP (order by id)
from T
group by id
GO
Solution for old versions of SQL Server
This solution was taken from this blog:
http://ariely.info/Blog/tabid/83/EntryId/152/Unsplit-Using-For-XML-with-group-by-columns.aspx
SELECT
id,
STUFF(
(
SELECT ',' + t_in.txt
FROM T t_in
WHERE (t_in.id = t_out.id)
FOR XML PATH ('')
),1,1,''
) AS NameValues
FROM T t_out
GROUP BY id
GO
I hope that this solve the question ;-)
I'm attempting to do an upsert-style statement for Postgres in jOOQ. The framework I'm running in takes care of concurrency concerns in this specific situation so I'm not worried about that. I'm only using jOOQ for creating the SQL, the actual execution is done via Spring's JdbcTemplate and a BeanPropertySqlParameterSource.
I've decided to go with a two-step "insert..where not exists" / "update .." process.
My SQL is:
insert into mytable (my_id, col1, col2) select :myId,
:firstCol, :secondCol where not exists (select 1
from mytable where my_id = :myId)
I'm using Postgres 9.4, jOOQ 3.5. I'm not sure how to express both the jOOQ params in the select and the "where not exists" clause in jOOQ.
Suggestions to change programming languages or databases aren't viable in my situation.
If you want to reuse a named parameter in jOOQ, ideally, you create the AST element outside of the query, as such:
// Assuming a static import
import static org.jooq.impl.DSL.*;
Param<Integer> myId = param("myId", Integer.class);
You can then use it multiple times in your query:
using(configuration)
.insertInto(MY_TABLE, MY_TABLE.MY_ID, MY_TABLE.COL1, MY_TABLE.COL2)
.select(
select(
myId,
param("firstCol", MY_TABLE.COL1.getType()),
param("secondCol", MY_TABLE.COL2.getType())
)
.whereNotExists(
selectOne()
.from(MY_TABLE)
.where(MY_TABLE.MY_ID.eq(myId))
)
);
My DBA has constructed me a stored procedure in a Sybase database, for which I don't have the definition.
If I run it, it returns a resultset with a set of columns and values. I would like to SELECT further to reduce the rows in the result set. Is this possible?
From this question it seems like I could insert the results into a temporary table, but I'm not sure I've got permissions to do this.
Is there any way I can SELECT certain rows, or if not, can someone give me example code for simulating with a temporary table?
In Sybase ASE, we can use this hack to select from a stored procedure via a "proxy table":
http://www.sypron.nl/proctab.html
Example:
sp_addserver loopback, null, ##servername
go
create existing table
sp_test12 (
Document_Name varchar(100),
Required_Status varchar(5),
Doc_ID varchar(10),
OrderBy int,
No_of_Copy_Retain int,
_p_EPEB_ID varchar(10) null,
_p_MY_NAME varchar(3) null,
_p_MY_NO varchar(10) null,
_p_EPEB_EDATE datetime null,
_TXN varchar(10) null,
_SUBTXN varchar(15) null,
_OwnType_ID1 varchar(5) null,
_OwnType_ID2 varchar(5) null,
_blnflag int null
)
external procedure
at 'loopback.MYDB.dbo.usp_xyz'
go
select
Doc_ID, No_of_Copy_Retain, _p_EPEB_ID, _p_EPEB_ID, _p_MY_NAME, _p_MY_NO
from #sp_test12
where
_p_EPEB_ID='EPEB1508'
and _p_MY_NAME='107'
and _p_MY_NO='2011000045'
and _p_EPEB_EDATE='2011-01-15 15:03:03.0'
and _TXN='TX012'
and _SUBTXN='TX012.001'
and _OwnType_ID1='ASSN'
and _OwnType_ID2='ASSN'
and _blnflag=0
go
Under Sybase IQ (12.6 and higher at least) you can select from a stored procedure and filter the results as if it was a table. I do not know if this works under ASE or ASA but you could give it a try.
So if you stored procedure is called myproc an the result set has a column ACTIVE which can be either 0 or 1 and you want to select only the ACTIVE = 1 rows you could do this.
SELECT * FROM myproc() WHERE ACTIVE = 1
Under IQ you can also use this as a derived table and JOIN it with other tables for example like this...
SELECT t1.name,t1.address,t2,active
FROM tbl_atable t1,
( SELECT * FROM myproc() WHERE ACTIVE = 1) t2
WHERE t1.active = t2.active
...which is kind of neat!
I hope that works for which ever version of Sybase you are running.
You will need to ask the DBA to change the stored procedure.
You could get it changed to select the results into a temporary table rater than a plain select and then you can write your own select on that temp table to return only the rows you want
It is possible with ASE but in a rather roundabout kind of way using CIS and proxy tables. The mechanism is described very well on Rob Verschoor's site:
http://www.sypron.nl/proctab.html
I tried it out once as a curiosity and indeed it does work. I did not delve into the tricky question of error-handling.
pjjH
As far as I know, this is not possible in Sybase ASE. Even using
insert #temp_table
exec my_procedure
doesn't work (at least on sybase 12.x).
Just a thought.
Perhaps your DBA could prepare a view instead of a stored procedure, if he wanted you for some reason not to look at the inner stuff or worry about it.
Another approach would be to see the stored procedure text (unless encrypted) with sp_helptext and rewrite it for your own purposes (eg. into a view) to be able to apply additional conditioning to the resultset.
In Sybase IQ, you can do this:
select < col1>, < col2> from < sp_name>('< sp_arg>') where < predicate>
Example:
select Object, DbspaceName, ObjSize from sp_iqindexinfo ('table xyz') where Object like '%col1_indx%'