I'm a postrgresql noob. I'm using psql on a remote server. I've made a query that I'd like to download as a csv. The query runs just fine on its own.
There are two tables involved in the query. The first:
CREATE TABLE players (username varchar(100), skill varchar(100),id int);
INSERT INTO players VALUES
('user1', 'flight', 1),
('user2', 'run', 2),
('user3', 'run', 3),
('user4', 'flight', 4),
('user5', 'flight', 5),
('user6', 'climb', 6),
('user7', 'flight', 7),
('user8', 'flight', 8),
('user9', 'flight', 9),
('user10', 'climb', 10);
The second table is temporary, I only need it to exist to extract this csv:
CREATE TEMP TABLE specific_players (username varchar(100));
INSERT INTO specific_players VALUES
('user2'),
('user5'),
('user7'),
('user9');
my query looks like this:
select * from players p inner join specific_players sp USING (username);
Like I said, that works just fine.
But when I try to wrap it in a copy command
\copy (select * from players inner join specific_players USING (username)) to '/Users/me/path/to/folder/filename.csv' with csv
I get an error:
ERROR: syntax error at or near "("
LINE 1: COPY ( select * from players p join specific_playe...
The file IS created on my computer, but it's empty. Any idea what I'm doing wrong?
Related
I am trying to save the output of below 'WITH' sub-query to a csv file.
WITH mer9 AS (
SELECT *,
substring(seq_window_mut_9mers, split9.start, 9)
FROM split9
),
mer23 AS (
SELECT *,
substring(seq_window_mut_23mers, split23.start, 23)
FROM split23
),
dataset AS (
SELECT *
FROM table
INNER JOIN mer9 ON mer9.seq_window_mut_9mers = table.seq_window_mut_9mers
INNER JOIN mer23 ON mer23.seq_window_mut_23mers = table.seq_window_mut_23mers
)
COPY (SELECT * FROM dataset) TO '/tmp/filename.csv' (format CSV);
After running the query, I am getting an error:
[Code: 0, SQL State: 42601] ERROR: syntax error at or near "COPY"
Position: 3566 [Script position: 3566 - 3570]
Result sets generated from a CTE cannot be accessed in a different query. A CTE creates a sort of "temporary table" that only exists in the current query. That being said, put your CTE inside of the COPY statement and it should work, e.g.
COPY (
WITH mer9 AS (
SELECT *, substring(seq_window_mut_9mers, split9.start, 9)
FROM split9),
mer23 AS (
SELECT *, substring(seq_window_mut_23mers, split23.start, 23)
FROM split23),
dataset AS (
SELECT * FROM table
INNER JOIN mer9 ON mer9.seq_window_mut_9mers = table.seq_window_mut_9mers
INNER JOIN mer23 ON mer23.seq_window_mut_23mers = table.seq_window_mut_23mers
)
) TO '/tmp/filename.csv' (format CSV);
Edit. As pointed out by #a_horse_with_no_name:
Keep in mind that this command will create a file in the server. If you wish to create a file with the output in the client, consider using STDOUT in your COPY command, e.g. using psql:
$ psql -d yourdb -h yourdbhost -U your_user -c "COPY (WITH..) TO STDOUT" > file.csv
See also this answer.
I created a temporary table with sqlalchemy (with an underlying postgres database) that is going to be joined with a database table. However, in some cases when a value is empty '' then postgres throws the error:
failed to find conversion function from unknown to text
SqlAlchemy assembles everything to the following context
[SQL: 'WITH temp_table AS \n(SELECT %(param_1)s AS id, %(param_2)s AS email, %(param_3)s AS phone)\n SELECT campaigns_contact.id, campaigns_contact.email, campaigns_contact.phone \nFROM campaigns_contact JOIN temp_table ON temp_table.id = campaigns_contact.id AND temp_table.email = campaigns_contact.email AND temp_table.phone = campaigns_contact.phone'] [parameters: {'param_1': 83, 'param_2': '', 'param_3': '+1234567890'}]
I assemble the temporary table as follows
stmts = []
for row in import_data:
row_values = [literal(row[value]).label(value) for value in values]
stmts.append(select(row_values))
subquery = union_all(*stmts)
subquery = subquery.cte(name="temp_table")
The problem seems to be the part here
...%(param_2)s AS email...
which after replacing the param_2 results in
...'' AS email...
which will cause the error mentioned above.
One way to solve the issue is to perform a cast
...''::text AS email...
However, I don't know how to perform ::text cast with sqlalchemy!?
I'm trying to traverse some nodes and I want to stop the traversal when the furthest reached node matches a certain condition.
My data is some sequentially connected nodes - there is a sample data creation script here (note this is much simplified from my real data, just to illustrate the problem):
create database plocal:people
create class Person extends V
create property Person.name string
create property Person.age float
create property Person.ident integer
insert into Person(name,age,ident) VALUES ("Bob", 30.5, 1)
insert into Person(name,age,ident) VALUES ("Bob", 30.5, 2)
insert into Person(name,age,ident) VALUES ("Carol", 20.3, 3)
insert into Person(name,age,ident) VALUES ("Carol", 19, 4)
insert into Person(name,age,ident) VALUES ("Laura", 75, 5)
insert into Person(name,age,ident) VALUES ("Laura", 60.5, 6)
insert into Person(name,age,ident) VALUES ("Laura", 46, 7)
insert into Person(name,age,ident) VALUES ("Mike", 16.3, 8)
insert into Person(name,age,ident) VALUES ("David", 86, 9)
insert into Person(name,age,ident) VALUES ("Alice", 5, 10)
insert into Person(name,age,ident) VALUES ("Nigel", 69, 11)
insert into Person(name,age,ident) VALUES ("Carol", 60, 12)
insert into Person(name,age,ident) VALUES ("Mike", 16.3, 13)
insert into Person(name,age,ident) VALUES ("Alice", 5, 14)
insert into Person(name,age,ident) VALUES ("Mike", 16.3, 15)
create class NEXT extends E
create edge NEXT from (select from Person where ident = 1) to (select from Person where ident = 3)
create edge NEXT from (select from Person where ident = 2) to (select from Person where ident = 4)
create edge NEXT from (select from Person where ident = 8) to (select from Person where ident = 12)
create edge NEXT from (select from Person where ident = 5) to (select from Person where ident = 15)
create edge NEXT from (select from Person where ident = 15) to (select from Person where ident = 14)
create edge NEXT from (select from Person where ident = 7) to (select from Person where ident = 13)
create edge NEXT from (select from Person where ident = 13) to (select from Person where ident = 10)
Let's define a sequence as a traversal of nodes from a node with no incoming links (a start node). I am trying to write a query that will return me all sequences of nodes up until the first occurrence of a specific name encountered in the traversal. Suppose the specific name is 'Mike'. So for this data I would want the following sequences to be found:
("Laura", 75, 5) -> ("Mike", 16.3, 15),
("Laura", 46, 7) -> ("Mike", 16.3, 13),
("Mike", 16.3, 8)
I can get the sequence from a specific node to the node just before a 'Mike'. The following query returns me record ("Laura", 75, 5), as the record after that one has name 'Mike'
traverse out('NEXT') from (select from Person where ident = 5) while name <> 'Mike'
The following query returns me two rows, one with record ("Laura", 75, 5) and one with record ("Mike", 16.3, 15), as the record after Mike has name 'Alice'.
traverse out('NEXT') from (select from Person where ident = 5) while name <> 'Alice'
I have two problems with this - firstly I'd like to include the node which matches the condition in the sequence (i.e. when checking for a Person called 'Mike', I'd like 'Mike' to be the final node in the sequence returned)
For that I assume I need to store the traversal in an object, and request one more out-Next for that object before returning. I've tried various approaches to storing the traversal in an object in the middle of a query, but I'm just not getting it. Here is an example (which errors):
select from (
select $seq from
(select from Person where ident = 5)
let $seq = traverse out('NEXT') from $current while name <> 'Alice'
<... here append the next node ...>
)
Secondly, this query just starts from one node - I'd like to start at all starting nodes and return a sequence ending in 'Mike' wherever there is one. I'm hoping that once I can store the traversal in an object, it should be relatively straight-forward to just run from multiple starting points rather than just one.
(Of course, another option for this specific query is to find all the nodes that match the specific condition (e.g. name= 'Mike') and work backwards from those, but I'd really like to see it work in the way I described initially as I'll need that general approach for more things later.)
I suspect quite a lot of my issue is that I'm really struggling to work out how to use the let statement in OrientDB - I'm really not understanding how the scope works, which objects exist at what stages of the query. If anyone knows any good documentation out there other than the official docs that would be really useful as I've read those and I'm still not getting it.
So any helpful hints on how to answer this question, or where to find more information on how to write this type of query would be really useful.
I hope it can help you
select expand($c) let $b=(select expand(out("NEXT")[name="Alice"]) from (select expand(last($a)) from (select from Person where ident = 5)
let $a = (traverse out('NEXT') from $current while name <> 'Alice')) limit 1), $c=unionAll($a,$b)
select name, list from (select name,$c.name as list from Person
let $b=( select expand(out("NEXT")[name="Alice"]) from (select expand(last($a)) from $parent.$current
let $a = (traverse out('NEXT') from $current while name <> 'Alice')) limit 1),
$c=unionAll($a,$b) where in("NEXT").size()=0)
where list contains "Alice"
I have imported one shapefile named tc_bf25 using qgis, and the following is my python script typed in pyscripter,
import sys
import psycopg2
conn = psycopg2.connect("dbname = 'routing_template' user = 'postgres' host = 'localhost' password = '****'")
cur = conn.cursor()
query = """
ALTER TABLE tc_bf25 ADD COLUMN source integer;
ALTER TABLE tc_bf25 ADD COLUMN target integer;
SELECT assign_vertex_id('tc_bf25', 0.0001, 'the_geom', 'gid')
;"""
cur.execute(query)
query = """
CREATE OR REPLACE VIEW tc_bf25_ext AS
SELECT *, startpoint(the_geom), endpoint(the_geom)
FROM tc_bf25
;"""
cur.execute(query)
query = """
CREATE TABLE node1 AS
SELECT row_number() OVER (ORDER BY foo.p)::integer AS id,
foo.p AS the_geom
FROM (
SELECT DISTINCT tc_bf25_ext.startpoint AS p FROM tc_bf25_ext
UNION
SELECT DISTINCT tc_bf25_ext.endpoint AS p FROM tc_bf25_ext
) foo
GROUP BY foo.p
;"""
cur.execute(query)
query = """
CREATE TABLE network1 AS
SELECT a.*, b.id as start_id, c.id as end_id
FROM tc_bf25_ext AS a
JOIN node AS b ON a.startpoint = b.the_geom
JOIN node AS c ON a.endpoint = c.the_geom
;"""
cur.execute(query)
query = """
ALTER TABLE network1 ADD COLUMN shape_leng double precision;
UPDATE network1 SET shape_leng = length(the_geom)
;"""
cur.execute(query)
I got the error at the second cur.execute(query),
But I go to pgAdmin to check result, even though no error occurs, the first cur.execute(query) didn't add new columns in my table.
What mistake did I make？ And how to fix it？
I am working with postgresql 8.4, python 2.7.6 under Windows 8.1 x64.
When using psycopg2, autocommit is set to False by default. The first two statements both refer to table tc_bf25, but the first statement makes an uncommitted change to the table. So try running conn.commit() between statements to see if this resolves the issue
You should run each statement individually. Do not combine multiple statements into a semicolon separated series and run them all at one. It makes error handling and fetching of results much harder.
If you still have the problem once you've made that change, show the exact statement you're having the problem with.
Just to add to #Talvalin you can enable auto-commit by adding
psycopg2.connect("dbname='mydb',user='postgres',host ='localhost',password = '****'")
conn.autocommit = True
after you connect to your database using psycopg2
How would I get the last inserted ID using a multirow insert?
Here is my code:
$sql='INSERT INTO t (col1, col2, col3) VALUES (1, 2, 3), (4, 5, 6), (7, 8, 9)'; // example
$stmt = $contactsTable->getAdapter()->prepare($sql);
$stmt->execute();
$rowsAdded=$stmt->rowCount(); // mysql_affected_rows
$lastId=$stmt->lastInsertId();
echo '<br>Last ID: '.$lastId;
Also, is there a method in ZF to get the next insert id before an insert?
thanks
$lastId=$contactsTable->getAdapter()->lastInsertId();
This worked.
So, here is the full working code I'm using to create a multi-row insert, getting the rows added and the last inserted id:
$contactsTable = new Contacts_Model_Contacts();
$sql='INSERT INTO t (col1, col2, col3) VALUES (1, 2, 3), (4, 5, 6), (7, 8, 9)'; // example
$stmt = $contactsTable->getAdapter()->prepare($sql);
$stmt->execute();
$rowsAdded=$stmt->rowCount(); // mysql_affected_rows
$lastId=$contactsTable->getAdapter()->lastInsertId(); // last inserted id
echo '<br>Last ID: '.$lastId;
An alternate solution. Move off sql code from controllers and place them in models. That is what they are for.
If you are using models, you can given the name of table which has auto increment column, in class variable.
protected $_name="<table_name>";
Then in your model class method, you can get last insert id from
$insertID= $this->getAdapter()->lastInsertId();
that code should work, but it will only get you the id of your last insert.
you can get the next autoincrement with this mysql query:
SELECT Auto_increment FROM information_schema.tables WHERE TABLE_SCHEMA = 'your_db_name' AND TABLE_NAME='the_table_you_want';