Copy output of WITH subquery to CSV in postgres - postgresql

Question

I am trying to save the output of below 'WITH' sub-query to a csv file.
WITH mer9 AS (
SELECT *,
substring(seq_window_mut_9mers, split9.start, 9)
FROM split9
),
mer23 AS (
SELECT *,
substring(seq_window_mut_23mers, split23.start, 23)
FROM split23
),
dataset AS (
SELECT *
FROM table
INNER JOIN mer9 ON mer9.seq_window_mut_9mers = table.seq_window_mut_9mers
INNER JOIN mer23 ON mer23.seq_window_mut_23mers = table.seq_window_mut_23mers
)
COPY (SELECT * FROM dataset) TO '/tmp/filename.csv' (format CSV);
After running the query, I am getting an error:
[Code: 0, SQL State: 42601] ERROR: syntax error at or near "COPY"
Position: 3566 [Script position: 3566 - 3570]

Answer 1

Result sets generated from a CTE cannot be accessed in a different query. A CTE creates a sort of "temporary table" that only exists in the current query. That being said, put your CTE inside of the COPY statement and it should work, e.g.
COPY (
WITH mer9 AS (
SELECT *, substring(seq_window_mut_9mers, split9.start, 9)
FROM split9),
mer23 AS (
SELECT *, substring(seq_window_mut_23mers, split23.start, 23)
FROM split23),
dataset AS (
SELECT * FROM table
INNER JOIN mer9 ON mer9.seq_window_mut_9mers = table.seq_window_mut_9mers
INNER JOIN mer23 ON mer23.seq_window_mut_23mers = table.seq_window_mut_23mers
)
) TO '/tmp/filename.csv' (format CSV);
Edit. As pointed out by #a_horse_with_no_name:
Keep in mind that this command will create a file in the server. If you wish to create a file with the output in the client, consider using STDOUT in your COPY command, e.g. using psql:
$ psql -d yourdb -h yourdbhost -U your_user -c "COPY (WITH..) TO STDOUT" > file.csv
See also this answer.