I want to import a CSV File (comma delimited with double quote) into SQLite using PowerShell script. I tried:
echo ".import export.csv mytable" | sqlite3.exe
I get this error:
export.csv:327: unescaped " character
I get this error for all lines. On SQLite's command line shell same command works:
sqlite > .import export.csv mytable
How can I make this command work using a PowerShell script?
This works for me in both PowerShell 5.1 and v7.0.
$params = #"
.mode csv
.separator ,
.import export.csv mytable
"#
$params | .\sqlite3.exe mydatabase.db
The following single command line works in both
cmd.exe (version 10.0.x.x via ver) and
powershell.exe (version 5.1.x.x via $PSVersionTable)
.\sqlite3.exe my.db ".import file1.csv table1 --csv"
This loads the contents of csv file file1.csv into table table1 within the sqlite database file my.db. The --csv flag will import the file and create the table structure based on the header column names in that file, and will approximately infer the data types.
You can import multiple files this way.
i.e.
.\sqlite3.exe my.db ".import file1.csv table1 --csv" ".import file2.csv table2 --csv"
You can chain commands together to immediately open the database for querying by appending ; .\sqlite3.exe my.db.
i.e.
.\sqlite3.exe my.db ".import file1.csv table1 --csv" ".import file2.csv table2 --csv; .\sqlite3.exe my.db"
what about sqlite3.exe ".import export.csv mytable".
You can check the documentation of sqlite3.exe to use it not being in its shell, how to pass parameter to sqlite3.exe.
You can try below line
Invoke-Expression 'sqlite3 yourdatabase.db < ".import export.csv mytable"'
Related
I'm attempting to dynamically create a script that gets saved as a bat file that will be scheduled to execute daily via Windows Task Scheduler. The script performs full database backups for each Postgres database using pg_dump.
The current script is as follows:
COPY (SELECT 'pg_dump '|| datname || ' > e:\postgresbackups\FULL\' || datname || '_%date:~4,2%-%date:~7,2%-%date:~10,4%_%time:~0,2%_%time:~3,2%_%time:~6,2%.dump' FROM pg_database) TO 'E:\PostgresBackups\Script\FULL_Postgres_Backup_Job_TEST.bat' (format csv, delimiter ';');
An example of the output is as follows:
pg_dump postgres > e:\postgresbackups\FULL\postgres_%date:~4,2%-%date:~7,2%-%date:~10,4%%time:~0,2%%time:~3,2%_%time:~6,2%.dump
I need help with updating my code so that the output will include double quotes around the name of the dump file; however, when I add this to my COPY script it adds more than what is necessary to the output. I would like the output to look like the following which includes the double-quotes:
pg_dump postgres > "e:\postgresbackups\FULL\postgres_%date:~4,2%-%date:~7,2%-%date:~10,4%%time:~0,2%%time:~3,2%_%time:~6,2%.dump"
Any help would be greatly appreciated!
Thanks to #Mike Organek's comment, my issue has been resolved by switching the format from CSV to TEXT. Now when I enclose the dump filename in double quotes, the output is more of what is expected and works as intended. The only odd thing now is that in the output it creates a second backslash in the filename. My code has been updated as follows:
COPY (SELECT 'pg_dump '|| datname || ' > "e:\postgresbackups\FULL\' || datname || '_%date:~4,2%-%date:~7,2%-%date:~10,4%_%time:~0,2%_%time:~3,2%_%time:~6,2%.dump"' FROM pg_database) TO 'E:\PostgresBackups\Script\FULL_Postgres_Backup_Job.bat' (format text, delimiter ';');
An example of the output that gets created within the bat file is as follows:
pg_dump postgres > "e:\\postgresbackups\\FULL\\postgres_%date:~4,2%-%date:~7,2%-%date:~10,4%_%time:~0,2%_%time:~3,2%_%time:~6,2%.dump"
As you can see, it adds a double backslash; however, the pg_dump executes successfully!
I have an sql file (import_noeud.sql) that copy a csv file into a table.
\copy public.import_noeud FROM 'T:\Affaires\102\171205_noeud.csv' DELIMITER ';' CSV;
I then use a batch file with psql statement to launch my sql file.
psql -h localhost -U postgres -d dev -v ON_ERROR_STOP=1 -f import_noeud.sql> 01.log 2>&1
For now on, every time the user want to reimport new points he has to go modify the from clause of the sql file.
My question is, could my batch file:
open the windows explorer
ask the user to fetch the csv file
store the result (for example 'T:\Affaires\102\171205_noeud.csv')
launch the sql file using the result
Let's incorporate vbs into a .bat or .cmd batch file. vbs is used to select the file.
Save both the vbs and cmd files in the same directory.
import.vbs
Set wShell=CreateObject("WScript.Shell")
Set oExec=wShell.Exec("mshta.exe ""about:<input type=file id=FILE><script>FILE.click();new ActiveXObject('Scripting.FileSystemObject').GetStandardStream(1).WriteLine(FILE.value);close();resizeTo(0,0);</script>""")
myFile = oExec.StdOut.ReadLine
wscript.echo myFile
runme.cmd
#echo off
for /f "tokens=*" %%a in ('cscript //nologo import.vbs') do (set "VBSoutput=%%a")
echo \copy public.import_noeud FROM '%VBSoutput%' DELIMITER ';' CSV; > SOME_IMPORT.sql
Now you can run the batch file, which will open a Choose File To Open window. Once selected and you choose open, it will then select the file and automatically push it to echo the \copy... to a .sql file. So let's say you browse to, and select:
T:\Affaires\102\171205_noeud.csv
it will echo the following to the sql file in this case named SOME_IMPORT.sql
\copy public.import_noeud FROM 'T:\Affaires\102\171205_noeud.csv' DELIMITER ';' CSV;
You can add the launching of the SQL file if you like as well to the batch file.
If you decide to change the name of the vbs script, please ensure you also change it in the batch string ('cscript //nologo import.vbs')
Hope it helps.
I have a SQL database. If I export it as CSV using sqlitebrowser there is an option to define a "Quote character". How can I set the quote character when doing the export with a script? The default is " and want it to be #.
Export script without quote-character definition:
sqlite3 my.db<<EOF
.mode csv
.separator |
.output out.csv
SELECT * FROM myTable
EOF
Desired output:
#This is#|# a
"table" with#|# three columns, #
# some
newlines,
and...
#|#two#|# "rows". #
I've looked in the documentation und couldn't find such an option. Does it exist?
I have SAS code that will write to a postgres table if it is already created but still empty. How can I create/alter a postgres table from SAS (or using a script that pulls in SAS macro variables) if it does not exist or already has data? The number of fields may change. Currently, I use the filename option along with the pipe to write to the postgres file.
filename pgout pipe %unquote(%bquote(')/data/dwight/IFS6.2/app/PLANO/sas_to_psql.sh
%bquote(")&f_out_schema.%bquote(").&file_name.
%bquote(')
)
;
I've tried using this version, but it does not work:
filename pgout pipe %unquote(%bquote(')/data/dwight/IFS6.2/app/PLANO/sas_to_psql.sh
%bquote('')CREATE TABLE mdo_backend.fob_id_desc
SELECT * FROM &library_name..&file_name.
%bquote(")&f_out_schema.%bquote(").&file_name./('')/
%bquote(')
)
;
This is the script I use:
LOAD_TO_PSQL.SH
#!/bin/bash
. /data/projects/ifs/psql/config.sh
psql -d $DB -tAq -c "COPY $1 FROM STDIN USING DELIMITERS '|'"
is possible in PSQL console export file with current date on the end of the file name?
The name of the exported file should be like this table_20140710.csv is it possible to do this dynamically? - the format of the date can be different than the above it isn't so much important.
This is example what i mean:
\set curdate current_date
\copy (SELECT * FROM table) To 'C:/users/user/desktop/table_ ' || :curdate || '.csv' WITH DELIMITER AS ';' CSV HEADER
The exception of the \copy meta command not expanding variables is (meanwhile) documented
Unlike most other meta-commands, the entire remainder of the line is always taken to be the arguments of \copy, and neither variable interpolation nor backquote expansion are performed in the arguments.
To workaround you can build, store and execute the command in multiple steps (similar to the solution Clodoaldo Neto has given):
\set filename 'my fancy dynamic name'
\set command '\\copy (SELECT * FROM generate_series(1, 5)) to ' :'filename'
:command
With this, you need to double (escape) the \ in the embedded meta command. Keep in mind that \set concatenates all further arguments to the second one, so quote spaces between the arguments. You can show the command before execution (:command) with \echo :command.
As an alternative to the local \set command, you could also build the command server side with SQL (the best way depends on where the dynamic content is originating):
SELECT '\copy (SELECT * FROM generate_series(1, 5)) to ''' || :'filename' || '''' AS command \gset
Dynamically build the \copy command and store it in a file. Then execute it with \i
First set tuples only output
\t
Set the output to a file
\o 'C:/users/user/desktop/copy_command.txt'
Build the \copy command
select format(
$$\copy (select * from the_table) To 'C:/users/user/desktop/table_%s.csv' WITH DELIMITER AS ';' CSV HEADER$$
, current_date
);
Restore the output to stdout
\o
Execute the generated command from the file
\i 'C:/users/user/desktop/copy_command.txt'