I have an sql file (import_noeud.sql) that copy a csv file into a table.
\copy public.import_noeud FROM 'T:\Affaires\102\171205_noeud.csv' DELIMITER ';' CSV;
I then use a batch file with psql statement to launch my sql file.
psql -h localhost -U postgres -d dev -v ON_ERROR_STOP=1 -f import_noeud.sql> 01.log 2>&1
For now on, every time the user want to reimport new points he has to go modify the from clause of the sql file.
My question is, could my batch file:
open the windows explorer
ask the user to fetch the csv file
store the result (for example 'T:\Affaires\102\171205_noeud.csv')
launch the sql file using the result
Let's incorporate vbs into a .bat or .cmd batch file. vbs is used to select the file.
Save both the vbs and cmd files in the same directory.
import.vbs
Set wShell=CreateObject("WScript.Shell")
Set oExec=wShell.Exec("mshta.exe ""about:<input type=file id=FILE><script>FILE.click();new ActiveXObject('Scripting.FileSystemObject').GetStandardStream(1).WriteLine(FILE.value);close();resizeTo(0,0);</script>""")
myFile = oExec.StdOut.ReadLine
wscript.echo myFile
runme.cmd
#echo off
for /f "tokens=*" %%a in ('cscript //nologo import.vbs') do (set "VBSoutput=%%a")
echo \copy public.import_noeud FROM '%VBSoutput%' DELIMITER ';' CSV; > SOME_IMPORT.sql
Now you can run the batch file, which will open a Choose File To Open window. Once selected and you choose open, it will then select the file and automatically push it to echo the \copy... to a .sql file. So let's say you browse to, and select:
T:\Affaires\102\171205_noeud.csv
it will echo the following to the sql file in this case named SOME_IMPORT.sql
\copy public.import_noeud FROM 'T:\Affaires\102\171205_noeud.csv' DELIMITER ';' CSV;
You can add the launching of the SQL file if you like as well to the batch file.
If you decide to change the name of the vbs script, please ensure you also change it in the batch string ('cscript //nologo import.vbs')
Hope it helps.
Related
Once again, I am asking for your help to manage a small batch script.
As announced, in a previous post, I'm used to work on linux and automation tasks are more obvious to me but I have to work today under Microsoft environment.
Writting a Batch file is definitly not an easy things.
I am able with this script to be connected to a PostgreSQL database and load data via ogr2ogr. Until then, no problem.
Currently, the path of the folder is hard written in my code, but I would like to have the possibility of choosing the working directory through windows explorer.
Also, concerning this directory, I would like to have the possibility to process the subfolders at the same time.
Here, below the piece of my *.bat code:
TITLE Upload Shapefile Files to PostgreSQL
#echo off
cls
color 9
echo ******************************************************************************
echo * Upload Shapefile Files to PostgreSQL *
echo ******************************************************************************
set /P Host=Please enter your Server Host (default:localhost):
set /P Port=Please enter your PGSQL port (default:5432):
set /p Database=Please enter your PGSQL Database Name (default:postgres):
set /P Schema=Please enter your Edigeo Schema (default:public):
set /P User=Please enter your PGSQL username (default:postgres):
set /P Password=Please enter your PGSQL password (default:postgres):
For /F %%H In ("C:\Users\stephj\Desktop\SHP\*.shp") do ogr2ogr -overwrite -t_srs EPSG:2154 -s_srs EPSG:2154 -f "PostgreSQL" PG:"host=%Host% port=%Port% user=%User% password=%Password% dbname=%Database%" -lco schema=%Schema%
pause
Thanks in advance for your time and your help.
Ok, here is a way. This will just demonstrate by echoing the files, you need to amend it to follow your command structure:
Note!! This is a batch/PowerShell hybrid script. It needs to be saved with a batch file extension, preferably .cmd:
#echo off
set "pshell="(new-object -COM 'Shell.Application').BrowseForFolder(0,'Select Folder',0,0).self.path""
for /f "delims=" %%a in ('powershell %pshell%') do set "workdir=%%a"
for /r "%workdir%" %%i in (*.shp) do echo "%%~i"
I am using pgAdmin version 4.3 and i want to export one table data to CSV file. I used this query
COPY (select * from product_template) TO 'D:\Product_template_Output.csv' DELIMITER ',' CSV HEADER;
but it shows error
a relative path is not allowed to use COPY to a file
How can I resolve this problem any help please ?
From the query editor, once you have executed your query, you just have to click on the "Download as CSV (F8)" button or use F8 key.
Source pgAdmin 4 Query Toolbar
Use absolute paths or cd to a known location so that you can ignore the path.
For example cd into documents directory then run the commands there.
If you are able to cd into your documents directory, then the command would be like this:
Assuming you are want to use PSQL from the command line.
cd ~/Documents && psql -h host -d dbname -U user
\COPY (select * from product_template) TO 'Product_template_Output.csv' DELIMITER ',' CSV HEADER;
The result would be Product_template_Output.csv in your current working directory(Documents folder).
Again using psql.
You have to remove the double quotes:
COPY (select * from product_template) TO 'D:\Product_template_Output.csv'
DELIMITER ',' CSV HEADER;
If your PgAdmin instance resides in a remote server, the aforementioned solutions might not be handy for you if you do not have remote access to the server. In this case, simply select all the query data and copy it. Open an excel file and you could paste it. Simple !! Tweaked.
You might have tough time if your query result is too much though.
Try this command:
COPY (select * from product_template) TO 'D:\Product_template_Output.csv' WITH CSV;
In PgAdmin export option is available in file menu.Execute the query, and then we can view the data in the Output pane. Click on the menu FILE -> EXPORT from query window.
PSQL to export data
COPY noviceusers(code, name) FROM 'C:\noviceusers.csv' DELIMITER ',' CSV HEADER;
https://www.novicetechie.com/2019/12/export-postgresql-data-in-to-excel-file.html for reference.
Write your query to select data on the query tool and execute
Click on the download button on the pgAdmin top bar (selected in red)
Rename the file to your liking
Select which folder to save the file
Congrats!!!
I want to export a postgresql table to a csv file.
I have tried two ways, however both are unsuccessful for different reasons.
In the first case, you can see what I run and what I get bellow:
COPY demand.das_april18_pathprocess TO '/home/katerina/das_april18_pathprocess.csv' DELIMITER ',' CSV HEADER;
No such file or directory
SQL state: 58P01
I need to mention that in the location /home/katerina/ I have created an empty file named das_april18_pathprocess.csv, for which I modified the Permission settings to allow Read and Write.
In my second try, the query is executed without any errors but I cannot see the csv file. The command that I run is the following:
COPY demand.das_april18_pathprocess TO '/tmp/das_april18_pathprocess.csv' DELIMITER ',' CSV HEADER;
In the /tmp directory there is no cvs file.
Any advice on how to export the table to csv file with any way is really appreciated!
Ah, you run into a common problem -- you're creating a file on the server's filesystem, not your local filesystem. That can be a pain.
You can, however, COPY TO STDOUT, then redirect the result.
If you're using linux or another unix, the easiest way to do this is from the command line:
$ psql <connection options> -c "COPY demand.das_april18_pathprocess TO STDOUT (FORMAT CSV)" > das_april18_pathprocess.csv
copy ( select * from demand.das_april18_pathprocess) to '/home/katerina/das_april18_pathprocess.csv' with CSV header ;
I am writing a shell script that fetches data (.csv file) form AWS S3, downloads it locally onto an EC2 Linux AMI Instance, and then copies the data to an RDS PostGresql database.
My Shell code is the following:
FILE="$(ls DB)"
PARAMETERFORDB= "'\\COPY table(x,y) FROM ''$FILE'' CSV HEADER'"
$(psql --host=XXXXX --port=XXXXX --username=XXXXX --password --dbname=XXXXX -c ${PARAMETERFORDB})
So when the data from S3 is downloaded, I store the files' name inside the FILE variable (it is the only file in the folder, the folder will be deleted after the Database query).
I get following error message:
./shellTest.sh: line 21: '\COPY table(x,y) FROM ''14.9.2016.csv'' CSV HEADER': command not found
psql: option requires an argument -- 'c'
Try "psql --help" for more information.
What am I doing wrong?
In the line
PARAMETERFORDB= "'\\COPY table(x,y) FROM ''$FILE'' CSV HEADER'"
remove the space after the = and remove one level of single quotes:
PARAMETERFORDB="\\COPY table(x,y) FROM '$FILE' CSV HEADER"
In the line where psql is invoked, enclose ${PARAMETERFORDB} in double quotes since it contains spaces.
I'd like to have a simple vbscript or even just a commandline one-liner that allows me to (in pseudocode):
for each file FL (*.ts) in folder
run command 'ffprobe -show_frames -select_streams 1 -print_format csv -i "folder\FL" > "folder\FL.Name.csv"'
next
Like the code says, I only want it to work on files of type .TS, and I need to run each file in the folder through that commandline and redirect the stdout to a csv file. I could create a small program in C#, but I'd rather have a lightweight script or something and I'm new to that. Any ideas?
Processing files with a particular extension in a given folder can be done like this:
Set sh = CreateObject("WScript.Shell")
Set fso = CreateObject("Scripting.FileSystemObject")
For Each f In fso.GetFolder("C:\foo").Files
If fso.GetExtensionName(f) = "ts" Then
sh.Run "...", 0, True
End If
Next
Redirecting the output to a CSV file is (IMHO) best done by running the VBScript with cscript.exe and redirecting STDOUT to a file:
cscript your.vbs > output.csv