This one is blowing my mind a bit. I am able to BCP OUT a file fine with just -c but I need the data to be pipe delimited so I include -t"|" and the data becomes all special characters. Here's the code:
SET #bcpsql =
' cd C:\Program Files\Microsoft SQL Server\100\Tools\Binn\ & BCP.EXE ' +
' "SELECT * FROM wire" queryout' +
' "C:\WIRE_'+#dte+'.txt" -T -c -t"|" >> "C:\mantas\log.txt"';
EXEC master..xp_cmdshell #bcpsql
If I put -t"^|" it will export fine but all the delimiters become ^|. So, somehow the pipe is throwing something off. Has anyone encountered something like this?
You need the space between the parameter and the pipe, but you also need the double quotes.
use -t "|"
(-T[space]"|")
Related
I'm attempting to dynamically create a script that gets saved as a bat file that will be scheduled to execute daily via Windows Task Scheduler. The script performs full database backups for each Postgres database using pg_dump.
The current script is as follows:
COPY (SELECT 'pg_dump '|| datname || ' > e:\postgresbackups\FULL\' || datname || '_%date:~4,2%-%date:~7,2%-%date:~10,4%_%time:~0,2%_%time:~3,2%_%time:~6,2%.dump' FROM pg_database) TO 'E:\PostgresBackups\Script\FULL_Postgres_Backup_Job_TEST.bat' (format csv, delimiter ';');
An example of the output is as follows:
pg_dump postgres > e:\postgresbackups\FULL\postgres_%date:~4,2%-%date:~7,2%-%date:~10,4%%time:~0,2%%time:~3,2%_%time:~6,2%.dump
I need help with updating my code so that the output will include double quotes around the name of the dump file; however, when I add this to my COPY script it adds more than what is necessary to the output. I would like the output to look like the following which includes the double-quotes:
pg_dump postgres > "e:\postgresbackups\FULL\postgres_%date:~4,2%-%date:~7,2%-%date:~10,4%%time:~0,2%%time:~3,2%_%time:~6,2%.dump"
Any help would be greatly appreciated!
Thanks to #Mike Organek's comment, my issue has been resolved by switching the format from CSV to TEXT. Now when I enclose the dump filename in double quotes, the output is more of what is expected and works as intended. The only odd thing now is that in the output it creates a second backslash in the filename. My code has been updated as follows:
COPY (SELECT 'pg_dump '|| datname || ' > "e:\postgresbackups\FULL\' || datname || '_%date:~4,2%-%date:~7,2%-%date:~10,4%_%time:~0,2%_%time:~3,2%_%time:~6,2%.dump"' FROM pg_database) TO 'E:\PostgresBackups\Script\FULL_Postgres_Backup_Job.bat' (format text, delimiter ';');
An example of the output that gets created within the bat file is as follows:
pg_dump postgres > "e:\\postgresbackups\\FULL\\postgres_%date:~4,2%-%date:~7,2%-%date:~10,4%_%time:~0,2%_%time:~3,2%_%time:~6,2%.dump"
As you can see, it adds a double backslash; however, the pg_dump executes successfully!
I would like to join strings for file name and shell script so I can run the command line in kdb for ftp transfer.
But there are quotations in quotation marks. not sure how to add / in there.
This is the code I have:
host:"abc.com";
usr:"def";
path:"get /home/eddie/abc.csv /home/terry/";
cmd:" " sv ("/home/kdb/eddie/ftp.sh";host;usr;path);
system cmd;
So the path will not have quotation mark and will be running error. How can I solve this problem?
You can escape quotes with \ e.g. "\"Matt\"" but I don't think that's your issue. It looks like you are attempting to use get in the system command. This is a kdb keyword and your OS will not recognise it. You should just be passing the location of the csv to your ftp script.
Edit:
You may also need sh in the system command.
cat test.sh
echo $1
system "test.sh hello"
sh: ./test.sh: Permission denied
'os
system "sh test.sh hello"
"hello"
Assuming that you simply want quotes within a string, it may just be as simple as using .Q.s1 aka -3!, see https://code.kx.com/q/ref/dotq/#qs1-string-representation
q)" " sv ("/home/kdb/eddie/ftp.sh";host;usr;.Q.s1 path)
"/home/kdb/eddie/ftp.sh abc.com def \"get /home/eddie/abc.csv /home/terry/\""
q)" " sv ("/home/kdb/eddie/ftp.sh";host;usr;-3!path)
"/home/kdb/eddie/ftp.sh abc.com def \"get /home/eddie/abc.csv /home/terry/\""
I want to create scipt to faciliate producing local text file extracts from Hive.
This is to basically execute commands like below:
hive -e "SET hive.cli.print.header=true;SELECT * FROM dropme"|perl -pe 's/(?:\t|^)\KNULL(?=\t|$)//g'>extract/outbound/dropme.txt
While the above works like a charm I find it quite problematic to implement through the parametrized following script (much simplified):
#!/bin/sh
TNAME=dropme
SQL="SELECT * FROM $TNAME"
echo $SQL
echo "SQL: $SQL"
EXTRACMD="hive -e \"SET hive.cli.print.header=true;$SQL\"|perl -pe 'BEGIN{if(defined(\$_=<ARGV>)){s/\b\w+\.//g;print}}s/(?:\t|^)\KNULL(?=\t|$)//g'>extract/outbound/$TNAME.txt"
echo "CMD: $EXTRACMD";
${EXTRACMD}
When run I get: Exception in thread "main" java.lang.NumberFormatException: For input string: "e"
I know there may be many flavours you can print the text or execute command. For instance the line echo $SQL prints me list of files in the directory instead:
SELECT file1.txt file2.txt file3.txt file4.txt FROM dropme
while the next one: echo "SQL: $SQL" gives just what I want: SQL: SELECT * FROM dropme
echo "CMD: $EXTRACMD" prints the (almost) the command to be executed. Almost, as I see \t in perl code being expanded:
CMD: hive -e "SET hive.cli.print.header=true;SELECT * FROM dropme"|perl -pe 'BEGIN{if(defined($_=<ARGV>)){s\w+\.//g;print}}s/(?: |^)\KNULL(?= |$)//g'>extract/outbound/dropme.txt
Maybe that's still ok, but what I want is to be able to copy&paste this command into (other) terminal and execute as the command I put at the top. Ideally I would like that command to be exactly the same (so with \t there)
Biggest problem I have comes when I try to execute it (${EXTRACMD} line). I'm getting the error:
Exception in thread "main" java.lang.NumberFormatException: For input string: "e" …and so on, irrelevant as bash treats every 'word' as single command here. I assume as I don't even know what is really tries to run (prior print attempt obviously doesn't help)
I'm aware that I have multiple options, like:
escaping special characters in the command definition string (like I did with doublequotes)
experimenting with echo and $VAR, '$VAR' or "$VAR"
experimenting with "${EXTRACMD}" or evaluating through eval "${EXTRACMD}"
experimenting with shopt -s extglob or set -f
but as number of combinations is quite large and with my little bash experience I feel it's better to ask for good practice here so my question is:
Is there a way to print a (complex/compound shell) command first and subsequently be able to execute it (exactly as per printed output)? In this case it would be printing the exact command from the top, then executing it the same way as by manually copying that output into terminal prompt and pressing Enter.
Do not construct commands as strings. See http://mywiki.wooledge.org/BashFAQ/050 for details.
That page also talks about a built-in way of getting the shell to tell you what it is running (section 6).
If that doesn't do what you want you can also, with bash, try using printf %q\\n "${arr[*]}".
I have a table that contains a Blob field to store a PDF or image (jpeg, gif, ...) file. There's a Java EE application to do this and it works fine.
But, sometimes I need to insert or update the Blob field via a command.
Is there any simple way to do this without coding? I'm looking for a pure command or some kind of tool.
DB2 is v9 at AIX and Windows.
Unfortunately I don't have DB2 handy right now but I think I can point you in the correct direction:
#!/bin/sh
FILENAME="$1"
HEX=`xxd -p $FILENAME | tr -d '\n'`
echo insert into $2 \($3\) values \(cast\(X\'$HEX\' as blob\( $4 \) \) \);
The idea is to use xxd to create proper hexadecimal string of your file. (AIX has xxd installed unless it is broken.) The hexadecimal string can then be used with the db2 command line processor ("db2") in SQL queries. All you have to do is to cast the hexadecimal string to blob.
The script I gave you will print out correct SQL query for testing. Fix it to match your DDL and other requirements. Then change the echo command to db2 command and you should have a working utility:
./myscript myfile mytable myblob myblobsize
I'm trying to build the .NET assembly file by executing this code in matlab2010b
workdir = 'C:\Users\H\Documents\Source Code\MatlabFiles';
outdir = fullfile(workdir, 'Output');
dnetdir = fullfile(workdir, 'dotnet');
%% Determine file names
mfile = fullfile(workdir, 'perform.m');
dnetdll = fullfile(dnetdir, 'dotnet.dll');
%% Create directories if needed
if (exist(outdir, 'dir') ~= 7)
mkdir(outdir);
end
if (exist(dnetdir, 'dir') ~= 7)
mkdir(dnetdir);
end
%% Build .NET Assembly
eval(['mcc -N -d ' dnetdir ' -W ''dotnet:dotnet,' ...
'dotnetclass,0.0,private'' -T link:lib ' mfile]);
I'm getting this error.
??? Error using ==> mcc
The output directory,
'C:\Users\H\Documents\Project\thesis\Source'
does not exist.
I'm pretty sure it's because of the space in the directory path "...\Source Code\...".
Because if I just use another path with no spaces it works perfectly fine.
Is there a way to make this work?
Thank you.
I think the actual problem occurs with your EVAL statement. You build a string to evaluate by concatenating strings like dnetdir and mfile, each of which will have a file path with a space in it. The resulting string you pass to EVAL will look like this:
mcc -N -d C:\Users\H\Documents\Source Code\MatlabFiles\dotnet -W ...
^--Look at that ugly space!
What you need to do is build your string so that there are apostrophes around these paths, like this:
eval(['mcc -N -d ''' dnetdir ''' -W ''dotnet:dotnet,' ...
'dotnetclass,0.0,private'' -T link:lib ''' mfile '''']);
Which will result in a string that looks like this:
mcc -N -d 'C:\Users\H\Documents\Source Code\MatlabFiles\dotnet' -W ...
And which will be evaluated properly now even with that nasty space in there.
I don't have any experience with mcc but some other functions may suffer from similar problems since most people are used to using the command mode (i.e. similar to the command prompt in DOS, Linux, Mac, ...). However, most functions are really functions such that you can use them in function mode and pass their arguments within parentheses.
You can also use mcc in function mode, as described in the help. That might look somewhat like:
mcc('-N', '-d', dnetdir, '-W', 'dotnet:dotnet,dotnetclass,0.0,private', '-T', 'link:lib', mfile);
That way you should not have to worry about escaping any character.
try changing the last line to:
eval(['mcc -N -d ''' dnetdir ''' -W ''dotnet:dotnet,' ...
'dotnetclass,0.0,private'' -T link:lib ' mfile]);
note the extra quotes around dnetdir