I've got a .bch file that i usually manually edit whenever I need to restore a database. The changes involve removing # from the start of specific lines and changing the database name of the lines where the # is removed to a new name that will be unique every time the script is run.
#DATABASE "YYYYY"
MOVE "YYYYYY"
#TO "H:\MSSQL.1\Data\YYYYY.mdf"
Change to
DATABASE "XXXXX"
MOVE "YYYYYY"
TO "H:\MSSQL.1\Data\XXXXX.mdf"
Would this be possible to do via a script in powershell?
Adding to question as I was pretty vague:
Hi Team, Sorry for the vague question I will get better as I ask more I'm sure. I'm aware of the replace function but in my research I couldn't really find anything to find specific characters. A function involving something like:
"On line Where character 1 from the left = # replace YYYY with XXX"
Then Step two of the shell script could be:
"on line Where string "XXXX" exists delete character 1 from the left"
I just can't find on google if functions like this exist
You haven't given much to go on, but have a start with:
cat somefile.bch | %{$_ -replace "expression","replace"} > newfile.bch
Related
I have a lengthy SQL script which contains MANY/multiple sections like this (amongst other script sections):
USE [NAVDB]
GO
IF NOT EXISTS (SELECT name FROM sys.database_principals
WHERE name = '##Placeholder##')
CREATE USER [MyDomain\adcsuser]
FOR LOGIN [MyDomain\adcsuser]
WITH DEFAULT_SCHEMA = [MyDomain\adcsuser]
GO
GRANT CONNECT TO [MyDomain\adcsuser] AS [dbo]
GO
I need to parse this script tile and modify only the IF NOT EXISTS...CREATE USER... lines of the script such that "##Placeholder##" is replaced by the text within the square brackets [] in the same line immediately following the CREATE USER string.
So, the line in the above snippet would become:
IF NOT EXISTS (SELECT name FROM sys.database_principals
WHERE name = 'MyDomain\adcsuser')
CREATE USER [MyDomain\adcsuser]
FOR LOGIN [MyDomain\adcsuser]
WITH DEFAULT_SCHEMA = [MyDomain\adcsuser]
The file is many hundreds of lines long with many dozen (at least) of these sections in it.
NotePad++ find-replace and macros couldn't handle it because of the "\" in the names between the []s and I couldn't find how to make NP++ copy the text between []s.
Also, I tried reviewing other related answers like: How can I replace every occurrence of a String in a file with PowerShell?, but remain stymied so far.
Because of the complex nature of the script file structure I'm leery of "read the whole file and just use Regex find/replace" approaches, so I was looking for a more... RBAR (Row By Agonizing Row) approach.
Dynamic SQL or parameterized approaches and similar suggestions are NOT APPROPRIATE for this situation as THE SCRIPT WAS ALREADY DYNAMICALLY GENERATED from an existing Production system (and I do not have the source code for the utility which generated this as output). I literally can't make wholesale structural changes like that.
Also, after reading this post once again, I should point out that the whole "IF NOT EXISTS...WITH DEFAULT_SCHEMA [*]" command is on ONE LINE in the script file (if that wasn't sufficiently clear).
For NotePad++ the find and replace support regex.
Example of how to find and replace all lines containing "CREATE USER [someusername]" with your replacement ##Placeholder## would be:
The .* is wildcard, brackets are special characters in regex so to include them as part of the search you have to escape them. So \[.*\] would find anything wrapped in brackets. I just added CREATE USER as an example on finding all those specific lines.
Making sure to select "Regular expression" in Search Mode.
PowerShell, with everything on one line, you can read in each line, find the match, extract the user and then replace; copying that line back out to a new file.
Input test file:
Example PowerShell script:
#create new output file
new-item "C:\temp\test2.txt" -Force
foreach ($line in (get-content "C:\temp\test.txt"))
{
#Find the the user between the brackets and remove the brackets.
$user = ([regex]"\[.*\]").Match($line).value.replace("[","").replace("]","")
#Replace PlaceHolder with user pulled from the brackets and write to new file
$line -replace '##PlaceHolder##', $user | out-file "C:\temp\test2.txt" -Append
}
Then the contents of the output file:
#Tim Mylott,
The entire issue arose from a script that was generated off my production system from Powershell's DBATools' Export-DbaLogin command.
The output script from that tool was like your first text.txt file in that it issues blind CREATE USER commands without first testing to see if the user already existed.
This was inadequate for our needs (our production DB and system have been around for years and now has old, bad 'lint' all over the place in it), so I took the script output into Notepad++ and added the IF NOT EXIST... test logic (see the script in the original question for details) as a prefix to the CREATE USER commands in the script with a global search & replace. Only, I had to put ##Placeholder## in for the username in the SELECT's WHERE clause for the test.
This left me with a script where I had to replace the ##Placeholder## text with the actual username strings in the existing CREATE USER text on the same line in the script file.
The solution, in NP++, which you led me to, was to use Regex in the NP++ Search to select the userid string in the same line. From there, it was fairly easy and straightforward to use a NP++ macro recording to automate the search & replace.
First, I found online a regex to select the text between the first matching pair of []s, which is: (?<=[).+?(?=])
The NP++ script recording was basically this (start Macro recording):
Find:##PlaceHolder## (non-regex search) [NP++ finds the next line to alter]
[Home] (to find the beginning of the line)
Find: (?<=[).+?(?=]) (regex search) [NP++ selects the UserID string in the CREATE USER part of the line)
[Ctrl+C] (copy UserID string to clipboard once Find dialog is closed)
[Home] (to find the beginning of the line again)
Find:##PlaceHolder## (non-regex search) [NP++ finds the ##Placeholder## text in the current line and selects it]
[Ctrl+V] (paste in the UserID string from the clipboard)
(this leaves the cursor on the same location where the corrected command can be visually verified as correct)
In NP++ Use the Play Macro button to manually replay this recorded macro a few times to assure the macro has it right...
Then, use the play multiple times (until the end of file is reached), and Voila!
The script file is all fixed up.
[Note: the DBATools GitHub now has 2 bug reports/requests to add this IF NOT EXISTS logic to the CREATE USER lines in the generated script, so this issue will eventually go away, but for now... there is at least one reasonable solution using NotePad++.]
So, Thank You, Tim, for leading me to a quick answer.
That said, though, your Powershell part of the answer was wrong, so no points there! (it was doing the wrong thing and putting "##Placeholder##" INTO the script, rather than replacing the existing "##PlaceHolder##" string in the script with the actual UserID strings already IN the script.
To repeat, the problem was (logically) to go:
-- FROM THIS:
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUser2]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUser3]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUser5]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUser1]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUser7]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUser8]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUserA]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUserB]...
IF NOT EXISTS (SELECT ... WHERE name = '##PlaceHolder##') CREATE USER [MyDomain\AUserC]...
-- TO THIS:
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUser2') CREATE USER [MyDomain\AUser2]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUser3') CREATE USER [MyDomain\AUser3]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUser5') CREATE USER [MyDomain\AUser5]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUser1') CREATE USER [MyDomain\AUser1]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUser7') CREATE USER [MyDomain\AUser7]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUser8') CREATE USER [MyDomain\AUser8]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUserA') CREATE USER [MyDomain\AUserA]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUserB') CREATE USER [MyDomain\AUserB]...
IF NOT EXISTS (SELECT ... WHERE name = 'MyDomain\AUserC') CREATE USER [MyDomain\AUserC]…
Revised Script from Tim's excellent start (version 2) :-)
#create new output file
new-item "C:\temp\test2.sql" -Force
foreach ($line in (get-content "C:\temp\test.sql"))
{
if ($line.Contains("##Placeholder##"))
{
#Find the the user between the brackets and remove the brackets.
$user = ([regex]"\[.+?\]").Match($line).value.replace("[","").replace("]","")
#Replace PlaceHolder with user pulled from the brackets and write to new file
$line -replace '##Placeholder##', $user | out-file "C:\temp\test2.sql" -Append
}
else
{
out-file "C:\temp\test2.sql" -Append -InputObject $line
}
}
...Had to tweak the Regex (to less permissive) and output all the rest of the script lines too.
First, I would like to apologize in case that the title is not descriptive enough, I'm having a hard time dealing with this problem. I'm trying to build an automation for a svn merge using a powershell script that will be executed for another process. The function that I'm using looks like this:
function($target){
svn merge $target
}
Now, my problem occurs when there are conflicts in the merge. The default behavior of the command is request an input from the user and proceed accordingly. I would like to automatize this process using predefined values (show the differences and then postpone the merge), but I haven't found a way to do it. In summary, the workflow that I am looking to accomplish is the following:
Detect whether the command execution requires any input to proceed
Provide a default inputs (in my particular case "df" and then "p")
Is there any way to do this in powershell? Thank you so much in advance for any help/clue that you can provide me.
Edit:
To clarify my question: I would like to automatically provide a value when a command executed within a powershell script require it, like in the following example:
Requesting user input
Edit 2:
Here is a test using the snippet provided by #mklement0. Unfortunately, It didn't work as expected, but I thought it was wort to add this edition to clarify the question per complete
Expected behavior:
Actual result:
Note:
This answer does not solve the OP's problem, because the specific target utility, svn, apparently suppresses prompts when the process' stdin input isn't coming from a terminal (console).
For utilities that do still prompt, however, the solution below should work, within the constraints stated.
Generally, before attempting to simulate user input, it's worth investigating whether the target utility offers programmatic control over the behavior, via its command-line options, which is both simpler and more robust.
While it would be far from trivial to detect whether a given external command is prompting for user input:
you can blindly send the presumptive responses,
which assumes that no situational variations are needed (except if a particular calls happens not to prompt at all, in which case the input is ignored).
Let's assume the following batch file, foo.cmd, which puts up 2 prompts and echoes the input:
#echo off
echo begin
set /p "input1=prompt 1: "
echo [%input1%]
set /p "input2=prompt 2: "
echo [%input2%]
echo end
Now let's send responses one and two to that batch file:
C: PS> Set-Content tmp.txt -Value 'one', 'two'; ./foo.cmd '<' tmp.txt; Remove-Item tmp.txt
begin
prompt 1: one
[one]
prompt 2: two
[two]
end
Note:
For reasons unknown to me, the use of an intermediate file is necessary for this approach to work on Windows - 'one', 'two' | ./foo.cmd does not work.
Note how the < must be represented as '<' to ensure that it is passed through to cmd.exe and not interpreted by PowerShell up front (where < isn't supported).
By contrast, 'one', 'two' | ./foo does work on Unix platforms (PowerShell Core).
You can store the SVN command line output into a variable and parse through that and branch as you desire. Each line of output is stored into a new enumerator (cli output stored in PS variables is in array format)
$var = & svn merge $target
$var
How do I take a substring where I don't know the length of the thing I want, but I know that the end of it is a CR/LF?
I'm communicating with a server trying to extract some information. The start point of the substring is well defined, but the end point can be variable. In other scripting languages, I'd expect there to be a find() command, but I haven't found one in PowerShell yet. Most articles and SE questions refer to Get-Content, substring, and Select-String, with the intent to replace a CRLF rather than just find it.
The device I am communicating with has a telnet-like command structure. It starts out with it's model as a prompt. You can give it commands and it responds. I'm trying to grab the hostname from it. This is what a prompt, command, and response look like in a terminal:
TSS-752>hostname
Host Name: ThisIsMyHostname
TSS-752>
I want to extract the hostname. I came across IndexOf(), which seems to work like the find command I am looking for. ":" is a good start point, and then I want to truncate it to the next CRLF.
NOTE: I have made my code work to my satisfaction, but in the interest of not receiving anymore downvotes (3 at the time of this writing) or getting banned again, I will not post the solution, nor delete the question. Those are taboo here. Taking into account the requests for more info from the comments has only earned me downvotes, so I think I'm just stuck in the SO-Catch-22.
You could probably have found the first 20 examples in c# outlining this exact same approach, but here goes with PowerShell examples
If you want to find the index at which CR/LF occurs, use String.IndexOf():
PS C:\> " `r`n".IndexOf("`r`n")
2
Use it to calculate the length parameter argument for String.Substring():
$String = " This phrase starts at index 4 ends at some point`r`nand then there's more"
# Define the start index
$Offset = 4
# Find the index of the end marker
$CRLFIndex = $string.IndexOf("`r`n")
# Check that the end marker was actually found
if($CRLFIndex -eq -1){
throw "CRLF not found in string"
}
# Calculate length based on end marker index - start index
$Length = $CRLFIndex - $Offset
# Generate substring
$Substring = $String.Substring($Offset,$Length)
I have been searching but so far I only found how to insert date into tables based on a csv files.
I have the following scenario:
Directory name = ticketID
Inside this directory I have a couple of files, like:
Description.txt
Summary.txt - Contains ticket header and has been imported succefully.
Progress_#.txt - this is everytime a ticket gets udpdated. I get a new file.
Solution.txt
Importing the Issue.txt was easy since this was actually a CSV.
Now my problem is with Description and Progress files.
I need to update the existing rows with the data from this files. Something on the line of
update table_ticket set table_ticket.description = Description.txt where ticket_number = directoryname
I'm using PostgreSQL and the COPY command is valid for new data and it would still fail due to the ',;/ special chars.
I wanted to do this using bash script, but it seem that it is it won't be possible:
for i in `find . -type d`
do
update table_ticket
set table_ticket.description = $i/Description.txt
where ticket_number = $i
done
Of course the above code would take into consideration connection to the database.
Anyone has a idea on how I could achieve this using shell script. Or would it be better to just make something in Java and read and update the record, although I would like to avoid this approach.
Thanks
Alex
Thanks for the answer, but I came across this:
psql -U dbuser -h dbhost db
\set content = `cat PATH/Description.txt`
update table_ticket set description = :'content' where ticketnr = TICKETNR;
Putting this into a simple script I created the following:
#!/bin/bash
for i in `find . -type d|grep ^./CS`
do
p=`echo $i|cut -b3-12 -`
echo $p
sed s/PATH/${p}/g cmd.sql > cmd.tmp.sql
ticketnr=`echo $p|cut -b5-10 -`
sed -i s/TICKETNR/${ticketnr}/g cmd.tmp.sql
cat cmd.tmp.sql
psql -U supportAdmin -h localhost supportdb -f cmd.tmp.sql
done
The downside is that it will create always a new connection, later I'll change to create a single file
But it does exactly what I was looking for, putting the contents inside a single column.
psql can't read the file in for you directly unless you intend to store it as a large object in which case you can use lo_import. See the psql command \lo_import.
Update: #AlexandreAlves points out that you can actually slurp file content in using
\set myvar = `cat somefile`
then reference it as a psql variable with :'myvar'. Handy.
While it's possible to read the file in using the shell and feed it to psql it's going to be awkward at best as the shell offers neither a native PostgreSQL database driver with parameterised query support nor any text escaping functions. You'd have to roll your own string escaping.
Even then, you need to know that the text encoding of the input file is valid for your client_encoding otherwise you'll insert garbage and/or get errors. It quickly lands up being easier to do it in a langage with proper integration with PostgreSQL like Python, Perl, Ruby or Java.
There is a way to do what you want in bash if you really must, though: use Pg's delimited dollar quoting with a randomized delimiter to help prevent SQL injection attacks. It's not perfect but it's pretty darn close. I'm writing an example now.
Given problematic file:
$ cat > difficult.txt <__END__
Shell metacharacters like: $!(){}*?"'
SQL-significant characters like "'()
__END__
and sample table:
psql -c 'CREATE TABLE testfile(filecontent text not null);'
You can:
#!/bin/bash
filetoread=$1
sep=$(printf '%04x%04x\n' $RANDOM $RANDOM)
psql <<__END__
INSERT INTO testfile(filecontent) VALUES (
\$x${sep}\$$(cat ${filetoread})\$x${sep}\$
);
__END__
This could be a little hard to read and the random string generation is bash specific, though I'm sure there are probably portable approaches.
A random tag string consisting of alphanumeric characters (I used hex for convenience) is generated and stored in seq.
psql is then invoked with a here-document tag that isn't quoted. The lack of quoting is important, as <<'__END__' would tell bash not to interpret shell metacharacters within the string, wheras plain <<__END__ allows the shell to interpret them. We need the shell to interpret metacharacters as we need to substitute sep into the here document and also need to use $(...) (equivalent to backticks) to insert the file text. The x before each substitution of seq is there because here-document tags must be valid PostgreSQL identifiers so they must start with a letter not a number. There's an escaped dollar sign at the start and end of each tag because PostgreSQL dollar quotes are of the form $taghere$quoted text$taghere$.
So when the script is invoked as bash testscript.sh difficult.txt the here document lands up expanding into something like:
INSERT INTO testfile(filecontent) VALUES (
$x0a305c82$Shell metacharacters like: $!(){}*?"'
SQL-significant characters like "'()$x0a305c82$
);
where the tags vary each time, making SQL injection exploits that rely on prematurely ending the quoting difficult.
I still advise you to use a real scripting language, but this shows that it is indeed possible.
The best thing to do is to create a temporary table, COPY those from the files in question, and then run your updates.
Your secondary option would be to create a function in a language like pl/perlu and do this in the stored procedure, but you will lose a lot of performance optimizations that you can do when you update from a temp table.
My script reads a log file once a minute and selects (and acts upon) the lines where the timestamp begins with the previous minute.
This is easy (the regex is simply "^$timestamp"), but when the log gets big it can take a while.
My thinking is the lines I want will always be near the bottom of the file, so I'd be searching far fewer lines if I started at the bottom and searched upwards, stopping when I get to the minute prior to the one I'm interested in.
My question is, how can I search from the bottom of the file instead of the top? Can I even say "read line $length", or even "read line n" (if so I could do a sort of binary search thing to find the length of the file and work backwards from there)?
Last question: would this even be faster (I'd still like to know how to do it even if it wouldn't be faster)?
Ideally, I'd like to do this all in my own code without installing anything extra.
Thanks
get-content bigfile.txt -tail 10
This words on huge files nearly instantly without any big memory usage.
I did it with a 22 GB text file in my testing.
Doing something like "get-context bigfile.txt | select -Last 10" works but it seems to have to load all of the lines (or objects in powershell) then does the select.
May I suggest just changing the regex to equal Get-Date + whatever time period you want?
For example (and this is without your log so i apologize)
$a = Get-Date
$hr = $a.Hour
$min = $a.Minute
Then work off those values to build out the regex to select the times you want. And if you don't already use it this website is awesome for building regex's quickly and easily http://gskinner.com/RegExr/ .
Got another fix, I think you will like this..
$a = get-content .\biglog.text
Use the length to slice the array from back to front change write host to select-string and your regex or whatever you want to do in reverse..
foreach($x in $a.length..0){ write-host $a[$x] }
Another option after the get-content cmdlet again, this option just reverse orders the array then you are reading $a from bottom to top
[array]::Reverse($a)
dc
If you only want the last bit of the file, depending on the format, you can just do this:
Get-Content C:\Windows\WindowsUpdate.log | Select -last 10
This will return the last 10 lines found in the file.