I'm trying to run the following command in Windows Server 2003 but sed creates a pile of files that I can't delete from the command line inside the current directory.
for /R %f in (*.*) do "C:\Program Files\gnuwin32\bin\sed.exe" -i "s/bad/good/g" "%f"
Does anyone have any suggestions? Mysteriously enough, I'm able to delete the files using Windows Explorer.
As requested, here are some example filenames:
sed0E3WZJ
sed5miXwt
sed6fzFKh
And, more troubleshooting info...
It occurs from both the command prompt & batch files
If I just need to run sed on a single directory, then I use sed "s/bad/good/g" *.* and everything is OK. Alas, I also need it to tackle all the subdirectories.
I only have Sed installed.
Sed is creating the files
I have replicated your setup and I have the following observations.
I dont think there is a problem in the loop. The simple command "C:\Program Files\gnuwin32\bin\sed.exe" -i "s/bad/good/g" . - creates the same set of temporary files.
The files are indeed created by sed. sed creates these temporary files when the "in place" (-i) option is turned on. In the normal course, sed actually deletes the files (that is what happens in cygwin) using a call to the 'unlink' library. In case of gnuwin32, it looks like the 'unlink' fails. I have not been able to figure out why. I took a guess that maybe the unlink call is dependent on the gnuwin32 'coreutils' library and tried to download and install the coreutils library - no dice.
If you remove the 'read-only' restriction in the parent folder before executing the sed command, you can delete the temporary files from windows command prompt. So that should give you some temporary respite.
I think we now have enough information to raise a bug report. If you agree, I think it may be a good idea to bring it to the notice of the good folks responsible for gnuwin32 and ask them for help.
Meanwhile, the following version cleans up its temporary file:
https://github.com/mbuilov/sed-windows
As this is a known bug in sed with the -i option you can run attrib -R <filename> to remove the read only attribute from file after sed completes.
Alternatively do not use the -i option and redirect the output to a new file and then delete and rename the input and output.
Cygwin hoses the ACLs on files sometimes, you'll probably have to use cacls or chmod to fix it up before you can delete the file.
Here is where a bit of troubleshooting comes into play. Does this happen when you run that command from the command-line and a batch file? What if you run sed on an individual file on the command line - does it create these files for every file, or just certain files/filetypes? Does it only happen for that replacement, or all replacements in general, or just always when you run sed.exe on a file? Is it only sed creating these files, or all Gnuwin32 exe's (eg. awk, cat, etc)? Does the same thing happen on a sed.exe from a new install of Gnuwin32? What error message does it give when you try to delete the files? Can you delete the files from explorer while the command prompt is still open? What if you close the command prompt and reopen it, then try to delete the files?
you can just run sed without for loop
c:\test> sed -i.bak "s/bad/good/g" file*.*
This is a stab in the dark, but it wouldn't surprise me in the least if the gnuwin32 implementation of sed is duff (i.e., faulty in some way). Can you try to replicate the problem using the AT&T U/Win POSIX support for windows? It is easy to install and includes the Korn shell, sed, and find, so you can use find instead of the FOR /R. (I'm wondering if part of the problem is that the MS FOR and gnuwin32 sed don't play nicely together.)
I realize this is an old thread but It's still an issue. My fix is to add
"DEL sed*" to the end of a batch file after sed. Quick and dirty.
I am using this command to clean up the temporary files created by gnuwin32's sed:
FOR /f "tokens=*" %%a in ('dir /b ^| findstr /i "^sed[0-9a-zA-Z][0-9a-zA-Z][0-9a-zA-Z][0-9a-zA-Z][0-9a-zA-Z][0-9a-zA-Z]$"') DO del %%a
i know this is old. But just want to share with people what I did that cause this.
It was in fact the temp file for an already open file through gvim example .swap file that causing the sed tmp file didnt get remove completely.
So sed trying to read and append into the opened file which the user is currently viewing and having trouble doing it which causes the tmp file error.
Related
I just have to install a software cvxopt for a project and the instructions tell me to do this:
Remove all instances of ‘-mno-cygwin’ from c:\Python27\Lib\distutils\cygwinccompiler.py using the command:
sed 's/-mno-cygwin//g' -i'.bak' c:\Python27\Lib\distutils\cygwinccompiler.py
Every time I try to run the command , it says unknown command ' ' '.
I don't know what sed is , I just want to install cvxopt. I have been struggling with it for the last 2 hours. Please Help!
Two things:
Firstly, the -i option doesn't work on all sed versions for Windows. It allows in-place editing. If yours doesn't, create an intermediate file like this and then overwrite your original if/when you are happy with it.
cd \Python27\Lib\distutils
sed "s/-mno-cygwin//g" cygwinccompiler.py > x.tmp
del cygwinccompiler.py
ren x.tmp cygwinccompiler.py
Secondly, Windows doesn't understand single quotes, so use double quotes like this:
cd \Python27\Lib\distutils
sed "s/-mno-cygwin//g" -i".bak" cygwinccompiler.py
I have been trying to do a recursive grep command on files in sub folders using grep in NTemacs and Cygwin. So far the "best" results have been using grep in eshell. When I use this:
grep "t" -r *
I get a list of all file names containing the letter t, in all sub folders one layer down but notthing else. In Cygwin i get nothing. I'm working on a directroy that is not in the Cygwin install. Don't know if that mather or not.
What I want is to match the content of a more complex string in all files (and not just the file names, but the content). And in all sub directories.
I would like to use eshell from emacs but I'm open to suggestions, apart form using LINUX. This is a work PC and I don't want to do all the setup of a LINUX install.
i just wrote a very similar answer to another question, but i suspect it's the same root problem:
my first thought is that your files have windows line endings (CRLF) as opposed to unix/linux line endings (LF), and that is messing with grep's ability to parse the file. try running this:
dos2unix filename
on each file you need to search then try your grep statement again.
if you need to convert many files across several directories, i suggest using dos2unix with the -exec action of find:
find . -exec dos2unix {} \;
(add whatever other options you need to find before running that, of course)
I am looking for a safe and reliable way to overwrite ONE line in a text file. I don't care if it's using sed, grep, perl whatever. It just needs to be portable and reliable. Specifically what I am trying to do is replace the value of a variable I have saved in a text file at runtime. Let's say I have a file named variables.txt which contains a line that reads userName=stephen. Let's say my program wants to change the userName to frank. Here's what I've come up with using sed:
sed -i '' 's/userName.*/userName=frank/' variables.txt
The concern I have with this is that I've read that on some versions of sed using the '-i' switch without specifying a backup file could cause the command to fail or risk possible file corruption. Not an option. What do you guys think?
Edit
For those asking where I read about command failure and file corruption. The manpage for my version of sed recommends against providing an empty value for the -i switch as well as take a look at the comments on this page here:
It seems that some versions of sed require the argument after -i and others do not. With GNU sed version 4.1.x, it seems that the -i does not require an argument and specifying an empty argument after it actually fails.
It sounds like the unanimous recommendation is to provide a backup file and then delete it after the command completes. However I'm still concerned about this solution since my version of sed doesn't even support the --version switch. My primary concern here is that the solution is both reliable and portable.
t=`tempfile`
sed -e 's/userName.*/userName=frank/' variables.txt >$tempfile
cp $tempfile >variables.txt
rm $tempfile
you can also use mv but that won't preserve file rights
if tempfile is not available then use some other method ($$.bak) to create the filename.
As long as you don't run on windows, sed -i is as safe as anything. Even if the machine crashes mid-process, variables.txt will either have the old content or the new content -- it should never be missing or corrupt.
The concern I have with this is that I've read that on some versions of sed using the '-i' switch without specifying a backup file could cause the command to fail or risk possible file corruption.
Where have you read it?
It is not strictly true: this is not an usual problem in sed but all programs (including sed) can fail and in very unlikely situations corrupt data. So you should not be too afraid of data corruption with sed.
Anyway, why do you not use -i with an extension (such as -i.bak) for granting more safety? In any case you can erase the backup file with rm...
You could copy variables.txt to variables.txt.bak before running the command if you wanted to keep a backup. Or go the other way around:
sed 's/userName.*/userName=frank/' variables.txt > variables.txt.fixed
if [ $? -eq 0 ]
then
cp variables.txt.fixed variables.txt
fi
I'm wondering if there is a way to change a specific word in all of the files within the /www/ directory using command line. Just looking for a faster way to change out a specific word so I don't need to open all the files manually! Thanks!
find /www -type f -exec sed -i 's/foo/bar/g' \{\} \;
This line will replace foo with bar every time foo occurs in any file in /www. Be very sure you know what's under /www and what the replacement would do to those files before running it.
You might be looking for a grep-sed solution to find and replace, if you are on a Mac (and referring to the Mac's Terminal app).
The file is initially
$cat so/app.yaml
application: SO
...
I run the following command. I get an empty file.
$sed s/SO/so/ so/app.yaml > so/app.yaml
$cat so/app.yaml
$
How can you use SED to edit the file and not giving me an empty file?
$ sed -i -e's/SO/so/' so/app.yaml
The -i means in-place.
The > used in piping will open the output file when the pipes are all set up, i.e. before command execution. Thus, the input file is truncated prior to sed executing. This is a problem with all shell redirection, not just with sed.
Sheldon Young's answer shows how to use in-place editing.
You are using the wrong tool for the job. sed is a stream editor (that's why it's called sed), so it's for in-flight editing of streams in a pipe. ed OTOH is a file editor, which can do everything sed can do, except it works on files instead of streams. (Actually, it's the other way round: ed is the original utility and sed is a clone that avoids having to create temporary files for streams.)
ed works very much like sed (because sed is just a clone), but with one important difference: you can move around in files, but you can't move around in streams. So, all commands in ed take an address parameter that tells ed, where in the file to apply the command. In your case, you want to apply the command everywhere in the file, so the address parameter is just , because a,b means "from line a to line b" and the default for a is 1 (beginning-of-file) and the default for b is $ (end-of-file), so leaving them both out means "from beginning-of-file to end-of-file". Then comes the s (for substitute) and the rest looks much like sed.
So, your sed command s/SO/so/ turns into the ed command ,s/SO/so/.
And, again because ed is a file editor, and more precisely, an interactive file editor, we also need to write (w) the file and quit (q) the editor.
This is how it looks in its entirety:
ed -- so/app.yaml <<-HERE
,s/SO/so/
w
q
HERE
See also my answer to a similar question.
What happens in your case, is that executing a pipeline is a two-stage process: first construct the pipeline, then run it. > means "open the file, truncate it, and connect it to filedescriptor 1 (stdout)". Only then is the pipe actually run, i.e. sed is executed, but at this time, the file has already been truncated.
Some versions of sed also have a -i parameter for in-place editing of files, that makes sed behave a little more like ed, but using that is not advisable: first of all, it doesn't support all the features of ed, but more importantly, it is a non-standardized proprietary extension of GNU sed that doesn't work on many non-GNU systems. It's been a while since I used a non-GNU system, but last I used one, neither Solaris nor OpenBSD nor HP-UX nor IBM AIX sed supported the -i parameter.
I believe that redirecting output into the same file you are editing is causing your problem.
You need redirect standard output to some temporary file and when sed is done overwrite the original file by the temporary one.