I have a file in which I have a particular line of this type:
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^ ...
Actually all the others lines are a list (a matrix) of numbers or *******. The problem is that I can not be able to open this file with normal editors and so I can not be able to remove this line.
I can open the file via shell using nano.
To eliminate this line (that is the second line from the top) I used the simple command:
sed '2d' fort.21.dat
But I can not be able to eliminate it.
Can someone help me to eliminate this line and make this file.dat normally readable ?
Thanks a lot
Try:
tr -d '\0' < fort.21.dat > fixed.21.dat
This uses the tr utility to delete the ^# (zero) bytes from the file.
Related
I got about 300 (.txt) files in a folder with "normal text" formatted with /n's. I want all text in these files, to be in the first line, because a program in my pipeline requires that.
Exampel:
(IS:)
GTCGCAGCCG
TCGGCTCGGA
TCTCGGCCTC
(SHOULD BE:)
GTCGCAGCCGTCGGCTCGGATCTCGGCCTC
If I could overwrite them all, with file names staying the same, that would be convenient. I don't want to crack out python yet, is there an easy UNIX tool / command line approach?
I was here before:
How do I remove newlines from a text file?
But how to do that for all 300 files in my folder? If i use tr -d '\n' < *.txt it tells me: "bash: *.txt: ambiguous redirect"
You need a shell loop to process each file in turn. Note that ed is a better choice than tr, as it is designed to work with files, not streams.
for f in *.txt; do
printf '%%j\nwq\n' | ed "$f"
done
%j is the ed command to join all lines in the buffer; the %% is necessary here to make printf output a literal %. wq is the command to write changes back to the file and quite. (The q is optional, as ed will exit after the end of its script anyway.)
I'd like to add a line on top of my output if my input file has a specific word.
However, if I'm just looking for specific string, then as I understand it, it's too late. The first line is already in the output and I can't prepend to it anymore.
Here's an exemple of input.
one
two
two
three
If I can find a line with, say, the word two, I'd like to add a new line before the first one, with for example FOUND. I want that line prepended only once, even if there are several matches.
So an input file without any two would remain unchanged, and the example file above would become:
FOUND
one
two
two
three
I know how to prepend with i\, but can't get the context right. From what I understood that would be around:
1{
/two/{ # This will will search "two" in the first line, how to look for it in the whole file ?
1i\
FOUND
}
}
EDIT:
I know how to do it using other languages/methods, that's not my question.
Sed has advanced features to work on several lines at once, append/prepend lines and is not limited to substitution. I have a sed file already filled with expressions to modify a python source file, which is why I'd prefer to avoid using something else. I want to be able to add an import at the beginning of a file if a certain class is used.
A Perl solution:
perl -i.bak -0077 -pE 'say "FOUND" if /two/;' in_file
The Perl one-liner uses these command line flags:
-p : Loop over the input one line at a time, assigning it to $_ by default. Add print $_ after each loop iteration.
-i.bak : Edit input files in-place (overwrite the input file). Before overwriting, save a backup copy of the original file by appending to its name the extension .bak.
-E : Tells Perl to look for code in-line, instead of in a file. Also enables all optional features. Here, enables say.
-0777 : Slurp files whole.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
sed is for doing s/old/new on individual strings, that's not what you're trying to do so you shouldn't bother trying to use sed. There's lots of ways to do this, this one will be very efficient, robust and portable to all Unix systems:
$ grep -Fq 'two' file && echo "FOUND"; cat file
FOUND
one
two
two
three
To operate on a stream instead of (or in addition to) a file and without needing to read the whole input into memory:
awk 'f{print; next} {buf[NR]=$0} /two/{print "FOUND"; for (i=1;i<=NR;i++) print buf[i]; f=1}'
e.g.:
$ cat file | awk 'f{print; next} {buf[NR]=$0} /two/{print "FOUND"; for (i=1;i<=NR;i++) print buf[i]; f=1}'
FOUND
one
two
two
three
That awk script will also work using any awk in any shell on every Unix box.
name//name/20000_random_test.txt: No space left on device
This is an example of a line I need to modify in a file with all lines with the same pattern as follows:
name//name/<nameoffile>: No space left on device
I tried using:
cat didnotmakeit.txt | sed 's|.*\\\(.*\)|\1|' > filestoupload.txt
and
sed 's|.*\\\(.*\)|\1|' didnotmakeit.txt > filestoupload.txt
But none of these did anything to the file.
A certain amount of files did not transfer to a folder because it ran out of space. I copied and pasted the list in a file.
I need to remove the characters before and after the file name. This way, I will have a list to then use another command something like:
grep NAME_REGEXP /path/to/filenames.list | rsync -a --files-from - . remote.example.com:
Please help me remove these unnecessary characters. Thank you!
The following should work :
sed 's!.*/\([^:].*\):.*!\1!' didnotmakeit.txt
I have been massaging with sed (found tutorial here: Grymoire ) ASCII files we get from our hardware suppliers. Files have a structure like so
Model-Manufacturer:D12-500
Test_Version:2.6.3
But some files we receive are randomly "broken" and miss an entry for "Model-Manufacturer:"
Model-Manufacturer:D12-500
Test_Version:2.6.3
Model-Manufacturer:H24-700
Test_Version:2.6.3
Test_Version:2.6.3
Model-Manufacturer:R15-300
Test_Version:2.6.3
I want to fix this problem with Sed and place the missing entry for "Model-Manufacturer:N/A" before the second occurence of "Test_Version:2.6.3" ; this is my code
sed -n '
/Test_Version/ {
# found "Test_Version" - read in next line
N
# look for "Test_Version" on the second line
# and print if there.
/\n.*Test_Version/ {
# found it - now edit making one line
s/Test_Version/Model-Manufacturer:N/A/
}
}' infile > outfile
It's not working. I believe I need to remember the position of each "Test_Version" and "Model_Manufacturer" before doing the replacement, correct? Can I do this with sed?
Thanks in advance for your input.
Change your substitution to:
s||\nModel-Manufacturer:N/A&|
Using an alternate delimiter means you don't have to escape the slash in "N/A". Using an empty left side reuses the most recent match. The ampersand copies the match into the right side.
Also, you need to remove the -n.
If I understand what you are trying to achieve, you are very close. I think changing the substitution command to the following makes it work:
s/\nTest_Version/\nMode-Manufacturer:N\/A\nTest_Version/
Greetings!
I have been tasked to create a report off files we receive from our hardware suppliers. I need to grep these files for two fields 'Test_Version' and 'Model-Manufacturer' ; for each field, I need to capture their corresponding values.
In a previous post, I found help to create a basic report like so:
find . -name "*.VER" -exec egrep -A 1 'Test_Version=|Model-Manufacturer:' {} ';'
Model-Manufacturer:^M
R22-100^M
Test_Version=2.6.3^M
Model-Manufacturer:^M
R16-300^M
Test_Version=2.6.3^M
However, the data that's output is riddled with DOS carriage returns "^M". My boss wants "Model-Manufacturer" to show like "Test_Version" i.e
Model-Manufacturer:R22-100
Test_Version=2.6.3
Model-Manufacturer:R16-300
Test_Version=2.6.3
Using sed, I attempted to remove the "^M" characters for "Model-Manufacturer" but to no avail:
find . -name "*.VER" -exec egrep -A 1 'Test_Version=|Model-Manufacturer:' {} ';' | sed 's/Model-Manufacturer:^M//g'
This command has not effect. What am I missing here?
Give this a try:
sed '/Model-Manufacturer:/s/\r//g'
If you also have newlines and you want to combine the two lines into one, you can use one of the techniques shown in the answers to your previous question.
you can remove the carriage returns using dos2unix if you have it. Or using tr
tr -d '\r' < file
If you're using Bash as your shell, or creating the script in vi, you should be able to do:
sed -e 's/<Ctrl-V><Ctrl-M>//g'
to remove the CRs.
Ctrl-V (the keystroke on your keyboard) inserts the next keystroke literally, and Ctrl-M is carriage return.