Is there a way to get relative paths to resulted coverage.xml (or strip off prefix) using coverage.py?
I could only manage to sed the console output. The bash code below will expand "../myrootmodule" and remove it from the coverage output. It will also adjust the spaces to preserve the columns alignment. You must adjust the myrootmodule path (first line only).
prefix="$$(readlink -e $$(dirname $$(pwd)))/myrootmodule/"; \
coverage report \
| sed "s|$$prefix||" \
| sed "s/^-\{$${#prefix}\}//" \
| sed "s/^Name \{$${#prefix}\}/Name/" \
| sed "s/^TOTAL \{$${#prefix}\}/TOTAL/"
As it is big, I include it in tests/Makefile.
Related
I wrote some code that generates a github contributions-style heatmap in the terminal given a csv file that contains timestamps and some unsigned value.
I'd like to generate a csv that contains dates and the number of github contributions I made on that date.
Is there a simple way to do this?
You could use git log and a custom format:
git log --date=short --format="%an %ad [%h] %s" | cut -d ' ' -f1 -f2 -f3 -f4- | sed -E 's/ /,/' | sed -E 's/ /,/' | sed -E 's/ /,/'
I get:
Lachlan,Miller,2019-03-25,[e20b847] Rename method
Lachlan,Miller,2019-03-25,[6c47dbf] Add a POC using JS
lmiller1990,2018-04-12,[c295307],Add song class
lmiller1990,2018-04-12,[876cbe2],Add timer
You could use grep for this job. Also, flags like i, A and color will help you cleaning things up a bit. Also, output the result in a .csv file using >
use man grep to know a more about its flags.
Try using:
git log | grep -E -A 2 --color "commit|Date" > output.csv
You could also add --summary flag to log.
I have tried to scan through the other posts in stack overflow for this, but couldn't get my code work, hence I am posting a new question.
Below is the content of file temp.
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/<env:Body><dp:response xmlns:dp="http://www.datapower.com/schemas/management"><dp:timestamp>2015-01-
22T13:38:04Z</dp:timestamp><dp:file name="temporary://test.txt">XJzLXJlc3VsdHMtYWN0aW9uX18i</dp:file><dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:file></dp:response></env:Body></env:Envelope>
This file contains the base64 encoded contents of two files names test.txt and test1.txt. I want to extract the base64 encoded content of each file to seperate files test.txt and text1.txt respectively.
To achieve this, I have to remove the xml tags around the base64 contents. I am trying below commands to achieve this. However, it is not working as expected.
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g' > test.txt
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g' > test1.txt
Below command:
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g'
produces output:
XJzLXJlc3VsdHMtYWN0aW9uX18i
<dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:response> </env:Body></env:Envelope>`
Howeveer, in the output I am expecting only first line XJzLXJlc3VsdHMtYWN0aW9uX18i. Where I am commiting mistake?
When i run below command, I am getting expected output:
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g'
It produces below string
lc3VsdHMtYWN0aW9uX18i
I can then easily route this to test1.txt file.
UPDATE
I have edited the question by updating the source file content. The source file doesn't contain any newline character. The current solution will not work in that case, I have tried it and failed. wc -l temp must output to 1.
OS: solaris 10
Shell: bash
sed -n 's_<dp:file name="\([^"]*\)">\([^<]*\).*_\1 -> \2_p' temp
I add \1 -> to show link from file name to content but for content only, just remove this part
posix version so on GNU sed use --posix
assuming that base64 encoded contents is on the same line as the tag around (and not spread on several lines, that need some modification in this case)
Thanks to JID for full explaination below
How it works
sed -n
The -n means no printing so unless explicitly told to print, then there will be no output from sed
's_
This is to substitute the following regex using _ to separate regex from the replacement.
<dp:file name=
Regular text
"\([^"]*\)"
The brackets are a capture group and must be escaped unless the -r option is used( -r is not available on posix). Everything inside the brackets is captured. [^"]* means 0 or more occurrences of any character that is not a quote. So really this just captures anything between the two quotes.
>\([^<]*\)<
Again uses the capture group this time to capture everything between the > and <
.*
Everything else on the line
_\1 -> \2
This is the replacement, so replace everything in the regex before with the first capture group then a -> and then the second capture group.
_p
Means print the line
Resources
http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
http://www.grymoire.com/Unix/Sed.html
/usr/xpg4/bin/sed works well here.
/usr/bin/sed is not working as expected in case if the file contains just 1 line.
below command works for a file containing only single line.
/usr/xpg4/bin/sed -n 's_<env:Envelope\(.*\)<dp:file name="temporary://BackUpDir/backupmanifest.xml">\([^>]*\)</dp:file>\(.*\)_\2_p' securebackup.xml 2>/dev/null
Without 2>/dev/null this sed command outputs the warning sed: Missing newline at end of file.
This because of the below reason:
Solaris default sed ignores the last line not to break existing scripts because a line was required to be terminated by a new line in the original Unix implementation.
GNU sed has a more relaxed behavior and the POSIX implementation accept the fact but outputs a warning.
I have huge number of html files. I need to replace all the , and " with html entities &nsbquo and &quto respectively.
I need to succeed in two steps for this:
1) Find all the text between tags. I need to replace only in this text between tags.
2) Replace all required strings using sed
My command for this is :
xmlstarlet sel -t -v "*//p" "index.html" | sed 's/,/\&nsbquo/'
This works, but now I dont know how to put back the changes to index.html file.
In sed we have -i option, but for that I need to specify the filename with sed command. But in my case, i have to use | to filter out the required string from html file.
Please help. I did a lot of search for this from 2 days but no luck.
Thank you,
Divya.
The main problem here is that in XML there is no difference between " and ", so you can't use xmlstarlet to do this directly. You could replace " with a special string and then use sed to replace that with ":
xmlstarlet ed -u "//p/text()" \
-x "str:replace(str:replace(., ',', '#NSBQUO#'), '\"', '#QUOT#')" \
quote.html | \
sed 's/#NSBQUO#/\&nsbquo\;/g; s/#QUOT#/\"\;/g' > quote-new.html
mv quote-new.html quote.html
NOTE: str:replace and other exslt functions were only added to xmlstarlet ed in version 1.3.0, so it was not available at the time this question was asked.
Example:
echo one two three | sed 's/ /\n/g' | sed 's/^/:/g'
output:
:one
:two
:three
Without piping:
echo one two three | sed 's/ /\n/g;s/^/:/g'
output:
:one
two
three
seems like first pattern isn't expanded before executing second one, but I really don't know much about sed
How can I use first example without piping twice?
PS Pattern used in examples is informative
The other way to do it is with repeated -e options:
echo one two three | sed -e 's/ /\n:/g' -e 's/^/:/g'
This is easier to understand when you have many operations to do; you can align the separate operations on separate lines:
echo one two three |
sed -e 's/ /\n:/g' \
-e 's/^/:/g'
For example, I have a script to generate outline documents from templates. One part of the script contains:
sed -e "s/[:]YEAR:/$(date +%Y)/g" \
-e "s/[:]TODAY:/$today/" \
-e "s/[:]BASE:/$BASE/g" \
-e "s/[:]base:/$base/g" \
-e "s/[:]FILE:/$FILE/g" \
-e "s/[:]file:/$file/g" \
$skeleton |
...
Although it could be done on one line, it would not promote readability.
The main problem here is that sed decides on what constitutes a line (a pattern that it works on) before executing any commands. That is, if you have only one pattern (one two three), it won't get reinterpreted as multiple lines after execution of s/ /\n/g. If would be still a single pattern, although that would be the one that contains newlines inside it.
The simplest workaround to make sed reinterpret patterns along the newly inserted newlines is just running sed twice, as you did.
Another workaround would be adding something like m option (multi-line buffer) to s command:
$ echo one two three | sed 's/ /\n/g;s/^/:/mg'
:one
:two
:three
You could put all that into one regular expression like this:
echo one two three | sed 's/\([^ ]\+\)\( \+\|$\)/:\1\n/g'
The first part \([^ ]\+\) selects your words (i.e. a string of characters which is not a space. The seconds part \( \+\|$\) matches either one or more spaces or the line end (which is required for the three which has no space after it.
Then we we just build the line by using a back-reference to the word matched in part 1.
This might work for you:
echo one two three | sed 'y/ /\n/;s/^/:/mg'
:one
:two
:three
Can anyone explain how to use sed to delete all characters up to & including the 2nd comma on a line in a CSV file?
The beginning of a typical line might look like
1234567890,ABC/DEF, and the number of digits in the first column varies i.e. there might be 9 or 10 or 11 separate digits in random order, and the letters in the second column could also be random. This randomness and varying length makes it impossible to use any explicit pattern searching.
You could do it with sed like this
sed -e 's/^\([^,]*,\)\{2\}//'
not 100% sure on the syntax, I tried it, and it seems to work though. It'll delete zero-or-more of anything-but-a-comma followed by a comma, and all that is matched twice in succession.
But even easier would be to use cut, like this
cut -d, -f3-
which will use comma as a delimiter, and print fields 3 and up.
EDIT:
Just for the record, both sed and cut can work with a file as a parameter, just append it at the end like so
cut -d, -f3- myfile.txt
or you can pipe the output of your program through them
./myprogram | cut -d, -f3-
sed is not the "right" choice of tool (although it can be done). since you have structured data, you can use fields/delimiter method instead of creating complicated regex.
you can use cut
$ cut -f3- -d"," file
or gawk
$ gawk -F"," '{$1=$2=""}1' file
$ gawk -F"," '{for(i=3;i<NF;i++) printf "%s,",$i; print $NF}' file
Thanks for all replies - with the help provided I have written the simple executable script below which does what I want.
#!/bin/bash
cut -d, -f3- ~/Documents/forex_convert/input.csv |
sed -e '1d' \
-e 's/-/,/g' \
-e 's/ /,/g' \
-e 's/:/,/g' \
-e 's/,D//g' > ~/Documents/forex_convert/converted_input
exit