I have the following file:
<CamcorderProfiles cameraId="0">
<EncoderProfile quality="720p" fileFormat="mp4" duration="30">
<Video codec="h264"
bitRate="8000000"
width="1280"
height="720"
frameRate="30" />
<Audio codec="aac"
bitRate="96000"
sampleRate="48000"
channels="1" />
</EncoderProfile>
<EncoderProfile quality="1080p" fileFormat="mp4" duration="30">
<Video codec="h264"
bitRate="12000000"
width="1920"
height="1080"
frameRate="30" />
<Audio codec="aac"
bitRate="96000"
sampleRate="48000"
channels="1" />
</EncoderProfile>
</CamcorderProfiles>
<CamcorderProfiles cameraId="1">
<EncoderProfile quality="720p" fileFormat="mp4" duration="30">
<Video codec="h264"
bitRate="8000000"
width="1280"
height="720"
frameRate="30" />
<Audio codec="aac"
bitRate="96000"
sampleRate="48000"
channels="1" />
</EncoderProfile>
<EncoderProfile quality="1080p" fileFormat="mp4" duration="30">
<Video codec="h264"
bitRate="12000000"
width="1920"
height="1080"
frameRate="30" />
<Audio codec="aac"
bitRate="96000"
sampleRate="48000"
channels="1" />
</EncoderProfile>
</CamcorderProfiles>
Here is my code:
camerastart=$(sed -n '{/<CamcorderProfiles cameraId="0">/=}' $file)
cameraend=$(sed -n $camerastart',$ {/<\/CamcorderProfiles>/=}' $file)
camera1080pstart=$(sed -n $camerastart','$cameraend' {/<EncoderProfile quality="1080p" fileFormat="mp4" duration="30">/=}' $file)
When I print the variables, I get the correct line numbers on all but camera1080p_start, which comes up blank.
camerastart = 3
cameraend = 29
camera1080pstart =
If I change up the first line of the code to cameraId="1", then I do get proper results for that case.
camerastart = 31
cameraend = 57
camera1080pstart = 45
If I change the code so the first two variables are hardcoded in, I do get the proper output of 17 camera1080pstart.
camerastart="3"
cameraend="29"
camera1080pstart=$(sed -n $camerastart','$cameraend' {/<EncoderProfile quality="1080p" fileFormat="mp4" duration="30">/=}' $file)
Anyone know what is going on?
Change the second command to:
cameraend=$(sed -n $camerastart',$ {/<\/CamcorderProfiles>/=}' $file | head -1)
because it returns multiple line numbers. However, this might not be the best way to tackle this particular problem.
Here's a script that will find and print the line you want to change:
$ cat tst.awk
BEGIN{ FS = "\"" }
/<CamcorderProfiles cameraId=/ { cameraId = $2 }
/<EncoderProfile quality=/ { quality = $2 }
/bitRate="12000000"/ {
if ( (cameraId == "0") && (quality == "1080p") ) {
print NR, "this is the line I want to change:"
print
}
}
$ awk -f tst.awk file
17 this is the line I want to change:
bitRate="12000000"
If you need to be able to invoke it with different cameraId and/or quality and/or any other values, that's a trivial tweak.
If you tell us in what way you want to change the line the script finds, that's trivial too.
EDIT: updated script based on OPs comments below.
$ cat tst.awk
BEGIN{ FS = "\"" }
/<CamcorderProfiles cameraId=/ { cameraId = $2 }
/<EncoderProfile quality=/ { quality = $2 }
/bitRate="12000000"/ {
if ( (cameraId == "0") && (quality == "1080p") ) {
sub(/12000000/,"20000000")
}
}
{ print }
$ awk -f tst.awk file > tmpfile
$ diff file tmpfile
17c17
< bitRate="12000000"
---
> bitRate="20000000"
If you want to update the original file, just run it as:
awk -f tst.awk file > tmpfile && mv tmpfile file
The solution that worked for me is:
camerastart=$(sed -n '{/<CamcorderProfiles cameraId="0">/=}' $file | head -1)
cameraend=$(sed -n $camerastart',$ {/<\/CamcorderProfiles>/=}' $file | head -1)
camera1080pstart=$(sed -n $camerastart','$cameraend' {/<EncoderProfile quality="1080p" fileFormat="mp4" duration="30">/=}' $file | head -1)
Related
I've searched through the various questions and tried different permutations of curl commands but yet to find one the works so posting the question as I'm likely missing something obvious and cannot see it for looking.
I'm running a curl command to try and upload a file for parsing.
S-MBP:project SG$ curl -i -X POST -F "data=#/Users/SG/Desktop/project/mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz" 'http://localhost:3030/upload/'
HTTP/1.1 100 Continue
HTTP/1.1 200 OK
Access-Control-Allow-Headers: accept, content-type
Access-Control-Allow-Methods: POST
Access-Control-Allow-Origin: *
Date: Sat, 25 Jul 2020 14:56:05 GMT
Content-Length: 19
Content-Type: text/plain; charset=utf-8
http: no such file
Some of the permutations tried based on previous answers:
curl -i -X POST -F "filename=#/Users/SG/Desktop/project/mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz" http://localhost:3030/upload/
curl -i -X POST -F "filename=#mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz" http://localhost:3030/upload
curl -i -X POST -F filename=#/Users/SG/Desktop/project/mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz http://localhost:3030/upload/
curl -i -X POST -F "filename=#/Users/SG/Desktop/project/mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz" "http://localhost:3030/upload/"
Interestingly, if I pass in the name of a file that doesn't exist I get the same error but if I change the name of the directory to one that doesn't exist the error is a curl(26) making me think that the command couldn't care less about the file at the moment. I am running this on a mac if that's of nay relevance, I saw a post that implied there may be an issue with brew curl.
The form that I'm trying to target is part of the docker image https://hub.docker.com/repository/docker/simagix/maobi
The form with some values omitted
<form action="/upload" id="maobi" class="dropzone dz-clickable dz-started">
<div class="dz-message">
Drop files here or click to upload.<br>
</div>
<div class="dz-filename"><span data-dz-name="">mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz</span></div></div></form>
and then the script I see in the page that I believe is used to upload and then parse the document generating the output.
<script>
Dropzone.options.maobi = {
timeout: 300000,
init: function() {
this.on("success", function(file, responseText) {
blob = new Blob([responseText], { type: 'text/html' }),
anchor = document.createElement('a');
filename = file.upload.filename;
if ((n = filename.indexOf("mdiag-")) == 0 ) {
n = filename.lastIndexOf(".json")
filename = filename.substring(0, n) + ".html";
} else if ((n = filename.lastIndexOf(".json")) > 0 ) {
//...
//...
} else if ((n = filename.indexOf(".log")) > 0 && (n = filename.lastIndexOf(".gz")) > 0) {
filename = filename.substring(0, n) + ".html";
} else if ((n = filename.lastIndexOf(".bson")) > 0 ) {
filename = filename.substring(0, n) + "-cluster.html";
}
anchor.download = filename;
anchor.href = (window.webkitURL || window.URL).createObjectURL(blob);
anchor.dataset.downloadurl = ['text/html', anchor.download, anchor.href].join(':');
anchor.click();
});
this.on("error", function(file, responseText) {
alert(responseText);
});
}
};
</script>
It seems to me that you are not passing the file in the right form field.
From looking at the Dropzone.js documentation, it seems the right field name is file (since that's the default for the paramName configuration), not data or filename. But to be entirely sure, it'd be best to look at the network request in your browser's devtools and see what POST field name is used there for passing the file.
curl -i -X POST -F "file=#/Users/SG/Desktop/project/mongodb-cluster-shard-00-02.lrqcr.mongodb.net_27017-cluster.bson.gz" 'http://localhost:3030/upload/'
There is a similar question on SO, however, the provided answer is all done in one line. For readability purposes I would like the solution on multiple lines:
This is what I tried although as append:
sed -i '/home_server localhost {/a\
home_server example-coa {\
type = coa\
ipaddr = 127.0.0.1\
port = 3799\
secret = '"${SECRET}"'\
coa {\
irt = 2\
mrt = 16\
mrc = 5\
mrd = 30\
}\
}\
home_server localhost {\
' /etc/freeradius/3.0/proxy.conf
This is an example of APPEND, it works, but I need to do a replace.
I would like to replace home_server localhost { with the above.
I'm literally trying to add a block above it so that it will look like this at the end:
home_server example-coa {
...
}
home_server localhost {
...
}
Why don't you just insert? E.g
sed -i '/home_server localhost {/i\
home_server example-coa {\
type = coa\
ipaddr = 127.0.0.1\
port = 3799\
secret = '"${SECRET}"'\
coa {\
irt = 2\
mrt = 16\
mrc = 5\
mrd = 30\
}\
}\
' /etc/freeradius/3.0/proxy.conf
This might work for you (GNU sed and bash):
cat <<! | sed '/home_server localhost {/e cat /dev/stdin' file
home_server example-coa {
type = coa
ipaddr = 127.0.0.1
port = 3799
secret = '"${SECRET}"'
coa {
irt = 2
mrt = 16
mrc = 5
mrd = 30
}
}
!
Use the e command to place a here-document in the output stream of a sed command.
I want to extract comment from SCSS file put them on a new markdown file and remove the comments from original file.
file.scss
/***
* # Typography
* We use
* [Roboto](https://fonts.google.com/specimen/Roboto)
*/
/**
* Body typography:
* `p`
*
* #example
* p This is a paragraph with #[strong bold ] and#[em italic ] styles.</p>
*/
body {
color: $base-font-color;
}
/**
* Heading 1:
* `h1` `.h1`
*/
h1,
.h1 {
#include h1;
}
Expected result
At the end of the process I would like to have:
Comments.md
# Typography
* We use
[Roboto](https://fonts.google.com/specimen/Roboto)
Body typography:
`p`
#example
p This is a paragraph with #[strong bold ] and#[em italic ] styles.</p>
Heading 1:
`h1` `.h1`
file.scss
body {
color: $base-font-color;
}
h1,
.h1 {
#include h1;
}
Question
How can I do that with awk, perl, sed or other tools?
I implement a solution myself
#!/usr/bin/env bash
# USAGE
# ## Development
#
# # single file
# bash ./scripts/extract-md.bash src/base/_typography.scss
#
# # all files
# bash ./scripts/extract-md.bash src/base/_typography.scss
#
# ## production (override files)
#
# export PRODUCTION=true
# bash ./scripts/extract-md.bash
shopt -s globstar
start() {
for legacy in $legacies; do
directory="${legacy%/*}"
filename="${legacy##*/}"
new_filename="${filename//.scss/.md}"
documentation="${directory}/${new_filename}"
extract_comment "$legacy" "$documentation"
stylesheet="${1//.scss/.new.scss}"
extract_code "$legacy" "$stylesheet"
[[ $PRODUCTION ]] && mv "$stylesheet" "$legacy"
done
}
extract_comment() {
local legacy="$1"
local documentation="$2"
awk '/^\/\*/{flag=1} flag; /^\s?\*\//{print;flag=0}' "$legacy" \
| perl -p -e 's/\/[\*]+//g' \
| perl -p -e 's/^\s*[\*]?[ \/]?//g' \
> "$documentation"
}
extract_code() {
local legacy="$1"
local stylesheet="$2"
awk 'BEGIN{flag=1}/^\/\*/{flag=0}; /^\s?\*\//{flag=1; next}flag' "$legacy" \
> "$stylesheet"
}
legacies=${1:-src/**/*.scss}
start $legacies
Are there minimal, or even larger, working examples of using SCons and knitr to generate reports from .Rmd files?
kniting an cleaning_session.Rmd file from the command line (bash shell) to derive an .html file, may be done via:
Rscript -e "library(knitr); knit('cleaning_session.Rmd')".
In this example, Rscript and instructions are fed to a Makefile:
RMDFILE=test
html :
Rscript -e "require(knitr); require(markdown); knit('$(RMDFILE).rmd', '$(RMDFILE).md'); markdownToHTML('$(RMDFILE).md', '$(RMDFILE).html', options=c('use_xhtml', 'base64_images')); browseURL(paste('file://', file.path(getwd(),'$(RMDFILE).html'), sep=''
In this answer https://stackoverflow.com/a/10945832/1172302, there is reportedly a solution using SCons. Yet, I did not test enough to make it work for me. Essentially, it would be awesome to have something like the example presented at https://tex.stackexchange.com/a/26573/8272.
[Updated] One working example is an Sconstruct file:
import os
environment = Environment(ENV=os.environ)
# define a `knitr` builder
builder = Builder(action = '/usr/local/bin/knit $SOURCE -o $TARGET',
src_suffix='Rmd')
# add builders as "Knit", "RMD"
environment.Append( BUILDERS = {'Knit' : builder} )
# define an `rmarkdown::render()` builder
builder = Builder(action = '/usr/bin/Rscript -e "rmarkdown::render(input=\'$SOURCE\', output_file=\'$TARGET\')"',
src_suffix='Rmd')
environment.Append( BUILDERS = {'RMD' : builder} )
# define source (and target files -- currently useless, since not defined above!)
# main cleaning session code
environment.RMD(source='cleaning_session.Rmd', target='cleaning_session.html')
# documentation of the Cleaning Process
environment.Knit(source='Cleaning_Process.Rmd', target='Cleaning_Process.html')
# documentation of data
environment.Knit(source='Code_Book.Rmd', target='Code_Book.html')
The first builder calls the custom script called knit. Which, in turn, takes care of the target file/extension, here being cleaning_session.html. Likely the suffix parameter is not needed altogether, in this very example.
The second builder added is Rscript -e "rmarkdown::render(\'$SOURCE\')"'.
The existence of $TARGETs (as in the example at Command wrapper) ensures SCons won't repeat work if a target file already exists.
The custom script (whose source I can't retrieve currently) is:
#!/usr/bin/env Rscript
local({
p = commandArgs(TRUE)
if (length(p) == 0L || any(c('-h', '--help') %in% p)) {
message('usage: knit input [input2 input3] [-n] [-o output output2 output3]
-h, --help to print help messages
-n, --no-convert do not convert tex to pdf, markdown to html, etc
-o output filename(s) for knit()')
q('no')
}
library(knitr)
o = match('-o', p)
if (is.na(o)) output = NA else {
output = tail(p, length(p) - o)
p = head(p, o - 1L)
}
nc = c('-n', '--no-convert')
knit_fun = if (any(nc %in% p)) {
p = setdiff(p, nc)
knit
} else {
if (length(p) == 0L) stop('no input file provided')
if (grepl('\\.(R|S)(nw|tex)$', p[1])) {
function(x, ...) knit2pdf(x, ..., clean = TRUE)
} else {
if (grepl('\\.R(md|markdown)$', p[1])) knit2html else knit
}
}
mapply(knit_fun, p, output = output, MoreArgs = list(envir = globalenv()))
})
The only thing, now, necessary is to run scons.
I'm trying to comment this lines:
passdb {
driver = pam
[session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=<n>]
[cache_key=<key>] [<service name>]
args = dovecot
}
via sed:
sed -i '1!N; s/passdb {\
driver = pam\
\[session=yes\] \[setcred=yes\] \[failure_show_msg=yes\] \[max_requests=\<n\>\]\
\[cache_key=\<key\>\] \[\<service name\>\]\
args = dovecot\
}/#passdb {\
# driver = pam\
# [session=yes] [setcred=yes] [failure_show_msg=yes] [max_requests=<n>]\
# [cache_key=<key>] [<service name>]\
# args = dovecot\
#}/' t
But it doesn't match what I need, can anyone tell me what I'm doing wrong here?
If all you are trying to do is comment the lines between passdb and }, then the following should suffice
sed -i '/^passdb {/,/}/s/^/#/g' file
Using awk
awk '/^passdb {/,/^}/ {$0="#"$0}1' file