Scala command line parser using Scallop - scala

I'm fairly new to Scala and need to build a really simple command line parser which provides something like the following which I created using JRuby in a few minutes:-
java -jar demo.jar --help
Command Line Example Application
Example: java -jar demo.jar --dn "CN=Test" --nde-url "http://www.example.com" --password "password"
For usage see below:
-n http://www.example.com
-p, --password set the password
-c, --capi set add to Windows key-store
-h, --help Show this message
-v, --version Print version
Scallop looks like it will do the trick, but I can't seem to find a simple example that works! All of the examples I've found seem to be fragmented and don't work for some reason or other.
UPDATE
I found this example which works, but I'm not sure how to bind it into the actual args within the main method.
import org.rogach.scallop._;
object cmdlinetest {
def main(args: Array[String])
val opts = Scallop(List("-d","--num-limbs","1"))
.version("test 1.2.3 (c) 2012 Mr Placeholder")
.banner("""Usage: test [OPTION]... [pet-name]
|test is an awesome program, which does something funny
|Options:
|""".stripMargin)
.footer("\nFor all other tricks, consult the documentation!")
.opt[Boolean]("donkey", descr = "use donkey mode")
.opt("monkeys", default = Some(2), short = 'm')
.opt[Int]("num-limbs", 'k',
"number of libms", required = true)
.opt[List[Double]]("params")
.opt[String]("debug", hidden = true)
.props[String]('D',"some key-value pairs")
// you can add parameters a bit later
.args(List("-Dalpha=1","-D","betta=2","gamma=3", "Pigeon"))
.trailArg[String]("pet name")
.verify
println(opts.help)
}
}

Well, I'll try to add more examples :)
In this case, it would be much better to use ScallopConf:
import org.rogach.scallop._
object Main extends App {
val opts = new ScallopConf(args) {
banner("""
NDE/SCEP Certificate enrollment prototype
Example: java -jar demo.jar --dn CN=Test --nde-url http://www.example.com --password password
For usage see below:
""")
val ndeUrl = opt[String]("nde-url")
val password = opt[String]("password", descr = "set the password")
val capi = toggle("capi", prefix = "no-", descrYes = "enable adding to Windows key-store", descrNo = "disable adding to Windows key-store")
val version = opt[Boolean]("version", noshort = true, descr = "Print version")
val help = opt[Boolean]("help", noshort = true, descr = "Show this message")
}
println(opts.password())
}
It prints:
$ java -jar demo.jar --help
NDE/SCEP Certificate enrollment prototype
Example: java -jar demo.jar --dn CN=Test --nde-url http://www.example.com --password password
For usage see below:
-c, --capi enable adding to Windows key-store
--no-capi disable adding to Windows key-store
--help Show this message
-n, --nde-url <arg>
-p, --password <arg> set the password
--version Print version

Did you read the documentation? It looks like all you have to do is call get for each option you want:
def get [A] (name: String)(implicit m: Manifest[A]): Option[A]
It looks like you might need to provide the expected return type in the method call. Try something like this:
val donkey = opts.get[Boolean]("donkey")
val numLimbs = opts.get[Int]("num-limbs")

If you're just looking for a quick and dirty way to parse command line arguments, you can use pirate, an extremely barebones way to parse arguments. Here is what it would look like to handle the usage you describe above:
import com.mosesn.pirate.Pirate
object Main {
def main(commandLineArgs: Array[String]) {
val args = Pirate("[ -n string ] [ -p string ] [ -chv ]")("-n whatever -c".split(" "))
val c = args.flags.contains('c')
val v = args.flags.contains('v')
val h = args.flags.contains('h')
val n = args.strings.get("n")
val p = args.strings.get("p")
println(Seq(c, v, h, n, p))
}
}
Of course, for your program, you would pass commandLineArgs instead of "-n whatever -c".
Unfortunately, pirate does not yet support GNU style arguments, nor the version or help text options.

Related

locust unrecognized arguments when running as lib

The following codes are from Locust examples - use_as_lib.
import gevent
from locust import HttpUser, task
from locust.env import Environment
from locust.stats import stats_printer, stats_history
from locust.log import setup_logging
setup_logging("INFO", None)
class MyUser(HttpUser):
host = "https://docs.locust.io"
#task
def t(self):
self.client.get("/")
env = Environment(user_classes=[MyUser])
runner = env.create_local_runner()
web_ui = env.create_web_ui("127.0.0.1", 8089)
env.events.init.fire(environment=env, runner=runner, web_ui=web_ui)
gevent.spawn(stats_printer(env.stats))
gevent.spawn(stats_history, env.runner)
runner.start(1, spawn_rate=10)
gevent.spawn_later(60, lambda: runner.quit())
runner.greenlet.join()
web_ui.stop()
If I run it with python use_as_lib.py, everything works fine. But if I run it with python use_as_lib.py -c argument01 -b argument02, it will fail with:
use_as_lib.py: error: unrecognized arguments: -c -b argument02
In my case, the snippet above is part of a big program, which has its own command line arguments.
I checked a bit, seems argument_parser.ui_extra_args_dict() here invoked by env.create_web_ui("127.0.0.1", 8089) will parse all the arguments, which cause this issue.
Any ideas on how to fix it ? Thanks!
You can pass a parsed set of parameters when you create the environment, slightly less hack-y than your suggestion. Something like this:
parser = locust.argument_parser.get_parser()
parsed_options = parser.parse_args("-f yourlocustfile.py --headless <other params>")
env = Environment(user_classes=[MyUser], parsed_options=parsed_options=parsed_options)
Would that work?
Here's the workaround I used. So far it works. But more like a hack.
# ...
tmp = sys.argv
sys.argv = [sys.argv[0]]
env.create_web_ui("127.0.0.1", 8089)
sys.argv = tmp
# ...

object not found when creating targets list programmatically

I'm trying to generate a {targets} list programmatically, via a function in an R package.
get_pipeline <- function(which_countries) {
countries <- NULL # avoid R CMD CHECK warning
print(which_countries) # Shows that which_countries is available
list(
targets::tar_target(
name = countries,
command = which_countries # But here, which_countries is not found
)
)
}
The _targets.R file looks like this:
library(targets)
couns <- c("USA", "GBR")
TargetsQuestions::get_pipeline(couns)
I see the following error:
> tar_make()
[1] "USA" "GBR"
Error in enexpr(expr) : object 'which_countries' not found
Error in `tar_throw_run()`:
! callr subprocess failed: object 'which_countries' not found
Note that the which_countries variable is printable, but not found in the call to tar_target.
How can I get create the countries target successfully so that it contains the vector c("USA", "GBR")?
This code is in a GitHub repository at https://github.com/MatthewHeun/TargetsQuestions. To reproduce:
git clone https://github.com/MatthewHeun/TargetsQuestions
Build the package in RStudio.
targets::tar_make() at the Console.
Thanks in advance for any suggestions!
Thanks to #landau for pointing to https://wlandau.github.io/targetopia/contributing.html#target-factories which in turn points to the metaprogramming section of Advanced R at https://adv-r.hadley.nz/metaprogramming.html.
The solution turned out to be:
get_pipeline <- function(which_countries) {
list(
targets::tar_target_raw(
name = "countries",
# command = which_countries # which_countries must have length 1
# command = !!which_countries # invalid argument type
command = rlang::enexpr(which_countries) # Works
)
)
}
With _targets.R like this:
library(targets)
couns <- c("USA", "GBR")
TargetsQuestions::get_pipeline(couns)
the commands tar_make() and tar_read(countries), give
[1] "USA" "GBR"
as expected!

Is there a way to convert juniper "json" or "xml" config to "set" or "show" config?

We use juniper hardware with junos version 15. In this version we can export our config as "json" or "xml" which we want to use to edit it with our automation tooling.
Importing however is only possible in "set" or "show" format.
Is there a tool to convert "json" or "xml" format to "set" or "show" format?
I can only find converters between "show" and "set".
We can't upgrade to version 16 where the import of "json" would be possible.
Here's a script I made at work, throw it in your bin and you can it via providing a filename or piping output. This assumes linux or mac so the os.isatty function works, but the logic can work anywhere:
usage demo:
person#laptop ~ > head router.cfg
## Last commit: 2021-04-20 21:21:39 UTC by vit
version 15.1X12.2;
groups {
BACKBONE-PORT {
interfaces {
<*> {
mtu 9216;
unit <*> {
family inet {
mtu 9150;
person#laptop ~ > convert.py router.cfg | head
set groups BACKBONE-PORT interfaces <*> mtu 9216
set groups BACKBONE-PORT interfaces <*> unit <*> family inet mtu 9150
set groups BACKBONE-PORT interfaces <*> unit <*> family inet6 mtu 9150
set groups BACKBONE-PORT interfaces <*> unit <*> family mpls maximum-labels 5
<... output removed... >
convert.py:
#!/usr/bin/env python3
# Class that attempts to parse out Juniper JSON into set format
# I think it works? still testing
#
# TODO:
# accumulate annotations and provide them as commands at the end. Will be weird as annotations have to be done after an edit command
from argparse import ArgumentParser, RawTextHelpFormatter
import sys, os, re
class TokenStack():
def __init__(self):
self._tokens = []
def push(self, token):
self._tokens.append(token)
def pop(self):
if not self._tokens:
return None
item = self._tokens[-1]
self._tokens = self._tokens[:-1]
return item
def peek(self):
if not self._tokens:
return None
return self._tokens[-1]
def __str__(self):
return " ".join(self._tokens)
def __repr__(self):
return " ".join(self._tokens)
def main():
# get file
a = ArgumentParser(prog="convert_jpr_json",
description="This program takes in Juniper style JSON (blah { format) and prints it in a copy pastable display set format",
epilog=f"Either supply with a filename or pipe config contents into this program and it'll print out the display set view.\nEx:\n{B}convert_jpr_json <FILENAME>\ncat <FILENAME> | convert_jpr_json{WHITE}",
formatter_class=RawTextHelpFormatter)
a.add_argument('file', help="juniper config in JSON format", nargs="?")
args = a.parse_args()
if not args.file and os.isatty(0):
a.print_help()
die("Please supply filename or provide piped input")
file_contents = None
if args.file:
try:
file_contents = open(args.file, "r").readlines()
except IOError as e:
die(f"Issue opening file {args.file}: {e}")
print(output_text)
else:
file_contents = sys.stdin.readlines()
tokens = TokenStack()
in_comment = False
new_config = []
for line_num, line in enumerate(file_contents):
if line.startswith("version ") or len(line) == 0:
continue
token = re.sub(r"^(.+?)#+[^\"]*$", r"\1", line.strip())
token = token.strip()
if (any(token.startswith(_) for _ in ["!", "#"])):
# annotations currently not supported
continue
if token.startswith("/*"):
# we're in a comment now until the next token (this will break if a multiline comment with # style { happens, but hopefully no-one is that dumb
in_comment = True
continue
if "inactive: " in token:
token = token.split("inactive: ")[1]
new_config.append(f"deactivate {tokens} {token}")
if token[-1] == "{":
in_comment = False
tokens.push(token.strip("{ "))
elif token[-1] == "}":
if not tokens.pop():
die("Invalid json supplied: unmatched closing } encountered on line " + f"{line_num}")
elif token[-1] == ";":
new_config.append(f"set {tokens} {token[:-1]}")
if tokens.peek():
print(tokens)
die("Unbalanced JSON: expected closing }, but encountered EOF")
print("\n".join(new_config))
def die(msg): print(f"\n{B}{RED}FATAL ERROR{WHITE}: {msg}"); exit(1)
RED = "\033[31m"; GREEN = "\033[32m"; YELLOW = "\033[33m"; B = "\033[1m"; WHITE = "\033[0m"
if __name__ == "__main__": main()
You can load XML configuration using edit-config RPC or load-configuration RPC. For more details:
https://www.juniper.net/documentation/en_US/junos/topics/reference/tag-summary/netconf-edit-config.html
https://www.juniper.net/documentation/en_US/junos/topics/reference/tag-summary/junos-xml-protocol-load-configuration.html
XML content can be loaded via an "op" script by placing the content inside a call to junos:load-configuration() template defined in "junos.xsl". Something like the following:
version 1.1;
ns jcs = "http://xml.juniper.net/junos/commit-scripts/1.0";
import "../import/junos.xsl";
var $arguments = {
<argument> {
<name> "file";
<description> "Filename of XML content to load";
}
<argument> {
<name> "action";
<description> "Mode for the load (override, replace, merge)";
}
}
param $file;
param $action = "replace";
match / {
<op-script-results> {
var $configuration = slax:document($file);
var $connection = jcs:open();
call jcs:load-configuration($connection, $configuration, $action);
}
}
Thanks,
Phil

Examples of using SCons with knitr

Are there minimal, or even larger, working examples of using SCons and knitr to generate reports from .Rmd files?
kniting an cleaning_session.Rmd file from the command line (bash shell) to derive an .html file, may be done via:
Rscript -e "library(knitr); knit('cleaning_session.Rmd')".
In this example, Rscript and instructions are fed to a Makefile:
RMDFILE=test
html :
Rscript -e "require(knitr); require(markdown); knit('$(RMDFILE).rmd', '$(RMDFILE).md'); markdownToHTML('$(RMDFILE).md', '$(RMDFILE).html', options=c('use_xhtml', 'base64_images')); browseURL(paste('file://', file.path(getwd(),'$(RMDFILE).html'), sep=''
In this answer https://stackoverflow.com/a/10945832/1172302, there is reportedly a solution using SCons. Yet, I did not test enough to make it work for me. Essentially, it would be awesome to have something like the example presented at https://tex.stackexchange.com/a/26573/8272.
[Updated] One working example is an Sconstruct file:
import os
environment = Environment(ENV=os.environ)
# define a `knitr` builder
builder = Builder(action = '/usr/local/bin/knit $SOURCE -o $TARGET',
src_suffix='Rmd')
# add builders as "Knit", "RMD"
environment.Append( BUILDERS = {'Knit' : builder} )
# define an `rmarkdown::render()` builder
builder = Builder(action = '/usr/bin/Rscript -e "rmarkdown::render(input=\'$SOURCE\', output_file=\'$TARGET\')"',
src_suffix='Rmd')
environment.Append( BUILDERS = {'RMD' : builder} )
# define source (and target files -- currently useless, since not defined above!)
# main cleaning session code
environment.RMD(source='cleaning_session.Rmd', target='cleaning_session.html')
# documentation of the Cleaning Process
environment.Knit(source='Cleaning_Process.Rmd', target='Cleaning_Process.html')
# documentation of data
environment.Knit(source='Code_Book.Rmd', target='Code_Book.html')
The first builder calls the custom script called knit. Which, in turn, takes care of the target file/extension, here being cleaning_session.html. Likely the suffix parameter is not needed altogether, in this very example.
The second builder added is Rscript -e "rmarkdown::render(\'$SOURCE\')"'.
The existence of $TARGETs (as in the example at Command wrapper) ensures SCons won't repeat work if a target file already exists.
The custom script (whose source I can't retrieve currently) is:
#!/usr/bin/env Rscript
local({
p = commandArgs(TRUE)
if (length(p) == 0L || any(c('-h', '--help') %in% p)) {
message('usage: knit input [input2 input3] [-n] [-o output output2 output3]
-h, --help to print help messages
-n, --no-convert do not convert tex to pdf, markdown to html, etc
-o output filename(s) for knit()')
q('no')
}
library(knitr)
o = match('-o', p)
if (is.na(o)) output = NA else {
output = tail(p, length(p) - o)
p = head(p, o - 1L)
}
nc = c('-n', '--no-convert')
knit_fun = if (any(nc %in% p)) {
p = setdiff(p, nc)
knit
} else {
if (length(p) == 0L) stop('no input file provided')
if (grepl('\\.(R|S)(nw|tex)$', p[1])) {
function(x, ...) knit2pdf(x, ..., clean = TRUE)
} else {
if (grepl('\\.R(md|markdown)$', p[1])) knit2html else knit
}
}
mapply(knit_fun, p, output = output, MoreArgs = list(envir = globalenv()))
})
The only thing, now, necessary is to run scons.

Unable to upload file via REST request in GRAILS

I am trying to upload a file via REST using GRAILS
curl -X POST -H "Cache-Control: no-cache" -H "Postman-Token: d5d7aef8-3964-311b-8b64-4a7a82c52323" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW" -F "file1=myfile.jpg" -F "fname=swateek" -F "lname=jena" 'http://localhost:8081/sampleFileREST/document/upload'
Here's how my controller looks like:
class DocumentController extends RestfulController<Document> {
static responseFormats = ['json', 'xml']
def upload() {
def fileLocation="<xml>empty</xml>"
def file = request.getParameter('file1')
def f1 = request.getParameter('fname')
def f2 = "<abc>"+request.getParameter('lname')+"</abc>"
def params = "gf"
if(file.empty) {
fileLocation = "<xml>"+"File cannot be empty"+"</xml><allprm>"+params+"</allprm>"
} else {
def documentInstance = new Document()
documentInstance.filename = file.originalFilename
documentInstance.fullPath = grailsApplication.config.uploadFolder + documentInstance.filename
file.transferTo(new File(documentInstance.fullPath))
documentInstance.save()
fileLocation = "<xml>"+documentInstance.fullPath+"</xml>"
}
/* return "File uploaded to: "+documentInstance.fullPath */
render(text: fileLocation, contentType: "text/xml", encoding: "UTF-8")
}
}
I am able to access the parameters of the request, anything except the file I am sending in the request.
Unable to figure out what's wrong here.
UPDATE
I had used .getParameter() to fetch a file. That's incorrect, the correct way is as below:
request.getFile('<filename>') // without the <>
This might raise an error in IntelliJ as "Symbol Not Found" or "Cannot Resolve Method", please follow the procedure in the answer below.
Damn the IDE that I was using, IntelliJ.
Also, this piece of code while getting the file:
def file = request.getParameter('file1')
should be replaced as
def file = request.getFile('file1')
Now previously, when I was using the request.getFile() method I was getting an "Symbol Not Found" error and it was failing to execute the request.
Solution:
Open IntelliJ
Click on "File"
Find the option "Invalidate Caches/Restart" and wait for IntelliJ to come back again.
If this doesn't work, the other way is mentioned in this answer:
IntelliJ IDEA JDK configuration on Mac OS