Parsing a simple libconfuse file with Augeas - configuration-files

I'm using libconfuse for my program's configuration files, and that's working nicely. Now I'm interested to parse the configuration files using Augeas. I found a mailing list post which says that there's no generic Augeas lens for libconfuse files, because it's a "context-free file format" (in essence, it allows infinite nesting).
My program's configuration files are quite simple, with just one level of sections containing configuration parameters. E.g.:
serial {
serial-device = "/dev/ttyUSB0"
baudrate = 115200
}
server-socket {
host = "localhost"
port = 12345
}
What would be involved in writing a generic Augeas lens for this simple variety of libconfuse configuration file? Are there any examples around? What would be the most straight-forward way to handle this?

The post you're referring to is from 2008. Augeas has since been able to parse recursive configuration files, using the rec keyword. See for example lvm.aug, which is quite similar to what you're trying to achieve.

Thanks to ℝaphink's answer, I started with the lvm.aug lens, which was a good starting point, and improved it a little.
Here's what I've got at the moment, which supports only a subset of libconfuse syntax—if you test it with the libconfuse example test.conf, it will fail in several places. But it works for the sub-set of the syntax in the config files I'm currently using, so it's "good enough" for my current purposes. Although I'd like to figure out how to get indentation of nested blocks looking nice (like the json.aug lens does; I haven't figured out how it does it yet).
(*
Module: LibconfuseSimple
Based on Module LVM
*)
module LibconfuseSimple =
(* See lvm2/libdm/libdm-config.c for tokenisation;
* libdm uses a blacklist but I prefer the safer whitelist approach. *)
(* View: identifier
* The left hand side of a definition *)
let identifier = /[a-zA-Z0-9_-]+/
(* strings can contain backslash-escaped dquotes, but I don't know
* how to get the message across to augeas *)
let str = [label "str". Quote.do_quote (store /[^"]*/)]
let int = [label "int". store Rx.integer]
let env = [label "env". del "${" "${" . store /[^}]*/ . del "}" "}"]
let const (r:regexp) = [ label "const" . store r ]
let rawstr = [label "rawstr". store Rx.space_in]
(* View: flat_literal
* A literal without structure *)
let flat_literal = int|str|env|const /true|false|null/|rawstr
(* allow multiline and mixed int/str, used for raids and stripes *)
(* View: list
* A list containing flat literals *)
let list = [
label "list" . counter "list"
. del /\[[ \t\n]*/ "["
.([seq "list". flat_literal . del /,[ \t\n]*/ ", "]*
. [seq "list". flat_literal . del /[ \t\n]*/ ""])?
. Util.del_str "]"]
(* View: val
* Any value that appears on the right hand side of an assignment *)
let val = flat_literal | list
(* View: comments
* Comments of various sorts *)
let comments =
Util.comment
| Util.comment_c_style
| Util.comment_multiline
(* View: nondef
* A line that doesn't contain a statement *)
let nondef =
Util.empty
| comments
(* View: indent
* Remove any input indentation; output 4 spaces indentation. *)
let indent = del /[ \t]*/ " "
(* Build.block couldn't be reused, because of recursion and
* a different philosophy of whitespace handling. *)
(* View: def
* An assignment, or a block containing definitions *)
let rec def = [
key identifier . (
(del /[ \t]*/ " " . [label "title" . store identifier])? . del /[ \t]*\{\n?/ " {\n"
.[label "dict" . (Util.empty | indent . comments | indent . def)*]
. Util.indent . Util.del_str "}\n"
|Sep.space_equal . val . Util.comment_or_eol)]
(* View: lns
* The main lens *)
let lns = (nondef | (Util.indent . def))*

Related

Strip margin of indented triple-quote string in Purescript?

When using triple quotes in an indented position I for sure get indentation in the output js string too:
Comparing these two in a nested let
let input1 = "T1\nX55.555Y-44.444\nX52.324Y-40.386"
let input2 = """T1
X66.324Y-40.386
X52.324Y-40.386"""
giving
// single quotes with \n
"T1\x0aX55.555Y-44.444\x0aX52.324Y-40.386"
// triple quoted
"T1\x0a X66.324Y-40.386\x0a X52.324Y-40.386"
Is there any agreed upon thing like stripMargin in Scala so I can use those without having to unindent to top level?
Update, just to clarify what I mean, I'm currently doing:
describe "header" do
it "should parse example header" do
let input = """M48
;DRILL file {KiCad 4.0.7} date Wednesday, 31 January 2018 'AMt' 11:08:53
;FORMAT={-:-/ absolute / metric / decimal}
FMAT,2
METRIC,TZ
T1C0.300
T2C0.400
T3C0.600
T4C0.800
T5C1.000
T6C1.016
T7C3.400
%
"""
doesParse input header
describe "hole" do
it "should parse a simple hole" do
doesParse "X52.324Y-40.386" hole
Update:
I was asked to clarify stripMargin from Scala. It's used like so:
val speech = """T1
|X66.324Y-40.386
|X52.324Y-40.386""".stripMargin
which then removes the leading whitespace. stripMargin can take any separator, but defaults to |.
More examples:
Rust has https://docs.rs/trim-margin/0.1.0/trim_margin/
Kotlin has in stdlib: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.text/trim-margin.html
I guess it might sound like asking for left-pad ( :) ) but if there's something there already I'd rather not brew it myself…
I'm sorry you didn't get a prompt response to this one, but I have implemented this function here. In case the pull request isn't merged, here's an implementation that just depends on purescript-strings:
import Data.String (joinWith, split) as String
import Data.String.CodeUnits (drop, dropWhile) as String
import Data.String.Pattern (Pattern(..))
stripMargin :: String -> String
stripMargin =
let
lines = String.split (Pattern "\n")
unlines = String.joinWith "\n"
mapLines f = unlines <<< map f <<< lines
in
mapLines (String.drop 1 <<< String.dropWhile (_ /= '|'))

swapping variable name and variable type

I have a text file with a long list of variables like this:
a VARCHAR(32),
b INT,
c TINYINT,
.
.
.
I want to quickly swap the order of the name and type so I have:
VARCHAR(32) a,
INT b,
TINYINT c
.
.
.
Im happy to use a bash terminal or notepad ++ and I do have a basic knowledge of regex but Im not sure how to tackle this problem.
How can I go about doing this?
you can use this app I just wrote:
http://codepen.io/franzskuffka/pen/Ndxejz
Or run it through this function
function swap (text) {
let lines = text.split(',\n')
let parts = lines.map(function (line) {
var lineParts = line.split(' ')
lineParts[2] = lineParts[0]
delete lineParts[0]
return lineParts.join(' ')
})
return parts.join(',\n')
}
Generally I recommend the text editor Kakoune for this task which is awesome for text processing in general due to the multicursor support and incremental editing.

How to translate words literally?

I want to translate the names of the businesses literally.
i.e :
FLAFEL ADAM I want to translate it to Hebrew,
F -> פ
L -> ל
A -> א
.
.
.
so it will be פלאפל אדם
both names have the same spelling ! I don't care a bout the meaning , I just want to have the same spelling in both languages !
How to do that ?
You could use table lookup, i.e. a table that translates every character from one language to another. In Obj-C, you could use a NSDictionary that you could initialize as
NSDictionary *table = #{#"F": #"פ",
#"L": #"ל",
#"A": #"א"
};
etc.
You can then do the translation e.g. as
NSString *translatedL = table[#"L"];
etc.

How to parse variable in REXX/Regina

When writing a macro, I have a variable, {$#2}, that either starts with a 1 or 2 digit number followed by a ".", or it doesn't. When it starts in that way, I want to put the number into a rexxvar, which I have called C.
I have tried
#if [ DATATYPE(LEFT({$#2},1), "W") ] \
#evaluate ^^ ^parse '{$#2} C '.' .^ \
#endif \
This, and every variation I can think of, gives errors saying the #IF line contains invalid characters.
How should I do this?
I am using PPWizard, and Regina - but I can't create either tag.
Thanks
Ian
I do not use PPWizard so this could all be wrong, but
Looking at PPWizard #if, the if should be either
#if DATATYPE(LEFT({$#2},1), "W")
or
#if [ DATATYPE(LEFT({$#2},1), "W") <> 0 ]
But I do not know wether you can imbed the {$#2} or not (I do not know PPWizard)
For the parse statement one of these may be what you want
#evaluate ^^ ^parse value '{$#2}' with C '.' .^
or
#evaluate ^^ ^parse var {$#2} C '.' .^
See rexx parse syntax
An alternative way might try the Define Rexx tag and do it in rexx. More people could help you with pure rexx. i.e the rexx would be
if DATATYPE(LEFT(value_to_process,1), "W") then do
parse var value_to_process C '.'
end
where value_to_process is the value to be checked (i.e. {$#2})
For others.
The final answer to the problem is to write
#RexxVar value_to_process = {$#2}
#evaluate+ ^^ ^if DATATYPE(LEFT(value_to_process,1), "W") then do; parse var value_to_process C '.'; end^ \
in the macro I was creating.
Thanks Bruce. Your reply was most helpful on my route to a solution.

Newbie transforming CSV files in Clojure

I'm both new and old to programming -- mostly I just write a lot of small Perl scripts at work. Clojure came out just when I wanted to learn Lisp, so I'm trying to learn Clojure without knowing Java either. It's tough, but it's been fun so far.
I've seen several examples of similar problems to mine, but nothing that quite maps to my problem space. Is there a canonical way to extract lists of values for each line of a CSV file in Clojure?
Here's some actual working Perl code; comments included for non-Perlers:
# convert_survey_to_cartography.pl
open INFILE, "< coords.csv"; # Input format "Northing,Easting,Elevation,PointID"
open OUTFILE, "> coords.txt"; # Output format "PointID X Y Z".
while (<INFILE>) { # Read line by line; line bound to $_ as a string.
chomp $_; # Strips out each line's <CR><LF> chars.
#fields = split /,/, $_; # Extract the line's field values into a list.
$y = $fields[0]; # y = Northing
$x = $fields[1]; # x = Easting
$z = $fields[2]; # z = Elevation
$p = $fields[3]; # p = PointID
print OUTFILE "$p $x $y $z\n" # New file, changed field order, different delimiter.
}
I've puzzled out a little bit in Clojure and tried to cobble it together in an imperative style:
; convert-survey-to-cartography.clj
(use 'clojure.contrib.duck-streams)
(let
[infile "coords.csv" outfile "coords.txt"]
(with-open [rdr (reader infile)]
(def coord (line-seq rdr))
( ...then a miracle occurs... )
(write-lines outfile ":x :y :z :p")))
I don't expect the last line to actually work, but it gets the point across. I'm looking for something along the lines of:
(def values (interleave (:p :y :x :z) (re-split #"," coord)))
Thanks, Bill
Please don't use nested def's. It doesn't do, what you think it does. def is always global! For locals use let instead. While the library functions are nice to know, here a version orchestrating some features of functional programming in general and clojure in particular.
(import 'java.io.FileWriter 'java.io.FileReader 'java.io.BufferedReader)
(defn translate-coords
Docstrings can be queried in the REPL via (doc translate-coords). Works eg. for all core functions. So supplying one is a good idea.
"Reads coordinates from infile, translates them with the given
translator and writes the result to outfile."
translator is a (maybe anonymous) function which extracts the translation from the surrounding boilerplate. So we can reuse this functions with different transformation rules. The type hints here avoid reflection for the constructors.
[translator #^String infile #^String outfile]
Open the files. with-open will take care, that the files are closed when its body is left. Be it via normal "drop off the bottom" or be it via a thrown Exception.
(with-open [in (BufferedReader. (FileReader. infile))
out (FileWriter. outfile)]
We bind the *out* stream temporarily to the output file. So any print inside the binding will print to the file.
(binding [*out* out]
The map means: take the seq and apply the given function to every element and return the seq of the results. The #() is a short-hand notation for an anonymous function. It takes one argument, which is filled in at the %. The doseq is basically a loop over the input. Since we do that for the side effects (namely printing to a file), doseq is the right construct. Rule of thumb: map: lazy => for result, doseq: eager => for side effects.
(doseq [coords (map #(.split % ",") (line-seq in))]
println takes care for the \n at the end of the line. interpose takes the seq and adds the first argument (in our case " ") between its elements. (apply str [1 2 3]) is equivalent to (str 1 2 3) and is useful to construct function calls dynamically. The ->> is a relatively new macro in clojure, which helps a bit with readability. It means "take the first argument and add it as last item to the function call". The given ->> is equivalent to: (println (apply str (interpose " " (translator coords)))). (Edit: Another note: since the separator is \space, we could here write just as well (apply println (translator coords)), but the interpose version allows to also parametrize the separator as we did with the translator function, while the short version would hardwire \space.)
(->> (translator coords)
(interpose " ")
(apply str)
println)))))
(defn survey->cartography-format
"Translate coords in survey format to cartography format."
Here we use destructuring (note the double [[]]). It means the argument to the function is something which can be turned into a seq, eg. a vector or a list. Bind the first element to y, the second to x and so on.
[[y x z p]]
[p x y z])
(translate-coords survey->cartography-format "survey_coords.txt" "cartography_coords.txt")
Here again less choppy:
(import 'java.io.FileWriter 'java.io.FileReader 'java.io.BufferedReader)
(defn translate-coords
"Reads coordinates from infile, translates them with the given
translator and writes the result to outfile."
[translator #^String infile #^String outfile]
(with-open [in (BufferedReader. (FileReader. infile))
out (FileWriter. outfile)]
(binding [*out* out]
(doseq [coords (map #(.split % ",") (line-seq in))]
(->> (translator coords)
(interpose " ")
(apply str)
println)))))
(defn survey->cartography-format
"Translate coords in survey format to cartography format."
[[y x z p]]
[p x y z])
(translate-coords survey->cartography-format "survey_coords.txt" "cartography_coords.txt")
Hope this helps.
Edit: For CSV reading you probably want something like OpenCSV.
Here's one way:
(use '(clojure.contrib duck-streams str-utils)) ;;'
(with-out-writer "coords.txt"
(doseq [line (read-lines "coords.csv")]
(let [[x y z p] (re-split #"," line)]
(println (str-join \space [p x y z])))))
with-out-writer binds *out* such that everything you print will go to the filename or stream you specify, rather than standard-output.
Using def as you're using it isn't idiomatic. A better way is to use let. I'm using destructuring to assign the 4 fields of each line to 4 let-bound names; then you can do what you want with those.
If you're iterating over something for the purpose of side-effects (e.g. I/O) you should usually go for doseq. If you wanted to collect up each line into a hash-map and do something with them later, you could use for:
(with-out-writer "coords.txt"
(for [line (read-lines "coords.csv")]
(let [fields (re-split #"," line)]
(zipmap [:x :y :z :p] fields))))