Using space depth to parse config file's contexts with Perl - perl

Question

I've been trying to create a config file parser to parse Cisco IOS configs and such. The final objective would be to show relevant data in contexts based on filters in a configuration file. For example, with such a config file it would display all interfaces where we've found the line "access vlan" as a child of the "interface" context and only show lines containing "speed", "duplex" and "description".
{
'Context' => '^interface',
'Types' => [
'Switch',
],
'Condition' => 'access vlan',
'Filter' => [
'speed',
'duplex',
'description'
]
};
So far, so good. I read the "running-config" and I index the lines depth (given that a non-empty line , not beginning with a space (\s) has a depth of 0) in an array.
Then, in another read I use that index to read the data again, this time using relative position based on depth to create the "childs" of a context. Here's the function :
sub getDeep {
my #data = (#_);
my ($bighash,$hash);
#First read
foreach my $idx (0.. $#data) {
my ($spaces, $content) = ($data[$idx] =~ m/^(\s*)(.*)/);
my $depth = length $spaces;
$bighash->{node}{$idx}{depth} = $depth;
}
# Variables for the first read
my $ldepth = 0;
my $lcontext;
my $lid;
# Second read
foreach my $id (0 .. $#data) {
$data[$id] =~ s/^\s*//;
next if ($data[$id] =~ /^!/);
my $depth = $bighash->{node}{$id}{depth};
if ($depth eq 0) {
push (#{$hash->{global}} , $data[$id]);
$lcontext = $data[$id];
$lid = $id;
}
if (($depth gt 0) && ($id - $lid eq 1)) {
push (#{$hash->{$lcontext}}, (" " x $depth. $data[$id]));
$lid = $id;
}
}
return $hash;
}
Using this sub, I can return a hash, then based on the presence of an arrayref for a given key, apply filters as explained. This works pretty well, so far very proud of this piece of code.
Problem comes when I want to find childs of childs. In the example below, the childs of "given param2" would reprensent my next challenge.
interface XYZ
given param1 -> child of "interface XYZ"
given param2 -> child of "interface XYZ"
given param2.1 -> child of "given param2"
given param2.2 -> child of "given param2"
given param3 -> child of "interface XYZ"
So after thinking about this for a while and failing with different approaches, my question comes in 2 separate parts :
1) Is there a better way to do this that I'm not seeing ?
2) How could I keep tagging childs of childs as the lines dig deeper and identify them properly in a data structure ?
Thank you for reading up to this line :)

Answer 1

This thread contains the solution I was hoping for :) Since it might benefit to others, here's the link :
https://perlmonks.org/?node_id=1224600
Cheers !

Answer 2

I have written something to do exactly this. I can't figure out how to put it on metacpan. However, im sure there is better already on there if I knew where to look. It's on of the first i wrote in perl, so it's a bit of a mess. But basically you can type "gettree -l Node interface" and on a XR device it'll pull all of the config. "gettree -s Node Description_keyword" will pull all of a single interface config. you can also use it with STDIN e.g., "cat file | gettree -l interface".
Program
#!/usr/bin/perl
use lib '/PATH_TO_Gettree.pm_MODULE/';
use strict;
use warnings;
use Gettree;
use Getopt::Std;
my %opts;
getopts('eislnpm' , \%opts);
my $searchstr;
my $filename;
my $debug=0;
if($ARGV[0]=~m/help/ || $ARGV[0]=~m/^\?$/ )
{ die usage(); }
if($#ARGV<0||$#ARGV>1)
{
usage();
killapp("Please specifiy node and search term, use --help for the help menu");
}
elsif($#ARGV==0)
{
Gettree::setopts( \%opts , \$ARGV[0] );
while(<STDIN>)
{
Gettree::gettree_stream_passline( \$_ );
}
print Gettree::gettree_getreturnstring();
}
else
{
$filename= $ARGV[0];
$filename="/CONFIGS_DIR/".lc $filename if ! $opts{e};
print Gettree::gettree_file ( \%opts , \$filename , \$ARGV[1]) ; #\$filename , $searchstring
}
sub killapp
{
print $_[0]."\n";
exit;
}
sub usage
{
print "
Usage: gettree [OPTION]... [NODE] STRING
Search for PATTERN in each FILE or standard input.
usage gettree <options> <node> <string>
-s include same level
-l include lower levels
-n show line numbers (do not use with STDIN, it wont work)
-i case insensitive
-e exact file location (rather than just the nodename)
-p print parent's same level lines
-m minimal print, do not print parents
Examples:
gettree Node text
gettree -sln NODE CCT_Ref
gettree -l NODE POS8/0
\n\n";
exit;
}
Module
#!/usr/bin/perl
package Gettree;
use strict;
use warnings;
my $line;
my $wsdiff = 0;
my $iopt = 0;
my $sopt = 0;
my $lopt = 0;
my $nopt = 0;
my $popt = 0;
my $mopt = 0;
my $linecounter = 0;
my $matched = -1;
my $debug = 0; ##remove later
my #arr;
my #sopt_arr;
my #popt_arr;
my $searchstr;
my $returnstring;
sub setopts # \%opthash , $searchstring
{
cleardata();
push #arr, [ 0, "",0];
my %hash=%{$_[0]};
$iopt = 1 if $hash{i};
$sopt = 1 if $hash{s};
$lopt = 1 if $hash{l};
$nopt = 1 if $hash{n};
$popt = 1 if $hash{p};
$mopt = 1 if $hash{m};
if ( defined $hash{qopts} )
{
$iopt = 1 if $hash{qopts} =~ /i/;
$lopt = 1 if $hash{qopts} =~ /l/;
$nopt = 1 if $hash{qopts} =~ /n/;
$sopt = 1 if $hash{qopts} =~ /s/;
$popt = 1 if $hash{qopts} =~ /p/;
$mopt = 1 if $hash{qopts} =~ /m/;
}
if ( ref($_[1]) ) { $searchstr=$iopt? qr/${$_[1]}/i : qr/${$_[1]}/ ; }
else { $searchstr=$iopt? qr/$_[1]/i : qr/$_[1]/ ; }
}
sub gettree_stream_passline # \$line
{
process_line(${$_[0]});
}
sub gettree_getreturnstring
{
return $returnstring;
}
sub gettree_varable # \%opthash , \$text , $searchstring
{
setopts($_[0] , $_[2]);
my $str=${$_[1]};
while($str=~m#(.*\n)#g)
{
process_line($1);
}
return $returnstring;
}
sub gettree_file # \%opthash , \$filename , $searchstring
{
setopts($_[0] , $_[2]);
my $filename;
if ( ref($_[1]) ) { $filename=${$_[1]}; }
else { $filename=$_[1] ; }
open FH, "<", $filename or die "\nFile ".$filename." cannot be found\nerror : ".$!."\n";
while(my $text=<FH>)
{
process_line($text);
}
close FH;
return $returnstring;
}
sub process_line
{
$line=shift;
if($line=~m/^([ \t]+)/) { $wsdiff=length($1) }
else { $wsdiff=0 };
if($wsdiff>$arr[$#arr][0])
{
push #arr, [ $wsdiff , $line , $linecounter ];
if ( $sopt || $popt )
{
#popt_arr=#sopt_arr if $popt;
#sopt_arr=() if defined $sopt_arr[0];
}
}
else
{
while( #arr && $arr[$#arr][0]>$wsdiff )
{
pop #arr;
#sopt_arr=#popt_arr if ( $sopt || $popt );
#popt_arr=() if $popt;
}
if($#arr<0)
{
push #arr, [ $wsdiff , $line, $linecounter ];
}
else
{
push #sopt_arr, $arr[$#arr] if $sopt || $popt ;
$arr[$#arr]=[ $wsdiff , $line , $linecounter ];
}
}
#sopt_arr=() if $#sopt_arr>200; ## to avoid filling the memory
#popt_arr=() if $#popt_arr>200; ## to avoid filling the memory
##used in l and s opts to print lines after match
if($matched>=0)
{
if($wsdiff>$matched)
{
printline(\$line) if $lopt==1 ;
}
elsif ($wsdiff<$matched)
{
$matched=-1;
}
else
{
if ($sopt )
{ printline(\$line) }
else
{ $matched=-1 }
}
}
if( $matched==-1 && $line=~m/$searchstr/ )
{
printtree();
$matched=$wsdiff if $sopt || $lopt;
}
$linecounter++;
}
sub printtree
{
if(!$mopt)
{
for (0..$#arr-(1+$popt))
{
printline( \$arr[$_][1] , \$arr[$_][2] );
}
}
if($popt)
{
for (0..$#popt_arr)
{
printline( \$popt_arr[$_][1] , \$popt_arr[$_][2] );
}
printline( \$arr[$#arr-1][1] , \$arr[$#arr-1][2] ); #print the parent
#popt_arr=() ;
}
if($sopt)
{
for (0..$#sopt_arr)
{
printline( \$sopt_arr[$_][1] , \$sopt_arr[$_][2] );
}
#sopt_arr=() ;
}
printline( \$arr[$#arr][1] , \$arr[$#arr][2] );
#arr=();
push #arr, [ $wsdiff , $line , $linecounter ];
}
sub printline
{
$nopt==1? $returnstring.= ${$_[1]}+1 ." : ".${$_[0]} : $returnstring.= ${$_[0]};
}
sub cleardata
{
$line="";
$wsdiff = 0;
$iopt = 0;
$sopt = 0;
$lopt = 0;
$nopt = 0;
$popt = 0;
$mopt = 0;
$linecounter = 0;
$matched = -1;
#arr=();
#sopt_arr=();
#popt_arr=();
$searchstr="";
$returnstring="";
}
1;
Breif explanation how it works
The program is just a link to the module. I's a module because i've used it in many programs and stand alone. Gettree.pm will send data line by line to process_line(). process line will get the white-space ($wsdiff) and use this as a marker. any line directly before an increment in whitespace will be stored in #arr. for printing if a match is found later. so the parent is stored. #sopt_arr is for the same line, so it stores previous lines of the same white-space. #popt_arr is for the parent match, but this doesn't work very well (i dont really use it, it could be removed). when a match for the search string is made, #arr,Sopt_arr, & #popt_arr are printed, $matched is set, this will be used for -l option. all lines after the match are printed until the white-space is < the matched white-space. so in summary, it will take each unique white-space before an increment. it works for Juniper and Alcatel too. I'm sure it would work on others too.
remember to modify CONFIGS_DIR and PATH_TO_Gettree.pm_MODULE to match your filesystem paths