I'm learning how to write parsers, and to do so, I'm writing parser for SQL.
Grammar I'm writing is processed by perl Parse::Eyapp module, which is very similar to standard yacc.
When I added support for single-operand operators (not sure what is the correct name - operators like 12!, or ## 'value'), when compiling the grammar to perl, I got:
14 shift/reduce conflicts
I had it also earlier, but I solved it by adding appropriate %left and %right, but this time I'm at loss, since the problem seems to be from conflict between 1-operand operators, and more traditional two-operand ones.
Full grammar is too long to put in here, so I'll just link to it.
To compile it, I use command:
eyapp -m Pg::SQL::Parser::SQL -o SQL.pm SQL.eyp
When running eyapp ... with verbose enabled, I get this output.
So, the question is: how to solve the problem in here?
Aargh. Looks like I misdiagnosed the problem. The real cause of the problem were not unary operators, but cast (expr '::' normal_type).
Adding %left '::' at the end of priority configs solved the problem.
In case you're curious - commit link.
Related
As a Perl beginner I am sometimes getting compilation errors and have to search a lot to find it. In the end it is just a missing semicolon at the end of a line. Some syntax errors with missing semicolon are checked by Perl but not in general. Is there a way to get this check?
edit:
I know about Perl::Critic but can't use it atm. And I don't know if it checks for missing semicolon in general.
Because semicolons actually mean something in Perl and aren't just there for decoration, it's not possible for any tool (even the Perl interpreter itself) to know in every case whether you actually meant to leave off the semi-colon or not. Thus, there's no general-case answer to your question; you'll just need to go through your code and make sure it's correct.
As mentioned in my comments, there are various tricks you can try with your editor to expedite the process of finding potentially-incorrect lines; you must, however, either examine and fix these by hand or risk introducing new problems.
The syntax check is perl -c, but that's no different than attempting to run the program outright. Due to its flexible/undecidable syntax, one cannot generally do what you want. That's the downside of comfort and expressiveness.
Upgrade to the latest stable Perl, the parser's error messages got better/more exact over the last years and will correctly recognise many circumstances of a missing semicolon.
Rule of thumb that works for many parsers/other languages: if the error makes no sense, look a couple of lines before.
use diagnostics; usually gives you a nice hint, same as use warnings;. Try to keep a consistent coding style, check perlstyle.
Also you can use Perl::Critic online.
Also as general advice learn how to use packages and modules, try to group code into subs and study the syntax of arrays, lists and hashes. A common mistake is forgetting the ; after an anonymous hashref assignment:
my $hashref = { a => 5, b => 10};
Is there a CoffeeScript fork with no changes other than fixed scoping, so that it's largely compatible with CoffeeScript (completely compatible if the code has no outer-variable assignments)? I'd consider acceptable ways to assign an outer variable to include := a la LiveScript, or nonlocal a la Python, or something else.
There's not. I think LiveScript is compatible with CoffeeScript, but that isn't anything near "a small patch".
Could be that Coco is your friend.
https://github.com/satyr/coco
I am going to try it myself now..
On its way to hide JavaScript's bad parts, CoffeeScript has
accumulated own quirks: horrible variable scope, awkward ranges,
confusing and/or pointless keywords, verbose file extension, and so
on. Coco tries to amend them, entwining good parts of both.
found in a list of coffee alternatives:
https://github.com/jashkenas/coffee-script/wiki/List-of-languages-that-compile-to-JS
EDIT:
There is still a caveat with the Coco solution, afaik.
The := syntax only helps you with assignments, not with read access.
I have been encountering errors in Perl code that are reported at places which have nothing to do with the error. I fixed one such error after hours of removing and re adding code line by line and then doing some trial and error. Two such errors are detailed below. My question is : If these issues happen in future, is there a way to ensure that perl compiler helps me fix this OR do I have to rewrite the code in some other language. (am considering Java).
My program looks like this:
use switch;
use strict;
use warnings;
...other modules;
sub log{
}
..various sub routines
switch {$val1)
{
log(..) #first invocation of log
case ($val2)
...
}
Now within sub log if I do this
{
$val3 = POSIX::floor($val2/$val4)*$val4;
$val5="/x/y/$logfilename";
}
I get an error saying that there is an error at the case statement.
If I move the line $val5="/x/y/$logfilename"; BEFORE $val3, there is no error.
OR if I remove the '/' in $val5 i.e. $val5="x", there is no error
OR if I say $val5=qq(/x/y/$logfilename); there is no error.
This time I consider myself lucky that I found a workaround but it was only after 3 hours of fighting this. Is there a way to make perl compiler report the errors accurately?
I have one more similar case to report and can add if necessary.
Inputs solicited
Ignoring the problems with your post, the issue you are probably running into is with the fact that the switch functionality in perl is implemented with a source filter. This basically means that the source code is preprocessed to convert what looks like a switch case structure into valid perl code before compilation. Perl's parser usually gets line numbers a little wrong on complex structures, but when source filters are involved, all bets are off. This is one of the reasons that Switch has been deprecated in modern versions of Perl.
The solution is to stop using switch, and either use Perl's other control structures, or use a version of perl new enough to support the given/when construct (5.10+), which is modern Perl's version of switch/case.
LIMITATIONS in Switch sounds like it's relevant:
Due to the heuristic nature of Switch.pm's source parsing, the presence of regexes with embedded newlines that are specified with raw /.../ delimiters and don't have a modifier //x are indistinguishable from code chunks beginning with the division operator /. As a workaround you must use m/.../ or m?...? for such patterns. Also, the presence of regexes specified with raw ?...? delimiters may cause mysterious errors. The workaround is to use m?...? instead.
Switch module is deep black magic. Please don't use it unless you're prepared to debug its problems.
I know I risk asking a speculative question, however, inspired by this recent question I wonder which editor does the best job of syntax highlighting Perl. Being well aware of the difficulties (impossibilities) of parsing Perl I know there will not be a perfect case. Still I wonder if there is a clear leader in faithful representation.
N.B. I use gedit and it works fine, but with known issues.
Komodo Edit does a good job and also scans your modules (including those installed via CPAN) and does well at generating autocomplete data for them.
I'm a loyal vim user and rarely encounter anything odd with the native syntax.vim, except for these cases (I'll edit in more if/when I find them; others please feel free also):
!!expression is better written !!!!expression (everything after two ! is rendered as a comment quoted string; four ! brings everything back to normal)
m## or s### renders everything after the # as a comment; I usually use {} as a delimiter when avoiding / for leaning toothpick syndrome
some edge cases for $hash{key} where key is not a simple alphanumeric string - although it's safer to enclose such key names in '' anyway so as to not have to look up the exact cases for when a bareword is treated as a key name
I haven't used it, but Padre should be good since it's written in Perl. IIRC It uses PPI for parsing
jEdit...with the tweaks that I have amassed over the years. It's got the most customizable syntax highlighting I've ever seen.
I use Emacs in CPerl mode. I think it does a terrific job, although similar to Ether's answer, it's not perfect. What's more, I usually use Htmlize to publish highlighted code to the web. It's kind of annoying to use fancier forums like this one that do their own syntax highlighting, since it's not really any easier and the results aren't as good.
It is "common knowledge" that source filters are bad and should not be used in production code.
When answering a a similar, but more specific question I couldn't find any good references that explain clearly why filters are bad and when they can be safely used. I think now is time to create one.
Why are source filters bad?
When is it OK to use a source filter?
Why source filters are bad:
Nothing but perl can parse Perl. (Source filters are fragile.)
When a source filter breaks pretty much anything can happen. (They can introduce subtle and very hard to find bugs.)
Source filters can break tools that work with source code. (PPI, refactoring, static analysis, etc.)
Source filters are mutually exclusive. (You can't use more than one at a time -- unless you're psychotic).
When they're okay:
You're experimenting.
You're writing throw-away code.
Your name is Damian and you must be allowed to program in latin.
You're programming in Perl 6.
Only perl can parse Perl (see this example):
#result = (dothis $foo, $bar);
# Which of the following is it equivalent to?
#result = (dothis($foo), $bar);
#result = dothis($foo, $bar);
This kind of ambiguity makes it very hard to write source filters that always succeed and do the right thing. When things go wrong, debugging is awkward.
After crashing and burning a few times, I have developed the superstitious approach of never trying to write another source filter.
I do occasionally use Smart::Comments for debugging, though. When I do, I load the module on the command line:
$ perl -MSmart::Comments test.pl
so as to avoid any chance that it might remain enabled in production code.
See also: Perl Cannot Be Parsed: A Formal Proof
I don't like source filters because you can't tell what code is going to do just by reading it. Additionally, things that look like they aren't executable, such as comments, might magically be executable with the filter. You (or more likely your coworkers) could delete what you think isn't important and break things.
Having said that, if you are implementing your own little language that you want to turn into Perl, source filters might be the right tool. However, just don't call it Perl. :)
It's worth mentioning that Devel::Declare keywords (and starting with Perl 5.11.2, pluggable keywords) aren't source filters, and don't run afoul of the "only perl can parse Perl" problem. This is because they're run by the perl parser itself, they take what they need from the input, and then they return control to the very same parser.
For example, when you declare a method in MooseX::Declare like this:
method frob ($bubble, $bobble does coerce) {
... # complicated code
}
The word "method" invokes the method keyword parser, which uses its own grammar to get the method name and parse the method signature (which isn't Perl, but it doesn't need to be -- it just needs to be well-defined). Then it leaves perl to parse the method body as the body of a sub. Anything anywhere in your code that isn't between the word "method" and the end of a method signature doesn't get seen by the method parser at all, so it can't break your code, no matter how tricky you get.
The problem I see is the same problem you encounter with any C/C++ macro more complex than defining a constant: It degrades your ability to understand what the code is doing by looking at it, because you're not looking at the code that actually executes.
In theory, a source filter is no more dangerous than any other module, since you could easily write a module that redefines builtins or other constructs in "unexpected" ways. In practice however, it is quite hard to write a source filter in a way where you can prove that its not going to make a mistake. I tried my hand at writing a source filter that implements the perl6 feed operators in perl5 (Perl6::Feeds on cpan). You can take a look at the regular expressions to see the acrobatics required to simply figure out the boundaries of expression scope. While the filter works, and provides a test bed to experiment with feeds, I wouldn't consider using it in a production environment without many many more hours of testing.
Filter::Simple certainly comes in handy by dealing with 'the gory details of parsing quoted constructs', so I would be wary of any source filter that doesn't start there.
In all, it really depends on the filter you are using, and how broad a scope it tries to match against. If it is something simple like a c macro, then its "probably" ok, but if its something complicated then its a judgement call. I personally can't wait to play around with perl6's macro system. Finally lisp wont have anything on perl :-)
There is a nice example here that shows in what trouble you can get with source filters.
http://shadow.cat/blog/matt-s-trout/show-us-the-whole-code/
They used a module called Switch, which is based on source filters. And because of that, they were unable to find the source of an error message for days.