I am writing a parser (in C++) and I have a small list of strings (less than 100) where each one represents a valid parser tag. I need to map each such known tag to an enum value for further processing.
As all strings are known at compile time, I have been looking into using a perfect hash function for this purpose.
I am aware of existing tools and algorithms for perfect hash function generation s.a. gperf, mph, cmph. However, all such tools/implementations are under some restrictive license (such as GPL, LGPL, MPL), while due to my limitations I am looking for some code which is under a relaxed license for reuse (such as MIT license) and preferably in C/C++ or C#.
Are you aware of any such tool or code ?
Yes, here's one that seems to fit your parameters:
https://www.codeproject.com/Articles/989340/Practical-Perfect-Hashing-in-Csharp
Note it's using a license agreement that I'm not particularly familiar with. But it doesn't look like its GPL related.
Related
I have developed a big-integer system (for numbers of dynamical length) in Forth (ANS-94), but it is too slow. Are there any fast systems available?
Also, any tip regarding the subject is welcome.
The Forth Scientific Library includes Arithmetic on big signed-magnitude numbers module. Although, a test is required to see what implementation is faster. To use this module in SP-Forth some libraries should be included:
REQUIRE [IF] lib/include/tools.f
REQUIRE M+ lib/include/double.f
S" big.fth" INCLUDED \ just for example
In any case, if Forth generates a code that performance is not sufficient (and algorithm is already optimized), any other implementation in form of dynamic library (DLL, SO) can be used.
Regarding performance, it is better to use dynamic memory (instead of dictionary space) for buffers in SP-Forth 4.21, since data-space and code-space are not separated.
I am kinda playing with the SHA-1 algorithm. I want to find out differences and variations in the results if I change few values in the SHA-1 algorithm for a college report. I have found a piece of java code to generate hash of a text. Its done by importing
java.security.MessageDigest
class. However, I want to change the h0-4 values and edit them but I don't know where can I find them? I had a look inside the MessageDigest class but couldn't find it there. Please help me out!
Thanx in advance.
I don't believe you can do that. Java doesn't provide any API for its MessageDigest Class, which can allow you change the values.
However, there are some workarounds (none of which I've ever tried). Take a look at this answer to the question "How to edit Java Platform Package (Built-in API) source code?"
If you're playing around with tweaks to an algorithm, you shouldn't be using a built-in class implementing that algorithm. The class you mention is designed to implement standard algorithms for people who just want to use them in production; if you're using SHA-1 (or any cryptographic algorithm) instead of playing around and tweaking it, it's never a good idea to change the algorithm yourself (e.g. by changing the initial hash value), so the class does not support modifying those constants.
Just implement the algorithm yourself; from Wikipedia's pseudocode, it doesn't look like it's all that complicated. I know that "don't implement your own crypto, use a standard and well-tested implementation" is a common mantra here, but that only applies to production-type code -- if you're playing around with an algorithm to see what effect tweaking it has, you should implement it yourself, so you have more flexibility in modifying it and seeing the effect of the modifications.
Basically adding to #Rahil's answer but too much for comments:
Even without API access, if MessageDigest were the implementation you could use reflection. But it's not.
Most of the java standard library is just commonly-useful classes in the usual way, e.g. java.util.ArrayList contains the implementation of ArrayList (or ArrayList<?> since 6), java.io.FileInputStream contains the implementation of FileInputStream (although it may use other classes in that implementation), etc. Java Cryptography uses a more complicated scheme where the implementations are not in the API classes but instead in "providers" that are mostly in their own jars (in JRE/lib and JRE/lib/ext) not rt.jar and mostly(?) don't have source in src.zip.
Thus the java.security.MessageDigest class does not have the code to implement SHA1, or SHA256, or MD5, etc etc. Instead it has code to search the JVM's current list of crypto providers to find an implementation of whatever algorithm is asked for, and instantiate and use that. Normally the list of providers used is set to (the list of) those included in the JRE distribution, although an admin or program can change it.
With the normal JRE7 providers, SHA1 is implemented by sun.security.provider.SHA.
In effect the API classes like MessageDigest Signature Cipher KeyGenerator etc function more like interfaces or facades by presenting the behavior that is common to possibly multiple underlying implementations, although in Java code terms they are actual classes and not interfaces.
This was designed back in 1990 or so to cope with legal restrictions on crypto in effect then, especially on export from the US. It allowed the base Java platform to be distributed easily because by itself it did no crypto. To use it -- and even if you don't do "real" crypto on user data in Java you still need things like verification of signed code -- you need to add some providers; you might have one set of providers, with complete and strong algorithms, used in US installations, and a different set, with fewer and weaker algorithms, used elsewhere. This capability is now much less needed since the US officially relaxed and in practice basically dropped enforcement about 2000, although there are periodically calls to bring it back. There is still one residual bit, however: JCE (in Oracle JREs) contains a policy that does not allow symmetric keys over 128 bits; to enable that you must download from the Oracle website and install an additional (tiny) file "JCE Unlimited Strength Policy".
TLDR: don't try to alter the JCE implementation. As #cpast says, in this case where you want to play with something different from the standard algorithm, do write your own code.
A friend of mine needs to implement some statistical calculations in hardware.
She wants it to be accomplished using VHDL.
(cross my heart, I haven't written a line of code in VHDL and know nothing about its subtleties)
In particular, she needs a direct analogue of MATLAB's betainc function.
Is there a good package around for doing this?
Any hints on the implementation are also highly appreciated.
If it's not a good idea at all, please tell me about it as well.
Thanks a lot!
There isn't a core available that performs an incomplete beta function in the Xilinx toolset. I can't speak for the other toolsets available, although I would doubt that there is such a thing.
What Xilinx does offer is a set of signal processing blocks, like multipliers, adders and RAM Blocks (amongst other things, filters, FFTs), that can be used together to implement various custom signal transforms.
In order for this to be done, there needs to be a complete understanding of the inner workings of the transform to be applied.
A good first step is to implement the function "manually" in matlab as a proof of concept:
Instead of using the built-in function in matlab, your friend can try to implement the function just using fundamental operators like multipliers and adders.
The results can be compared with those produced by the built-in function for verification.
The concept can then be moved to VHDL using the building blocks that are provided.
Doing this for the incomplete beta function isn't something for the faint-hearted, but it can be done.
As far as I know there is no tool which allow interface of VHDL and matlab.
But interface of VHDL and C is fairly easy, so if you can implement your code(MATLAB's betainc function) in C then it can be done easily with FLI(foreign language interface).
If you are using modelsim below link can be helpful.
link
First of all a word of warning, if you haven't done any VHDL/FPGA work before, this is probably not the best place to start. With VHDL (and other HDL languages) you are basically describing hardware, rather than a sequential line of commands to execute on a processor (as you are with C/C++, etc.). You thus need a completely different skill- and mind-set when doing FPGA-development. Just because something can be written in VHDL, it doesn't mean that it actually can work in an FPGA chip (that it is synthesizable).
With that said, Xilinx (one of the major manufacturers of FPGA chips and development tools) does provide the System Generator package, which interfaces with Matlab and can automatically generate code for FPGA chips from this. I haven't used it myself, so I'm not at all sure if it's usable in your friend's case - but it's probably a good place to start.
The System Generator User guide (link is on the previously linked page) also provides a short introduction to FPGA chips in general, and in the context of using it with Matlab.
You COULD write it yourself. However, the incomplete beta function is an integral. For many values of the parameters (as long as both are greater than 1) it is fairly well behaved. However, when either parameter is less than 1, a singularity arises at an endpoint, making the problem a bit nasty. The point is, don't write it yourself unless you have a solid background in numerical analysis.
Anyway, there are surely many versions in C available. Netlib must have something, or look in Numerical Recipes. Or compile it from MATLAB. Then link it in as nav_jan suggests.
As an alternative to VHDL, you could use MyHDL to write and test your beta function - that can produce synthesisable (ie. can go into an FPGA chip) VHDL (or Verilog as you wish) out of the back end.
MyHDL is an extra set of modules on top of Python which allow hardware to be modelled, verified and generated. Python will be a much more familiar environment to write validation code in than VHDL (which is missing many of the abstract data types you might take for granted in a programming language).
The code under test will still have to be written with a "hardware mindset", but that is usually a smaller piece of code than the test environment, so in some ways less hassle than figuring out how to work around the verification limitations of VHDL.
If I want to write my engine which will generate all the code solving the task described in simple declarative style, what languages should I look at?
Prolog. Definitely Prolog. I know it's not the vanilla option, so here is the rationale:
Prolog has a flexible syntax that can be made even more flexible by using its interpret-time macro expansion mechanism (a.k.a term expansion).
In case that the native syntax won't do, Prolog has a good built-in parsing mechanism: Definite Clauses Grammar (DCG).
Prolog is intended for finding solutions based on declarations.
Prolog has several useful libraries for declarative computing, such as constraint solving, linear equations solver, etc.
Prolog searches for all solutions, thus making variations and optimizations more natural.
Prolog uses flexible data structures (functors) which it can both examine and generate, so generating complex structures in rather natural. You don't need string generation: You can generate functors and then print them. DCG also helps with this.
I actually did this sort of projects: Generators from natural-looking Prolog to languages like SQL and Erlang. Getting to know Prolog takes some time, but in my experience it's worth your while.
That's an extremely broad topic, so it deserves an extremely broad answer.
An engine designed to implement arbitrary processing and generation of code is the DMS Software Reengineering Toolkit. DMS parses a wide variety of langauges, will accept definitons of more languages (including specification or modelling languages), provides pattern matching and transformation using declarative patterns written at source level syntax, etc.
DMS isn't a single language; rather, it is a set of Domain-Specific Languages (DSLs) each of which provides support for one of the issues that a code metaprogramming tool must address: langauge grammars, attribute computations, pattern matching/transformation, flow analysis, task scripting.
DMS is extremely powerful; it has been used to build many code analysis, generation, and transformation tools (you can see a wide variety of examples at the web site offered as COTS tools). It is not necessarily easy, because analyzing/transforming code for real langauges such as Java, C# and C++ is complicated, because those languages are complicated, and because the fundamental problem of transforming code from one level of abstraction to another and producing optimized results is fundamentally complicated. I'll claim DMS is as easy as practical for the problem targeted.
(Full disclosure: I'm the principal behind DMS).
I am doing a compilers discipline at college and we must generate code for our invented language to any platform we want to. I think the simplest case is generating code for the Java JVM or .NET CLR. Any suggestion which one to choose, and which APIs out there can help me on this task? I already have all the semantic analysis done, just need to generate code for a given program.
Thank you
From what I know, on higher level, two VMs are actually quite similar: both are classic stack-based machines, with largely high-level operations (e.g. virtual method dispatch is an opcode). That said, CLR lets you get down to the metal if you want, as it has raw data pointers with arithmetic, raw function pointers, unions etc. It also has proper tailcalls. So, if the implementation of language needs any of the above (e.g. Scheme spec mandates tailcalls), or if it is significantly advantaged by having those features, then you would probably want to go the CLR way.
The other advantage there is that you get a stock API to emit bytecode there - System.Reflection.Emit - even though it is somewhat limited for full-fledged compiler scenarios, it is still generally enough for a simple compiler.
With JVM, two main advantages you get are better portability, and the fact that bytecode itself is arguably simpler (because of less features).
Another option that i came across what a library called run sharp that can generate the MSIL code in runtime using emit. But in a nicer more user friendly way that is more like c#. The latest version of the library can be found here.
http://code.google.com/p/runsharp/
In .NET you can use the Reflection.Emit Namespace to generate MSIL code.
See the msdn link: http://msdn.microsoft.com/en-us/library/3y322t50.aspx