I have a C file that has a bunch of #defines for bits that I'd like to reference from python. There's enough of them that I'd rather not copy them into my python code, instead is there an accepted method to reference them directly from python?
Note: I know I can just open the header file and parse it, that would be simple, but if there's a more pythonic way, I'd like to use it.
Edit:
These are very simple #defines that define the meanings of bits in a mask, for example:
#define FOO_A 0x3
#define FOO_B 0x5
Running under the assumption that the C .h file contains only #defines (and therefore has nothing external to link against), then the following would work with swig 2.0 (http://www.swig.org/) and python 2.7 (tested). Suppose the file containing just defines is named just_defines.h as above:
#define FOO_A 0x3
#define FOO_B 0x5
Then:
swig -python -module just just_defines.h ## generates just_defines.py and just_defines_wrap.c
gcc -c -fpic just_defines_wrap.c -I/usr/include/python2.7 -I. ## creates just_defines_wrap.o
gcc -shared just_defines_wrap.o -o _just.so ## create _just.so, goes with just_defines.py
Usage:
$ python
Python 2.7.3 (default, Aug 1 2012, 05:16:07)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import just
>>> dir(just)
['FOO_A', 'FOO_B', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '_just', '_newclass', '_object', '_swig_getattr', '_swig_property', '_swig_repr', '_swig_setattr', '_swig_setattr_nondynamic']
>>> just.FOO_A
3
>>> just.FOO_B
5
>>>
If the .h file also contains entry points, then you need to link against some library (or more) to resolve those entry points. That makes the solution a little more complicated since you may have to hunt down the correct libs. But for a "just defines case" you don't have to worry about this.
You might have some luck with the h2py.py script found in the Tools/scripts directory of the Python source tarball. While it can't handle complex preprocessor macros, it might be sufficient for your needs.
Here is a description of the functionality from the top of the script:
Read #define's and translate to Python code.
Handle #include statements.
Handle #define macros with one argument.
Anything that isn't recognized or doesn't translate into valid
Python is ignored.
Without filename arguments, acts as a filter.
If one or more filenames are given, output is written to corresponding
filenames in the local directory, translated to all uppercase, with
the extension replaced by ".py".
By passing one or more options of the form "-i regular_expression"
you can specify additional strings to be ignored. This is useful
e.g. to ignore casts to u_long: simply specify "-i '(u_long)'".
#defines are macros, that have no meaning whatsoever outside of your C compiler's preprocessor. As such, they are the bane of multi-language programmers everywhere. (For example, see this Ada question: Setting the license for modules in the linux kernel from two weeks ago).
Short of running your source code through the C-preprocessor, there really is no good way to deal with them. I typically just figure out what they evalutate to (in complex cases, often there's no better way to do this than to actually compile and run the damn code!), and hard-code that value into my program.
The (well one of the) annoying parts is that the C preprocessor is considered by C coders to be a very simple little thing that they often use without even giving a second thought to. As a result, they tend to be shocked that it causes big problems for interoperability, while we can deal with most other problems C throws at us fairly easily.
In the simple case shown above, by far the easiest way to handle it would be to encode the same two values in constants in your python program somewhere. If keeping up with changes is a big deal, it probably wouldn't be too much trouble to write a Python program to parse those values out of the file. However, you'd have to realise that your C code would only re-evaluate those values on a compile, while your python program would do it whenever it runs (and thus should probably only be run when the C code is also compiled).
If you're writing an extension module, use http://docs.python.org/3/c-api/module.html#PyModule_AddIntMacro
I had almost exactly this same problem so wrote a Python script to parse the C file. It's intended to be renamed to match your c file (but with .py instead of .h) and imported as a Python module.
Code: https://gist.github.com/camlee/3bf869a5bf39ac5954fdaabbe6a3f437
Example:
configuration.h
#define VERBOSE 3
#define DEBUG 1
#ifdef DEBUG
#define DEBUG_FILE "debug.log"
#else
#define NOT_DEBUGGING 1
#endif
Using from Python:
>>> import configuration
>>> print("The verbosity level is %s" % configuration.VERBOSE)
The verbosity level is 3
>>> configuration.DEBUG_FILE
'"debug.log"'
>>> configuration.NOT_DEBUGGING is None
True
Related
Program compiles, but freezes after starting. If replace the format and include with 32-bit versions or comment out the MessageBox, then everything works fine.
format PE64 GUI
include 'E:\Fresh\include\win64a.inc'
entry start
section '.data' data readable writeable
text db 'Hello world!',0
section '.text' code readable executable
start:
invoke MessageBox,0,text,text,0
invoke ExitProcess,0
section '.idata' import data readable writeable
library kernel32,'KERNEL32.DLL', user32, 'USER32.DLL'
import kernel32, ExitProcess, 'ExitProcess'
import user32, MessageBox, 'MessageBoxA'
Your stack is not aligned to 16 bytes, as the ABI requires. Add and rsp, -16 to the beginning of your code, and it will work.
Regarding this exchange in the comments:
Ruslan: What does the disassembly look like? Are invoke macros expanded as expected?
rancid_rot: Not sure, there is MessageBox in cs instead of ds. And mov rcx,0 instead push 0.
I'd recommend avoiding invoke and similar macros until you learn what they should expand to. Otherwise you think you write in assembly, but actually you write in a high-level language only resembling assembly, not even knowing what code you will get in the end—thus defying the whole purpose of using an assembler.
To actually learn to call functions in Win64 assembly, see the documentation on Win64 calling conventions.
Simple question here, just can't seem to pass it google in a way it can understand.
Say I wanted to execute a line of actual programming code (c++ or java or python... etc) like SetCursorPos or printf from the command prompt command line. I vaguely imagine I would have to invoke the compiler and pass the command to it like a parameter, where from it would then be converted into machine language and passed to... where exactly?
Okay so that was kind of two questions.
How to run actual code from the command line and
what exactly is happening when a fully compiled program, or converted line of code (presuming these are essentially binary containers at that point), is executed?
Question one takes priority obviously. Unfortunately, I can not find any documentation on it, just a bunch of stuff vaguely related to it.
How to run actual code from the command line
Without delving into the vast amounts of blurriness between them, there are two major categories of language implementations: interpreters and compilers.
With many interpreters (or implementations with implicit compilation, such as V8 JavaScript's jit compiler, or pretty much anything with a repl), running a single line from the command line should be fairly trivial. CPython (the standard implementation of Python) has the -c command option:
$ python -c 'print("Hello, world!")'
Hello, world!
Language implementations with explicit compilation steps will tend to be decidedly less simple. In particular, the compiler would need to either accept source either from directly out of the argument list, or from standard input (via piping or redirection). On the output side, your compiler would have to support immediately executing that program, or outputting it to standard out, so that an operating system feature (if it exists) can execute it from a pipe.
To my knowledge, most explicit compilers are not designed with such usage in mind. In such cases, your best bet is to see if there is a REPL available for the language in question, preferably one as compatible with your compiler as possible, or to create (or find) a wrapper that makes it look like your language has a REPL. The wrapper would:
Accept input along the lines of CPython above.
Create a temporary source file behind the scenes with the code to be run and any necessary boilerplate.
Pass that file to the compiler.
Automatically run the resulting executable.
Delete the source file and executable. These may be cleaned up by the operating system later instead, if they're in a temp directory.
From the point of view of the user, this should look pretty similar to the CPython example, as they wouldn't have to interact with or see the compiler or temporary files.
I have a third party piece of code that works differently when I add a macro via Makefile e.g. -DMacro instead of doing #define MACRO in a top level header file
(which as their documentation implies is included in ALL files).
I Googled if there are any differences in defining it in different ways but could not come up with much except Precedence of -D MACRO and #define MACRO.
I am wondering if I am missing anything about make documentation / C standards before I start debugging and determining the issue.
Thanks for any answers.
Usually, it's exactly the same but neither make nor the ISO standard have anything to say about it. It's up to the compiler itself, some may not even have a -D option.
To make, it's just running the command (such as gcc) with whatever options it takes. ISO doesn't specify anything about how to run a compiler, just how the compiler (and the things it creates) behaves.
For gcc, the preprocessor options can be found here so it looks like it is identical to #define.
Could you please tell me how to get the file separator of current operating system, for example \ in Windows and / in Unix, in Fortran at run-time.
You can use Fortran 2003 Standard intrinsic procedure GET_ENVIRONMENT_VARIABLE to do something like this. Example:
CHARACTER(LEN=99999) :: path
CHARACTER(LEN=1) :: path_separator
CALL GET_ENVIRONMENT_VARIABLE('PATH',path)
path_separator=path(1:1)
WRITE(*,*)'Path separator is ',path_separator
END
This program will output "/" as a path separator in UNIX or Linux. You could get this from other environment variables as well. Notice that this example is hardwired for UNIX/Linux. You would need a bit different logic to extract e.g. "\" for Windows, but I am not familiar with this system. I vaguely remember from Win95 having something like "c:\.....", so it is likely that in case of Windows you would look for "\" in path(3:3).
Hope this helps.
As far as I know the Fortran standard does not say anything about the file system path separator. The best I can suggest is to define a macro which defines the appropriate separator. For example
#if __unix__
character(len=1), parameter :: path_sep='/'
#elif _WIN32
character(len=1), parameter :: path_sep='\'
#else
#error "path_sep not defined. Set this constant for your system."
#endif
However, on Windows you can use either / or \ as your path separator (try it!) so on most systems there is no need to do anything special with the path separator.
Note that you don't need to determine the path separator at run-time. Since you must recompile your Fortran source for each new system you want to run it on, you just need to ensure that the correct path separator is specified at compile time for each new system (the path separator won't change between runs of your program on the same system).
Using preprocessor macros like those above it the obvious way of doing this: you can just add a new #elif clause for each new system you port your code to. This is alot easier than introducing picemeal solutions which check particular environment variables like the other answers suggest.
In addition the specifying the path separator you may also need to set parameters for other aspects of the operating system environment. For example, the current and parent directory characters (. and .. respectively under Windows and POSIX systems).
Finally, note that the preprocessor defines __unix__ and _WIN32 are widely used macros which indicate which system you are on and one, depending which system you are on, should be set by most compilers. If they are not set you can usually use the flag -D__unix__=1 or -D_WIN32=1 when you are compiling your source to set one of these macros. To preprocess your code using the -cpp (gfortran) or -fpp (ifort) command line options or change your source files extensions from .f to .F or from .f90 to .F90.
Following IRO-bot suggestion, you could make a new environment variable, eg
export DELIMITER='/'
in unix
Now make a corresponding variable in windows, just with '\', and you are good to go.
If you don't want to set a global variable in windows, or doesn't have the rights to do so, put the following in an .bat file
set DELIMITER=\
For more on set, see http://www.ss64.com/nt/set.html
You may have to run the fortran program from CMD when using set.
Finaly put the following in your fortran file
CHARACTER :: delimiter
CALL get_environment_variable('DELIMITER',delimiter)
I rather favor the solution of 4 above, but with one modification:
function delimiter_detect() result(delim)
character(LEN=1) :: delim
character(LEN=258) :: path
integer :: ii, stat_out
call GET_ENVIRONMENT_VARIABLE('HOME',path,status=stat_out)
if (stat_out/=0) go to 999
! xxx path='xx\' ! alternate test
do ii=1, 258
delim=path(ii:ii)
if (delim=='/'.or.delim=='\') return
! ... Note: emacs does not recognize the second delimiter test
enddo
999 continue
write(*,'(1x,a/1x,a/1x,a)') 'Could not obtain home directory path.', &
'Restart program and manually insert path', &
'to folder/directory containing input files.'
stop
end function delimiter_detect
That said, I haven't tested the concept on windows OS.
Take an undocumented executable of unknown origin. Trying /?, -h, --help from the command line yields nothing. Is it possible to discover if the executable supports any command line options by looking inside the executable? Possibly reverse engineering? What would be the best way of doing this?
I'm talking about a Windows executable, but would be interested to hear what different approaches would be needed with another OS.
In linux, step one would be run strings your_file which dumps all the strings of printable characters in the file. Any constants chars will thus be shown, including any "usage" instructions.
Next step could be to run ltrace on the file. This shows all function calls the program does. If it includes getopt (or familiar), then it is a sure sign that it is processing input parameters. In fact, you should be able to see exactly what argument the program is expecting since that is the third parameter to the getopt function.
For Windows, you can see this question about decompiling Windows executables. It should be relatively easy to at least discover the options (what they actually do is a different story).
If it's a .NET executable try using Reflector. This will convert the MSIL code into the equivalent C# code which may make it easier to understand. Unfortunately private and local variable names will be lost, as these are not stored in the MSIL but it should still be possible to follow what's going on.