Error in Makefile calling sed with comment character - sed

I'm trying to build lstrip, a Lua utility for compressing Lua source code. I'm trying to build for Lua 5.1.3 on OS X v10.10.3.
I have downloaded and extracted the Lua 5.1.3 source code and modified the Makefile for lstrip to point to this directory. However, when I run make, I get this output:
cc -I/usr/local/src/lua-5.1.3/src -I/usr/local/src/lua-5.1.3/src -O2 -Wall -Wextra -O2 -c -o lstrip.o lstrip.c
lstrip.c:33:14: warning: unused parameter 'argc' [-Wunused-parameter]
int main(int argc, char* argv[])
^
1 warning generated.
sed '/void luaX_next/i#include "proxy.c"' /usr/local/src/lua-5.1.3/src/llex.c > llex.c
sed: 1: "/void luaX_next/i#inclu ...": command i expects \ followed by text
make: *** [llex.c] Error 1
This is what the relevant Makefile command looks like:
llex.c:
sed '/void luaX_next/i#include "proxy.c"' $(LUASRC)/$# > $#
I think that this is because the # in the sed command is being treated like an actual comment, but I'm not sure.
How can I fix the Makefile, or manually run the steps, to get lstrip built?
Full copy of the Makefile follows, in case it matters:
# makefile for lstrip
# change these to reflect your Lua installation (Lua 5.1!)
LUA= /usr/local/src/lua-5.1.3
LUAINC= $(LUA)/src
LUALIB= $(LUA)/src
LUASRC= $(LUA)/src
# no need to change anything below here
CFLAGS= $(INCS) $(WARN) -O2 $G
WARN= -O2 -Wall -Wextra
INCS= -I$(LUAINC) -I$(LUASRC)
LIBS= -L$(LUALIB) -llua -lm
MYNAME= lstrip
MYLIB= $(MYNAME)
T= $(MYNAME)
OBJS= $(MYNAME).o llex.o
TEST= test.lua
all: test
test: $T
$T $(TEST)
$T: $(OBJS)
$(CC) -o $# $(OBJS) $(LIBS)
llex.c:
sed '/void luaX_next/i#include "proxy.c"' $(LUASRC)/$# > $#
llex.o: proxy.c
clean:
-rm -f $(OBJS) core core.* a.out $(MYNAME)
# eof
Hand-build solution:
cp /usr/local/src/lua-5.1.3/src/llex.c .
Hand-edit llex.c to add the line #include "proxy.c" before the line starting with void luaX_next (line 446 for me).
Now run make, which will succeed.

You can find the answer in the sed manual, and it is in the Makefile lines
llex.c:
sed '/void luaX_next/i#include "proxy.c"' $(LUASRC)/$# > $#
Some variable expansion takes place here:
$(LUASRC) is expanded to the variable set above -> $(LUA)/src. Recurse as needed.
$# is replaced with the current target (llex.c)
So this recipe says:
In order to obtain the target llex.c (which will be processed later through other recipes), apply a stream editing command to the file $LUASRC/llex.c and write it to llex.c.
The stream editing command is: look for text "void luaX_next", before printing it, insert line "#include "proxy.c"".
Problem is, the command to do this is not "i" but "i\(newline)", which conflicts with a requirement of Makefiles that recipes must be on a single line.
I suspect that in order to fix your Makefile you need to use a different command than sed; awk can fit the bill although it's a bit more complex.

This line works in Mac OS X and in Linux:
sed '/void luaX_next/{h;s/.*/#include "proxy.c"/;p;g;}' $(LUASRC)/$# > $#

Related

How to make a Linux Makefile work on Windows?

(Disclaimer: I am not at all a computer science genius, far from that. I may use bad terminology and I appologize.)
I need to run some tests using the google test library and I was provided with a Makefile to handle the execution but it won't run on my Windows machine (I use Visual Studio). It was made for a Linux environment so I'm not sure what i would need to modify in order to run it. I have used MinGW to run Makefiles of my own making in the past.
Here's how the Makefile looks like:
CC=g++
CFLAGS=-c -Wall -ggdb -I.
LDFLAGS=
SOURCES=main.c singlelinklist.c
TESTS=single_test.cpp
#TODO: Need a more elegant way of specifying objects and tests
GTESTDIR=~/environment/googletest
OBJECTS=$(SOURCES:.cpp=.o)
FLAGS = -Iinclude
#all: $(SOURCES) $(EXECUTABLE)
# These next lines do a bit of magic found from http://stackoverflow.com/questions/2394609/makefile-header-dependencies
# Essentially it asks the compiler to read the .cpp files and generate the needed .h dependencies.
# This way if any .h file changes the correct .cpp files will be recompiled
depend: .depend
.depend: $(SOURCES)
rm -f ./.depend
$(CC) $(CFLAGS) -MM $^ >> ./.depend;
include .depend
# End .h file magic
#$(EXECUTABLE): $(OBJECTS)
# $(CC) $(LDFLAGS) $(OBJECTS) -o $#
#.cpp.o:
# $(CC) $(CFLAGS) $< -o $#
clean:
# rm -rf *o $(EXECUTABLE) test_executable
rm -f ./.depend
rm $(GTESTDIR)/libgtest.a
rm $(GTESTDIR)/gtest-all.o
# Google test section
$(GTESTDIR)/libgtest.a:
$(CC) -isystem $(GTESTDIR) -I $(GTESTDIR) -pthread -c $(GTESTDIR)/gtest/gtest-all.cc -o $(GTESTDIR)/gtest-all.o
ar -rv $(GTESTDIR)/libgtest.a $(GTESTDIR)/gtest-all.o
# This will also recompile if any source file is changed.
test_executable: $(GTESTDIR)/libgtest.a $(TESTS) depend
$(CC) -isystem $(GTESTDIR) -pthread -ggdb $(TESTS) $(GTESTDIR)/gtest/gtest_main.cc $(GTESTDIR)/libgtest.a -o test_executable
test: test_executable
./test_executable --gtest_print_time=0
And here's how my workspace is structured.
(Seems like I can't embed pictures).

Why can't I build 'true' pragma module on Strawberry Perl 5.28.1?

I'm rebuilding a Windows 10 (64 bit) workstation from scratch, and I've hit a snag in building one particular CPAN module: true. This is a well-established module— it was last updated in 2011!— and I've used it for years on previous versions of Strawberry Perl, up to and including 5.26. I'm mystified as to what's going wrong.
I build the module as follows:
C:\Strawberry\cpan\build\true-0.18-0> perl Makefile.PL
Checking if your kit is complete...
Looks good
Generating a gmake-style Makefile
Writing Makefile for true
Writing MYMETA.yml and MYMETA.json
C:\Strawberry\cpan\build\true-0.18-0> gmake
cp lib/true/VERSION.pm blib\lib\true\VERSION.pm
cp lib/true.pm blib\lib\true.pm
Running Mkbootstrap for true ()
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command -e chmod -- 644 "true.bs"
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command::MM -e cp_nonempty -- true.bs blib\arch\auto\true\true.bs 644
"C:\Strawberry\perl\bin\perl.exe" "C:\Strawberry\perl\lib\ExtUtils/xsubpp" -typemap C:\STRAWB~1\perl\lib\ExtUtils\typemap true.xs > true.xsc
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command -e mv -- true.xsc true.c
gcc -c -IC:\Strawberry\perl\site\lib\B\Hooks\OP\Annotation\Install -IC:\Strawberry\perl\vendor\lib\B\Hooks\OP\Check\Install -s -O2 -DWIN32 -DWIN64 -DCONSERVATIVE -D__USE_MINGW_ANSI_STDIO -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO -fwrapv -fno-strict-aliasing -mms-bitfields -O3 -Wall -W -DVERSION=\"0.18\" -DXS_VERSION=\"0.18\" "-IC:\STRAWB~1\perl\lib\CORE" true.c
In file included from true.xs:4:0:
true.xs: In function 'true_leave':
C:\STRAWB~1\perl\lib\CORE/perl.h:174:22: warning: unused parameter 'my_perl' [-Wunused-parameter]
# define pTHX tTHX my_perl PERL_UNUSED_DECL
^
true.xs:25:24: note: in expansion of macro 'pTHX'
STATIC void true_leave(pTHX) {
^
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Mksymlists \
-e "Mksymlists('NAME'=>\"true\", 'DLBASE' => 'true', 'DL_FUNCS' => { }, 'FUNCLIST' => [], 'IMPORTS' => { }, 'DL_VARS' => []);"
g++ true.def -o blib\arch\auto\true\true.xs.dll -mdll -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib" true.o "C:\STRAWB~1\perl\lib\CORE\libperl528.a" "C:\Strawberry\perl\site\lib\auto\B\Hooks\OP\Annotation\Annotation.a" "C:\Strawberry\perl\vendor\lib\auto\B\Hooks\OP\Check\Check.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libmoldname.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libkernel32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libuser32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libgdi32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libwinspool.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libcomdlg32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libadvapi32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libshell32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libole32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\liboleaut32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libnetapi32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libuuid.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libws2_32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libmpr.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libwinmm.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libversion.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libodbc32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libodbccp32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libcomctl32.a" -Wl,--enable-auto-image-base
"C:\Strawberry\perl\bin\perl.exe" -MExtUtils::Command -e chmod -- 755 blib\arch\auto\true\true.xs.dll
dlltool --def true.def --output-lib blib\arch\auto\true/true.a --dllname true.xs.dll blib\arch\auto\true\true.xs.dll
I don't know if the warning about pTHX is significant or ignorable. Anyhow, it does generate an output file true.xs.dll:
C:\Strawberry\cpan\build\true-0.18-0> dir blib\arch\auto\true\true.xs.dll
Volume in drive C is Local SSD Disk
Volume Serial Number is B4E3-5132
Directory of C:\Strawberry\cpan\build\true-0.18-0\blib\arch\auto\true
01/23/2019 09:53 17,920 true.xs.dll
1 File(s) 17,920 bytes
0 Dir(s) 873,458,094,080 bytes free
But the file is unusable:
C:\Strawberry\cpan\build\true-0.18-0> perl -I blib\arch\auto -Iblib\arch\auto\true -I blib\lib\auto -I blib\lib\ -Mtrue
Can't load 'blib\arch\auto\true/true.xs.dll' for module true: load_file:The specified module could not be found at C:/Strawberry/perl/lib/DynaLoader.pm line 193.
at - line 0.
Compilation failed in require.
BEGIN failed--compilation aborted.
The documentation on Dynaloader doesn't really help, but the code in Dynaloader.pm near line 193 does provide a hint:
# Many dynamic extension loading problems will appear to come from
# this section of code: XYZ failed at line 123 of DynaLoader.pm.
# Often these errors are actually occurring in the initialisation
# C code of the extension XS file. Perl reports the error as being
# in this perl code simply because this was the last perl code
# it executed.
So my working theory is that “something” changed in the latest Perl, causing code rot in the true module. But there's a flaw in my theory: at least one CPAN tester has successfully built true (although they did it with 32-bit perl).
I'm at a loss. Anyone got any ideas I can try?
I'm not sure whether to post this as an answer or not, but it is at least a resolution. After spending a day and a half of snooping through source code and trying various experiments, I gave up, removed the 64-bit Strawberry 5.28.1, and installed 32-bit Strawberry 5.28.1. Then true built, tested, and installed like a champ.
My only guess (and it's a wild guess) is that some part of the process was somehow finding and interacting with some 32-bit binaries on the network— although I expressly did not have PERL5LIB or other Perl env vars set, and nothing in my PATH pointed to any network Perl stuff.
But I don't have time to investigate this any further, and 32-bit Perl is good enough for what I do. A big thank you to #Corion and #ikegami for their time and comments.

Makefile with multiple outcomes

This is my first attempt to write a makefile and therefore there is a lot of room for improvements. I need to generate several mex functions for matlab on mac using the package sedumi and intel compiler studio.
Here is the makefile
# define matlab dir
MDIR = /Applications/MATLAB_R2017b.app
# compiles mex files using g++
#CC = gcc
# compiler flags for g++
#CCFLAGS = -O3 -fpic
# to use the intel compiler instead, uncomment CC and CCFLAGS below:
# compiles mex file using the intel compiler
CC = icc
# compiler flags for intel compiler
CCFLAGS = -O3 -fPIC -D__amd64
# Figure out which platform we're on
UNAME = $(shell uname -s)
# Linux
ifeq ($(findstring Linux,${UNAME}), Linux)
# define which files to be included
CINCLUDE = -I$(MDIR)/extern/include -Ic++ -shared
# define extension
EXT = mexa64
endif
# Mac OS X
ifeq ($(findstring Darwin,${UNAME}), Darwin)
# define which files to be included
CINCLUDE = -L$(MDIR)/bin/maci64 -Ic++ -shared -lmx -lmex -lmat -lmwblas
# define extension
EXT = mexmaci64
# CCFLAGS += -std=c++11
endif
SRC:=$(wildcard *.c)
*.o: $(SRC)
for i in $(SRC) ; do \
$(CC) $(CCFLAGS) -I$(MDIR)/extern/include -c -Ic++ $$i -o $${i%.c}.o; \
done
OBJ0:=bwblkslv.o sdmauxFill.o sdmauxRdot.o
OBJ1:=choltmpsiz.o
all:
$(CC) $(CCFLAGS) $(CINCLUDE) $(OBJ0) -o bwblkslv.$(EXT)
$(CC) $(CCFLAGS) $(CINCLUDE) $(OBJ1) -o choltmpsiz.$(EXT)
# clean up
clean:
rm -f *.o *.$(EXT)
As it is, Makefile works just fine. make and then make all return the mex functions needed within matlab.
I have OBJ0 to OBJ37 and although adding the lines solves the problem I wonder if there is a simpler way to accomplish the same results.
Many thanks.
Ed
PS. Thanks to the author of https://github.com/jtilly/mex for parts of the makefile.
There are several aspects that you could improve:
Use make automatic variables ($#, $<, $^...)
Use a more make-oriented style for your C compilations, thanks to a make pattern rule:
SRC := $(wildcard *.c)
OBJ := $(patsubst %.c,%.o,$(SRC))
%.o: %.c
$(CC) $(CCFLAGS) -I$(MDIR)/extern/include -c -Ic++ $< -o $#
This has several advantages over you shell loop. The most important one being that only one compilation is run when only one C file changes. With your solution all C files are recompiled when only one changes. Moreover, *.o is a wildcard that expands to existing object files. Not what you want, usually. Note that in your case you could also use a static pattern rule:
$(OBJ): %.o: %.c
$(CC) $(CCFLAGS) -I$(MDIR)/extern/include -c -Ic++ $< -o $#
which looks almost the same but limits the rule to the specified list of targets $(OBJ).
Use a macro and foreach-eval-call to instantiate the linker rules:
.PHONY: all clean
EXTS :=
# $(1): variable name
define MY_rule
$(1)_EXT := $$(patsubst %.o,%.$$(EXT),$$(firstword $$($(1))))
EXTS += $$($(1)_EXT)
$$($(1)_EXT): $$($(1))
$$(CC) $$(CCFLAGS) $$(CINCLUDE) $$^ -o $$#
endef
OBJVARS := OBJ0 OBJ1 ... OBJ37
OBJ0 := bwblkslv.o sdmauxFill.o sdmauxRdot.o
OBJ1 := choltmpsiz.o
$(foreach v,$(OBJVARS),$(eval $(call MY_rule,$(v))))
all: $(EXTS)
clean:
rm -f $(OBJ) $(EXTS)
Note the $$ to escape the first expansion added by eval. For example, the first iteration of foreach instantiates the following rule:
OBJ1_EXT := $(patsubst %.o,%.$(EXT),$(firstword $(OBJ1))
EXTS += $(OBJ1_EXT)
$(OBJ1_EXT): $(OBJ1)
$(CC) $(CCFLAGS) $(CINCLUDE) $^ -o $#
An easy way to design the macro consists in writing first what you want with, for instance, the OBJ1 variable, replace each $ by $$ and finally replace OBJ1 by $(1).

Can someone explain me this recipe?

$(Q)makedepend $(CFLAGS) -o.o -f- $< 2> nul: | sed -e s!$(<:.cpp=.o)!$#! -e s!\\!/!g > $(#:.o=.d)
It is part of following rule in contiki makefile.
CUSTOM_RULE_CPP_TO_OBJECTDIR_O = 1
$(OBJECTDIR)/%.o: %.cpp | $(OBJECTDIR)
$(TRACE_CC)
$(Q)cl -nologo $(VCFLAGS) -c $< -Fo$#
$(Q)makedepend $(CFLAGS) -o.o -f- $< 2> nul: | sed -e s!$(<:.cpp=.o)!$#! -e s!\\!/!g > $(#:.o=.d)
Please let me know if you want more info before downvoting. Thanks
I'm not sure what exactly you want explained. This is a command line which invokes the makedepend program, redirects its stderr to nul: (looks like this expects to be run on a DOS system? But using UNIX tools like sed?) and sends its output to sed. The sed program is converting the .o filename in the makedepend output to the actual target name (including the path), and converting backslashes to forward slashes.
Then the results are written to a .d file. Presumably somewhere in the makefile you'll find a line that uses include to include all the .d files.
Basically, this is a way of auto-generating header file dependencies so that make will rebuild object files when they change, without you having to write them all by hand in the makefile.
ETA
$(Q)
A make variable reference which probably expands to either # or not, to control verbosity of the makefile.
makedepend $(CFLAGS) -o.o -f- $<
Invoke makedepend with the CFLAGS options and write the results to stdout (-f-), reading the source file ($<)
2> nul:
Redirect stderr to nul: which on Windows is a special character device which means "throw it away".
|
Send the output of the previous command to the input of the next command.
sed
Run the sed (stream editor) program, which is a UNIX/POSIX program that manipulates files on a line-by-line basis (usually).
-e s!$(<:.cpp=.o)!$#!
The first manipulation substitutes the plain name of the object file (e.g., if you're compiling foo.cpp then $< is foo.cpp, and $(<:.cpp=.o) replaces the .cpp with a .o so the result is foo.o) with the name of the target: $# will be the full target name like obj/foo.o, if OBJECTDIR is obj.
-e s!\\!/!g
The second substitution replaces all backslashes (escaped from the shell, so \\) with forward slashes.
> $(#:.o=.d)
Write the output of the sed command to the file $# with the .o replaced by .d, so if OBJECTDIR is obj it will write to obj/foo.d

How to write a Makefile rule to download a file only if it is missing?

I'm trying to write a Makefile which should download some sources if and only if they are missing.
Something like:
hello: hello.c
gcc -o hello hello.c
hello.c:
wget -O hello.c http://example.org/hello.c
But of course this causes hello.c to be downloaded every time make command is run. I would like hello.c to be downloaded by this Makefile only if it is missing. Is this possible with GNU make and how to do this if it is?
My guess is that wget doesn't update the timestamp on hello.c, but retains the remote timestamp. This causes make to believe that hello.c is old and attempts to download it again. Try
hello.c:
wget ...
touch $#
EDIT: The -N option to wget will prevent wget from downloading anything unless the remote file is newer (but it'll still check the timestamp of the remote file, of course.)
Since the Makefile should be working as you want, you need to check a few unlikely cases:
1) Check that you don't have any .PHONY rules mentioning the source file.
2) Check that the source target name matches the file path you are downloading.
You could also try running make -d to see why make thinks it needs to 're-build' the source file.
The Makefile you wrote downloads hello.c only if it's missing. Perhaps you are doing something else wrong? See for example:
hello: hello.c
gcc -o hello hello.c
hello.c:
echo 'int main() {}' > hello.c
And:
% make
echo 'int main() {}' > hello.c
gcc -o hello hello.c
% rm hello
% make
gcc -o hello hello.c
% rm hello*
% make
echo 'int main() {}' > hello.c
gcc -o hello hello.c
(the echo command was not executed the second time)
If the prerequisite for hello.c has changed or is empty and Make continues to download the file when it exists, then one option to prevent Make from re-downloading the file is to use a flag in the body of the target to detect if the file exists:
hello.c:
test -f $# || wget -O hello.c http://example.org/hello.c
The test command will return true if the hello.c file exists, otherwise it will return false and the wget command will run.