Running perl script on multiple files using Ant - perl

I have a set of files that I want to run a perl script on. The problem is they are not in the same directory. In fact they are all part of an ordered but complex directory hierarchy.
I wrote a Bash script that does what I want:
if [ ${HOME:+1} ]; then
for A in ${HOME}/a/b/*; do
for B in ${A}/c/*; do
for C in ${B}/*; do
if [ -f $C/somefile ]; then
some_perl_script $C/somefile;
fi
done;
done;
done;
fi
So that works, and it's not too complicated. However, I need that functionality to be available using Ant. Of course I could just call that script from Ant, but I want to know if there is anyway of having Ant do it in the first place, which seems like a more straightforward way of doings things. Essentially, I want something similar to the following build.xml:
<project name="application" basedir=".">
<target name="apply_to_all">
<exec executable="perl" dir="${basedir}">
<arg value="some_perl_script.pl"/>
<arg value="a/b/**/c/**/**/somefile"/>
</exec>
</target>
</project>
I found this SO post. However, I'd like to do it without ant-contrib.

Use the apply task to invoke your perlscript on all files that are included in the nested fileset(s) :
<apply executable="perl">
<arg value="-your args.."/>
<fileset dir="/some/dir">
<patternset>
<include name="**/*.pl"/>
</patternset>
</fileset>
<fileset refid="other.files"/>
</apply>
Optionally use the attribute parallel = true, means run the command only once, appending all files as arguments. If false, command will be executed once for every file, default =false => see Ant manual on apply
You may use include / exclude patterns and one or more filesets. F.e. you may use **/*.pl for
all your perl scripts.

Related

Conditional <exec> based on results of <copy>

In our nant build script for our web-based application, we <copy> a set of files to a target directory and then run aspnet_compiler over them via <exec>.
<copy> only copies files that have changed, however here is no way to pass this information to <exec>, and I want to avoid running aspnet_compiler when nothing has actually changed.
Options I've tried to find are: <copy> setting a property when any file is copied that can then be checked with <if>; or being able to create a file before the copy and doing something like <if test="any-file-newer-than(targetdir, timestampfile)">. Even better would be if <copy> could return a list of copied files that I can then iterate over to avoid having to process the entire tree, but I think that might be asking a bit too much.
So far, I've drawn a blank: is what I'm looking for possible without writing a custom extension?
Why don't you just simply replace copy task with robocopy? (you're on Windows, right?)
Robocopy returns different exit codes on different successful copy situations:
https://ss64.com/nt/robocopy-exit.html
For example:
0 - ok, nothing copied
1 - ok, something copied
You could do something like this:
<exec program="robocopy.exe" commandline="${SourceDir} ${DestDir}" failonerror="false" resultproperty="ExitCode" />
<fail unless="${ExitCode < 8}" message="Failed to copy"/> <!-- Anything between 0..7 is OK for robocopy -->
<exec unless="${ExitCode == 0}" ...

Simple way to enumerate the files in a fileset in phing

I am new to phing and trying to verify if my build.xml works as expected. I am looking for a convenient way to enumerate the files in a phing fileset.
The only thing that I've been able to get working is foreach (like in how to iterate (loop) through directories in phing?). However, it feels way too complex: I have to create a subtask, and phing gets called once for every file, making the outupt list hard to parse visually.
Any better alternatives? Thanks!
With Phing 2.4.8, the <echo> task supports filesets:
http://www.phing.info/trac/ticket/792
There is currently no better way. You could grep the output, though :)
<task name="dummy">
<foreach param="filename" absparam="absfilename" target="echoFilesetFile">
<fileset refid="co"/>
</foreach>
</task>
<target name="echoFilesetFile">
<echo>file: rel:${filename}|abs:${absfilename}</echo>
</target>
then $ phing dummy | grep 'file:'

Nant, SqlCmd -v switch and spaces in nant property fails build with invalid argument

I have a nant script that ...
1. takes the content of disc-file
2. assigns that content to a nant property
3. and then calls sqlcmd with a -v passing in that property containg the content of the disc file
4. inside the sql script the contents of the file should be used by a stored proc.
The problem is that when the content of the file contains a space the nant build stops with a "Invalid argument" issue
Anone know a way around this ?
The top part of the nant script is ...
<?xml version="1.0"?>
<!-- the main name of this project -->
<project name="Hops" default="all">
<!-- BuildHistory -->
<property name="buildHistoryContents" value="" />
<xmlpeek xpath="/" file="BuildNotes.xml" property="buildHistoryContents"></xmlpeek>
<!-- <echo message="${buildHistoryContents}" /> -->
<!-- ***************** -->
<target name="ExecSql">
<echo message="running sql script : ${SqlBuildScriptsDir}${sqlBuildFileName}" />
<exec program="${SqlCmd}" commandline="-S ${SqlServerInstanceName} -E -d HBus -i ${SqlBuildScriptsDir}${sqlBuildFileName} -v vSchemaVersion=${buildHistoryContents} " />
</target>
The sql script contains the line ...
exec lsp_SchemaVersionUpsert '1.4', N'$(vSchemaVersion)'
A disc file content that works is ...
<BuildNotes>
<Note>
<buildVer>HasNotSpace</buildVer>
</Note>
</BuildNotes>
A disc file content that does not works is ...
<BuildNotes>
<Note>
<buildVer>Has Space</buildVer>
</Note>
</BuildNotes>
The use of all this is pass xml build comments to a table logging version build history for the db schema.
Does anyone know an alternate method or know a way through this ?
The next part, added after Phillip Keeley correcty solved first part (the SPACE Problem)
I simplified the original task to simplify the question.
There is also a Quoted Attribute Problem ; xml quoted attributes cause the nant build to fail with "Invalid Argument".
eg this will cause nant to choke but removing the dt attribute will enable the nant build to succeed ...
<BuildNotes>
<Note>
<buildVer>1.4</buildVer>
<dateStarted>09/24/2009 11:25:42</dateStarted>
<Item dt="20091008" >SpacesAndNoQuotedAttribute</Item>
</Note>
</BuildNotes>
Any ideas ... ?
Your problem is (of course) in line
<exec program="${SqlCmd}" commandline="-S ${SqlServerInstanceName} -E -d HBus -i ${SqlBuildScriptsDir}${sqlBuildFileName} -v vSchemaVersion=${buildHistoryContents} " />
specifically, in
-v vSchemaVersion=${buildHistoryContents}
The NAnt expression replaces property ${buildHistoryContents} with the stored value--which will include any embedded spaces. Problem is, when calling SQLCMD (I"m assuming that's what ${SqlCmd} resolves to) from a command window, the values for any and all -v parameters are space delimited -- that is, the parser hits -v, reads the next characters through the "=" as the variable name, then reads all characters after the = and through the next space (or end of line) as the value to assign to the variable, and that embedded space will mess you up bigtime.
On the command line, the work-around is to wrap the variable value in quotes:
- v MyVariable=Hello World
becomes
- v MyVariable="Hello World"
That doesn't work here, because it's XML and you have to wrap the commandline attribute of the exec element with quotes... and embedded quotes will, once again, mess you up bigtime.
I believe the work-around here is to use XML macro substitution (I quite possibly have the formal titles of these concepts wrong) for those embedded quotes. This value should be
"
Which means that the following should work:
<exec program="${SqlCmd}" commandline="-S ${SqlServerInstanceName} -E -d HBus -i ${SqlBuildScriptsDir}${sqlBuildFileName} -v vSchemaVersion="${buildHistoryContents}" " />
Please try this and see -- I may have to do something like this myself some day soon.

Nant cmd.exe redirection creating file called 'program' on c:\ drive

I have NAnt script which as part of its project calls a batch file using the following task:
<target name="makeplane">
<exec program="C:\WINDOWS\system32\CMD.EXE"
commandline="/C ${make.file} > ${make.log}"
verbose="false"
workingdir="${make.dir}"
basedir="${make.dir}">
</exec>
<delete>
<fileset basedir="c:\">
<include name="program" />
</fileset>
</delete>
</target>
Unfortunately i dont have control over the contents on the batch file and it spews out a lot of garbage onto the screen which is of no use in the log. So to get around this im redirecting the output from the bat file to a text file using the
> ${make.log}
part which equates to "> log.txt".
This redirection seems to create a file called "program" on the C drive and messes up all sorts of services and windows generally doesnt like it. To get around this Im manually deleting this file after the bat file has executed.
The problem is i now need to run a similar task for another project entirely and if they run at the same time then the first will lock the file called "program" and the second will fail. Not exactly a great situation for Continuous integration.
I searched on the net but because the file is called program i get all sorts of rubbish results. Anyone got any ideas on a work around. I tried the output parameter on the exec task but the issue remains the same.
If the file path to the log contains spaces, one generally would want to surround the path in quotes. In order to do this in nant one can use the " entity.
It sounds like this is what's happening in your particular situation. Therefore, if you change your example to the following I think things should work as expected.
<target name="makeplane">
<exec program="C:\WINDOWS\system32\CMD.EXE"
commandline="/C ${make.file} > "${make.log}""
verbose="false"
workingdir="${make.dir}"
basedir="${make.dir}">
</exec>
</target>
Usually this happens because the script is trying to create a file with a long file name with space in it (c:\program files in your case), but it is not using quotes around the long file name.
Here is what I did. I think it is a bit cleaner for complex commands.
<property name="cmd.label" value="\${ss.previous.label}#$Project.SSPath" />
<echo message="Getting $Project.Name source code with label \${cmd.label}" />
<property name="cmd" value=""\${tfs.root}\tf.exe" get $Project.SSPath "/version:L\${cmd.label}" /force /recursive /noprompt"/>
<exec program="cmd.exe"
workingdir="\${shadow.dir}"
failonerror="true"
verbose="true">
<arg value="/c" />
<arg value=""\${cmd}"" />
<arg value="> nul" />
</exec>

Nant : change file permission

I have an ASP.NET application.
Basically the delivery process is this one :
Nant builds the application and creates a zip file on the developer's computer with the application files without SVN folders and useless files. This file is delivered with a Nant script.
The zip and nant files are copied to the client's computer
the Nant script replaces the current website files with the file contained in the zip file.
My problem is that with this process I have an Unauthorized access error when I try to open the website.
It seems that the files need to have a permission set for the user "IIS_WPG".
I don't have the power to change IIS configuration so I have to manually change the permissions of each file. And each time I replace the files the permissions are removed and I need to set them again.
So I have two questions :
Can I change files permissions with Nant ? How to do it ?
Is it possible to avoid this problem ? (developers don't have this user on their computers)
#Jeff Fritz
Ouch...
Your suggestion is the right solution but the parameters are... dangerous :).
On dev computers I'm logged as administrator and I tried your suggestion with cmd.
It replaces all the permissions set in order to set only the ones defined in the command (so, after the command, accessing files resulted in a "Access denied" even with my admin user)
It applied on the C:\WINDOWS\ directory, while I called the command from the wwwroot folder. :)
So, after some tests, the right command is :
cacls [full folder path] /T /E /G IIS_WPG:F
/T : applies on specified folder and subfolders
/E : edits the ACL instead of replacing it :)
You need to run the CACLS program in windows to grant permissions to files and folders. From Nant, you can do this with the EXEC task.
Try a tag block like:
<exec program="cacls">
<arg value="*" />
<arg value="/G IIS_WPG:F" />
</exec>
We ended up writing our own task for this with some fairly straight forward code:
[TaskName("addusertodir")]
public class AddUserToDirectorySecurity : Task
{
[TaskAttribute("dir", Required=true)]
public string DirPath { get; set; }
[TaskAttribute("user", Required=true)]
public string UserName { get; set; }
protected override void ExecuteTask()
{
FileSystemAccessRule theRule1 = new FileSystemAccessRule(UserName, FileSystemRights.ListDirectory, AccessControlType.Allow);
FileSystemAccessRule theRule2 = new FileSystemAccessRule(UserName, FileSystemRights.ReadAndExecute, AccessControlType.Allow);
FileSystemAccessRule theRule3 = new FileSystemAccessRule(UserName, FileSystemRights.Read, AccessControlType.Allow);
DirectorySecurity theDirSecurity = new DirectorySecurity();
theDirSecurity.AddAccessRule(theRule1);
theDirSecurity.AddAccessRule(theRule2);
theDirSecurity.AddAccessRule(theRule3);
Directory.SetAccessControl(DirPath, theDirSecurity);
}
}
Then you can write a nant script that loads the custom task and executes:
<loadtasks>
<fileset>
<include name="MyTask.dll"/>
</fileset>
</loadtasks>
<addusertodir dir="MyDir" user="IIS_WPG"/>
Obviously, this could be modified for your certain rules or you could even parameterize this in the task if you so wish. We preferred this over the using the exec task as it have us a bit more control over permissions that were being applied.
CACLS is now deprecated. Here's a version that uses ICACLS, the replacement.
Let's say we have the following:
The root folder of our installation is "c:\inetpub\wwwroot", and it's stored in the NANT variable ${paths.myprogram.inetpub}
The folder we want to modify is called "uploads", and it's stored in ${upload.foldername}
The user we want to grant access to is "IIS_UPLOAD_USER", stored in ${iis.upload.user}
The permission level we want to grant is "M", for "modify" permissions, stored in ${iis.user.permissionlevel}
With these assumptions, our task is this:
<exec program="icacls">
<arg value="${path::combine(paths.myprogram.inetpub, upload.foldername)}" />
<arg value="/grant" />
<arg value="${iis.upload.user}:${iis.user.permissionlevel}" />
</exec>
Hope this helps!