I need to do some kind of modification in post-synthesis hardware description language (HDL) files of field-progammable gate array (FPGA) design. I am using Microsemi's Libero system on chip (SoC) tool for that design process.
Totally, it has three different design processes such as
Pre-modification process
Modification process
Post-modification process on HDL codes
All three steps need to be performed in sequence using the Windows command line. I wrote a Tcl script for the first and third processes and a Perl script for the second process. In the first and third processes, some FPGA design tools for synthesis, simulation, place and route will be called through the command line.
This is the overall idea of the work. My question is how can I run all three different scripts (Tcl, Perl and Tcl) in sequence using a single script (or any other file format) in command line?
Note: In few steps, it is required to get user data through STDIN. I am working in windows platform..
Create a batch file (extension .bat) something like:
start /b tcl script1
start /b perl script1
start /b tcl script2
I'm not familar with Tcl but an example using Perl would be (from the DOS command-line):
perl hi.pl > log.txt & perl mid.pl >> log.txt & perl low.pl >> log.txt
The 3 perl script run in succession with the output of all 3 scripts put in a file called log.txt.
You can incorporate STDIN as needed.
Related
In olden days, I remember a trick we used to use to run Perl scripts within Windows cmd.exe as a simple invocation of a cmd file rather than having to run perl.exe with the script name, something like:
#rem = '
#perl -x -S %0 %*
#goto :eof
#rem ';
<insert Perl script here>
This used the rather neat trick of exploiting differences in how cmd.exe and perl.exe would handle the input script. Windows' cmd.exe would read the first three lines as three separate commands which would:
have a comment with no echo;
run perl with the same input file and arguments, without echo; and
goto the end of the file (exit) with no echo.
On the other hand, perl.exe would treat the first four lines as an assignment statement, then go on to execute the Perl script proper.
Now it's often a pain to run Powershell scripts from the cmd.exe command line since you have to use something like:
powershell -file go.ps1
I'm wondering if there's a way to encode both batch and powershell commands into a single cmd file, similar to the Perl trick, in such a way that it starts running under cmd.exe but switches to Powershell quickly after that.
If that were possible, you could run your script go.cmd simply by entering:
go
at the command line, rather than some convoluted invocation of powershell.
I know that you can ship separate cmd and ps1 files but that's in fact what I'm trying to get away from. I'm looking for a single file solution if possible.
I also know that you can base-64 encode your script or execute it as a string, provided you replace all newlines with semicolons. But that means the Powershell stuff is no longer easily editable or readable in the resulting file.
Fortunately or otherwise, Powershell knows exactly what to do with .cmd files: use CMD to run them.
#set $a=%0 && powershell -encodedcommand ZwBjACAAKAAoAGQAaQByACAAZQBuAHYAOgBgACQAYQApAC4AdgBhAGwAdQBlACAALQByAGUAcABsAGEAYwBlACAAJwAiACcALAAnACcAKQAgAHwAIABzAGUAbABlAGMAdAAgAC0AcwBrAGkAcAAgADIAIAB8ACAAcABvAHcAZQByAHMAaABlAGwAbAAgAC0A
#exit /b
"Hello from Powershell! You know it's me because cmd would never know that 2 + 2 = $(2 + 2)!"
The encoded command is
gc ((dir env:`$a).value -replace '\"','') | select -skip 2 | powershell -
Which fetches the contents of the script invoking itself, skips the stuff intended for CMD and, yes, runs Powershell on the rest.
Since we're piping commands through stdin and not having them in a proper script file there may be weirdness for more complicated scripts, doubly so if you start nesting these hybrids. I wouldn't trust this in production, but hey, it's something.
I have a series of perl scripts that I want to run one after another on a unix system. What type of file would this be / could I reference it as in documentation? BASH, BATCH, Shell Script File?
Any help would be appreciated.
Simply put the commands you would use to run them manually in a file (say, perlScripts.sh):
#!/bin/sh
perl script1.pl
perl script2.pl
perl script3.pl
Then from the command line:
$ sh perlScripts.sh
Consider using Perl itself to run all of the scripts. If the scripts don't take command line arguments, you can simply use:
do 'script1.pl';
do 'script2.pl';
etc.
do 'file_name' basically copies the file's code into the current script and executes it. It gives each file its own scope, however, so variables won't clash.
This approach is more efficient, because it starts only one instance of the Perl interpreter. It will also avoid repeated loading of modules.
If you do need to pass arguments or capture the output, you can still do it in a Perl file with backquotes or system:
my $output = `script3.pl file1.txt`; #If the output is needed.
system("script3.pl","file1.txt"); #If the output is not needed.
This is similar to using a shell script. However, it is cross-platform compatible. It means your scripts only rely on Perl being present, and no other external programs. And it allows you to easily add functionality to the calling script.
On OpenVMS, it is possible to write DCL (DIGITAL Command Language) command scripts that interpret lines without the $ prompt as input to the preceding command.
For example, let's assume that we have a simple application ADD.EXE that asks for input to two questions, "Enter first value:" and "Enter second value:", and then displays the sum of these two values. Then in OpenVMS DCL it would be possible to write a command script ADD.COM like this:
$ RUN ADD.EXE
5
7
When this command script is executed (by typing #ADD.COM if I remember correctly), the output would be
12
I have tried to find a way to do the same using Windows batch scripts, but so far without success. Can it be done using batch scripts, or is there any alternative approach of accomplishing this under Windows?
There is no direct replacement of this OpenVMS feature, but the work-around is very simple:
(
echo 5
echo 7
) | add.exe
This generate a temporary file with two lines and pipe it to the input of ADD.EXE
How would I determine what script, program, or shell executed my Perl script?
Example: I might want to have human readable output if executed from shell (customized for each type of shell), a different type of output if called as a script from another perl script, and a machine readable format if executed from a program such as a continuous integration server.
Motivation: I have a tool that changes its output based on which shell executes it. I'd normally implement this behavior as an option to the script, but this tool's design doesn't allow for options. Other shells have environment variables that indicate what shell is running. I'm working on a patch to support Powershell, which has no such special variable.
Edit: Many of these answers happen to be linux specific. Unfortuantely, Powershell is for Windows. getppid, the $ENV{SHELL} variable, and shelling out to ps won't help in this case. This script needs to run cross-platform.
You use getppid(). Take this snippet in child.pl:
my $ppid = getppid();
system("ps --no-headers $ppid");
If you run it from the command line, system will show bash or similar (among other things). Execute it with system("perl child.pl"); in another script, e.g. parent.pl, and you will see that perl parent.pl executed it.
To capture just the name of the process with arguments (thanks to ikegami for the correct ps syntax):
my $ppid = getppid();
my $ps = `ps --no-headers -o cmd $ppid`;
chomp $ps;
EDIT: An alternative to this approach, might be to create soft links to your script, make the different contexts use different links to access your script and inspect $0 to build logic around that.
I would suggest a different approach to accomplish your goal. Instead of guessing at the context, make it more explicit. Each use case is wholly separate, so have three different interfaces.
A function which can be called inside a Perl program. This would likely return a Perl data structure. This is far easier, faster and more reliable than parsing script output. It would also serve as the basis for the scripts.
A script which outputs for the current shell. It can look at $ENV{SHELL} to discover what shell is running. For bonus points, provide a switch to explicitly override.
A script which can be called inside a non-Perl program, such as your continuous integration server, and issue machine readable output. XML and/or JSON or whatever.
2 and 3 would be just thin wrappers to format the data coming out of 1.
Each is tailored to fit its specific need. Each will work without heuristics. Each will be far simpler than trying to guess the context and what the user wants.
If you can't separate 2 and 3, have the continuous integration server set an environment variable and look for it.
Depending on your environment, you may be able to pick it up from the environment variables. Consider the following code:
/usr/bin/perl -MData::Dumper -e 'print Dumper(\%ENV);' | grep sh
On my Ubuntu system, it gets me:
'SHELL' => '/bin/bash',
So I guess that says I'm running perl from a bash shell. If you use something else, the SHELL variable may give you a hint.
But let's say you know you're in bash, but perl is run from a subshell. Then try:
/bin/sh -c "/usr/bin/perl -MData::Dumper -e 'print Dumper(\%ENV);'" | grep sh
You will find:
'_' => '/bin/sh',
'SHELL' => '/bin/bash',
So the shell is still bash, but bash has a variable $_ which also show the absolute filename of the shell or script being executed, which may also give a valuable hint. Similarily, for other environments there will most probably be clues left in the perl %ENV hash that should give you valuable hints.
If you're running PowerShell 2.0 or above (most likely), you can infer the shell as a parent process by examining the environment variable %psmodulepath%. By default, it points to the system modules under %windir%\system32\windowspowershell\v1.0\modules; this is what you would see if you examine the variable from cmd.exe.
However, when PowerShell starts up, it prepends the user's default module search path to this environment variable which looks like: %userprofile%\documents\windowspowershell\modules. This is inherited by child processes. So, your logic would be to test if %psmodulepath% starts with %userprofile% to detect powershell 2.0 or higher. This won't work in PowerShell 1.0 because it does not support modules.
This is on Windows XP with PowerShell v2.0, so take it with a grain of salt.
In a cmd.exe shell, I get:
PSModulePath=C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules\
whereas in the PowerShell console window, I get:
PSModulePath=E:\Home\user\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsP
owerShell\v1.0\Modules\
where E:\Home\user is where my "My Documents" folder is. So, one heuristic may be to check if PSModulePath contains a user dependent path.
In addition, in a console window, I get:
!::=::\
in the environment. From the PowerShell ISE, I get:
!::=::\
!C:=C:\Documents and Settings\user
We're starting Matlab from our Jenkins buildserver. As the build may take some time it would be nice to get some log-outputs while matlab is running. Is there a way to print text to standard output? disp, fprintf and java.lang.System.out.printline only write to the matlab console, not to standard output.
Using a logfile or a pipe won't help, as Jenkins only reads from standard-output during a build step.
How can we write log-statements to the standard output while matlab is running?
EDIT:
We're running Matlab 2010b on Windows
Depending what you are doing with Matlab you could probably launch it in command line without GUI. I used this on a server and it behaves pretty much like a shell script and writes to standards outputs.
See the startup options.
I used the following:
/path/to/matlab -nojvm -nodisplay -nosplash -nodesktop -r /path/to/mfile
EDIT: forgot to mention one very important little detail, place an exit command at the end of your mfile or Matlab will hang there waiting.
It seems that the combination of -wait and -log (not -logfile) clones the command window output to the parent console's stdout, but only if you call the MATLAB executable in [MATLABROOT]\bin, not [MATLABROOT]\bin\win64 (the subdirectory for current arch).
Tested on Windows with R2015b and R2016b:
C:\MATLAB\bin\matlab.exe -wait -log
NOT
C:\MATLAB\win64\bin\matlab.exe -wait -log
Remember to put an exit/quit in your script if you are running with -r.
The only trouble is that I can't seem to find any documentation for the -log option! Meh.
There don't seem to be any good ways to do this from within MATLAB. The easiest way I can think of doing this is by using a shell script. You could write a small shell script which would simply print any input to stdout, and then call that shell script from within matlab using the unix (or system) commands. Jenkins should be able to read the command-line output of the script and work with that.
I figured out a way to do this and am also doing it for Jenkins Matlab interface on windows.
Basic idea is that you will use diary command, but then tail -f the file, but you need a smart way to kill the tail command if you open multiple matlab instances because there will be name collisions. So the method I'm using is to name the file log.txt where the PID used is MATLAB's PID it is using when it opens.
There is an undocumented feature in MATLAB that allows you to get its PID. So now, both your batch file and MATLAB know the PID without having to read/write to a random text file that will get messy when executing multiple jobs. So the PID you use that as your unique identifier. The PID of "tail -f" is also used by MATLAB to kill tail -f to make the batch file die and is found by MATLAB using the commandline details associated with the process invocation since it uses again the unique PID log file name.
This uses some wmic commands and needs Windows Vista/7 or above. With XP you probably have to work harder to get the process ID's but should be still possible.
Here is what to do:
1) Get gnu awk for windows: http://gnuwin32.sourceforge.net/packages/gawk.htm
2) Get tail.exe from windows resource kit: http://www.microsoft.com/en-us/download/details.aspx?id=17657
3) Make sure tail and awk are in your path (the windows resourece kit I don't think automatically puts them in the path)
3) Create a batch file called matlabrun.bat as follows, (note: you need the #echo off, also the entire command is quite long, scroll right..)
#echo off
wmic process call create "c:\matlab\bin\win64\matlab.exe -r \"cd('c:\jenkins\workspace\test'); workdir=pwd; outpath=[pwd '\output'] ; try; run('C:\MATLAB\work\test_run'); end; quit; \" " | findstr ProcessId | awk "{print $3}" | awk -F";" "{ print $1 }"
4) Create another batch file called run.bat with:
for /f %%i in ('matlabrun.bat') do (
echo MATLAB Log... > log%%i.txt
tail -f log%%i.txt
set logfilename=log%%i.txt
goto next
)
:next
del /f %logfilename%
5) The run.bat file will execute matlabrun.bat and since -wait is not passed, matlab will immediately return to the command line and execute the tail -f command. That will block the batch file from completing until you kill it. matlabrun.bat returns the PID of matlab.
6) Another important note: since you are using "wmic process create" which will provide you with a PID that MATLAB is using, but will default to a working directory of c:\windows\system32. So that is why I pass the work directory to matlab. wmic process create is also a bit particular about what parameters you put into your command string for matlab to run. So it appears to have a problem with using commas in your command string. So I suggest not to use those, or figure out how to escape them (it might be that ^, works, but I just removed my commas anyway in my matlab run command).
6) The "test_run.m" file contains the following code to write to the correct log file and to kill correct tail -f instance.
matlabpid=feature('getpid');
filename=['log',num2str(matlabpid),'.txt'];
filenamefull=[workdir,'\',filename];
diary(filenamefull);
disp('Script starting...')
%%% put your code here %%%
disp('Script completed...');
diary off;
%%% FIND PID of tail.exe and kill it
%%% by using the name of the log file in the process command line
[a,b]=dos(['wmic process get Commandline,ProcessId']);
C=textscan(b,'%s','delimiter','\n');C=C{1};
for jj=1:size(C,1),
if strfind(C{jj},filename),
D=textscan(C{jj},'%s');D=D{1};
dos(['taskkill /f /pid ',D{4}]) %kills tail.exe which is the log watcher
break
end
end
7) You start it by doing run.bat. It will go and execute matlab, then start tailing the output while MATLAB runs in real-time. Then when done it will delete the log file.
8) My directory structure / files are in these locations (I'm using win7 64bit):
c:\jenkins\workspace\test\tail.exe
c:\jenkins\workspace\test\awk.exe
c:\jenkins\workspace\test\matlabrun.bat
c:\jenkins\workspace\test\run.bat
c:\matlab\work\test_run.m
c:\matlab\bin\win64\matlab.exe
If you are using 32bit matlab, point it to the win32 directory. To get the correct PID, you need to specify the actualy matlab.exe binary in the win32 or win64 directory.
You can do this by pointing the -logfile option to the Jenkins log file. Something like the following:
"C:\path\to\matlab.exe" "-r" "functionToRun" "-logfile" "%JENKINS_HOME%\jobs\%JOB_NAME%\builds\%BUILD_NUMBER%\log" /wait
You can use the diary mode. Not sure if it will fit your specific implementation.
http://www.mathworks.com/help/techdoc/ref/diary.html
I didn't find a real solution. Mathworks created some wrapper tool. But this will only output the results after matlab has exited. You won't get any ouput during execution.
http://www.mathworks.de/support/solutions/en/data/1-ACT3YN/index.html?product=ML&solution=1-ACT3YN
So I'll have live without real live-output...
Or try using '-logfile' option in matlab.
matlab.exe -nodisplay -nosplash -nodesktop -wait -logfile logfile.txt -r "try script.m ;catch err; disp(err.message); end ; exit"
I prefer using bash (Execute shell) in Jenkins, then you can tail the log-file while matlab is running.
matlab.exe <...> &
matpid=$!
tail -f logfile.txt &
tailpid=$!
wait $matpid
matexit=$?
kill $tailpid
sleep 1 # Just to make sure kill is done before Jenkins step ends and no zombie processes
exit $matexit