I'm trying to execute sh-file from resources.
Executed file is located at the root of resources: src/main/resources/hiveCommand.sh
import sys.process._
"./hiveCommand.sh" !!
But receive IOException: not such file or directory
What am I doing wrong?
Scala's Process integration does not know how to handle shell scripts. It can only start programs. To run a shell script you need start a shell (e.g. bash) and give it the file to run as an argument.
There is a complication however. Since the script is a resource (located at src/main/resources/hiveCommand.sh during compile time), it is located in a jar at run time.
So in short:
First extract the shell script (use getClass.getResourceAsStream("/hiveCommand.sh") to read the resource) and store it on disk.
Then start with something like:
"bash /tmp/hiveCommand.sh".!!
Related
I am not understanding how to properly run a simple test(feature file and python file)
with the library pytest-bdd.
From the official documentation, I can't understand what command to issue to run a test.
I tried using pytest command, but I saw the NO test ran.
Do I need to use another library behave to run a feature file?
I figured out trying for 2 days,that ,
for running a pytest-bdd test, there are certain requirements, at least in my view.
put both the feature file and python file in the same directory (maybe this can be changed with configuration files)
the python file name needs to start with test_
the python file needs to contain a method of which name will start with test_
the method starting with test_ , need to be assigned to the #scenario sentence
to run the test, issue pytest command in the same directory(maybe it is also configurable)
After issuing you will only see the method with the name starting with test_ has passed, but all the tests actually ran. To test, you can assert False in any #when or #then annotated method, it will throw errors.
The system contained : pytest-bdd==3.0.2 (copied from pip freeze output)
Features files and python files can be placed in different folders using the bdd_features_base_dir hook provided by pytest-bdd; I think it is better having features files in different folders too.
Here you can see a working example (a simple hello world BDD test):
https://github.com/davidemoro/pytest-play-docker/tree/master/tests
https://github.com/davidemoro/pytest-play-docker/blob/master/tests/pytest.ini (see bdd_features_base_dir in [pytest] section)
https://github.com/davidemoro/pytest-play-docker/tree/master/tests/bdd
If you want to try out pytest-bdd without installation you can use Docker. Create a folder with inside your pytest BDD files and if you want a separate features folder targeted in bdd_features_base_dir and run:
docker run --rm -it -v $(pwd):/src davidemoro/pytest-play:latest
I've found out, that in the python file you don't have to put:
the method starting with test_ , need to be assigned to the #scenario sentence
You can just add: scenarios("") - to allow the tests to be started, which are using steps defined in this specific python file.
Remember to import scenarios!: from pytest_bdd import scenarios
Example:
Code example
Command..
pytest -v path_to_test_file.py
Things to note here..
Check format of feature file as filename.feature
Always __init__ modules, otherwise test-runner will not find test files
Glue right step definitions to test function
Add feature in features module
If you are using python3 execute test with python3
So,
python3 -m pytest -v path_to_test_file.py
Documentation
https://pytest-bdd.readthedocs.io/en/stable/#
Let's consider the following perl script:
#!/usr/bin/perl
system("C:/Program Files (x86)/Microsoft Visual Studio/2017/Enterprise/Common7/Tools/VsDevCmd.bat");
system("msbuild");
The batch file invoked with the first system call is supposed to set up some environment variables so that the msbuild executable in the second system call can be found.
When I run this perl script I get the following error:
'msbuild' is not recognized as an internal or external command,
operable program or batch file.
So it looks like the environment variables set in the batch file are not made available to the context of the perl script. What can I do to make this work?
Note 1
Running first the batch file from a console window and then running msbuild works fine. So the batch file works as expected and msbuild is actually available.
Note 2
My real-world perl script is much longer. This example here is a massive simplification which allows to reproduce the problem. So I cannot easily replace the perl script with a batch file, for example.
Note 3
The funny thing is: I've been using this perl script for one or two years without any problems. Then suddenly it stopped working.
Your process has an associated environment which contains things like the search path.
When a sub-process starts, the new process has a new, separate, environment which starts as a copy of the parent process's environment.
Any process (including sub-processes) can change their own environment. They cannot, however, change their parent's process's environment.
Running system() creates a new environment.
So when you call system() to set up your environment, it starts a new sub-process with a new environment. Your batch program then changes this new environment. But then the sub-process exits and its environment ceases to exist - taking all of the changes with it.
You need to run the batch file in a parent process, before running your Perl program.
I have abc.jar file to deploy and run in remote machine.
I have transferred the file using jenkins, now what I have done is, call a a.bat batch file on remote machine using psexec in Execute Windows Batch Command.
a.bat executes the abc.jar
When the jar begins execution, the command prompt texts are stored in a file.
using java -jar abc.jar >> a.log 2>&1
Now what I want is to display the a.log contents in the jenkins console when the jar file is being executed
(the file is continuously being written and I want to show it in jenkins console as it is being written)
I have tried to do it using parallel processing by calling start twice, one for calling batch file, another using type for displaying.
But when I use start I get Process leaked file descriptor .
Is there any other way I can achieve this. Be it calling powershell or scheduled task in jenkins.
You need to look for tee equivalents in windows , there are few like GNU utilities for Win32, however if you have cygwin you can still use tee which will easy the prcoess.
Now the wuestion arises how to run my jar file on cygwin from jenkins ?
you can still use execute windows[batch] shell. and add cygwin installation path to the PATH variable and start using linux command like a BOSS.
or you can use powershell tee in built command from batch.
How to run the job within springbatchadmin.war from command line?
I used the following command in command prompt to run the job.
D:\apache-tomcat-6.0.35\webapps\springbatchadmin>java -classpath "lib\*;src" com.companyname.batch.BatchLauncher job1Cfg job1
But I got the error and I cannot run the job.
Please give me right directions.
If you are using tomcat for deploying the admin, then the .war file should have deployed itself into a folder as soon as you start the server once. Now you can trace to the location of the xml files inside that folder, and execute jobs from command-line.
eg: in my case, the xml files are in the folder,
tomcat\webapps\my_project_war\WEB-INF\classes\springbatch
So just in case you want to run a job (say 'abcJob') inside an xml, say 'xyz.xml',
use the command,
java org.springframework.batch.core.launch.support.CommandLineJobRunner "tomcat\webapps\my_project_war\WEB-INF\classes\springbatch\xyz.xml" "abcJob" parameter1=value1 etc
While running from command-line, you can also set the classpath variable to some constant location, so that you can run all relative jobs with reference to that location. (in my case it is'tomcat\webapps\my_project_war\WEB-INF\classes')
So I would use the command,
java org.springframework.batch.core.launch.support.CommandLineJobRunner "classpath*:springbatch/xyz.xml" "abcJob" parameter1=value1
I'm executing this code:
p = subprocess.Popen(['/path/to/my/script.sh','--flag'] , stdin=subprocess.PIPE)
p.communicate(input='Y')
p.wait()
It works when executing it on the shell using "python scriptName.py",
BUT when executing using PyDev in Eclipse, it fails, the reason:
/path/to/my/script.sh: line 111: service: command not found
This bash script "script.sh" contains the following command which causes the error:
service mysqld restart
So "service" is not recognized when running the .sh script from the context of PyDev.
I guess it has to do with some ENV VAR configurations, couldn't find how to do it.
BTW - Using "shell=True" when calling subprocess.Popen didn't solve it.
service usually is located in /usr/sbin, and that this directory isn't on the PATH. As this usually contains administrative binaries and scripts which arn't designed to be run by everyone (only by admins/root), the sbin directories arn't always added to the PATH by default.
To check this, try to print PATH in your script (or add an env command).
To fix it, you could either
set the PATH in your python script using os.setenv
pass an env dict containing the correct PATH to Popen
set the PATH in your shellscript
use the full path in your shellscript
set the PATH in eclipse