I'm new in Ubuntu and programming.
I'm testing a program that I found on github, to download and import OSM data into postgis.
It works when I run it from terminal (url and name are fake):
make all NAME=dbname URL='http://myurl'
using postgres user.
Now I need to run this command every day.
So I wrote that script:
#!/bin/bash
# go to the directory with Makefile
cd /PCuserhome/directory/to/Makefile/
# run Makefile
make all NAME=dbname URL='http://myurl'
and it works when i run it from terminal.
So I have added it to crontab (of postgres user) in this way:
0,15,30,45 * * * * /PCuserhome/myscript.sh
It create the db but probably fail in running osmosis selection (Osmosis is in the path for all users).
Any idea to solve this? Thank you!
crontab commands are executed only with minimal environment variables, i.e.
PATH=/usr/bin:/bin (on debian anyway),
so if you are relying on programs that are in your $PATH, it will fail.
Consider specifying an absolute path to the osmosis program wherever it's called from.
Alternatively you can change $PATH itself in your script
export PATH="/my/bin:$PATH"
p.s.: you can check the environment by adding a simple cron job
* * * * * env > /tmp/env.txt
Related
I have a script that I want to schedule to run each lets say every 10min.
this script downloads a csv file in this folder.
C:\Users\rsharma\scripts\cylance\external_devices
Script name is external_devices.py and I have created it in visual studio code (WSL).
now I know I can use Crontab and windows scheduler to schedule it but nothing seems to be working.
this is my script path
C:\Users\rsharma\scripts\cylance\External_devices.py
these are the crontab entries I have tried
*/10 * * * * /mnt/c/Users/rsharma/scripts/cylance/External_devices.py
*/10 * * * * /usr/bin/python3 /mnt/c/Users/rsharma/scripts/cylance/External_devices.py
windows scheduler entries -
program - C:\WINDOWS\System32\wsl.exe
argument - /mnt/c/Users/rsharma/scripts/cylance/External_devices.py
Please move the script out of "/mnt/c/Users...." directory and copy to another directory in the C drive. The WSL mount point is only available when the lxss Hyper-V WSL is started so will not be mounted unless VSCode is running. It's now much better to use the WSL2 (Ubuntu 20.4) version which you can download from the Microsoft store and then install into as an extension "VSCode Remote Extensions for WSL" which will mount into "/home/username" and is much much faster and feature rich, especially if you plan to use Docker as it is directly integrated.
Install Windows Python into a directory of your choice for example - c:\Python. You will obviously need to install any libraries that the Unix version uses. There will be a binary
c:\Python\Python3.x\python.exe
That will need to be coded into a batch windows command file (xxx.cmd). To run your script change to the directory the script is saved and run the python command.
Example code for xxx.cmd batch file:
cd c:\scriptdirectory
c:\Python\Python3.x\python.exe External_devices.py
Run this on the command line to ensure it works, then schedule in the Windows Scheduler as appropriate.
I'd like to run a script using kpcli (http://kpcli.sourceforge.net/) via cronjob. All works fine when running it interactively.
When running it the following minimum example via cronjob I get the following error-message.
46 22 * * * myuser /usr/local/bin/kpcli --command "vers"
Here the error message
No usable Term::ReadLine::* modules found.
This list was tried:
* Term::ReadLine::Gnu
* Term::ReadLine::Perl
* Term::ReadLine::Perl5
For more information, read the documentation: perldoc kpcli
Here is the output when running it via command line
kpcli: 3.4
Perl: v5.26.1
Operating system: linux ("Ubuntu 18.04.4 LTS")
ReadLine being used: Term::ReadLine::Gnu
Pivotal Perl Modules for kpcli
* File::KeePass: 2.03
* Term::ShellUI: 0.92
* Term::ReadKey: 2.37
* Term::ReadLine: 1.16
* Capture::Tiny: 0.48
* Clipboard: 0.13
* Math::Random::ISAAC: 1.003
* Term::ReadLine::Gnu: 1.35
* Authen::OATH: not installed (optional)
* Sub::Install: not installed (optional)
I also tried to set the variable
PERL5LIB=/usr/lib/x86_64-linux-gnu/perl5/5.26
But nothing changed :(
What's wrong?
I have it :). I have unset all environment variables step by step by shell script. The result was that the variable TERM="xterm" was set in my env but not in the cron env. After setting it in cron all works fine
The PERL5LIB is a user's environment variable; the system's utility cron knows nothing about it.
Further, what you invoke from crontab mostly runs "out of" your home directory; this can depend on the system but it is generally not from where the script is. So then that place is the script's working directory.
Clearly the module Term::ReadLine::Gnu is installed in a non-standard location and when the script runs via cron it altogether cannot find that module (and perhaps yet others).
There are various ways about this
Set PERL5LIB right on the command line in the crontab
SHELL=/bin/bash
MAILTO=your#email.address
* * * * * PERL5LIB=/path/as/needed /full/path/script.pl arguments
The environment variables I set in the beginning are often already set to reasonable values (but they may not be). Consider what other environment may be needed, if any. See your system's documentation for crontab(5) and test
Change to a directory from which the script is known to work
* * * * * (cd /path/to/where/it/works; ./script.pl arguments)
Make the script itself change to the desired working directory by adding to it
use FindBin qw($RealBin);
BEGIN {
chdir $RealBin or die "Can't chdir to $RealBin: $!";
};
This must be done before any use statements that need the script to already be in the particular directory; perhaps all those that refer to non-standard locations.
The solution where the order of compile-time statements matters can be fragile here. On the other hand, in that way the working directory is set up right in the script and sometimes that is precisely what is needed.
There is certainly a rationale for doing both, change the working directory and set PERL5LIB
* * * * * (cd /full/path; PERL5LIB=path1:path2:... ./script.pl arguments)
I am able to execute scrip from command line.
I'm executing it like this:
/path/to/script run
But while executing it from cron like below, the page is not comming:
55 11 * * 2-6 /path/to/script.pl run >> /tmp/script.log 2>&1
The line which is getting a webpage uses LWP::Simple:
my $site = get("http://sever.com/page") ;
I'm not modyfing anything. The page is valid and accessible.
I'm getting enpty page only when I execute this script from crontab. I am able to execute itfrom command line!
Crontab is owned by root. And job is executed as root.
Thanks in advance for any clue!
It's difficult to say what might be causing this, but there are differences between your environment, and the environment created by crontab.
You could try running it through a shell with appropriate args to construct your user environment:
55 11 * * 2-6 /bin/tcsh -l /path/to/script.pl run >> /tmp/script.log 2>&1
I'm assuming you are running it by cron with your own user ID of course. If you aren't, then obviously you should try running it manually with the user ID that cron is using to run it.
If it's not a difference in environment variables (e.g. those that specify a proxy to use), I believe you are running afoul of SElinux. Among other things, it prevents background applications (e.g. cron jobs) from accessing the internet unless you explicitly allow them to do so. I don't know how to do so, but you should be able to find out quite easily.
I have a perl script (which syncs delicious to wp) which:
runs via the shell but
does not run via cron (and i dont get an error)
The only thing I can think of is that it read the config file wrongly but... it is defined via the full path (i think).
I read my config file as:
my $config = Config::Simple->import_from('/home/12345/data/scripts/delicious/wpds.ini',
\my %config);
(I am hosted on mediatemple)
Does anybody have a clue?
update 1: HERE is the complete code: http://plugins.svn.wordpress.org/wordpress-23-compatible-wordpress-delicious-daily-synchronization-script/trunk/ (but I have added the path as above to the configuration file location as difference)
update 2: crossposted on https://forums.mediatemple.net/viewtopic.php?pid=31563#p31563
update 3: the full path did the trick, solved
The difference between a cron job and a job run from the shell is 'environment'. The primary difference is that your profile and the like are not run for a cron job, so any environment variables you have set in your normal shell environment are not set the same in the cron environment - no extensions to PATH, no environment variables identifying where Delicious and/or WP are hosted, etc.
Suggestion: create a cron job that simply reports the environment to a known file:
env > /home/27632/tmp/env.27632
Then see what is set in your own shell environment in comparison. Chances are, that will reveal the trouble.
Failing that, other environmental differences are that a cron job has no terminal, and has /dev/null for input and output - so interactive stuff does not work well.
it seems the problem is not in running perl, but locating the Config library
you should try:
perl -e "print #INC"
and run a similar perl script in cron, and read the output
it possible that they differ
I suggest looking at my answer to How to simulate the environment cron executes a script with?
This is an similar Jonathan's answer but goes a bit further.
Based on your crontab, and depending on your installation, the problem might be the "perl". As others note the environment, particularly the $PATH variable, is different for cron. perl may not be in the path so you need to put the full path to perl in the cron command.
You can determine the path with the command $ type perl
I run into the same problem ...
Perl script works but not via CRON => error: "perl: command not found"
... after an update from Plesk 12.0 to Plesk 12.5. But the existing answers were not very helpful for me.
It took some time, but than I found this thread in the Odin forum which helps me: https://talk.plesk.com/threads/scheduled-tasks-always-fail.331821/
They suggest the following:
/usr/local/psa/bin/server_pref -u -crontab-secure-shell ""
That deletes in the /var/spool/cron/crontabs files the line:
SHELL="/opt/psa/bin/chrootsh"
After that, my cron jobs run with out any error.
(Ubuntu 14.04 with Plesk 12.5)
If the perl script runs fine manually, but not from crontab, then
there is some environment path needed by the some package that is not
getting through `cron`. Run your command as follows:
suppose your cron entry like:
* 13 * * * /usr/bin/perl /home/username/public_html/cron.pl >/dev/null 2>&1
env - /home/username/public_html/cron.pl
The output will show you the missing package. export that package path in
$PATH variables
I have a bunch of Perl scripts that all run fine, yet need to have use Plibdata; up top.
I set up a cron job that runs (I get the confirmation email from root) and it spits back the following error message:
Can't locate Plibdata.pm in #INC (#INC contains: /install/lib /opt/perl58/lib/5.8.8/IA64.ARCHREV_0-thread-multi /opt/perl58/lib/5.8.8 /opt/perl58/lib/site_perl/5.8.8/IA64.ARCHREV_0-thread-multi /opt/perl58/lib/site_perl/5.8.8 /opt/perl58/lib/site_perl .) at ./x line 5.
BEGIN failed--compilation aborted at ./x line 5.
Line 5 is... you guessed it.... use Plibdata;
I am also attempting to set the environment as such:
use lib "$ENV{CARSPATH}/install/lib";
so maybe if I found the location of this plibdata, I could explicitly direct it that way?
My cron commands will be executed using /usr/bin/sh says crontabs...
Any suggestions?
This script works from the command line.
You don't say what Plibdata is. You also don't state if this works at your command prompt. I assume that it does.
Try this:
perl -MPlibdata -e 1
Assuming that doesn't spit the same error, try this:
perl -MPlibdata -le 'print $INC{"Plibdata.pm"}'
That will tell you where. (It's probably in your PERL5LIB env var if this works.) Then you can just add the appropriate "use lib" to the directory Plibdata.pm is in.
Also, be sure you're using the same perl in both locations - command line ("which perl") and in the cron job (try "BEGIN { print $^X }" at the top of your script).
Cron uses a different user env than your env when logged in. Are you able to run the script from the command line? If so, just set your env variables inside the cron above your current commands.
Clearly, Plibdata.pm is not installed in the default module paths on your system:
/install/lib /opt/perl58/lib/5.8.8/IA64.ARCHREV_0-thread-multi /opt/perl58/lib/5.8.8 /opt/perl58/lib/site_perl/5.8.8/IA64.ARCHREV_0-thread-multi /opt/perl58/lib/site_perl/5.8.8 /opt/perl58/lib/site_perl
You have three choices:
Install Plibdata.pm in a known Perl system path (site_perl is the classic option).
Make the PERL5LIB shell environment (or the equivalent command line -I option for Perl) include the installation path of the module.
Use use lib in your script. Remember that the use lib action is done at compile time, so your variable in the path may not be initialised. Try using the variable in a BEGIN block like this:
my $env;
BEGIN {
$env = $ENV{CARSPATH};
}
use lib "$env/install/lib";
Running your program from a wrapper script as others have suggested is probably my preferred method, but there may be a few other solutions:
If you're using a modern cron you may be able to do something like this in your crontab entry:
* * * * * CARSPATH=/opt/carsi x
replacing the asterisks with the appropriate schedule designators.
This will set CARSPATH for the x process and allow the use lib statement that passes the environment variable to work.
You can also, depending on your shell and cron implementation, store your environment setup in a file and do something like:
* * * * * source specialenv.sh && x
Where specialenv.sh contains lines like (for bash)
export CARSPATH=/opt/carsi
You may also be able to set environment variables directly in the crontab, should you choose to do so.
cron does not setup an environment for you when it runs your code, so the environment variable $CARSPATH does not exist. I suggest only running shell scripts from cron, setting the environment inside of the shell script and then running the program you really wanted to run.
example wrapper script:
#!/bin/bash
source ~username/.bash_profile
cd ~username
./script.pl
If you are using ksh or sh you may need to say
#!/bin/sh
. ~username/.profile
cd ~username
./script.pl
Remember to replace username with your username on the system. Also, if the script is not in your home directory you will want to execute it with the path to it, not ./.
You say source, or period space, to load a given shell script in the current shell. You load it in the current shell so that any environment settings stay with the current shell.
~/.bash_profile, ~/.bashrc, ~/.profile, /etc/profile, etc. are all files that commonly hold your environment setup. Which one you are using depends heavily on your OS and who set it up.
Although its not an 'answer', I resolved this issue by using DBI instead of the Plibdata.
Which is kind of milky because now I will have to change a couple scripts around... ahhhhh I wish there was something I could do to make the Plibdata work
I'm still going to try Chas. Owens answer to see if that works
didn't work for me... "interpreter "/bin/bash" not found"maybe it would work with people who have that interpreter
* * * * * CARSPATH=/opt/carsi ./x
works