How does one submit a job requesting a specific GPU in condor without a submission file? - hpc

I want to do:
condor_submit -i request_cpus=8 request_gpus=1 -requirements="(CUDADeviceName!='Tesla K40m')"
but due to the spaces and commas, I cannot do this. I also tried:
condor_submit -i request_cpus=2 request_gpus=1 -requirements=(CUDADeviceName!=TeslaK40m)
but it failed.
It does work with a submission file e.g.
Request_gpus = 1
Request_cpus = 2
requirements = (CUDADeviceName != "Tesla K40m")
# requirements = (CUDADeviceName == "Quadro RTX 6000")
Queue
so please don't recommend that I already have that working. I want it in one line on the command line.
Thanks!

try
-requirements='(CUDADeviceName != "Tesla K40m")'

Related

Raspberry Pi not picking up Wii controller or not communicating

I am trying to connect my Wii-mote's to my Raspberry Pi, so I can use them in pygame. I know to use the Wii controller, I need to make some configurations, and I have made them. However, when I run my .sh script to connect the remotes, I am thrown some errors. Here are the scripts:
/mywinput
#WiiMote
Wiimote.A = BTN_A
Wiimote.B = BTN_B
Wiimote.Dpad.X = ABS_Y
Wiimote.Dpad.Y = -ABS_X
Wiimote.Minus = BTN_SELECT
Wiimote.Plus = BTN_START
Wiimote.Home = BTN_MODE
Wiimote.1 = BTN_X
Wiimote.2 = BTN_Y
# Nunchuk
Nunchuk.C = BTN_C
Nunchuk.Z = BTN_Z
Plugin.led.Led1 = 1
#Plugin.led.Led2 = 1
Plugin.led.Led3 = 1
#Plugin.led.Led4 = 1
/wiicontroller.sh
#!/bin/bash
sleep 1 # Wait until Bluetooth services are fully initialized
hcitool dev | grep hci >/dev/null
if test $? -eq 0 ; then
wminput -d -c /home/pi/mywinput CC:FB:65:2C:52:8F &
else
echo "Blue-tooth adapter not present!"
exit 1
fi
I have made wiicontoller.sh executable:
sudo chmod 775 /home/pi/bin/connectwii.sh
Supposedly, I should press one and two one the controller and it should connect. However, the script returns immediately. And even then, when I press one and two, I get thrown errors. I don't think that should be possible as the script has returned, but here is the output:
Socket connect error (control channel)
I am also aware that using cwiid in python is another option. But that doesn't work either, as it does not detect my remote. wmgui also cannot pickup my remote. The only way I am able to detect my remote is through the hcitool scan command. Any help would be appreciated!

GRIB1 to GRIB2 eccodes conversion failing due to paramId = 0

So what I am trying is converting a GRIB1 file to GRIB2 containing mostly wind data.
Usually you can just change the edition to 2 and the eccodes lib does everything else.
Now the issue is that my GRIB1 file has only paramId=0 and shortName=unknown messages.
What the hell am I supposed to do? When I load it into a viewer (e.g. PredictWind Offshore) it displays just fine. Any ides on how I can convert this to GRIB2 without knowing the messages? Am I missing something?
What I already tried:
❯ grib_set -s edition=2 e1.grib e2.grib
ECCODES ERROR : concept: no match for paramId=0
ECCODES ERROR : Please check the Parameter Database 'https://apps.ecmwf.int/codes/grib/param-db/?id=0'
ECCODES ERROR : concept: input handle edition=2
ECCODES ERROR : grib_set_values[0] edition (type=long) failed: Concept no match
What my GRIB1 (e1.grib) looks like:
❯ grib_dump e1.grib | egrep 'paramId|shortName'
...
shortName = unknown;
paramId = 0;
...
❯ grib_dump dsv1.grib | egrep 'paramId' | wc -l
679 # all unknown and 0
Edit:
For future readers:
I ended up not using eccodes at all. My solution now involves using PyNIO which worked great for me. It is able to read GRIB1.
For further info contact me directly.

how to specify multiple log files pattern in fail2ban jail?

I have log files on my server as follows
vpn_20191007.log
vpn_20191008.log
vpn_20191009.log
vpn_20191010.log
vpn_20191011.log
vpn_20191012.log
vpn_20191013.log
vpn_20191014.log
vpn_20191015.log
vpn_20191016.log
Is it possible to add log files pattern in fail2ban jail config?
[application]
enabled = false
filter = example
action = iptables
logpath = /var/log/vpn_%D.log
maxretry = 1
Well, conditionally it is possible...
Although wildcards are basically allowed at the moment, so :
logpath = /var/log/vpn_*.log
will do the job, but it is a bit ugly in your case:
fail2ban cumulate the list of files only by start of service, so the list remains obtained in fail2ban (unless it gets reloaded) - this means you should notify fail2ban that the log file name got changed (see https://github.com/fail2ban/fail2ban/issues/1379, the work is in progress).
since only one file will get new messages, the monitoring of other files is unneeded, especially if polling backend is used.
So better create some logrotate rules for that:
in order to rename/compress all previous log-files (to avoid match for obsolete files);
either create hard- or sym-link for last/active file with a fixed name (so fail2ban is always able to find it with the same name, and you'd not need wildcard at all);
or to notify fail2ban to reload the jail if logfile-name got changed(fail2ban-client reload vpn).
Here is an example for logrotate amendment:
postrotate
nfn="/var/log/vpn_$(date +%Y%m%d).log"
touch "$nfn"
ln -fs "$nfn" /var/log/vpn.log
You can add wilcard :
logpath = /var/log/vpn_*.log
and/or you can use multiple lines :
logpath = /var/log/vpn_20191007.log
/var/log/vpn_20191008.log
/var/log/vpn_20191009.log
/var/log/vpn_20191010.log
/var/log/vpn_20191011.log
/var/log/vpn_20191012.log
/var/log/vpn_20191013.log
/var/log/vpn_20191014.log
/var/log/vpn_20191015.log
/var/log/vpn_20191016.log
(You can combine the two)

config.site for vendor libs on Fedora x86_64

I'm having trouble building a few Autotool-based libraries on Fedora 26, x86_64. The 64-bit Fedora puts third party and vendor libraries in /usr/local/lib64. Ubuntu 17 uses /usr/local/lib so the same projects build OK.
I've been using --libdir=/usr/local/lib64 but three libraries resist it. I lack a config.site for /usr/local so I am trying to add one. The Autoconf manual on Site Defaults is a tad bit confusing to me when it discusses usr/local's config.site. It says:
[discussion of /usr version of confg.site] ...
Likewise, on platforms where 64-bit libraries are built by default,
then installed in /usr/local/lib64 instead of /usr/local/lib, it is
appropriate to install /usr/local/share/config.site:
# /usr/local/share/config.site for platforms that prefer
# the directory /usr/local/lib64 over /usr/local/lib.
test "$libdir" = '${exec_prefix}/lib' && libdir='${exec_prefix}/lib64'
The problem I am having is, is the modification above appended to the /usr/local version of config.site? Or does it replace an existing block of code? Or can I just copy it where it belongs without modification?
Or maybe, what does a cat of /usr/local/share/config.site look like?
Here is the config.site for /usr. Its not clear to me if it needs modification or how to modify it.
$ cat /usr/share/config.site
# This is the config.site file to satisfy FHS defaults when installing below
# /usr.
#
# You may override this file by your config.site using the CONFIG_SITE env
# variable.
#
# Note: This file includes also RHEL/Fedora fix for installing libraries into
# "/lib/lib64" on 64bit systems.
if test -n "$host"; then
# skip when cross-compiling
return 0
fi
if test "$prefix" = /usr \
|| { test "$prefix" = NONE && test "$ac_default_prefix" = /usr ; }
then
test "$sysconfdir" = '${prefix}/etc' && sysconfdir=/etc
test "$sharedstatedir" = '${prefix}/com' && sharedstatedir=/var
test "$localstatedir" = '${prefix}/var' && localstatedir=/var
ARCH=`uname -m`
for i in x86_64 ppc64 s390x aarch64; do
if test $ARCH = $i; then
test "$libdir" = '${exec_prefix}/lib' && libdir='${exec_prefix}/lib64'
break
fi
done
fi
config.site for vendor libs on Fedora x86_64
This answers the question of what config.site looks like for /usr/local/share/config.site. It does not answer the question of why --libdir=/usr/local/lib64 fails to set the directory, as #John Bollinger pointed out in the comments.
The /usr/local/share/config.site is wrong. Though it was copied from Fedora's config.site and placed in /usr/local/share, the prefix directories are wrong. The prefix test should use /usr/local and not /usr.
Below is the corrected one.
$ cat /usr/local/share/config.site
...
if test -n "$host"; then
# skip when cross-compiling
return 0
fi
if test "$prefix" = /usr/local \
|| { test "$prefix" = NONE && test "$ac_default_prefix" = /usr/local ; }
then
test "$sysconfdir" = '${prefix}/etc' && sysconfdir=/etc
test "$sharedstatedir" = '${prefix}/com' && sharedstatedir=/var
test "$localstatedir" = '${prefix}/var' && localstatedir=/var
ARCH=`uname -m`
for i in x86_64 ppc64 s390x aarch64; do
if test $ARCH = $i; then
test "$libdir" = '${exec_prefix}/lib' && libdir='${exec_prefix}/lib64'
break
fi
done
fi
I'm not sure if these are correct, however. They were not modified.
test "$sysconfdir" = '${prefix}/etc' && sysconfdir=/etc
test "$sharedstatedir" = '${prefix}/com' && sharedstatedir=/var
test "$localstatedir" = '${prefix}/var' && localstatedir=/var
Now, the next question is, why Fedora's /usr/share/config.site is not handling prefix=/usr/local properly. That's an open question at Issue 1510073 : Autoconf does not honor libdir in config.site for "libdir=#libdir#" in *.pc file, which has been closed as NOT A BUG.

how do i track downloads for files in my google cloud storage

I need a way of tracking downloads by users of my site, for mp3 files in my cloud storage? Aside from storage logs, are there any other solutions.
There's a storage logging feature: https://cloud.google.com/storage/docs/access-logs
This question is pretty old, but I thought I would share my solution in case anyone is looking for a solution still. You need to enable access logs and then write some scriptage to download and parse the access logs.
I used Ruby and here's the meat of my script:
#!/usr/bin/env ruby
require 'fileutils'
temp_dir = "/tmp/access-logs"
output_file = "/tmp/download-count.csv"
# Clean up the existing access logs
FileUtils.rm_rf(temp_dir)
FileUtils.mkdir(temp_dir)
`/usr/bin/gsutil -m cp "gs://my_access_logs/FusionAuthAccesssLog_usage_*" #{temp_dir} > /dev/null 2>&1`
# Collect the counts
counts = Hash.new(0)
Dir.foreach(temp_dir) do |file|
if File.file?("#{temp_dir}/#{file}")
date = file.gsub(/MyAccesssLog_usage_([0-9]{4})_([0-9]{2})_([0-9]{2}).*/, '\1\2\3')
File.readlines("#{temp_dir}/#{file}").each do |l|
if l =~ /my-file.zip"/
counts[date] = counts[date] + 1
end
end
end
end
File.open(output_file, "w", :encoding => "UTF-8") do |f|
# Write the header
f.puts("Date,Download count\n")
# Write the counts
counts.sort.each do |date,count|
f.puts("#{date},#{count}\n")
end
end
I wrote a blog post on this that goes over the script in detail as well. Here's the link: https://fusionauth.io/blog/2018/09/20/download-counts-from-google-cloud-storage