In order to get online predictions, I'm creating a model version on the ai-platform. It works fine unless I want to use the --accelerator parameter.
Here is the command that works:
gcloud alpha ai-platform versions create [...] --model [...] --origin=[...] --python-version=3.5 --runtime-version=1.14 --package-uris=[...] --machine-type=mls1-c4-m4 --prediction-class=[...]
Here is the parameter that makes it not work:
--accelerator=^:^count=1:type=nvidia-tesla-k80
This is the error message I get:
ERROR: (gcloud.alpha.ai-platform.versions.create) INVALID_ARGUMENT: Request contains an invalid argument.
I expect it to work, since 1) the parameter exists and uses these two keys (count and type), 2) I use the correct syntax for the parameter, any other syntaxes would return a syntax error, and 3) the "nvidia-tesla-k80" value exists (it is described in --help) and is available in the region in which the model is deployed.
Make sure you are using a recent version of the Google Cloud SDK.
Then you can use the following command:
gcloud beta ai-platform versions create $VERSION_NAME \
--model $MODEL_NAME \
--origin gs://$MODEL_DIRECTORY_URI \
--runtime-version 1.15 \
--python-version 3.7 \
--framework tensorflow \
--machine-type n1-standard-4 \
--accelerator count=1,type=nvidia-tesla-t4
For reference you can enable logging during model creation:
gcloud beta ai-platform models create {MODEL_NAME} \
--regions {REGION}
--enable-logging \
--enable-console-logging
The format for the --accelerator parameter as you can check in the official documentation is:
--accelerator=count=1,type=nvidia-tesla-k80
I think that might cause your issue, let me know.
How do I use PREMIRRORS in Bitbake local configurations or recipes?
I want to provide my own download locations for some slow or inaccessible third-party URLs, but the official PREMIRRORS documentation is vague and lacks examples.
Note: These results are based on experimentation with Yocto 2.3, but probably apply to 2.5 as well.
A simple example
Suppose that your recipe file contains this target URL:
SRC_URI = "http://download.example.com:8080/foo/bar/baz-1.0.tar.gz"
Then in your local.conf, you can define your custom download location as:
PREMIRRORS_prepend = "http://download\.example\.com:8080/.* http://my-mirror.example.com/copies/\n"
In this default case (with no special placeholders) Bitbake does not include the additional /foo/bar path elements, and instead tries to download just the filename from http://my-mirror.example.com/copies/baz-1.0.tar.gz
Advanced examples
These samples use special predefined placeholders, which are detailed in the next section.
HTTP/HTTPS with same file structure
Recipe: SRC_URI = "https://example.com:1234/foo/bar.zip"
Setting: PREMIRRORS_prepend = "http(s)?://example\.com(:\d+)?/.* http://mirror.local/PATH\n"
Attempts: http://mirror.local/foo/bar.zip
HTTP/HTTPS with flat structure
Recipe: SRC_URI = "https://example.com:1234/foo/bar.zip"
Setting: PREMIRRORS_prepend = "http(s)?://example\.com(:\d+)?/.* http://mirror.local/MIRRORNAME\n"
Attempts: http://mirror.local/example.com.1234.foo.bar.zip
Just switch the hostname
Recipe: SRC_URI = "ftp://example.com:1234/foo/bar.zip"
Setting: PREMIRRORS_prepend = "(\w+)://example\.com(:\d+)?/.* TYPE://mirror.local/PATH\n"
Attempts: ftp://mirrors.local/foo/bar.zip
Placeholders in replacement URI
PREMIRRORS parses all matched URIs and provides five special placeholder values in the target URI. Supposing the matched URI is http://host.example.com:1234/foo/bar/baz.txt:
TYPE https
HOST host.example.com%3A1234
PATH foo/bar/baz.txt
BASENAME baz.txt
MIRRORNAME host.example.com.1234.foo.bar.baz.txt
Altering the PREMIRRORS variable
The PREMIRRORS variable consists of series of lines (separated by \n) each with a regular expression to match a URI, and then a replacement string, with both portions separated by a space.
Bitbake tries them in order of appearance, and you generally want your private mirrors to take priority, so prepend onto PREMIRRORS, ex:
PREMIRRORS_prepend = "http://original/location/.* http://alternate/location/\n"
What file should I edit?
You can add entries to PREMIRRORS inside your bitbake recipes, but it is not recommended, since a major use of PREMIRRORS is for people reusing your recipe in some other context or location.
Instead, you can put it inside your local.conf file in an existing build directory. Alternately, edit the source template which Poky script use when creating a new local.conf in a fresh build-directory.
Other questions
What about SOURCE_MIRROR_URL?
The SOURCE_MIRROR_URL is a quick way to add a series of PREMIRROR entries for all supported protocols. For example, this setting:
INHERIT += "own-mirrors"
SOURCE_MIRROR_URL = "TYPE://mirror.local/PATH"
is the same as writing:
PREMIRRORS_prepend = "\
cvs://.*/.* TYPE://mirror.local/PATH \
svn://.*/.* TYPE://mirror.local/PATH \
git://.*/.* TYPE://mirror.local/PATH \
gitsm://.*/.* TYPE://mirror.local/PATH \
hg://.*/.* TYPE://mirror.local/PATH \
bzr://.*/.* TYPE://mirror.local/PATH \
p4://.*/.* TYPE://mirror.local/PATH \
osc://.*/.* TYPE://mirror.local/PATH \
https?$://.*/.* TYPE://mirror.local/PATH \
ftp://.*/.* TYPE://mirror.local/PATH \
npm://.*/?.* TYPE://mirror.local/PATH \
"
It seems the INHERIT+SOURCE_MIRROR_URL directives will still work if used in your local.conf (as opposed to a particular recipe.) However, Bitbake will emit warnings, so it may not be the intended use-case. Ex:
WARNING: Invalid protocol in PREMIRRORS: ('cvs://.*/.*', 'TYPE://mirror.local/PATH')
How can I check and debug my settings?
The -D debug flag will cause bitbake to emit information about what URLs it attempts to download from. You can also use -C do_fetch, which will force it to try the fetch step and re-download anything needed for the given recipe.
bitbake -D -C do_fetch software-recipe-name-here
Here's some example debug output, showing the PREMIRROR URL it attempts to access:
DEBUG: some-software-1.0 do_fetch: Trying PREMIRRORS
DEBUG: some-software-1.0 do_fetch: Fetcher accessed the network with the command /usr/bin/env wget -t 2 -T 30 -nv --passive-ftp --no-check-certificate -P /home/user/build_foo/DL_DIR 'http://mirror.local/path/to/the/filename.ext
If you need to experiment and run bitbake many times, it will be faster to temporarily put your new PREMIRRORS_prepend directive into a particular test-recipe, as opposed to modifying the local.conf. This is because Bitbake won't need to re-parse all the other recipes whenever you change it.
What if I want to isolate a port-number, e.g. http://host:123/foo?
Apparently there's no easy way to get the 123 on its own. While PREMIRRORS allows you to match with regular expressions, it does not seem to support using captured text from the match inside the replacement URI.
The port number is present inside HOST and MIRRORNAME, but there's no standard mechanism to split those values apart.
Fllow up. What if I want to isolate a port-number, e.g. http://host:123/foo?
Maybe you can use captured.
like:
org: "http://somewhere.org:1234/somedir1/somedir2/somefile_1.2.3.tar.gz"
reg: "http://somewhere.org(:\d+)?/.*"
sub: "http://somewhere2.org\1/somedir3"
result: "http://somewhere2.org:1234/somedir3/somefile_1.2.3.tar.gz"
I have an autotools-based BitBake recipe which I would like to have binaries installed in /usr/local/bin and libraries installed in /usr/local/lib (instead of /usr/bin and /usr/lib, which are the default target directories).
Here's a part of the autotools.bbclass file which I found important.
CONFIGUREOPTS = " --build=${BUILD_SYS} \
--host=${HOST_SYS} \
--target=${TARGET_SYS} \
--prefix=${prefix} \
--exec_prefix=${exec_prefix} \
--bindir=${bindir} \
--sbindir=${sbindir} \
--libexecdir=${libexecdir} \
--datadir=${datadir} \
--sysconfdir=${sysconfdir} \
--sharedstatedir=${sharedstatedir} \
--localstatedir=${localstatedir} \
--libdir=${libdir} \
...
I thought that the easiest way to accomplish what I wanted to do would be to simply change ${bindir} and ${libdir}, or perhaps change ${prefix} to /usr/local, but I haven't had any success in this area. Is there a way to change these installation variables, or am I thinking about this in the wrong way?
Update:
Strategy 1
As per Ross Burton's suggestion, I've tried adding the following to my recipe:
prefix="/usr/local"
exec_prefix="/usr/local"
but this causes the build to fail during that recipe's do_configure() task, and returns the following:
| checking for GLIB... no
| configure: error: Package requirements (glib-2.0 >= 2.12.3) were not met:
|
| No package 'glib-2.0' found
This package can be found during a normal build without these modified variables. I thought that adding the following line might allow the system to find the package metadata for glib:
PKG_CONFIG_PATH = " ${STAGING_DIR_HOST}/usr/lib/pkgconfig "
but this seems to have made no difference.
Strategy 2
I've also tried Ross Burton's other suggestion to add these variable assignments into my distribution's configuration file, but this causes it to fail during meta/recipes-extended/tzdata's do_install() task. It returns that DEFAULT_TIMEZONE is set to an invalid value. Here's the source of the error from tzdata_2015g.bb
# Install default timezone
if [ -e ${D}${datadir}/zoneinfo/${DEFAULT_TIMEZONE} ]; then
install -d ${D}${sysconfdir}
echo ${DEFAULT_TIMEZONE} > ${D}${sysconfdir}/timezone
ln -s ${datadir}/zoneinfo/${DEFAULT_TIMEZONE} ${D}${sysconfdir}/localtime
else
bberror "DEFAULT_TIMEZONE is set to an invalid value."
exit 1
fi
I'm assuming that I've got a problem with ${datadir}, which references ${prefix}.
Do you want to change paths for everything or just one recipe? Not sure why you'd want to change just one recipe to /usr/local, but whatever.
If you want to change all of them, then the simple way is to set prefix in your local.conf or distro configuration (prefix = "/usr/local").
If you want to do it in a particular recipe, then just assigning prefix="/usr/local" and exec_prefix="/usr/local" in the recipe will work.
These variables are defined in meta/conf/bitbake.conf, where you can see that bindir is $exec_prefix/bin, which is probably why assigning prefix didn't work for you.
Your first strategy was on the right track, but you were clobbering more than you wanted by changing only "prefix". If you look in sources/poky/meta/conf/bitbake.conf you'll find everything you are clobbering when you set the variable "prefix" to something other than "/usr" (like it was in my case). In order to modify only the install path with what would manually be the "--prefix" option to configure, I needed to set all the variables listed here in that recipe:
prefix="/your/install/path/here"
datadir="/usr/share"
sharedstatedir="/usr/com"
exec_prefix="/usr"
I have a strange issue with Sphinx, I am trying to be able to match things like:
L.A. Confidential
So people can search for "LA Confidential" and still get that title. Similarly for "P.M."
to be able to match "PM", etc.
I tried putting the period (full stop character U+002E) in the ignore_char list. This
didn't make any difference.
So then I tried implementing index_sp =1. This did not solve the issue either.
According to what I understand of the documentation, either of these should have solved
this issue correct?
I wonder if it has somthing to do with our math mode which is set to extended2, using
Sphinx 2.0.3.
Any help would be greatly appreciated.
Edit, here is my thinking_sphinx.yml config:
Note that the period character (U+002E) is not used anywhere else in my config except in the ignore_chars line.
production:
mem_limit: 512M
morphology: stem_en
wordforms: "db/sphinx/wordforms.txt"
stopwords: "db/sphinx/stopwords.txt"
ngram_chars: "U+4E00..U+9FBB, U+3400..U+4DB5, U+20000..U+2A6D6, U+FA0E, U+FA0F, U+FA11, \
U+FA13, U+FA14, U+FA1F, U+FA21, U+FA23, U+FA24, U+FA27, U+FA28, U+FA29, U+3105..U+312C, \
U+31A0..U+31B7, U+3041, U+3043, U+3045, U+3047, U+3049, U+304B, U+304D, U+304F, U+3051, \
U+3053, U+3055, U+3057, U+3059, U+305B, U+305D, U+305F, U+3061, U+3063, U+3066, U+3068, \
U+306A..U+306F, U+3072, U+3075, U+3078, U+307B, U+307E..U+3083, U+3085, U+3087, \
U+3089..U+308E, U+3090..U+3093, U+30A1, U+30A3, U+30A5, U+30A7, U+30A9, U+30AD, \
U+30AF, U+30B3, U+30B5, U+30BB, U+30BD, U+30BF, U+30C1, U+30C3, U+30C4, U+30C6, \
U+30CA, U+30CB, U+30CD, U+30CE, U+30DE, U+30DF, U+30E1, U+30E2, U+30E3, U+30E5, \/
U+30E7, U+30EE, U+30F0..U+30F3, U+30F5, U+30F6, U+31F0, U+31F1, U+31F2, U+31F3, \
U+31F4, U+31F5, U+31F6, U+31F7, U+31F8, U+31F9, U+31FA, U+31FB, U+31FC, U+31FD, \
U+31FE, U+31FF, U+AC00..U+D7A3, U+1100..U+1159, U+1161..U+11A2, U+11A8..U+11F9, \
U+A000..U+A48C, U+A492..U+A4C6"
ngram_len: 1
ignore_chars: "U+0027, U+2013, U+2014, U+0026, U+002E, ., &"
(huge char_set entry here for different languages, ommited.)
I ran a test locally with the following in my thinking_sphinx.yml for Thinking Sphinx v3.0.4 and it worked:
development:
ignore_chars: U+002E
The same in sphinx.yml for Thinking Sphinx v2.0.14 worked too. I am using Sphinx 2.0.8, but I'll be a little surprised if that's the problem. It's certainly unrelated to your match mode.
I have successfully configure perl for cross comile by using configure options:
./Configure -des -Dusecrosscompile \
-Dtargethost=172.17.185.91 \
-Dtargetdir=/home/perl/ \
-Dtargetuser=root \
-Dtargetarch=arm-linux \
-Dcc=arm-linux-gcc \
-Dusrinc=/opt/Mozart_Toolchain/arm-eabi-uclibc/include/ \
-Dincpth=/opt/Mozart_Toolchain/arm-eabi-uclibc/include/ \
-Dlibpth=/opt/Mozart_Toolchain/arm-eabi-uclibc/lib/
And the configure script tell me "Now you must run 'make'." But I encounter such as error when I make:
`sh cflags "optimize='-O2'" miniperlmain.o` miniperlmain.c
CCCMD = arm-linux-gcc -DPERL_CORE -c -DOVR_DBL_DIG=14 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -Wall
In file included from perl.h:38,
from miniperlmain.c:40:
config.h:4425:12: error: operator '==' has no left operand
In file included from miniperlmain.c:40:
perl.h:713:14: error: operator '>=' has no left operand
... ...
In config.h, some macro is left blank, for example:
#define INTSIZE /**/
#define LONGSIZE /**/
#define SHORTSIZE /**/
... much more ...
And I think it is the undefined macro result in the make error. I have no idea how to fix it. Why the macro is blank even if successfully configure?
Are there some guides to cross compile Perl?
There is a Cross directory that features a README file that includes the following instructions for arm-linux:
1) You should be reading me (README) in perl-5.x.y/Cross
2) Make sure you are in the Cross directory.
3) Edit the file 'config' to contain your target platform information.
4) make patch ## This will patch the existing source-tree.
5) make perl ## Will make perl
(Read the whole thing.)
I got the easiest way to cross compile Perl for arm-linux.Please refer to Cross-compiling perl. It's a great work! It saved my life.
Just according to instructions that give, you can get what you want. You may encounter such error when 'make':
pp_sys.c:78: error: non-thread-local declaration of 'h_errno' follows thread-local declaration
Simply comment that line.
Enjoy it!