I build the libtensorflowLite.so for android and it works well.But the binary size is 3.8M and it is too big as it is just reported with hundreds of KB. So, do i have improper build configuration?
I add the android ndk and sdk path in tensorflow congfire.
I add following in tensorflow/lite/BUILD:
cc_binary(
name = "libtensorflowLite.so",
linkopts = ["-shared", "-Wl,-soname=libtensorflowLite.so"],
visibility = ["//visibility:public"],
linkshared = 1,
copts = tflite_copts(),
deps = [
":framework",
"//tensorflow/contrib/lite/kernels:builtin_ops",
],
)
The build commit:
bazel build -c opt //tensorflow/contrib/lite:libtensorflowLite.so
--crosstool_top=//external:android/crosstool
--cpu=arm64-v8a
--host_crosstool_top=#bazel_tools//tools/cpp:toolchain
--cxxopt="-std=c++11"
--verbose_failures
Related
Currently working on a react app and I'm getting this error after installing metaplex.
My react-scripts version is 4.0.3
./node_modules/#metaplex-foundation/js-next/dist/esm/programs/token/gpaBuilders/TokenGpaBuilder.mjs
Can't import the named export 'ACCOUNT_SIZE' from non EcmaScript module (only default export is available)
I found out the solution in the git of metaplex here. I will leave you here the whole answer anyway.
Getting Started with Metaplex and CRA 5
This example sets up a new React app with Metaplex using "Create React App" (CRA) version 5 — i.e. using Webpack 5.
This example has been generated using the following steps:
Create a new project using the "Create React App" command.
npx create-react-app getting-started-react-cra5
cd getting-started-react-cra5
Install the Metaplex and the Solana SDKs.
npm install #metaplex-foundation/js #solana/web3.js
Install some polyfills.
npm install assert util crypto-browserify stream-browserify
Install and use react-app-rewired.
# Installs react-app-rewired.
npm install react-app-rewired
# Replaces "react-scripts" with "react-app-rewired" in package.json scripts.
sed -i '' 's/react-scripts /react-app-rewired /g' package.json
Override Webpack 5 configurations.
Create a new file to override Webpack 5 configurations.
touch config-overrides.js
Copy the following code inside the new config-overrides.js file.
const webpack = require("webpack");
module.exports = function override(webpackConfig) {
// Disable resolving ESM paths as fully specified.
// See: https://github.com/webpack/webpack/issues/11467#issuecomment-691873586
webpackConfig.module.rules.push({
test: /\.m?js/,
resolve: {
fullySpecified: false,
},
});
// Ignore source map warnings from node_modules.
// See: https://github.com/facebook/create-react-app/pull/11752
webpackConfig.ignoreWarnings = [/Failed to parse source map/];
// Polyfill Buffer.
webpackConfig.plugins.push(
new webpack.ProvidePlugin({ Buffer: ["buffer", "Buffer"] })
);
// Polyfill other modules.
webpackConfig.resolve.fallback = {
crypto: require.resolve("crypto-browserify"),
stream: require.resolve("stream-browserify"),
util: require.resolve("util"),
assert: require.resolve("assert"),
fs: false,
process: false,
path: false,
zlib: false,
};
return webpackConfig;
};
Update your browser requirements.
Update the browserslist object of your package.json to include the following production requirements.
"browserslist": {
"production": [
- ">0.2%",
- "not dead",
- "not op_mini all"
+ "chrome >= 67",
+ "edge >= 79",
+ "firefox >= 68",
+ "opera >= 54",
+ "safari >= 14"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
That's it!
You should not use #metaplex-foundation/js-next SDK, the actual repo is #metaplex-foundation/js, the name was changed and the updated repo is this, so try using #metaplex-foundation/js instead
Useful info
I am building a webapp and using as back-end rocket.rs, with diesel.rs to connect to the Postgres database. I wanto to dockerize this webapp: I'm on Windows 11, and I want to cross-compile for Linux alpine, so I can just add the executable in the container.
Toolchain: nightly-2021-12-05-x86_64-pc-windows-msvc
Target: x86_64-unknown-linux-musl
Cargo.toml file:
[package]
name = "project-api"
version = "1.0.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
rocket = { version = "0.4.10", default-features = false }
diesel = { version = "1.4.8", features = ["postgres"] }
serde = { version = "1.0.131", features = ["derive"]}
dotenv = "0.15.0"
[dependencies.rocket_contrib]
version = "0.4.10"
default-features = false
features = ["json", "serve", "diesel_postgres_pool"]
.cargo/config.toml file:
[target.x86_64-unknown-linux-musl]
linker = "rust-lld"
Compile command: cargo build --target x86_64-unknown-linux-musl --release
In my PATH I already added:
.../PostgreSQL/14/bin
.../PostgreSQL/14/lib
.../PostgreSQL/14/include
The problem
When I compile the project, I get the following error:
error: linking with `rust-lld` failed: exit code: 1
...
= note: rust-lld: error: unable to find library -llibpq
This is the complete error:
Compiling project-api v1.0.0 (D:\projects\project-management-system)
error: linking with `rust-lld` failed: exit code: 1
|
= note: "rust-lld" "-flavor" "gnu" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\self-contained\\rcrt1.o" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\self-contained\\crti.o" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\self-contained\\crtbeginS.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.0.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.1.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.10.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.11.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.12.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.13.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.14.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.15.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.2.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.3.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.4.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.5.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.6.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.7.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.8.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.project_api.f9c82bac-cgu.9.rcgu.o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7.22qos1iwh0pu1l3f.rcgu.o" "--as-needed" "-L" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps" "-L" "D:\\projects\\project-management-system\\target\\release\\deps" "-L" "D:/programs/PostgreSQL/14/lib" "-L" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib" "-Bstatic" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libdotenv-17787ac5e36ba405.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\librocket_contrib-908e672c5a94e5a1.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libdiesel-b618b4aafb14a3d4.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libpq_sys-0deff7ab575185bc.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libbitflags-cd7b464110d56932.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libr2d2-17991d433d6b8815.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libscheduled_thread_pool-ed3fe870ef2578f4.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libparking_lot-084e1692fb5bda8c.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libparking_lot_core-57505ee8fbe2715a.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\liblock_api-c4af51bd9cc3c613.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libscopeguard-d10f930610969f1b.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libinstant-5b170197aa126111.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libserde_json-f02c39ef48f71da7.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libryu-62ce0cec239f97c8.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libitoa-8ec78a5baa91caac.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\librocket-28b96e263956caad.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libatty-11406b7a761cb014.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libbase64-d9fe97a8863a409c.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libmemchr-1dd949379d427674.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libtoml-6efc66767f0cacd7.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libserde-4add0bb05fd83ce8.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libyansi-077b82a357bfe296.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\librocket_http-9426f414126ee4c8.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libhyper-5854b3142fe943a8.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libmime-41e7a782b7da09c1.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\liblog-8290fbf24ebfc650.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\liblog-52629d6d7adb881b.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libcfg_if-b1347d601d516de4.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\liblanguage_tags-1f120d4eb4c64568.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libtypeable-37b90a66c0e53fd4.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libtraitobject-e42a82912129d07a.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libnum_cpus-2c9ebf56e8506369.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libhttparse-5fdd6843d2b90a53.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libunicase-1ff8862237f0ad7e.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\liburl-cb2f1703c2051404.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libidna-2dccc330fa120c40.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libunicode_normalization-05c479f908735c18.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libtinyvec-89542cb3fe5ec50d.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libtinyvec_macros-0d79dc173afd5963.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libunicode_bidi-25a8617db7cb03aa.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libmatches-5a438e594be46ce5.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libbase64-1d678e0cfd664cb9.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libsafemem-7dc2f1cd7551f3b9.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libbyteorder-2481812d3e7c910d.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libunicode_xid-d2b897ae1242d958.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libstate-d915de2c2db68eef.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libindexmap-3190e1dd9e2809b7.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libhashbrown-4794792eab7bd8e3.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libcookie-b22759f25ba38a69.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libtime-6373f03850cf300d.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\liblibc-d520e3667288b8da.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libpercent_encoding-ee9e622516b87769.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libsmallvec-2817ef9eba1ed532.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libpercent_encoding-dd42f0c66c80313d.rlib" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\libpear-f2f7fd587bab036d.rlib" "--start-group" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libstd-84fa96c5a6567c2a.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libpanic_unwind-2694d6b5ee3337f3.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libminiz_oxide-58e7d510a302cb06.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libadler-3b97f36d6f391442.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libobject-7f2f74160dfc1203.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libmemchr-bc62538a6b3d16cf.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libaddr2line-531fc0db5fed2ff4.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libgimli-267be56b670e3626.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libstd_detect-5efe0f28e08ac5c2.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\librustc_demangle-5cb8086f7dea7d11.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libhashbrown-26229185d2048651.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\librustc_std_workspace_alloc-5e539cff88cca34d.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libunwind-a33879496a1412b2.rlib" "-lunwind" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libcfg_if-cb8b44b450e6e086.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\liblibc-9946a456dc1d9b49.rlib" "-lc" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\liballoc-7fa17c9ac5f3bbe0.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\librustc_std_workspace_core-d23d1fdf2f9c2e82.rlib" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libcore-4927c240c984c2ab.rlib" "--end-group" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\libcompiler_builtins-ff05cc303f3269e0.rlib" "-Bdynamic" "-llibpq" "--eh-frame-hdr" "-znoexecstack" "-L" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib" "-L" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\self-contained" "-o" "D:\\projects\\project-management-system\\target\\x86_64-unknown-linux-musl\\release\\deps\\project_api-565490f9d632eaa7" "--gc-sections" "-static" "-pie" "--no-dynamic-linker" "-z" "text" "-zrelro" "-znow" "-O1" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\self-contained\\crtendS.o" "C:\\Users\\user\\.rustup\\toolchains\\nightly-2021-12-05-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-unknown-linux-musl\\lib\\self-contained\\crtn.o"
= note: rust-lld: error: unable to find library -llibpq
error: could not compile `project-api` due to previous error
What I tried
I already tried adding the libpq.dll and libpq.lib files to the x86_64-unknown-linux-musl/lib folder, but I got the same error. I tried adding it to the default target too, but it didn't work. I found out it could be a problem of some build.rs file in a library... but I don't know which one. I also tried compiling with the regular nightly toolchain.
I am trying to setup a simple flink application from scratch using Bazel. I've bootstrapped the project by running
sbt new tillrohrmann/flink-project.g8
and after that I have added some files in order for Bazel to take control of the building (i.e., migrate from sbt). This is how the WORKSPACE looks like
# WORKSPACE
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
skylib_version = "1.0.3"
http_archive(
name = "bazel_skylib",
sha256 = "1c531376ac7e5a180e0237938a2536de0c54d93f5c278634818e0efc952dd56c",
type = "tar.gz",
url = "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/{}/bazel-skylib-{}.tar.gz".format(skylib_version, skylib_version),
)
rules_scala_version = "5df8033f752be64fbe2cedfd1bdbad56e2033b15"
http_archive(
name = "io_bazel_rules_scala",
sha256 = "b7fa29db72408a972e6b6685d1bc17465b3108b620cb56d9b1700cf6f70f624a",
strip_prefix = "rules_scala-%s" % rules_scala_version,
type = "zip",
url = "https://github.com/bazelbuild/rules_scala/archive/%s.zip" % rules_scala_version,
)
# Stores Scala version and other configuration
# 2.12 is a default version, other versions can be use by passing them explicitly:
load("#io_bazel_rules_scala//:scala_config.bzl", "scala_config")
scala_config(scala_version = "2.12.11")
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
scala_repositories()
load("#io_bazel_rules_scala//scala:toolchains.bzl", "scala_register_toolchains")
scala_register_toolchains()
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_binary", "scala_test")
# optional: setup ScalaTest toolchain and dependencies
load("#io_bazel_rules_scala//testing:scalatest.bzl", "scalatest_repositories", "scalatest_toolchain")
scalatest_repositories()
scalatest_toolchain()
load("//vendor:workspace.bzl", "maven_dependencies")
maven_dependencies()
load("//vendor:target_file.bzl", "build_external_workspace")
build_external_workspace(name = "vendor")
and this is the BUILD file
package(default_visibility = ["//visibility:public"])
load("#io_bazel_rules_scala//scala:scala.bzl", "scala_library", "scala_test")
scala_library(
name = "job",
srcs = glob(["src/main/scala/**/*.scala"]),
deps = [
"#vendor//vendor/org/apache/flink:flink_clients",
"#vendor//vendor/org/apache/flink:flink_scala",
"#vendor//vendor/org/apache/flink:flink_streaming_scala",
]
)
I'm using bazel-deps for vendoring the dependencies (put in the vendor folder). I have this on my dependencies.yaml file:
options:
buildHeader: [
"load(\"#io_bazel_rules_scala//scala:scala_import.bzl\", \"scala_import\")",
"load(\"#io_bazel_rules_scala//scala:scala.bzl\", \"scala_library\", \"scala_binary\", \"scala_test\")",
]
languages: [ "java", "scala:2.12.11" ]
resolverType: "coursier"
thirdPartyDirectory: "vendor"
resolvers:
- id: "mavencentral"
type: "default"
url: https://repo.maven.apache.org/maven2/
strictVisibility: true
transitivity: runtime_deps
versionConflictPolicy: highest
dependencies:
org.apache.flink:
flink:
lang: scala
version: "1.11.2"
modules: [clients, scala, streaming-scala] # provided
flink-connector-kafka:
lang: java
version: "0.10.2"
flink-test-utils:
lang: java
version: "0.10.2"
For downloading the dependencies, I'm running
bazel run //:parse generate -- --repo-root ~/Projects/bazel-flink-scala --sha-file vendor/workspace.bzl --target-file vendor/target_file.bzl --deps dependencies.yaml
Which runs just fine, but then when I try to build the project
bazel build //:job
I'm getting this error
Starting local Bazel server and connecting to it...
ERROR: Traceback (most recent call last):
File "/Users/salvalcantara/Projects/me/bazel-flink-scala/WORKSPACE", line 44, column 25, in <toplevel>
build_external_workspace(name = "vendor")
File "/Users/salvalcantara/Projects/me/bazel-flink-scala/vendor/target_file.bzl", line 258, column 91, in build_external_workspace
return build_external_workspace_from_opts(name = name, target_configs = list_target_data(), separator = list_target_data_separator(), build_header = build_header())
File "/Users/salvalcantara/Projects/me/bazel-flink-scala/vendor/target_file.bzl", line 251, column 40, in list_target_data
"vendor/org/apache/flink:flink_clients": ["lang||||||scala:2.12.11","name||||||//vendor/org/apache/flink:flink_clients","visibility||||||//visibility:public","kind||||||import","deps|||L|||","jars|||L|||//external:jar/org/apache/flink/flink_clients_2_12","sources|||L|||","exports|||L|||","runtimeDeps|||L|||//vendor/commons_cli:commons_cli|||//vendor/org/slf4j:slf4j_api|||//vendor/org/apache/flink:force_shading|||//vendor/com/google/code/findbugs:jsr305|||//vendor/org/apache/flink:flink_streaming_java_2_12|||//vendor/org/apache/flink:flink_core|||//vendor/org/apache/flink:flink_java|||//vendor/org/apache/flink:flink_runtime_2_12|||//vendor/org/apache/flink:flink_optimizer_2_12","processorClasses|||L|||","generatesApi|||B|||false","licenses|||L|||","generateNeverlink|||B|||false"],
Error: dictionary expression has duplicate key: "vendor/org/apache/flink:flink_clients"
ERROR: error loading package 'external': Package 'external' contains errors
INFO: Elapsed time: 3.644s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
Why is that? Anyone can help? It would be great having detailed instructions and project templates for Flink/Scala applications using Bazel. I've put everything together in the following repo: https://github.com/salvalcantara/bazel-flink-scala, feel free to send a PR or whatever.
I have this BUILD file:
package(default_visibility = ["//visibility:public"])
load("#npm_bazel_typescript//:index.bzl", "ts_library")
ts_library(
name = "lib",
srcs = glob(
include = ["**/*.ts"],
exclude = ["**/*.spec.ts"]
),
deps = [
"//packages/enums/src:lib",
"//packages/hello/src:lib",
"#npm//faker",
"#npm//#types/faker",
"#npm//express",
"#npm//#types/express",
],
)
load("#io_bazel_rules_docker//nodejs:image.bzl", "nodejs_image")
nodejs_image(
name = "server",
data = [":lib"],
entry_point = ":index.ts",
)
load("#io_bazel_rules_docker//container:container.bzl", "container_push")
container_push(
name = "push_server",
image = ":server",
format = "Docker",
registry = "gcr.io",
repository = "learning-bazel-monorepo/server",
tag = "dev",
)
load("#io_bazel_rules_k8s//k8s:object.bzl", "k8s_object")
k8s_object(
name = "k8s_deploy",
kind = "deployment",
namespace = "default",
template = ":server.yaml",
images = {
"deploy_server:do_not_delete": ":server"
},
)
But when running the k8s_deploy rule I get this error:
INFO: Analyzed target //services/server/src:k8s_deploy (1 packages loaded, 7 targets configured).
INFO: Found 1 target...
Target //services/server/src:k8s_deploy up-to-date:
bazel-bin/services/server/src/k8s_deploy.substituted.yaml
bazel-bin/services/server/src/k8s_deploy
INFO: Elapsed time: 0.276s, Critical Path: 0.01s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
2019/12/22 07:45:14 Unable to publish images: unable to publish image deploy_server:do_not_delete
The lib, server and push_server rules work fine. So I don't know what's the issue as there is no specific error message.
A snippet out of my server.yaml file:
spec:
containers:
- name: server
image: deploy_server:do_not_delete
You can try it yourself by running bazel run //services/server/src:k8s_deploy on this repo: https://github.com/flolude/minimal-bazel-monorepo/tree/de898eb1bb4edf0e0b1b99c290ff7ab57db81988
Have you pushed images using this syntax before?
I'm used to using the full repository tag for both the server.yaml and the k8s_object images.
So, instead of just "deploy_server:do_not_delete", try "gcr.io/learning-bazel-monorepo/deploy_server:do_not_delete".
I tried to configure PostgreSQL as node's database in Corda 3.0 and in Corda 4.0.
I have added following things in build.gradle file. (Testdb1 is Database name. I have tried with postgres also)
node{
...
// this part i have added
extraConfig = [
jarDirs: ['path'],
'dataSourceProperties': [
'dataSourceClassName': 'org.postgresql.ds.PGSimpleDataSource',
'"dataSource.url"' : 'jdbc:postgresql://127.0.0.1:5432/Testdb1',
'"dataSource.user"' : 'postgres',
'"dataSource.password"': 'admin#123'
],
'database': [
'transactionIsolationLevel': 'READ_COMMITTED'
]
]
// till here
}
this part in reference.conf file
dataSourceProperties = {
dataSourceClassName = org.postgresql.ds.PGSimpleDataSource
dataSource.url = "jdbc:postgresql://127.0.0.1:5432/Testdb1"
dataSource.user = postgres
dataSource.password = "admin#123"
}
database = {
transactionIsolationLevel = "READ_COMMITTED"
}
jarDirs = ["path"]
I got the follwing Error while deploying the nodes:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':java-source:deployNodes'.
In node-info-gen.log file it showed the CAPSULE EXCEPTION. Then I updated my JDK to 8u191 but still got the same error.
I have go-through the followings to get the things done. One can get reference from here.
https://docs.corda.net/node-database.html ,
https://github.com/corda/corda/issues/4037 ,
How can the Corda node be extended to work with databases other than H2?
You need to add those properties in node.conf in each of your corda nodes. After you do "deployNodes"
After you add these properties in node.conf file , just run the corda jar . It will automatically start . But before that you need to create the tables (The migration to other DBs is already provided in corda documentation)
I have added following things in .conf file of each node and one referene.conf file. I have given all the privileges for the user postgres which are mentioned in corda documentation.
https://docs.corda.r3.com/node-database.html
(previously I used postgresql-42.2.5.jar file but that didn't work so I used one downgrade version of it postgresql-42.1.4.jar.
one can download jar files from https://jdbc.postgresql.org/download.html )
After deployed Nodes successfully add following things:
dataSourceProperties = {
dataSourceClassName = org.postgresql.ds.PGSimpleDataSource
dataSource.url = "jdbc:postgresql://127.0.0.1:5432/Testdb1"
dataSource.user = postgres
dataSource.password = "admin#123"
}
database = {
transactionIsolationLevel = "READ_COMMITTED"
}
jarDirs = ["path"]
(path = jar file's location) after adding this configuration run file called runnodes.bat