AOSP: Static Prebuilt Modules and their dependents - android-source

I have prebuilt binaries of opencv as static libraries built using the android NDK r16 for arm64-v8a & armeabi-v7a.
I'm trying to use these binaries as is in an AOSP build for the same above two architectures.
The output of the opencv prebuild seems to have separated the opencv specific .a files from the 3rdparty .a files. I gather this means to link successfully I will have to include both the opencv library (eg libopencv_core.a) as well as the third-party dependency (libtbb.a). I'm not sure how to specify the 3rdparty dependency in the static prebuild module description.
This is folder structure of the opencv output
sdk
└── native
├── 3rdparty
│ └── libs
│   ├── arm64-v8a
│   │   ├── libcpufeatures.a
│   │   └── libtbb.a
│   ├── armeabi-v7a
│   │   ├── libcpufeatures.a
│   │   └── libtbb.a
│   ├── x86
│   │   ├── libcpufeatures.a
│   │   ├── libittnotify.a
│   │   └── libtbb.a
│   └── x86_64
│   ├── libcpufeatures.a
│   ├── libittnotify.a
│   └── libtbb.a
├── jni
│ ├── android.toolchain.cmake
│ ├── include
│ │ ├── opencv
│ │ │ ├── cvaux.h
| | | ...
│ │ └── opencv2
│ │ ├── core
└── libs
├── arm64-v8a
│ ├── libopencv_core.a
│ ├── libopencv_features2d.a
│ ├── libopencv_imgproc.a
│ └── libopencv_video.a
├── armeabi-v7a
│ ├── libopencv_core.a
│ ├── libopencv_features2d.a
│ ├── libopencv_imgproc.a
│ └── libopencv_video.a
├── x86
│ ├── libopencv_core.a
│ ├── libopencv_features2d.a
│ ├── libopencv_imgproc.a
│ └── libopencv_video.a
└── x86_64
├── libopencv_core.a
├── libopencv_features2d.a
├── libopencv_imgproc.a
└── libopencv_video.a
The dependencies are in the 3rdparty folder.
I created an Android.mk file that makes PREBUILDs out of all the opencv libraries, but I'm not sure how to add the dependencies to the prebuilt module. This is my current Android.mk file
ROOT_DIR := $(call my-dir)
LOCAL_PATH := $(ROOT_DIR)/core-libs/opencv/Android/sdk/native/libs
include $(CLEAR_VARS)
LOCAL_MODULE := opencv_core_prebuilt
LOCAL_MULTILIB := both
LOCAL_MODULE_CLASS := STATIC_LIBRARIES
LOCAL_MODULE_SUFFIX := .a
LOCAL_SRC_FILES_64 := arm64-v8a/libopencv_core.a
LOCAL_SRC_FILES_32 := armeabi-v7a/libopencv_core.a
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_PATH)/../jni/include
LOCAL_EXPORT_CFLAGS_64 := -L$(LOCAL_PATH)/../3rdparty/libs/arm64-v8a -llibtbb.a -llibcpufeatures.a
LOCAL_EXPORT_CFLAGS_32 := -L$(LOCAL_PATH)/../3rdparty/libs/armeabi-v7a -llibtbb.a -llibcpufeatures.a
include $(BUILD_PREBUILT)
include $(CLEAR_VARS)
LOCAL_MODULE := opencv_features2d_prebuilt
LOCAL_MULTILIB := both
LOCAL_MODULE_CLASS := STATIC_LIBRARIES
LOCAL_MODULE_SUFFIX := .a
LOCAL_SRC_FILES_64 := arm64-v8a/libopencv_features2d.a
LOCAL_SRC_FILES_32 := armeabi-v7a/libopencv_features2d.a
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_PATH)/../jni/include
include $(BUILD_PREBUILT)
include $(CLEAR_VARS)
LOCAL_MODULE := opencv_imgproc_prebuilt
LOCAL_MULTILIB := both
LOCAL_MODULE_CLASS := STATIC_LIBRARIES
LOCAL_MODULE_SUFFIX := .a
LOCAL_SRC_FILES_64 := arm64-v8a/libopencv_imgproc.a
LOCAL_SRC_FILES_32 := armeabi-v7a/libopencv_imgproc.a
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_PATH)/../jni/include
include $(BUILD_PREBUILT)
include $(CLEAR_VARS)
LOCAL_MODULE := opencv_video_prebuilt
LOCAL_MULTILIB := both
LOCAL_MODULE_CLASS := STATIC_LIBRARIES
LOCAL_MODULE_SUFFIX := .a
LOCAL_SRC_FILES_64 := arm64-v8a/libopencv_video.a
LOCAL_SRC_FILES_32 := armeabi-v7a/libopencv_video.a
LOCAL_EXPORT_C_INCLUDE_DIRS := $(LOCAL_PATH)/../jni/include
include $(BUILD_PREBUILT)
When I try to link to the opencv prebuilds sure enough I get undefined references for calls into libtbb.
How do I specify the libtbb dependency and how to do I make sure the corresponding 3rdparty .a file is linked together.
Can I add it to the LOCAL_SRC_FILES variable ?

Related

How to change the default directory structure of dh_make so that dpkg-buildpackage does not throw any errors

I am trying to create a debian package for a postgreSQL extension Apache-age release 1.1.1 and created the directory structure using dh_make command.
The directory structure is as follows:
age-1.1.1 (project root)
├── debian
│   ├── changelog
│   ├── compat
│   ├── control
│   ├── docs
│   ├── examples
│   ├── links
│   ├── manpages
│   ├── menu
│   ├── postinst
│   ├── postrm
│   ├── preinst
│   ├── prerm
│   ├── rules
│   ├── source
│   └── watch
├── src
└── Makefile
The dpkg-buildpackage -b when run from project-root folder it looks for debian folder, then reads the rule file, then reads the Makefile located in the project root to build the package.
I want to change the directory structure to the following:
.project root
├── packaging
│ ├── debian
│ │ ├── control
│ │ ├── control.in
│ │ ├── changelog
│ │ ├── copyright
│ │ ├── pgversions
│ │ ├── rules
│ │ └── ...
│ └──
├── src
├── LICENSE
├── README.md
├── Makefile
└── ...
I want to change the directory structure so that the dpkg-buildpackage -b command can be run from the packaging folder and it should build the package.
Inside your Makefile
Modify the install paths accordingly. It should point to your packaging/debian/* where * is the filename.
This way the Makefile can point to the correct file path target inside the new folder structure.
I'm not sure if this is the best way to do this but it's working for me:
Here are the steps:
First run the dh_make_pgxs command from the project root directory.
Create a packaging directory in the project root and move the debian directory created in step 1 to this directory along with the Makefile, age.control and the age--1.1.1.sql.
Your file structure should look like this:
.project root
├── packaging
│   ├── debian
│   │   ├── control
│   │   ├── control.in
│   │   ├── changelog
│   │   ├── copyright
│   │   ├── pgversions
│   │   ├── rules
│   │   └── ...
│   ├── age--1.1.1.sql
│   ├── age.control
│   ├── Makefile
│   └── ...
├── src
├── LICENSE
├── README.md
└── ...
Change the file paths in the Makefile like:
src/backend/age.o should be ../src/backend/age.o.
./tools/ should be ./../tools/.
and so on.
Now you can simply run the dpkg-buildpackage -b command from the packaging directory to build the debian package.
Note: In step 1 we are running dh_make_pgxs in the project root first, this is to make sure that the project name in the control files and the version in the changelog file are correct. In this case the name/source in control, control.in & changelog files should be apache-age and the version number in changelog file should be 1.1.1-1.
Alternatively, you can run the command from the packaging directory and manually change the name and version in the control and changelog files.

Scala share resources folder to open a file

This is my folder structure and I am trying to load the "grammar.txt" file from resources folder but I get not found error.
val source = Source.fromResource("grammar.txt")
Folder structure:
➜ cfg-tools tree -L 4
.
├── build.sbt
├── src
│   ├── main
│   │   └── scala
│   │   ├── Builer.scala
│   │   ├── Driver.scala
│   │   ├── tokens.scala
│   │   └── Tools.scala
│   ├── resources
│   │   └── grammar.txt
build.sbt
name := "cfg-tools"
version := "0.1"
scalaVersion := "3.0.2"
Compile / unmanagedResourceDirectories += sourceDirectory.value / "resources"
You don't need the custom SBT configuration: just use the standard place for resources which is src/main/resources (note that it's in main subfolder compared to your current structure).

Factoring out common components when kustomizing kubernetes manifests

I'm using kustomize binary to craft specific kubernetes deployment yaml files.
{Version:kustomize/v3.8.1
GitCommit:0b359d0ef0272e6545eda0e99aacd63aef99c4d0
BuildDate:2020-07-16T00:58:46Z GoOs:linux GoArch:amd64}
Here is my directory structure:
overlays
├── parser
│   ├── dev
│   │   └── nonsec
│   │   ├── dev-patches-parsers-tests.yaml
│   │   ├── dev-patches-parsers.yaml
│   │   └── kustomization.yaml
│   ├── prod
│   │   ├── nonsec
│   │   │   ├── kustomization.yaml
│   │   │   ├── prod-patches-parsers-tests.yaml
│   │   │   └── prod-patches-parsers.yaml
│   │   ├── sec
│   │   │   ├── kustomization.yaml
│   │   │   ├── prod-patches-parsers-sec-tests.yaml
│   │   │   ├── prod-patches-parsers-sec.yaml
│   │   │   ├── prod-patches-parsers-tests.yaml
│   │   │   └── prod-patches-parsers.yaml
│   │   └── v3r
│   │   └── empty.txt
│   ├── stage
│   └── suit
Notice that overlays/parser/prod/nonsec and /overlays/parser/prod/sec contain two identical patch file sets. I want to factor out the common files and push them up one level under prod - I don't want copies of same patches in multiple dirs. I want both nonsec and sec builds to use the same set of prod-patches but I don't know how to do that with kustomize. I've tried to put them in the dir above, but kustomize doesn't allow any references to patch files in the directories above - which is what I NEED to do.
How can I avoid duplication of these prod-patches*.yaml files in multiple dirs?!
Here is my nonsec kustomization.yaml
# Use this as the base code
resources:
- ../../../../base
# Decorate the base code with the following components
components:
- ../../../../components/common-all
- ../../../../components/common-prod
- ../../../../components/parser
# Then finally, patch the results from above with this:
patchesStrategicMerge:
- prod-patches-parsers.yaml
- prod-patches-parsers-tests.yaml
And this is my sec kustomization.yaml:
# Use this as the base code
resources:
- ../../../../base
# Decorate the base code with the following components
components:
- ../../../../components/common-all
- ../../../../components/common-prod
- ../../../../components/parser
# Then finally, patch the results from above with this:
patchesStrategicMerge:
- prod-patches-parsers.yaml
- prod-patches-parsers-tests.yaml
- prod-patches-parsers-sec.yaml
- prod-patches-parsers-sec-tests.yaml
You could do the following:
overlays
├── parser
│ ├── dev
│ │ └── nonsec
│ │ ├── dev-patches-parsers-tests.yaml
│ │ ├── dev-patches-parsers.yaml
│ │ └── kustomization.yaml
│ ├── prod
│ │ ├── kustomization.yaml
│ │ ├── prod-patches-parsers-tests.yaml
│ │ ├── prod-patches-parsers.yaml
│ │ ├── nonsec
│ │ │ └── kustomization.yaml
│ │ ├── sec
│ │ │ ├── kustomization.yaml
│ │ │ ├── prod-patches-parsers-sec-tests.yaml
│ │ │ └── prod-patches-parsers-sec.yaml
│ │ └── v3r
│ │ └── empty.txt
with overlays/parser/prod/kustomization.yaml:
resources:
- ../../../../base
- ../../../../components/common-all
- ../../../../components/common-prod
- ../../../../components/parser
patchesStrategicMerge:
- prod-patches-parsers.yaml
- prod-patches-parsers-tests.yaml
overlays/parser/prod/nonsec/kustomization.yaml:
resources:
- ../
overlays/parser/prod/sec/kustomization.yaml:
resources:
- ../
patchesStrategicMerge:
- prod-patches-parsers.yaml
- prod-patches-parsers-tests.yaml
The overall directory structure is a bit complex and you might want to simplify or flatten it.

Adding a custom resource directory to SBT's default unmanagedResourceDirectories

I have two SBT projects as outlined below.
├── Project 1
│ │
│ └── src
│ └── main
│ ├── scala
│ │ └── com
│ │ └── xyz
│ │ └── <*.scala>
│ └── resources
│ └── <Typesafe & Log4J config files>
│
│
└── Project 2
│
├── src
│ └── main
│ ├── scala
│ │ └── com
│ │ └── xyz
│ │ └── <*.scala>
│ └── resources
│ └── <Typesafe & Log4J config files>
│
├── resources
│ └── <JS, HTML, Image files etc.>
├── other-dir-1
│
├── other-dir-2
│
└── other-dir-3
Compiling Project 1 (actually running SBT exportedProducts task) produces the following directory structure. unmanagedResourceDirectories points to Project1/src/main/resources. I believe this is the default resourceDirectory (as mentioned in Customizing Paths). In other words, files in default resource directory are automatically added by exportedProducts
├── Project 1
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
└── <Typesafe & Log4J config files>
For Project 2, I want the following directory structure to be produced by exportedProducts.
├── Project 2
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
├── <Typesafe & Log4J config files>
│
└── resources
└── <JS, HTML, Image files etc.>
To do this I added the following to SBT build file in the appropriate project definition.
unmanagedResourceDirectories in Compile += baseDirectory.value
excludeFilter in unmanagedResources := HiddenFileFilter || "other-dir-*"
includeFilter in unmanagedResources :=
new SimpleFileFilter(_.getCanonicalPath.startsWith((baseDirectory.value / "resources").getCanonicalPath))
This correctly includes resources directory but doesn't include the files from Project2\src\main\resources. The target directory looks like the
├── Project 2
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
└── resources
└── <JS, HTML, Image files etc.>
Adding a custom resource directory in some way masks the content of the default resource directory. I tried something along the lines of what was mentioned in this SO post but wasn't successful.
The other thing that I tried was to set unmanagedResourceDirectories in Compile += baseDirectory.value / "resources" and remove both includeFilter and excludeFilter. This adds the files from Project2\src\main\resources correctly but adds the files & directories from Project2\resources directly to Project2\target\scala-2.10\classes. The target directory looks like the following
├── Project 2
└── target
└── scala-2.10
└── classes
├── com
│ └── xyz
│ └── <*.class>
├── <Typesafe & Log4J config files>
│
└── <JS, HTML, Image files etc.>

Standard CoffeeScript output layout

Is there any de facto standard for the filesystem output layout of compiled CoffeeScript output?
Or: where should the .js and .map files end up?
I currently have a file watcher1 compiling the output to ./grounds/[whatever] to keep the source folder clean, so I end up with something like:
index.html
/js
├──foo.js
├──bar.js
/coffee
├──a.coffee
├──b.coffee
├──/grounds
│ ├──a.js
│ ├──a.map
│ ├──b.js
│ ├──b.map
├──/some-module
│ ├──c.coffee
│ ├──/grounds
│ │ ├──c.js
│ │ ├──c.map
Just curious if I missed the boat on some existing standard for the output file layout, or if folks generally just let them all be siblings in the same folder and are happy with that.
1: Pycharm
I don't think they is any standard, but for development, I use the same structure for the JS than for the coffee:
eg:
coffee
├── AdvancedStatsModule.coffee
├── board
│   ├── Board.coffee
│   ├── Card.coffee
├── controllers
│   ├── directives.coffee
│   ├── factory.coffee
│   ├── filters.coffee
│   ├── ListController.coffee
public/js
├── AdvancedStatsModule.js
├── board
│   ├── Board.js
│   ├── Card.js
├── controllers
│   ├── directives.js
│   ├── factory.js
│   ├── filters.js
│   ├── ListController.js
Advantages for this:
JS and coffee are not mixed
The coffee directory is not public
For production, I usually concat and minify my files anyway.
Update:
I use gulp for that. My Gulpfile:
var watch= require('gulp-watch');
var coffee= require('gulp-coffee');
gulp.src(paths.coffee)
.pipe(watch(function(files) {
return files.pipe(coffee())
.pipe(gulp.dest(paths.js));
}));