Cleanly handling /usr/local/ with Swift package manager and libevent - swift

I have 2 dependencies in my project libevent and libressl. Both of which are installed locally ( respectively under /usr/local/include and /usr/local/opt/libressl/include )
What I am looking for to achieve is for SPM to automatically understand to search in those directories.
I know I can pass flags to swift build to achieve this; but my ultimate goal is that I can properly generate xcode projects from the command line without having to constantly add custom build flags in Xcode.
I'm pretty sure it is possible, since I do not have to enter the custom settings for PostgreSQL.
Swift-tools version is at 4.0.x
Package.swift for reference:
// swift-tools-version:4.0
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "CEvent",
providers: [
.brew(["libevent"]),
.apt(["libevent-dev"])
],
products: [
// Products define the executables and libraries produced by a package, and make them visible to other packages.
.library(
name: "CEvent",
targets: ["CEvent"]),
],
dependencies: [
],
targets: [
.target(
name: "CEvent",
dependencies: []
),
]
)
Module map:
module CEvent [system] {
header "shim.h"
link "event"
export *
}
And my current build script ( build.sh ):
#!/usr/local/bin/fish
swift build -Xcc -O0 -Xcc -fblocks -Xswiftc -lbcrypt -Xswiftc -I/usr/local/include -Xswiftc -L/usr/local/lib -Xswiftc -ltls -Xswiftc -lcrypto -Xswiftc -lssl -Xswiftc -L/usr/local/opt/postgresql/lib -Xswi$
As for the reason that I want this. If I add/update/remove dependencies in swift I want to generate a new xcode project, and not have to fix its settings on respective build machines; as well as apt/ubuntu /usr/lib instead.

What you've found out, and documented in your answer, is a good start but not the full story. Yes, SwiftPM uses pkg-config to determine where certain libraries are installed. Yes, SwiftPM uses the pkgConfig name which it'll pass on to pkg-config. However the search paths are a bit more involved. On macOS it uses the following list as a base search path:
/usr/local/lib/pkgconfig
/usr/local/share/pkgconfig
/usr/lib/pkgconfig
/usr/share/pkgconfig
PKG_CONFIG_PATH environment variable
However SwiftPM doesn't use the pkg-config command, but instead parses .pc files directly. By setting the pkgConfig parameter on your package, it knows what filename to look for in the paths listed above. And, for the example in your answer, the story stops here. If there's a libevent.pc file found it parses that file, and any flags returned are passed on to the compiler and linker.
However if you were to define package providers, e.g.:
providers: [
.Brew("libsodium"),
.Apt("libsodium-dev")
]
Then SwiftPM adds additional search paths depending on the package provider for the platform it is building for. Continuing the example of macOS, SwiftPM will run brew --prefix. If this returns a path, the following path is added as a additional search path:
[brewPrefix]/opt/[packageName]/lib/pkgconfig
In my example of libsodium, SwiftPM is now able to infer the location of the library without requiring brew link or symlinks at all. In my verbose build output it lists the libsodium library path in my cellar: -L/usr/local/Cellar/libsodium/1.0.11/lib.

Alright so the thing I ignored from analysing other projects ( IBM-Swift/CLibpq in particular ) seems to be making use of the tool pkg-config which is not something I personally ever touched before.
pkg-config looks in /usr/lib/pkgconfig /usr/share/pkgconfig and the local variants for config files used in during the build process.
Inside Package.swift, after the name parameter you need to insert something for example:
let package = Package(
name: "CEvent",
pkgConfig: "libevent",
Some caveats I discovered with this:
The bcrypt library I am using does not have a full fletched install or build inside the makefile so I compiled it using the new options found in swift4 PM instead found here: BCrypt example on github and the Swift docs for more help here: SPM API Redesign
LibreSSL found in Homebrew will not install its pkgconfig on the system; so it is easiest or in my eyes best maintenance wise to either manually add it or to compile LibreSSL-portable from source.
Overall great learning experience for me today.

Related

No such module in a Swift Package using Xcode - package listed in dependencies

I create a blank template package:
> swift package init --name Temp
> open Package.swift
Xcode Version 13.2.1 (13C100) opens the package.
I add a dependency to the package.
dependencies: [
.package(url: "https://github.com/johnsundell/publish.git", from: "0.7.0")
],
Xcode > Product > Build succeeds at this point.
I edit Temp/Sources/Temp/Temp.swift to insert the first line the package that is defined in dependencies.
import Publish
A build now generates the following error:…/Temp/Sources/Temp/Temp.swift:1:8: error: no such module 'Publish'.
I feel certain this is an Apple bug. Or I could be missing something.
There are several posts about this problem when there is an xcodeproj and the additional structure that provides. Some of them hint at workarounds that help some people.
Has anyone seen this and/or know of how to resolve it?
Apple's Creating a Standalone Swift Package with Xcode document doesn't provide any insight.
thanks for the chatter in the comments, #Larme & #koen, it helped
The issue was user error (and/or a documentation lapse). Living on the (bleeding) edge.
Sometimes updates from changes are slow or require a clean or a relaunch.
Xcode auto-generates Schemes from the targets defined in your package. My build was targeting MyTarget.
Two things were missing:
name: "Publish" was not included in the package dependency - it's needed so you can reference it below (or maybe this can be derived, it's hard to tell because of Xcode refresh issues), and
a reference is needed in the dependencies for each target using the package-dependency, i needed to add dependencies: ["Publish"] in the related target
dependencies: [
.package(name: "Publish", url: "https://github.com/johnsundell/publish.git", from: "0.7.0")
],
…
targets: [
.target(
name: "MyTarget",
dependencies: ["Publish"]),
]

How can I enable ABI stability for a SwiftPM package?

Given my bog standard Package Description of
let package = Package(
name: "MyLib",
products: [
.library(name: "MyLib", targets: ["MyLib"]),
],
dependencies: [],
targets: [
.target(
name: "MiniRxSwift",
dependencies: [],
swiftSettings: [
.define("<see below>")
]),
...
I'm trying to get swiftpm to pass the -enable-library-evolution flag through to swiftc, but I've been unsuccessful.
Using swiftSettings of .define("-enable-library-evolution"), I get a compile error which states
"error: conditional compilation flags must be valid Swift identifiers (rather than '-enable-library-evolution')"
I get the same error if I omit the leading hyphen e.g. `.define("enable-library-evolution")
I've tried the Xcode setting of .define("BUILD_LIBRARIES_FOR_DISTRIBUTION") which doesn't result in a compile error, but also doesn't result in the flag getting set, it instead results in -DBUILD_LIBRARIES_FOR_DISTRIBUTION on the command line for swiftc, which doesn't do anything.
After a bit more research, I worked out that .define in swiftSettings is hardwired to produce things with -D - hence it's name.
Instead I needed to use unsafeFlags, which does indeed result in the flag getting passed correctly to the swift compiler:
swiftSettings: [
.unsafeFlags(["-enable-library-evolution"])
]
BUT then when I attempt to consume this package, Xcode fails to load the package, with the error:
The package product 'MyLib' cannot be used as a dependency of this target because it uses unsafe build flags.
If I can't enable library evolution without unsafe build flags, and I can't use unsafe build flags in a library, then what can I do? What is the point of having unsafeFlags if you can't use any libraries which set them?
Why do you need ABI stability if you have access to the package? You should avoid it if possible. That said, you should be able to use unsafeFlags with editable packages (ones you have dragged into the project) but not ones you include the normal way via URL.
If you need to vend someone a swift package as a binary you will need to build your package as an XCFramework and then you can put that in a place where SwiftPM can depend on it. That means it can be hosted at a static place where you give the URL or it can be embedded in the git repo with the Package.swift manifest that you have for vending the package. The only tool I know of to help with this is found here which works for Swift only package when I have used it.

Cant run/find Executable Swift Package

Bottom line I am trying to run an executable from a swift package I created and linked to my target.
I think I'm missing something here, so I've created an executable swift package with the content
// swift-tools-version:5.2
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "AutoLocalized",
platforms: [
.macOS(.v10_15), .iOS(.v12)
],
products: [
.executable(name: "autolocalized", targets: ["AutoLocalized"])
],
targets: [
.target(name: "AutoLocalized", dependencies: []),
.testTarget(name: "AutoLocalizedTests", dependencies: ["AutoLocalized"]),
]
)
so as I understand adding the ".executable" outputs a file I can then run via Xcode.
I am trying to run a build phase script using that output but I dont know how to access it.
When I try to add dependencies to my target I can find the "autolocalized" executable
but building the project I get an error:
After a lot of searching I found A way, maybe not the best but it workes.
So what I did is adding the package via SPM but without a dependency to any target, that means I had to make sure I build the package manually, finding it in the Build folder and using it directly from there, this was the only way I found to run mac os scripts using build phases in Xcode, I am positive others tried but no one found a good answer, so here it is.

How to generate a framework from a SwiftPM-generated XCode project?

I used SwiftPM to set up an XCode project for a framework; based on
// swift-tools-version:4.0
import PackageDescription
let package = Package(
name: "MyThing",
products: [
.library(
name: "MyThing",
targets: ["MyThing"]),
],
dependencies: [
],
targets: [
.target(
name: "MyThing",
dependencies: [])
)
I ran swift package generate-xcodeproj and then pod install (based on a Podfile whose content shouldn't matter). I obtain MyThing.xcworkspace.
Now I figured that
xcodebuild -workspace MyThing.xcworkspace -scheme MyThing clean build
should create the .framework -- but it doesn't, only a binary file appears. I suspect some automatism is at work here since the source folder contains a file named main.swift, among others.
What do I have to do to get the framework built?
I need to script the whole process, to please no manual workarounds.
As of Swift 4.0 swift package generate-xcodeproj doesn't automatically generate schemes for all targets, but still makes those targets accessible from Xcode.
If creating a new scheme manually once is acceptable, you can do so and add your framework target as a scheme build target.
Otherwise, new schemes can be created programmatically with libraries like xcodeswift/xcproj, which allow you to parse a newly generated Xcode project and generate a new scheme with its XCSharedData class.
After that new framework scheme is created you can run the build with xcodebuild -workspace MyThing.xcworkspace -scheme MyThingFramework clean build.
Mixing script files and framework code in one target seems to confuse SwiftPM. Moving main.swift out of the source folder clears up things.
Scheme MyThing-Package creates all build products as specified in Package.swift, including the framework.
If you want to have the script files in the same XCode Workspace (for editing convenience), put them in their own target. You can even create an executable product (which won't create anything useful since an executable can't link to dynamic frameworks).

Swift Package Manager throws nonWhitelistedFlags error

The problem occurs with system modules where it's necessary to use pkgConfig and pkgConfig contains flag definitions.
ImageMagick (homebrewed)
I create two packages: CMagicWand, type system-module
module.modulemap
module CMagickWand [system] {
header "/usr/local/Cellar/imagemagick/7.0.5-0/include/ImageMagick-7/MagickWand/MagickWand.h"
link "MagickWand"
export *
}
Package.swift
import PackageDescription
let package = Package(
name: "CMagickWand",
pkgConfig: "MagickWand"
)
Then I try to consume it from the package MagicWand type library
Package.swift
import PackageDescription
let package = Package(
name: "MagickWand",
dependencies: [
.Package(url: "../CMagickWand", majorVersion: 1)
]
)
pkgConfig MagickWand.pc
prefix=/usr/local/Cellar/imagemagick/7.0.5-0
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include/ImageMagick-7
includearchdir=/usr/local/Cellar/imagemagick/7.0.5-0/include/ImageMagick-7
libname=MagickWand-7.Q16HDRI
Name: MagickWand
Description: MagickWand - C API for ImageMagick (ABI Q16HDRI)
URL: https://www.imagemagick.org
Version: 7.0.5
Requires: MagickCore
*Cflags: -I${includearchdir} -I${includedir} -DMAGICKCORE_HDRI_ENABLE=1 -DMAGICKCORE_QUANTUM_DEPTH=16*
Libs: -L${libdir} -l${libname}
Libs.private: -L${libdir} -l${libname} -L/usr/local/opt/freetype/lib -lfreetype -L/usr/local/Cellar/xz/5.2.3/lib -llzma -lbz2 -lz -lltdl -lm -lm
With this setup I execute swift build for the second package and the output is following
Cloning /bla-bla-bla/Libraries/CMagickWand
HEAD is now at 30ed4b4 Initial commit
Resolved version: 1.0.0
error: nonWhitelistedFlags("Non whitelisted flags found: [\"-DMAGICKCORE_HDRI_ENABLE=1\", \"-DMAGICKCORE_QUANTUM_DEPTH=16\", \"-DMAGICKCORE_HDRI_ENABLE=1\", \"-DMAGICKCORE_QUANTUM_DEPTH=16\"] in pc file MagickWand")
I tried to remove problematic CFlags from the corresponding .pc file and this doesn't help, even if I figure out how to remove them (I always can create my own .pc file) I don't find it sustainable. These flags are there for a reason.
I have the same problem with mysqlclient on my target system (Ubuntu), the problem is not reproducible on OS X but it doesn't help me:
error: nonWhitelistedFlags("Non whitelisted flags found: [\"-fabi-version=2\", \"-fno-omit-frame-pointer\"] in pc file mysqlclient")
The error comes from the func whitelist (https://github.com/apple/swift-package-manager/blob/master/Sources/PackageLoading/Module%2BPkgConfig.swift) and I don't see any way how to enhance the list during runtime. I don't believe that I'm the only one who struggles with this limitation but I can't find a workaround for a few days already.
I solved the problem with the following parameters:
swift build -Xcc -I/usr/local/include/ImageMagick-7/MagickWand/ -Xcc -I/usr/local/include/ImageMagick-7/ -Xcc -DMAGICKCORE_HDRI_ENABLE=0 -Xcc -DMAGICKCORE_QUANTUM_DEPTH=16 -Xlinker -L/usr/local/lib
I think you have to adjust the paths.
module.modulemap:
module CMagickWand [system] {
header "shim.h"
header "/usr/local/include/ImageMagick-7/MagickWand/MagickWand.h"
link "MagickWand-7.Q16HDRI"
export *
}
link has the value from pkgConfig MagickWand.pc libname.
Package.swift:
import PackageDescription
let package = Package(
name: "CMagickWand"
)
shim.h (don't know if this file is needed):
#include <stdio.h>
I contacted the author of this code and the reply is following:
The problem here is that we don't allow all flags from a pkg config
file because SwiftPM can't reason about them. We have build settings
proposal coming up soon which will solve these issues. For now, you can
manually pass the flags using swift build -Xcc -Xswiftc
-Xld
So, the only proper way to solve it, for the time being, is to replace flags from .pc files, specify them explicitly while building your project where these packages are imported and pray that those flags will never interfere in case you have several dependencies.
I created copies for .pc files (yes, files, there are dependencies inside), removed flags and created new links from /usr/local/lib/pkgconfig since I don't want this change to have any side-effects. Works flawlessly with ImageMagick on OS X, will try later on Ubuntu.
Thank you, Ankit!