AWS CodeBuild S3 cache for swift lambda - swift

Question

So I have some aws swift lambdas that I deployed via sam deploy. That works fine.
The swift lambda looks like this: aws-samples
I am currently in the middle of building a CI/CD pipeline using Codepipeline and Codebuild. My Codebuild project executes the following buildspec.yml and is configured to cache to S3:
version: 0.2
phases:
build:
commands:
- sam build
- sam package -t template.yml --s3-bucket bucketName --output-template-file packaged.yaml
artifacts:
files:
- packaged.yaml
cache:
paths:
- ".aws-sam/**/*"
To build the swift lambdas sam buildexecuted the following makefile as the function in template.yml is set to: BuildMethod: makefile
Makefile:
### Add functions here and link them to builder-bot format MUST BE "build-FunctionResourceName in template.yaml"
build-ExpiredMediaItemProcessorLambda: builder-bot
builder-bot:
$(eval $#PRODUCT = $(subst build-,,$(MAKECMDGOALS)))
$(eval $#BUILD_DIR = $(PWD)/.aws-sam/build-$($#PRODUCT))
$(eval $#STAGE = $($#BUILD_DIR)/lambda)
$(eval $#ARTIFACTS_DIR = $(PWD)/.aws-sam/build/$($#PRODUCT))
# prep directories
mkdir -p $($#BUILD_DIR)/lambda $($#ARTIFACTS_DIR)
# Compile application
swift build --product $($#PRODUCT) -c release --build-path $($#BUILD_DIR)
# copy deps
ldd '/$($#BUILD_DIR)/release/$($#PRODUCT)' | grep swift | cut -d' ' -f3 | xargs cp -Lv -t /$($#BUILD_DIR)/lambda
# copy binary to stage
cp $($#BUILD_DIR)/release/$($#PRODUCT) $($#BUILD_DIR)/lambda/bootstrap
# copy app from stage to artifacts dir
cp $($#STAGE)/* $($#ARTIFACTS_DIR)
I got the makefile from the linked aws-samples project above which I modified slightly to work on codebuild.
Now to my question: How do I get codebuild S3 cache to work with swift lambdas?
when I cache the .aws-sam/**/* folder swift cannot compile because the build paths are different due to the different build machines. The error looks like so:
swift build --product LambdaName -c release --build-path /codebuild/output/src409372700/src/.swift-build/build-LambdaName
33 [1/863] Compiling CSotoExpat xmltok_impl.c
34 [2/863] Compiling CSotoExpat xmltok_ns.c
35 [3/865] Compiling INIParser INIParser.swift
36 <unknown>:0: error: PCH was compiled with module cache path '.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB', but the path is currently '/codebuild/output/src409372700/src/.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB'
37 <unknown>:0: error: missing required module 'SwiftShims'
38 [4/865] Compiling Logging Locks.swift
39 <unknown>:0: error: PCH was compiled with module cache path '.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB', but the path is currently '/codebuild/output/src409372700/src/.aws-sam/build-LambdaName/x86_64-unknown-linux-gnu/release/ModuleCache/1LD7OVICEM9JB'
40 <unknown>:0: error: missing required module 'SwiftShims'
As far as I understand this is due to the fact that swift build cannot cope with a changing or relative path.
After some digging I found this stack overflow post which has a similar problem. I tried there solution and the swift compile error is gone but after a rebuild swift compiles the entire project again even if no code was changed.
Any help is much appreciated!
Thanks!