Can somebody please explain to me what I'm missing here?
Given
let index: Int32 = 100
Why is this not okay:
// Use of extraneous '&'
let ptr = &index // Type inference?
Or even:
// Use of extraneous '&'
let ptr: UnsafePointer<Int32> = &index
But this is:
{
func point(num: UnsafePointer<Int32>) -> UnsafePointer<Int32> {
return num
}
let ptr = point(num: &index)
}
This would be the simple equivalent of this in C:
int index = 100;
int *ptr = &index;
Do I really have to define a function that literally takes the referenced value and passes back the very same reference? Something feels wrong about it. It seems like I'm missing something here, maybe even fundamental.
How do I assign an UnsafePointer to the memory address of the type it is (Int32 in this case)???
Thanks!
Edit:
Ultimately what I'm attempting to accomplish is, I need to write several various structures into a binary file. The variable index would be a property of a structure. The path I'm going down now involves a file OutputStream. I don't mind receiving suggestions on this, but gets out of scope of the original question.
I don't know the precise rationale, but presumably let ptr = &index isn't allowed because there's no guarantee that you can dereference ptr without invoking undefined behaviour (assuming index isn't a global or static stored variable – the only cases where Swift guarantees stable and unique pointer values).
Unlike other languages, Swift doesn't guarantee that a local variable will remain initialised until the end of the scope it's declared in – the optimiser is free to deinitialise it earlier. Allowing let ptr = &index would therefore make it far too easy to write unsound code.
It's worth noting that your example of:
func point(num: UnsafePointer<Int32>) -> UnsafePointer<Int32> {
return num
}
let ptr = point(num: &index)
is also unsound. Attempting to dereference ptr from let ptr = point(num: &index) is undefined behaviour, as the inout-to-pointer argument conversion produces a temporary pointer only valid for the duration of the function call.
If you want a scoped temporary pointer to a value, you can use withUnsafePointer(to:) – for example:
func baz() {
var foo = 5
withUnsafePointer(to: &foo) { ptr in
// use `ptr` here – do not escape it!
}
// In Swift 4.2 you can also use `withUnsafePointer(to:)` on let constants.
let bar = 5
withUnsafePointer(to: bar) { ptr in
// use `ptr` here – do not escape it!
}
}
Note that the pointer is only valid for the duration of the closure – attempting to escape it will lead to undefined behaviour.
Related
I want to get the memory address of a value type of constant in Swift. For example with the variables it looks like:
My code:
var value = 10
withUnsafePointer(to: &value) {
print(" str value \(value) has address: \($0)")
}
There is a variant of withUnsafePointer(to:,_:) that accepts a non-inout argument as its first input argument.
let immutableValue = 1
withUnsafePointer(to: immutableValue, { pointer -> Void in
print(pointer)
})
As MartinR pointed out, this pointer though is valid only for the invocation of the closure.
It is also important to note that with values known at compile time, the compiler might substitute the value in place of the variable, so the variable might not even exist in memory at runtime.
I've updated to swift 5 and one of the dependencies I use won't compile in swift 5. I've fixed it, but now I'm getting 350+ deprecation warnings all over the file. They're all similar to this:
withUnsafeMutableBytes is deprecated: use withUnsafeMutableBytes<R>(_: (UnsafeMutableRawBufferPointer) throws -> R) rethrows -> R instead
And this is a snipit of the code (it's basically just calling a c library's functions):
var k = Data(count: crypto_generichash_keybytes())
k.withUnsafeMutableBytes { kPtr in
flutter_sodium.crypto_generichash_keygen(kPtr)
}
For reference, in the above crypto_generichash_keybytes() just returns a size_t and crypto_generichash_keygen's signature is void crypto_generichash_keygen(unsigned char k[crypto_generichash_KEYBYTES]);.
I figured out (as this answer states) that the way to get around this should be to call kPtr.baseAddress:
var k = Data(count: crypto_generichash_keybytes())
k.withUnsafeMutableBytes { kPtr in
flutter_sodium.crypto_generichash_keygen(kPtr.baseAddress)
}
as that should use the withUnsafeMutableBytes<ResultType> variant rather than the deprecated withUnsafeMutableBytes<ResultType, ContentType>. However, this instead results in the error
value of type 'UnsafeMutablePointer<_>' has no member 'baseAddress'.
If I explicitly specify the resultType and kPtr:
var k = Data(count: crypto_generichash_keybytes())
k.withUnsafeMutableBytes { (kPtr: UnsafeMutableRawBufferPointer) -> Void in
flutter_sodium.crypto_generichash_keygen(kPtr.baseAddress)
}
I instead get
UnsafeMutableRawBufferPointer' is not convertible to 'UnsafeMutablePointer<_>'.
Are there any swift experts out there that can help me figure out the right way to do this? I know the warnings are just warnings, but I prefer to have code that compiles with no warnings.
I took a look at Swift 5.0: 'withUnsafeBytes' is deprecated: use `withUnsafeBytes<R>(...) before posting this question and it doesn't help my situation as I'm not loading the pointer but rather using the data. Also, I've done exactly what the documentation tells me to but that still isn't helping.
EDIT: To be a bit more clear, some of the 350+ warnings were related to code where the Data is allocated in the code, however some of them are where I receive Data from an external source. That looks something like this:
let args = call.arguments as! NSDictionary
let server_pk = (args["server_pk"] as! FlutterStandardTypedData).data
let server_sk = (args["server_sk"] as! FlutterStandardTypedData).data
let client_pk = (args["client_pk"] as! FlutterStandardTypedData).data
var rx = Data(count: flutter_sodium.crypto_kx_sessionkeybytes())
var tx = Data(count: flutter_sodium.crypto_kx_sessionkeybytes())
let ret = rx.withUnsafeMutableBytes { rxPtr in
tx.withUnsafeMutableBytes { txPtr in
server_pk.withUnsafeBytes { server_pkPtr in
server_sk.withUnsafeBytes { server_skPtr in
client_pk.withUnsafeBytes { client_pkPtr in
flutter_sodium.crypto_kx_server_session_keys(rxPtr, txPtr, server_pkPtr, server_skPtr, client_pkPtr)
}
}
}
}
}
with the corresponding method call
SODIUM_EXPORT
int crypto_kx_client_session_keys(unsigned char rx[crypto_kx_SESSIONKEYBYTES],
unsigned char tx[crypto_kx_SESSIONKEYBYTES],
const unsigned char client_pk[crypto_kx_PUBLICKEYBYTES],
const unsigned char client_sk[crypto_kx_SECRETKEYBYTES],
const unsigned char server_pk[crypto_kx_PUBLICKEYBYTES])
__attribute__ ((warn_unused_result));
(and I know that the code is not really optimal swift, but when dealing with interoperability between dart and swift this is what the flutter team came up with for how to do it).
When I asked the question I was trying to distill it down to the simplest case but that case had a specific answer which differs to the overall problem I'm having.
I wouldn't use Data here – Data represents an untyped collection of "raw" bytes, however crypto_generichash_keygen wants a mutable pointer to typed memory. The reason why the UnsafeMutablePointer<T> variant of withUnsafeMutableBytes was deprecated is that it's fundamentally the wrong abstraction to be providing on untyped memory.
The simplest way to get a buffer of typed memory in Swift is with an Array:
var k = [UInt8](repeating: 0, count: crypto_generichash_keybytes())
flutter_sodium.crypto_generichash_keygen(&k)
You can always turn the resulting buffer into a Data value afterwards by saying Data(k).
Another option is to use an UnsafeMutableBufferPointer:
let k = UnsafeMutableBufferPointer<UInt8>.allocate(capacity: crypto_generichash_keybytes())
defer {
k.deallocate()
}
flutter_sodium.crypto_generichash_keygen(k.baseAddress!)
// Now use the buffer `k` – just make sure you finish using it before the end of
// the scope when `deallocate()` gets called!
Unlike Array, this avoids having to pre-fill the resulting buffer with zeros before being passed off to the C API, however this likely isn't of concern. But just like Array, you can turn such a buffer into a Data by just saying Data(k).
For cases where you get handed a Data value from some external source and need to pass it off to an API as a typed pointer, the simplest and safest option is to just turn it into an array before passing it by saying Array(someData).
For example:
let args = call.arguments as! NSDictionary
let server_pk = (args["server_pk"] as! FlutterStandardTypedData).data
let server_sk = (args["server_sk"] as! FlutterStandardTypedData).data
let client_pk = (args["client_pk"] as! FlutterStandardTypedData).data
var rx = [UInt8](repeating: 0, count: flutter_sodium.crypto_kx_sessionkeybytes())
var tx = [UInt8](repeating: 0, count: flutter_sodium.crypto_kx_sessionkeybytes())
flutter_sodium.crypto_kx_server_session_keys(
&rx, &tx, Array(server_pk), Array(server_sk), Array(client_pk)
)
You probably could use withUnsafeBytes and call bindMemory on the underlying pointer, but I would discourage it, as it changes the type of the underlying memory which could subtly impact the soundness of any other Swift code sharing that memory due to the fact that you're switching out the type from under it.
I got some unexpected behavior using UnsafeMutablePointer on an observed property in a struct I created (on Xcode 10.1, Swift 4.2). See the following playground code:
struct NormalThing {
var anInt = 0
}
struct IntObservingThing {
var anInt: Int = 0 {
didSet {
print("I was just set to \(anInt)")
}
}
}
var normalThing = NormalThing(anInt: 0)
var ptr = UnsafeMutablePointer(&normalThing.anInt)
ptr.pointee = 20
print(normalThing.anInt) // "20\n"
var intObservingThing = IntObservingThing(anInt: 0)
var otherPtr = UnsafeMutablePointer(&intObservingThing.anInt)
// "I was just set to 0."
otherPtr.pointee = 20
print(intObservingThing.anInt) // "0\n"
Seemingly, modifying the pointee on an UnsafeMutablePointer to an observed property doesn't actually modify the value of the property. Also, the act of assigning the pointer to the property fires the didSet action. What am I missing here?
Any time you see a construct like UnsafeMutablePointer(&intObservingThing.anInt), you should be extremely wary about whether it'll exhibit undefined behaviour. In the vast majority of cases, it will.
First, let's break down exactly what's happening here. UnsafeMutablePointer doesn't have any initialisers that take inout parameters, so what initialiser is this calling? Well, the compiler has a special conversion that allows a & prefixed argument to be converted to a mutable pointer to the 'storage' referred to by the expression. This is called an inout-to-pointer conversion.
For example:
func foo(_ ptr: UnsafeMutablePointer<Int>) {
ptr.pointee += 1
}
var i = 0
foo(&i)
print(i) // 1
The compiler inserts a conversion that turns &i into a mutable pointer to i's storage. Okay, but what happens when i doesn't have any storage? For example, what if it's computed?
func foo(_ ptr: UnsafeMutablePointer<Int>) {
ptr.pointee += 1
}
var i: Int {
get { return 0 }
set { print("newValue = \(newValue)") }
}
foo(&i)
// prints: newValue = 1
This still works, so what storage is being pointed to by the pointer? To solve this problem, the compiler:
Calls i's getter, and places the resultant value into a temporary variable.
Gets a pointer to that temporary variable, and passes that to the call to foo.
Calls i's setter with the new value from the temporary.
Effectively doing the following:
var j = i // calling `i`'s getter
foo(&j)
i = j // calling `i`'s setter
It should hopefully be clear from this example that this imposes an important constraint on the lifetime of the pointer passed to foo – it can only be used to mutate the value of i during the call to foo. Attempting to escape the pointer and using it after the call to foo will result in a modification of only the temporary variable's value, and not i.
For example:
func foo(_ ptr: UnsafeMutablePointer<Int>) -> UnsafeMutablePointer<Int> {
return ptr
}
var i: Int {
get { return 0 }
set { print("newValue = \(newValue)") }
}
let ptr = foo(&i)
// prints: newValue = 0
ptr.pointee += 1
ptr.pointee += 1 takes place after i's setter has been called with the temporary variable's new value, therefore it has no effect.
Worse than that, it exhibits undefined behaviour, as the compiler doesn't guarantee that the temporary variable will remain valid after the call to foo has ended. For example, the optimiser could de-initialise it immediately after the call.
Okay, but as long as we only get pointers to variables that aren't computed, we should be able to use the pointer outside of the call it was passed to, right? Unfortunately not, turns out there's lots of other ways to shoot yourself in the foot when escaping inout-to-pointer conversions!
To name just a few (there are many more!):
A local variable is problematic for a similar reason to our temporary variable from earlier – the compiler doesn't guarantee that it will remain initialised until the end of the scope it's declared in. The optimiser is free to de-initialise it earlier.
For example:
func bar() {
var i = 0
let ptr = foo(&i)
// Optimiser could de-initialise `i` here.
// ... making this undefined behaviour!
ptr.pointee += 1
}
A stored variable with observers is problematic because under the hood it's actually implemented as a computed variable that calls its observers in its setter.
For example:
var i: Int = 0 {
willSet(newValue) {
print("willSet to \(newValue), oldValue was \(i)")
}
didSet(oldValue) {
print("didSet to \(i), oldValue was \(oldValue)")
}
}
is essentially syntactic sugar for:
var _i: Int = 0
func willSetI(newValue: Int) {
print("willSet to \(newValue), oldValue was \(i)")
}
func didSetI(oldValue: Int) {
print("didSet to \(i), oldValue was \(oldValue)")
}
var i: Int {
get {
return _i
}
set {
willSetI(newValue: newValue)
let oldValue = _i
_i = newValue
didSetI(oldValue: oldValue)
}
}
A non-final stored property on classes is problematic as it can be overridden by a computed property.
And this isn't even considering cases that rely on implementation details within the compiler.
For this reason, the compiler only guarantees stable and unique pointer values from inout-to-pointer conversions on stored global and static stored variables without observers. In any other case, attempting to escape and use a pointer from an inout-to-pointer conversion after the call it was passed to will lead to undefined behaviour.
Okay, but how does my example with the function foo relate to your example of calling an UnsafeMutablePointer initialiser? Well, UnsafeMutablePointer has an initialiser that takes an UnsafeMutablePointer argument (as a result of conforming to the underscored _Pointer protocol which most standard library pointer types conform to).
This initialiser is effectively same as the foo function – it takes an UnsafeMutablePointer argument and returns it. Therefore when you do UnsafeMutablePointer(&intObservingThing.anInt), you're escaping the pointer produced from the inout-to-pointer conversion – which, as we've discussed, is only valid if it's used on a stored global or static variable without observers.
So, to wrap things up:
var intObservingThing = IntObservingThing(anInt: 0)
var otherPtr = UnsafeMutablePointer(&intObservingThing.anInt)
// "I was just set to 0."
otherPtr.pointee = 20
is undefined behaviour. The pointer produced from the inout-to-pointer conversion is only valid for the duration of the call to UnsafeMutablePointer's initialiser. Attempting to use it afterwards results in undefined behaviour. As matt demonstrates, if you want scoped pointer access to intObservingThing.anInt, you want to use withUnsafeMutablePointer(to:).
I'm actually currently working on implementing a warning (which will hopefully transition to an error) that will be emitted on such unsound inout-to-pointer conversions. Unfortunately I haven't had much time lately to work on it, but all things going well, I'm aiming to start pushing it forwards in the new year, and hopefully get it into a Swift 5.x release.
In addition, it's worth noting that while the compiler doesn't currently guarantee well-defined behaviour for:
var normalThing = NormalThing(anInt: 0)
var ptr = UnsafeMutablePointer(&normalThing.anInt)
ptr.pointee = 20
From the discussion on #20467, it looks like this will likely be something that the compiler does guarantee well-defined behaviour for in a future release, due to the fact that the base (normalThing) is a fragile stored global variable of a struct without observers, and anInt is a fragile stored property without observers.
I'm pretty sure the problem is that what you're doing is illegal. You can't just declare an unsafe pointer and claim that it points at the address of a struct property. (In fact, I don't even understand why your code compiles in the first place; what initializer does the compiler think this is?) The correct way, which gives the expected results, is to ask for a pointer that does point at that address, like this:
struct IntObservingThing {
var anInt: Int = 0 {
didSet {
print("I was just set to \(anInt)")
}
}
}
withUnsafeMutablePointer(to: &intObservingThing.anInt) { ptr -> Void in
ptr.pointee = 20 // I was just set to 20
}
print(intObservingThing.anInt) // 20
Let's say I do the following in C++:
int i = 1;
int* ptr = &i;
*ptr = 2;
cout << i << '\n';
And I want to do something similar in swift. Could I do the following?
var i : Int = 1
var iptr : UnsafeMutablePointer<Int> = &i
iptr.memory = 2
print(i)
And achieve the same result?
Yes-ish.
You can't do it exactly as you've attempted in the question. It won't compile. Swift won't let you directly access the address of a value like this. At the end of the day, the reason is mostly because there's simply no good reason to do so.
We do see the & operator in Swift however.
First of all, there is the inout keyword when declaring function parameters:
func doubleIfPositive(inout value: Float) -> Bool {
if value > 0 {
value *= 2
return true
}
return false
}
And to call this method, we'd need the & operator:
let weMadeARadian = doubleIfPositive(&pi)
We can see it similarly used when we have a function which takes an argument of type UnsafeMutablePointer (and other variants of these pointer structs). In this specific case, it's primarily for interoperability with C & Objective-C, where we could declare a method as such:
bool doubleIfPositive(float * value) -> bool {
if (value > 0) {
value *= 2;
return true;
}
return false;
}
The Swift interface for that method ends up looking somethin like this:
func doubleIfPositive(value: UnsafeMutablePointer<Float>) -> Bool
And calling this method from Swift actually looks just like it did before when using the inout approach:
let weMadeARadian = doubleIfPositive(&pi)
But these are the only two uses of this & operator I can find in Swift.
With that said, we can write a function that makes use of the second form of passing an argument into a method with the & operator and returns that variable wrapped in an unsafe mutable pointer. It looks like this:
func addressOf<T>(value: UnsafeMutablePointer<T>) -> UnsafeMutablePointer<T> {
return value
}
And it behaves about as you'd expect from your original code snippet:
var i: Int = 1
var iPtr = addressOf(&i)
iPtr.memory = 2
print(i) // prints 2
As noted by Kevin in the comments, we can also directly allocate memory if we want.
var iPtr = UnsafeMutablePointer<Int>.alloc(1)
The argument 1 here is effectively the mount of space to allocate. This says we want to allocate enough memory for a single Int.
This is roughly equivalent to the following C code:
int * iPtr = malloc(1 * sizeof(int));
BUT...
If you're doing any of this for anything other than interoperability with C or Objective-C, you're most likely not Swifting correctly. So before you start running around town with pointers to value types in Swift, please, make sure it's what you absolutely need to be doing. I've been writing Swift since release, and I've never found the need for any of these shenanigans.
Like this (not the only way, but it's clear):
var i : Int = 1
withUnsafeMutablePointer(&i) {
iptr -> () in
iptr.memory = 2
}
print(i)
Not a very interesting example, but it is completely parallel to your pseudo-code, and we really did reach right into the already allocated memory and alter it, which is what you wanted to do.
This sort of thing gets a lot more interesting when what you want to do is something like cycle thru memory just as fast as doing pointer arithmetic in C.
I'm working with a third party c API I'm trying to call one of the functions with a simple string. Something like this:
some_c_func("aString");
I get a build error:
Type 'UnsafeMutablePointer<char_t>' does not conform to protocol 'StringLiteralConvertible'
I've seen some suggestions to use utf8 on String or similar conversions, which gets nearly there, but with the following error:
some_c_func("aString".cStringUsingEncoding(NSUTF8StringEncoding));
'UnsafePointer<Int8>' is not convertible to 'UnsafeMutablePointer<char_t>'
How can I create an UnsafeMutablePointer?
It all depends on what char_t is.
If char_t converts to Int8 then the following will work.
if let cString = str.cStringUsingEncoding(NSUTF8StringEncoding) {
some_c_func(strdup(cString))
}
This can be collapsed to
some_c_func(strdup(str.cStringUsingEncoding(NSUTF8StringEncoding)!))
WARNING! This second method will cause a crash if func cStringUsingEncoding(_:) returns nil.
Updating for Swift 3, and to fix memory leak
If the C string is only needed in a local scope, then no strdup() is needed.
guard let cString = str.cString(using: .utf8) else {
return
}
some_c_func(cString)
cString will have the same memory lifecycle as str (well similar at least).
If the C string needs to live outside the local scope, then you will need a copy. That copy will need to be freed.
guard let interimString = str.cString(using: .utf8), let cString = strdup(interimString) else {
return
}
some_c_func(cString)
//…
free(cString)
it may be simpler than that - many C APIs pass strings around as char * types, and swift treats these as unsafe.
try updating the C API (good) or hack it's header files (bad) to declare these as const char * instead.
in my experience this allows you to pass standard swift String types directly to the C API.
apparently a constant is required, in order to conform to the protocol.
I haven't tried passing strings like that, but I have a C function that I call from Swift, that takes a lot more parameters than shown here, among which is a reference to a Swift C typecast buffer to hold an error string. The compiler doesn't complain and the function call works. Hopefully this will steer you closer to the answer and you can provide an update with the final answer or someone else can.
var err = [CChar](count: 256, repeatedValue: 0)
var rv = somefunc((UnsafeMutablePointer<Int8>)(err))
if (rv < 0) {
println("Error \(err)")
return
}