I'm trying to read bare Data into a Swift 4 struct using the withUnsafeBytes method. The problem
The network UDP packet has this format:
data: 0102 0A00 0000 0B00 0000
01 : 1 byte : majorVersion (decimal 01)
02 : 1 byte : minorVersion (decimal 02)
0A00 0000 : 4 bytes: applicationHostId (decimal 10)
0B00 0000 : 4 bytes: versionNumber (decimal 11)
Then I have an extension on Data that takes a start and the length of bytes to read
extension Data {
func scanValue<T>(start: Int, length: Int) -> T {
return self.subdata(in: start..<start+length).withUnsafeBytes { $0.pointee }
}
}
This works correctly when reading the values one by one:
// correctly read as decimal "1"
let majorVersion: UInt8 = data.scanValue(start: 0, length: 1)
// correctly read as decimal "2"
let minorVersion: UInt8 = data.scanValue(start: 1, length: 1)
// correctly read as decimal "10"
let applicationHostId: UInt32 = data.scanValue(start: 2, length: 4)
// correctly read as decimal "11"
let versionNumber: UInt32 = data.scanValue(start: 6, length: 4)
Then I created a struct that represents the entire packet as follows
struct XPLBeacon {
var majorVersion: UInt8 // 1 Byte
var minorVersion: UInt8 // 1 Byte
var applicationHostId: UInt32 // 4 Bytes
var versionNumber: UInt32 // 4 Bytes
}
But when I read the data directly into the structure I have some issues:
var beacon: XPLBeacon = data.scanValue(start: 0, length: data.count)
// correctly read as decimal "1"
beacon.majorVersion
// correctly read as decimal "2"
beacon.minorVersion
// not correctly read
beacon.applicationHostId
// not correctly read
beacon.versionNumber
I it supposed to work to parse an entire struct like this?
Reading the entire structure from the data does not work because
the struct members are padded to their natural boundary. The
memory layout of struct XPLBeacon is
A B x x C C C C D D D D
where
offset member
0 A - majorVersion (UInt8)
1 B - minorVersion (UInt8)
2 x x - padding
4 C C C C - applicationHostId (UInt32)
8 D D D D - versionNumber (UInt32)
and the padding is inserted so that the UInt32 members are
aligned to memory addresses which are a multiple of their size. This is
also confirmed by
print(MemoryLayout<XPLBeacon>.size) // 12
(For more information about alignment in Swift, see
Type Layout).
If you read the entire data into the struct then the bytes are assigned
as follows
01 02 0A 00 00 00 0B 00 00 00
A B x x C C C C D D D D
which explains why major/minorVersion are correct, but applicationHostId and versionNumber
are wrong. Reading all members separately from the data is the correct solution.
Since Swift 3 Data conforms to RandomAccessCollection, MutableCollection, RangeReplaceableCollection. So you can simply create a custom initializer to initialise your struct properties as follow:
struct XPLBeacon {
let majorVersion, minorVersion: UInt8 // 1 + 1 = 2 Bytes
let applicationHostId, versionNumber: UInt32 // 4 + 4 = 8 Bytes
init(data: Data) {
self.majorVersion = data[0]
self.minorVersion = data[1]
self.applicationHostId = data
.subdata(in: 2..<6)
.withUnsafeBytes { $0.load(as: UInt32.self) }
self.versionNumber = data
.subdata(in: 6..<10)
.withUnsafeBytes { $0.load(as: UInt32.self) }
}
}
var data = Data([0x01,0x02, 0x0A, 0x00, 0x00, 0x00, 0x0B, 0x00, 0x00,0x00])
print(data as NSData) // "{length = 10, bytes = 0x01020a0000000b000000}\n" <01020a00 00000b00 0000>
let beacon = XPLBeacon(data: data)
beacon.majorVersion // 1
beacon.minorVersion // 2
beacon.applicationHostId // 10
beacon.versionNumber // 11
Following on Leo Dabus answer, I created a slightly more readable constructor:
extension Data {
func object<T>(at index: Index) -> T {
subdata(in: index ..< index.advanced(by: MemoryLayout<T>.size))
.withUnsafeBytes { $0.load(as: T.self) }
}
}
struct XPLBeacon {
var majorVersion: UInt8
var minorVersion: UInt8
var applicationHostId: UInt32
var versionNumber: UInt32
init(data: Data) {
var index = data.startIndex
majorVersion = data.object(at: index)
index += MemoryLayout.size(ofValue: majorVersion)
minorVersion = data.object(at: index)
index += MemoryLayout.size(ofValue: minorVersion)
applicationHostId = data.object(at: index)
index += MemoryLayout.size(ofValue: applicationHostId)
versionNumber = data.object(at: index)
}
}
What is not part of this is of course the checking for the correctness of the data. As other mentioned in comments, this could be done either by having a failable init method or by throwing an Error.
Related
I have an array of UInt8 and I want to calculate CheckSum8 Modulo 256.
If sum of bytes is less than 255, checkSum function returns correct value.
e.g
let bytes1 : [UInt8] = [1, 0xa1]
let validCheck = checkSum(data : bytes1) // 162 = 0xa2
let bytes : [UInt8] = [6, 0xB1, 27,0xc5,0xf5,0x9d]
let invalidCheck = checkSum(data : bytes) // 41
Below function returns 41 but expected checksum is 35.
func checkSum(data: [UInt8]) -> UInt8 {
var sum = 0
for i in 0..<data.count {
sum += Int(data[i])
}
let retVal = sum & 0xff
return UInt8(retVal)
}
Your checkSum method is largely right. If you want, you could simplify it to:
func checkSum(_ values: [UInt8]) -> UInt8 {
let result = values.reduce(0) { ($0 + UInt32($1)) & 0xff }
return UInt8(result)
}
You point out a web site that reports the checksum8 for 06B127c5f59d is 35.
The problem is that in your array has 27, not 0x27. If you have hexadecimal values, you always need the 0x prefix for each value in your array literal (or, technically, at least if the value is larger than 9).
So, consider:
let values: [UInt8] = [0x06, 0xB1, 0x27, 0xc5, 0xf5, 0x9d]
let result = checkSum(values)
That’s 53. If you want to see that in hexadecimal (like that site you referred to):
let hex = String(result, radix: 16)
That shows us that the checksum is 0x35 in hexadecimal.
Here I am trying to concatenate 5 bytes into the single Integer value, I am getting an issue with UnsafePointer withMemoryRebound method.
when I am debugging and checking logs it will gives the correct value. But when I try without debug, it will give the wrong value.(4 out of 5 times wrong value). I got confuses on this API. Is it correct way I am using?
case 1:
let data = [UInt8](rowData) // rowData is type of Data class
let totalKM_BitsArray = [data[8],data[7],data[6],data[5],data[4]]
self.totalKm = UnsafePointer(totalKM_BitsArray).withMemoryRebound(to:UInt64.self, capacity: 1) {$0.pointee}
case 2:
Below code will work for both Enable or Disable debug mode And gives the correct value.
let byte0 : UInt64 = UInt64(data[4])<<64
let byte1 : UInt64 = UInt64(data[5])<<32
let byte2 : UInt64 = UInt64(data[6])<<16
let byte3 : UInt64 = UInt64(data[7])<<8
let byte4 : UInt64 = UInt64(data[8])
self.totalKm = byte0 | byte1 | byte2 | byte3 | byte4
Please suggest me UnsafePointer way of using? Why will this issue come?
Addtional infomation :
let totalKm : UInt64
let data = [UInt8](rowData) // data contain [100, 200, 28, 155, 0, 0, 0, 26, 244, 0, 0, 0, 45, 69, 0, 0, 0, 4, 246]
let totalKM_BitsArray = [data[8],data[7],data[6],data[5],data[4]] // contain [ 244,26,0,0,0]
self.totalKm = UnsafePointer(totalKM_BitsArray).withMemoryRebound(to:UInt64.self, capacity: 1) {$0.pointee}
// when print log gives correct value, when run on device give wrong 3544649566089386 like this.
self.totalKm = byte0 | byte1 | byte2 | byte3 | byte4
// output is 6900 This is correct as expected
There are a few problems with this approach:
let data = [UInt8](rowData) // rowData is type of Data class
let totalKM_BitsArray = [data[8], data[7], data[6], data[5], data[4]]
self.totalKm = UnsafePointer(totalKM_BitsArray)
.withMemoryRebound(to:UInt64.self, capacity: 1) { $0.pointee }
Dereferencing UnsafePointer(totalKM_BitsArray) is undefined behaviour, as the pointer to totalKM_BitsArray's buffer is only temporarily valid for the duration of the initialiser call (hopefully at some point in the future Swift will warn on such constructs).
You're trying to bind only 5 instances of UInt8 to UInt64, so the remaining 3 instances will be garbage.
You can only withMemoryRebound(_:) between types of the same size and stride; which is not the case for UInt8 and UInt64.
It's dependant on the endianness of your platform; data[8] will be the least significant byte on a little endian platform, but the most significant byte on a big endian platform.
Your implementation with bit shifting avoids all of these problems (and is generally the safer way to go as you don't have to consider things like layout compatibility, alignment, and pointer aliasing).
However, assuming that you just wanted to pad out your data with zeroes for the most significant bytes, with rowData[4] to rowData[8] making up the rest of the less significant bytes, then you'll want your bit-shifting implementation to look like this:
let rowData = Data([
100, 200, 28, 155, 0, 0, 0, 26, 244, 0, 0, 0, 45, 69, 0, 0, 0, 4, 246
])
let byte0 = UInt64(rowData[4]) << 32
let byte1 = UInt64(rowData[5]) << 24
let byte2 = UInt64(rowData[6]) << 16
let byte3 = UInt64(rowData[7]) << 8
let byte4 = UInt64(rowData[8])
let totalKm = byte0 | byte1 | byte2 | byte3 | byte4
print(totalKm) // 6900
or, iteratively:
var totalKm: UInt64 = 0
for byte in rowData[4 ... 8] {
totalKm = (totalKm << 8) | UInt64(byte)
}
print(totalKm) // 6900
or, using reduce(_:_:):
let totalKm = rowData[4 ... 8].reduce(0 as UInt64) { accum, byte in
(accum << 8) | UInt64(byte)
}
print(totalKm) // 6900
We can even abstract this into an extension on Data in order to make it easier to load such fixed width integers:
enum Endianness {
case big, little
}
extension Data {
/// Loads the type `I` from the buffer. If there aren't enough bytes to
/// represent `I`, the most significant bits are padded with zeros.
func load<I : FixedWidthInteger>(
fromByteOffset offset: Int = 0, as type: I.Type, endianness: Endianness = .big
) -> I {
let (wholeBytes, spareBits) = I.bitWidth.quotientAndRemainder(dividingBy: 8)
let bytesToRead = Swift.min(count, spareBits == 0 ? wholeBytes : wholeBytes + 1)
let range = startIndex + offset ..< startIndex + offset + bytesToRead
let bytes: Data
switch endianness {
case .big:
bytes = self[range]
case .little:
bytes = Data(self[range].reversed())
}
return bytes.reduce(0) { accum, byte in
(accum << 8) | I(byte)
}
}
}
We're doing a bit of extra work here in order to we read the right number of bytes, as well as being able to handle both big and little endian. But now that we've written it, we can simply write:
let totalKm = rowData[4 ... 8].load(as: UInt64.self)
print(totalKm) // 6900
Note that so far I've assumed that the Data you're getting is zero-indexed. This is safe for the above examples, but isn't necessarily safe depending on where the data is coming from (as it could be a slice). You should be able to do Data(someUnknownDataValue) in order to get a zero-indexed data value that you can work with, although unfortunately I don't believe there's any documentation that guarantees this.
In order to ensure you're correctly indexing an arbitrary Data value, you can define the following extension in order to perform the correct offsetting in the case where you're dealing with a slice:
extension Data {
subscript(offset offset: Int) -> Element {
get { return self[startIndex + offset] }
set { self[startIndex + offset] = newValue }
}
subscript<R : RangeExpression>(
offset range: R
) -> SubSequence where R.Bound == Index {
get {
let concreteRange = range.relative(to: self)
return self[startIndex + concreteRange.lowerBound ..<
startIndex + concreteRange.upperBound]
}
set {
let concreteRange = range.relative(to: self)
self[startIndex + concreteRange.lowerBound ..<
startIndex + concreteRange.upperBound] = newValue
}
}
}
Which you can use then call as e.g data[offset: 4] or data[offset: 4 ... 8].load(as: UInt64.self).
Finally it's worth noting that while you could probably implement this as a re-interpretation of bits by using Data's withUnsafeBytes(_:) method:
let rowData = Data([
100, 200, 28, 155, 0, 0, 0, 26, 244, 0, 0, 0, 45, 69, 0, 0, 0, 4, 246
])
let kmData = Data([0, 0, 0] + rowData[4 ... 8])
let totalKm = kmData.withUnsafeBytes { buffer in
UInt64(bigEndian: buffer.load(as: UInt64.self))
}
print(totalKm) // 6900
This is relying on Data's buffer being 64-bit aligned, which isn't guaranteed. You'll get a runtime error for attempting to load a misaligned value, for example:
let data = Data([0x01, 0x02, 0x03])
let i = data[1...].withUnsafeBytes { buffer in
buffer.load(as: UInt16.self) // Fatal error: load from misaligned raw pointer
}
By loading individual UInt8 values instead and performing bit shifting, we can avoid such alignment issues (however if/when UnsafeMutableRawPointer supports unaligned loads, this will no longer be an issue).
I need to sign-extend an 8bit value to 12 bits. In C, I can do it this way. I read Apple's BinaryInteger protocol documentation, but it didn't explain sign extending to a variable number of bits (and i'm also pretty new at Swift). How can I do this in Swift, assuming val is UInt8 and numbits is 12?
#define MASKBITS(numbits) ((1 << numbits) - 1)
#define SIGNEXTEND_TO_16(val, numbits) \
( \
(int16_t)((val & MASKBITS(numbits)) | ( \
(val & (1 << (numbits-1))) ? ~MASKBITS(numbits) : 0) \
))
You can use Int8(bitPattern:) to convert the given unsigned
value to a signed value with the same binary representation,
then sign extend by converting to Int16, make unsigned again, and finally truncate
to the given number of bits:
func signExtend(val: UInt8, numBits: Int) -> UInt16 {
// Sign extend to unsigned 16-bit:
var extended = UInt16(bitPattern: Int16(Int8(bitPattern: val)))
// Truncate to given number of bits:
if numBits < 16 {
extended = extended & ((1 << numBits) - 1)
}
return extended
}
Example:
for i in 1...16 {
let x = signExtend(val: 200, numBits: i)
print(String(format: "%2d %04X", i, x))
}
Output:
1 0000
2 0000
3 0000
4 0008
5 0008
6 0008
7 0048
8 00C8
9 01C8
10 03C8
11 07C8
12 0FC8
13 1FC8
14 3FC8
15 7FC8
16 FFC8
I had the same question in the context of bitstream parsing. I needed code to parse n bit two's complement values into Int32. Here is my solution that works without any condition:
extension UInt32 {
func signExtension(n: Int) -> Int32 {
let signed = Int32.init(bitPattern: self << (32 - n))
let result = signed >> (32 - n)
return result
}
}
And a unit test function showing how to use that code:
func testSignExtension_N_2_3() {
let unsignedValue: UInt32 = 0b110
let signedValue: Int32 = unsignedValue.signExtension(n: 3)
XCTAssertEqual(signedValue, -2)
}
So, I have a stream of well-formed data coming from some hardware. The stream consists of a bunch of chunks of 8-bit data, some of which are meant to form into 32-bit integers. That's all good. The data moves along and now I want to parcel the sequence up.
The data is actually a block of contiguous bytes, with segments of it mapped to useful data. So, for example, the first byte is a confirmation code, the following four bytes represent a UInt32 of some application-specific meaning, followed by two bytes representing a UInt16, and so on for a couple dozen bytes.
I found two different ways to do that, both of which seem a bit..overwrought. It may just what happens when you get close to the metal.
But — are these two code idioms generally what one should expect to do? Or am I missing something more compact?
// data : Data exists before this code, and has what we're transforming into UInt32
// One Way to get 4 bytes from Data into a UInt32
var y : [UInt8] = [UInt8](repeating:UInt8(0x0), count: 4)
data.copyBytes(to: &y, from: Range(uncheckedBounds: (2,6)))
let u32result = UnsafePointer(y).withMemoryRebound(to: UInt32.self, capacity: 1, {
$0.pointee
})
// u32result contains the 4 bytes from data
// Another Way to get 4 bytes from Data into a UInt32 via NSData
var result : UInt32 = 0
let resultAsNSData : NSData = data.subdata(in: Range(uncheckedBounds: (2,6))) as NSData
resultAsNSData.getBytes(&result, range: NSRange(location: 0, length: 4))
// result contains the 4 bytes from data
Creating UInt32 array from well-formed data object.
Swift 3
// Create sample data
let data = "foo".data(using: .utf8)!
// Using pointers style constructor
let array = data.withUnsafeBytes {
[UInt32](UnsafeBufferPointer(start: $0, count: data.count))
}
Swift 2
// Create sample data
let data = "foo".dataUsingEncoding(NSUTF8StringEncoding)!
// Using pointers style constructor
let array = Array(UnsafeBufferPointer(start: UnsafePointer<UInt32>(data.bytes), count: data.length))
I found two other ways of doing this which is leading me to believe that there are plenty of ways to do it, which is good, I suppose.
Two additional ways are described in some fashion over on Ray Wenderlich
This code dropped into your Xcode playground will reveal these two other idioms.
do {
let count = 1 // number of UInt32s
let stride = MemoryLayout<UInt32>.stride
let alignment = MemoryLayout<UInt32>.alignment
let byteCount = count * stride
var bytes : [UInt8] = [0x0D, 0x0C, 0x0B, 0x0A] // little-endian LSB -> MSB
var data : Data = Data.init(bytes: bytes) // In my situtation, I actually start with an instance of Data, so the [UInt8] above is a conceit.
print("---------------- 1 ------------------")
let placeholder = UnsafeMutableRawPointer.allocate(bytes: byteCount, alignedTo:alignment)
withUnsafeBytes(of: &data, { (bytes) in
for (index, byte) in data.enumerated() {
print("byte[\(index)]->\(String(format: "0x%02x",byte)) data[\(index)]->\(String(format: "0x%02x", data[index])) addr: \(bytes.baseAddress!+index)")
placeholder.storeBytes(of: byte, toByteOffset: index, as: UInt8.self)
}
})
let typedPointer1 = placeholder.bindMemory(to: UInt32.self, capacity: count)
print("u32: \(String(format: "0x%08x", typedPointer1.pointee))")
print("---------------- 2 ------------------")
for (index, byte) in bytes.enumerated() {
placeholder.storeBytes(of: byte, toByteOffset: index, as: UInt8.self)
// print("byte \(index): \(byte)")
print("byte[\(index)]->\(String(format: "0x%02x",byte))")
}
let typedPointer = placeholder.bindMemory(to: UInt32.self, capacity: count)
print(typedPointer.pointee)
let result : UInt32 = typedPointer.pointee
print("u32: \(String(format: "0x%08x", typedPointer.pointee))")
}
With output:
---------------- 1 ------------------
byte[0]->0x0d data[0]->0x0d addr: 0x00007fff57243f68
byte[1]->0x0c data[1]->0x0c addr: 0x00007fff57243f69
byte[2]->0x0b data[2]->0x0b addr: 0x00007fff57243f6a
byte[3]->0x0a data[3]->0x0a addr: 0x00007fff57243f6b
u32: 0x0a0b0c0d
---------------- 2 ------------------
byte[0]->0x0d
byte[1]->0x0c
byte[2]->0x0b
byte[3]->0x0a
168496141
u32: 0x0a0b0c0d
Here's a Gist.
let a = [ 0x00, 0x00, 0x00, 0x0e ]
let b = a[0] << 24 + a[1] << 16 + a[2] << 8 + a[3]
print(b) // will print 14.
Should I describe this operation ?
I have some binary data that encodes a two byte value as a signed integer.
bytes[1] = 255 // 0xFF
bytes[2] = 251 // 0xF1
Decoding
This is fairly easy - I can extract an Int16 value from these bytes with:
Int16(bytes[1]) << 8 | Int16(bytes[2])
Encoding
This is where I'm running into issues. Most of my data spec called for UInt and that is easy but I'm having trouble extracting the two bytes that make up an Int16
let nv : Int16 = -15
UInt8(nv >> 8) // fail
UInt8(nv) // fail
Question
How would I extract the two bytes that make up an Int16 value
You should work with unsigned integers:
let bytes: [UInt8] = [255, 251]
let uInt16Value = UInt16(bytes[0]) << 8 | UInt16(bytes[1])
let uInt8Value0 = uInt16Value >> 8
let uInt8Value1 = UInt8(uInt16Value & 0x00ff)
If you want to convert UInt16 to bit equivalent Int16 then you can do it with specific initializer:
let int16Value: Int16 = -15
let uInt16Value = UInt16(bitPattern: int16Value)
And vice versa:
let uInt16Value: UInt16 = 65000
let int16Value = Int16(bitPattern: uInt16Value)
In your case:
let nv: Int16 = -15
let uNv = UInt16(bitPattern: nv)
UInt8(uNv >> 8)
UInt8(uNv & 0x00ff)
You could use init(truncatingBitPattern: Int16) initializer:
let nv: Int16 = -15
UInt8(truncatingBitPattern: nv >> 8) // -> 255
UInt8(truncatingBitPattern: nv) // -> 241
I would just do this:
let a = UInt8(nv >> 8 & 0x00ff) // 255
let b = UInt8(nv & 0x00ff) // 241
extension Int16 {
var twoBytes : [UInt8] {
let unsignedSelf = UInt16(bitPattern: self)
return [UInt8(truncatingIfNeeded: unsignedSelf >> 8),
UInt8(truncatingIfNeeded: unsignedSelf)]
}
}
var test : Int16 = -15
test.twoBytes // [255, 241]