I think I get the basic idea behind ASN.1 parsing. Walk the bytes, interpret them and do something useful with them. Alas I am stuck on the implementation.
Apple has no sample code (that I could find), probably for security reasons. OpenSSL is not very well documented, so I can only guess at what the functions actually do. The only sample code in swift I did find doesn't handle case 17 (the in-app purchases), which is the one thing I am interested in.
I tried figuring out where in the data stream the pointer is located, but I always get the same nonsensical result of 49.
let receiptContents = NSData(contentsOfURL: receiptLocation)!
let receiptBIO = BIO_new(BIO_s_mem())
BIO_write(receiptBIO, receiptContents.bytes, Int32(receiptContents.length))
contents = d2i_PKCS7_bio(receiptBIO, nil)
//parsing
var currentIndex = UnsafePointer<UInt8>()
var endIndex = UnsafePointer<UInt8>()
let octets = pkcs7_d_data(pkcs7_d_sign(self.contents).memory.contents)
var ptr = UnsafePointer<UInt8>(octets.memory.data)
let end = ptr.advancedBy(Int(octets.memory.length))
println(ptr.memory) //always 49 ???
println(end.memory) //always 0 ???
println(octets.memory.length)
I tried parsing the NSData myself, but well, what is the type of the binaire data?
receiptContents = NSData(contentsOfURL: receiptLocation)!
//get bytes
let count = receiptContents.length / sizeof(UInt8)
var bytes = [UInt8](count: count, repeatedValue: 0)
receiptContents.getBytes(&bytes, length:count * sizeof(UInt8))
//parsing
for index in 0...5
{
let value = Int(bytes[index])
println(value)
}
I get this output:
48
130
21
57
6
9
but if understand the ASN.1 format correctly, it's supposed to start with a value of 17 (set), then 3 bytes for the length (Int24?), then a value of 16 (first sequence), sequence length (1 byte), sequence type (1 byte), sequence payload, (repeat).
Other types like Int32, Int16 makes even less sense to me.
Not sure how to proceed here. Any suggestions?
From the outset, I suppose I should disclaim this by saying I'm not very familiar with many of the underlying technologies (Swift, OpenSSL, the Biometrics standards that I think you're working with).
That said, you're probably running afoul of BER's tagging rules. The Wiki article on X.690 has some introductory comments about how BER tags are constructed, but really you'll want to consult Annex A of X.690 for an example encoding and X.680 §8 for information about tagging.
A SET type can appear in several different forms; but in your case 49 = 0x31 = 0b00110001 = UNIVERSAL 17 (SET, constructed). Other forms may occur, but this is the only one that's explicitly identified as a SET from the tag itself: the specification may use a different tag for a SET type.
Related
Situation: I was solving LeetCode 3. Longest Substring Without Repeating Characters, when I use the Dictionary using Swift the result is Time Limit Exceeded that failed to last test case, but using the same notion of code with C++ it acctually passed with runtime just fine. I thought in swift Dictionary is same thing as UnorderdMap.
Some research: I found some resources said use NSDictionary over regular one but it requires reference type instead of Int or Character etc.
Expected result: fast performance in accessing Dictionary in Swift
Question: I know there are better answer for the question, but the main goal here is Is there a effiencient to access and write to Dictionary or someting we can use to substitude.
func lengthOfLongestSubstring(_ s: String) -> Int {
var window:[Character:Int] = [:] //swift dictionary is kind of slow?
let array = Array(s)
var res = 0
var left = 0, right = 0
while right < s.count {
let rightChar = array[right]
right += 1
window[rightChar, default: 0] += 1
while window[rightChar]! > 1 {
let leftChar = array[left]
window[leftChar, default: 0] -= 1
left += 1
}
res = max(res, right - left)
}
return res
}
Because complexity of count in String is O(n), so that you should save count in a variable. You can read at chapter
Strings and Characters in Swift Book
Extended grapheme clusters can be composed of multiple Unicode scalars. This means that different characters—and different representations of the same character—can require different amounts of memory to store. Because of this, characters in Swift don’t each take up the same amount of memory within a string’s representation. As a result, the number of characters in a string can’t be calculated without iterating through the string to determine its extended grapheme cluster boundaries. If you are working with particularly long string values, be aware that the count property must iterate over the Unicode scalars in the entire string in order to determine the characters for that string.
The count of the characters returned by the count property isn’t always the same as the length property of an NSString that contains the same characters. The length of an NSString is based on the number of 16-bit code units within the string’s UTF-16 representation and not the number of Unicode extended grapheme clusters within the string.
let str = "This is a swift bug"
let data = Data(str.utf8)
print("data size = ", data.endIndex, data.count)
let trimmed = data[2..<data.endIndex]
print("trimmed size = ", trimmed.endIndex, trimmed.count)
The result is
data size = 19 19
trimmed size = 19 17
According to the Apple doc about endIndex:
This is the “one-past-the-end” position, and will always be equal to the count.
Is it a bug? or I'm missing something?
You should open an Apple Feedback for the documentation of Data.endIndex. It's incorrect.
The startIndex of Data is not promised to be zero, and this is an example of when it isn't. Using the Int subscript on Data is unfortunately very dangerous unless you know precisely how the Data was constructed (and specifically that it has a zero index).
Data uniquely mixes two facts that make it tricky to use correctly:
It is its own Slice
Its Index is Int
For some discussion of this, and suggested patterns, see Data.popFirst(), removeFirst() adjust indices. Also see Data ranged subscribe strange behavior for another version of this question.
When you use an expression like array[2..<array.endIndex] you are creating a slice. A slice is a sort of window onto an array (or something similar to an array). Its startIndex is not necessarily 0 and its endIndex is not necessarily one after the last index of the original.
Example:
let arr = Array(1...10)
print(arr.startIndex) // 0
print(arr.endIndex) // 10
let slice = arr[2...4]
print(slice.startIndex) // 2
print(slice.endIndex) // 5
print(slice.count) // 3
You see how this works? The slice has its own logic. Its size (count) is the size of the slice, but its index numbers come from the original array, because the slice is nothing but a pointer into a section of the original array. It has no independent existence; it is just a way of seeing, as it were.
An important consequence is that slice[0] will crash: the first available index of slice is 2, as we have already been told. This is why it is crucial to know whether you're dealing with an original array or a slice.
However, at least you have reason to know that this issue might exist, because slice has a special type — Array<Int>.SubSequence, meaning an ArraySlice. But the fact that you are encountering this by way of Data makes it more tricky, because trimmed is typed as a Data, not as a DataSlice! It is in fact a Data.SubSequence, but you have no simple way of finding that out! That's because Data.SubSequence is typealiased to Data itself. This is to be regarded as a flaw in the Data implementation.
Nevertheless, it is exactly the same phenomenon. These answers should look strangely familiar:
let str = "This is a swift bug"
let data = Data(str.utf8)
let trimmed = data[2...4]
print(trimmed.startIndex) // 2
print(trimmed.endIndex) // 5
print(trimmed.count) // 3
The best way to solve this is Don't Do That. To take a subrange of a Data as a true Data, use subdata:
let trimmed2 = data.subdata(in: 2..<5)
print(trimmed2.startIndex) // 0, and so on; it's an independent copy
I am having a hard time getting a valid value out of the HR characteristics. I am clearly not handling the values properly in Dart.
Example Data:
List<int> value = [22, 56, 55, 4, 7, 3];
Flags Field:
I convert the first item in the main byte array to binary to get the flags
22 = 10110 (as binary)
this leads me to believe that it is U16 (bit[0] is == 1)
HR Value:
Because it is 16 bit I am trying to get the bytes in the 1 & 2 indexes. I then try to buffer them into a ByteData. From there I get convert them to Uint16 with the Endian set to Little. This is giving me a value of 14136. Clearly I am missing something fundamental about how this is supposed to work.
Any help in clearing up what I am not understanding about how to process the 16 bit BLE values would be much appreciated.
Thank you.
/*
Constructor - constructs the heart rate value from a BLE message
*/
HeartRate(List<int> values) {
var flags = values[0];
var s = flags.toRadixString(2);
List<String> flagsArray = s.split("");
int offset = 0;
//Determine whether it is U16 or not
if (flagsArray[0] == "0") {
//Since it is Uint8 i will only get the first value
var hr = values[1];
print(hr);
} else {
//Since UTF 16 is two bytes I need to combine them
//Create a buffer with the first two bytes after the flags
var buffer = new Uint8List.fromList(values.sublist(1, 3)).buffer;
var hrBuffer = new ByteData.view(buffer);
var hr = hrBuffer.getUint16(0, Endian.little);
print(hr);
}
}
Your updated data looks much better. Here's how to decode it, and the process you'd use to figure this out yourself from scratch.
Determine the format
The Bluetooth site has been reorganized recently (~2020), and in particular they got rid of some of the document viewers, which makes things much harder to find and read IMO. All the documentation is in the Heart Rate Service (HRS) document, linked from the main GATT page, but for just parsing the format, the best source I know of is the XML for org.bluetooth.characteristic.heart_rate_measurement. (Since the reorganization, I don't know how you can find this page without searching for it. It doesn't seem to be linked anymore.)
Byte 0 - Flags: 22 (0001 0110)
Bits are numbered from LSB (0) to MSB (7).
Bit 0 - Heart Rate Value Format: 0 => UINT8 beats per minute
Bit 1-2 - Sensor Contact Status: 11 => Supported and detected
Bit 3 - Energy Expended Status: 0 => Not present
Bit 4 - RR-Interval: 1 => One or more values are present
The meaning of RR-intervals is explained in the HRS document, linked above. It sounds like you just want the heart rate value, so I won't go into them here.
Byte 1 - UINT8 BPM: 56
Since Bit 0 of flags was 0, this is the beats per minute. 56.
Bytes 2-5 - UINT16 RR Intervals: 55, 4, 7, 3
You probably don't care about these, but there are two UINT16 values here (there can be an arbitrary number of RR-Interval values). BLE is always little-endian, so [55, 4] is 1,079 (55 + 4<<8), and [7, 3] is 775 (7 + 3<<8).
I believe the docs are a little confusing on this one. The XML suggests that these values are in seconds, but the comments say "Resolution of 1/1024 second." The normal way to express this would be <BinaryExponent>-10</BinaryExponent>, and I'm certain that's what they meant. So these would be:
RR0: 1.05s (1079/1024)
RR1: 0.76s (775/1024)
Swift 5, Xcode 10.
I'm looping through an array of Strings (size probably < 20), each of them looks something like this:
johnsmith.20190202102030.conf
janedoe.19700101115959.conf
I know the first part (the name) beforehand but want to extract the middle part (birthday: 8, 12 or 14 characters long).
Version 1:
let f = "johnsmith.20190202102030.conf"
let name = "johnsmith"
let start = f.index(f.startIndex, offsetBy: name.count+1)
let end = f.index(f.startIndex, offsetBy: f.count-5)
let birthday = String(f[start..<end])
Version 2:
let f = "johnsmith.20190202102030.conf"
let farr = f.split(separator: ".").map(String.init)
let birthday = farr[1]
I'm currently only doing this for 10 Strings and (of course) didn't notice any difference in speed. Even with 100 Strings there probably won't be much of a difference anyway but I'm curious:
Ignoring the length of the code and potential errors, is there a reason (apart from personal preference) to prefer using one version over the other (e.g. speed with 100k Strings - I'm not asking for actual measurements!)?
From my very rough testing, it seems that the substring version is faster. However, in your case I would opt for using the version using split. The code is much more readable to me.
I would like to know how to read a binary file into memory (writing it to memory like an "Array Buffer" from JavaScript), and write to different parts of memory 8-bit, 16-bit, 32-bit etc. values, even 5 bit or 10 bit values.
extension Binary {
static func readFileToMemory(_ file) -> ArrayBuffer {
let data = NSData(contentsOfFile: "/path/to/file/7CHands.dat")!
var dataRange = NSRange(location: 0, length: ?)
var ? = [Int32](count: ?, repeatedValue: ?)
data.getBytes(&?, range: dataRange)
}
static func writeToMemory(_ buffer, location, value) {
buffer[location] = value
}
static func readFromMemory(_ buffer, location) {
return buffer[location]
}
}
I have looked at a bunch of places but haven't found a standard reference.
https://github.com/nst/BinUtils/blob/master/Sources/BinUtils.swift
https://github.com/apple/swift/blob/master/stdlib/public/core/ArrayBuffer.swift
https://github.com/uraimo/Bitter/blob/master/Sources/Bitter/Bitter.swift
In Swift, how do I read an existing binary file into an array?
Swift - writing a byte stream to file
https://apple.github.io/swift-nio/docs/current/NIO/Structs/ByteBuffer.html
https://github.com/Cosmo/BinaryKit/blob/master/Sources/BinaryKit.swift
https://github.com/vapor-community/bits/blob/master/Sources/Bits/Data%2BBytesConvertible.swift
https://academy.realm.io/posts/nate-cook-tryswift-tokyo-unsafe-swift-and-pointer-types/
https://medium.com/#gorjanshukov/working-with-bytes-in-ios-swift-4-de316a389a0c
I would like for this to be as low-level as possible. So perhaps using UnsafeMutablePointer, UnsafePointer, or UnsafeMutableRawPointer.
Saw this as well:
let data = NSMutableData()
var goesIn: Int32 = 42
data.appendBytes(&goesIn, length: sizeof(Int32))
println(data) // <2a000000]
var comesOut: Int32 = 0
data.getBytes(&comesOut, range: NSMakeRange(0, sizeof(Int32)))
println(comesOut) // 42
I would basically like to allocate a chunk of memory and be able to read and write from it. Not sure how to do that. Perhaps using C is the best way, not sure.
Just saw this too:
let rawData = UnsafeMutablePointer<UInt8>.allocate(capacity: width * height * 4)
If you're looking for low level code you'll need to use UnsafeMutableRawPointer. This is a pointer to a untyped data. Memory is accessed in bytes, so 8 chunks of at least 8 bits. I'll cover multiples of 8 bits first.
Reading a File
To read a file this way, you need to manage file handles and pointers yourself. Try the the following code:
// Open the file in read mode
let file = fopen("/Users/joannisorlandos/Desktop/ownership", "r")
// Files need to be closed manually
defer { fclose(file) }
// Find the end
fseek(file, 0, SEEK_END)
// Count the bytes from the start to the end
let fileByteSize = ftell(file)
// Return to the start
fseek(file, 0, SEEK_SET)
// Buffer of 1 byte entities
let pointer = UnsafeMutableRawPointer.allocate(byteCount: fileByteSize, alignment: 1)
// Buffer needs to be cleaned up manually
defer { pointer.deallocate() }
// Size is 1 byte
let readBytes = fread(pointer, 1, fileByteSize, file)
let errorOccurred = readBytes != fileByteSize
First you need to open the file. This can be done using Swift strings since the compiler makes them into a CString itself.
Because cleanup is all for us on this low level, a defer is put in place to close the file at the end.
Next, the file is set to seek the end of the file. Then the distance between the start of the file and the end is calculated. This is used later, so the value is kept.
Then the program is set to return to the start of the file, so the application starts reading from the start.
To store the file, a pointer is allocated with the amount of bytes that the file has in the file system. Note: This can change inbetween the steps if you're extremely unlucky or the file is accessed quite often. But I think for you, this is unlikely.
The amount of bytes is set, and aligned to one byte. (You can learn more about memory alignment on Wikipedia.
Then another defer is added to make sure no memory leaks at the end of this code. The pointer needs to be deallocated manually.
The file's bytes are read and stored in the pointer. Do note that this entire process reads the file in a blocking manner. It can be more preferred to read files asynchronously, if you plan on doing that I'll recommend looking into a library like SwiftNIO instead.
errorOccurred can be used to throw an error or handle issues in another manner.
From here, your buffer is ready for manipulation. You can print the file if it's text using the following code:
print(String(cString: pointer.bindMemory(to: Int8.self, capacity: fileByteSize)))
From here, it's time to learn how to read manipulate the memory.
Manipulating Memory
The below demonstrates reading byte 20..<24 as an Int32.
let int32 = pointer.load(fromByteOffset: 20, as: Int32.self)
I'll leave the other integers up to you. Next, you can alos put data at a position in memory.
pointer.storeBytes(of: 40, toByteOffset: 30, as: Int64.self)
This will replace byte 30..<38 with the number 40. Note that big endian systems, although uncommon, will store information in a different order from normal little endian systems. More about that here.
Modifying Bits
As you notes, you're also interested in modifying five or ten bits at a time. To do so, you'll need to mix the previous information with the new information.
var data32bits = pointer.load(fromByteOffset: 20, as: Int32.self)
var newData = 0b11111000
In this case, you'll be interested in the first 5 bits and want to write them over bit 2 through 7. To do so, first you'll need to shift the bits to a position that matches the new position.
newData = newData >> 2
This shifts the bits 2 places to the right. The two left bits that are now empty are therefore 0. The 2 bits on the right that got shoved off are not existing anymore.
Next, you'll want to get the old data from the buffer and overwrite the new bits.
To do so, first move the new byte into a 32-bits buffer.
var newBits = numericCast(newData) as Int32
The 32 bits will be aligned all the way to the right. If you want to replace the second of the four bytes, run the following:
newBits = newBits << 16
This moves the fourth pair 16 bit places left, or 2 bytes. So it's now on position 1 starting from 0.
Then, the two bytes need to be added on top of each other. One common method is the following:
let oldBits = data32bits & 0b11111111_11000001_11111111_11111111
let result = oldBits | newBits
What happens here is that we remove the 5 bits with new data from the old dataset. We do so by doing a bitwise and on the old 32 bits and a bitmap.
The bitmap has all 1's except for the new locations which are being replaced. Because those are empty in the bitmap, the and operator will exclude those bits since one of the two (old data vs. bitmap) is empty.
AND operators will only be 1 if both sides of the operator are 1.
Finally, the oldBits and the newBits are merged with an OR operator. This will take each bit on both sides and set the result to 1 if the bits at both positions are 1.
This will merge successfully since both buffers contain 1 bits that the other number doesn't set.