My app is, like oh so many apps, retrieving JSON from an API and converting it using the new Codable protocol in Swift 4. Most of the time, this works fine and as expected. However, sometimes the API will send me unexpected garbage. Incorrect types, arrays with just null inside, that kind of thing.
The problem is that the objects involved can be large and complicated, and when I'm parsing a child object and it fails, the whole object fails, all the way up to the root. I'm including a very simple playground example to illustrate the concept; the actual objects involved are way more complex.
let goodJSON = """
{
"name": "Fiona Glenanne",
"vehicles": [
{
"make": "Saab",
"model": "9-3",
"color": "Black"
},
{
"make": "Hyundai",
"model": "Genesis",
"color": "Blue"
}
]
}
"""
let goodJSONData = goodJSON.data(using: .utf8)!
let badJSON = """
{
"name": "Michael Westen",
"vehicles": {
"make": "Dodge",
"model": "Charger",
"color": "Black"
}
}
"""
let badJSONData = badJSON.data(using: .utf8)!
struct Character: Codable {
let name: String
let vehicles: [Vehicle]
}
struct Vehicle: Codable {
let make: String
let model: String
let color: String
}
do {
let goodCharacter = try JSONDecoder().decode(Character.self, from: goodJSONData)
print(goodCharacter)
} catch {
print(error)
}
do {
let badCharacter = try JSONDecoder().decode(Character.self, from: badJSONData)
print(badCharacter)
} catch DecodingError.typeMismatch(let type, let context) {
print("Got \(type); \(context.debugDescription) ** Path:\(context.codingPath)")
} catch {
print("Caught a different error: \(error)")
}
Output:
Character(name: "Fiona Glenanne", vehicles: [__lldb_expr_20.Vehicle(make: "Saab", model: "9-3", color: "Black"), __lldb_expr_20.Vehicle(make: "Hyundai", model: "Genesis", color: "Blue")])
Got Array<Any>; Expected to decode Array<Any> but found a dictionary instead. ** Path:[CodingKeys(stringValue: "vehicles", intValue: nil)]
vehicles is expected to be an array of objects, but in the badJSON case, it is a single object, which causes the .typeMismatch exception and kills the parsing right there.
What I'm looking for is a way to allow errors like this one to kill the parsing for the child object only and allow parsing of the parent object to continue. I'm looking to do this in a generic fashion, so I don't have to special case each and every object in my app to specifically handle whatever bad data the API delivers. I'm not sure if there even is a solution for this, I haven't had any luck finding anything, but it would sure improve my quality of life if there is. Thanks!
you can try to custom the init(from decoder: Decoder) as suggested in comments it would be something like this,
struct Character: Codable {
let name: String
let vehicles: [Vehicle]
private enum CodingKeys: String, CodingKey { case name, vehicles }
init(from decoder: Decoder) throws {
let container = try decoder.container(keyedBy: CodingKeys.self)
name = try container.decode(String.self, forKey: .name)
do {
let vehicle = try container.decode(Vehicle.self, forKey: .vehicles)
vehicles = [vehicle]
} catch DecodingError.typeMismatch {
vehicles = try container.decode([Vehicle].self, forKey: .vehicles)
}
}
Related
Thanks for your help. I need interaction with Toml files in my macOS Swift application. I am using the TOMLDecoder library to parse the Toml format. The library works by specifying a Swift struct type that conforms to Codable, and have the library create the object for us. From the docs:
struct Discography: Codable {
struct Album: Codable {
let name: String
struct Song: Codable {
let name: String
}
let songs: [Song]
}
let albums: [Album]
}
If we take a sample Toml file:
[[albums]]
name = "Born to Run"
[[albums.songs]]
name = "Jungleland"
[[albums.songs]]
name = "Meeting Across the River"
[[albums]]
name = "Born in the USA"
[[albums.songs]]
name = "Glory Days"
[[albums.songs]]
name = "Dancing in the Dark"
We can parse it with:
let tomlData = try? Data(contentsOf: URL(fileURLWithPath: "/path/to/file"))
let discography = try? TOMLDecoder().decode(Discography.self, from: tomlData)
Here comes my question. The library does not provide a way to reverse the process, so to serialize back the object, so I would like to write that on my own, and, possibly, I would like to achieve a solution in clean Swift, if I understand correctly, by the use of the T type, thus allowing any kind of Codable conforming object to be serializable. The decode function in the library is:
public func decode<T: Decodable>(_ type: T.Type, from text: String) throws -> T {
let topLevel: Any
do {
topLevel = try TOMLDeserializer.tomlTable(with: text)
} catch {
throw DecodingError.dataCorrupted(DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid TOML.", underlyingError: error))
}
let decoder = TOMLDecoderImpl(referencing: self, options: self.options)
guard let value = try decoder.unbox(topLevel, as: type) else {
throw DecodingError.valueNotFound(type, DecodingError.Context(codingPath: [], debugDescription: "The given data did not contain a top-level value."))
}
return value
}
I have started to write my encode function like the following:
class TOMLEncoder: TOMLDecoder {
func encode<T>(sourceObject: T) -> String {
return "Sample serialized text..."
}
}
I really don't know how to proceed... from my very limited knowledge I should iterate somehow on the sourceObject properties and create the TOML file from the contents of those properties, but I am not sure if that is the correct approach and how to achieve it. Any help is greatly appreciated. Thanks
I have types for which I provide CodingKeys with custom names for their data members as appropriate. I would like to encode, as required by different call sites, only a subset of a type's CodingKeys before sending the data to a server. The required encodings change from call site to call site, so there's no default implementation to consider.
Example types (for demonstration purposes):
struct Account: Codable
{
let name: String;
var balance: Float;
mutating func set(balance: Float)
{
self.balance = balance; // verifications, etc.
}
}
struct User: Codable
{
enum CodingKeys: String, CodingKey
{
case
id = "db_id",
name = "db_name",
account
// many more keys
}
let id: Int;
let name: String;
var account: Account;
// many more data members
}
Creating an instance:
var user = User(
id: 1, name: "john", account: Account(name: "checking", balance: 10_000));
Using a JSONEncoder works as expected; it produces the following:
{
"db_id" : 1,
"db_name" : "john",
"account" : {
"balance" : 10000,
"name" : "checking"
}
}
I want to encode a subset of the User type in order to send that data back to a server so that I can update specific data fields instead of updating the entire set of properties of my type. Mock usage example:
user.account.set(balance: 15_000);
let jsonEncoding = JSONEncodeSubset.of(
user, // instance to encode
keys: [User.CodingKeys.id, User.CodingKeys.account] // data to include
);
The resulting produced JSON would look like so:
{
"db_id" : 1,
"account" : {
"balance" : 15000,
"name" : "checking"
}
}
Server side, we now have the exact data we need to perform our desired update.
Another example: somebody entered the wrong name for our user, therefore another update request looks like this:
user.name = "jon"; // assume the model was modified to make this mutable
let jsonEncoding = JSONEncodeSubset.of(
user, // instance to encode
keys: [User.CodingKeys.id, User.CodingKeys.name] // only encode id & name
);
The expected resulting JSON encoding:
{
"db_id" : 1,
"name" : "jon"
}
Note that we exclude the information that isn't part of the update (user's account). The objective is to optimize the encoding to include only data that is relevant to that specific request.
Considering I have a large list of objects to update, with different call sites updating different things, I'd like to have a succinct way to perform the encoding task.
Encoding a subset of a type's data members improves encoding performance/memory footprint.
Sending a larger count of smaller encoded objects to the server becomes more efficient.
Does Swift provide any such support for encoding subsets of a type's CodingKeys?
Expanding on Joakim Danielson's answer you can use CodingUserInfoKey to pass data to userInfo property of JSONEncoder.
extension CodingUserInfoKey {
static let keysToEncode = CodingUserInfoKey(rawValue: "keysToEncode")!
}
extension JSONEncoder {
func withEncodeSubset<CodingKeys>(keysToEncode: [CodingKeys]) -> JSONEncoder {
userInfo[.keysToEncode] = keysToEncode
return self
}
}
You need to make CodingKeys conform to CaseIterable and implement a custom encode(to:) method:
struct User: Codable {
enum CodingKeys: String, CodingKey, CaseIterable { // add `CaseIterable`
...
}
...
func encode(to encoder: Encoder) throws {
let keysToEncode = encoder.userInfo[.keysToEncode] as? [CodingKeys] ?? CodingKeys.allCases
var container = encoder.container(keyedBy: CodingKeys.self)
for key in keysToEncode {
switch key {
case .id:
try container.encode(id, forKey: .id)
case .account:
try container.encode(account, forKey: .account)
case .name:
try container.encode(name, forKey: .name)
}
}
}
}
And use it like this:
let encoder = JSONEncoder().withEncodeSubset(keysToEncode: [User.CodingKeys.id, .account])
let encoded = try encoder.encode(user)
print(String(data: encoded, encoding: .utf8)!)
// prints: {"db_id":1,"account":{"name":"checking","balance":15000}}
This solution requires that you write a custom encode(to:) for all properties but once that is done it should be easy to use.
The idea is to let the encode(to:) encode only those properties/keys that exists in a given array. So given the example above the function would look like this
func encode(to encoder: Encoder) throws {
var container = encoder.container(keyedBy: CodingKeys.self)
for key in selectedCodingKeys {
switch key {
case .id:
try container.encode(id, forKey: .id)
case .account:
try container.encode(account, forKey: .account)
case .name:
try container.encode(name, forKey: .name)
}
}
}
selectedCodingKeys is new property in the struct
var selectedCodingKeys = [CodingKeys]()
and we could also add a specific function for encoding
mutating func encode(for codingKeys: [CodingKeys]) throws -> Data {
self.selectedCodingKeys = codingKeys
return try JSONEncoder().encode(self)
}
and then the decoding could be done like in this example
var user = User(id: 1, name: "John", account: Account(name: "main", balance: 100.0))
do {
let data1 = try user.encode(for: [.id, .account])
let data2 = try user.encode(for: [.id, .name])
} catch {
print(error)
}
Im running into an issue with ObjectMapper and the way a json response is coming back from a server.
Is there a way to have ObjectMapper parse this response and create an array of rooms?
Currently I cannot straight map the json as the keys are unique, and will change every request, as well I cannot use the ObjectMapper way of accessing nested objects with dot notation, rooms.0.key as I am unsure how many objects there are, as well as the key that will show up.
Is there a simple way of parsing this reponse.
"rooms": {
"165476": {
"id": "165476",
"area": {
"square_feet": 334,
"square_meters": 31
},
"165477": {
"id": "165477",
"area": {
"square_feet": 334,
"square_meters": 31
},
Use Codable to parse the above JSON response.
Models:
struct Rooms: Codable {
let rooms: [String:Room]
}
struct Room: Codable {
let id: String
let area: Area
}
struct Area: Codable {
let squareFeet: Int
let squareMeters: Int
}
Parse the JSON data like so,
do {
let decoder = JSONDecoder()
decoder.keyDecodingStrategy = .convertFromSnakeCase
let response = try decoder.decode(Rooms.self, from: data)
print(response.rooms.first?.key) //for the above json it will print "165476"
} catch {
print(error)
}
Let's say you have a JSON with several fields:
Students: [
student: {
name: "Harry"
surname: "Smith"
age: 24
},
...
]
and decodables:
struct Students : Decodable {
let student: [Student]
}
struct Student : Decodable {
let name: String
let surname: String
let age: Int
}
If you want to test invalid data, do you write a sequence of tests with te following fake data?
Test1 data:
{
name: null
surname: "Smith"
age: 24
}
Test2 data:
{
name: "Harry"
surname: null
age: 24
}
Test3 data:
{
name: "Harry"
surname: "Smith"
age: null
}
And maybe a test with a wrong type?
{
name: "Harry"
surname: "Smith"
age: "24" //<- string
}
So do you write all these tests, one for each of the above jsons or does this not make sense to you?
It makes sense writing separate test for each key-value pair if we were not able to see the reason of failure and we had to handle it ourself for each pair. For example, old way of JSONSerialization to get Dictionary and then manually checking each key-value if they exist, if yes, does type matches etc. If we were able to get all the values and expected then we would use to initialize the type e.g, Student. In this kind of scenario, instead of putting so many if-else to report proper failure one can think of separate cases for each pair.
Now that we have Codable and error thrown clearly provide all the information for failure so a single test is enough.
The Decodable protocol provides a series of specific Decoding errors.
As already mentioned by Kamran use one test and catch all possible Decoding errors for example
do {
XCTAssertNoThrow(try JSONDecoder().decode(Students.self, from: data))
} catch let DecodingError.dataCorrupted(context) {
XCTFail("Data corrupted: \(context.debugDescription)")
} catch let DecodingError.keyNotFound(key, context) {
let message = "Key '\(key)' not found: \(context.debugDescription), codingPath: \(context.codingPath)"
XCTFail(message)
} catch let DecodingError.valueNotFound(value, context) {
let message = "Value '\(value)' not found: \(context.debugDescription), codingPath: \(context.codingPath)"
XCTFail(message)
} catch let DecodingError.typeMismatch(type, context) {
let message = "Type '\(type)' mismatch: \(context.debugDescription), codingPath: \(context.codingPath)"
XCTFail(message)
} catch {
XCTFail(error.localizedDescription)
}
I'd primarily test the success way with at least one case.
Also I'd test a few cases with "borderline" values like unusual characters in String values or Integer values around the 32/64bit threshold.
Finally I'd test for one case where all the required (non-optional) values are null. If you later change the model for some properties to be optional this test would fail so you can check if you need to make adaptions because of the changed circumstance.
I have a decodable class:
struct AuthenticationResponse : Decodable {
var status: String
var error: Error
var access_token: String? = ""
var expires_in: Double? = 0
var token_type: String? = ""
var scope: String? = ""
var refresh_token: String? = "
}
struct Error : Decodable {
var desc: String
var code: String
}
In the Error class I have:
And to decode to this class, I have:
URLSession.shared.dataTask(with: request) { (data:Data?, response:URLResponse?, error:Error?) in
if let jsonData = data{
let decoder = JSONDecoder()
print("hey")
print("response: \(String(data:jsonData, encoding:.utf8))")
completion(try! decoder.decode(AuthenticationResponse.self, from: jsonData))
}
}.resume()
As some of the responses I receive are (Success response):
{
“status”: “SUCCESS” “error”: null, "access_token":
"MWVmOWQxMDYwMjQyNDQ4NzQyNTdkZjQ3NmI4YmVjMGZjZGM5N2IyZmNkOTA1 N2M0NDUzODEwYjM5ZWQyNGNkZg",
"expires_in": 3600, "token_type": "bearer", "scope": null,
"refresh_token":
"ZGEwOGZiOWZhMzhhYjBmMzAyOGRmZTA5NjJhMjY2MTk3YzMyMmE1ZDlkNWI2N mJjYmIxMjNkMjE1NWFhNWY0Mg"
}
And then a failed response just contains an error object with desc and code in it.
What i am trying to achieve is a decodable class suitable for both scenarios (When a response is successful and failed) however im not sure how to achieve this. I'm aware i can make 2 separate decodable classes but this would make things messier as i'd have to determine if the response is an error and populate to return different classes.
Does anyone know how i should acheive this>
I will give it a try, but first we need to sort out what I consider a somewhat shoddy question. Since Error is the name of a (famous and widely used) protocol it should be renamed and since you want to be able to leave it empty in your AuthenticationResponse it must obviously be an optional there (bearing the question why it is in the Response at all, but I will leave this aside). This leaves us with the following:
struct AuthError : Decodable {
var desc: String
var code: String
}
struct AuthenticationResponse : Decodable {
var status: String
var error: AuthError?
var access_token: String? = ""
var expires_in: Double? = 0
var token_type: String? = ""
var scope: String? = ""
var refresh_token: String? = ""
}
Then we need some example data for the two relevant cases in question, I used:
let okData = """
{
"status": "SUCCESS",
"error": null,
"access_token":
"MWVmOWQxMDYwMjQyNDQ4NzQyNTdkZjQ3NmI4YmVjMGZjZGM5N2IyZmNkOTA1N2M0NDUzODEwYjM5ZWQyNGNkZg",
"expires_in": 3600,
"token_type": "bearer",
"scope": null,
"refresh_token":
"ZGEwOGZiOWZhMzhhYjBmMzAyOGRmZTA5NjJhMjY2MTk3YzMyMmE1ZDlkNWI2NmJjYmIxMjNkMjE1NWFhNWY0Mg"
}
""".data(using: .utf8)!
let errData = """
{
"desc": "username or password incorrect",
"code": "404"
}
""".data(using: .utf8)!
Now we can define a single enum return type which allows for all our cases:
enum AuthResult {
case ok(response: AuthenticationResponse)
case authError(error: AuthError)
case parseError(description: String)
case fatal
}
which finally allows us to write our parse function for the received authentication data:
func parse(_ jsonData:Data) -> AuthResult {
let decoder = JSONDecoder()
do {
let authRes = try decoder.decode(AuthenticationResponse.self, from: jsonData)
return .ok(response: authRes)
} catch {
do {
let errRes = try decoder.decode(AuthError.self, from: jsonData)
return .authError(error: errRes)
} catch let errDecode {
return .parseError(description: errDecode.localizedDescription)
}
}
}
All this in a Playground will permit usage as in
switch parse(okData) {
case let .ok(response):
print(response)
case let .authError(error):
print(error)
case let .parseError(description):
print("You threw some garbage at me and I was only able to \(description)")
default:
print("don't know what to do here")
}
That is still elegant compared to the mess you would make in most other languages, but the call is still out on wether it would not make more sense to just define AuthenticationResponse as the (regular) return type of the parse function and provide the rest by throwing some enum (conforming to Error) and some suitable payload.
Coming (mainly) from Java I still shun from using exceptions as "somewhat" regular control flow (as in a "regular" login failure), but given Swifts much more reasonable approach to exceptions this might have to be reconsidered.
Anyways, this leaves you with a function to parse either case of your services replies and a decent way to handle them in a "uniform" manner. As you might not be able to modify the behaviour of the service handling your request this might be the only viable option. However, if you are able to modify the service you should strive for a "uniform" reply that would be parseable by a single call to JSONDecoder.decode. You would still have to interpret the optionals (as you should in the above example, since they are still a pain to work with, even given Swifts brilliant compiler support forcing you to "do the right thing"), but it would make your parsing less error prone.