I’m struggling on how to retrieve an attachment from a CouchDB in Server-Swift.
I understand that CouchDB has a kind of ‘flag’ (called stub) to indicate that there is an attachment.
In the JSON structure I get back from CouchDB I can spot under the key “_attachments” metadata (Content-Type, length etc)
I know how to retrieve that attachment from the CLI (curl -X GET ip:port{id}/{attachment_file}) but I’m drawing a blind how to do it from Swift.
The code to go through the results of a database.retrieveAll() call:
if let docs = docs {
for document in docs["rows"].arrayValue {
var plaatje = [String: Any]()
plaatje["ordernum"] = document["doc"]["orderNumber"].stringValue
plaatje["img"] = // what to put here?????
plaatjes.append(plaatje)
}
}
Is there a method of the database instance or any other way to get the binary of the attachment from Swift?
Related
I am using the following:
Apollo iOS
Neo4j Database (with the GraphQL plugin)
GraphQL
Singleton Class for Connection
My code below is to connect to the GraphQL endpoint which happens to be my Neo4j Database.
class Network {
static let shared = Network()
private(set) lazy var apollo: ApolloClient = {
let url = URL(string: "http://localhost:7474/graphql/")!
let keychain = KeychainSwift()
let configuration = URLSessionConfiguration.ephemeral
configuration.httpAdditionalHeaders = ["Authorization": "\(String(describing: keychain.get("neo.auth")))"]
return ApolloClient(
networkTransport: HTTPNetworkTransport(url: url, session: URLSession(configuration: configuration))
)
}()
}
When I create data inside my database via my app this works great however if I then query to the database to return all the data, the newly created data is not found. The data is in the database because I can see it in the browser and also if I restart my app and then run the exact same code to return all the data; it returns it.
The other thing I have tried to do is run two instances of my app side by side in the simulator. Both apps have the same features and can import/export data. When I create new data via one instance the data is created in the database successfully however upon importing the data in the other instance of the app - it returns nothing (Both apps being ran at the same time).
My Import Code
func importData(){
let apollo = Network.shared.apollo
//Import all data from the graph database
apollo.fetch(query: GetAllQuery()) { result in
guard let data = try? result.get().data else { return }
print(data.jsonObject.values)
}
}
The only thing I can think of is that the session is not updating when new data is created in the database. The reason I feel this is because if I relaunch the app and run my import function it actually returns all the new data. I need the connection to update when new data is created, is there a way I can refresh the connection upon data creation?
I found a fix that is working so far for me:
apollo.clearCache()
I am using this before I run a query to the database and so far it has pulled down all the correct data. My issue was that when I exported data to the database and then tried importing it (or reading it) directly after it never returned anything or returned data that already existed in the system.
I'm trying to add records to my Firebase database as follows:
So basically we have matches -> user_supplied_id -> {id,location}
This is achievable using the following code and the Swift API:
let matches = Database.database().reference().child("users").child(UserDefaults.standard.object(forKey: "uid") as! String).child("matches").child((all_listings?[index].listingId)!)
let newBookData = [
"id": all_listings?[index].listingId,
"location" : all_listings?[index].location
] as [String : Any]
matches.setValue(newBookData)
I am now trying to replicate this behaviour using the Firebase REST API. I'm basically sending a POST request to the address:
https://PROJECTID.firebaseio.com/.../matches/-LOpJmU9yj4hAocHjnrB.json
with the following data:
"{\"id\":\"-LOpJmU9yj4hAocHjnrB\",\"location\":\"Edinburgh\"}"
However, this results in the following outcome instead:
As you can see, it creates an additional ID and level of nesting before adding the elements to the database. How can I fix this?
Don't use POST. According to the documentation:
To accomplish the equivalent of the JavaScript push() method (see Lists of Data), you can issue a POST request.
You don't want a push here. A push operation creates a new random push ID and makes that the key of the data you provided.
If already you know the location you want to set (it looks like you already have a known push id), just use a PUT to set the data at that location.
so I already finished all of the actual app for this. I just need to setup the backend. I figured Firebase was the best solution since Parse is no longer a thing. What I wanted was:
Users with profiles - These profiles can be viewed by added friends but only edited (written) to by the actual profile owner.
So I read through the Firebase Docs and still cannot really figure out how to do this. They only have 1 Swift application example that does not do anything similar and the one Obj C twitter one, will not even build. All of their docs still have println for Swift which just makes me think it is not updated frequently.
Does anyone have any good examples / tutorials of this? I keep trying to search for things but nothing is as similar enough to what I want. I am more looking on how to setup the db for each user and access it rather actually using Firebase in Swift.
As I wrote in my comment to your question, this answer is based on what we do in a real social app Impether using Swift + Firebase.
Data structure
Let's assume that you want to store the following information for a single user:
email
username
name
followers - number of people who follow a particular user
following - number of people who a particular user follows
avatar_url - url of their avatar
bio - some additional text
Since in Firebase everything is stored a JSON objects, you can store the above structure under node with path like users/$userId, where $userId is Firebase User UID which is created for each registered user if you use simple email/password Firebase authorization.
Firebase email/password authorization is described in their docs:
https://www.firebase.com/docs/ios/guide/user-auth.html
https://www.firebase.com/docs/ios/guide/login/password.html
Notice that there are both Obj-C and Swift snippets. I find Firebase documentation really great as it helped me a lot when I was building our app.
For the purpose of this answer let's assume that we have user with username jack and Firebase User UID equal to jack_uid (in reality this will be a string generated by Firebase).
Then an example data for this user will be store under a path users/jack_uid and can look like this:
{
"email" : "jack#example.com",
"username" : "jack",
"name" : "Jack",
"followers" : 8,
"following" : 11,
"avatar_url" : "http://yourstoragesystem.com/avatars/jack.jpg",
"bio" : "Blogger, YouTuber",
}
Firebase email/password authorization works really well, but let's be honest, if user wants to sign in into the app, it's a lot better for him to use his username than his email he gave while he registering his account.
In order to do that, we decided to store a mapping from usernames to user ids. The idea is that if user inputs his username and password in a login form, we use that mapping to retrieve his user id and then we try to sign him in using his user id and provided password.
The mapping can be stored for example under a path username_to_uid and looks like this:
{
"sample_username_1": "firebase_generated_userid_1",
"sample_username_2": "firebase_generated_userid_2",
...
"jack": "jack_uid",
"sample_username_123": "firebase_generated_userid_123"
}
Then creating a profile may looks like this and it's done as soon as registration of a new account was successful (this snippet is very close to the exact code we use in the production):
func createProfile(uid: String, email: String,
username: String, avatarUrl: String,
successBlock: () -> Void, errorBlock: () -> Void) {
//path to user data node
let userDataPath = "/users/\(uid)"
//path to user's username to uid mapping
let usernameToUidDataPath = "/username_to_uid/\(username)"
//you want to have JSON object representing user data
//and we do use our User Swift structures to do that
//but you can just create a raw JSON object here.
//name, avatarUrl, bio, followers and following are
//initialized with default values
let user = User(uid: uid, username: username, name: "",
avatarUrl: avatarUrl, bio: "",
followers: 0, following: 0)
//this produces a JSON object from User instance
var userData = user.serialize()
//we add email to JSON data, because we don't store
//it directly in our objects
userData["email"] = email
//we use fanoutObject to update both user data
//and username to uid mapping at the same time
//this is very convinient, because either both
//write are successful or in case of any error,
//nothing is written, so you avoid inconsistencies
//in you database. You can read more about that technique
//here: https://www.firebase.com/blog/2015-10-07-how-to-keep-your-data-consistent.html
var fanoutObject = [String:AnyObject]()
fanoutObject[userDataPath] = userData
fanoutObject[usernameToUidDataPath] = uid
let ref = Firebase(url: "https://YOUR-FIREBASE-URL.firebaseio.com/images")
ref.updateChildValues(fanoutObject, withCompletionBlock: {
err, snap in
if err == nil {
//call success call back if there were no errors
successBlock()
} else {
//handle error here
errorBlock()
}
})
}
In addition to this you possibly want to store for each user a list of his followers and a separate list of users he follows. This can be done just by storing user ids at a path like followers/jack_uid, for example it can look like this:
{
"firebase_generated_userid_4": true,
"firebase_generated_userid_14": true
}
This is the way we store sets of values in our app. It very convenient, because it is really user to update it and check if some value is there.
In order to count the number of followers, we put this counter into user's data directly. This makes reading the counter very efficient. However, updating this counter requires using transactional writes and the idea is almost exactly the same as in my answer here: Upvote/Downvote system within Swift via Firebase
Read/write permissions
A part of your question is how to handle permissions to data you store. The good news is that Firebase is exceptionally good here. If you go to your Firebase dashboard there is a tab named Security&Rules and this is the place where you control permissions to your data.
What's great about Firebase rules is that they are declarative, which makes them very easy to use and maintain. However, writing rules in pure JSON is not the best idea since it's quite hard to control them when you want to combine some atomic rules into a bigger rule or your app simple grows and there are more and more different data you store in your Firebase database. Fortunately, Firebase team wrote Bolt, which is a language in which you can write all rules you need very easily.
First of all I recommend to read Firebase docs about Security, especially how does permission to a node influences permission for its children. Then, you can take a look at Bolt here:
https://www.firebase.com/docs/security/bolt/guide.html
https://www.firebase.com/blog/2015-11-09-introducing-the-bolt-compiler.html
https://github.com/firebase/bolt/blob/master/docs/guide.md
For example, we use rules for managing users data similar to this:
//global helpers
isCurrentUser(userId) {
auth != null && auth.uid == userId;
}
isLogged() {
auth != null;
}
//custom types, you can extend them
//if you want to
type UserId extends String;
type Username extends String;
type AvatarUrl extends String;
type Email extends String;
type User {
avatar_url: AvatarUrl,
bio: String,
email: Email,
followers: Number,
following: Number,
name: String,
username: Username,
}
//user data rules
path /users/{$userId} is User {
write() { isCurrentUser($userId) }
read() { isLogged() }
}
//user's followers rules
//rules for users a particular
//user follows are similar
path /followers/{$userId} {
read() { isLogged() }
}
path /followers/{$userId}/{$followerId} is Boolean {
create() { isCurrentUser($followerId) && this == true }
delete() { isCurrentUser($followerId) }
}
//username to uid rules
path /username_to_uid {
read() { true }
}
path /username_to_uid/{$username} is UserId {
create() { isCurrentUser(this) }
}
The bottom line is that you write rules you want using Bolt, then you compile them into JSON using Bolt compiler and then you deploy them into your Firebase, using command line tools or by pasting them into dashboard, but command line is way more efficient. A nice additional feature is that you can test your rules by using tools in Simulator tab in your dashboard.
Summary
For me Firebase is a great tool for implementing a system you want. However, I recommend to start with simple features and learn how to use Firebase in the first place. Implementing social app with functionality like for example Instagram is quite a big challenge, especially if you want to do it right :) It's very tempting to put all functionality there very quickly and Firebase makes it relatively easy to do, but I recommend to be patient here.
In addition, take your time and invest in writing tools. For example, we have two separated Firebase databases, one for production and second for testing, which is really important if you want to write unit and UI tests efficiently.
Also, I recommend building permission rules from the beginning. Adding them later may be tempting, but also quite overwhelming.
Last but not least, follow Firebase blog. They post regularly and you can be up to date with their latest features and updates - this is how I learnt how to use concurrent writes using fanout technique.
I am building a REST API and facing this issue: How can REST API pass very large JSON?
Basically, I want to connect to Database and return the training data. The problem is in Database I have 400,000 data. If I wrap them into a JSON file and pass through GET method, the server would throw Heap overflow exception.
What methods we can use to solve this problem?
DBTraining trainingdata = new DBTraining();
#GET
#Produces("application/json")
#Path("/{cat_id}")
public Response getAllDataById(#PathParam("cat_id") String cat_id) {
List<TrainingData> list = new ArrayList<TrainingData>();
try {
list = trainingdata.getAllDataById(cat_id);
Gson gson = new Gson();
Type dataListType = new TypeToken<List<TrainingData>>() {
}.getType();
String jsonString = gson.toJson(list, dataListType);
return Response.ok().entity(jsonString).header("Access-Control-Allow-Origin", "*").header("Access-Control-Allow-Methods", "GET").build();
} catch (SQLException e) {
logger.warn(e.getMessage());
}
return null;
}
The RESTful way of doing this is to create a paginated API. First, add query parameters to set page size, page number, and maximum number of items per page. Use sensible defaults if any of these are not provided or unrealistic values are provided. Second, modify the database query to retrieve only a subset of the data. Convert that to JSON and use that as the payload of your response. Finally, in following HATEOAS principles, provide links to the next page (provided you're not on the last page) and previous page (provided you're not on the first page). For bonus points, provide links to the first page and last page as well.
By designing your endpoint this way, you get very consistent performance characteristics and can handle data sets that continue to grow.
The GitHub API provides a good example of this.
My suggestion is no to pass the data as a JSON but as a file using multipart/form-data. In your file, each line could be a JSON representing a data record. Then, it would be easy to use a FileOutputStream to receive te file. Then, you can process the file line by line to avoid memory problems.
A Grails example:
if(params.myFile){
if(params.myFile instanceof org.springframework.web.multipart.commons.CommonsMultipartFile){
def fileName = "/tmp/myReceivedFile.txt"
new FileOutputStream(fileName).leftShift(params.myFile.getInputStream())
}
else
//print or signal error
}
You can use curl to pass your file:
curl -F "myFile=#/mySendigFile.txt" http://acme.com/my-service
More details on a similar solution on https://stackoverflow.com/a/13076550/2476435
HTTP has the notion of chunked encoding that allows you send a HTTP response body in smaller pieces to prevent the server from having to hold the entire response in memory. You need to find out how your server framework supports chunked encoding.
My task is to display the supported document types on an iPhone with OS 3.x, such as .pdf, .rtf, .doc, .ppt, .png, .tiff etc.
Now, I have stored these files only encrypted on disk. For security reasons, I want to avoid storing them unencrypted on disk.
Hence, I prefer to use loadData:MIMEType:textEncodingName:baseURL: instead of loadRequest: to display the document because loadData allows me to pass the content in a NSData object, i.e. I can decrypt the file in memory and have no need to store it on disk, as it would be required when using loadRequest.
The problem is that loadData does not appear to work with all file types:
Testing shows that all picture types seem to work fine, as well as PDFs, while the more complex types don't. I get a errors such as:
NSURLErrorDomain Code=100
NSURLErrorDomain Code=102
WebView appears to need a truly working URL for accessing the documents as a file, despite me offering all content via the NSData object already.
Here's the code I use to display the content:
[webView loadData:data MIMEType:type textEncodingName:#"utf-8" baseURL:nil];
The mime-type is properly set, e.g. to "application/msword" for .doc files.
Does anyone know how I could get loadData to work with all types that loadRequest supports? Or, alternatively, is there some way I can tell which types do work for sure (i.e. officially sanctioned by Apple) with loadData? Then I can work twofold, creating a temp unencrypted file only for those cases that loadData won't like.
Update
Looks like I'm not the first one running into this. See here:
http://osdir.com/ml/iPhoneSDKDevelopment/2010-03/msg00216.html
So, I guess, that's the status quo, and nothing I can do about it.
Someone suggested a work-around which might work, though:
http://osdir.com/ml/iPhoneSDKDevelopment/2010-03/msg00219.html
Basically, the idea is to provide a tiny http server that serves the file (from memory in my case), and then use loadRequest. This is probably a bit more memory-intensive, though, as both the server and the webview will probably both hold the entire contents in memory as two copies then, as opposed to using loadData, where both would rather share the same data object. (Mind you, I'll have to hold the decrypted data in memory, that's the whole point here).
I experienced a very similar issue (i get my files from a server however) and saw your post and thought it was a dead end and then just by chance started to experiment on the device (iPad, in this instance) and it worked when i gave the baseURL as what i used to get it from the server and it worked but does not work on the simulator. I would try that, otherwise I would submit a bug report to Apple.
Here is solution via NSURLProtocol:
class CoreDataFileURLProtocol : NSURLProtocol {
var connection: NSURLConnection!
override class func canInitWithRequest(request: NSURLRequest) -> Bool {
return (request.URL.scheme == "coredatafile")
}
override class func canonicalRequestForRequest(request: NSURLRequest) -> NSURLRequest {
return request
}
override func startLoading() {
if let file_id = self.request.URL.absoluteString?.lastPathComponent {
if let file = SAFile.MR_findFirstByAttribute("file_id", withValue: file_id) as? SAFile {
let response = NSURLResponse(URL: request.URL, MIMEType: file.mime, expectedContentLength: Int(file.filesize), textEncodingName: "utf-8")
client?.URLProtocol(self, didReceiveResponse: response, cacheStoragePolicy: .NotAllowed)
client?.URLProtocol(self, didLoadData: file.data)
client?.URLProtocolDidFinishLoading(self)
}
}
}
override func stopLoading() {
}
}
Now you need only register class:
NSURLProtocol.registerClass(CoreDataFileURLProtocol.self)
And create a request with file_id:
let url = NSURL(scheme: "coredatafile", host: "myapp.com", path: "/\(file.file_id)")
webView.loadRequest(NSURLRequest(URL: url!))