i am making a REST API in golang and i want to add support for filtering fields but i don't know the best way to implement that, lets say i have this structure representing an Album model
type Album struct {
ID uint64 `json:"id"`
User uint64 `json:"user"`
Name string `json:"name"`
CreatedDate time.Time `json:"createdDate"`
Privacy string `json:"privacy"`
Stars int `json:"stars"`
PicturesCount int `json:"picturesCount"`
}
and a function that returns an instance of an Album
func GetOne(id uint64, user uint64) (Album, error) {
var album Album
sql := `SELECT * FROM "album" WHERE "id" = $1 AND "user" = $2;`
err := models.DB.QueryRow(sql, id, user).Scan(
&album.ID,
&album.User,
&album.Name,
&album.CreatedDate,
&album.Privacy,
&album.Stars,
&album.PicturesCount,
)
return album, err
}
and the client was to issue a request like this
https://api.localhost.com/albums/1/?fields=id,name,privacy
obvious security issues aside, my first thought was to filter the fields in the database using something like this
func GetOne(id uint64, user uint64, fields string) {
var album Album
sql := fmt.Sprintf(`SELECT %s FROM "album" WHERE "id" = $1 AND "user" = $2;`, fields)
// i don't know what to do after this
}
and then i thought of adding omitempty tag to all the fields and setting the fields to their zero value before encoding it to JSON,
would this work?
which one is the better way?
is there a best way?
how would i go about implementing the first method?
Thank you.
For your first proposal (querying only the requested fields) there are two approaches (answering "would this work?" and "how would I go about implementing the first method?"):
Dynmaically reate a (possibly anonymous) struct and generate JSON from there using encoding/json.
Implement a wrapper that will translate the *database/sql.Rows you get back from the query into JSON.
For approach (1.), you will somehow need to create structs for any combination of attributes from your original struct. As reflect cannot create a new struct type at runtime, your only chance would be to generate them at compile time. The combinatorial explosion will bloat your binary, so do not do that.
Approach (2.) is to be handled with caution and can only be a last resort. Taking the list of requested fields and writing out JSON with the values you got from DB sounds straightforward and does not involve reflection. However your solution will be (very likely) much more unstable than encoding/json.
When reading your question I too thought about using the json:"omitempty" struct tag. And I think that it is the preferable solution. It does neither involve metaprogramming nor writing your own JSON encoder, which is a good thing. Just be aware of the implications in case some fields are missing (client side maybe has to account for that). You could query for all attributes always and override the unwanted ones using reflection.
In the end, all above solutions are suboptimal, and the best solution would be to not implement that feature at all. I hope you have a solid reason to make attributes variable, and I am happy to further clarify my answer based on your explaination. However, if one of the attributes of a resource is too large, it maybe should be a sub-resource.
Related
I'm loading some data from a CSV file that has some extra notes fields that I don't want in the DB. Is there an option to just ignore extra fields when storing to the DB?
I think mongoose did this by default - which does have a downside that stuff goes missing without warning if your schema is wrong but... thats what i want in this case.
Otherwise what is a way to reflect and get the schema so I can remove extra fields from the data manually?
I'm getting this error on .create
Unknown arg `notes` in data.notes for type WalletCreateInput.
Did you mean `name`?
Available args:
...
It's not allowed to add extra fields while interacting with Prisma Query.
The current behaviour is intentional and it throws the error as an extra validation layer to make sure you're passing the right data.
There is a feature request that discusses allowing extra fields while interacting with Query.
As of now, destructuring the fields which are necessary and passing only the required fields is the only option.
Late to the party, but there is a way around this.
If you use the "fieldReference" preview feature:
generator client {
provider = "prisma-client-js"
previewFeatures = ["fieldReference"]
}
You can then create the following to strip out any extra keys.
function stripPrisma<T extends {}>(input: {fields:{}},data: T) : T {
let validKeys = Object.keys(input.fields);
let dataCopy: any = {...data};
for(let key of Object.keys(data)) {
if(!(validKeys.includes(key))) {
delete dataCopy[key];
}
}
return dataCopy as T;
}
And use it like this
data = stripPrisma(prisma.myTable, data);
prisma.myTable.create({data:data});
It is not perfect, since it will only be able to use "checked input", meaning you can only use the foreign key in your input and not the foreign object.
Let's say we have an HTTP endpoint to get all elements by name?
GET /elements?name={name}
{name} can have a value of CSV or be absent
valid:
GET /elements?name=Bill,Mary,Ann
GET /elements?name=Mike
GET /elements
invalid:
GET /elements?name=
Somehow we find out in controller that name is not passed. We know that the contract implies to return all values for elements. Possible decisions on further actions (I've seen in different projects) are:
using a NULL or a "dummy" substitution like a secret char sequence "!#$#%#$" and juggling them in database while building a query
using if (present) { executeQueryA } else { executeQueryB } logic
I am not sure I like either of these approaches, because when there is more than one optional filter these designs become unmaintainable. Something makes me believe that there is a better way to handle the situation.
What would be a proper design on back-end and in database query to handle the case "select all" when nothing is given? Just a general idea and some pseudo-code will be much appreciated.
I work with an array of string, each string var is a coded object.
I want to decode the object, when I print a string var I get something structured like that :
"firstName=\"Elliot\" lastName=\"Alderson\" gender=\"male\" age=\"33\",some description I also need to get"
Is that a standard format to store key value properties ? I can't find anything on internet. The keys are always the same so that's not a big deal to get theses values as a dictionary but I would like to know if there is like a best practice method to get theses data instead of just searching for each key and then reach value from the first quote to the second one (for each value)
Because my file is 30000 lines so I better choose the more optimized way.
Thanks !
So I am really new to parse and I have no idea how to store a famous quote and it's own author, so then I can use that information in my iOS app made with swift.
I have imported all the frameworks in order to make parse work but I don't know to to store that quotes with the authors and then retrieve the information to display the quote in a label and the author in another label. Please don't be rude, I don't get who to make this work.
So if you want to save a text and retrieve to Parse
let's say you want to save some text into a class called Data
Save
var data = "Swift is nice"
var object = PFObject(className:"Data")
object["message"] = data
object.saveInBackground()
So I use the saveInBackground method just for simplicity however if you should other saveInBackground method where you could check if there is no error while you are saving into Parse.
Retrieve
#IBOutlet weak var textlabel:UILabel!
var query = PFQuery(className:"Data")
query.getFirstObjectInBackgroundWithBlock({ (objects:PFObject?, error:NSError?) -> Void in
if error == nil {
let retrieveData = objects?.objectForKey("message") as! String
//so lets say you had a UILAbel
self.textlabel.text = retrieveData
}
})
I used the getFirstObjectInBackgroundWithBlock method because I assume that I have at least one object save in Parse. So if you are retrieve a lot of data you could findObjects method .
Hope that helps :)
To answer your comment on Lamars's fine answer:
Create a class ("Add class") "Quotes" in Parse, if you haven't already. Create a column in that class named "Quote", set it's type to String. Click "Add Row" and you will get a new row in your table. Double Click it's field and you can write your quote there.
This is me editing the usernameField in my Parse Class:
You could have another column named "Author" of type String, and just do the same thing, but if your app gets more advanced and you would like to display all the quotes from a specific author, you should add another class, named "Author". Add a column named "Name", double click and submit your name.
In your "Quotes" class, add a column named "Author", type Pointer, and make it point to your Author Class. Then copy the correct objectId from Pointer (let's sat Steve Jobs has objectId "12345678") and paste it to the "Author" column in Quotes. Now, if there's another quote by Steve Jobs, you can re-use that objectId, not having to store the name "Steve Jobs" more than once.
I understand you're new to Parse.com and maybe databases as well, but this way of creating relations is very good knowledge, if you want to design stuff in the future.
Parse has a great documentation, in Obj-C and Swift, check it out:
https://www.parse.com/docs/ios/guide
How does one use Firebase to do basic auto-completion/text preview?
For example, imagine a blog backed by Firebase where the blogger can tag posts with tags. As the blogger is tagging a new post, it would be helpful if they could see all currently-existing tags that matched the first few keystrokes they've entered. So if "blog," "black," "blazing saddles," and "bulldogs" were tags, if the user types "bl" they get the first three but not "bulldogs."
My initial thought was that we could set the tag with the priority of the tag, and use startAt, such that our query would look something like:
fb.child('tags').startAt('bl').limitToFirst(5).once('value', function(snap) {
console.log(snap.val())
});
But this would also return "bulldog" as one of the results (not the end of the world, but not the best either). Using startAt('bl').endAt('bl') returns no results. Is there another way to accomplish this?
(I know that one option is that this is something we could use a search server, like ElasticSearch, for -- see https://www.firebase.com/blog/2014-01-02-queries-part-two.html -- but I'd love to keep as much in Firebase as possible.)
Edit
As Kato suggested, here's a concrete example. We have 20,000 users, with their names stored as such:
/users/$userId/name
Oftentimes, users will be looking up another user by name. As a user is looking up their buddy, we'd like a drop-down to populate a list of users whose names start with the letters that the searcher has inputted. So if I typed in "Ja" I would expect to see "Jake Heller," "jake gyllenhaal," "Jack Donaghy," etc. in the drop-down.
I know this is an old topic, but it's still relevant. Based on Neil's answer above, you more easily search doing the following:
fb.child('tags').startAt(queryString).endAt(queryString + '\uf8ff').limit(5)
See Firebase Retrieving Data.
The \uf8ff character used in the query above is a very high code point
in the Unicode range. Because it is after most regular characters in
Unicode, the query matches all values that start with queryString.
As inspired by Kato's comments -- one way to approach this problem is to set the priority to the field you want to search on for your autocomplete and use startAt(), limit(), and client-side filtering to return only the results that you want. You'll want to make sure that the priority and the search term is lower-cased, since Firebase is case-sensitive.
This is a crude example to demonstrate this using the Users example I laid out in the question:
For a search for "ja", assuming all users have their priority set to the lowercased version of the user's name:
fb.child('users').
startAt('ja'). // The user-inputted search
limitToFirst(20).
once('value', function(snap) {
for(key in snap.val()){
if(snap.val()[key].indexOf('ja') === 0) {
console.log(snap.val()[key];
}
}
});
This should only return the names that actually begin with "ja" (even if Firebase actually returns names alphabetically after "ja").
I choose to use limitToFirst(20) to keep the response size small and because, realistically, you'll never need more than 20 for the autocomplete drop-down. There are probably better ways to do the filtering, but this should at least demonstrate the concept.
Hope this helps someone! And it's quite possible the Firebase guys have a better answer.
(Note that this is very limited -- if someone searches for the last name, it won't return what they're looking for. Hence the "best" answer is probably to use a search backend with something like Kato's Flashlight.)
It strikes me that there's a much simpler and more elegant way of achieving this than client side filtering or hacking Elastic.
By converting the search key into its' Unicode value and storing that as the priority, you can search by startAt() and endAt() by incrementing the value by one.
var start = "ABA";
var pad = "AAAAAAAAAA";
start += pad.substring(0, pad.length - start.length);
var blob = new Blob([start]);
var reader = new FileReader();
reader.onload = function(e) {
var typedArray = new Uint8Array(e.target.result);
var array = Array.prototype.slice.call(typedArray);
var priority = parseInt(array.join(""));
console.log("Priority of", start, "is:", priority);
}
reader.readAsArrayBuffer(blob);
You can then limit your search priority to the key "ABB" by incrementing the last charCode by one and doing the same conversion:
var limit = String.fromCharCode(start.charCodeAt(start.length -1) +1);
limit = start.substring(0, start.length -1) +limit;
"ABA..." to "ABB..." ends up with priorities of:
Start: 65666565656565650000
End: 65666665656565650000
Simples!
Based on Jake and Matt's answer, updated version for sdk 3.1. '.limit' no longer works:
firebaseDb.ref('users')
.orderByChild('name')
.startAt(query)
.endAt(`${query}\uf8ff`)
.limitToFirst(5)
.on('child_added', (child) => {
console.log(
{
id: child.key,
name: child.val().name
}
)
})