APIs Explorer's Firestore's projects.databases.documents.list has incorrect regex - google-cloud-firestore

Summary: APIs Explorer "Try this API" for projects.databases.documents.list appears (!?) to have an overly restrictive/incorrect regex on parent and requirement on collectionId.
Neither gcloud firestore nor firebase firestore: provides functionality to list collections, I'm planning to write a simple app to do so.
As always, I explored the APIs methods using the excellent APIs Explorer but the projects.databases.documents.list "Try this API" appears (!?) to have an overly restrictive|incorrect regex on parent.
The documentation correctly states that:
https://firestore.googleapis.com/v1/{parent=projects/*/databases/*/documents/*/**}/{collectionId}
And:
Required. The parent resource name. In the format:
projects/{project_id}/databases/{database_id}/documents or
projects/{project_id}/databases/{database_id}/documents/{document_path}.
For example: projects/my-project/databases/my-database/documents or projects/my-project/databases/my-database/documents/chatrooms/my-chatroom
Using Google's first format example: projects/my-project/databases/my-database/documents does not work:
APIs Explorer only accepts the second format example for parent but then requires a value for collectionId which may not be desired:
APIs Explorer appends collectionId to the parent to from the URL. In the case of parent ending /documents (which isn't permitted), this would make sense to access the chatrooms collection or in the case that parent ends /documents/chatrooms/my-chatroom to then get the messages collection (with my-chatroom) but the requirement prohibits using APIs Explorer (!) to query projects/my-project/databases/my-database/documents/chatrooms; collectionId is required and would need to be chatrooms but a parent of projects/my-project/databases/my-database/documents is not permitted.
Using one of my projects (${PROJECT}) and (default) for {database_id}, I can use the documentation's examples correctly in curl:
TOKEN=$(gcloud auth print-access-token)
PROJECT=...
PARENT=projects/${PROJECT}/databases/(default)/documents
COLLECTION=...
curl \
--header "Authorization: Bearer ${TOKEN}" \
--header 'Accept: application/json' \
--write-out '%{response_code}' \
--output /dev/null \
--silent \
https://firestore.googleapis.com/v1/${PARENT}/${COLLECTION}
200
The APIs works correctly navigating down through subcollections too.
Posting this here in the hopes that, if APIs Explorer is indeed incorrect, fixing it can help other developers not encounter this issue and be discouraged.
Note: Since I'm posting feedback, the tool doesn't correctly adjust cURL, HTTP or JavaScript generated examples to reflect the checkbox value on "API Key"; when the "API Key" is deselected, the parameter should not be included in the calls.

It seems to be a problem with the regular expression in the API explorer. I need to enter the parent as projects/fire-template-24/databases/(default)/documents/users/ (including the forward slash) and also enter the collectionId as it's required (users in my case). This seems to be making a query at:
https://[GOOGLE+APIS_URL]/projects/[ID]/databases/(default)/documents/users//users?key=[API_KEY]
This returns an error about the extra trailing slash which the API explorer forced me to add and collectionId field is also appended to it with another /.
The following regular expression seems to be working:
^projects\/[^/]+\/databases\/[^/]+\/documents(\/[^/]+(\/[^/]+\/[^/]+)*)$
The above RegEx doesn't required a forward slash in parent and matches a collection path only. If the collectionId is removed then it should work. Working example of the RegEx can be found here.

I could also observe the same in the API explorer page. I observed that there is an existing issue in the Public Issue Tracker for the same. As mentioned in the #2 comment of the issue, this issue report has been forwarded to the Firestore Engineering team to investigate, but there is no ETA for a resolution. I would suggest you star the issue so that you will get notified whenever there is any update on the issue.

Related

Github API to get the author/committer of specific line of code

I'm using Github code search API to get search for some text in a given repository. The response doesn't seem to contain any information related to the content that matched, sha/hash of the commit or the author/committer info:
https://api.github.com/search/code?q=addClass+repo:jquery/jquery
Is there any way to get the author/committer of the given line of code in a repo using the Github API?
Thanks!
For searching commits we can use the new Commit Search API which is currently available for developers for preview . During the preview period we need to explicitly specify a custom media type in the 'Accept' header. For example your search using this API could be - curl -H 'Accept: application/vnd.github.cloak-preview' \https://api.github.com/search/commits?q=addClass+repo:jquery/jquery
. ref: Search Commits

How to search for code in GitHub with GitHub API?

I'm trying to search for some piece of code using the GitHub API V3 given only the keyword, not limiting by user, organization, or repository.
For example, if I want to search for all pieces of code that contain the keyword "addClass", the results would be
https://github.com/search?q=addClass&type=Code&ref=searchresults without using GitHub API.
But how can I do the same thing through GitHub API? I tried https://api.github.com/search/code?q=addClass
It says "Must include at least one user, organization, or repository". How can I fix this?
You can do a code search without specifying a user/org/repo if you authenticate.
First, generate a personal access token for use for this purpose, from your Profile on GitHub's website:
Settings -> Developer Settings -> Personal Access Token -> Generate New Token (you can leave all access options unticked, since you're just using to make web requests)
Now, your original GET request will work and return results, if you append the token to it:
https://api.github.com/search/code?q=addClass&access_token=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
UPDATE: OCT 2021
As pointed out by a comment below, passing the token in via a query parameter (like above) is deprecated. You must now add it as an Authorization header.
e.g.
curl --location --request GET 'https://api.github.com/search/code?q=addClass +in:file +language:csharp' \
--header 'Authorization: Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
or in Python:
import requests
url = "https://api.github.com/search/code?q=addClass +in:file +language:csharp"
headers = {
'Authorization': 'Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
}
response = requests.request("GET", url, headers=headers)
print(response.text)
2020: As detailed in Mark Z.'s answer, using an authentication (Authorization': 'Token xxxx') allows for a code search.
get /search/code
You can use:
either a dedicated command-line tool like feinoujc/gh-search-cli
ghs code --extension js "import _ from 'lodash'"
or the official GitHub CLI gh, (after a gh auth login) as show in issue 5117:
gh api --method=GET "search/code?q=filename:test+extension:yaml+org:new-org"
Or even:
gh api --method=GET search/code -f q='filename:test extension:yaml org:new-org' \
--jq '.items[] | [.repository.full_name,.path,.sha] | #tsv'
That would get a line-based, tab-separated list of fields in this order: repo name, file path, git sha. (see gh help formatting)
2014 (original answer): That seems related to the new restriction "New Validation Rule for Beta Code Search API" (Oct. 2013)
In order to support the expected volume of requests, we’re applying a new validation rule to the Code Search API. Starting today, you will need to scope your code queries to a specific set of users, organizations, or repositories.
So, the example of the API search code mentions now:
Suppose you want to find the definition of the addClass function inside jQuery. Your query would look something like this:
https://api.github.com/search/code?q=addClass+in:file+language:js+repo:jquery/jquery
While Gihub does not currently support code search without repo, user, or organization (see VonC's answer), codesearch does index some sources from Github via the codesearch API, albeit with an API less fully featured out than Github's.
For example, to search for wget invocations indexed from Github, call
curl "https://searchcode.com/api/codesearch_I/?q=wget&src=2"
The optional src parameter picks the code source (e.g., Github, BitBucket) that should be searched, and you can find its integer value for a source by altering the parameters of faceted search in the codesearch UI. The current value of src for Github is 2.
You can verify that the returned results from the above example come from github.com by viewing the the repo property of results items.

Use soundcloud embed to create embed code from the command line for private sounds

I understand that it's possible to use oembed to get embed code for private files, but I'm running into "403 forbidden" when I try it.
http://soundcloud.com/albert-albala-1/audio-recording-on-monday/s-VzgUg
is accessible in a browser, but
curl -iL http://soundcloud.com/oembed?url=http%3A//soundcloud.com/albert-albala-1/audio-recording-on-monday/s-VzgUg&format=json&auto_play=false
yields a HTTP/1.1 403 Forbidden error. The same call with a public sound works fine:
curl -iL http://soundcloud.com/oembed?url=http%3A//soundcloud.com/osmconcerts/rosemarie-landryv2&format=json&auto_play=false
Related question: Rendering SoundCloud widget for a private track using PHP API, although that one uses PHP. I'm trying to get this to work on the command line.
It's possible to embed private tracks using a secret token if the track's settings allow embedding.
Embedding is allowed by default, but if you're receiving a 403 with a valid secret token, you've probably disallowed public embedding for the track.
You can enable this through the web UI on your track's edit page under Settings -> Advanced -> Widget Settings -> Click to enable -> For everybody:
Just had this problem myself, and the track/sound settings have changed layout/structure since pje's answer from 30/09/2013.
Currently, under Permissions, you need to make sure that 'Display Embed code' is turned on - despite the fact that you may not want this to happen! Doing so should mean a 200 status code is returned along with the embed metadata.
Essentially what this is doing is allowing the track to be publicly embedded on other sites (despite the track being private in terms of not being listed on the main account page).

How can I post in Facebook wall programmatically?

I'm very new to Facebook (yeah, I'm the only one) but I'd like to publish messages in the wall programmatically using Python.
What do I need to do to achieve that? I'm very confused. I have seen that most question about this are focused on how to let an app interact in some way with Facebook, but I'd like to post as if I were a simple user (a user which happens to "own" that particular Facebook page). Do I really need to go and create an app? (By the way, I'm having problems even with that... it tells me that my account seems fake or invalid even after I validated it via SMS). I imagine that there is a simpler way, introducing my credentials, using a POST request but I haven't read something similar.
I would have thought this is what I needed, but it leads to a page to create an app.
Some pointers?
By the way, I got a token via a publish_stream permission in the Graph API Explorer (using the extended permissions tab). Seems like this permission doesn't have expiration date but it's not recommended. Is there a better choice for a token?.
Thanks
Facebook is a bit trickier than other sites it would appear.
While watching the POST tab under NET in Firebug, while logging in, I see the following POST parameters:
charset_test €,´,€,´,水,Д,Є
default_persistent 1
email myusername
lgnjs 1340241652
lgnrnd 182050_CuPx
locale en_US
lsd AVqwALVx
pass mypassword
persistent 1
timezone 240
I don't yet know what lsd paramater is, but I did find out what the lgnrnd was by viewing the source of the login page before I logged in. It appears to be there to make it more difficult to automate authentication.
You first want your script to scrape that login page for that value, store it in a variable. Then, a cURL example would be something like:
curl -L "https://www.facebook.com/login.php?login_attempt=1" \
--cookie-jar /tmp/cookies.txt \
--data-urlencode "email=myusername" \
--data-urlencode "lgnrnd=$lgnrnd" \
--data-urlencode "pass=mypassword"
and so forth, with all the parameters...
This is untested of course, and I don't know what some of those parameter values are yet, but you should get the idea from that. You could probably use a python module like Beautiful Soup or something to apply the above basic concept.
Then you would do the same thing, watching the NET tab to find the parameters sent when you post to your facebook wall, and put it all together.
Note that I'm storing and using cookies with curl's --cookie-jar option. You will need a way to save and use cookies with the request as well, which is why I showed that.
Hope that helps you get started anyway..
What about this?.
import urllib2, urllib
parameters = {}
parameters['access_token'] = MY_TOKEN
parameters['message'] = 'Hello world'
target = 'https://graph.facebook.com/MY_ID/feed'
parameters = urllib.urlencode(parameters)
handler = urllib2.urlopen(target, parameters)
while True:
if handler.code < 400:
print handler.read() # Gets post_id
break
elif handler.code >= 400:
print 'Error' # :-(
break
Are there problems with this? Maybe a better way? Comments?
Thanks

Open Graph custom action that links to other user

I have question about the beta Open Graph stuffs.
The documentation I am looking at is this https://developers.facebook.com/docs/beta/opengraph/
I successfully define custom objects and actions. However, I cannot figure out how to link current user with other users.
For example, say I define a custom action 'kick'. I want the current user to be able to 'kick' one of his/her friends.
The closest object I can think of is 'profile', but when I pass user_id, 'http://www.facebook.com/profile.php?id=', or http://graph.facebook.com/, it does not work.
This is the requests I tried
POST https://graph.facebook.com/me/myapp:kick?access_token=abc&profile=http%3A%2F%2Fwww.facebook.com%2F123
POST https://graph.facebook.com/me/myapp:kick?access_token=&profile=123
POST https://graph.facebook.com/me/myapp:kick?access_token=abc&profile=http%3A%2F%2Fwww.facebook.com%2Fprofile.php%3Fid%3D123
POST https://graph.facebook.com/me/myapp:kick?access_token=abc&profile=http%3A%2F%2Fgraph.facebook.com%2F123
This is the response I get
{"error":{"message":"An unexpected error has occurred. Please retry your request later.","type":"OAuthException"}}
The profile given is for an external website. They call it external profile. Pretty misleading terms.
As you can see I used the following
curl -F 'access_token=TOKEN' \
-F 'profile=http://graph.facebook.com/zuck' \
'https://graph.facebook.com/me/MYAPP:kick'
And it gave the above.
I guess you can fill the external profile with meta info from the Facebook users and it will show alright (for example I can send the profile URL facebook.com/zuck instead), but it seems backwards, inefficient and not the intended usage.
For example feeding it my link.
curl -F 'access_token=TOKEN' \
-F 'profile=http://facebook.com/username' \
'https://graph.facebook.com/me/MYAPP:kick'
But the thing is ... I am not a musician.
It does though seem to look alright in aggregation view.
Which is interesting/weird since one of objects shown in this picture is a Facebook profile, so you would think they would give an example with Facebook profiles as objects.
Though they did say
We are now extending the Open Graph to include arbitrary actions and objects created by 3rd party apps and enabling these apps to integrate deeply into the Facebook experience
Which most likely means they want you to create arbitrary objects outside of Facebook.
Can you still link to other users?
Yes, but through tagging using tags=FacebookID1,FacebookID2 but it will be in the form of
phwd kicked a musician with Friend1 and Friend2 on [APP NAME]