I want to use the binary protocol in pyorient (faster performance then HTTP).
I want to process JSON response though from query like below:
response = client.query("select #this.toJson('out_*:-1') from Worker where userName = '" + userName + "'")
However I get a JSON serialisation issue from the Pyorient library - is it possible (NOT using http API) to process JSON from query response using only binary protocol?
Any help appreciated!
try this as your query:
response = client.query("select #this.toJson('fetchPlan:out_*:-1') from Worker where userName = '" + userName + "'")
Related
I am trying to get all data into excel using the Get data from web option but I am not sure how to properly configure it. I did get the API key.
EDIT:
Below is the path I use, but that is only the time entries for one user, I need them for ALL users.
https://api.clockify.me/api/v1/workspaces/myWorkspace/user/myUser/time-entries
Any help is appreciated.
EDIT: I didn't read that you wanted to use Power Query in Excel. The Advanced Editor only works with this code in Power BI Desktop.
You'll need to program the request in the Advanced Editor in Power Query.
Here is an example of connecting to the Get Workspace Clients endpoint:
let
baseUrl = "https://api.clockify.me/api/v1/",
apiKey = "your-api-key",
workspaceID = "5dfb421c1b30342708229760",
GetWorkspace = (workspaceID as text) =>
let
options = [Headers = [#"X-Api-Key"= apiKey, #"Content-Type" = "application/Json"],
RelativePath = "workspaces/" & workspaceID & "/clients"],
call = Web.Contents(baseUrl, options),
jsonParsed = Json.Document(call)
in
jsonParsed
in
GetWorkspace
Using this function you just have to change the parameters you need according to the endpoint you want to hit.
The baseUrl will be the same, you'll need to change the RelativePath with the rest of the url and if you need to pass some query parameters, put them after RelativePath in a record, like this:
RelativePath = "workspaces/" & workspaceID & "/clients", Query = [page = "1"]],
I recommend using Postman to make the calls and Fiddler to track how the Url is being constructed. Then compare Postman requests with your power query requests, to check the differences.
Here are some other threads about the subject:
How to access Clockify API through Power Query
How to pull data from Toggl API with Power Query?
I'm having trouble writing to an Azure Block Blob from C++ using a SAS (Shared Access Signature). I'm using the Blob REST API and Poco. The HTTP request returns error 404 (resource does not exist), but I can't figure out what I'm doing wrong.
I generate the SAS on the server in C# like this (seems to work fine):
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("my-blob");
container.CreateIfNotExists();
SharedAccessBlobPolicy sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(40);
sasConstraints.Permissions = SharedAccessBlobPermissions.Write | SharedAccessBlobPermissions.List;
string sasContainerToken = container.GetSharedAccessSignature(sasConstraints);
return Request.CreateResponse(HttpStatusCode.OK, container.Uri + sasContainerToken);
In the Azure portal I can indeed see the Blob container being created as expected. I receive this SAS in C++ using an HTTP request. What I get looks like this (some names and signature replaced for security reasons):
https://myname.blob.core.windows.net/my-blob?sv=2012-02-12&se=2016-06-07T11%3A13%3A19Z&sr=c&sp=wl&sig=%%%%%%%%%%%%%%%%%%%%%%%
Then I try to create the file using Poco and the Blob REST API. That looks like this:
std::string cloudUrl = sasURI + "&restype=container";
std::string fileName = "fname.ext";
Poco::URI* uri = new Poco::URI(cloudUrl.c_str());
std::string* path = new std::string(uri->getPathAndQuery());
Poco::Net::HTTPSClientSession* session = new Poco::Net::HTTPSClientSession(uri->getHost(), uri->getPort());
std::string method = Poco::Net::HTTPRequest::HTTP_PUT;
Poco::Net::HTTPRequest* request = new Poco::Net::HTTPRequest(method, *path, Poco::Net::HTTPMessage::HTTP_1_1);
request->add("x-ms-blob-content-disposition", "attachment; filename=\"" + fileName + "\"");
request->add("x-ms-blob-type", "BlockBlob");
request->add("x-ms-meta-m1", "v1");
request->add("x-ms-meta-m2", "v2");
Poco::Net::HTTPResponse* httpResponse = new Poco::Net::HTTPResponse();
int fileContent = 42;
request->setContentLength(sizeof(int));
request->setKeepAlive(true);
std::ostream& outputStream = session->sendRequest(*request);
outputStream << fileContent;
std::istream &is = session->receiveResponse(*httpResponse);
Poco::Net::HTTPResponse::HTTPStatus status = httpResponse->getStatus();
std::ostringstream outString;
Poco::StreamCopier::copyStream(is, outString);
if (status != Poco::Net::HTTPResponse::HTTP_OK)
{
Logger::log("Connection failed\nstatus:", status, "\nreason:", httpResponse->getReason(), "\nreasonForStatus:", httpResponse->getReasonForStatus(status), "\nresponseContent:", outString.str());
}
I've looked up here how the REST API works. I found here that when using a SAS I don't need to do regular authentication.
What am I doing wrong here? Why am I getting error 404?
I believe most of your code is correct, all you need to do is insert the file name in your SAS URL.
Now that I have seen this question more carefully, this is what is happening:
You're creating a SAS on a blob container (my-blob) and using this SAS to upload a file (let's call it fname.ext). However you're not including the file name in the SAS URL so Azure Storage Service is assuming that you're trying to upload a file called my-blob in a $root container so on the service side when Azure Blob Service tries to validate the SAS, it validates it against $root container. Because you created the SAS for my-blob container and Azure Service is using $root container, the SAS does not match and that's why you're getting 403 error.
What you need to do is insert the file name in your SAS URL. So your SAS URL (or Request URL) would be something like (notice that I added fname.ext there):
https://myname.blob.core.windows.net/my-blob/fname.ext?sv=2012-02-12&se=2016-06-07T11%3A13%3A19Z&sr=c&sp=wl&sig=%%%%%%%%%%%%%%%%%%%%%%%
Also, you don't need the following two lines of code:
request->add("x-ms-version", "2015-02-21");
request->add("x-ms-date", "2016-06-07");
As these are not really needed when using SAS.
I've finally figured out what was going wrong here. :)
There were two problems in the above code. The first is that the filename needed to be inserted into the URL, as Gaurav Mantri explained. This does the trick:
int indexOfQuestionMark = cloudUrl.find('?');
cloudUrl = cloudUrl.substr(0, indexOfQuestionMark) + "/" + fileName + cloudUrl.substr(indexOfQuestionMark);
The other problem is that I wasn't uploading enough bytes. sizeof(int) is 4 bytes while pushing 42 into a stream turns it into characters, making it only 2 bytes. The server then keeps waiting for the remaining 2 bytes. That makes this the correct line in the example code above:
request->setContentLength(2);
Also, it works without these three lines so I suppose they're not needed:
request->add("x-ms-blob-content-disposition", "attachment; filename=\"" + fileName + "\"");
request->add("x-ms-meta-m1", "v1");
request->add("x-ms-meta-m2", "v2");
Similarly, adding this doesn't seem needed: "&restype=container".
Finally, for writing the SharedAccessBlobPermissions.List rights aren't needed so those can be left out in SAS generation on the server side.
One possible reason for your error could be the request date being too old. You're setting the request date as Midnight UTC tonight. Azure Storage allows about 15 minutes of clock skewness. Request date/time being "too old" is one of the major reasons for this 403 error (apart from incorrect account key and expired token in case of a SAS).
This is how you're setting x-ms-date request header.
request->add("x-ms-date", "2016-06-07");
This header's value should be formatted in the following format:
request->add("x-ms-date", "Sun, 11 Oct 2009 21:49:13 GMT");
Usually in C# world, we would do a DateTime.UtcNow.ToString("R") to get the date/time in correct format.
Please change your code accordingly and see if that solves the problem.
I'm trying to download the file by it's unique id (GUID) and in 99% it's working fine, but there some cases where I'm getting http not found - 404. But I know that file is exists and I know it's unique id, from the metadata I got from the document library.
How file can't be found if I have it's metadata in hands ?
I'm doing the next (Java):
//itemID is taken from the library listing (using web service) as uniqueid (guid)
String requestUrl = site + "/_api/web/GetFileById('" + itemID.substring(1, itemID.lastIndexOf('}')) + "')/$value"; //Remove {} from the unique id
HttpGet request = new HttpGet(requestUrl);
// setting auth credentials to the request.
RequestConfig requestConfig = RequestConfig.custom().setConnectTimeout(60000).build();
request.setConfig(requestConfig);
request.addHeader("accept", "application/json; odata=verbose");
HttpResponse result = httpClient.execute(request);
int iRet = result.getStatusLine().getStatusCode(); //Here iRet = 404, sometimes.
Thanks
From previous experience I would look for a lock by concurrent connections. I see you have a 60 s timeout (setConnectTimeout(60000)) in your code. For instance, have you tried if having a single query returns a valid result, and after (say 120s) you try twice within the 60s timeout, it returns the 404?
In elastic search java api, suppose I am building the indexed data using this client
Node node = nodeBuilder().clusterName("es").node();
Client client = node.client();
IndexResponse response = client.prepareIndex("twitter", "tweet", "1")
.setSource(jsonBuilder()
.startObject()
.field("user", "kimchy")
.field("postDate", new Date())
.field("message", "trying out Elasticsearch")
.endObject()
)
.execute()
.actionGet();
This indexed data is stored in folder XYZ/elasticsearch/data.
My question is how to retrieve this indexed data in java from some other client or some other code. Is there any way in which I can give the path and the already indexed data can be imported, then I can perform queries on it?
Edit :
The code for client on other computer
Node node = nodeBuilder().clusterName("es").node();
Client client = node.client();
MatchQueryBuilder qb = QueryBuilders.matchQuery("user", "kimchy");
SearchRequestBuilder srb = client.prepareSearch("twitter").setTypes("tweet");
SearchResponse big = srb.setQuery(qb).execute().actionGet();
SearchHit[] results = big.getHits().getHits();
This shows
search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed
Thanks
Arya
I'm not sure I understand the question, but if it's just querying in java here's an example :
MatchQueryBuilder qb = QueryBuilders.matchQuery("user", "kimchy);
SearchRequestBuilder srb = client.prepareSearch("twitter").setTypes("tweet");
srb.setQuery(qb);
SearchResponse response = srb.execute().actionGet();
//Here goes your code where you use response.getHits() that contains your items
The message you get by using my snippet usually indicates that your client doesn't manage to connect to your server. Check your server status (i.e. that you launched your server properly) and try using this.
Node node = NodeBuilder.nodeBuilder().client(true).node();
client = node.client();
I don't think you need to specify the clustername unless you have some custom configuration i'm not aware of. Also check your index name.
My query url is:
var url = 'https://api.mongolab.com/api/1/databases/database/collections/collection?'
+ 'q={'
+ '\"visible\": true'
+ ', \"date\": ' + JSON.stringify( jsonDate )
+ ', \"country\": \"' + country + "\""
+ '}'
+ '&s={"date": -1}'
+ '&apiKey=' + this.key;
I have and option to sort after with Backbone or Jquery, but I hope to do it with query.
The url you generated is correct, however it is not a valid URL unless you do proper URL encoding of special characters like spaces.
Use encodeURIcomponent of your query parameters, and it should work.
Or use jQuery to pass your parameter as javascript object in jQuery.get method.
Query is working. Problem was in PhoneGap framework, because I've open platform related JS/HTML code and made changes in there. But changes should be made in special www directory, one with source for every platforms.
I would prefer using cursor.sort(sort).
cursor.sort(date)
check this http://docs.mongodb.org/manual/reference/method/cursor.sort/