I'm having a bit of an issue with a shell script, replacing url in a json file with sed, with an url stored on a variable. The code is this:
sed -i "30s+.*+\"uri\":\"$MONGO_URI\",+" temp.json
To replace the MONGO_URI in the json:
{
...
"requestsDB": {
"uri": "MONGO_URI",
"password":"MONGO_PW",
"database": "temp-db"},
"mlAPI": {
"server": "..."}
...
}
#into
{
...
"requestsDB": {
"uri": "mongodb://admin:password#*****725/compose?authSource=admin?ssl=true?retryWrites=false",
"password":"MONGO_PW",
"database": "temp-db"},
"mlAPI": {
"server": "...}
...
}
But instead I'm getting this:
"uri" :"mongodb://admin:password#*****725/compose?authSource=admin "uri": "MONGO_URI",ssl=true "uri": "MONGO_URI",retryWrites=false",
Don't really know what's causing the issue, don't know if anybody can help?
As noted in the comments, using jq docs would be a much better solution to alter JSON
An example using JQ where the temp.json looks like this
{
"requestsDB":
{
"uri": "MONGO_URI",
"password": "MONGO_PW",
"database": "temp-db"
},
"mlAPI":
{
"server": "..."
}
}
We can use the following bash calls to alter the uri:
# Hardcoded URL
jq --arg MOGO_URI "Example URL" '.requestsDB.uri |= $MOGO_URI' temp.json
# If your new url is available through a bash var, you can use:
MOGO_URI='Example URL'
jq --arg MOGO_URI "$MOGO_URI" '.requestsDB.uri |= $MOGO_URI' temp.json
This will produce:
{
"requestsDB": {
"uri": "Example URL",
"password": "MONGO_PW",
"database": "temp-db"
},
"mlAPI": {
"server": "..."
}
}
JqPlay Demo
Related
I have very basic mapping.json
{
"mappings": [
{
"priority": 1,
"request": {
"method": "GET",
"url": "/your/url?and=query"
},
"response": {
"status": 200,
"statusMessage": "query param test"
}
},
{
"priority": 2,
"request": {
"method": "GET",
"url": "/your"
},
"response": {
"status": 200,
"statusMessage": "no query param"
}
}
]
}
It's the exact same example as given in the documentation.
Result:
admin ~ % curl -i http://localhost:8081/your
HTTP/1.1 200 no query param
Matched-Stub-Id: 6ff84303-8abb-48d0-bd27-679de118afc7
Transfer-Encoding: chunked
Server: Jetty(9.2.z-SNAPSHOT)
admin ~ % curl -i http://localhost:8081/your/url?and=query
zsh: no matches found: http://localhost:8081/your/url?and=query
admin ~ %
Cannot figure out what I am doing wrong here. It's exactly the same example give in the documentation. I tried putting query parameter like this:
"queryParameters" : {
"search_term" : {
"equalTo" : "WireMock"
}
},
This also didn't help.
TIA
Check out the answer and comment from this question, but the tl;dr is that if you want to include query parameters in a CURL request, you have to have the URL in quotes.
That would explain why Postman worked, and the CURL request without query parameters also worked, but the CURL request with query parameters did not.
curl -i 'http://localhost:8081/your/url?and=query' should be enough to solve your problem (might need double quotes instead of single?)
Hello I am looking to write a script which uses firebase firestore and writes some json to a specific collection in firestore. I have done this with the realtime db but firestore is a tad different below is my Realtime db snippet that works.
curl -X POST \
-d '{"param1":"'"$1"'", "param2":"'"$2"'"}' \
https://xxxx.firebaseio.com/xxxx.json?
Thanks for the help
After Reading the documentation I got to this
curl -X POST \
-H "Content-Type: application/json" \
-d'{
"fields": {
"Field1": {
"stringValue": "'"$var1"'"
},
"Field2": {
"stringValue": "'"$var2"'"
},
"Field3": {
"stringValue": "$var3"
}
}
}'\"https://firestore.googleapis.com/v1beta1/projects/**PROJECT_ID**/databases/(default)/documents/**COLLECTION_ID**?&key=(YOUR API KEY)"
The accepted answer helped me, but it took me a long time to figure out how can I use data types other than stringValues, so I am adding this answer hoping someone finds this helpful in the future.
curl -X POST \
-H "Content-Type: application/json" \
-d' {
"fields": {
"Field1": {
"arrayValue": {
"values": [{
"mapValue": {
"fields": {
"key1": {
"stringValue": "val1"
},
"key2": {
"stringValue": "val2"
}
}
}
}]
}
},
"Field2": {
"integerValue": <intValue>
},
"Field3": {
"stringValue": "var3"
}
}
}'\"https://firestore.googleapis.com/v1beta1/projects/**PROJECT_ID**/databases/(default)/documents/**COLLECTION_ID**?&key=<YOUR WEB API KEY>"
Use this for reference.
example field value
var data = { "fields": { "productName":{"stringValue": dealname.toString()}, "companyname":{"stringValue": companyName.toString()}, "contact":{"stringValue": contact.toString()}, "email":{"stringValue": email.toString()}, "domain":{"stringValue": domain.toString()}, "createdate":{"stringValue": createdate.toString()}, "salesCode":{"stringValue": code.toString()}, "price":{"stringValue": amount.toString()}, "phone":{"stringValue": phone.toString()}, "orderId":{"stringValue": orderId.toString() } } };
more information firestore information
Attempting to make a plain GET request to one of the BigQuery REST APIs gives an error that looks like this:
curl https://www.googleapis.com/bigquery/v2/projects/$PROJECT_ID/jobs/$JOBID
Output:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "required",
"message": "Login Required",
"locationType": "header",
"location": "Authorization",
...
What is the correct way to invoke one of the REST APIs from the command-line, such as the query or insert APIs? The API reference has a "Try this API", but the examples don't translate directly to something you can run from the command-line.
As a disclaimer, when working from the command-line, using the bq tool will usually be sufficient, or for more complex use cases, the BigQuery client libraries enable programming with BigQuery from multiple languages. It can still be useful sometimes to make plain requests to the REST APIs to see how certain APIs work at a low level, however.
First, make sure that you have installed the Google Cloud SDK. This should include the gcloud and bq command-line tools. If you haven't already, authorize your account by running this command from your terminal:
gcloud auth login
This should prompt you to log in and then give you an access code that you can paste into your terminal. (The exact process may change over time).
Now let's try a query using the BigQuery REST API, calling the jobs.query method. Modify this script with your own project name, which you can find from the Google Cloud Console, then paste the script into your terminal:
PROJECT="YOUR_PROJECT_NAME"
QUERY="\"SELECT 1 AS x, 'foo' AS y;\""
REQUEST="{\"kind\":\"bigquery#queryRequest\",\"useLegacySql\":false,\"query\":$QUERY}"
echo $REQUEST | \
curl -X POST -d #- -H "Content-Type: application/json" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
https://www.googleapis.com/bigquery/v2/projects/$PROJECT/queries
If it worked, you should see output that looks like this:
{
"kind": "bigquery#queryResponse",
"schema": {
"fields": [
{
"name": "x",
"type": "INTEGER",
"mode": "NULLABLE"
},
{
"name": "y",
"type": "STRING",
"mode": "NULLABLE"
}
]
},
"jobReference": {
"projectId": "<your project ID>",
"jobId": "<your job ID>"
},
"totalRows": "1",
"rows": [
{
"f": [
{
"v": "1"
},
{
"v": "foo"
}
]
}
],
"totalBytesProcessed": "0",
"jobComplete": true,
"cacheHit": false
}
If you haven't set up the bq command-line tool, you can use bq init from your terminal to do so. Once you have, you can try running the same query using it:
bq query --use_legacy_sql=False "SELECT 1 AS x, 'foo' AS y;"
You can also see the REST API requests that the bq tool makes by passing the --apilog= option:
bq --apilog= query --use_legacy_sql=False "SELECT [1, 2, 3] AS x;"
Now let's try an example using the jobs.insert method instead of the query API. Run this script, replacing YOUR_PROJECT_NAME with your project name:
PROJECT="YOUR_PROJECT_NAME"
QUERY="\"SELECT 1 AS x, 'foo' AS y;\""
REQUEST="{\"configuration\":{\"query\":{\"useLegacySql\":false,\"query\":${QUERY}}}}"
echo $REQUEST | \
curl -X POST -d #- -H "Content-Type: application/json" \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
https://www.googleapis.com/bigquery/v2/projects/$PROJECT/jobs
Unlike the query API, which returned a response immediately, you will see a result that looks similar to this:
{
"kind": "bigquery#job",
"etag": "\"<etag string>\"",
"id": "<project name>:<job ID>",
"selfLink": "https://www.googleapis.com/bigquery/v2/projects/<project name>/jobs/<job ID>",
"jobReference": {
"projectId": "<project name>",
"jobId": "<job ID>"
},
"configuration": {
"query": {
"query": "SELECT 1 AS x, 'foo' AS y;",
"destinationTable": {
"projectId": "<project name>",
"datasetId": "<anonymous dataset>",
"tableId": "<anonymous table>"
},
"createDisposition": "CREATE_IF_NEEDED",
"writeDisposition": "WRITE_TRUNCATE",
"useLegacySql": false
}
},
"status": {
"state": "RUNNING"
},
"statistics": {
"creationTime": "<timestamp millis>",
"startTime": "<timestamp millis>"
},
"user_email": "<your email address>"
}
Notice the status:
"status": {
"state": "RUNNING"
},
If you want to check on the job now, you can use the jobs.get method. Similar to before, run this from your terminal, using the job ID from the output in the previous step:
PROJECT="YOUR_PROJECT_NAME"
JOB_ID="YOUR_JOB_ID"
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \
https://www.googleapis.com/bigquery/v2/projects/$PROJECT/jobs/$JOB_ID
If the query is done, you'll get a response that indicates as much:
...
"status": {
"state": "DONE"
},
...
Finally, we can make a request to fetch the query results, also using the REST API.
curl -H "Authorization: Bearer $(gcloud auth print-access-token)" \
https://www.googleapis.com/bigquery/v2/projects/$PROJECT/queries/$JOB_ID
The output will look similar to when we used the jobs.query method above:
{
"kind": "bigquery#getQueryResultsResponse",
"etag": "\"<etag string>\"",
"schema": {
"fields": [
{
"name": "x",
"type": "INTEGER",
"mode": "NULLABLE"
},
{
"name": "y",
"type": "STRING",
"mode": "NULLABLE"
}
]
},
"jobReference": {
"projectId": "<project ID>",
"jobId": "<job ID>"
},
"totalRows": "1",
"rows": [
{
"f": [
{
"v": "1"
},
{
"v": "foo"
}
]
}
],
"totalBytesProcessed": "0",
"jobComplete": true,
"cacheHit": true
}
I am newbie with mongodb and loopback. I want to send and save data from my app to database. how can I do that?
Update
shops model:
{
"shopname": "string",
"tel": "string",
"latlng": "string",
"address": "string",
"id": "string",
"personId": "string"
}
CURL:
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{ \
"shopname": "spring", \
"tel": "12345678", \
"latlng": "52.1106986,21.7768998", \
"address": "05-319 Skwarne, Poland" \
}' 'http://localhost:3000/api/shops'
Now what should I write in shops.js to give an api for using in app to send data to database ?
'use strict';
module.exports = function(Shops) {
};
you should have provided more info about the steps you have already done.
let me begin with the first step:
download and install mongodb on your server: link
after running mongodb, add your desired database info to datasources.json file. e.g.
{
"db": {
"name": "db",
"connector": "memory"
},
"shopDB": {
"host": "localhost",
"port": 27017,
"url": "mongodb://localhost:27017/shopDB",
"database": "shopDB",
"password": "",
"name": "shopDB",
"user": "",
"connector": "mongodb"
}
}
add loopback-connector-mongodb to your project via npm.
now define your model(you can utilize loopback's user friendly command line interface to do so. call command "slc loopback:model" in your projects root folder)
after you finish the step 4, loopback will create 2files for you: shop.js and shop.json => these files are located in your projectFolder/common/models/ directory. note that it's a good practice to follow the loopback's convention in naming models and name your model in singular form(shop). (it uses the plural forms of the model names in other parts of the project). your shop.json should look like the below code:
{
"name": "shop",
"plural": "shops",
"base": "PersistedModel",
"idInjection": true,
"options": {
"validateUpsert": true
},
"properties": {
"shopname": {
"type": "string",
"required": true
},
"tel": {
"type": "string",
"required": true
},
"latlng": {
"type": "string",
"required": true
},
"address": {
"type": "string"
},
"personId": {
"type": "string",
"required": true
}
},
"validations": [],
"relations": {},
"acls": [],
"methods": {}
}
now you can post your shop json to http://localhost:3000/api/shops/ . note that our shop model inherits from PersistedModel base model and has some built-in functions to do crud operations. and if you want just create some shop instances in your db, you won't need to add anything to your shop.js file!
In one of my project, I am trying to use Elasticsearch (1.7) to query data. But, It returns different result for unicode characters depending on if they are uppercased or not. I try to use icu_analyzer to get rid of problem.
Here is a small example to demonstrate my problem. My index is like this,
$ curl -X PUT http://localhost:9200/tr-test -d '
{
"mappings": {
"names": {
"properties": {
"name": {
"type": "string"
}
}
}
},
"settings": {
"index": {
"number_of_shards": "5",
"number_of_replicas": "1",
"analysis": {
"filter": {
"nfkc_normalizer": {
"type": "icu_normalizer",
"name": "nfkc"
}
},
"analyzer": {
"my_lowercaser": {
"tokenizer": "icu_tokenizer",
"filter": [
"nfkc_normalizer"
]
}
}
}
}
}
}'
Here is a test data to demonstrate my problem.
$ curl -X POST http://10.22.20.140:9200/tr-test/_bulk -d '
{"index": {"_type":"names", "_index":"tr-test"}}
{"name":"BAHADIR"}'
Here is a similar query. If I query using BAHADIR as query_string, I can easily find my test data.
$ curl -X POST http://10.22.20.140:9200/tr-test/_search -d '
{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "BAHADIR"
}
}
}
}
}'
In Turkish, lowercased version of of BAHADIR is bahadır. I am expecting same result while querying with bahadır. But Elasticsearch cannot find my data. And I cannot fix that with using ICU for analysis. It works perfectly fine if I query with bahadir.
I already read Living in a Unicode World and Unicode Case Folding. But cannot fix my problem. I still cannot make elasticsearch to use correct case folding.
Update
I also try to create my Index like this.
$ curl -X PUT http://localhost:9200/tr-test -d '
{
"mappings": {
"names": {
"properties": {
"name": {
"type": "string",
"analyzer" : "turkish"
}
}
}
},
"settings": {
"index": {
"number_of_shards": "5",
"number_of_replicas": "1"
}
}
}'
But I am getting same results. My data can be found if I search using BAHADIR or bahadir but it cannot be found by searching bahadır which is correct lowercased version of BAHADIR.
You should try to use the Turkish Language Analyzer in your setting.
{
"mappings": {
"names": {
"properties": {
"name": {
"type": "string",
"analyzer": "turkish"
}
}
}
}
}
As you can see in the implementation details, it also defines a turkish_lowercase so I guess it'll take care of your problems for you. If you don't want all the other features of the Turkish Analyzer, define a custom one with only turkish_lowercase
If you need a full text search on your name field, you should also change the query method to match query, which is the basic full text search method on a single field.
{
"query": {
"match": {
"name": "bahadır"
}
}
}
On the other hand, query string query is more complex and searches on multiple fields allowing an advanced syntax; It also has an option to pass the analyzer you want to use, so if you really needed this kind of query you should have tried passing "analyzer": "turkish" within the query. I'm not an expert of query string query though.