I have configured the server instance in code
"instance": [
{ "address": 127.0.0.1,
"port": 6514
},
{"address": 127.0.0.1,
"port": 6515
}
]
I'm also having file(/root/home/inst.txt) which has a format
{ "address": 127.0.0.1,
"port": 6514
},
{"address": 127.0.0.1,
"port": 6515
},
{"address": 127.0.0.1,
"port": 6516
}
I want to replace server instances in code from file.
I had tried
sed -e '/"instances": \[/,/\]/{/r /root/home/inst.txt/!d}'
but failed
This might work for you (GNU sed):
sed -ne '/instances/,/\]/!{p;b};/instances/{p;r addrFile' -e '};/\]/p' mainFile
Turn off implicit printing -n.
If the current line is not between instances and ], print the current line.
If the current line is between instances and ] and matches instances, print the current line and the addFile.
If the current line is between instances and ] and matches ], print the current line.
Alternatives:
sed -e '/instances/!b;p;r addrFile' -e ':a;N;/\]/!ba;s/.*\n//' mainFile
Or:
sed -e '/instances/{p;r addrFile' -e ':a;N;/\],/!s/\n//;ta;D}' mainFile
Or:
sed -ne '/instances/{p;r addrFile' -e ':a;/\],/!{n;ba}};p' mainFile
Related
I'm triyng run the crawler with docker image, but it's returning this error.
PS C:\Users\Santosgab\Desktop\documentation> docker run -it --env-file=.env -e "CONFIG=$(cat config.json | jq -r tostring)" algolia/docsearch-scraper
Traceback (most recent call last):
File "/root/src/config/config_loader.py", line 101, in _load_config
data = json.loads(config, object_pairs_hook=OrderedDict)
File "/usr/lib/python3.6/json/__init__.py", line 367, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.6/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.6/json/decoder.py", line 355, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
i used the config provided by Algolia, just made some small changes to run in my project, my config.json has no problem.
{
"index_name": "Sequor",
"sitemap_urls": ["http://localhost:3000/sitemap.xml"],
"sitemap_alternate_links": true,
"stop_urls": ["/tests"],
"selectors": {
"lvl0": {
"selector": "(//ul[contains(#class,'menu__list')]//a[contains(#class, 'menu__link menu__link--sublist menu__link--active')]/text() | //nav[contains(#class, 'navbar')]//a[contains(#class, 'navbar__link--active')]/text())[last()]",
"type": "xpath",
"global": true,
"default_value": "Documentation"
},
"lvl1": "header h1",
"lvl2": "article h2",
"lvl3": "article h3",
"lvl4": "article h4",
"lvl5": "article h5, article td:first-child",
"lvl6": "article h6",
"text": "article p, article li, article td:last-child"
},
"strip_chars": " .,;:#",
"custom_settings": {
"separatorsToIndex": "_",
"attributesForFaceting": ["language", "version", "type", "docusaurus_tag"],
"attributesToRetrieve": [
"hierarchy",
"content",
"anchor",
"url",
"url_without_anchor",
"type"
]
},
"conversation_id": ["833762294"],
"nb_hits": 46250
}
Two things I notice:
There's no path on your config file. Can you try it with pathing (i.e. ./config.json)
Your config is missing any start_urls as entry points to crawling.
I have a Ansible group_vars directory with the following file within it:
$ cat inventory/group_vars/env1
...
...
ldap_config: !vault |
$ANSIBLE_VAULT;1.1;AES256
31636161623166323039356163363432336566356165633232643932623133643764343134613064
6563346430393264643432636434356334313065653537300a353431376264333463333238383833
31633664303532356635303336383361386165613431346565373239643431303235323132633331
3561343765383538340a373436653232326632316133623935333739323165303532353830386532
39616232633436333238396139323631633966333635393431373565643339313031393031313836
61306163333539616264353163353535366537356662333833653634393963663838303230386362
31396431636630393439306663313762313531633130326633383164393938363165333866626438
...
...
This Ansible encrypted string has a Kubernetes secret encapsulated within it. A base64 blob that looks something like this:
IyMKIyBIb3N0IERhdGFiYXNlCiMKIyBsb2NhbGhvc3QgaXMgdXNlZCB0byBjb25maWd1cmUgdGhlIGxvb3BiYWNrIGludGVyZmFjZQojIHdoZW4gdGhlIHN5c3RlbSBpcyBib290aW5nLiAgRG8gbm90IGNoYW5nZSB0aGlzIGVudHJ5LgojIwoxMjcuMC4wLjEJbG9jYWxob3N0CjI1NS4yNTUuMjU1LjI1NQlicm9hZGNhc3Rob3N0Cjo6MSAgICAgICAgICAgICBsb2NhbGhvc3QKIyBBZGRlZCBieSBEb2NrZXIgRGVza3RvcAojIFRvIGFsbG93IHRoZSBzYW1lIGt1YmUgY29udGV4dCB0byB3b3JrIG9uIHRoZSBob3N0IGFuZCB0aGUgY29udGFpbmVyOgoxMjcuMC4wLjEga3ViZXJuZXRlcy5kb2NrZXIuaW50ZXJuYWwKIyBFbmQgb2Ygc2VjdGlvbgo=
How can I decrypt this in a single CLI?
We can use an Ansible adhoc command to retrieve the variable of interest, ldap_config. To start we're going to use this adhoc to retrieve the Ansible encrypted vault string:
$ ansible -i "localhost," all \
-m debug \
-a 'msg="{{ ldap_config }}"' \
--vault-password-file=~/.vault_pass.txt \
-e#inventory/group_vars/env1
localhost | SUCCESS => {
"msg": "ABCD......."
Make note that we're:
using the debug module and having it print the variable, msg={{ ldap_config }}
giving ansible the path to the secret to decrypt encrypted strings
using the notation -e#< ...path to file...> to pass the file with the encrypted vault variables
Now we can use Jinja2 filters to do the rest of the parsing:
$ ansible -i "localhost," all \
-m debug \
-a 'msg="{{ ldap_config | b64decode | from_yaml }}"' \
--vault-password-file=~/.vault_pass.txt \
-e#inventory/group_vars/env1
localhost | SUCCESS => {
"msg": {
"apiVersion": "v1",
"bindDN": "uid=readonly,cn=users,cn=accounts,dc=mydom,dc=com",
"bindPassword": "my secret password to ldap",
"ca": "",
"insecure": true,
"kind": "LDAPSyncConfig",
"rfc2307": {
"groupMembershipAttributes": [
"member"
],
"groupNameAttributes": [
"cn"
],
"groupUIDAttribute": "dn",
"groupsQuery": {
"baseDN": "cn=groups,cn=accounts,dc=mydom,dc=com",
"derefAliases": "never",
"filter": "(objectclass=groupOfNames)",
"scope": "sub"
},
"tolerateMemberNotFoundErrors": false,
"tolerateMemberOutOfScopeErrors": false,
"userNameAttributes": [
"uid"
],
"userUIDAttribute": "dn",
"usersQuery": {
"baseDN": "cn=users,cn=accounts,dc=mydom,dc=com",
"derefAliases": "never",
"scope": "sub"
}
},
"url": "ldap://192.168.1.10:389"
}
}
NOTE: The above section -a 'msg="{{ ldap_config | b64decode | from_yaml }}" is what's doing the heavy lifting in terms of converting from Base64 to YAML.
References
How to run Ansible without hosts file
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#filters-for-formatting-data
Base64 Decode String in jinja
How to decrypt string with ansible-vault 2.3.0
If you need a one liner that works with any yaml file (not only in inventory) containing inlined vault vars, and if you are ready to install a pip package for that, there is a solution using yq, a yaml processor built on top of jq
prerequesite: Install yq
pip install yq
Usage
You can get your result with the following command:
yq -r .ldapconfig inventory/group_vars/env1 | ansible_vault decrypt
If you need to type your vault pass interactively, don't forget to add the relevant option
yq -r .ldapconfig inventory/group_vars/env1 | ansible_vault --ask-vault-pass decrypt
Note: the -r option to yq is mandatory to get a raw result without the quotation marks around the value.
It seems when using PHP variables like $_SERVER the code snippet ignores the $. For example
{
// Example:
"IP Address Test": {
"scope": "php",
"prefix": "iptest",
"body": [
"// Debugging",
"if($_SERVER[\"REMOTE_ADDR\"]=='${1:ipaddress}'){",
"\t//run only my ip",
"\t$0",
"}"
],
"description": "Test only from IP address"
}
}
outputs :
// Debugging
if(_SERVER["REMOTE_ADDR"]=='xxx.xxx.xxx.xxx'){
//run only my ip
}
You can't use \ you have to use double $ ..
eg.
// Debugging
if($$_SERVER["REMOTE_ADDR"]=='xxx.xxx.xxx.xxx'){
//run only my ip
}
I have used this code it only enter with new ipaddress
sed -r 's/(\b[0-9]{1,3}.){3}[0-9]{1,3}\b'/$newip/
But i need to enter with system ipaddress to the file directly,can anyone help it out with this
Thanks in advance
The command you have already does this. Just need to add the -i option to the sed:
$ cat file
{ "MONGODB_CONNECTION_URL" : "mongodb://127.0.0.1/genieacs",
"CWMP_INTERFACE" : "0.0.0.0",
"CWMP_PORT" : 7547,
"CWMP_SSL" : false,
"NBI_INTERFACE" : "0.0.0.0",
"NBI_PORT" : 7557,
"FS_INTERFACE" : "0.0.0.0",
"FS_PORT" : 7567,
"FS_HOSTNAME" : "acs.example.com",
"DEBUG" : false
}
Set the IP address in a variable:
$ newip="192.168.0.20"
Run the sed command:
$ sed -i -r 's/(\b[0-9]{1,3}.){3}[0-9]{1,3}\b'/$newip/ file
File has updated IP:
$ cat file
{ "MONGODB_CONNECTION_URL" : "mongodb://255.255.25.0/genieacs",
"CWMP_INTERFACE" : "255.255.25.0",
"CWMP_PORT" : 7547,
"CWMP_SSL" : false,
"NBI_INTERFACE" : "255.255.25.0",
"NBI_PORT" : 7557,
"FS_INTERFACE" : "255.255.25.0",
"FS_PORT" : 7567,
"FS_HOSTNAME" : "acs.example.com",
"DEBUG" : false
}
sed -E 's/([^/])\b([0-9]{1,3}\.){3}[0-9]{1,3}\b([^/])'/\1192.168.0.28\2/g config.json
if as variable e.g. obtained from ifconfig etc, after testing do real editing, add -i option
$ newip=192.168.0.28
$ sed -Ei 's/([^/])\b([0-9]{1,3}\.){3}[0-9]{1,3}\b([^/])'/\1$newip\2/g config.json
[
{
"Node": "consul-staging-a-1.org",
"Address": "10.0.11.221",
"ServiceID": "mesos-slave-staging-a-1.org:determined_bartik:5000",
"ServiceName": "service1",
"ServiceTags": null,
"ServiceAddress": "",
"ServicePort": 4003
},
{
"Node": "consul-staging-a-1.org",
"Address": "10.0.11.221",
"ServiceID": "mesos-slave-staging-a-1.org:angry_hypatia:5000",
"ServiceName": "service1",
"ServiceTags": null,
"ServiceAddress": "",
"ServicePort": 4007
}
]
This is what I get from querying the Consul service API (/v1/catalog/service/service1).
Commands that I used to start registrator and services:
docker run -d -v /var/run/docker.sock:/tmp/docker.sock -h $HOSTNAME gliderlabs/registrator consul://consul-staging-a-1.org:8500
docker run -d -p 4003:5000 -e "SERVICE_NAME=service1" docker-training/hello-world
docker run -d -p 4007:5000 -e "SERVICE_NAME=service1" docker-training/hello-world
Any steps I'm doing wrong? How do you assign the hostname to ServiceAddress field?
I had similar issue and found that before v6 Consul Registrator leaves ServiceAddress empty, but in v6 (at the moment the latest) it is "0.0.0.0". I've tried to use "-ip" option but it did not help, somehow it assigns internal IP address of the container. Found related issues at:
https://github.com/gliderlabs/registrator/issues/240
In my case I fixed it by binding container to IP address like:
docker run -d -p 10.0.0.3:4003:5000 -e "SERVICE_NAME=service1" docker-training/hello-world