Neo4j: Change legacy index from exact to fulltext - rest

In my Neo4j (2.1.1 Community Edition) database I have Lucene legacy index in place called node_auto_index:
GET http://localhost:7474/db/data/index/node/
{
"node_auto_index": {
"template": "http://localhost:7474/db/data/index/node/node_auto_index/{key}/{value}",
"provider": "lucene",
"type": "exact"
}
}
Now I would like to change the type from "exact" to "fulltext". How can I do that using REST? I tried the following approaches but neither of them worked:
DELETE and recreate
I tried to delete it first before recreating as "fulltext", but it is read-only:
DELETE http://localhost:7474/db/data/index/node/node_auto_index/node_auto_index
{
"message": "read only index",
"exception": "UnsupportedOperationException",
"fullname": "java.lang.UnsupportedOperationException",
"stacktrace": [
"org.neo4j.kernel.impl.coreapi.AbstractAutoIndexerImpl$ReadOnlyIndexToIndexAdapter.readOnlyIndex(AbstractAutoIndexerImpl.java:254)",
"org.neo4j.kernel.impl.coreapi.AbstractAutoIndexerImpl$ReadOnlyIndexToIndexAdapter.delete(AbstractAutoIndexerImpl.java:290)",
"org.neo4j.server.rest.web.DatabaseActions.removeNodeIndex(DatabaseActions.java:437)",
"org.neo4j.server.rest.web.RestfulGraphDatabase.deleteNodeIndex(RestfulGraphDatabase.java:935)",
"java.lang.reflect.Method.invoke(Unknown Source)",
"org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139)",
"java.lang.Thread.run(Unknown Source)"
]
}
POST to replace
POST http://localhost:7474/db/data/index/node/
{
"name" : "node_auto_index",
"config" : {
"to_lower_case" : "true",
"type" : "fulltext",
"provider" : "lucene"
}
}
 
{
"message": "Supplied index configuration:\n{to_lower_case=true, type=fulltext, provider=lucene}\ndoesn't match stored config in a valid way:\n{provider=lucene, type=exact}\nfor 'node_auto_index'",
"exception": "IllegalArgumentException",
"fullname": "java.lang.IllegalArgumentException",
"stacktrace": [
"org.neo4j.kernel.impl.coreapi.IndexManagerImpl.assertConfigMatches(IndexManagerImpl.java:168)",
"org.neo4j.kernel.impl.coreapi.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java:149)",
"org.neo4j.kernel.impl.coreapi.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerImpl.java:209)",
"org.neo4j.kernel.impl.coreapi.IndexManagerImpl.getOrCreateNodeIndex(IndexManagerImpl.java:314)",
"org.neo4j.kernel.impl.coreapi.IndexManagerImpl.forNodes(IndexManagerImpl.java:302)",
"org.neo4j.server.rest.web.DatabaseActions.createNodeIndex(DatabaseActions.java:398)",
"org.neo4j.server.rest.web.RestfulGraphDatabase.jsonCreateNodeIndex(RestfulGraphDatabase.java:830)",
"java.lang.reflect.Method.invoke(Unknown Source)",
"org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139)",
"java.lang.Thread.run(Unknown Source)"
]
}

for all the future readers of this question.
I faced the similar situation and found a much cleaner approach to fix this. Instead of deleting the node_auto_index, try the following steps.
open the shell / command line for the db:
- neo4j-sh (0)$ index --get-config node_auto_index
- ==> {
- ==> "provider": "lucene",
- ==> "type": "exact"
- ==> }
- neo4j-sh (0)$ index --set-config node_auto_index type fulltext
- ==> INDEX CONFIGURATION CHANGED, INDEX DATA MAY BE INVALID
- neo4j-sh (0)$ index --get-config node_auto_index
- ==> {
- ==> "provider": "lucene",
- ==> "type": "fulltext"
- ==> }
- neo4j-sh (0)$
Worked perfectly fine for me . Hope this helps someone in need :-)

Neo4j does not allow deleting the auto indexes node_auto_index and relationship_auto_index, nor by REST nor by any other API.
However there's a dirty trick to do the job. This trick will delete all auto and other legacy indexes. It does not touch the schema indexes. Be warned, it's a potentially dangerous operation, so make sure you have a valid backup in place. Stop the database and then do a
rm -rf data/graph.db/index*
Restart the database and all auto and legacy indexes are gone.

Related

Opensearch Failed to set number of replicas due no permissions

I have the problem with running index managment policy for new indices. I get following error on "set number_of_replicas" step:
{
"cause": "no permissions for [indices:admin/settings/update] and associated roles [index_management_full_access, own_index, security_rest_api_access]",
"message": "Failed to set number_of_replicas to 2 [index=sample.name-2022.10.22]"
}
The indices are created by logstash with "sample.name-YYYY.MM.DD" name template, so in the index policy I have "sample.name-*" index pattern.
My policy:
{
"policy_id": "sample.name-*",
"description": "sample.name-* policy ",
"schema_version": 16,
"error_notification": null,
"default_state": "set replicas",
"states": [
{
"name": "set replicas",
"actions": [
{
"replica_count": {
"number_of_replicas": 2
}
}
]
],
"ism_template": [
{
"index_patterns": [
"sample.name-*"
],
"priority": 1
}
]
}
I don't understand the reason of this error.
Am I doing something wrong?
Retry of the policy doesn't work.
The policy works only if I manually reassign it to index by Dashboards or API.
Opensearch version: 2.3.0
First time I created the policy using API under custom internal user with mapped “security_rest_api_access” security role only.
So I added all_access rights to my internal user and re-created policy and it works!
Seems that the policy runs under my internal user, which created it

MongoDB Replica Set - The value of parameter linuxConfiguration.ssh.publicKeys.keyData is invalid

This is concerning the Azure Deployment Template for a MongoDB Replica Set defined here mongodb-replica-set-centos.
When I run the recommended deployment commands to deploy the replica set, namely
az group create --name <resource-group-name> --location <resource-group-location> # Use this command when you need to create a new resource group for your deployment.
az deployment group create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/migr8/AzureDeploymentTemplates/main/mongo/mongodb-replica-set-centos/azuredeploy.json
where the resource group is already set up. I receive the following error:
{
"status": "Failed",
"error": {
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "Conflict",
"message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"BadRequest\",\r\n \"message\": \"{\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"InvalidParameter\\\",\\r\\n \\\"message\\\": \\\"The value of parameter linuxConfiguration.ssh.publicKeys.keyData is invalid.\\\",\\r\\n \\\"target\\\": \\\"linuxConfiguration.ssh.publicKeys.keyData\\\"\\r\\n }\\r\\n}\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"
},
{
"code": "Conflict",
"message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"BadRequest\",\r\n \"message\": \"{\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"InvalidParameter\\\",\\r\\n \\\"message\\\": \\\"The value of parameter linuxConfiguration.ssh.publicKeys.keyData is invalid.\\\",\\r\\n \\\"target\\\": \\\"linuxConfiguration.ssh.publicKeys.keyData\\\"\\r\\n }\\r\\n}\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"
}
]
}
}
The problem field is in both primary-resources.json and secondary-resources.json appears to be
"variables": {
"subnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('subnet').vnet, parameters('subnet').name)]",
"securityGroupName": "[concat(parameters('namespace'), parameters('vmbasename'), 'nsg')]",
"linuxConfiguration": {
"disablePasswordAuthentication": true,
"ssh": {
"publicKeys": [
{
"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
"keyData": "[parameters('adminPasswordOrKey')]"
}
]
}
}
},
And ascociated with the variable adminPasswordOrKey. I have tried changing this to be both standard passwords and SSH keys of varying bit-depth, no luck...
How can I fix this?
Repro steps
Run az group create --name <resource-group-name> --location <resource-group-location> where resource group exists.
Run az deployment group create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/migr8/AzureDeploymentTemplates/main/mongo/mongodb-replica-set-centos/azuredeploy.json and step through the prompts
Enter the relevant in formation.
Further Investigation
I have just seen this answer (https://stackoverflow.com/a/60860498/626442) saying specifically that
Note: Please note that the only allowed path is /home//.ssh/authorized_keys due to a limitation of Azure.
I have changed this value of the path, no joy, same error. :'[
You forgot to pass parameters in az deployment group create .... --parameters azuredeploy.parameters.json. You can download azuredeploy.parameters.json and change values as needed. See https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/template-tutorial-use-parameter-file?tabs=azure-cli#deploy-template for details.
Specifically the error in the question complains about adminUsername parameter being empty. Bear in mind this user name is also being used in the home directory path, so limit yourself to lowcase ASCII a-z, numbers, underscore. No spaces, not special characters, no utf.
Not related to the error, but be aware these necromancers use mongo 3.2 which was buried 4 years ago: https://www.mongodb.com/support-policy/lifecycles. Considering they open it wide to the internet you may have way more problems if you actually deploy it.
UPDATE
An example of the parameters I used:
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"adminUsername": {
"value": "yellow"
},
"mongoAdminUsername": {
"value": "phrase"
},
"mongoAdminPassword": {
"value": "settle#SING"
},
"secondaryNodeCount": {
"value": 2
},
"sizeOfDataDiskInGB": {
"value": 2
},
"dnsNamePrefix": {
"value": "written"
},
"centOsVersion": {
"value": "7.7"
},
"primaryNodeVmSize": {
"value": "Standard_D1_v2"
},
"secondaryNodeVmSize": {
"value": "Standard_D1_v2"
},
"zabbixServerIPAddress": {
"value": "Null"
},
"adminPasswordOrKey": {
"value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdNRTU0XF3xazhhDmwXXWGG7wp4AaQC1r89K7sZFRXp9VSUtydV59DHr67mV5/0DWI5Co1yWK713QJ00BPlBIHNMNLuoBBq8IkOx8fBZF1g9YFm5Zy4ay+CF4WgDITAsyxhKvUWL6jwG5M3XIdVYm49K+EFOCWSSaNtCk8tHhi3v6/5HFkwc2r0UL/WWWbbt5AmpJ8QOCDk/x+XcgCjP9vE5jYYGsFz9F6V1FdOpjVfDwi13Ibivj/w2wOZh2lQGskC+qDjd2upK13+RfWYHY3rr+ulNRPckHRhOqmZ2vlUapO4T0X9mM6ugSh1FprLP5nHdVCUls2yw4BAcSoM9NMiyafE56Xkp9h3bTAfx5Ufpe5mjwQp+j15np1pVpwDaEgk7ZeaPoZPhbalpvZGyg9KiKfs9+KUYHfGklIOHKJ3RUoPE286rg1U4LGswil5RARRSf86kBBHXaIPxy1X0N6QryeWhk0aM6LWEdl7mVbQksa7ilANnsaVMl7FSdY/Cc="
}
}
}
DANGER: It will deploy publicly accessible mongodb replica set with publicly accessible credentials, so please delete the resources as soon as you are happy with testing/debugging
This is how deployment looks like on the portal:

TYPO3 CMS 8.7.27: Call to a member function getPackagePath() on null

After installing extensions in typo3 CMS 8.7.27, I got following error.. Seems like the ExtensionManagementUtility can't load the ah_contentapi.
This is my composer.json file in root (/var/www/html/typo3) for loading my extensions:
{
"repositories":[
{
"type":"composer",
"url":"https://composer.typo3.org/"
},
{
"type":"package",
"package":{
"name":"Bm/ah-content-api",
"version":"0.0.1",
"type":"typo3-cms-extension",
"source":{
"url":"https://user#bitbucket.org/company/ah_config_typo3.git",
"type":"git",
"reference":"master"
}
}
},
{
"type":"package",
"package":{
"name":"Bm/ah-contentelements",
"version":"0.0.1",
"type":"typo3-cms-extension",
"source":{
"url":"https://user#bitbucket.org/company/ah_contentelements_typo3.git",
"type":"git",
"reference":"master"
}
}
}
],
"name":"typo3/cms-base-distribution",
"description":"TYPO3 CMS Base Distribution",
"license":"GPL-2.0-or-later",
"require":{
"helhum/typo3-console":"^4.9.3 || ^5.2",
"typo3/cms-about":"^8.7.10",
"typo3/cms-belog":"^8.7.10",
"typo3/cms-beuser":"^8.7.10",
"typo3/cms-context-help":"^8.7.10",
"typo3/cms-documentation":"^8.7.10",
"typo3/cms-felogin":"^8.7.10",
"typo3/cms-fluid-styled-content":"^8.7.10",
"typo3/cms-form":"^8.7.10",
"typo3/cms-func":"^8.7.10",
"typo3/cms-impexp":"^8.7.10",
"typo3/cms-info":"^8.7.10",
"typo3/cms-info-pagetsconfig":"^8.7.10",
"typo3/cms-rte-ckeditor":"^8.7.10",
"typo3/cms-setup":"^8.7.10",
"typo3/cms-sys-note":"^8.7.10",
"typo3/cms-t3editor":"^8.7.10",
"typo3/cms-tstemplate":"^8.7.10",
"typo3/cms-viewpage":"^8.7.10",
"typo3/cms-wizard-crpages":"^8.7.10",
"typo3/cms-wizard-sortpages":"^8.7.10",
"typo3/cms":"^8.7",
"dmitryd/typo3-realurl":"2.*",
"GridElementsTeam/Gridelements":"8.2.*",
"clickstorm/cs_seo":"3.*",
"Bm/ah-content-api":"0.0.1",
"Bm/ah-contentelements":"0.0.1"
},
"scripts":{
"typo3-cms-scripts":[
"typo3cms install:fixfolderstructure",
"typo3cms install:generatepackagestates"
],
"post-autoload-dump":[
"#typo3-cms-scripts"
]
},
"extra":{
"typo3/cms":{
"web-dir":"public"
},
"helhum/typo3-console":{
"comment":"This option is not needed ay more for helhum/typo3-console 5.x",
"install-extension-dummy":false
}
},
"autoload":{
"psr-4":{
"Bm\\AhContentelements\\":"public/typo3conf/ext/ah_contentelements/Classes",
"Bm\\AhContentapi\\":"public/typo3conf/ext/ah_content_api/Classes"
}
}
}
I already cleared cache in install tool at:
1. -> important actions -> clear all cache
2. -> clean up -> Clean typo3temp/ folder
piece from composer.lock:
{
"_readme": [
"This file locks the dependencies of your project to a known state",
"Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
"This file is #generated automatically"
],
"content-hash": "954afd2318d54ec9b1dd0e4d7f9b445b",
"packages": [
{
"name": "Bm/ah-content-api",
"version": "0.0.1",
"source": {
"type": "git",
"url": "https://stevenhippovibe#bitbucket.org/hippovibe/ah_config_typo3.git",
"reference": "master"
},
"type": "typo3-cms-extension"
},
{
"name": "Bm/ah-contentelements",
"version": "0.0.1",
"source": {
"type": "git",
"url": "https://stevenhippovibe#bitbucket.org/stevenhippovibe/ah_contentelements_typo3.git",
"reference": "master"
},
"type": "typo3-cms-extension"
},
The Error occurs when the extension folder name under typo3conf/ext/<folder_name> doesn't match extension key used in some places of the system (e.g. using EXT:your_extension_key/... syntax in TypoScript).
Changing folder name fixed similar problem for me.
Check the PHP version and try to change it from i.e. 7.4 to 7.3.
I once had this problem with an extension that should be compatible with PHP 7.4, but wasn't in real life. This solved the problem for me.
Question here is:
How did you update to 8.7.27 (which composer command was executed)
How does your composer.lock look like?
Do you use TYPO3 console or any other special composer plugins / CLI commands to e.g. generate PackageStates.php?
I just ran into the same error message under TYPO3 9.5.5.
Solution:
Deinstall one TYPO3 extension after the other and try it out again. This will lead you to the extension which has an error. Most probably the error is inside of the file ext_localconf.php or ext_tables.php .
I got this error detail:
PHP Warning: Use of undefined constant FH_DEBUG_EXT - assumed 'FH_DEBUG_EXT' (this will throw an Error in a future version of PHP) in /var/www/html/global-extensions/ext/div2007/ext_localconf.php line 15
This has nothing to do with your error. But it can be that you have an error in one of your installed extensions or even in a backup of an extension, e.g. a folder named as extensionname.bak .
Also these recommendations can help:
https://wiki.typo3.org/Exception/CMS/1476107295

StreamEnabled table property causes Serverless failure

I'm using Serverless to deploy my AWS cloudformation stack. On one of my tables, I enable streams via "StreamEnabled": true. When this is enabled, I get an error on deployment: Encountered unsupported property StreamEnabled.
If I remove the property, I get a validation exception: ValidationException: Stream StreamEnabled was null.
I found a git issue that was addressed and apparently fixed (here), but after upgrading to v1.3, I'm still getting the same errors on deployment.
Can anyone lend insight as to what the issue may be?
It is enabled by default. You can check it from shell:
aws dynamodbstreams list-streams
{
"Streams": [
{
"TableName": "MyTableName-dev",
"StreamArn": "arn:aws:dynamodb:eu-west-2:0000000000000:table/MyTableName-dev/stream/2018-10-26T15:06:25.995",
"StreamLabel": "2018-10-26T15:06:25.995"
}
]
}
And:
aws dynamodbstreams describe-stream --stream-arn "arn:aws:dynamodb:eu-west-2:00000000000:table/MyTableName-dev/stream/2018-10-26T15:06:25.995"
{
"StreamDescription": {
"StreamLabel": "2018-10-26T15:06:25.995",
"StreamStatus": "ENABLED",
"TableName": "MyTableName-dev",
"Shards": [
{
"ShardId": "shardId-000000000000000-0000000f",
"SequenceNumberRange": {
"StartingSequenceNumber": "00000000000000000000000"
}
}
],
"CreationRequestDateTime": 1540566385.987,
"StreamArn": "arn:aws:dynamodb:eu-west-2:0000000000000000:table/MyTableName-dev/stream/2018-10-26T15:06:25.995",
"KeySchema": [
{
"KeyType": "HASH",
"AttributeName": "application_id"
}
],
"StreamViewType": "KEYS_ONLY"
}
}
It is not a solution, but found that fact I realized that I don't have an issue.

MongoError additional information

I'm interested by Mongo Errors, the only useful link I found is not that useful actually.
http://www.mongodb.org/about/contributors/error-codes/
"name": "MongoError",
"err": "E11000 duplicate key error index: AyolanDB.users.$email_1 dup key: { : \"copain6#gmail.com\" }",
"code": 11000,
"n": 0,
"connectionId": 838,
"ok": 1
For example, what does mean the "ok"? That the DB is still running? What's "n"?
If anyone has more information about that... Because it's quite poor actually!
ok - means a command successfully completed
n - number of document affected (insert | update | remove)
For more info, see getLastError