Spring Boot Rest search 404 - spring-data

I noticed periodically that /search doesn't seem to work on my #RepositoryRestResource:
Spring Boot Output
2014-08-25 13:37:51.526 INFO 10645 --- [ main] o.s.d.r.w.RepositoryRestHandlerMapping : Mapped "{[/{repository}/search],methods=[HEAD],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public org.springframework.http.HttpEntity<?> org.springframework.data.rest.webmvc.RepositorySearchController.headForSearches(org.springframework.data.rest.webmvc.RootResourceInformation)
2014-08-25 13:37:51.526 INFO 10645 --- [ main] o.s.d.r.w.RepositoryRestHandlerMapping : Mapped "{[/{repository}/search],methods=[GET],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public org.springframework.hateoas.ResourceSupport org.springframework.data.rest.webmvc.RepositorySearchController.listSearches(org.springframework.data.rest.webmvc.RootResourceInformation)
2014-08-25 13:37:51.526 INFO 10645 --- [ main] o.s.d.r.w.RepositoryRestHandlerMapping : Mapped "{[/{repository}/search/{search}],methods=[GET],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public org.springframework.http.ResponseEntity<java.lang.Object> org.springframework.data.rest.webmvc.RepositorySearchController.executeSearch(org.springframework.data.rest.webmvc.RootResourceInformation,org.springframework.web.context.request.WebRequest,java.lang.String,org.springframework.data.domain.Pageable,org.springframework.data.rest.webmvc.PersistentEntityResourceAssembler)
2014-08-25 13:37:51.526 INFO 10645 --- [ main] o.s.d.r.w.RepositoryRestHandlerMapping : Mapped "{[/{repository}/search/{search}],methods=[GET],params=[],headers=[],consumes=[],produces=[application/x-spring-data-compact+json],custom=[]}" onto public org.springframework.hateoas.ResourceSupport org.springframework.data.rest.webmvc.RepositorySearchController.executeSearchCompact(org.springframework.data.rest.webmvc.RootResourceInformation,org.springframework.web.context.request.WebRequest,java.lang.String,java.lang.String,org.springframework.data.domain.Pageable,org.springframework.data.rest.webmvc.PersistentEntityResourceAssembler)
2014-08-25 13:37:51.526 INFO 10645 --- [ main] o.s.d.r.w.RepositoryRestHandlerMapping : Mapped "{[/{repository}/search/{search}],methods=[HEAD],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public org.springframework.http.ResponseEntity<java.lang.Object> org.springframework.data.rest.webmvc.RepositorySearchController.headForSearch(org.springframework.data.rest.webmvc.RootResourceInformation,java.lang.String)
curl on /persons
curl http://localhost:8080/persons
{
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons{?page,size,sort}",
"templated" : true
}
},
"_embedded" : {
"persons" : [ {
"firstName" : "Jimmy",
"lastName" : "Neutron",
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons/2"
}
}
}, {
"firstName" : "Jimmy",
"lastName" : "Page",
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons/3"
}
}
}, {
"firstName" : "Jimmy",
"lastName" : "Johns",
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons/4"
}
}
} ]
},
"page" : {
"size" : 20,
"totalElements" : 3,
"totalPages" : 1,
"number" : 0
}
cURL on /search
curl -v http://localhost:8080/persons/search
* Adding handle: conn: 0x7f9903004400
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f9903004400) send_pipe: 1, recv_pipe: 0
* About to connect() to localhost port 8080 (#0)
* Trying ::1...
* Connected to localhost (::1) port 8080 (#0)
> GET /persons/search HTTP/1.1
> User-Agent: curl/7.30.0
> Host: localhost:8080
> Accept: */*
>
< HTTP/1.1 404 Not Found
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< Content-Length: 0
< Date: Mon, 25 Aug 2014 18:55:27 GMT
<
* Connection #0 to host localhost left intact
Usually I can restart the Application and it randomly appears with 0 changes:
curl http://localhost:8080/persons
{
"_links" : {
"self" : {
"href" : "http://localhost:8080/persons{?page,size,sort}",
"templated" : true
},
"search" : {
"href" : "http://localhost:8080/persons/search"
}
},
"page" : {
"size" : 20,
"totalElements" : 0,
"totalPages" : 0,
"number" : 0
}
Is there something needed to enable search functionality 100% of the time?

I had the same problem and found out that causes it.
Spring loads only one repository for your domain type to find search resource mappings. If you have multiple repositories for your domain type, spring somehow randomly selects one and uses that one. If that one does not contain your custom search method, it does not load.

I don't quite buy the randomness argument, but if you can really reproduce it, feel free to open a ticket in our tracker.
The general rule is that the search resource is only exposed if there's at least a single query method exposed (i.e. a query method that's not actively configured to not be exported) on the repository in question.
Generally speaking, you shouldn't really let clients construct URIs on the fly but rather inspect the response for links and follow them. Thus, I'd argue that if you get the first response, performing a call to /search is invalid as the client has no reason to assume the resource to even exist. If you start with the second response, you can discover a search link advertising a search resource being available and follow it.

Getting random "Not Found" error can happen in case you have multiple repositories defined for the same domain object. It is a known Spring Data Rest bug which happens due to undeterministic order in which those repositories are discovered/registered.
Please vote at https://jira.spring.io/browse/DATAREST-923 to get it fixed.

Related

Service unavailable error while using MongoDB, ElasticSearch and transporter

I am trying to use the transporter plugin to create a pipeline to sync a MongoDB database and ElasticSearch. I am using a Linux virtual machine (ubuntu) for this.
I have created a MongoDB collection my_application with the following data in it:
db.users.find().pretty();
{
"_id" : ObjectId("6008153cf979ac0f18681765"),
"firstName" : "Sammy",
"lastName" : "Shark"
}
{
"_id" : ObjectId("60081544f979ac0f18681766"),
"firstName" : "Gilly",
"lastName" : "Glowfish"
}
I configured ElasticSearch and the transporter pipeline and now exported MongoDB_URI and Elastic_URI.
I then ran my transporter pipeline.js to obtain this:
INFO[0005] metrics source records: 2 path=source ts=1611154492641006368
INFO[0005] metrics source/sink records: 2 path="source/sink" ts=1611154492641013556
I then try to view my ElasticSearch but get this error:
curl $ELASTICSEARCH_URI/_search?pretty=true
{
"error" : {
"root_cause" : [
{
"type" : "cluster_block_exception",
"reason" : "blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];"
}
],
"type" : "cluster_block_exception",
"reason" : "blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];"
},
"status" : 503
}
Here is my elasticsearch.yml:
# Use a descriptive name for the node:
node.name: node-1
path.data: /var/lib/elasticsearch
# Path to log files:
path.logs: /var/log/elasticsearch
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 0.0.0.0
# Set a custom port for HTTP:
http.port: 9200
# Bootstrap the cluster using an initial set of master-eligible nodes:
cluster.initial_master_nodes: ["node-1", "node-2"]
Here is my elasticsearch node:
{
"name" : "node-1",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "_na_",
"version" : {
"number" : "7.7.1",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "ad56dce891c901a492bb1ee393f12dfff473a423",
"build_date" : "2020-05-28T16:30:01.040088Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
I have tried deleting indices and restarting the server but the error repeats. Would like to know the solution to this. I am using elasticsearch 7.10

Spring Cloud Contract consumer side test returns 404

While running Spring Contract test on consumer side using stubs. I got following response first for endpoint creation then for request that was sent to it and then for the response.
Both type1 and type2 are enum fields, the other fields are string.
127.0.0.1 - POST /mappings
Connection: [keep-alive]
User-Agent: [Apache-HttpClient/4.5.12 (Java/11.0.5)]
Host: [localhost:11291]
Content-Length: [834]
Content-Type: [text/plain; charset=UTF-8]
{
"id" : "7d0b6496-be88-43c0-891d-bdb8ef8ae033",
"request" : {
"url" : "/testEndpoint",
"method" : "PUT",
"headers" : {
"Content-Type" : {
"matches" : "application/json;charset=UTF-8.*"
}
},
"bodyPatterns" : [ {
"matchesJsonPath" : "$[?(#.['name'] == 'Test')]"
}, {
"matchesJsonPath" : "$[?(#.['type1'] == 'NONE')]"
}, {
"matchesJsonPath" : "$[?(#.['type2'] == 'NONE')]"
}, {
"matchesJsonPath" : "$[?(#.['description'] == null)]"
}, {
"matchesJsonPath" : "$[?(#.['comment1'] == null)]"
}, {
"matchesJsonPath" : "$[?(#.['comment2'] == null)]"
} ]
},
"response" : {
"status" : 201,
"transformers" : [ "response-template" ]
},
"uuid" : "7d0b6496-be88-43c0-891d-bdb8ef8ae033"
}
127.0.0.1 - PUT /testEndpoint
Connection: [keep-alive]
User-Agent: [Apache-HttpClient/4.5.12 (Java/11.0.5)]
Host: [localhost:11291]
Accept-Encoding: [gzip,deflate]
Content-Length: [117]
Content-Type: [application/json; charset=UTF-8]
{"name":"Test","type1":"NONE","type2":"NONE","description":null,"comment1":null,"comment2":null}
The response I got was:
Matched response definition:
(no response definition configured)
Response:
HTTP/1.1 404
(no headers)
Actually I see that the problem is that the regex for json content type is "application/json;charset=UTF-8.*" and the actual is "application/json; charset=UTF-8" (notice the space). You should change your contract to support a regex for that space (e.g. application/json.*)

Spring-Data #RepositoryRestResource deleteByName uses wrong HTTP-Method when deleting a resource

I'm faced with a scenario where the custom #RepositoryRestResource interface-method is involved by the wrong HTTP-Method. For example:
#RepositoryRestResource(path = "matches", collectionResourceRel = "matches")
public interface MatchRepo extends Neo4jRepository<Match, Long> {
Collection<Match> findAllByCodeName(#Param("codeName") String codeName);
#Transactional
Long deleteAllByCodeName(#Param("codeName") String codeName);
}
Request:
curl -i -X GET 'http://localhost:8003/spring-data/api/v1/matches/search/findAllByCodeName?codeName=Test-CodeName-1'
Note the above GET HTTP-Method. This is expected, & i'm happy with the Response:
HTTP/1.1 200
Content-Type: application/hal+json;charset=UTF-8
Transfer-Encoding: chunked
Date: Tue, 20 Nov 2018 15:32:49 GMT
{
"_embedded" : {
"matches" : [ {
"id" : "1",
"codeName" : "Test-CodeName-1",
"round" : 1,
"me" : "ROCK",
"pc" : "ROCK",
"result" : "D",
"timestamp" : "Nov 20, 2018, 05:32:27 AM",
"lastUpdated" : "Nov 20, 2018, 05:32:27 AM",
"created" : "Nov 20, 2018, 05:32:27 AM",
"_links" : {
"self" : {
"href" : "http://localhost:8003/spring-data/api/v1/matches/22"
},
"match" : {
"href" : "http://localhost:8003/spring-data/api/v1/matches/22"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "http://localhost:8003/spring-data/api/v1/matches/search/findAllByCodeName?codeName=Test-CodeName-1"
}
}
}%
This is what appears on the Intelli-J Console-Mappings:
http://localhost:8003/spring-data/api/v1/{repository}/search
& I implemented the request as indicated in the mappings, as shown below. But the problem becomes evident when I am deleting a resource with a GET HTTP-Method as shown below:
Request:
curl -i -X GET 'http://localhost:8003/spring-data/api/v1/matches/search/deleteAllByCodeName?codeName=Test-CodeName-1'
Response:
HTTP/1.1 200
Content-Type: application/hal+json;charset=UTF-8
Transfer-Encoding: chunked
Date: Tue, 20 Nov 2018 15:51:33 GMT
{
"10":
}
I need to find a way to make my custom deleteAllByCodeName(#Param) interface-method from the MatchRepo class to execute with the correct HTTP-Method. Must use DELETE HTTP-Method & not the GET HTTP-Method and adhere to the REST-API Design Principles.
The manual notes that search resources only support GET requests.
https://docs.spring.io/spring-data/rest/docs/3.1.2.RELEASE/reference/html/#repository-resources.search-resource
You can prevent this repo method from being exported:
#RestResource(exported = false)
Long deleteAllByCodeName(#Param("codeName") String codeName);
and create a normal Spring MVC controller that handles the delete request.

JFrog Artifactory API query for object properties does not return requested detail

I am requesting label properties for docker artifact, perhaps the url is not correct? I get response object (json) but label properties are not included. Code example:
response = Net::HTTP.get_with_headers("http://myrepo:8081/artifactory/api/storage/dockerv2-local/anonymizer/functional/manifest.json;docker.label.com.company.info.build='*'",
{'Authorization' => 'Bearer <REDACTED>'})
if response.code.to_s == "200"
puts ("Artifactory response "+ response.body)
puts ("response object: "+response.inspect())
else
puts ("Artifactory request returned "+response.code.to_s)
end
Connecting to artifactory
Artifactory response {
"repo" : "dockerv2-local",
"path" : "/anonymizer/functional/manifest.json",
"created" : "2018-03-14T14:52:22.681-07:00",
"createdBy" : "build",
"lastModified" : "2018-03-15T15:52:34.225-07:00",
"modifiedBy" : "build",
"lastUpdated" : "2018-03-15T15:52:34.225-07:00",
"downloadUri" : "http://myrepo:8081/artifactory/dockerv2-local/anonymizer/functional/manifest.json",
"mimeType" : "application/json",
"size" : "1580",
"checksums" : {
"sha1" : "bf2a1f85c7ab8cec14b64d172b7fdaf420804fcb",
"md5" : "9c1bbfc77e2f44d96255f7c1f99d2e8d",
"sha256" : "53e56b21197c57d8ea9838df7cffb3d8f33cd714998d620efd8a34ba5a7e33c0"
},
"originalChecksums" : {
"sha256" : "53e56b21197c57d8ea9838df7cffb3d8f33cd714998d620efd8a34ba5a7e33c0"
},
"uri" : "http://myrepo:8081/artifactory/api/storage/dockerv2-local/anonymizer/functional/manifest.json"
}
response object: #<Net::HTTPOK 200 OK readbody=true>
If I understand you correctly, you want to get the properties of the manifest.json file, "docker.label.com.company.info.build" in particular.
From looking at your command:
response = Net::HTTP.get_with_headers("http://myrepo:8081/artifactory/api/storage/dockerv2-local/anonymizer/functional/manifest.json;docker.label.com.company.info.build='*'",
It seems that you are using a semicolon to get the properties, which is not the right way. As you can see in this REST API, in order to use the get properties you should use the ampersand sign, so your command should look like:
response = Net::HTTP.get_with_headers("http://myrepo:8081/artifactory/api/storage/dockerv2-local/anonymizer/functional/manifest.json&docker.label.com.company.info.build='*'",

Could not to inspect the dedicated host provisioning process

When I used slcli(softlayer-python command) to create a dedicated host, the command return the order id. And I check the order's status was 'APPROVED'. But I can not get the host in the result of 'SoftLayer_Account/getDedicatedHosts'.
So I check the billing item and it is 'dedicated_virtual_hosts' rightly. Did SoftLayer API support another approach to inspect the dedicated host provisioned? Or did I do something wrong?
Yes, the dedicated host should be listed when calling to SoftLayer_Account::getDedicatedHosts method, or when using the "slcli dedicatedhost list" command. I suggest to check your permissions and device access, verify that "View Virtual Dedicated Host Details" is checked.
Below are some slcli commands I executed to order and list dedicated hosts.
To order a dedicated host:
slcli dedicatedhost create -H slahostname -D example.com -d mex01 -f 56_CORES_X_242_RAM_X_1_4_TB
To list dedicated hosts:
slcli dedicatedhost list
:.......:...................:..........:..............:................:............:............:
: id : name : cpuCount : diskCapacity : memoryCapacity : datacenter : guestCount :
:.......:...................:..........:..............:................:............:............:
: 11111 : slahostname : 56 : 1200 : 242 : mex01 : - :
:.......:...................:..........:..............:................:............:............:
Below an example about how to see the details:
slcli dedicatedhost detail 11111
:.................:...........................:
: name : value :
:.................:...........................:
: id : 11111 :
: name : slahostname :
: cpu count : 56 :
: memory capacity : 242 :
: disk capacity : 1200 :
: create date : 2018-02-01T09:53:46-04:00 :
: modify date : :
: router id : 333333 :
: router hostname : bcr01a.mex01 :
: owner : owner001 :
: guest count : 0 :
: datacenter : mex01 :
:.................:...........................:
Using RestFul the response when calling to SoftLayer_Account::getDedicatedHosts should be something like below:
GET:
https://[userName]:[apiKey]#api.softlayer.com/rest/v3/SoftLayer_Account/getDedicatedHosts
RESPONSE:
{
"cpuCount": 56,
"createDate": "2018-02-01T09:53:46-04:00",
"diskCapacity": 1200,
"id": 11111,
"memoryCapacity": 242,
"modifyDate": null,
"name": "slahostname"
}
Also you can use SoftLayer_Virtual_DedicatedHost::getObject method:
GET:
https://[userName]:[apiKey]#api.softlayer.com/rest/v3/SoftLayer_Virtual_DedicatedHost/11111/getObject