I wanna query crateDB through ES client.But failed
I set the es.api.enabled: true in the crate.yml and I used the ES java-rest-high-level-client to connect crateDB via port 4200, but I cannot get any response
public class crateAPI {
public static void main(String[] args) throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("localhost", 4200, "http")));
SearchRequest searchRequest =new SearchRequest();
SearchSourceBuilder searchSourceBuilder =new SearchSourceBuilder();
searchSourceBuilder.query
(QueryBuilders.termQuery("test.news.content","test"));
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest);
for(SearchHit hit : searchResponse.getHits().getHits()){
System.out.println(hit.getSourceAsString());
}
client.close();
}
}
You probably shouldn't (and soon be deprecated) as advised in their official documentation here
But if you have to in the mean time, Crate will respond to a standard ES URL search like so
http://your-server:4200/*your-index-name*/_search?q=your-field:your-search-value
Related
I'm trying to make a Kafka producer and consumer, but my project is in dotnet Core 2.0 and it doesn't seem to work well with kafka. This is the proof-of-concept I've tried coming up with. I'm using Visual Studio 2017 with the kafka-net nuget package:
using
using KafkaNet;
using KafkaNet.Model;
using KafkaNet.Protocol;
producer
static void Main(string[] args)
{
string payload = "Welcome to Kafka!";
string topic = "IDGTestTopic";
Message msg = new Message(payload);
Uri uri = new Uri("localhost:9092");
var options = new KafkaOptions(uri);
var router = new BrokerRouter(options);
var client = new Producer(router);
client.SendMessageAsync(topic, new List<Message> { msg }).Wait();
Console.ReadLine();
}
consumer
static void Main(string[] args)
{
string topic = "IDGTestTopic";
Uri uri = new Uri("http://localhost:9092");
var options = new KafkaOptions(uri);
var router = new BrokerRouter(options);
var consumer = new Consumer(new ConsumerOptions(topic, router));
foreach (var message in consumer.Consume())
{
Console.WriteLine(Encoding.UTF8.GetString(message.Value));
}
Console.ReadLine();
}
When I try to run the producer first, I get an error message on the BrokerRouter:
$exception {System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: port
at System.Net.IPEndPoint..ctor(IPAddress address, Int32 port)
at KafkaNet.DefaultKafkaConnectionFactory.Resolve(Uri kafkaAddress, IKafkaLog log)
at KafkaNet.Model.KafkaOptions.<get_KafkaServerEndpoints>d__0.MoveNext()
at KafkaNet.BrokerRouter..ctor(KafkaOptions kafkaOptions)
at SampleKafkaProducer.Program.Main(String[] args) in C:\v4target\SampleKafka\SampleKafkaProducer\SampleKafkaProducer\Program.cs:line 18} System.ArgumentOutOfRangeException
How is a port of 9092 out of range? My Visual Studio projects are running on ports in the 55000's. Multiple sources I've researched use 9092 as a kafka port.
Does anyone understand the error message? Is part of the main problem because I'm using a version of Kafka not compatible with dotnet core?
The problem is with the uri.
Uri uri = new Uri("localhost:9092");
If you print out the uri.Port, it's -1. Hence the ArgumentOutOfRangeException.
Try this instead:
Uri uri = new Uri("http://localhost:9092");
From the KafkaNet Repository. This is how they setup the URI:
var options = new KafkaOptions(new Uri("http://CSDKAFKA01:9092"), new Uri("http://CSDKAFKA02:9092"))
{
Log = new ConsoleLog()
};
I am developing a REST client using JBOSS app server and RESTEasy 2.3.6. I've included the following line at the beginning of my code:
RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
Here's the rest of the snippet:
RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(host, port, AuthScope.ANY_REALM), new UsernamePasswordCredentials(userid,password));
ClientExecutor executor = createAuthenticatingExecutor(httpclient, host, port);
String uriTemplate = "http://myhost:8080/webapp/rest/MySearch";
ClientRequest request = new ClientRequest(uriTemplate, executor);
request.accept("application/json").queryParameter("query", searchArg);
ClientResponse<SearchResponse> response = null;
List<MyClass> values = null;
try
{
response = request.get(SearchResponse.class);
if (response.getResponseStatus().getStatusCode() != 200)
{
throw new Exception("REST GET failed");
}
SearchResponse searchResp = response.getEntity();
values = searchResp.getValue();
}
catch (ClientResponseFailure e)
{
log.error("REST call failed", e);
}
finally
{
response.releaseConnection();
}
private ClientExecutor createAuthenticatingExecutor(DefaultHttpClient client, String server, int port)
{
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local auth cache
BasicScheme basicAuth = new BasicScheme();
HttpHost targetHost = new HttpHost(server, port);
authCache.put(targetHost, basicAuth);
// Add AuthCache to the execution context
BasicHttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.AUTH_CACHE, authCache);
// Create ClientExecutor.
ApacheHttpClient4Executor executor = new ApacheHttpClient4Executor(client, localContext);
return executor;
}
The above is a fairly simple client that employs the ClientRequest/ClientResponse<T> technique. This is documented here. The above code does work (only left out some trivial variable declarations like host and port). It is unclear to me from the JBOSS documentation as to whether I need to run RegisterBuiltin.register first. If I remove the line completely - my code still functions. Do I really need to include the register method call given the approach I have taken? The Docs say I need to run this once per VM. Secondly, if I am required to call it, is it safe to call more than one time in the same VM?
NOTE: I do understand there are newer versions of RESTEasy for JBOSS, we are not there yet.
Currently I am using Apache http components client V4.3.5. In my case, I can upload small file(1kb), but it is not working on large file(100kb) when I run the code and get the exception "org.apache.http.NoHttpResponseException: 192.168.128.109:443 failed to respond". Can anyone take a look at my code and let me know what causes my issue?
Here is my code:
public static void main(String[] args) throws IOException,
KeyStoreException {
try {
SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(
null, new TrustStrategy() {
public boolean isTrusted(X509Certificate[] chain,
String authType) throws CertificateException {
return true;
}
}).build();
SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(
sslContext,
SSLConnectionSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
HttpClientBuilder builder = HttpClientBuilder.create();
builder.disableContentCompression();
builder.setSSLSocketFactory(sslsf);
SocketConfig config = SocketConfig.custom().setSoKeepAlive(true).setSoTimeout(300000).build();
builder.setDefaultSocketConfig(config);
CloseableHttpClient httpclient = builder.build();
HttpPost httppost = new HttpPost("https://192.168.128.109/upload");
String encodedAuthorization = DatatypeConverter
.printBase64Binary("admin:itronitr".getBytes());
httppost.setHeader("Authorization", "Basic " + encodedAuthorization);
FileBody bin = new FileBody(new File("c:\\test.txt"));
String boundary = "hK1oPL5_XSfbm6lkCNlKI63rltrew5Bqik0ul";
HttpEntity reqEntity = MultipartEntityBuilder.create()
.setBoundary(boundary).addPart("upfile", bin).build();
httppost.setEntity(reqEntity);
System.out.println(httppost.getEntity().getContentLength());
System.out
.println(httppost.getEntity().getContentType().toString());
CloseableHttpResponse response = httpclient.execute(httppost);
HttpEntity resEntity = response.getEntity();
System.out.println(response.getStatusLine());
String content = EntityUtils.toString(resEntity);
System.out.println(content);
} catch (Exception exc) {
exc.printStackTrace();
}
}
Thanks,
Bill
Finally I fix the issue and it is caused by buffer size. By default, buffer size of httpclient is 8k. So I change it to 4k and my code works well.
Here is the code that changes buffer size:
ConnectionConfig connectionConfig = ConnectionConfig.custom()
.setBufferSize(4128)
.build();
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultConnectionConfig(connectionConfig)
.build();
This is what worked for me; may or may not work for you!!
I recently encountered the same issue and tried all the suggestions whatever I was able to find on internet i.e upgrading httpClient to latest version and adding a re-try handler ; but none fixed it for me.
I already had a re-try handler built in my code and was running on the latest Apache client, but it was still failing with the exception Caused by: org.apache.http.NoHttpResponseException: xxxxx:443 failed to respond
So, took me almost 2 days to debug this issue and find the root cause (at-least in my case)
There seems to be a bug in older Java versions up to Java 11.0.3 included that prevents Apache HTTP Client from sending payloads bigger than 16368 bytes caused by https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8214339.
I was running on java 11.0.2 and when I upgraded to 11.0.10, it worked for me and I was able to send the bigger payload without any code changes
I also faced the similar problem. I went through many blogs and forums and tried various things but none worked for me. So, I tried a workaround. I added retry handler as below. And it worked for me:
HttpClientBuilder.create()
.setDefaultCredentialsProvider(provider)
.setRetryHandler(new DefaultHttpRequestRetryHandler() {
#Override
public boolean retryRequest(final IOException exception, final int executionCount, final HttpContext context) {
if (exception instanceof NoHttpResponseException) {
return true;
}
return super.retryRequest(exception, executionCount, context);
}
})
.build();
Although it is not a correct fix and just a workaround but it is working for me for now. I'll stick to this solution till I won't get any permanent solution. Sharing it here in case someone might get benefit from it.
Could someone explain why this httpunit test case keeps failing in wc.getResponse with "bad file descriptor". I added the is.close() as a guess and moved it before and after the failure but that had no effect. This tests put requests to a Dropwizard app.
public class TestCircuitRequests
{
static WebConversation wc = new WebConversation();
static String url = "http://localhost:8888/funl/circuit/test.circuit1";
#Test
public void testPut() throws Exception
{
InputStream is = new FileInputStream("src/test/resources/TestCircuit.json");
WebRequest rq = new PutMethodWebRequest(url, is, "application/json");
wc.setAuthentication("FUNL", "foo", "bar");
WebResponse response = wc.getResponse(rq);
is.close();
}
No responses? So I'll try myself based on what I learned fighting this.
Httpunit is an old familiar tool that I'd use if I could. But it hasn't been updated in more than two years, so I gather its support for #PUT requests isn't right.
So I converted to Jersey-client instead. After a bunch of struggling I wound up with this code which does seem to work:
#Test
public void testPut() throws Exception
{
InputStream is = new FileInputStream("src/test/resources/TestCircuit.json");
String circuit = StreamUtil.readFully(is);
is.close();
Authenticator.setDefault(new MyAuthenticator());
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
com.sun.jersey.api.client.WebResource service = client.resource(url);
Builder builder = service.accept(MediaType.APPLICATION_JSON);
builder.entity(circuit, MediaType.APPLICATION_JSON);
builder.put(String.class, circuit);
return;
}
This intentionally avoids JAX-RS automatic construction of beans from JSON strings.
I am new to Restlet framework and I have the following time issue in the post method of my server resource.
My post method code
#Post
public Representation represent(Representation entity){
try{
//Thread.sleep(1000);
long start = System.currentTimeMillis();
Form aForm = new Form(getRequestEntity());
System.err.println("FORM Instantiation TIME: " + (System.currentTimeMillis()-start));
}catch(Exception ex){
ex.printStackTrace();
}
return new StringRepresentation("hello");
}
On different trails, the output that I am getting is 1900-1999 ms. But if I uncomment the line Thread.sleep(1000), then the time output is 900-999 ms. Can any one please confirm what is happening when instantiation the Form object and why the time is always 1900+ ms. Sorting out this time issue is important for me as I have to implement token based authentication to reduce the post method processing time.
Sorry for late reply. The restlet version I am using is 2.0.7
Here is the details
public static void main(String[] args) throws Exception {
Component component = new Component();
component.getServers().add(Protocol.HTTP, 8182);
VirtualHost aHost = component.getDefaultHost();
aHost.attach("/sample", new MyApplication());
component.getLogger().setLevel(Level.OFF);
component.start();
System.err.println("REST SERVICE STARTED ON PORT NUMBER 8182...");
}
I am running this application in local and not in any Web/App Server.