I've got several gRPC microservices deployed via Istio in my k8s pod behind a gateway that handles the routing for web clients. Things work great when I need to send an RPC from client (browser) to any of these services.
I'm now at the point where I want to call service A from service B directly. How do I go about doing that?
Code for how both the servers are instantiated:
const server = new grpc.Server();
server.addService(MyService, new MyServiceImpl());
server.bindAsync(`0.0.0.0:${PORT_A}`, grpc.ServerCredentials.createInsecure(), () => {
server.start();
});
A Service Account is being used with GOOGLE_APPLICATION_CREDENTIALS and a secret in my deployment YAML.
To call service A from service B, I was thinking the code in service B would look something like:
const serviceAClient: MyServiceClient = new MyServiceClient(`0.0.0.0:${PORT_A}`, creds);
const req = new SomeRpcRequest()...;
serviceAClient.someRpc(req, (err: grpc.ServiceError, response: SomeRpcResponse) => {
// yay!
});
Is that naive? One thing I'm not sure about is the creds that I need to pass when instantiating the client. I get complaints that I need to pass ChannelCredentials, but every mechanism I've tried to create those creds has not worked.
Another thing I'm realizing is that 0.0.0.0 can't be correct because each service is in its own container paired with a sidecar proxy... so how do I route RPCs properly and attach proper creds?
I'm trying to construct the creds this way:
let callCreds = grpc.CallCredentials.createFromGoogleCredential(myOauthClient);
let channelCreds = grpc.ChannelCredentials.createSsl().compose(callCreds);
const serviceAClient = new MyServiceClient(`0.0.0.0:${PORT_A}`, channcelCreds);
and I'm mysteriously getting the following error stack:
UnhandledPromiseRejectionWarning: TypeError: Channel credentials must be a ChannelCredentials object
at new ChannelImplementation (/bish/proto/activities/node_modules/#grpc/grpc-js/build/src/channel.js:69:19)
at new Client (/bish/proto/activities/node_modules/#grpc/grpc-js/build/src/client.js:58:36)
at new ServiceClientImpl (/bish/proto/activities/node_modules/#grpc/grpc-js/build/src/make-client.js:58:5)
at PresenceService.<anonymous> (/bish/src/servers/presence/dist/presence.js:348:44)
at step (/bish/src/servers/presence/dist/presence.js:33:23)
at Object.next (/bish/src/servers/presence/dist/presence.js:14:53)
at fulfilled (/bish/src/servers/presence/dist/presence.js:5:58)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
Which is odd because channelCreds is a ComposedChannelCredentialsImpl which, in fact, extends ChannelCredentials
Ok, at least the root cause of the "Channel credentials must be a ChannelCredentials object" error is now known. I'm developing node packages side-by-side as symlinks and each of the dependencies has their own copy of grpc-js in it.
https://github.com/npm/npm/issues/7742#issuecomment-257186653
Related
I have the following error trying to create a static website inspired by https://github.com/aws-samples/aws-cdk-examples/blob/master/typescript/static-site/static-site.ts
const certificateArn = new acm.DnsValidatedCertificate(
this,
"SiteCertificateR53",
{
domainName: props.siteDomain,
hostedZone: props.zone,
region: "us-east-1", // Cloudfront only checks this region for certificates.
}
).certificateArn;
new cdk.CfnOutput(this, "CertificateR53", {value: certificateArn});
Error:
Received response status [FAILED] from custom resource. Message returned: Resource is not in the state certificateValidated
If you don't need to do cross region stuff (e.g. us-east-1 needs a resource from us-west-2) using the following method provides the same benefit as DnsValidatedCertificate
const certificate = new acm.Certificate(this, `SiteCertificateR53`, {
domainName: props.siteDomain,
validation: acm.CertificateValidation.fromDns(props.zone)});
If you still gotta do cross-region stuff, then you should create and deploy your zone via AWS console first. That won't guarantee a fix though, this page can help if you're still stuck: https://docs.aws.amazon.com/acm/latest/userguide/troubleshooting-DNS-validation.html
I need to create many MongoDB Atlas endpoint connections using terraform.
I successfully create first, using this code:
#Private endpoint connection
resource "mongodbatlas_private_endpoint" "dbpe" {
project_id = var.prj_id
provider_name = "AWS"
region = var.aws_region
}
#AWS endpoint for secure connect to mongo db
resource "aws_vpc_endpoint" "ec2" {
vpc_id = var.sh_vpc
#service_name = "com.amazonaws.${var.aws_region}.ec2"
service_name = mongodbatlas_private_endpoint.dbpe.endpoint_service_name
vpc_endpoint_type = "Interface"
security_group_ids = [
aws_security_group.lb_sg.id,
]
subnet_ids = [
aws_subnet.subnet1.id,
var.sh_subnet
]
tags = {
"Name" = local.tname
}
#private_dns_enabled = true
}
But when I try to use this code second time in another folder (another tfstate) it failed cause error:
Error: error creating MongoDB Private Endpoints Connection: POST https://cloud.mongodb.com/api/atlas/v1.0/groups/***/privateEndpoint: 409 (request "Conflict") A PrivateLink Endpoint Service already exists for AWS region US_EAST_2.
As I understand, a second "mongodbatlas_private_endpoint" "dbpe" trying to create another one Endpoint service. But, when I creating second Endpoint manually through WebUI, it using the same service like first Endpoint.
How I can tell to second Endpoint to use the existing service?
Or maybe it all wrong?
Please, help!
Thank you!
I found the solution.
Creating the "Endpoint Connection" really creates Endpoint only when you do it at first time. All of next times is creating an only association between Atlas endpoint and new AWS Endpoint.
In terraform I tried to create an Atlas endpoint second time and catch an error (because of limit - 1 endpoint per region). All I need to do - is create "Basic Endpoint" one time (by separate folder with own tfstate) and don't delete it. And for each new AWS endpoint need to create a new link from AWS Endpoint to "Basic". I do it by a terraform resource:
mongodbatlas_private_endpoint_interface_link
Resource "mongodbatlas_private_endpoint" is not need now. A "service_name" parameter in "aws_vpc_endpoint" you can hardcoded from "Basic" Endpoint. Use "output" to see mongodbatlas_private_endpoint.test.private_link_id - this is what you need.
Compiler error when using example provided in Flink documentation. The Flink documentation provides sample Scala code to set the REST client factory parameters when talking to Elasticsearch, https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html.
When trying out this code i get a compiler error in IntelliJ which says "Cannot resolve symbol restClientBuilder".
I found the following SO which is EXACTLY my problem except that it is in Java and i am doing this in Scala.
Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
I tried copy pasting the solution code provided in the above SO into IntelliJ, the auto-converted code also has compiler errors.
// provide a RestClientFactory for custom configuration on the internally created REST client
// i only show the setMaxRetryTimeoutMillis for illustration purposes, the actual code will use HTTP cutom callback
esSinkBuilder.setRestClientFactory(
restClientBuilder -> {
restClientBuilder.setMaxRetryTimeoutMillis(10)
}
)
Then i tried (auto generated Java to Scala code by IntelliJ)
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
import org.apache.http.auth.AuthScope
import org.apache.http.auth.UsernamePasswordCredentials
import org.apache.http.client.CredentialsProvider
import org.apache.http.impl.client.BasicCredentialsProvider
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.client.RestClientBuilder
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
esSinkBuilder.setRestClientFactory((restClientBuilder) => {
def foo(restClientBuilder) = restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = { // elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
}
})
foo(restClientBuilder)
})
The original code snippet produces the error "cannot resolve RestClientFactory" and then Java to Scala shows several other errors.
So basically i need to find a Scala version of the solution described in Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
Update 1: I was able to make some progress with some help from IntelliJ. The following code compiles and runs but there is another problem.
esSinkBuilder.setRestClientFactory(
new RestClientFactory {
override def configureRestClientBuilder(restClientBuilder: RestClientBuilder): Unit = {
restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
// elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
httpClientBuilder.setSSLContext(trustfulSslContext)
}
})
}
}
The problem is that i am not sure if i should be doing a new of the RestClientFactory object. What happens is that the application connects to the elasticsearch cluster but then discovers that the SSL CERT is not valid, so i had to put the trustfullSslContext (as described here https://gist.github.com/iRevive/4a3c7cb96374da5da80d4538f3da17cb), this got me past the SSL issue but now the ES REST Client does a ping test and the ping fails, it throws an exception and the app shutsdown. I am suspecting that the ping fails because of the SSL error and maybe it is not using the trustfulSslContext i setup as part of new RestClientFactory and this makes me suspect that i should not have done the new, there should be a simple way to update the existing RestclientFactory object and basically this is all happening because of my lack of Scala knowledge.
Happy to report that this is resolved. The code i posted in Update 1 is correct. The ping to ECE was not working for two reasons:
The certificate needs to include the complete chain including the root CA, the intermediate CA and the cert for the ECE. This helped get rid of the whole trustfulSslContext stuff.
The ECE was sitting behind an ha-proxy and the proxy did the mapping for the hostname in the HTTP request to the actual deployment cluster name in ECE. this mapping logic did not take into account that the Java REST High Level client uses the org.apache.httphost class which creates the hostname as hostname:port_number even when the port number is 443. Since it did not find the mapping because of the 443 therefore the ECE returned a 404 error instead of 200 ok (only way to find this was to look at unencrypted packets at the ha-proxy). Once the mapping logic in ha-proxy was fixed, the mapping was found and the pings are now successfull.
I'm trying to understand in what scenarios endpoint definitions are required in the ServiceManifest. I have a stateful service with the multiple service remoting listeners defined. My implementation of CreateServiceReplicaListeners:
protected override IEnumerable<ServiceReplicaListener> CreateServiceReplicaListeners()
{
return new[]
{
new ServiceReplicaListener(context => this.CreateServiceRemotingListener(context)),
new ServiceReplicaListener(context =>
{
return new FabricTransportServiceRemotingListener(context,
new CustomService<string>(),
new FabricTransportRemotingListenerSettings
{
EndpointResourceName = "ustdfdomgfasdf"
});
}, name: "CustomListener")
};
}
The endpoint resource name for the custom listener is garbage. I have not defined that endpoint in the service manifest's resources:
<Resources>
<Endpoints>
<Endpoint Name="ServiceEndpoint" />
<Endpoint Name="ReplicatorEndpoint" />
</Endpoints>
</Resources>
However, in testing I find I'm still able to get a proxy to CustomListener:
InventoryItem i = new InventoryItem(description, price, number, reorderThreshold, max);
var applicationInstance = FabricRuntime.GetActivationContext().ApplicationName.Replace("fabric:/", String.Empty);
var inventoryServiceUri = new Uri("fabric:/" + applicationInstance + "/" + InventoryServiceName);
//Doesn't fail
ICustomService customServiceClient = ServiceProxy.Create<ICustomService>(inventoryServiceUri,
i.Id.GetPartitionKey(),
listenerName: "CustomListener");
//Still doesn't fail
var added = await customServiceClient.Add(1, 2);
To me, this indicates endpoint definitions aren't required for service remoting as long as the endpoint and listener names are unique. Is that so? If not, why does my example work?
Endpoints are required to tell service fabric to allocate ports in the node for the services being started on that node, this will prevent port collision when many services are opening ports in the node.
Once allocated, these are create as Environment Variables in the service process, something like: Fabric_Endpoint_<EndpointName> : port
When you create the Listeners, they are responsible to open the ports, generally using the ports allocated via Endpoints, but not prevents you creating a custom listener to Open any port (If running with enough privilege to do so)
CreateServiceRemotingListener(context) creates the default listeners
EndpointResourceName setting tell which endpoint to be used by a listener, if not defined, DefaultEndpointResourceName setting is used as the default Endpoint, the default value is "ServiceEndpoint"
What I am not sure to answer right now is: if EndpointResourceName is not found, it uses DefaultEndpointResourceName, I assume so, need to check the code to confirm that.
When multiple listeners are using the same port, they generally have a path to identify each of them, something like: tcp://127.0.0.1:1234/endpointpath
I am trying to run a simple program of jcloud. The program is as follows:
String provider = "openstack-nova";
String identity = "Tenant:usename"; // tenantName:userName
String credential = "pass";
novaApi = ContextBuilder.newBuilder(provider).endpoint("http://openstack.infosys.tuwien.ac.at/identity/v2.0")
.credentials(identity, credential).modules(modules).buildApi(NovaApi.class);
regions = novaApi.getConfiguredRegions();
The openstack.infosys is connect via SOCKS proxy on port 7777. I have also enlisted the same on eclipse(Window->Preferences->General->Network Config->SOCKS(Manual)) . However, everytime I run the code I get the following error:
ERROR o.j.h.i.JavaUrlHttpCommandExecutorService - Command not considered safe to retry because request method is POST:
Which is then caused by
Caused by: java.net.SocketTimeoutException: connect timed out
I am able to access the horizon web interface of the same without any issues.
Can someone please help me in understanding what is the possible problem.
You need to tell Apache jclouds about your proxy configuration when creating the context. Have a look at these properties, and pass the ones you need to the overrides method of the ContextBuilder:
Proxy type
Proxy host
Proxy port
Proxy user
Proxy password