Problem -> I want to deploy JanusGraph as separate service on Kubernetes. which storage backend should i use for cassandra. Is it CQL or cassandrathrift?? Cassandra is running as stateful service on Kubernetes.
Detailed Description-> As per JanusGraph doc, in case of Remote Server Mode, storage backend should be cql.
JanusGraph graph = JanusGraphFactory.build().
set("storage.backend", "cql").
set("storage.hostname", "77.77.77.77").
open();
Even they mentioned that Thrift is deprecated going ahead with Cassandra 2.1 & I am using Cassandra 3.
But in some blog, they have mentioned that rest api call from JanusGraph to Cassandra is possible only through Thrift.
Is Thrift really required? Can't we use CQL as storage backend for rest api call as well?
Yes, you absolutely should use the cql storage backend.
Thrift is deprecated, disabled by default in the current version of Cassandra (version 3), and has been removed from Cassandra version 4.
I would also be interested in reading the blog post you referenced. Are you talking about IBM's Rest API mentioned in their JanusGraph-utils Git repo? That confuses me as well, because I see both Thrift and CQL config happening there. In any case, I would go with the cql settings and give it a shot.
tl;dr;
Avoid Thrift at all costs!
Related
Is there any other way to deploy connector config rather than POSTing connector config to kafka connect REST api? https://docs.confluent.io/platform/current/connect/references/restapi.html#tasks
I am thinking of any form of persistent approach like a volume or s3, where connect during bootstrap would grap those configs would be great. Don't know/can't find if thats anywhere available.
regards
The REST API is the only way.
You can use abstractions like Terraform or Kubernetes resources, however, which wrap an HTTP client.
If you use other storage, that'll require you to write extra code to download files and call the REST API.
I am planning to implement a mongoDB storage backend for JanusGraph. The reason for picking MongoDB is mainly because all of our existing infrastructure and services use MongoDB, so it would be less net new maintenance requirement. Where do I get started? Is there a list of APIs that JanusGraph provides that need to be implemented by a custom backend? I couldn't find any documentation.
As of now Janusgraph supports only limited number of storage-backends as of now (Cassandra, HBase, BigTable, Berkeley). You can find more info here https://docs.janusgraph.org/storage-backend/.
We are using Kafka Connectors (JDBC and others), and configuring them using the REST API (using curl in shell scripts). Right now, when testing/developing, we are including secrets (for the JDBC connect - database user/pw) directly in the request. This is obviously bad, as those are then readily available for everybody to see when reading them out using the GET request.
Is there a good way to give secrets to the connectors? We can bring them in safely using environment variables or config files (injected fom OpenShift) - but is there a syntax available when starting a connector via the REST API for that?
EDIT: This is for the distributed mode of connectors; i.e., configuration by REST API, not connector config files...
A pluggable interface for this was implemented in Apache Kafka 2.0 through KIP-297. You can see more details in the documented example here.
Actually I am not very familiar with Dynamodb and I would like to launch a NoSQL database with local Dynamodb (downloadable version) but not hosted on amazon AWS. I would appreciate it if someone could let me know is it possible to make such a clustered or does downloadable version of Dynamodb support to be cluster locally ?
You can very easily run DynamoDB locally, but it only supports running a single instance—not a cluster. It's intended to be used for local testing/debugging.
DynamoDB is provided as a hosted service. Does not exist a DynamoDB code that you can download and install to use as a host or service provider.
As part of SDK for a lot of languages, AWS Team developed some wrappers that permits you to execute local versions of DynamoDB to test your particular code. These wrappers respect the DynamoDB API contract. In that case you can code to the DynamoDB interface and get the responses like it were hosted in AWS environment. But you can't host any database or even serve data as a service using this solutions.
Using Spark 1.3.1, when a master node is started with ./sbin/start-master.sh, a RESTful webservice is started on that machine (for me port 6066). Is there any documentation on how to interact with that service?
I found this code, but I was not able to find the corresponding Scaladoc let alone some sort of guide.
Here's the JIRA ticket, contains the Design Doc that motived this feature.
The goal is to create a new submission gateway that is stable across Spark versions
Additionally,
It is also not a goal to expose the new gateway as a general mechanism
for users of Spark to submit their applications. The new gateway will
be used strictly internally between Spark submit and the standalone
Master.