I am wondering if I could have json data type in the column family:
My table will have a unique row key, and column name of "tweets_json" and column value of json content.
How would I create such a table in CQL/cassandra or using python Driver or CQLengine?
tweet_json = json.encode({
"tweet_id" : tweet_id,
"body" : tweet_body,
"user_name" : this_user,
"timestamp" : timestamp
})
Basicall
Just have a text column? If you're not looking to filter on the json, then that should be fine. If you are, then you'll need to store it as proper columns, rather than as json.
Related
I am using kafka connect to sink some data from a kafka topic to a cassandra table. I want to add a column with a timestamp when the insert/update happens into cassandra. That is easy with postgres with functions and triggers. To use the same in cassandra that is not going to happen. I can not add java code to the cassandra cluster.
So I am thinking on how to add this value injecting a now() at some point of the kafka connect and inserted on the cassandra table as the result of the execution of the function.
I read kafka connect to cassandra uses de JSON cassandra insert api.
https://docs.datastax.com/en/cql-oss/3.3/cql/cql_using/useInsertJSON.html
I tried to insert with different formats but nothing worked.
INSERT INTO keyspace1.table1 JSON '{ "lhlt" : "key 1", "last_updated_on_cassandra" : now() }';
InvalidRequest: Error from server: code=2200 [Invalid query] message="Could not decode JSON string as a map: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'now': was expecting 'null', 'true', 'false' or NaN
at [Source: { "lhlt" : "key 1", "last_updated_on_cassandra" : now() }; line: 1, column: 74]. (String was: { "lhlt" : "key 1", "last_updated_on_cassandra" : now() })"
The CQL grammar does not support the use of functions when inserting data in JSON format.
The INSERT INTO ... JSON command only accepts valid JSON. For example:
INSERT INTO tstamp_tbl JSON '{
"id":1,
"tstamp":"2022-10-29"
}';
You will need to use the more generic INSERT syntax if you want to call CQL functions. For example:
INSERT INTO tstamp_tbl (id, tstamp)
VALUES ( 1, toTimestamp(now()) );
Cheers!
everyone. I am trying to create a stream from another stream (using CREATE FROM SELECT) that contains JSON data stored as VARCHAR. My first intention was to use EXTRACTJSONFIELD to extract specific JSON fields to separate columns. But on an attempt to query a new stream I am getting the null values for fields created using EXTRACTJSONFIELD.
Example:
My payload looks like this:
{
"data": {
"some_value": 3702,
},
"metadata": {
"timestamp": "2022-05-23T23:54:38.477934Z",
"table-name": "some_table"
}
}
I create stream using
CREATE STREAM SOME_STREAM (data VARCHAR, metadata VARCHAR) WITH (KAFKA_TOPIC='SOME_TOPIC', VALUE_FORMAT='JSON');
Data inside looks like this:
Then I am trying to create new stream using:
CREATE STREAM SOME_STREAM_QUERYABLE
AS SELECT
DATA,
METADATA,
EXTRACTJSONFIELD(METADATA, '$.timestamp') as timestamp
FROM SOME_STREAM
EMIT CHANGES
The attempt to query this new stream gives me null for timestamp field. Data and metadata fields have valid json.
I want to store a table for variables with table definition like below
variable
--------
id int
var_type int 0: number, 1: string, 2: json
var_name int
var_value ??? varchar or jsonb?
If I use varchar, how to store the json type variable and if, I am using jsonb how to store the int and string?
The example json value that is stored,
[{"name": "Andy", "email" : "andy#mail.id"},{"name": "Cindy", "email" : "cindy#mail.id"}]
TIA
Beny
When you have data and you don't know the structure, use a single jsonb column. JSON can handle strings, numbers, and more JSON.
{
"string": "basset hounds got long ears",
"number": 23.42,
"json": [1,2,3,4,5]
}
Don't try to cram them all into a single array. Put them in separate rows.
One row: {"name": "Andy", "email" : "andy#mail.id"}
Another row: {"name": "Cindy", "email" : "cindy#mail.id"}
However, your example feels like its avoiding designing a schema. JSONB is useful, but overusing it defeats the point of a relational database.
create table people (
id bigserial primary key,
// Columns for known keys which can have constraints.
name text not null,
email text not null,
// JSONB for extra keys you can't predict.
data jsonb
)
Use the JSON operators to query individual pairs.
select
name, email, data->>'favorite dog breed'
from some_table
I'm using the mongo-go driver for Go to save some documents on mongoDb. Everyhting works fine, but i'm wondering if there is a way to change how the ID is auto-generated. Right now the document model in the code has the primitive.ObjectID type, is something like this
type Review struct {
ID primitive.ObjectID `json:"id,omitempty" bson:"_id,omitempty"`
Title string `json:"title"`
Text string `json:"text"`
Rate float64 `json:"rate"`
Date time.Time `json:"date"`
Product Product `json:"product"`
}
And the document created is something like this
{
"_id" : ObjectId("5d6f739a20d42db438016cb1"),
"title" : "test-id",
"text" : "rev-text",
"rate" : 5.0,
"date" : ISODate("2019-09-02T12:18:00.000+02:00"),
"status" : "pending"
}
So far so good. However i want the ID to be a UUID, not an ObjectId. I know i can change the struct ID type to UUID or string and set that field when i'm saving the document, however i want to know if there is a way to change the default ObjectID generator of mongoDb so it generates a UUID automatically when saving a new document
You don't have to use an ObjectID for _id:
ID string `json:"id,omitempty" bson:"_id,omitempty"`
Then you can insert any value you want for _id. However, you won't be able to auto-generate ID this way, you have to generate the ID yourself, and insert it with the generated ID.
I have a class etest1 with properties name and text. Then I have a class etest2 with properties lala and etest (the embedded etest1). But I am not able to insert anything.
If I do:
insert into etest2 content {"lala" : "test lala", etest : {"name" : "das", "text" : "dasd"}}
I get an exception: The field 'etest2.etest' has been declared as EMBEDDED but an incompatible type is used. Value: {name=das, text=dasd}
If I do:
insert into etest2 content {"lala" : "test lala", etest : "#17:10"}
I get another exception: The field 'etest2.etest' has been declared as EMBEDDED but the value is the RecordID #17:10
So how can I insert a embedded document. If it is somehow possible I would like something like the first method I tried. I would like more doing just one insert instead of two inserts.
If this is your situation:
create class etest1
create property etest1.name string
create property etest1.text string
create class etest2
create property etest2.lala string
create property etest2.etest embedded etest1
you can do:
insert into etest2 content {"lala" : "test lala", etest : {"#type":"d", "#class":"etest1", "name" : "das", "text" : "dasd"}}