End goal: Read column values from DB, convert to strings, split and return the unique values.
I have a postgres table with 2 columns(both text datatype) that I am trying to read from such that key1 = column1_name, value1 = column1 values and key2 = column2_name, value2 = column2 values. Or get each col value in a separate variable altogether(key-value pair not needed then).
Table data looks like -
cities countries
New York, LA America
LA, Delhi, Mumbai America, India
GORM has been used for ORM so I tried an approach -
Type Values struct {
cities string
countries string
}
var values []Values
db, err := gorm.Open(...)
result := db.Table("name").Select("cities, countries").Scan(&values)
// values[0] returns first row with col1, col2
// values[1] returns 2nd row "
// values looks like [{New York, LA, America}, {LA, Delhi, Mumbai, America, India}]
// I want - [{cities - New York, LA, Delhi, Mumbai}, {countries - America, India}]
// LA and America are repeated values which should either be removed at the time of querying or later
I want the result to look like -
col1_name: col1 values =>
cities - New York, LA, Delhi, Mumbai
col2_name: col2 values =>
countries - America, India
There are repeated values in the columns from which I need to return unique values for each column.
Please help me retrieve data in the required format to be ultimately returned as unique strings that can be passed into another function.
Related
I have a parquet file with the following format
id = 2a1ed0848022
raw_value:
[{"state":"MO","city":"O Fallon","location_name":"Jackson Hewitt Tax Service","top_category":"Accounting, Tax Preparation, Bookkeeping, and Payroll Services"},
{"state":"IL","city":"Collinsville","location_name":"L E Smith Jewelry","top_category":"Jewelry, Luggage, and Leather Goods Stores"},
{"state":"MO","city":"O Fallon","location_name":"Bagwasi Family Eyecare","top_category":"Health and Personal Care Stores"},
{"state":"MO","city":"O Fallon","location_name":"Rally's Drive-In Restaurants","top_category":"Restaurants and Other Eating Places"},
{"state":"IL","city":"Collinsville","location_name":"BP","top_category":"Gasoline Stations"}
I would like to create a table in Athena on this parquet file and run a query like this
select maid from test12 where state="MD" and city="Baltimore".
How can I search state and city from the second column which has nested JSON.
The key to this is to use UNNEST. I'm assuming that raw_value is typed as array<struct<state:string,city:string,location_name:string,top_category:string>>.
SELECT id
FROM the_table CROSS JOIN UNNEST(raw_value) rv (location)
WHERE location.state = 'MD' AND city = 'Baltimore'
Using UNNEST like this expands each row in the table to one row per element in the array.
If raw_value is a string column, you need to parse it first. You can find an example of this in this answer: https://stackoverflow.com/a/56176204/1109
When I created the table Tab, I specified the columns as string,
Tab: ([Key1:string()] Col1:string();Col2:string();Col3:string())
But the column datatype (t) is empty. I suppose specifying the column as string has no effect.
meta Tab
c t f a
--------------------
Key1
Col1
Col2
Col3
After I do a bulk upsert in Java...
c.Dict dict = new c.Dict((Object[]) columns.toArray(new String[columns.size()]), data);
c.Flip flip = new c.Flip(dict);
conn.c.ks("upsert", table, flip);
The datatypes are all symbols:
meta Tab
c t f a
--------------------
Key1 s
Col1 s
Col2 s
Col3 s
How can I specify the datatype of the columns as string and have it remain as string?
You cant define a column of the empty table with as strings as they are merely lists of lists of characters
You can just set them as empty lists which is what your code is doing.
But the column will then take on the type of whatever data is inserted into it.
Real question is what is your java process sending symbols when it should be sending strings. You need to make the change there before publishing to KDB
Note if you define as chars you still wont be able to upsert strings
q)Tab: ([Key1:`char$()] Col1:`char$();Col2:`char$();Col3:`char$())
q)Tab upsert ([Key1:enlist"test"] Col1:enlist"test";Col2:enlist"test";Col3:enlist "test")
'rank
[0] Tab upsert ([Key1:enlist"test"] Col1:enlist"test";Col2:enlist"test";Col3:enlist "test")
^
q)Tab: ([Key1:()] Col1:();Col2:();Col3:())
q)Tab upsert ([Key1:enlist"test"] Col1:enlist"test";Col2:enlist"test";Col3:enlist "test")
Key1 | Col1 Col2 Col3
------| --------------------
"test"| "test" "test" "test"
KDB does not allow to define column types as list during creation of table. So that means you can not define your column type as String because that is also a list.
To do that only way is to define column as empty list like:
q) t:([]id:`int$();val:())
Then when you insert data to this table the column will automatically take type of that data.
q)`t insert (4;"row1")
q) meta t
c | t f a
---| -----
id | i
val| C
In your case, one option is to send string data from your Java process as mentioned by user 'emc211' or other option is to convert your data to string in KDB process before insertion.
I have a postgresql table of cities (1 row = 1 city) with a jsonb colum containing the name of the city in different languages (as a list, not an array). For example for Paris(France) I have:
id_city (integer) = 7444
name_city (text) = Paris
names_i18n (jsonb) = {"name:fr":"Paris","name:zh":"巴黎","name:it":"Parigi",...}
In reality in my table I have around 20 different languages. So I try to find a city looking for any name:xx's value that could match a parameter given by the user, but I can't find how to query the jsonb column in that way. I've tried something like the request below but it doesn't seem to be the good syntaxe
select * from jsonb_each_text(select names_i18n from CityTable)
where value ilike 'Parigi'
I have also tried the following
select * from CityTable where names_i18n ? 'Parigi';
But it seems to work only for the key part of the jsonb, is there any similar operator for the value part? I also need a way to know what name:XX has been found, not only the city name.
Anyone has a clue?
with CityTable (id_city, name_city, names_i18n) as (values(
7444, 'Paris',
'{"name:fr":"Paris","name:zh":"巴黎","name:it":"Parigi"}'::jsonb
))
select *
from CityTable, jsonb_each_text(names_i18n) jbet (key, value)
where value ilike 'Parigi'
;
id_city | name_city | names_i18n | key | value
---------+-----------+--------------------------------------------------------------+---------+--------
7444 | Paris | {"name:fr": "Paris", "name:it": "Parigi", "name:zh": "巴黎"} | name:it | Parigi
I have a table where a user inputs name, dob, etc. and I have a User_Name column that I want automatically populated from other columns.
For example input is: Name - John Doe, DOB - 01/01/1900
I want the User_Name column to be automatically populated with johndoe01011900 (I already have the algorithm to concatenate the column parts to achieve the desired result)
I just need to know how (SQL, Trigger) to have the User_Name column filled once the user completes imputing ALL target columns. What if the user skips around and does not input the data in order? Of course the columns that are needed are (not null).
This should do it:
you can use a calculated field with the following calculation:
LOWER(REPLACE(Name, ' ', ''))+CONVERT( VARCHAR(10), DateOfBirth, 112))
In the below sample I have used a temp table but this is the same for regular tables as well.
SAMPLE:
CREATE TABLE #temp(Name VARCHAR(100)
, DateOfBirth DATE
, CalcField AS LOWER(REPLACE(Name, ' ', ''))+CONVERT( VARCHAR(10), DateOfBirth, 112));
INSERT INTO #temp(Name
, DateOfBirth)
VALUES
('John Doe'
, '01/01/1900');
SELECT *
FROM #temp;
RESULT:
I have application with 2 columns viz. city, category. I want to fit this in hypertable.
There is also id which I would like to add as ROW.
create table ads (city, category);
insert into ads values ("1", "city:mumbai", "1");
insert into ads values ("1", "category:cars", "1");
insert into ads values ("2", "city:pune", "1");
insert into ads values ("2", "category:bikes", "1");
My question is how do I fetch rows where city = mumbai which should fetch 2 rows of ROW = 1.
So suppose I would make similar query in MySQL..
select * from ads where city = "mumbai";
I will get 1 row having category=cars, city=mumbai and id=1..How can same be achived in hypertable query?
Thanks.
it seems to be a current limitation as writed in:
http://hypertable.com/documentation/reference_manual/hql/#select
when you query for column you can get as results only the column values.
they say:
(these limitations will be removed in future versions of Hypertable)
Column Value Predicate
When specifying a column value predicate, the column family must be identical to the column family used in the SELECT clause, and exactly one column family must be selected. The following examples are valid:
SELECT col FROM test WHERE col = "foo";
SELECT col FROM test WHERE col =^ "prefix";
The following examples are not valid because they select more than one column family or because the column family in the select clause is different from the one in the predicate (these limitations will be removed in future versions of Hypertable):
SELECT * FROM test WHERE col = "foo";
SELECT col, col2 FROM test WHERE col =^ "prefix";
SELECT foo FROM test WHERE bar = "value";