I'm trying to update a column from a varchar type column into a JSON but the column is already filled in with varchars how can I cast them to turn into a JSON with additional information.
Table structure with data:
-----+--------+
| id | value |
+-----+--------+
| 1 | value1 |
| 2 | value2 |
| 3 | value3 |
| 4 | value4 |
| 5 | value5 |
| 6 | value6 |
+-----+--------*
Expected Result:
+-----+----------------------------------+
| id | value |
+-----+----------------------------------+
| 1 | {"t1": "val", "value": "value1"} |
| 2 | {"t1": "val", "value": "value2"} |
| 3 | {"t1": "val", "value": "value3"} |
| 4 | {"t1": "val", "value": "value4"} |
| 5 | {"t1": "val", "value": "value5"} |
| 6 | {"t1": "val", "value": "value5"} |
+-----+----------------------------------*
Kindly help me to resolve this query
demo:db<>fiddle
When altering a column type you can add an expression to convert the data from the old type into the new one. You have to add the USING clause in order to do so:
ALTER TABLE mytable
ALTER COLUMN "value" TYPE json USING json_build_object('t1', 'val', 'value', "value");
In this case use the json_build_object() function to create the expected JSON object incl. the old value
Related
Basically I have a table called cities which looks like this:
+------+-----------+---------+----------+----------------+
| id | name | lat | lng | submitted_by |
|------+-----------+---------+----------+----------------|
| 1 | Pyongyang | 39.0392 | 125.7625 | 15 |
| 2 | Oslo | 59.9139 | 10.7522 | 8 |
| 3 | Hebron | 31.5326 | 35.0998 | 8 |
| 4 | Hebron | 31.5326 | 35.0998 | 10 |
| 5 | Paris | 48.8566 | 2.3522 | 12 |
| 6 | Hebron | 31.5326 | 35.0998 | 7 |
+------+-----------+---------+----------+----------------+
Desired result:
+-----------+---------+
| name | count |
|-----------+---------|
| Hebron | 3 |
| Pyongyang | 1 |
| Oslo | 1 |
| Paris | 1 |
| Total | 6 | <-- The tricky part
+-----------+---------+
In other words, what I need to do is SELECT the SUM of the COUNT in the query I'm currently using:
SELECT name, count(name)::int FROM cities GROUP BY name;
But apparently nested aggregated functions are not allowed in PostgreSQL. I'm guessing I need to use ROLLUP in some way but I can't seem to get it right.
Thanks for the help.
You need to UNION ALL the total sum.
WITH ROLLUP works by summing up the total for every group separate and can't be used here.
CREATE TABLE cities (
"id" INTEGER,
"name" VARCHAR(9),
"lat" FLOAT,
"lng" FLOAT,
"submitted_by" INTEGER
);
INSERT INTO cities
("id", "name", "lat", "lng", "submitted_by")
VALUES
('1', 'Pyongyang', '39.0392', '125.7625', '15'),
('2', 'Oslo', '59.9139', '10.7522', '8'),
('3', 'Hebron', '31.5326', '35.0998', '8'),
('4', 'Hebron', '31.5326', '35.0998', '10'),
('5', 'Paris', '48.8566', '2.3522', '12'),
('6', 'Hebron', '31.5326', '35.0998', '7');
SELECT name, COUNT(name)::int FROM cities GROUP BY name
UNION ALL
SELECT 'Total', COUNT(*) FROM cities
name | count
:-------- | ----:
Hebron | 3
Pyongyang | 1
Oslo | 1
Paris | 1
Total | 6
db<>fiddle here
I got a large DB with around 1mil entries. I'm using Pymongo to convert it to CSV. The problem is, some of the key/value pairs are missing or are placed in wrong order in the DB.
This results in CSV file fields being shifted.
DB
[
{
"_id": "123abc",
"device":
{
"Chanel": "1",
"Vendror": "undefined",
"first_seen": "1611606675",
"last_seen": "1612606695",
},
{
"Chanel": "1",
"first_seen": "1612342635",
"Vendror": "abc",
"last_seen": "1631606928",
},
{
"Chanel": "1",
"first_seen": "1612342635",
"Vendror": "abc"
},
}
]
TRIED
1. Hardcoding headers, compare to key and write value. If key not existent write value 'N/A'
with open(fn, 'w', encoding="utf-8") as wf:
header = ['_id', 'Chanel', 'Vendor', 'first_seen', 'last_seen']
for key in header:
write_file.write(key + ',')
write_file.write('\n')
#timeframe is a list of all documents within certain timeframe
for batch in timeframe:
for data_items in batch['device']:
for key, value in data_items.items():
if key not in header:
wf.write('NA')
else:
value = str(value)
value = value.replace(',', '-')
write_file.write(f'{value},')
wf.write('\n')
wf.write('\n')
return fn
2. Tried converting to json and use Pandas to convert json to CSV - same issue
EXPECTED OUTPUT
_id | Chanel | Vendor | first_seen | last_seen
------------------------------------------------------
123abc | 1 | undefined | 1611606675 | 1612606695
------------------------------------------------------
123abc | 1 | undefined | 1612342635 | 1631606928
------------------------------------------------------
123abc | 1 | undefined | 1612342635 | N/A
------------------------------------------------------
ACTUAL OUTPUT
_id | Chanel | Vendor | first_seen | last_seen
------------------------------------------------------
123abc | 1 | undefined | 1611606675 | 1612606695
------------------------------------------------------
123abc | 1 | 1612342635| undefined | 1631606928
------------------------------------------------------
123abc | 1 | 1612342635 | undefined |
------------------------------------------------------
There are three columns id : integer auto increment , col_jsonb: jsonb , date: timestamp.
I want to merge col_jsonb row values into json build object based on date,
the required output
Table:
+----+----------------+------------+
| id | col_jsonb | date |
+----+----------------+------------+
| 1 | {"Morning":10} | 2020-08-09 |
| 2 | {"Evening":20} | 2020-08-09 |
| 3 | {"Night":30} | 2020-08-09 |
| 4 | {"Morning":20} | 2020-08-10 |
+----+----------------+------------+
Expected o/p:
+----+----------------------------------------------+------------+
| id | col_jsonb | date |
+----+----------------------------------------------+------------+
| 1 | [{"Morning":10},{"Evening":20},{"Night":30}] | 2020-08-09 |
| 2 | {"Morning":20} | 2020-08-10 |
+----+----------------------------------------------+------------+
Try This query:
select
row_number() over (order by date_) as "id",
jsonb_agg(col_jsonb),
date_ as "Date"
from
example
group by
date_
row_number is added for numbering of rows if required
DEMO
I have a table like this in database
+---+-------------+--------------+
|id | service_name| doc_id |org_id|
+---+-------------+--------------+
| 1 | new service | 12 | 119 |
| | | | |
| 2 | new service | 24 | 119 |
| | | | |
| 3 | old service | 13 | 118 |
| | | | |
| 4 | old service | 14 | 118 |
| | | | |
| 5 | new service | 20 | 119 |
+---+-------------+--------------+
I want to group all the doc_id's according to service_name column
I have tried using
IN my controller
$where_person['org_id'] = $this->post('org_id');
$result_insert = $this->$model_name->fetch_doctor_services($where_person);
In my Model
function fetch_doctor_services($where){
$this->db->select('service_name,doc_id')->from('services');
$this->db->group_by('service_name');
$this->db->where($where);
return $this->db->get()->result();
}
But it does not output data as i desire, by grouping by service_name and all the doc_id's according to that service_name.
where am i going wrong here?
Currently my output is like this.
{ "data":
[ { "service_name": "new service", "doc_id": "12" },
{
"service_name": "old service", "doc_id": "13" }
]
}
You need to use GROUP_CONCAT. See below code on how to use it
$this->db->select('service_name, GROUP_CONCAT( doc_id) ')->from('services');
$this->db->group_by('service_name');
$this->db->where($where);
return $this->db->get()->result();
I have an aggregate table that needs to be updated frequently from a table that is generated by new data every few hours. Since
the aggregate table grows a lot every update, I need an efficient way to update the aggregate table.
Can someone please show me how to merge information from a new table to an aggregate table as shown below?
For example
val aggregate_table = Seq(("A", 10),("B", 20),("C", 30),("D", 40)).toDF("id", "total")
| id | total |
|-----+-------|
| "A" | 10 |
| "B" | 20 |
| "C" | 30 |
| "D" | 40 |
val new_info_table = Seq(("X"),("B"),("C"),("B"),("A"),("A"),("C"),("B")).toDF("id")
| id |
|-----|
| "X" |
| "B" |
| "C" |
| "B" |
| "A" |
| "A" |
| "C" |
| "B" |
Resulting aggregate_table after aggregate_table has been merged with
new_info_table
aggregate_table
| id | total |
|-----+-------|
| "A" | 12 |
| "B" | 23 |
| "C" | 33 |
| "D" | 40 |
| "X" | 1 |