My csv table looks like this:
class, duration, rooms
"Engine Drawing", 30, "eng010, eng201"
"Linear Systems", 30, "eng110"
"Linear Algebra", 24, "mat015, art603, hum202"
I'd like to flatten it out to this form:
class, duration, room
"Engine Drawing", 30, "eng010"
"Engine Drawing", 30, "eng201"
"Linear Systems", 30, "eng110"
"Linear Algebra", 24, "mat015"
"Linear Algebra", 24, "art603"
"Linear Algebra", 24, "hum202"
It's a simple enough transformation, duplicating row for each value in comma delimited field.
Is there a way to do this in Orange without resporting to code?
If you go to the .tab format, you can mark the last column as basket; see http://docs.orange.biolab.si/reference/rst/Orange.data.formats.html#baskets. In this way you will load the last column as a list of values, not a single value -- don't forget to remove the quotes.
But from the on you're on you own. There's no such flattening in Orange, you'll have to code it. It shouldn't be more than a few lines.
Related
I'm trying to use JSONB data type in PostgreSQL for my product table, like the following example:
Then, when I execute the below INSERT query for this table, the query is not executed !
INSERT INTO PRODUCT
VALUES
(7, 'iPhone8', 'Grey', 700.00, 34, '{"storage": 256, "RAM": 16, "resolution": 1700X456 }')
You forgot double quotes in value "1700X456" (it is a string, not a number).
INSERT INTO PRODUCT
VALUES
(7, 'iPhone8', 'Grey', 700.00, 34, '{"storage": 256, "RAM": 16, "resolution": "1700X456" }')
What I'm trying to achieve is inferring values to certain DataFrame columns taking into account values of each individual row.
.withColumn("date", when(col("date").isNull, lit(new DateTime(col("timestamp").as[Long]).getYear)))
The problem is that I can't wrap my head around how to retrieve, for each of the Row objects, its value for the given column. I've seen other solutions but they either list the whole set of values for all of the rows, or just get the first value of them, which isn't what I'm trying to achieve.
Image an example DF like this...
(year, val1, val2, val3, timestamp)
(null, 10, 12, null, 123456789)
(null, 11, 12, null, 234567897)
And what I want to see after applying individual functions (for example, extracting year from timestamp) to each of the Rows is...
(year, val1, val2, val3, timestamp)
(2018 [using DateTime class], 10, 12, 1012, 123456789)
(2018 [using DateTime class], 12, 12, 1212, 234567897)
Is there any way of doing this?
Thats where UDFs come into play :
val udf_extractYear = udf((ts:Long) => new DateTime(ts).getYear)
then you can use this using e.g.
df
.withColumn("year", when(col("year").isNull, udf_extractYear(col("timestamp"))).otherwise(col("year")))
.show()
As you can see your timestamp column is automatically mapped to Long
I have a problem regarding timestamps in MongoDB.
So here's the case:
I read in the json's and add timestamp to them:
my_json['insertTime'] = datetime.datetime.now()
mongodb.collection.insert_one(my_json)
will result in the DB like that:
"insertTime" : ISODate("2017-05-24T12:39:34.844Z")
After I read it then from the DB and try to write the same document into another mongoDB table I get the following error:
TypeError: datetime.datetime(2017, 5, 24, 12, 39, 46, 671000) is not JSON serializable
I have read solutions that will convert the datetime into a string value, but I'd like it to be in the ISODate format like in the first table.
That's how the timestamp looks like after getting it from table A:
'insertTime': datetime.datetime(2017, 5, 24, 12, 39, 46, 671000)
What should I do to insert it into the second table with same format (ISODate)?
PS: the way I load data into table B is the following:
tableB.insert_one(json.loads(json.dumps(docFromTableA)))
There is no need to pass through a JSON representation in order to save a document to MongoDB. Just do:
tableB.insert_one(docFromTableA)
MongoDB's native data format is not JSON, it is BSON, a binary structure with many more types than JSON. PyMongo converts MongoDB documents between BSON and Python dicts automatically.
I have a table master where there song details are stored against their ids.
I have another table playlist that is actually a user playlist and only ids from the first table are inserted selectively here.
So the first table holds data like:
1,2,3,4,5,6,7,8,9,10..........112,113........500 etc along with other fields.
The second table only have selective ids.
45, 7, 98, 83, 6, 65, 8, 234, 51, 78.
Now I am running the following query
SELECT * FROM master WHERE id IN (SELECT * FROM playlist)
but the result set returned comes sorted as
6, 7, 8, 45, 51, 65,78, 83, 98, 234 with their corresponding details.
How do I prevent this automatic sorting?
Added a column serialno to the table playlist.
Then used the following query.
select * from audiomaster a inner join playlist_top10 b on b.audioid = a.id order by b.serialno
The rows now are always sorted in the order they are meant to be.
Basicalyy i want to sum data by certain condition.These are example of my data :
Group 1 - Value:
10,
20,
30,
Group 2-Value:
10,
20,
30,
Group 3-Value:
10,
20,
30,
I want to sum group 1 and group 3 only.How can do it,like in sql we can write select sum(group) from table1 where group <>'Group 2'.How can we do that using formula.Or there's other way to do it.