I have a dataframe that contains a series of dates, e.g.:
0 2014-06-17
1 2014-05-05
2 2014-01-07
3 2014-06-29
4 2014-03-15
5 2014-06-06
7 2014-01-29
What I would like to do is convert these dates to integers by the month, since all the values are within the same year. So I would like to get something like this:
0 6
1 5
2 1
3 6
4 3
5 6
7 1
Is there a quick way to do this with Pandas?
EDIT: Answered by jezrael. Thanks so much!
Use function dt.month:
print (df)
Dates
0 2014-06-17
1 2014-05-05
2 2014-01-07
3 2014-06-29
4 2014-03-15
5 2014-06-06
6 2014-01-29
print (df.Dates.dt.month)
0 6
1 5
2 1
3 6
4 3
5 6
6 1
Name: Dates, dtype: int64
Related
I'm fairly new to PostgreSQL and have a situation where I need to flatten a text column to two separate columns.
Below is the query I'm using:
SELECT s.CoolerShelf,
s.ShelfPosition,
FROM planogram
CROSS JOIN LATERAL UNNEST(string_to_array(shelves, ','))
WITH ORDINALITY s(CoolerShelf,ShelfPosition)
The desired results should look like as follows:
coolershelf shelfposition
[["8d2cf35d-5708-45e0-9cb6-acad358e0f92" 0
"5a91f7a2-029a-46d7-8440-9337dd1b87d3" 1
"521562a9-9d33-438d-8156-1e6b1874ec8e" 2
"e14817e4-6630-4dca-a188-ac71060dcac9" 3
"76967052-ba9d-43f5-afd4-b4bbe1452d7e" 4
"2e5a6fb2-071e-426b-ac55-69f16baa0b42" 5
"108f263d-ee78-4124-a94b-2c5641f90321" 6
"0dbe5016-9e78-4173-b6e6-ff3e0199ca2e"] 7
["9bd83b79-186d-4ae5-9373-956dbd515070" 0
"b6172191-fa44-436d-879d-c883e4d240ed" 1
"093b72ba-74cd-48b9-86df-7e7d9341ae53" 2
"88b6c7f8-1d23-4e82-b959-8cb3400cc039" 3
"8279d979-8a57-4595-b9d3-346f6b05924e" 4
"735e6139-0fce-4bb7-a4a2-00ceb86c9b07" 5
"0ad84c4f-e0d8-4606-b563-8b2e32cc632f" 6
"5a86f7ea-0763-4473-ba09-91398e938be7"] 7
["62b2b9c6-1991-48f7-8533-76fa877e9736" 0
"35b56ed8-74f0-42f8-ab1c-ede41605b7bd" 1
"71848241-6348-4fde-935e-74a5c369ede1" 2
"722f05a6-5672-4be6-902d-635372e04758" 3
"b2a45221-aafb-4949-8018-5fed6cf7c7fe" 4
"dbb49783-5f75-4b3c-a793-a933ea321679" 5
"6bb25395-6647-4668-9e6e-158ad5f0b8af" 6
"1b32e613-8e72-420f-b31f-7bc95650386c"] 7
["636e2084-fdeb-4594-a400-10f6ef2791d7" 0
"8ac273ab-b8b2-46af-a8b4-f8fb22afe8e4" 1
"372e4f00-4ce9-4a9f-927b-d34c5a4968c1" 2
"f821abb1-d97e-4d99-b630-f74de5d106c1" 3
"d40b9b64-e81d-4133-bde2-54975806c087" 4
"07937692-680f-4cb0-8d17-98684141b92c" 5
"3b2039d0-de86-4cd7-9fb2-21397932f14c" 6
"16c24542-65c8-45db-97dc-014e66db7ef0"] 7
["f67efbcc-898d-4b50-8c15-21ac4fbbc500" 0
"64c020c7-9bd1-4e00-968f-180e3d68e100" 1
"3667915a-8e10-41fb-8f00-035cc10324a6" 2
"b7bc23c3-f5a1-486c-a99a-6c61357ed000" 3
"11292acd-ef71-4e0c-8281-0f50007cf850" 4
"210cca62-61b4-4ed9-ad42-653a909d3045" 5
"1dd2468a-0a3c-43e6-aae6-bd702d1c8a74" 6
"d7f8e5ee-1e05-42e1-8ff4-89529a210a76"] 7
["a1de7674-fa0b-49e6-af6a-2522798c4861" 0
"5cca7cd7-f50b-4538-ad89-a85b2d72a555" 1
"30cd353c-ee8c-4a94-9fbd-166372c2fd96" 2
"7407ab86-fdf6-4bf1-8282-e1218e021ed3" 3
"20ce7593-b1e2-4401-9b7c-1af18ec37f6c" 4
"541e995a-1416-4f4b-9696-2827cdcbd64e" 5
"3247610e-0486-4891-8fce-32f2e03fcaec" 6
"6492db47-54af-4390-9c88-11f43c3eaef0"]] 7
The numbering needs to start from the beginning after every open and closed parentheses. It's half working as expected but how can I accomplish this?
I have tried using the following function to derive a table consisting of 3 columns with one column data holding a list of an arbitrary schema.
fn:{
flip `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
};
which works well on input with multiple rows i.e.:
q)x:flip `a`b`c`d`e!(5#enlist 5?10)
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time data id
-----------------
8 8 5 2 8 6 8
5 8 5 2 8 6 5
2 8 5 2 8 6 2
8 8 5 2 8 6 8
6 8 5 2 8 6 6
However fails when using input with a single row i.e.
q)x:`a`b`c`d`e!5?10
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time data id
------------
8 7 7
8 8 7
8 4 7
8 4 7
8 6 7
which is obviously incorrect.
One might fix this by using enlist i.e.
q)x:enlist `a`b`c`d`e!5?10
q)fn[`time`data`id!(x`b;(x`a`b`c`d`e);x`a)]
time| 8
data| 7 8 4 4 6
id | 7
Which is correct, however if one were to apply this in the function i.e.
fn:{
flip enlist `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
};
...
time| 2 5 8 7 9
data| 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9 2 5 8 7 9
id | 2 5 8 7 9
Which has the wrong format of data values.
My question here is how might one avert this conversion issue and derive the same field values whether the argument is a multi row or single row table.
Or otherwise what is the canonical implementation of this in kdb+/q
Thanks
Edit:
To clarify: my problem isn't necessarily with the data input as one could just apply enlist if it is only one row. My question pertains to how one might use enlist in the fn function to make single row input conform to the logic seen when using multi row tables. i.e. how to replace fn enlist input with fn data (how to make the function input agnostic) Thanks
Are you meaning to flip the data perpendicular to the rest of the table? Your 5 row example works because there are 5 rows and 5 columns. The single row doesn't work due to 1 row to 5 columns.
Correct me if I'm wrong but I think this is what you want:
fn:{([]time:x`b;data:flip x`a`b`c`d`e;id:x`a)};
--------------------------------------------------
t1:flip `a`b`c`d`e!(5#enlist til 5);
a b c d e
---------
0 0 0 0 0
1 1 1 1 1
2 2 2 2 2
3 3 3 3 3
4 4 4 4 4
fn[t1]
time data id
-----------------
0 0 0 0 0 0 0
1 1 1 1 1 1 1
2 2 2 2 2 2 2
3 3 3 3 3 3 3
4 4 4 4 4 4 4
--------------------------------------------------
t2:enlist `a`b`c`d`e!til 5;
a b c d e
---------
0 1 2 3 4
fn[t2]
time data id
-----------------
1 0 1 2 3 4 0
Note without the flip you get this:
([]time:t1`b;data:t1`a`b`c`d`e;id:t1`a)
time data id
-----------------
0 0 1 2 3 4 0
1 0 1 2 3 4 1
2 0 1 2 3 4 2
3 0 1 2 3 4 3
4 0 1 2 3 4 4
In this case the time is no longer in line with the data but it works because of 5 row and cols.
Edit - I can't think of a better way to convert a dictionary to a table when needed other than using count first in a conditional. Note if the first key is a nested list this wouldn't work
{ $[1 = count first x;enlist x;x] } `a`b`c`d`e!til 5
Note, your provided function doesn't work with this:
{
flip `time`data`id!(x`b;(x`a`b`c`d`e);x`a)
}{$[1 = count first x;enlist x;x]} `a`b`c`d`e!til 5
I have a table
t: flip `ref`a`b`c`d`e!(til 10;10?10;10?10;10?10;10?10;10?10)
ref a b c d e
0 5 3 3 9 1
1 1 9 0 0 0
2 5 9 4 1 7
3 0 0 5 1 3
4 2 6 8 9 3
5 3 2 0 6 6
6 7 6 4 9 8
7 4 8 9 7 2
8 7 0 8 8 3
9 7 9 0 4 8
how can I set all values in columns a,b,c ,.. to 0Ni if their value equals the value in column ref without having to do a single line update for every columns?
So something taht would look a bit like (which returns ERROR:type)
{update x:?[x=t;0Ni;x] from t} each `a`b`c`....
The previous answer involves working with strings, this can often get very messy. To avoid this it is possible to build up the query by passing column names as symbols instead. The dictionary for the functional select can be built up using the following function:
q){y!enlist[({?[y=x;0Ni;y]};x)],/:y:(),y}[`ref;`a`b`c]
a| ({?[y=x;0Ni;x]};`ref) `a
b| ({?[y=x;0Ni;x]};`ref) `b
c| ({?[y=x;0Ni;x]};`ref) `c
The initial column is x and it allows to any number of columns to be passed as y for comparison.
This can then be added into the functional select:
q)![t;();0b;{y!enlist[({?[y=x;0Ni;y]};x)],/:y:(),y}[`ref;`a`b`c]]
ref a b c d e
-------------
0 4 5 8 4 8
1 2 6 9 1
2 8 4 7 2 9
3 0 1 2 7 5
4 5 3 0 4
5 8 3 1 6
6 5 7 4 9 6
7 2 8 2 2 1
8 2 7 1 8
9 6 1 8 8 5
You may reuse the next code snippet
t: flip `ref`a`b`c`d`e!(til 10;10?10;10?10;10?10;10?10;10?10);
columns: `a`b`c`d`e;
![t;();0b;columns!{parse "?[",x,"=ref;0Ni;",x,"]" }each string columns]
Where updated columns are put in columns list.
And functional update, which maps every column X to value ?[x=ref;0Ni;x], is used
I have a list like this:
a:1 1 1 1 2 3 1 1 4 4 4 5 6 4
How can I sum all of the consecutive identical numbers in a, so that it will become:
a:4 2 3 2 12 5 6 4
There are many ways - one method:
q) a:1 1 1 1 2 3 1 1 4 4 4 5 6 4
q) sum each where[differ a] _ a
4 2 3 2 12 5 6 4
Another method to achieve this using prev & <>:
sum each cut[where a<>prev[a]; a]
4 2 3 2 12 5 6 4
I have two different arrays. I want to make them similar. For example
If difference between coordinates of two array is 1 then it will make it similar otherwise not.It would be nice If anybody help me out.
Let
A =
1 5
2 6
3 7
4 8
5 9
6 1
7 2
and
B =
2 6
3 7
4 8
5 9
7 1
8 2
7 5
ismember() will give you all the indices which have difference equal to 1
ismember(B-A,1)
ans =
1 1
1 1
1 1
1 1
0 0
0 1
0 0
and then
>> B(ismember(B-A,1)) = A(ismember(B-A,1))
B =
1 5
2 6
3 7
4 8
7 1
8 1
7 5
as you can see replaces all the values in B which have difference equal to 1 as B