I've imported an excel sheet into PowerBI and I'm trying to trying to transform the date column from text format (March 21st 2020, 00:00:000) into date format (21/03/2020).
Unfortunately when I select "detect type" it only selects text, and selecting "Date" presents an error. I've removed the ", 00:00:000" from the columns but still receiving an error message.
Does anyone have any suggestions on how I can transform this column into dd/mm/yyyy?
Many thanks,
After removing the , 00:00:000, you can use a transformation like one of these to remove the st part in 21st so that March 21st 2020 --> March 21 2020, which can be automatically converted to a date.
This transformation strips all characters except space and numbers after the first space:
each Text.BeforeDelimiter(_, " ") & Text.Select(_, {" ", "0".."9"})
This one deletes the two characters to the left of the last space:
each Text.ReplaceRange(_, Text.PositionOf(_, " ", Occurrence.Last)-2, 2, "")
The full query (using the first option) would look like this:
let
Source = <Your Data Source>,
Remove00 = Table.TransformColumns(Source, {{"DateText", each Text.BeforeDelimiter(_, ","), type text}}),
StripOrdinal = Table.TransformColumns(Remove00, {{"DateText", each Text.BeforeDelimiter(_," ") & Text.Select(_, {" ","0".."9"}), type text}}),
TextToDate = Table.TransformColumnTypes(StripOrdinal,{{"DateText", type date}})
in
TextToDate
You could use this PowerQuery Script:
let
Source = Excel.Workbook(File.Contents("Dates.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Changed Type" = Table.TransformColumnTypes(Sheet1_Sheet,{{"Column1", type text}}),
#"Split Column by Delimiter" = Table.SplitColumn(#"Changed Type", "Column1", Splitter.SplitTextByDelimiter(" ", QuoteStyle.Csv), {"Column1.1", "Column1.2", "Column1.3"}),
#"Added Month" = Table.AddColumn(#"Split Column by Delimiter", "Month", each if [Column1.1] = "January" then 1 else if [Column1.1] = "February" then 2 else if [Column1.1] = "March" then 3 else if [Column1.1] = "April" then 4 else if [Column1.1] = "May" then 5 else if [Column1.1] = "June" then 6 else if [Column1.1] = "July" then 7 else if [Column1.1] = "August" then 8 else if [Column1.1] = "September" then 9 else if [Column1.1] = "October" then 10 else if [Column1.1] = "November" then 11 else if [Column1.1] = "December" then 12 else 0),
#"Added Day" = Table.AddColumn(#"Added Month", "Day", each Text.Select([Column1.2], {"0".."9"})),
#"Removed Columns" = Table.RemoveColumns(#"Added Day",{"Column1.1", "Column1.2"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Column1.3", "Year"}}),
#"Changed Type2" = Table.TransformColumnTypes(#"Renamed Columns",{{"Year", Int64.Type}, {"Month", Int64.Type}, {"Day", Int64.Type}}),
#"Inserted Merged Column" = Table.AddColumn(#"Changed Type2", "Date", each Text.Combine({Text.From([Day], "de-DE"), "/", Text.From([Month], "de-DE"), "/", Text.From([Year], "de-DE")}), type text),
#"Changed Type3" = Table.TransformColumnTypes(#"Inserted Merged Column",{{"Date", type date}}),
#"Removed Columns2" = Table.RemoveColumns(#"Changed Type3",{"Year", "Month", "Day"})
in
#"Removed Columns2"
The conversion to your final data format is depending on your local settings and is usally done outside PowerQuery.
Related
So I have a table with StartDate and EndDate and I need to know if this dates overlap in a natural year. In example January 1st 2021 and December 31st 2021 and know how many days where in that period.
I have a formula that works in PowerBI that is:
Note: "DATA" is the name of the query.
OVERLAP days 2021 = MAX(MIN(DATE(2021,12,31),'DATA'[StartDate].[Date])-MAX(DATE(2021,1,1),'DATA'[EndDate].[Date])+1,0)
But now I need to translate it to PowerQuery.
Thanks!
How about
let Source = #table({"StartDate", "EndDate"},{{#date(2021,5,31),#date(2021,7,31)}}),
year=2021,
#"Added Custom" = Table.AddColumn(Source, "Overlap", each
if [EndDate]<#date(year,1,1) or [StartDate]>#date(year,12,31) or [EndDate]<[StartDate] then null else
Number.From(if [EndDate]<#date(year,1,1) then #date(year,1,1)
else if [EndDate]>#date(year,12,31) then #date(year,12,31)
else [EndDate]) -
Number.From(if [StartDate]<#date(year,1,1) then #date(year,1,1)
else if [StartDate]>#date(year,12,31) then #date(year,12,31)
else [StartDate]))
in #"Added Custom"
The basic algorithm is pretty much the same. But I confess I got a bit carried away with implementing it.
The problem is in how you want to define the years of interest, and how you want to display the result when there are multiple years.
To use Power Query
Select some cell in your Data Table
Data => Get&Transform => from Table/Range or from within sheet
When the PQ Editor opens: Home => Advanced Editor
Make note of the Table Name in Line 2
Paste the M Code below in place of what you see
Change the Table name in line 2 back to what was generated originally.
Read the comments and explore the Applied Steps to understand the algorithm
Here is a method using various M-Coding to develop
a list of all the years that are represented in the table
List.Generate to generate a list of days in any one Year
List.Accumulate to iterate through List.Generate for each year
Some other functions to create the table with the additional columns.
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Start Date", type date}, {"End Date", type date}}),
//add columns for each year
//create list of years for the table
Years =
let
firstYear = List.Transform(#"Changed Type"[Start Date], each Date.Year(_)),
lastYear = List.Transform(#"Changed Type"[End Date], each Date.Year(_))
in
List.Numbers(List.Min(firstYear), List.Max(lastYear) - List.Min(firstYear)+1),
//create lists of days count for each year, based on the start and end dates
newcols = List.Accumulate(Years, {}, (state, current) => state &
{let
ends = List.Transform(#"Changed Type"[End Date], each List.Min({#date(current,12,31), _})),
starts = List.Transform(#"Changed Type"[Start Date], each List.Max({#date(current-1,12,31), _})),
dys = List.Generate(
()=>[d=List.Max({0,Duration.Days(ends{0} - starts {0})}), idx=0],
each [idx] < List.Count(ends),
each [d=List.Max({0,Duration.Days(ends{[idx]+1} - starts {[idx]+1})}), idx=[idx]+1],
each [d]
)
in
dys}),
//add the new columns to original table
newColNames = List.Transform(Years, each "Days in " & Text.From(_)),
newTable = Table.FromColumns(
Table.ToColumns(#"Changed Type") & newcols,
Table.ColumnNames(#"Changed Type") & newColNames),
//Set data types
typed = Table.TransformColumnTypes(newTable, List.Transform(newColNames, each {_, Int64.Type}))
in
typed
I have two date fields
"MonthEndDate" and "date" I want a condition to satisfy that if date is less than 15 then take previous month end date and if date is more than 15 then take current month end date. Eg-
MonthEndDate
date
ResultDate
6/30/1999
7/2/1999
6/30/1999
10/31/1999
11/1/1999
10/31/1999
5/31/2009
5/28/2009
5/31/2009
You can achieve this using power query
let
Source = Excel.Workbook(File.Contents("C:\Ashok\Power BI\Stack Overflow\data_08_nov_21.xlsx"), null, true),
Data_Sheet = Source{[Item="Data",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Data_Sheet, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"MonthEndDate ", type date}, {" date ", type date}}),
#"Inserted Day" = Table.AddColumn(#"Changed Type", "Day", each Date.Day([#" date "]), Int64.Type),
#"Added Custom" = Table.AddColumn(#"Inserted Day", "Custom", each if[Day] < 15
then Date.EndOfMonth( Date.AddMonths([#" date "], -1))
else Date.EndOfMonth([#" date "])),
#"Renamed Columns" = Table.RenameColumns(#"Added Custom",{{"Custom", "ResultDate"}})
in
#"Renamed Columns"
I need to convert a date string to a DateType, but I've several challenges using to_date.
Formatting for day works well (1 or 2 digits), month is a Dutch abbreviation and doesn't work (works only if the abbreviation is equal to English), and year is 2 or 4 digits (missing centuries!).
What's the best way to convert these all to a DateType?
I couldn't find an option to set locale to NL using the formatting.
I created an UDF, but don't know if this is the best way to fix this.
The 19 for century is debatable.
Code:
#F.udf(T.StringType())
def convert_date(s):
month_dict = {"jan":"01", "feb":"02", "mrt":"03", "apr":"04", "mei":"05", "jun":"06", "jul":"07", "aug":"08", "sep":"09", "okt":"10", "nov":"11", "dec":"12" }
day, month, year = s.split("-")
if len(day) == 1:
day = '0' + day
if len(year) < 4:
year = '19' + year
date = day + "-" + month_dict[month] + "-" + year
return date
df = df.withColumn('DateOfBirth_new', F.to_date(convert_date(F.col("DateOfBirth"), "dd-M-yyyy"))
DateFrame:
df = spark.createDataFrame([
["2-feb-1966"],
["05-mei-1974"],
["3-mrt-83"],
["05-mrt-1983"],
["12-jun-75"]
]).toDF("DateOfBirth")
Here is a similar solution without a UDF, using a when expression for the month conversion.
month_conversion = F.expr("""CASE
WHEN (month = 'jan') THEN 01
WHEN (month = 'feb') THEN 02
WHEN (month = 'mrt') THEN 03
WHEN (month = 'apr') THEN 04
WHEN (month = 'mei') THEN 05
WHEN (month = 'jun') THEN 06
WHEN (month = 'jul') THEN 07
WHEN (month = 'aug') THEN 08
WHEN (month = 'sep') THEN 09
WHEN (month = 'okt') THEN 10
WHEN (month = 'nov') THEN 11
WHEN (month = 'dec') THEN 12
ELSE NULL END
""").alias("m")
day_conversion = F.when(F.length("day") == 1, F.concat(F.lit("0"), F.col("day"))).otherwise(F.col("day"))
year_conversion = F.when(F.length("year") < 4, F.concat(F.lit("19"), F.col("year"))).otherwise(F.col("year"))
(df.withColumn("split",
F.split("DateOfBirth", "-")
)
.withColumn("day",
F.col("split").getItem(0)
)
.withColumn("month",
F.col("split").getItem(1)
)
.withColumn("year",
F.col("split").getItem(2)
)
.select(
F.concat_ws("-",
day_conversion,
month_conversion,
year_conversion
).alias("DateOfBirth_new")
)
.show())
I have a google sheet where I need to get the next upcoming date based on the start date set in column A
Any pointers are greatly appreciated? I am unable exhibit the efforts as I am completely new to this sort of recurrence using Google sheets
https://docs.google.com/spreadsheets/d/1g_UNg4MjDy3gFufpjZtMRkbBz80K3scQdZOaiAnZi7I/edit#gid=0
This behavior (the next date from today including today) could be implemented manually by this formula:
={
"Next date from today";
ARRAYFORMULA(
IFS(
A2:A >= TODAY(),
A2:A,
B2:B = "Daily",
TODAY() + MOD(TODAY() - A2:A, C2:C),
B2:B = "Weekly",
TODAY() + MOD(TODAY() - A2:A, 7 * C2:C),
B2:B = "Monthly",
EDATE(A2:A, ROUNDUP((12 * (YEAR(TODAY()) - YEAR(A2:A)) + (MONTH(TODAY()) - MONTH(A2:A)) - IF(DAY(TODAY()) < DAY(A2:A), 1, 0)) / C2:C, 0) * C2:C),
True,
""
)
)
}
For additional options (like "every 2nd monday of the month" and others) additional options should be implemented in that IFS part.
If you are interested in a trivial case where the next date from the start date (column F:F on the screenshot) is needed, then the formula would be much simpler:
={
"Next date";
ARRAYFORMULA(
IFS(
B2:B = "Daily",
A2:A + C2:C,
B2:B = "Weekly",
A2:A + 7 * C2:C,
B2:B = "Monthly",
EDATE(A2:A, C2:C),
True,
""
)
)
}
Again, for additional options you'll need to add corresponding part to the IFS.
You could use IFS to check Frequency, and:
If Daily, add Count value to start date.
If Weekly, add Count value multiplied by 7.
If Monthly, since not all months have the same duration, retrieve the YEAR, MONTH and DAY indexes, add Count to the MONTH index, and set a new DATE, EDIT: or as suggested by kishkin, use EDATE.
It could be something like this:
=ARRAYFORMULA(IFNA(IFS(
B2:B = "Daily", A2:A + C2:C,
B2:B = "Weekly", A2:A + 7 * C2:C,
B2:B = "Monthly", EDATE(A2:A,C2:C)
)))
I have a Calendar program, that reside on the server, this program stores the dates into the database in the format :
YYYYMMDD
is there a "easy" way to manage these dates in asp? for example, let's assume we have the April 20th 2013 as a date:
20130420
we want to add 20 days to this date, so we want to produce:
20130510
any idea? is there a integerTOdate conversion in asp? or something i could use to add "days" to the number 20130420 ?
Something like this (untested):
Dim strDate, theDate, theDatePlusTwentyDays
Dim Year, Month, Day
' The date is stored as a string...
strDate = "20130420"
' Extract the components of the date
Year = Mid(strDate, 1, 4)
Month = Mid(strDate, 5, 2)
Day = Mid(strDate, 7, 2)
' Convert the components of the date into a datetime object
theDate = DateSerial(Year, Month, Day)
' Add 20 days using DateAdd
theDatePlusTwentyDays = DateAdd("d", 20, theDate)
Yes, you can use the DateAdd function
Response.Write(DateAdd("d",1,Now()))
You need to format your date first though into something like
<%
dim _dateOnServer
dim _formattedDate
dim _day
dim _month
dim _year
_dateOnServer = 20130420
_year = Left(_dateOnServer,4)
_month = Mid(_dateOnServer,5,2)
_day = Right(_dateOnServer,2)
_formattedDate = _month &"-"& _day &"-"& _year
dim _newDate
_newDate = DateAdd("d",20, _formattedDate )
_day = Left(_newDate,2)
_month = Mid(_newDate,4,2)
_year = Right(_newDate,4)
dim _newDateFormat
_newDateFormat = _year & _month & _day
%>
A solution that enables full control over order and format...
strDay = Day(Date)
strMonth = Month(Date)
strYear = Year(Date)
strHours = Hour(Now)
strMins = Minute(Now)
strSecs = Second(Now())
if len(strMonth) = 1 then
strMonth = "0" & strMonth
end if
if len(strDay) = 1 then
strDay = "0" & strDay
end if
if len(strHours) = 1 then
strHours = "0" & strHours
end if
if len(strMins) = 1 then
strMins = "0" & strMins
end if
if len(strSecs) = 1 then
strSecs = "0" & strSecs
end if
strDateAdded = strYear & "-" & strMonth & "-" & strDay
strDateAddedTime = strDateAdded & " " & strHours & ":" & strMins
Using this method you have complete control over the order and even when running your web app in different time zones, you still maintain DD/MM format... or whatever order you want such as MM-DD-YY (by reordering and trimming the year). Personally I prefer YYYY-MM-DD because sorting by ASC and DESC is a lot easier to work with, ie: easier to read because all rows will have the same number of characters like:
2013-04-01 03:15
2013-04-09 10:15
2013-04-22 07:15
2013-04-23 10:15
2013-04-23 10:60
2013-10-25 12:01
2013-10-25 12:59
Instead of:
2013-4-1 3:15
2013-4-9 10:15
2013-4-22 7:15
2013-4-23 10:15
2013-4-23 10:60
2013-10-25 12:1
2013-1-25 12:59