I have a data frame that I want to export to excel xlsx with a date/time stamp to the end of the file name.
I have the date time:
from datetime import datetime
date = datetime.now().strftime("%d-%m-%Y_%I:%M_%p")
datestr = date + 'xlsx'
df.to_excel(r"C:\somewhere\df + datestr")
any help would be great thank you !
This one should work (python 3.6+):
file_path = 'C:\\somewhere\\df'
date = datetime.now().strftime("%d-%m-%Y_%I:%M_%p")
df.to_excel(f'{file_path}{date}.xlsx')
Related
This is the original format of my date column when i import it. How do I change to it something like 30April2019
[
Thanks
Seems like your import options used datetime format, instead of date format.
Try declaring import format as DATE9. format
Your DATE column is a datetime value, which is different than a date value.
Use the format DTDATE9. so that the datetime value is rendered (represented) as dd-mon-yyyy when viewing. There are many other DT* formats available for rendering the value in other representations.
Example:
data have;
mydate = datetime(); * current date time;
mydate_raw_unformatted = mydate;
format mydate dtdate9.;
run;
proc print data=have;
run;
I have a dataframe in pyspark that has a timestamp string column in the following format:
"11/21/2018 07:21:49 PM"
This is in 24 hours format.
I want to filter the rows in the dataframe based on only the time portion of this string timestamp regardless of the date. For example I want to keep all rows that fall between the hours of 2:00pm and 4:00pm inclusive.
I tried the below to extract the HH:mm:ss and use the function between but it is not working.
# Grabbing only time portion from datetime column
import pyspark.sql.functions as F
time_format = "HH:mm:ss"
split_col = F.split(df['datetime'], ' ')
df = df.withColumn('Time', F.concat(split_col.getItem(1),F.lit(' '),split_col.getItem(2)))
df = df.withColumn('Timestamp', from_unixtime(unix_timestamp('Time', format=time_format)))
df.filter(F.col("Timestamp").between('14:00:00','16:00:00')).show()
Any ideas on how to filter rows only based on the HH:mm:ss portion in a timestamp column regardless of the actual date, would be very appreciated.
Format your timestamp to HH:mm:ss then filter using between clause.
Example:
df=spark.createDataFrame([("11/21/2018 07:21:49 PM",),("11/22/2018 04:21:49 PM",),("11/23/2018 12:21:49 PM",)],["ts"])
from pyspark.sql.functions import *
df.withColumn("tt",from_unixtime(unix_timestamp(col("ts"),"MM/dd/yyyy hh:mm:ss a"),"HH:mm:ss")).\
filter(col("tt").between("12:00","16:00")).\
show()
#+----------------------+--------+
#|ts |tt |
#+----------------------+--------+
#|11/23/2018 12:21:49 PM|12:21:49|
#+----------------------+--------+
Is there any way to get current timestamp in Date format in scala. I needed to create a date histogram and new Date() gives time in seconds and not in dd-mm-yyyy hh:mm format
Try to use the classes from the java.time package:
import java.time._
import java.time.format._
val format = DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss");
LocalDateTime.now().format(format)
Hello I would like to convert string date to date format:
for example from 190424 to 2019-01-24
I try with this code :
tx_wd_df = tx_wd_df.select(
'dateTransmission',
from_unixtime(unix_timestamp('dateTransmission', 'yymmdd')).alias('dateTransmissionDATE')
)
But I got this format : 2019-01-24 00:04:00
I would like only 2019-01-24
Any idea please?
Thanks
tx_wd_df.show(truncate=False)
You can simply use to_date(). This will discard the rest of the date, and pick up only the format that matches the input date format string.
import pyspark.sql.functions as F
date_column = "dateTransmission"
# MM because mm in Java Simple Date Format is minutes, and MM is months
date_format = "yyMMdd"
df = df.withColumn(date_column, F.to_date(F.col(date_column), date_format))
OK, I am trying to convert a date string from a format like:
2014-01-21 00:00:00
to
01/21/2014
I have tried many variations and am crashing and burning. The issue is that to test I have to create the script, export it in a process in Bonita (a BPM software), Import it and then create some cases. This all takes a long time.
Hopefully someone knows how to do this.
Also, is there a simple groovy editor out there? That would help me learn how to write groovy very quickly.
Groovy Dates have methods parse and format for converting to and from strings in various formats:
def format1 = '2014-01-21 00:00:00'
def format2 = Date.parse("yyyy-MM-dd hh:mm:ss", format1).format("dd/MM/yyyy")
assert format2 == '01/21/2014'
The format of the format strings are the same as Java's SimpleDateFormat.
String olddate='2014/01/21 00:00:00'
Date date = Date.parse("yyyy/MM/dd HH:mm:ss",olddate)
String newDate = date.format( 'MM/dd/yyyy' )
log.info newDate