Azure Data Factory copy activity with json data - azure-data-factory

I am using a copy activity with request method as POST to copy the JSON response from a rest api. The rest api requires a body which contains date and time details in the below format
[
{
"start_date": "2023-02-07"
"start_time": "06:42:06"
"end_date": "2023-02-07"
"end_date": "08:47:06"
}
]
When am using the date and time as hardcoded values like the one shown above, it works fine. When I try to make it dynamic i.e. startdate as currentdate - 1 and start time as current time - 24hrs and end date as currentdate and end time as current time its failing.
In simple words I want to make the body of the api as dynamic
I tried the end date as "{#formatdatetime(utcnow(),'yyyy-MM-dd')}"
and end time as end time as "{#formatdatetime(utcnow(),'hh-MM-ss')}"
and few other combinations of the above code but its failing. Can someone please help me with this issue.

Since I don't have access to a REST API like yours, I have taken set variable activities to demonstrate how to create the required body using dynamic content.
I have used addDays() function to remove 1 day from both current date (1 day) and current time (24 hours and hence 1 day).
Then I have formatted the result using formatDateTime in the following way:
[ { "start_date": "#{formatDateTime(addDays(utcNow(),-1),'yyyy-MM-dd')}" "start_time": "#{formatDateTime(addDays(utcNow(),-1),'hh:mm:ss')}" "end_date": "#{formatDateTime(utcNow(),'yyyy-MM-dd')}" "end_date": "#{formatDateTime(utcNow(),'hh:mm:ss')}"} ]
The result would be as shown below:

Related

Power Query From Web dynamical DAX

I am courious about how to make M coding more dynamically when finding data. Currently i am trying to gather day-to-day stock data. I am Currently using this line of M-code to get the data
= Web.Page(Web.Contents("https://finance.yahoo.com/quote/MATAS.CO/history?period1=1372377600&period2=" & "1600214400" & "&interval=1d&filter=history&frequency=1d"))
The part "1600214400" is supposed to change each day. However, I tried different ways to make this dynamic. For instance, I used Advanced mode when using Get From Web. But that also failed. I also tried to use other queries, tables and output from functions. Does anyone have any idea or thoughts about to make this part dynamic?
I have enclosed a picture on how it looks on advanced.
I Googled this problem and some pictures illustrated the possibility of using parameters when using From Web. But I do not have that feature...
You can create a small function (see unixTimestampForStartOfToday in code below), which gives you a Unix timestamp in seconds for the start of "today" (where "today" is the date (per UTC) on which the code is running.
You can then use it as the value of the period2 query string parameter -- meaning period2 should always reflect the current date (per UTC).
let
unixTimestampForStartOfToday = () as number =>
// Should return the unix timestamp for today's date (as per UTC)
// at 00:00 hours.
let
todayFloored = DateTime.Date(DateTimeZone.FixedUtcNow()),
unixTimestampInSeconds = Duration.TotalSeconds(todayFloored - #date(1970, 1, 1))
in unixTimestampInSeconds,
Source = Web.Page(
Web.Contents("https://finance.yahoo.com/quote/MATAS.CO/history", [
Query = [
period1 = "1372377600",
period2 = Number.ToText(unixTimestampForStartOfToday()),
interval = "1d",
filter = "history",
frequency = "1d"
]
])
)
in
Source

Watson assistant putting the day of the week in a variable based on user input of #sys-date

I'm making a bot that can take dinner reservations. When it asks for the users reservation time, I'd like the bot to populate my variable $dayoftheweek with the day of the week based on the date the user inputs. How would I be able to do that?
here is my slot "If Found".
Basically I'd like to know what "value" I could put in to get the day of the week populated.
The easiest would be to use the new system entities (currently in beta). Enable the beta in your skills options. Then, in your dialog node, assign the following:
Check for: #sys-data.day_of_week
save as: mydayofweek
Note that the day names are lower case.
You can use the method reformatDateTime of the expession language to get the day from #sys-date.
For example setting $dayofweek to
<? #sys-date.reformatDateTime('u') ?>
will set $dayofweek to a value between "1" (Monday) and "7" (Sunday)
or setting $dayofweek to
<? #sys-date.reformatDateTime('E') ?>
will set $dayofweek to a value between "Mo" and "Su"
The value of #sys-date is stored as a string format "yyyy-MM-dd" e.g. "2016-11-21".
If you want to know what day of the week that date is, you can use the method reformatDateTime:
"<? #sys-date.reformatDateTime('EEEE') ?>"
Here all the posibles values of the method
So your node would look something like this.
Or in JSON format:
{
"context": {
"dayoftheweek": "<? #sys-date.reformatDateTime('EEEE') ?>"
}
}
And here a small test of what would be stored in $dayoftheweek:
Watson: Hello. How can I help you?
User: Day of the week test
Watson: Tell me a date
User: September 13, 2019
Watson: Friday
Here is the JSON Skill for the test.

Can not use expression language functions from Events Trigger body

I am using events trigger in ADF v2. My trigger parameter values look like this:-
"parameters": {
"FolderPath": "#triggerBody().folderPath",
"FileName": "#triggerBody().fileName",
"Year": "#{split(#triggerBody().folderPath,'/')[2]}",
"Month": "#{split(#triggerBody().folderPath,'/')[3]}",
"Day": "#{split(#triggerBody().folderPath,'/')[4]}",
}
I initially ran it with just 2 initial parameters and the sample values for FolderPath look like this as seen from ADF monitor after a run is succeeded:-
test/poc/2019/09/20/00
But when I add other parameters to it, like Year, Month, Day as shown here, while publishing the changes, I get an error 'Trigger activation failed':-
When I hard coded Year, Month, Day in the trigger, it again started working. I am guessing some issues in calling expression language functions from event based trigger. I am basically trying to dynamically calculate individual elements from FolderPath such as 2019 as Year, 09 as Month and so on. How to achieve this?
Pasting DraganB's comment as an answer here:-
Try
"Year": "#{split(triggerBody().folderPath,'/')[2]}", "Month": "#{split(triggerBody().folderPath,'/')[3]}", "Day": "#{split(triggerBody().folderPath,'/')[4]}"
Basically delete # before triggerBody

IBM Watson Assistant: user input time frame validation

I am using IBM Watson Assistant and try to validate user input against time frames / spans. What function should I use?
I was using #sys-date.before(now()) method to validate the time frame between past and present of a user input, but it does now work for me. If I input now or today Watson Assistant still thinks it is a past...
#sys-date.before(now()):
Looks "like you're trying to make a reservation in the past. Try again."
true $day it is
I expect if user inputs now or today to make reservations work, and if user inputs time in the past to response as
"like you're trying to make a reservation in the past. Try again."
The code snippet in question looks like this:
{ "context":
{ "day": "#sys-date" } }
{ "context":
{ "time": "<? #sys-time.reformatDateTime('h:mm a') ?>" } }
I solved it! What I did is I put (today) for here #sys-date.before(today()) and now for here #sys-time.before(now())
With IBM Watson Assistant you can use systen entities #sys-date and #sys-time to turn user input into data / time values. And you can use expression language functions for date and time to evaluate and transform those values. It seems that you are already using both. Now it is up to the logic...
There are functions before and after, but also sameMoment or sameOrAfter and more for comparisons. If the user inputs today, it is not before now because both would refer to the same day and date. Are you looking for the time value to be before the current timestamp or before the current date? You would need to compare time to time or date to date. You can reformat the values to snip off the time component and turn them into dates.
For more fancy validations you could even use java.util.Date functionality.
This should be true if it is a past moment:
now.after(yourInput)

Ember.js and Dates - showing as one day earlier than actual

This has been giving me trouble for a long time now, and I wouldn't think this would be so hard. I have a model with some dates, and date data coming from the API like so:
{
...
tollgate1: '2016-04-15',
tollgate2: '2017-01-01',
projectClose: '2016-10-21',
}
I created a format-date helper (which uses moment.js) to format the dates in view mode, like so:
And that's shows it correctly. However, when I switch to edit mode, the input elements are still referencing the same values, but now they all go one day earlier!
This has been maddening for some time. I've thought it might be due to a like of time zone information in the data, but since I can't change the data that's fed to my app, how can I get it to just display the date in the data, regardless of timezone? For example, with the Tollgate 1 date, I would want it to show April 15 no matter where the user is in the world.
Okay, so as with many things, this isn't an Ember thing so much as just a JavaScript thing. It's really hard learning both at once!
Since my dates coming down from the API don't have a time zone, they're assumed to be GMT, and so my EST timezone of -4 hours makes it show as the day before. Apparently moment.js has some built-in handling so that's why the format-date helper works fine.
What I did to solve is to just add computed properties on my model for each date, and create a new Date object by pulling out the parts from the input date, like so:
function convertDateToUtc(d) {
if(d) {
return new Date(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate());
} else {
return null;
}
}
export default DS.Model.extend({
...
tollgate1Date: Ember.computed('tollgate1', function() {
return convertDateToUtc(this.get('tollgate1'));
})
});