I am trying to create a job to insert all my data from Mysql to MongoDB
and this is my configuration :
how to auto generate date_created and last_updated?
so every data insert will auto fill date_created to new Date() or current date time and every data updated will auto update field last_updated with new Date() or current date.
You are looking for the Get System Info step.
It gives you a number of variables realted to the environement, including the time of the run. I usually use the system date (fixed), so that all the date-created are the same for the run, which ease retrieval.
For the last_update, it is that same technique, if you can setup your Match field for update in the MongoDB output step. If you cannot, have a look at the Merge Row (diff) : it tells you if the record is new, deleted, identical or changed.
Related
I already have a date time column which is updated every time I ingest new data and it works, but I would like to do the same for a new column but just with the date, not the time. Please see the picture attached. What should I write in the Default option? Table column definitions
You could set the default value to CURRENT_DATE ON UPDATE CURRENT_DATE in your EventDate field.
But wouldn't it be better to extract date from the already existing EventDateTime field with DATE_FORMAT(EventDateTime, '%Y-%m-%d') or DATE(EventDateTime) functions?
I am looking through the documentation for Google Cloud Spanner, and it looks like write operations return a timestamp when the row was actually written.
But when reading rows, it doesn't seem possible to re-capture that timestamp (either as a column that can be read or as a column that could be limited and sorted on).
I assume that I could just update the row after it is written to append a new column (created_at), but ideally it would be nice to have that field automatically appended.
Is there any way to access the original transaction timestamp when querying spanner? I also noticed that there was a CURRENT_TIMESTAMP() sql function. Is that equivalent to the transaction timestamp?
You can create commit timestamp columns, and Cloud Spanner writes the timestamp as part of the transaction:
https://cloud.google.com/spanner/docs/commit-timestamp
Currently, updating the timestamp column is the closest we can get.
CURRENT_TIMESTAMP() returns the current time.
See for more information:
https://cloud.google.com/spanner/docs/functions-and-operators#current_timestamp
I have the need to compare a date column with the max(date column) while making a filter selection.
E.g., when I compare [Date] = {max([Date])}, it finds the max/latest date in the entire data and compares. This gives me correct result when the latest month is included in the filter, but fails if I keep all months except the latest.
Is there a way in which the latest date can be searched in the subset of the data (based on filter selection)?
I am working with Redshift database (live connection).
look at the attached. https://www.dropbox.com/s/5zdkw9n003rxgvl/170524%20stack%20question.twbx?dl=0
{fixed : max(date)} will reflect only what is in the context filter.
I have a microservice and I use REST methods such as GET PUT POST, for this issue lets take an example POST !
I want to add a record to one table, but I need to only add two values and the remaining two are id and date. ID is auto increment and have used Sequence Generator, hence working even if I do not give an ID as input.
I would like to do the same for Date but would like to not input date plus make sure the date set for that record is the SYS Date in the Oracle DB, I have made Date variable default as SYSDate in DB but its still not taking in a null value as its not null. How do I implement this ?
Help is truly appreciated !
I have never used DBIx::Class until today, so I'm completely new at it.
I'm not sure if this is possible or not, but basically I have a table in my SQLite database that has a timestamp column in it. The default value for the timestamp column is "CURRENT_TIMESTAMP". SQLite stores this in the GMT timezone, but my server is in the CDT timeszone.
My SQLite query to get the timestamp in the correct timezone is this:
select datetime(timestamp, 'localtime') from mytable where id=1;
I am wondering if it is possible in my DBIx schema for "MyTable" to force it to apply the datetime function every time it is retrieving the "timestamp" field from the database?
In the cookbook it looks like it is possible to do this when using the ->search() function, but I am wondering if it's possible to make it so if I'm using search(), find(), all(), find_or_new(), or any function that will pull this column from the database, it will apply the datetime() SQLite function to it?
DBIx::Class seems to have great documentation - I think I'm just so new at it I'm not finding the right places/things to search for.
Thanks in advance!
I've used InflateColumn::DateTime in this way and with a timestamp, and I can confirm it works, but I wonder if you have this backward.
If your column is in UTC, mark the column UTC, and then it should be a UTC time when you load it. Then when you set_timezone on the DateTime (presumably that would be an output issue - it's at output that you care it's locally zoned) you can set it to local time and it will make the necessary adjustment.