I am currently trying to extract from a database A (Oracle) different ID to db B (MySQL).
I use Talend. However the purpose is to automate this process so that newcomers to database A can be added to db B automatically.
I would need to automate the job I have built. I have no idea how to do that using talend. Moreover, I would like to do the automation using a condition variable_date_db_A < 'today's date' however I don't know how to "access" today's date on talend ...
Thank you very much for your help
This can be used in a tMap or tJavaRow to get the current date:
TalendDate.formatDate("yyyy-MM-dd HH:mm:ss",TalendDate.getCurrentDate())
Regarding automation there are two basic methods that I can point out to you:
if you have the paid version then it comes with a job scheduler (Talend Administration Console)
otherwise in the free version (Talend Open Studio), you can compile an executable and run it in any scheduler (Windows scheduler for free, or any enterprise scheduler your organization may be utilizing).
I don't know how to "access" today's date on talend.
To access today's date in Talend you can use Java's Date method or by using methods/ routines given by Talend itself i.e. TalendDate.getCurrentDate();
If you want to format the date, then you can use TalendDate.formatDate("yyyy-MM-dd", TalendDate.getCurrentDate());
As you want to compare the dates, then it's better to store the date in Date Type format, so that you may use TalendDate.compareDate(Date date1, Date date2); inbuilt function.
As you want to insert only newly created data, then you can use inner join - reject in tmap. So that you will insert those data which is not present in the system.
I would need to automate the job I have built
If you are using Talend Enterprise version, then you can schedule it using TAC (Talend Administration Console).
If you are using Open Studio Community version, then create an executable build, by right-clicking on the job from the repository and build job and create `Standalone Job. And you can schedule it, using Windows scheduler or using cron in linux.
Related
I use Metabase to generate dashboards and reports. I need to generate files using the scheduler and instead of sending them by e-mail, I need to make them available in SFTP. Do you have any suggestion on how to automate this processes ?
I use PostgreSQL as a database source.
I can also try other open source tools if needed.
I didn't find much information on how to do it yet.
I need it to automate modification of SQL Server Scheduled jobs and want to reuse functionality of SSMS Scheduled job wizard.
Is it possible to popup SSMS Scheduled job wizard using PowerShell, use it and then close?
Update: I am aware of how to modify jobs using SPs and PS. The wizard is just one step in automated process. The next steps are to read the job settings and generate job script as per company's standard (that is important as out of the box Drop and Create script is not good enough), set encoding of the file etc.
You call a job via integrated MS SQL procedure sp_start_job. You call this SQL via Invoke-SqlCmd.
To modify/create the job you have to use sp_add_job, sp_add_jobstep, sp_update_job and sp_add_jobschedule.
I'm using Tableau Desktop 9.0 to create a visualization by extracting the data from DB2 database. But this database is updated everyday. That is, at the every time the data in the database changes. So is it possible for me to schedule a task of refreshing the extracted data source automatically at specific interval of time so that, after updating the report should reflect the results accordingly. Can this be done through Tableau Desktop automatically? The main thing is that it can easily be done on Tableau Server as known. But I cannot afford for a Server so I'm trying to get an answer is it possible with the Tableau Desktop or not.
Use the Tableau Data Extract Command-Line Utility in a batch script (like DOS/cmd or PowerShell) and schedule the batch script to run on the Windows Task Scheduler.
Assuming you're using Windows. Use Task Scheduler to schedule (a variation of) the following Powershell script on a daily basis.
C:\Program Files\Tableau\Tableau 9.0\bin>tableau refreshextract--server
https://blah_blah_server_name --username YourServerSignIn --password
YourServerPwd --datasource "Some_Table" --source-username YourDatabaseSignIn
--source-password YourDatabasePassword
I am currently creating reports with Eclipse embedded Report Designer for BIRT reports. In my next step, I would like to schedule this reports in such a way:
It will run once per day on a set time.
It will be produced as PDF.
It will be sent by email to a concrete email address.
I am working on Windows. Is it possible to do all of that by running bash script as a scheduled task in Windows Task Scheduler?
Thanks.
You can either do this yourself with the free BIRT runtime or download the free BIRT iHub F-Type server at http://www.actuate.com/resources/product-downloads/.
How to use the BIRT runtime:
Download the BIRT runtime and install it.
http://download.eclipse.org/birt/downloads/#runtime
Test the runtime installation (if you use Windows it will look like the following)
C:\birt\ReportEngine>echo %BIRT_HOME%
C:\birt
C:\birt\ReportEngine>genReport.bat
C:\birt\ReportEngine>ECHO off
Help for ReportRunner
--mode/-m [run|render|runrender] [options] [rptdesign|rptdocument]
The default mode is runrender.
To see options for run mode, use:
--help/-h run
To see options for render mode, use:
--help/-h render
To see options for runrender mode, use:
--help/-h runrender
Print current message, use --help/-h
Test the runtime in console with a job like
C:\birt\ReportEngine>genReport.bat -o test.pdf -f PDF new_report.rptdesign
Create a batch file for your job (the sample you ran earlier).
In Windows Task Scheduler, make a task to run the batch file for your job.
Run your task on demand to make sure it runs correctly.
Windows task scheduler does have a simple email function but you will probably have to find a better program to send the generated file as email. Then add the script to run this program to your batch file. Make sure you read up on how to configure a task for Windows task scheduler correctly.
It sounds like you need a way of
running the a BIRT report, through the BIRT reporting engine,
using scheduler to control execution of the report a on a particular schedule
specifying output format
and specifying how the output should be delivered in the scheduled job.
You could role your own or you could use a Reporting/BI server that already has all of those capabilities. Depending on your other needs, this may be overkill or exactly right. I am using SpagoBI for that exact purpose. SpagoBI supports BIRT as one of the reporting formats and engines. It uses the quartz scheduler, and does support emailing a PDF version of a report to a specific email address (or list). You might also look into iHub or other products.
http://www.spagobi.org/
http://birt.actuate.com/products/birt-ihub
In SpagoBI, you would deploy your BIRT report to the SpagoBI server from SpagoBI Studio. Then, in SpagoBI Server you would setup the parameters on the report document and create a schedule for the report. On the schedule, you would specify the parameters (if other than default), a schedule, output format, and how you want the output stored/sent.
I have some task/ program to run on an windows XP machine at some specific dates. The dates are stored inside a .dbf and the dates will be dynamic. How can I schedule the tasks on the specific dates?
I am planning to write another program (perhaps c# service) to run cmd with the command at to create tasks in task scheduler. Is there any ways to specify all the dates into one task? And will it be possible to link the task scheduler with a DB? Thanks!
I wrote a C# wrapper, that is script callable, that will work with the Task Scheduler on both XP and Windows 7. The download and source code is at https://taskscheduler.codeplex.com. With that wrapper, you can create a task that executes an action that has multiple triggers. Those triggers can be set for single days, weeks, months, etc. with different repetition patterns. There are many samples both in the documentation and discussion areas to help with your coding.