I am new to using Pentaho Spoon. I have about 100 text files in a folder, none of which have file extensions. I have found that if I create a job and move a file, one at a time, that I can simply rename that file, adding a .txt extension to the end. What I'd like to do is create a job that goes through and renames each file and adds the .txt extension. I've tried using the regex, but can't seem to get it to work because there's no file extension.
Any help would be greatly appreciated.
It's a pretty straightforward solution but you need to use a Transformation, as Job steps won't do it, ok?
You need the following steps:
Get File Names: just add your folder and the RegExp ".*" (without the double quotes), so everything is listed. Check if it's ok with "Show filename(s)..." button.
Modified Java Script Value: declare a new_filename var concatenating the desired extension. Remember to click "Get Variables" after adding the script to output the new field.
var new_filename = filename + '.txt';
Process Files: select Operation = Move and filename/new_filename as your source/target filenames.
That's it!
Renaming a group of files is one thing I wouldn't use Kettle for. Why not let the shell do what the shell does best?
rem example for Windows CMD shell
ren absolute-path-to-folder\*. *.txt
This can be done using a Shell job entry, if you find reason to do it in Kettle at all.
I've seen "just use a shell script" answers for this before. Works great if you can guarantee you're Kettle server is on the same OS as the developer workstation. I'm in an environment where the Dev/Spoon instance is Windows, but the Prod/Kettle environment is Linux, so you can't write one script file to rule them all.
As for "Why on earth would you do this?", my scenario is an integration scenario. We're using Pentaho for Data Integration, but a different tool for Enterprise Integration. I want a Pentaho Job to produce an output file, and I want my Enterprise Integration tool to pick up the file and do something with it, but not before Pentaho is done writing the file. Renaming helps avoid a race condition when the Enterprise Integration solution recognizes the file is there, but Pentaho isn't done writing it yet.
If I could rename a set of files, for example change from test..csv.processing to test..csv, then Pentaho would create the file initially with the .processing extension, and then remove the extension once it's done. The Enterprise Integration solution that's looking for test.*.csv won't start processing the file until Pentaho renames it. Bingo, no race condition.
Related
I am storing integration logs in CosmosDB which have a timestamp in the filename including a ":" character. I'm not able to open the files in VS code because of this (and maybe the "." character as well). I found a github issue talking about this, but that issue is closed and I'm still not able to open the files. Need some guidance on what I can do to open this files from inside VS code. Yes, I could change the filename but I'd prefer just to make it work. I can open fine in the Azure Portal.
Example file name: 2019-09-04T14:38:35.308Z
Error message: "ENOENT:no such file or directory"
It seems this is a limitation based on allowable Windows filenames, so the option is either to rename the file or accept that I cannot read this files within VS Code and use Azure Storage Explorer instead.
I'm happy to accept a different answer if anyone out there has any other options!
I'm using VS 2015, so the TFSCMDLets add-in for 2015. TFS server is 2013.
The old "New-TfsPendingChange -Rename" syntax that I found here on SO is no longer supported, both per the docs and per the error messages I get when I try.
I can't just check them out and rename them using the filesystem rename command, because that causes TFS to lose track of the file.
I've also tried this using tf.exe, where the commands I need are available, but it can't seem to figure out the workspace to use, even though the containing folder only maps to a single workspace. In this case it works fine when passed a literal filename, but fails when passing a path using foreach, like this:
tf workspaces /collection:devtfs\DeltaWA_ITA_BI
dir "C:\TFS\BusinessIntelligence\Database\Reporting_Prod\Test Views\" -filter "*.View.sql" | foreach { $newname = $.Name -replace ".View.sql", ".sql"; tf rename $ $newname }
Does anyone have a sample script that works in VS2015, TFS 2013?
You should avoid renaming items managed by TFVC using your operating system (for example, using Windows File Explorer, or the rename command in the Windows command prompt). Instead, do this in Source Control Explorer:
In Source Control Explorer, select the file that you want to edit,
open its shortcut menu, and choose Rename.
Type the desired name for the item.
If you want to use command prompt or script to do this, you could use tf rename command. Not sure the detail error message for your workspace error.
However, you need to make sure that you are running the commands from a mapped folder, you can run tf workfold to double check if the current folder is mapped or not.
If it's mapped and the error still exists, you might have a problem with your workspace cache. Try to remove cache through tf workspaces command:
tf workspaces /remove:(*|workspace1[,workspace2,...])
/collection:(*|TeamProjectCollectionUrl)
The overall answer turned out be in two parts:
1. You really must execute tf.exe in the folder where you want the work done. I saw this in the code I looked at, but didn't understand it was a requirement.
2. This means you have to get tf.exe into your path. Got help from a coworker on that.
After that, my tiny script worked as desired.
I still don't see a way to do this with TFS CMDLets. This is kind of a pity, bu not really important.
Thanks for the assistance!
We have a customer with their SFTP site, and I would like to copy files from specific folder, by using any automated process.
One of the example which I found, is winscp.net, but I have not managed how to use it, for my purpose.
http://www.itworld.com/article/2928599/windows/how-to-automate-sftp-file-transfers-in-microsoft-windows.html
QUESTION: All I will need is not run script and the file should be copied from their directory to my local folder. Is it possible at all?
I found the way of using PSFTP(Putty) to connect to the server, but how to make it automatically, do not know.
I think its a good solution for your problem.
You have to install winscp and the code some files and it will do automaticaly. I used 3 or 4 times. Also you need the key for your SFTP to connect throught it.
Here a link to the guide step by step.
https://winscp.net/eng/docs/guides
Here is a link to the scripting webpage
https://winscp.net/eng/docs/scripting
I recommend you to create an ini file to load all of the characteristics of your ftp connection and then execute an script over this.
It can be launched like that
WinSCP.com /ini=[your ini file] /script=[Your script file(what you want to do when it is connect)]
Hope this helps!.
I'm using code first migrations with my context class in a class library (ie not the startup project) and I want to make batch files for the common operations to save having to pass in the parameters each time I want to add-migration and update-database. I ran the "dir" command in the console and it appears to be in the solution root folder so I have tried creating a .bat,.cmd or .ps1 file in the Solution Items folder but the package manager powershell doesn't seem to be able to find it?
At this very moment I am happening to read this from Bruce Payette's "Powershell in Action" (Wonderful book) so share something with you, lucky guy:
"In this example (Poster: an example in the book), even though hello.ps1 is in the current directory, you had to put ./ in front of it to run it. This is because Powershell doesn't execute commands out of the current directory by default. This prevents accidental execution of the wrong command."
Looks like I needed to just put a ".\" on the beginning of the batch file name - not sure if Powershell requires this to execute?
So I have been seeing a lot of this command line stuff to install/update frameworks/modules over the internet but never actually tried to use it. I would just manually copy files.
I am starting to learn how to use Laravel and also trying to learn about all this command line stuff, I downloaded and installed composer and now this is where I'm stuck.
What do I do with a command line like this php artisan bundle:install bob ?? Do I have to copy it in a cmd.exe or copy it in the address bar of my browser.
I cannot find a tutorial which explains just the basic of the basics of using this and am pretty lost.
From the looks of it you're currently using Laravel 3.
I just want to let you know that Laravel 4 is way on it's way and I'd recommend that you have a look at it instead of L3. Here's a link.
As for your question: You need to define the path to your php.exe file.
Open up your start menu and write "Environment". Without the "".
In the first window you'll see a bunch of variables and values. Edit the "path" variable.
Go to the very end of the "value" field. And - if there isn't already one - type a ";". Without the "".
After that paste the path to your php.exe. Just the folder.
I.e:
C:\xampp\php
not
C:\xampp\php\php.exe
And then you're done.
You might have to restart your computer for this to apply though.