I have a Talend Job that currently does the following:
Input csv (tFileInputDelimited) --> tMap --> Output csv(tFileInputDelimited)
The goal of my job, is keep a value from the tMap, and use it to rename the output file.
I've tried to use a context and specify the row and the column I want to use, but it didn't work.
I'm a beginner, I use talend during an intership, I started 6 years ago, so I don't know many things ^^
Thank you for you future help !
You can use a tJavaRow to capture the value from the flow and assign it to the variable, the code will be like this :
// get the value of wanted_field of the id 40
if (input.id == 40) context.myvar = input.wanted_field
Your job will look like this:
Input csv (tFileInputDelimited) --> tJavaRow --> tMap --> Output csv(tFileInputDelimited)
Related
I've got a dataflow with a csv file as source. The column NewPositive is a string and it contains numbers formatted in European style with a dot as thousand seperator e.g 1.019 meaning 1019
If I use the function toInteger to convert my NewPositive column to an int via toInteger(NewPositive,'#.###','de'), I only get the thousand cipher e.g 1 for 1.019 and not the rest. Why? For testing I tried creating a constant column: toInteger('1.019','#.###','de') and it gives 1019 as expected. So why does the function not work for my column? The column is trimmed and if I compare the first value with equality function: equals('1.019',NewPositive) returns true.
Please note: I know it's very easy to create a workaround by toInteger(replace(NewPositive,'.','')), but I want to learn how to use the toInteger function with the locale and format parameters.
Here is sample data:
Dato;NewPositive
2021-08-20;1.234
2021-08-21;1.789
I was able to repro this and probably looks to be a bug to me . I have reported this to the ADF team , will let you know once I hear back from them . You already have a work around please go ahead that to unblock yourself .
I am new to using Talend.
The first table is fed into tHashOutput1. This part works fine.
The tHashOutput1 is fed into tHashOut2 and tHashOutput3.
From tMap_3 is fed from user . When i try to feed tHashInput3 into the tMap i am not allowed to do this. What is wrong with this.
Please make sure that tHashInput3have the schema of tHashOutput1, as you can see there is a small yellow warning on the tHashInput3 but it not exist on tHashInput2. Also, there is no output from the tMap3 to see if it works.
i've got the following data:
ID;NAME;SKILL
1;JOE;XML
1;JOE;JAVA
1;JOE;ORACLE
2;JOHN;JAVA
2;JOHN;API
I need a counter that will give me this structure:
ID;NAME;COUNTER;SKILL
1;JOE;1;XML
1;JOE;2;JAVA
1;JOE;3;ORACLE
2;JOHN;1;JAVA
2;JOHN;2;API
How can i achieve that in Talend? I tried to use a Number.sequence but i dont know how to get the dependency with the column ID. So every time a new id occures i need to reset the Sequence Number.
Any advice?
You can do it in following way.
yourInput---tJavaRow---tMap--youroutput
Create context variable named as oldID as int.\
In tJavaRow add following code.
if(!input_row.ID.equalsIgnoreCase(context.oldID)){
Numeric.resetSequence("i", 0);
context.oldID=input_row.ID;
}
Add tmap after tJavaRow and add additional column with name COUNTER
In COUNTER column add following code.
Numeric.sequence("i",1, 1);
Now execute your job will get expected output.
my output.
[statistics] connected
1|JOE|1|XML
1|JOE|2|JAVA
1|JOE|3|ORACLE
2|JOHN|1|JAVA
2|JOHN|2|API
[statistics] disconnected
There is a simpler way to do it : you can have a schema like input->tMap->output.
Then in tMap, just add a new Integer column "counter" and fill it with Numeric.sequence(row1.NAME,1,1)
It will automatically restart the sequence every time the input name changes.
I have one of jmeter User defined variable as a "comma separated value" - ${countries} = IN,US,CA,ALL .
(I was first trying to get it as a list/array - [IN,US,CA,ALL] )
I want to use the variable to test a web service - GET /${country}/info . IS it possible using ForEach controller or Loop controller ?
Only thing is that I want to save it or read it as IN,US,..,ALL and use it in the request path.
Thanks
The CSV should be as per the format mentioned in the image attached.
Refer to the link on how to use CSV in Jmeter: http://ivetetecedor.com/how-to-use-a-csv-file-with-jmeter/
Thread Group Settings
No. of threads: 1
Ramp-up period: 1
Loop Count: 4
Hope this will help.
CSV config is a red herring, you don't need it.
You can use a regular expression extractor to split up the variable into another variable (eg MyVar), using something like:
(.+?)[,\n]
This is trying to match each item before a , or newline. It will place the values in variables like MyVar_1, MyVar_2, etc. This is as close to an array as JMeter understands natively.
You can then loop on the contents of the matches using MyVar_matchNr, and MyVar_1 to MyVar_n (you will need to use __V() function to access the 'array' contents.
I have searched all over, and read this post.
But it doesn't seem complete and doesn't work.
The situation: I need to get the last modified file from a directory on the local machine. I then need to pass that file into the fileinputdelimited component.
I currently have:
tfilelist --> iterate --> titeratetoflow --> tsamplerow
-->tflowtoiterate -> tinpufiledelimited ---> tlogrow (just to make sure its pulling the right file)
But it doesn't work. I have configured it. so that titeratetoflow has a column called
"FileName" with "((String)globalMap.get("CURRENT_FILE"))" as the value,
"FileDirectory" with ((String)globalMap.get("CURRENT_FILEDIRECTORY")) as value, and
"FileAndDirectory" with ((String)globalMap.get("CURRENT_FILEPATH")) as value.
The tsamplerow is limited to "1".
The tiflowtoiterate is set so that
"FileNameOnly" is value of "FileName"
"FileDirectoryOnly" is "FileDirectory" and
"FilePathComplete" is "FileAndDirectory"
In the File location field of the tinputfiledelimited, I have "((String)globalMap.get("FilePathComplete"))"
When it runs I get an error saying cannot find file or path. If I cut out the fileinput component and have it send straight to the tlogrow, it shows a single line of blank entry.
Any ideas?
I'm not sure if you've just slightly misconfigured the job here but it seems to work fine for me.
Here's a few screenshots showing my job design:
The only thing I can think of just by looking at your post is that you might have slightly messed up the key value pair combinations in the tFlowToIterate. I tend to find that the default settings there work fine pretty much all of the time and it makes it a little more obvious what it's doing as well.
EDIT: Actually, it looks like you might be using the wrong values in your tIterateToFlow. The tFileList will throw the values for the file paths etc in to the global map but it will preface it with the unique component name. If you hit ctrl+space in the value window it should prompt you with a list of available values (these are also specified in the "Outline" tab of the studio). It typically makes an implicit conversion to String but for this you will need to explicitly convert it so use .toString() instead of (String).
Another way to get last modified file is as below
tFileList(sorted DESC by file modified date) ------> tFixedFlowInput (schema - filename, filenumber) ----->tHashOutput
here in tFixedFlowInput
filename = file(String)globalMap.get("tFileList_1_CURRENT_FILEPATH")+"/"+(String)globalMap.get("tFileList_1_CURRENT_FILE")
filenumber = (Integer)globalMap.get("tFileList_1_NB_FILE")
What above will accomplish is get list of all files in the directory with their number/rank - where the file last modified will have file number =1 and next to that will have 2...and so on.
Now on SubJobOK of above tFileList you can have tHashInput which will read from above tHashOutput and filter only row where filenumber==1 - which means the last modified file.
tHashInput (link to tHashoutput) ---->tFilterRow(filenumber==1)------>tLogRow
One reason why you are getting null is probably you have used globalMap.get("CURRENT_FILEPATH) instead of globalMap.get("tFileList_1_CURRENT_FILEPATH")
The Simple Solution for above problem could be as below:
tFileList(sorted ASC by file modified date)--> tIterateToFlow --> tJava( just to end the subjob).
Then on
subjob ok --> tfileinput ( use (String)globalMap.get("tFileList_1_CURRENT_FILE") or (String)globalMap.get("tFileList_1_CURRENT_FILEPATH") as a file name/file path)
Explanation:
Since tFileList iterates all the files in ASC order, it will always have Latest file name stored in globalMap for the last iteration. The list is only iterated till tIterateToFlow hence after this component (String)globalMap.get("tFileList_1_CURRENT_FILE") will always give the last file name from the iterated list, which is the latest file in out case.
Main Flow :
Component View: