Get the results of a flow into a global variable - talend

I have a flow in my Talend job that returns just one value (one row with one column), and I want to store this value in a global variable. What is the best way to do it? So far I have been using tFlowToIterate in this scenario, but this seems like a hack - because I really do not need to iterate. I'm wondering if there are better alternatives.

there is different solutions personnaly I also use the tFlowToIterate method but you can also use the tJava components or juste use the tSetGlobalVar component which is the component use in this case (one row).
Here is how to use it :
I hope this answers your question :)

Related

Reusing variable defined in one scenario in another

I assigned JSON response to a variable in the first scenario and I wanna use the same variable in other scenarios. Note that first scenario gets the response and assigns it to the variable defined in the background. Is it possible?
No. You have to do the assignment in the Background: itself in order for it to be re-usable across multiple scenarios.
Please take the time to read through the documentation and demo examples, really.

How to inject (dynamic?) Parameters in Tableau CustomSQL

I currently try to solve the following issue in Tableau:
In the end, I would like to have a Tableau dashboard where the user can select a Customer, and then can see the Customer's KPIs. Nothing spectacular so far.
To obtain a Customer's KPIs, there is a CustomSQL query with a parameter "CustomerName" (that returns the KPIs for that Customer).
Now the thing:
I don't want to have a hardcoded list of CustomerNames, as it would be possible with Tableau Parameters. Instead, the CustomerNames should be fetched from another datasource. I did not find a way to "link" a Parameter to a DataSource, and/or inject something other than static Parameters into CustomSQL.
My Question: Is there really no solution for this, or am I just doing something wrong (I hope so).
I found this workaround here https://www.interworks.com/de/blog/daustin/2015/12/17/dynamic-parameters-tableau that seems to work, but that looks like... a workaround.
Few background info:
I have to stick to using a CustomSQL because
It is not viable for me to calculate all KPIs for all CustomerNames
and then filter by Tableau, since the data amount is too big.
It is not viable to replace the CustomSQL with Tableau Calculations
and Filters (already tried that, ended up in having Tableau pulling
too much data instead of pushing the work to the database).
I cannot believe that Tableau does not offer a solution here, since the use case is pretty common I believe.
Do you have some input for me?
Thank you for your help in advance!
Kind Regards
have you tried using rawsql() functions together with stored functions on the database side? I found it pretty useful when needed to load single value from the dataset completely not related to currently used datasource.
For example, running foo stored function which accepts 2 dates and calculated sum of something, Syntax should be something like:
rawsql_int(your_db_schema.foo(%1,%2),[startDateFieldTableau],[endDateFieldTableau])
but you can access it directly:
rawsql_int("select sum(bar) from sales")
but this is bit risky.
Drawbacks:
it relies on the current connection (you create a calculated field (duh!)
it will not work with extract (but you are using custom sql anyways so I believe you are more into live connection

Handing Arrays in soap webservice testing using fitnesse

Is there a way to dynamically create tables in wiki?
Usecase : I'm trying to mimic similar to soap sonar in fitnesse. SOAP SOANR 1. Once we import the wsdl, soap sonar generates inputs for operations in wsdl. 2. Choose a operation, Enter input and then execute the operation. 3. In case of arrays, we can select size of array and enter values in respective array.
Fitnesse 1. I'm able to achieve point 1 using soapui jars. 2. This i'm able to achieve using xmlhttptest fixture
I'm stuck in 3rd point. Is there a way i can do this in fitnesse? (My idea is from point 1, i can get sample input for each operation, from which i will get to know that there are arrays/complex types present in input.xml but how do we represent this in wiki dynamically?
Thanks in advance
What I've done in the past is use ListFixture (and MapFixture) to dynamically fill a List (and Map/Hashes for each element's properties) and then use these as input values to a XmlHttpTest's feature to create the body to be sent using a FreeMarker template (which allows iteration over a list, which I use to create elements in the array based on the list).
But this gets quite complex quickly. Is that level of flexibility truly required? I found that quite often hard coding the number of elements in arrays/lists in the wiki is simpler to do and makes the test far easier to understand/maintain.
I most cases I prefer to create a script (or scenario) with the right number of elements for the test case(s) in with the request in the wiki page. The use of scenarios allows me to test with different values (but the same number of elements). Another element count gets its own script/scenario.
Being able to dynamically change the number of elements is only worthwhile if you need to test for many different counts, otherwise the added complexity of dynamically creating the body is just not worth it.

How to setup multi-level MDO formulation with openmdao 1.x?

As an openmdao 1.x problem handles only one driver and as optimizers are still supposed to be drivers, how a multi-level formulation could be implemented? Should I use two problems? Should I call an optimizer directly within a solve_nonlinear component method? Thanks.
There have been some changes, and Problem is no longer a system. The best way to do this now is to create a Component that contains the sub-problem, tells it when to run, and passes data in and out. See example here:
How to use nested problems in OpenMDAO 1.x?
The planned way to handle this is going to be to use nested Problem instances. This is not implemented yet in the problem class, but it very easy to implement by hand.
All you would need to do is define your own solve_nonlinear method in a SubClass of Problem. if you're going to use analytic derivatives you would also need to implement a Jacobian and apply_linear methods as well and do use post-optimiality sensitivities if you had nested optimizers. Or you could force finite difference to happen in the containing parent group.
Your solve_linear will take in params, unknowns, and resids dictionaries and passes the relevant variables down into the problems vector. Essentially, the framework was designed to not know if you're using nested problems. The top level framework thinks that the inner one is just a regular component.

Dynamic data range references for charts

I have an OpenOffice Calc spread sheet that I'm using to track some data. I have three charts made from the data. I periodically add more data to the spreadsheet. My current way to propagate this to the chart is to alter the data ranges manually of each chart. I'd like to automate this, or at least not have to redundantly change each chart separately.
My current idea was to do something like $A$1:$A{$F$1} for the ranges where $F$1 holds the current last line. Unfortunately, OpenOffice doesn't recognize this, but I thought there might be a function or work around for it. I haven't been able to find one yet.
So, is there a way to execute my idea, or perhaps a better way to do it?
There is a very similar question to this, but the asker asked for many more features and the answer was to use something other than a spreadsheet. It was never answered whether this specific feature was possible.
Also:
First method is to extend the range of the graph way down, with lots of empty space.
Second method is to include only one extra line of data in the graph and when you add data, always insert it above that line.