How can I do breaks and subtotals in a report? - perl

I need to generate a business report using perl + Template Toookit and LaTeX.
Things are working really well, but I am struggling with the problem of having breaks (for example page breaks, or special headers) and subtotals whenever a field changes.
So, for example, every time the field "category" changes, I'd need to have a total of sales for that category, and a header showing that another category listing is starting; and then do the same when the field "group" - with the added interest that "group" is made up of categories, so the two things should nest.
I guess anyone that has built reports with Microsoft Access (or probably any other business reporting application) should be familiar with the problem.
Ideally this would be solved at a meta-level, so I don't have to rebuild the code every time, but only to specify what fields should generate breaks or subtotals.
I am (voluntarily) constrained to LaTeX and TT: LaTeX because of the control it gives over typography, and the possibility of generating custom graphics, and TT (or anything else that works in perl) because of learning curves.

There's no built-in subtotaling feature in TT, but you could possibly put your data into a Data::Table object, that would give you some ability to handle subtotaling at the 'meta' level, as you say.
Depending on the number of columns involved though, it might be just as simple to create local hashes to maintain running totals: NB: untested, example code only
[%-
MACRO printrow(rowtype, line) BLOCK;
# however you print the row as LaTeX
# rowtype is 'row', 'subtotal' or 'grandtotal' for formatting purposes
END;
SET sumcols = [ 'col3', 'col4', 'col5' ]; # cols to be accumulated
SET s_tot = {}; SET g_tot = {};
FOREACH i IN sumcols;
SET s_tot.$i = 0; # initialise
SET g_tot.$i = 0;
END;
FOREACH row IN data;
IF s_tot.col2 AND s_tot.col2 <> row.col2; # start of new group
printrow('subtotal', s_tot);
FOREACH i IN sumcols;
SET s_tot.$i = 0; #re-init
END;
END;
printrow('row', row);
SET s_tot.col2 = row.col2; # keep track of group level
FOREACH i IN sumcols;
SET s_tot.$i = s_tot.$i + row.$i;
SET g_tot.$i = g_tot.$i + row.$i;
END;
END;
printrow('grandtotal', g_tot);
-%]
Of course, if you have more than a couple of grouping levels, this can get quite messy. You could make s_tot an array of hashes to manage each level, to avoid hard-coding the levels. That's left as an exercise for the reader, as they say.

Related

How to create a variable in Matlab where certain subjects are coded as 1?

I want to create a variable called 'flag_artifact' where certain subjects from my dataset (for whom I know have bad quality images) are coded as e.g., 1. My dataset is stored in a table T with a certain number of rows and 'subject' is the 1st column in the table.
I managed to do it by creating a for loop. Surely there is a more efficient way to do this by perhaps directly creating the variable and using less lines of code? Could anyone give some advice?
Thank you very much! Here is what I have:
flag_artifact = {'T_300'}; %flagging subject number 300 for example
for i = 1:size(T,1)
if isequal(table2cell(T(i, 1)), flag_artifact)
T(i, 1) = {'1'};
end
end
However, when creating the variable flag_artifact = {'T_300'}, I would like it to include more than one subject. I tried using flag_artifact = {'T_300'}; {'T_301'}, as well as flag_artifact = {'T_300', 'T_301'} but it doesn't work because these subject identifiers do not get replaced with 1s.

Crystal reports select critera with comma delimited

I have a crystal report I need to change the select criteria on. Currently criteria compare's a database field to the parameter I created in the report.
{MaterialCR.MaterialId} = {?MaterialId}
I am now have a field that has comma delimited data in it I need to make sure the parameter includes any of the other ids in the new field.
Materialused has this data in it. "MA0161 ,MA0167" (No double quotes) . This doesn't work
{MaterialCR. MaterialUsed} = {?MaterialId}
I have tried to create a function to compare the two but it does not seem to work. It does not see the parameter as a string array.
My material match function that does not work
Function MaterialMatch (MaterialUsed as string,v1 () As String)
dim MyArray() as string
MyArray = Split (MaterialUsed,"," )
dim Match as boolean
Match = false
dim x as number
For x = 1 To count(v1) Step 1
IF "ALL" in v1 then
Match = true
x = count(MyArray)
end if
if MyArray(x) in v1 then
Match = true
x = count(MyArray)
end if
Next x
MaterialMatch = Match
End Function
This is what the data I am looking at looks like. We have many materials with a Material ID in it. We also have associated time that we need to select. It does not have a material id as it is a many to one situation. I need to retrieve all the records associated with the material including the time. Getting the material with ids is not the issue. I need to get the Time records also. I modified the view this report uses to include the material that overlaps the time. This is where I am stuck.
This is what my select expert formula looks like now. I do know the material used part is wrong.
(
{JobTimeMaterialCR.MaterialId} = {?MaterialId}
or
(
{JobTimeMaterialCR.Type} = "Time"
and
{JobTimeMaterialCR.MaterialUsed} = {?MaterialId}
))
I was able to write a formula that worked for me using the logic I described in my comment. Use this formula as your Record Selection Formula. In Formula Workshop these are found in Selection Formulas > Record Selection.
Local StringVar array values := Split({?Search Values},",");
Local NumberVar indexCount := Count(values);
Local BooleanVar found := false;
Local NumberVar counter;
For counter := 1 to indexCount Step 1 Do
(
If InStr({ARINVT.DESCRIP}, values[counter]) > 0 Then
found := true
);
found;
It's rough, but a good start. The search is case sensitive, so you may need to tweak it with some Lower() functions if you want case insensitive searches. Also, if there is a space between the delim character and the search string in your CSV string, then the space is included in the search. A Replace() function can help you get around this, but that would prevent you from using spaces in the search strings. If you need to use spaces in searches, then just take care when building your CSV String that there are no spaces before or after the comma that is your delim character.
If you need any help understanding the syntax of my formula feel free to comment and I will answer any questions.
I used a parameter field called {?Search Values} to simulate the CSV string data. {ARINVT.DESCRIP} is a field name from my test database I used to search thousands of records for key words I typed into my parameter field. You will want to replace these field names in the formula with your field names and you should be able to get this working without much trouble.

how to detect the end of (only) one file when merging SAS datasets

Question
To do something on the last observation in SAS, one can set an end -flag, e.g.
data myResult;
merge myInput1 (in=in1) myInput2 (in=in2) end=last;
by myKey;
** Do some stuff **;
if last then ** Do special stuff **;
run;
But how do I react on only one of my inputs reaching the end?
myInput1 (in=in1 end=end1) does not work.
Context
I want to compare the content of two large datasets and thus limit to 999 observations. If myInput1 contains some extra observations in the middle, my program reports that, but also some lines from myInput2 that were excluded by the obs=999 clause in myInput1. I actually would like to write:
data diff_&memName.;
format problem $20.;
merge myInput1(in=in1 end=end1 obs=999)
myInput2(in=in2 end=end2 obs=999)
by moederartikelnr;
select;
when (end1 or end2) delete;
when (not inFrom) problem = "only in 1_PROD";
when (not inTo) problem = "only in 2_DVLP";
when (not first.moederartikelnr or not last.moederartikelnr)
problem = 'duplicate key';
otherwise delete;
end;
run;
If you can help me in another way, please do so too.
Given your context, I think it might be easier to just stop the DATA step after 1000 iterations. On each iteration, one record is read from either dataset or both. So after 999 iterations, 999 records will have been read (not necessarily 999 from both). With this method, you do not need the OBS= option, so this will avoid false mis-matches. Something like:
data diff_&memName.;
if _n_=1000 then stop; *stop on 1000th iteration;
format problem $20.;
merge myInput1(in=in1)
myInput2(in=in2)
;
by moederartikelnr;
select;
when (not in1) problem = "only in myinput2";
when (not in2) problem = "only in myinput1";
when (not first.moederartikelnr or not last.moederartikelnr)
problem = 'duplicate key';
otherwise delete;
end;
run;

Assigning a whole DataStructure its nullind array

Some context before the question.
Imagine file FileA having around 50 fields of different types. Instead of all programs using the file, I tried having a service program, so the file could only be accessed by that service program. The programs calling the service would then receive a DataStructure based on the file structure, as an ExtName. I use SQL to recover the information, so, basically, the procedure would go like this :
Datastructure shared by service program :
D FileADS E DS ExtName(FileA) Qualified
Procedure called by programs :
P getFileADS B Export
D PI N
D PI_IDKey 9B 0 Const
D PO_DS LikeDS(FileADS)
D LocalDS E DS ExtName(FileA) Qualified
D NullInd S 5i 0 Array(50) <-- Since 50 fields in fileA
//Code
Clear LocalDS;
Clear PO_DS;
exec sql
SELECT *
INTO :LocalDS :nullind
FROM FileA
WHERE FileA.ID = :PI_IDKey;
If SqlCod <> 0;
Return *Off;
EndIf;
PO_DS = LocalDS;
Return *On;
P getFileADS E
So, that procedure will return a datastructure filled with a record from FileA if it finds it.
Now my question : Is there any way I can assign the %nullind(field) = *On without specifying EACH 50 fields of my file?
Something like a loop
i = 1;
DoW (i <= 50);
if nullind(i) = -1;
%nullind(datastructure.field) = *On;
endif;
i++;
EndDo;
Cause let's face it, it'd be a pain to look each fields of each file every time.
I know a simple chain(n) could do the trick
chain(n) PI_IDKey FileA FileADS;
but I really was looking to do it with SQL.
Thank you for your advices!
OS Version : 7.1
First, you'll be better off in the long run by eliminating SELECT * and supplying a SELECT list of the 50 field names.
Next, consider these two web pages -- Meaningful Names for Null Indicators and Embedded SQL and null indicators. The first shows an example of assigning names to each null indicator to match the associated field names. It's just a matter of declaring a based DS with names, based on the address of your null indicator array. The second points out how a null indicator array can be larger than needed, so future database changes won't affect results. (Bear in mind that the page shows a null array of 1000 elements, and the memory is actually relatively tiny even at that size. You can declare it smaller if you think it's necessary for some reason.)
You're creating a proc that you'll only write once. It's not worth saving the effort of listing the 50 fields. Maybe if you had many programs using this proc and you had to create the list each time it'd be a slight help to use SELECT *, but even then it's not a great idea.
A matching template DS for the 50 data fields can be defined in the /COPY member that will hold the proc prototype. The template DS will be available in any program that brings the proc prototype in. Any program that needs to call the proc can simply specify LIKEDS referencing the template to define its version in memory. The template DS should probably include the QUALIFIED keyword, and programs would then use their own DS names as the qualifying prefix. The null indicator array can be handled similarly.
However, it's not completely clear what your actual question is. You show an example loop and ask if it'll work, but you don't say if you had a problem with it. It's an array, so a loop can be used much like you show. But it depends on what you're actually trying to accomplish with it.
for old school rpg just include the nulls in the data structure populated with the select statement.
select col1, ifnull(col1), col2, ifnull(col2), etc. into :dsfilewithnull where f.id = :id;
for old school rpg that can't handle nulls remove them with the select statement.
select coalesce(col1,0), coalesce(col2,' '), coalesce(col3, :lowdate) into :dsfile where f.id = :id;
The second method would be easier to use in a legacy environment.
pass the key by value to the procedure so you can use it like a built in function.
One answer to your question would be to make the array part of a data structure, and assign *all'0' to the data structure.
dcl-ds nullIndDs;
nullInd Ind Dim(50);
end-ds;
nullIndDs = *all'0';
The answer by jmarkmurphy is an example of assigning all zeros to an array of indicators. For the example that you show in your question, you can do it this way:
D NullInd S 5i 0 dim(50)
/free
NullInd(*) = 1 ;
Nullind(*) = 0 ;
*inlr = *on ;
return ;
/end-free
That's a complete program that you can compile and test. Run it in debug and stop at the first statement. Display NullInd to see the initial value of its elements. Step through the first statement and display it again to see how the elements changed. Step through the next statement to see how things changed again.
As for "how to do it in SQL", that part doesn't make sense. SQL sets the values automatically when you FETCH a row. Other than that, the array is used by the host language (RPG in this case) to communicate values back to SQL. When a SQL statement runs, it again automatically uses whatever values were set. So, it either is used automatically by SQL for input or output, or is set by your host language statements. There is nothing useful that you can do 'in SQL' with that array.

Get all top level entry values from NotesView

What is the easiest way to get all top level entry values from a notes view? I have found the property "TopLevelEntryCount" returning the number of alle categories, but I want the values, too.
To iterate the view entries and column values need too much time.
Set ecl = view.Allentries
Set ve = ecl.Getfirstentry()
While Not(ve Is Nothing)
If IsArray(ve.Columnvalues(0)) Then
If flag = "" Then
arr = ve.Columnvalues(0)
Else
arr = ArrayUnique(ArrayAppend(arr, ve.Columnvalues(0)))
End If
Else
'error if arr is not already an array
arr = ArrayUnique(ArrayAppend(arr, ve.Columnvalues(0)))
End If
flag = "1"
Set ve = ecl.Getnextentry(ve)
Wend
Anyone know a faster way? It must not be written in LotusScript, actually I would prefer Java, JS or SSJS.
Idea 1: You can use the NotesViewNavigator class, and call the getFirst() and getNextCategory() methods. This should be faster than walking all the entries.
Idea 2: You can use NotesSession.Evaluate() with a formula that does an #Unique on #DbColumn for the first column of view. That should bring you back an array, and you can get the ubound of the array. Formulas tend to be very fast, but evaluate() has to compile them first, so I don't know if this will be faster or not. The disadvantage of this approach is that for very large views, this could exceed formula language's size limits. But if it does prove to be faster, you could catch the exception and fall-back to the slower method of iterating.
Richard's response is perfect!
For a java version it's almost the same:
see the Domino help for getNextCategory the given example below ViewNavigator really answer your need: link here
The evaluate method is also available in java Session