OLAP/MDX - define calculated member, sum all time to date data - postgresql

I would like to define "all time to date" calculated member in OLAP cube. I'm able to calculate YTD by using the following:
SUM(YTD([Time].[Month].CurrentMember), [Measures].[Suits])
How can I include all dates since the beginning of my data? My time dimension looks like:
<Dimension type="TimeDimension" visible="true" foreignKey="granularity" highCardinality="false" name="Time">
<Hierarchy name="Time" visible="true" hasAll="true" primaryKey="eom_date">
<Table name="v_months" schema="bizdata">
</Table>
<Level name="Year" visible="true" column="year_number" type="String" uniqueMembers="false" levelType="TimeYears" hideMemberIf="Never">
</Level>
<Level name="Quarter" visible="true" column="quarter_number" type="String" uniqueMembers="false" levelType="TimeQuarters" hideMemberIf="Never">
</Level>
<Level name="Month" visible="true" column="month_number" type="String" uniqueMembers="false" levelType="TimeMonths" hideMemberIf="Never">
</Level>
</Hierarchy>
</Dimension>
Not sure if relevant: I'm using mondrian olap server (running on tomcat), Saiku as frontend, postgres as database
I've tried a lot of combinations, but I can't figure it out.
Update: I've tried to use syntax suggested by Gonsalu:
<CalculatedMember name="YTD Suits" formatString="" formula="SUM(YTD([Time].[Month].CurrentMember), [Measures].[Suits])" dimension="Measures" visible="true">
</CalculatedMember>
<CalculatedMember name="PTD Suits" formatString="" formula="Sum({NULL:[Time].[Month].CurrentMember },[Measures].[Suits])" dimension="Measures" visible="true">
</CalculatedMember>
Using this I get the following error message when starting mondrian (note that YTD function works well without the second calculated member):
Caused by: mondrian.olap.MondrianException: Mondrian Error:Failed to parse query
'WITH
MEMBER [Measures].[Measures].[YTD Suits]
AS 'SUM(YTD([Time].[Month].CurrentMember), [Measures].[Suits])',
[$member_scope] = 'CUBE',
MEMBER_ORDINAL = 6
MEMBER [Measures].[Measures].[PTD Suits]
AS 'Sum({NULL:[Time].[Month].CurrentMember },[Measures].[Suits])',
[$member_scope] = 'CUBE',
MEMBER_ORDINAL = 7
SELECT FROM [Project Performance]'
Thank you for any ideas.

I haven't used Mondrian, but in SQL Server Analysis Services (SSAS), using the NULL member causes the range to go from one end of the level to the specified member.
In your case, the calculated member you're looking for might be something like this:
Sum( { NULL : [Time].[Month].CurrentMember }
, [Measures].[Suits]
)
You could also do a to the end of times calculated member using the NULL member on the other end, like so:
{ [Time].[Month].CurrentMember : NULL }

You can use PeriodsToDate function along with the allMember.
In your case it would be:
PeriodsToDate([Time.Time].[all_Time_member_name],[Time.Time].CurrentMember)

Related

Postgres XML parsing - Calculate the sum of multiple nodes of the same name using XPath

I have an xml snippet as below:
<OpCodeLaborInfo JobStatus="F" UpSellFlag="N" JobNo="1" OpCode="02TTZ10K" OpCodeDesc="10K SERVICE">
<TechInfo ActualHrsWorked="2.50" CustTechRate="27.00" TechHrs="0.00" TechName="JEFF SELLERS" TechNo="4816" />
<TechInfo ActualHrsWorked="0.00" CustTechRate="27.00" TechHrs="0.70" TechName="JEFF SELLERS" TechNo="4816" />
<BillTimeRateHrs BillRate="129.97" />
<CCCStmts Correction="PERFORMED 10K SERVICE" Complaint="LUBE OIL FILTER CHANGE, TIRE ROTATION, PERFORM MULTI POINT" />
<CCCStmts Correction="X" Complaint="INSPECTION, INSPECT FILTERS AND RECOMMEND, INSPECT BRAKES," />
<CCCStmts Complaint="BELTS AND HOSES" />
<RoAmts DlrCost="18.90" PayType="Cust" AmtType="Job" TotalAmt="59.12" />
</OpCodeLaborInfo>
<OpCodeLaborInfo JobStatus="F" UpSellFlag="N" JobNo="2" OpCode="02TTZ10K" OpCodeDesc="10K SERVICE">
<TechInfo ActualHrsWorked="2.50" CustTechRate="27.00" TechHrs="1.00" TechName="JEFF SELLERS" TechNo="4816" />
<TechInfo ActualHrsWorked="0.00" CustTechRate="27.00" TechHrs="0.00" TechName="JEFF SELLERS" TechNo="4816" />
<BillTimeRateHrs BillRate="129.97" />
<CCCStmts Correction="PERFORMED 10K SERVICE" Complaint="LUBE OIL FILTER CHANGE, TIRE ROTATION, PERFORM MULTI POINT" />
<CCCStmts Correction="X" Complaint="INSPECTION, INSPECT FILTERS AND RECOMMEND, INSPECT BRAKES," />
<CCCStmts Complaint="BELTS AND HOSES" />
<RoAmts DlrCost="18.90" PayType="Cust" AmtType="Job" TotalAmt="59.12" />
</OpCodeLaborInfo>
I need to calculate the sum of the TechInfo/#TechHrs for each OpCodeLaborInfo. I tried the following:
unnest(xpath('sum(//dns:RepairOrder/dns:RoRecord/dns:Rolabor/dns:OpCodeLaborInfo/dns:TechInfo/#TechHrs[1])'::text,
data_detail.ro_data_xml,
ARRAY[ARRAY['dns'::text, 'http://www.starstandards.org/STAR'::text]]))::text::numeric AS lbrsoldhours
but this seems to return the sum of the Tech Hours inside both the OpCodeLaborInfo nodes. Could someone be able to tell me how I can tweak the xpath so as to get the desired result.
So basically I need :
Job
Tech Hrs
1
sum(0.00+0.70)
2
sum(1.00+0.00)
I would solve this using xmltable()
select d.job, sum(t.hours)
from data_detail d
cross join xmltable (
'/OpCodeLaborInfo/TechInfo'
passing d.ro_data_xml
columns hours numeric path '#TechHrs') as t
group by d.job;
Online example
The XPath is probably not correct as the XPath you have shown doesn't match your sample XML data. Your sample XML also doesn't contain a namespace, so I am not sure why you are passing one to xpath()
But if you need one, you can use something like this:
cross join xmltable (
xmlnamespaces ('http://www.starstandards.org/STAR' as dns),
'/dns:OpCodeLaborInfo/dns:TechInfo'
passing d.ro_data_xml
columns hours numeric path '#TechHrs') as t
xpath can also work fine:
SELECT job
, SUM((xpath('//#TechHrs', element))[1]::text::decimal)
FROM data_detail
, LATERAL (SELECT unnest(xpath('/OpCodeLaborInfo/TechInfo', ro_data_xml))) u(element)
GROUP BY job;
fiddle
(based on the fiddle of #a_horse_with_no_name)

Liquibase: how to load data with CURRENT_TIMESTAMP?

I am trying to update my project, going from
<liquibase.version>3.5.5</liquibase.version>
<liquibase-hibernate5.version>3.6</liquibase-hibernate5.version>
to
<liquibase.version>3.9.0</liquibase.version>
<liquibase-hibernate5.version>3.8</liquibase-hibernate5.version>
and I'm having some troubles while loading data from CSV files, which contain some columns having the current timestamp as value.
E.g my CSV file:
id;name;created;modified
1;Book A;now();now()
2;Book B;now();now()
The Book table is created with:
<createTable tableName="book">
<column name="id" type="bigint" autoIncrement="${autoIncrement}">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(255)">
<constraints nullable="false"/>
</column>
<column name="created" type="timestamp">
<constraints nullable="false"/>
</column>
<column name="modified" type="timestamp">
<constraints nullable="false"/>
</column>
</createTable>
And the data is loaded with:
<property name="now" value="current_timestamp" dbms="postgresql"/>
<changeSet id="20180508144233-1" author="developer">
<loadData catalogName="public"
encoding="UTF-8"
file="config/liquibase/books.csv"
schemaName="public"
separator=";"
quotchar="'"
tableName="book">
</loadData>
</changeSet>
With the previous version of Liquibase it was working fine, however after the update I am getting the following error:
2020-06-26 16:49:57 [project-Executor-1] [ERROR] liquibase.changelog.ChangeSet - Change Set config/liquibase/changelog/20180508144233_added_books_data.xml::20180508144233-1::developer failed. Error: liquibase.exception.DateParseException: Improper value in 'NOW' value: now(). 'NOW' must be followed by + or -, then numeric offset, then units (h{our{s}}, m{inute{s}}, d{ay{s}}, or y{ears}
Hibernate: select answerweig0_.id as id1_1_, answerweig0_.likelihood as likeliho2_1_, answerweig0_.question_type as question3_1_, answerweig0_.weight as weight4_1_ from answer_weight answerweig0_
2020-06-26 16:49:57 [project-Executor-1] [ERROR] i.g.j.c.l.AsyncSpringLiquibase - Liquibase could not start correctly, your database is NOT ready: Migration failed for change set config/liquibase/changelog/20180508144233_added_books_data.xml::20180508144233-1::developer:
Reason: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DateParseException: Improper value in 'NOW' value: now(). 'NOW' must be followed by + or -, then numeric offset, then units (h{our{s}}, m{inute{s}}, d{ay{s}}, or y{ears}
liquibase.exception.MigrationFailedException: Migration failed for change set config/liquibase/changelog/20180508144233_added_books_data.xml::20180508144233-1::developer:
Reason: liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.DateParseException: Improper value in 'NOW' value: now(). 'NOW' must be followed by + or -, then numeric offset, then units (h{our{s}}, m{inute{s}}, d{ay{s}}, or y{ears}
at liquibase.changelog.ChangeSet.execute(ChangeSet.java:646)
at liquibase.changelog.visitor.UpdateVisitor.visit(UpdateVisitor.java:53)
at liquibase.changelog.ChangeLogIterator.run(ChangeLogIterator.java:83)
at liquibase.Liquibase.update(Liquibase.java:202)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.integration.spring.SpringLiquibase.performUpdate(SpringLiquibase.java:366)
at liquibase.integration.spring.SpringLiquibase.afterPropertiesSet(SpringLiquibase.java:314)
at org.springframework.boot.autoconfigure.liquibase.DataSourceClosingSpringLiquibase.afterPropertiesSet(DataSourceClosingSpringLiquibase.java:46)
at io.github.jhipster.config.liquibase.AsyncSpringLiquibase.initDb(AsyncSpringLiquibase.java:118)
at io.github.jhipster.config.liquibase.AsyncSpringLiquibase.lambda$afterPropertiesSet$0(AsyncSpringLiquibase.java:93)
at io.github.jhipster.async.ExceptionHandlingAsyncTaskExecutor.lambda$createWrappedRunnable$1(ExceptionHandlingAsyncTaskExecutor.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
As a workaround, I was able to set the values for such columns by using a default value in the table creation changelog:
<createTable tableName="book">
<column name="id" type="bigint" autoIncrement="${autoIncrement}">
<constraints primaryKey="true" nullable="false"/>
</column>
<column name="name" type="varchar(255)">
<constraints nullable="false"/>
</column>
<column name="created" type="timestamp" defaultValueComputed="CURRENT_TIMESTAMP">
<constraints nullable="false"/>
</column>
<column name="modified" type="timestamp" defaultValueComputed="CURRENT_TIMESTAMP">
<constraints nullable="false"/>
</column>
</createTable>
and removing the corresponding columns from the CSV file:
id;name
1;Book A
2;Book B
However, I'm still looking for a way to keep the CURRENT_TIMESTAMP value in the CSV file.
Your workaround is pretty much correct. But depending on database you will have to add a way that your update_timestamp or modified timestamp will change based on every addition, subtraction or edit you have in this database row. Here is how it can be setup in PostgresSQL for example.
<changeSet id="{UNIQUE_ID}" author="{YOUR_NAME}">
<preConditions onFail="MARK_RAN">
<dbms type="postgresql"/>
</preConditions>
<createProcedure>
CREATE OR REPLACE FUNCTION refresh_updated_date_column()
RETURNS TRIGGER AS $$
BEGIN
NEW.updated_date = now();
RETURN NEW;
END;
$$ language 'plpgsql';
</createProcedure>
<rollback>
DROP FUNCTION IF EXISTS refresh_updated_date_column;
</rollback>
</changeSet>`
This will be an additional changeset that you will add after your original changeset of creating the table and adding the required columns.
You need to add CURRENT_TIMESTAMP in your csv file like:
"id","date_created"
"11236,CURRENT_TIMESTAMP
And your changeset like:
<column header="date_modified"
name="date_modified"
type="DATE"/>
Note: Im using liquibase version 4.2.0+

Insert new data using Liquibase, postgreSQL and sequence

How do I make an insert for PostgreSQL using Liquibase if my ids are sequential. I try with the following:
<changeSet author="rparente" id="service-1.1-2019-01-09-01">
<insert tableName="tenant">
<column name="id"defaultValueSequenceNext="hibernate_sequence"/>
<column name="description" value="Prueba"/>
<column name="name" value="antel"/>
<column name="service_id" value="antel"/>
</insert>
</changeSet>
and I try with
<changeSet author="rparente" id="service-1.1-2019-01-09-01">
<insert tableName="tenant">
<column name="id" value="nextval('hibernate_sequence')"/>
<column name="description" value="Prueba"/>
<column name="name" value="antel"/>
<column name="service_id" value="antel"/>
</insert>
</changeSet>
The error is:
ERROR: null value in column "id" violates not-null constraint
I found the solution to insert data in Postgres with sequence (no default) ids
<changeSet author="author_name" id="service-1.1-2019-01-09-01">
<insert tableName="tenant">
<column name="id" valueSequenceNext="name_sequence"/>
<column name="description" value="TEST"/>
<column name="name" value="test"/>
<column name="service_id" value="testl"/>
<column name="status" value="ACTIVE"/>
</insert>
</changeSet>
Check out the ColumnConfig doc. You should be able to set a valueComputed property and in it call the Postgres function:
<column name="id" valueComputed="nextval('hibernate_sequence')"/>
As for me, I have to create a sequence first and then use it.
<changeSet>
<createSequence sequenceName="runtime_name_seq" dataType="bigint" incrementBy="1" maxValue="10000" minValue="1" startValue="1"/>
</changeSet>
<changeSet>
<createTable tableName="runtime_name">
<column name="id" type="INTEGER" defaultValueComputed="nextval('runtime_name_seq')">
<constraints nullable="false" primaryKey="true" primaryKeyName="pk_runtime_name"/>
</column>
</createTable>
</changeSet>
This will create a SQL by Liquibase (v3.8.1 I am using)
CREATE TABLE public.runtime_name
(
index INTEGER DEFAULT nextval('runtime_name_seq') NOT NULL
)

create unique index with function in hsqldb / liquibase

Is there a way to use function index in hsqldb?
I tried those 4:
<column name="LOWER(name)"/>
<column name="LCASE(name)"/>
<column name="LOWER(name)" computed="true"/>
<column name="LCASE(name)" computed="true"/>
Inside my createIndex changeset:
<changeSet author="dgt" id="unique-postgres" dbms="hsqldb">
<createIndex indexName="lower_case_index" tableName="users" unique="true">
<column name="LOWER(name)" computed="true"/>
</createIndex>
In documentation I noticed that hsqldb got: LOWER and LCASE built in function, but any of those do not work for me.
Every single time I've got an error:
Reason: liquibase.exception.DatabaseException: unexpected token: (
required: ) [Failed SQL: CREATE UNIQUE INDEX PUBLIC.lower_case_index
ON PUBLIC.users(LOWER(name))]
I know about a solution that I can change column type from VARCHAR to VARCHAR_IGNORECASE, but it's not a case for me, because I need a solution to work on both db: hsqldb and postgres.
My ideal solution should look like this:
<changeSet author="dgt" id="users-unique-index-postgres" dbms="hsqldb">
<createIndex indexName="name_index" tableName="users" unique="true">
<column name="LOWER(name)" computed="true"/>
</createIndex>
</changeSet>
<changeSet author="dgt" id="users-unique-index-hsqldb" dbms="postgresql">
<createIndex indexName="name_index" tableName="users" unique="true">
<column name="lower(name)" computed="true"/>
</createIndex>
</changeSet>
But it doesn't work.
HSQLDB does not support function based indexes at all, so you need to find a different solution. You could e.g. define the column as varchar_ignorecase instead of varchar and then create a "normal" unique index on that column.
You can keep a single table definition by using properties.
That could look like this:
<changeSet author="dgt" id="create-users-table">
<property name="users_name_type" value="varchar" dbms="postgresql"/>
<property name="users_name_type" value="varchar_ignorecase" dbms="hsqldb"/>
<createTable tableName="users">
<column name="name" type="${users_name_type}">
<constraints nullable="false"/>
</column>
</createTable>
</changeSet>
<changeSet author="dgt" id="users-unique-index-postgres" dbms="postgresql">
<createIndex indexName="name_index" tableName="users" unique="true">
<column name="lower(name)" computed="true"/>
</createIndex>
</changeSet>
<changeSet author="dgt" id="users-unique-index-hsqldb" dbms="hsqldb">
<createIndex indexName="name_index" tableName="users" unique="true">
<column name="name"/>
</createIndex>
</changeSet>

Adding an entity key when no key is inferred from a view

I have a database view which joins across a number of tables in SQL Server 2005. It looks something like this:
SELECT
m1.MenuName AS menu_name, m2.MenuName AS sub_menu_name, p.ProductName, od.amount
FROM
dbo.tblMenus AS m1
FULL OUTER JOIN
dbo.tblMenus AS m2
FULL OUTER JOIN
dbo.tblProductsRelMenus AS pm ON m2.Id = pm.SubMenuId ON m1.Id = pm.MenuId
FULL OUTER JOIN
(SELECT
dbo.tblOrderDetails.ProductId, SUM(dbo.tblOrderDetails.Ammount) AS amount
FROM
dbo.tblOrderDetails
FULL OUTER JOIN
dbo.tblOrders AS o ON dbo.tblOrderDetails.OrderId = o.OrderId
WHERE (o.OrderDestroyed = 0)
GROUP BY dbo.tblOrderDetails.ProductId) AS od
RIGHT OUTER JOIN
dbo.tblProducts AS p ON od.ProductId = p.ProductId ON pm.ProductId = p.ProductId
When I try to create an ADO .Net entity data model it complains about not having a primary key in the SSDL secion. I then found this:
http://msdn.microsoft.com/en-us/library/dd163156.aspx
but I don't understand the part about a defining query. Surely I just want a column with unique numbers to define the key, or?
<EntityType Name="SoldItemsView">
<Key>
<PropertyRef Name="SoldItemsViewId" />
</Key>
<Property Name="SoldItemsView" Type="int" Nullable="false" />
<Property Name="menu_name" Type="nvarchar" MaxLength="100" />
<Property Name="sub_menu_name" Type="nvarchar" MaxLength="100" />
<Property Name="ProductName" Type="nvarchar" MaxLength="50" />
<Property Name="amount" Type="int" />
</EntityType>
But how do I populate this column with unique numbers?
Thanks,
Barry
You can use only columns from the view. To define an entity key you must select column (or set of columns) from your view which uniquely identifies record from the view.