Pyspark error Invalid call to qualifier on unresolved object - pyspark
Hi I have table with a nested structure and trying to insert the data into the table using spark
CREATE TABLE IF NOT EXISTS dbname.tbname (
client_id BIGINT COMMENT 'Client id ',
visit_id string COMMENT 'total time spent per visit in seconds',
visit_start_time timestamp COMMENT 'minimum timestamp per visit level',
visit_end_time timestamp COMMENT 'maximum timestamp per visit level',
time_spent_per_visit_in_seconds integer COMMENT 'total time spent per visit in seconds',
visit_metrics STRUCT <
distinct_processes_count:integer COMMENT 'Total number of distinct process browsed through the visit through web activity',
distinct_page_count:integer COMMENT 'Total number of distinct page browsed through the visit through web activity'
> COMMENT 'Visit Level Metrics' ,
open_an_account STRUCT <
existing_client:STRUCT <
process_initiated:integer COMMENT 'Open an account of exisisting client intiated count through web activity',
process_submitted:integer COMMENT 'Open an account of exisisting client submitted count through web activity'
> COMMENT 'Open and account counts through web activity' ,
unspecified:STRUCT <
process_initiated:integer COMMENT 'Open an account of client intiated initiated count through web activity',
process_submitted:integer COMMENT 'Open an account of client intiated initiated count through web activity'
> COMMENT 'Open and account counts through web activity'
> COMMENT 'Open and account web activity count metrics',
transfer_of_assets STRUCT <
process_initiated:integer COMMENT 'Transfer of assests start indicator through web activity',
process_submitted:integer COMMENT 'Transfer of assests end indicator through web activity'
> COMMENT 'Transfer of account metrics',
direct_rollover STRUCT <
process_initiated:integer COMMENT 'Direct Rollover process intitiation count through web activity',
process_submitted:integer COMMENT 'Direct Rollover process submission count through web activity'
> COMMENT 'Direct Rollover Metrics',
trade_path STRUCT <
process_initiated:integer COMMENT 'Trade Path process intitiation count through web activity',
process_submitted:integer COMMENT 'Trade Path process intitiation count through web activity'
> COMMENT 'Trade Path Process Metrics',
account_management STRUCT <
account_activity:STRUCT <
alert_change_counts:STRUCT <
process_initiated:integer COMMENT 'Account Management AccountActivityAlerts change edit initiated count through web activity',
process_submitted:integer COMMENT 'Account Management AccountActivityAlerts change edit submitted through web activity'
> COMMENT 'Account Activity Alert Change counts through web activity'
> COMMENT 'Account Management Account Activity' ,
contact_info:STRUCT <
address_or_phone_change:STRUCT <
process_initiated:integer COMMENT 'Account Management addressphone change initiated count through web activity',
process_submitted:integer COMMENT 'Account Management addressphone change submitted count through web activity'
> COMMENT 'Account Activity Alert Change counts',
email_change:STRUCT <
process_initiated:integer COMMENT 'Account Management email change initiated count through web activity',
process_submitted:integer COMMENT 'Account Management email change submitted count through web activity'
> COMMENT 'Account Activity Email activity counts',
username_password_change:STRUCT <
process_initiated:integer COMMENT 'Account Management UsernamePassword change initiated count through web activity',
process_submitted:integer COMMENT 'Account Management UsernamePassword change submitted count through web activity'
> COMMENT 'Account Activity Username Password activity counts'
> COMMENT 'Account Management Contact Info Changes' ,
automatic_investments:STRUCT <
add_operation:STRUCT <
process_initiated:integer COMMENT 'Account Management Auto investment Add initiated count through web activity',
process_submitted:integer COMMENT 'Account Management Auto investment Add submitted count through web activity'
> COMMENT 'Account Activity add Auto investment counts',
edit_operation:STRUCT <
process_initiated:integer COMMENT 'Account Management Auto investment initiated count through web activity',
process_submitted:integer COMMENT 'Account Management Auto investment submitted count through web activity'
> COMMENT 'Account Activity edit Auto investment counts',
delete_operation:STRUCT <
process_initiated:integer COMMENT 'Account Management Auto investment delete initiated count through web activity',
process_submitted:integer COMMENT 'Account Management Auto investment delete submitted count through web activity'
> COMMENT 'Account Management delete Auto investment counts'
> COMMENT 'Account Management Auto investment Changes' ,
automatic_withdrawals:STRUCT <
process_initiated:integer COMMENT 'Account Management Auto withdrawal initiated count',
process_submitted:integer COMMENT 'Account Management Auto withdrawal submitted count'
> COMMENT 'Account Management Auto withdrawal process counts' ,
banking_info_addition:STRUCT <
process_initiated:integer COMMENT 'Account Management bank information initiated count',
process_submitted:integer COMMENT 'Account Management bank information submitted count'
> COMMENT 'Account Management bank information addition process counts' ,
beneficiaries_change:STRUCT <
process_initiated:integer COMMENT 'Account Management beneficiaries change initiated count',
process_submitted:integer COMMENT 'Account Management beneficiaries change submitted count'
> COMMENT 'Account Management beneficiaries change counts' ,
cost_basis_method_change:STRUCT <
process_initiated:integer COMMENT 'Account Management Costbasis method change initiated count',
process_submitted:integer COMMENT 'Account Management Costbasis method change submitted count'
> COMMENT 'Account Management Costbasis method counts' ,
holding_dividends_capital_gains_change:STRUCT <
process_initiated:integer COMMENT 'Account Management HoldingDividendsCapitalGains initiated count',
process_submitted:integer COMMENT 'Account Management HoldingDividendsCapitalGains submitted count'
> COMMENT 'Account Management HoldingDividendsCapitalGains counts' ,
instantbank_change:STRUCT <
process_initiated:integer COMMENT 'Account Management instant bank add initiated count',
process_submitted:integer COMMENT 'Account Management instant bank add submitted count'
> COMMENT 'Account Management HoldingDividendsCapitalGains counts' ,
required_minimum_distribution:STRUCT <
calculate:STRUCT <
process_initiated:integer COMMENT 'Account Management required minimum distribution calculate initiated count through web activity',
process_submitted:integer COMMENT 'Account Management required minimum distribution calculate submitted count through web activity'
> COMMENT 'Account Activity Alert Change counts',
one_time_distribution:STRUCT <
process_initiated:integer COMMENT 'Account Management required minimum distribution OneTimeDistribution intiated through web activity',
process_submitted:integer COMMENT 'Account Management required minimum distribution OneTimeDistribution submitted through web activity'
> COMMENT 'Account Activity Email activity counts'
> COMMENT 'Account Management Activity Changes'
> COMMENT ' Account Management Metrics'
)
COMMENT 'All journey visits provides a different Journey process visited during each visit an client made'
PARTITIONED BY (visit_date date COMMENT 'Date of the visit. Partition column')
STORED AS PARQUET
LOCATION 's3://bucketname/db_name/table_name/'
The loading of the data logic is as below i am just restructuring using the named_struct function nothing much than that.
I has to use sql to do this as the frameworkwe have accepts only sql and not dataframes.
spark.sql('''with web_cust_copy as
(
select
cast ('1' as bigint) client_id,
'213213' visit_id,
cast ('10' as integer) as time_spent_per_visit_in_seconds,
'2022-09-17 00:01:23' as visit_start_time,
'2022-09-17 00:01:23' as visit_end_time,
cast ('0' as integer) total_distinct_page_count ,
cast ('2' as integer) total_distinct_process_count,
cast ('1' as integer ) OpenanAccountJourney_initiated,
cast ('1' as integer ) OpenanAccountJourney_submitted,
cast ('1' as integer ) Client_OpenanAccountJourney_initiated,
cast ('1'as integer ) Client_OpenanAccountJourney_submitted,
cast ('1' as integer ) TransferofAssets_initiated,
cast ('1' as integer ) TransferofAssets_submitted,
cast ('1' as integer ) DirectRollover_initiated,
cast ('1' as integer ) DirectRollover_submitted,
cast ('1' as integer ) Trade_Path_initiated,
cast ('1' as integer ) Trade_Path_submitted,
cast ('1' as integer ) AM_UserNamePassword_Change_initiated,
cast ('1' as integer ) AM_UserNamePassword_Change_submitted,
cast ('1' as integer ) AM_RMD_OneTimeDistribution_initiated,
cast ('1' as integer ) AM_RMD_OneTimeDistribution_submitted,
cast ('1' as integer ) AM_RMD_Calculate_initiated,
cast ('1' as integer ) AM_RMD_Calculate_submitted,
cast ('1' as integer ) AM_InstantBank_Add_initiated,
cast ('1' as integer ) AM_InstantBank_Add_submitted,
cast ('1' as integer ) AM_HoldingDividendsCapitalGains_Change_initiated,
cast ('1' as integer ) AM_HoldingDividendsCapitalGains_Change_submitted,
cast ('1' as integer ) AM_Email_Change_initiated,
cast ('1' as integer ) AM_Email_Change_submitted,
cast ('1' as integer ) AM_CostBasisMethod_Change_initiated,
cast ('1' as integer ) AM_CostBasisMethod_Change_submitted,
cast ('1' as integer ) AM_Beneficiaries_Change_initiated,
cast ('1' as integer ) AM_Beneficiaries_Change_submitted,
cast ('1' as integer ) AM_BankInformation_Add_initiated,
cast ('1' as integer ) AM_BankInformation_Add_submitted,
cast ('1' as integer ) AM_AutomaticWithdrawal_Add_initiated,
cast ('1' as integer ) AM_AutomaticWithdrawal_Add_submitted,
cast ('1' as integer ) AM_AutomaticInvestment_Edit_initiated,
cast ('1' as integer ) AM_AutomaticInvestment_Edit_submitted,
cast ('1' as integer ) AM_AutomaticInvestment_Delete_intitated,
cast ('1' as integer ) AM_AutomaticInvestment_Delete_submitted,
cast ('1' as integer ) AM_AutomaticInvestment_Add_intitated,
cast ('1' as integer ) AM_AutomaticInvestment_Add_submitted,
cast ('1' as integer ) AM_AddressPhone_Change_intitated,
cast ('1' as integer ) AM_AddressPhone_Change_submitted,
cast ('1' as integer ) AM_AccountActivityAlerts_Change_intitated,
cast ('1' as integer ) AM_AccountActivityAlerts_Change_submitted,
cast ('2022-09-17' as date) visit_date
),
all_journey_visits_data_struct as
(
SELECT client_id
,visit_id
,visit_start_time
,visit_end_time
,time_spent_per_visit_in_seconds
,named_struct('distinct_processes_count',total_distinct_process_count,'distinct_page_count',total_distinct_page_count) as visit_metrics
,named_struct('process_initiated',openanaccountjourney_initiated,'process_submitted',openanaccountjourney_submitted) as open_an_acct_journey
,named_struct('process_initiated',client_openanaccountjourney_initiated,'process_submitted',client_openanaccountjourney_submitted) as client_open_an_acct_journey
,named_struct('process_initiated',transferofassets_initiated,'process_submitted',transferofassets_submitted) as transfer_of_assets
,named_struct('process_initiated',directrollover_initiated,'process_submitted',directrollover_submitted) as direct_rollover
,named_struct('process_initiated',trade_path_initiated,'process_submitted',trade_path_submitted) as trade_path
,named_struct('process_initiated',am_accountactivityalerts_change_intitated,'process_submitted',am_accountactivityalerts_change_submitted) as alert_change_counts
,named_struct('process_initiated',am_email_change_initiated,'process_submitted',am_email_change_submitted) as email_change
,named_struct('process_initiated',am_usernamepassword_change_initiated,'process_submitted',am_usernamepassword_change_submitted) as username_password_change
,named_struct('process_initiated',am_addressphone_change_intitated,'process_submitted',am_addressphone_change_submitted) as address_or_phone_change
,named_struct('process_initiated',am_automaticinvestment_add_intitated,'process_submitted',am_automaticinvestment_add_submitted) as automatic_investments_add_operation
,named_struct('process_initiated',am_automaticinvestment_edit_initiated,'process_submitted',am_automaticinvestment_edit_submitted) as automatic_investments_edit_operation
,named_struct('process_initiated',am_automaticinvestment_delete_intitated,'process_submitted',am_automaticinvestment_delete_submitted) as automatic_investments_delete_operation
,named_struct('process_initiated',am_automaticwithdrawal_add_initiated,'process_submitted',am_automaticwithdrawal_add_submitted) as automatic_withdrawals
,named_struct('process_initiated',am_bankinformation_add_initiated,'process_submitted',am_bankinformation_add_submitted) as banking_info_addition
,named_struct('process_initiated',am_beneficiaries_change_initiated,'process_submitted',am_beneficiaries_change_submitted) as beneficiaries_change
,named_struct('process_initiated',am_costbasismethod_change_initiated,'process_submitted',am_costbasismethod_change_submitted) as cost_basis_method_change
,named_struct('process_initiated',am_holdingdividendscapitalgains_change_initiated,'process_submitted',am_holdingdividendscapitalgains_change_submitted) as holding_dividends_capital_gains_change
,named_struct('process_initiated',am_instantbank_add_initiated,'process_submitted',am_instantbank_add_submitted) as instantbank_change
,named_struct('process_initiated',am_rmd_calculate_initiated,'process_submitted',am_rmd_calculate_submitted) as rmd_calculate
,named_struct('process_initiated',am_rmd_onetimedistribution_initiated,'process_submitted',am_rmd_onetimedistribution_submitted) as rmd_one_time_distribution
,visit_date from web_cust_copy where visit_date = '2022-09-17'
)
INSERT OVERWRITE TABLE dbname.tbname partition(visit_date)
SELECT /*+ COALESCE(4) */ client_id
,visit_id
,visit_start_time
,visit_end_time
,time_spent_per_visit_in_seconds
, visit_metrics
,named_struct('existing_client',client_open_an_acct_journey,'unspecified',open_an_acct_journey) as open_an_account
,transfer_of_assets
,direct_rollover
,trade_path
,named_struct(
'account_activity',(named_struct('alert_change_counts',alert_change_counts))
,'contact_info',(named_struct('address_or_phone_change',address_or_phone_change,'email_change',email_change,'username_password_change',username_password_change))
,'automatic_investments',(named_struct('add_operation',automatic_investments_add_operation,'edit_operation',automatic_investments_edit_operation,'delete_operation',automatic_investments_delete_operation))
,'automatic_withdrawals',(named_struct('automatic_withdrawals',automatic_withdrawals))
,'banking_info_addition',(named_struct('banking_info_addition',banking_info_addition))
,'beneficiaries_change',(named_struct('beneficiaries_change',beneficiaries_change))
,'cost_basis_method_change',(named_struct('cost_basis_method_change',beneficiaries_change))
,'holding_dividends_capital_gains_change',(named_struct('holding_dividends_capital_gains_change',holding_dividends_capital_gains_change))
,'instantbank_change',(named_struct('instantbank_change',instantbank_change))
,'required_minimum_distribution',(named_struct('calculate',rmd_calculate,'one_time_distribution',rmd_one_time_distribution))
) as account_management
,visit_date
FROM all_journey_visits_data_struct''')
Error iam encountering is below
An error was encountered:
"Invalid call to qualifier on unresolved object, tree: 'account_management"
Traceback (most recent call last):
File "/mnt/yarn/usercache/umh7/appcache/application_1663754629436_0006/container_1663754629436_0006_01_000001/pyspark.zip/pyspark/sql/session.py", line 767, in sql
return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)
File "/mnt/yarn/usercache/appcache/application_1663754629436_0006/container_1663754629436_0006_01_000001/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/mnt/yarn/usercache/appcache/application_1663754629436_0006/container_1663754629436_0006_01_000001/pyspark.zip/pyspark/sql/utils.py", line 71, in deco
raise AnalysisException(s.split(': ', 1)[1], stackTrace)
pyspark.sql.utils.AnalysisException: "Invalid call to qualifier on unresolved object, tree: 'account_management"
Not sure how to fix the error
Related
Validate email address will be accepted by sp_send_dbmail?
How do you validate that a text string is a valid email address which will be accepted by the sp_send_dbmail function? I've looked through other questions like this one, which yes works great, until a user copies their email address from outlook and comes through like Jane Doe <DoeJ#xdomain.com>, which fails to send via the system proc. I also want users to be able to supply multiple emails in a single string separated by semicolons, which are accepted by sp_send_dbmail. Thanks!
You can try this (there are other ways),
--PARAM:START
DECLARE #EmailList varchar(max);
SET #EmailList = 'Jane Doe <DoeJ#xdomain.com>;info.support#mssqltips.com;.info#mssqltips.com;
info..#mssqltips.com;info#mssqltips.c;info#support#mssqltips.com;info.support#mssql_tips.com;
+info#mssqltips.com;info Support#mssqltips.com;info#mssql tips.com;NULL;22#mssqltips.com;#mssqltips.com';
--PARAM:END
--VALIDATION:START
DECLARE #EmailTable TABLE
(
Email varchar(max),
IsValid bit
);
INSERT INTO #EmailTable(Email, IsValid)
SELECT LTRIM(RTRIM(value))
,CASE WHEN LTRIM(RTRIM(value)) = '' THEN 0
WHEN LTRIM(RTRIM(value)) LIKE '% %' THEN 0
WHEN LTRIM(RTRIM(value)) LIKE ('%["(),:;<>\]%') THEN 0
WHEN SUBSTRING(LTRIM(RTRIM(value)),CHARINDEX('#',LTRIM(RTRIM(value))),LEN(LTRIM(RTRIM(value)))) LIKE ('%[!#$%&*+/=?^`_{|]%') THEN 0
WHEN (LEFT(LTRIM(RTRIM(value)),1) LIKE ('[-_.+]') OR RIGHT(LTRIM(RTRIM(value)),1) LIKE ('[-_.+]')) THEN 0
WHEN (LTRIM(RTRIM(value)) LIKE '%[%' or LTRIM(RTRIM(value)) LIKE '%]%') THEN 0
WHEN LTRIM(RTRIM(value)) LIKE '%#%#%' THEN 0
WHEN LTRIM(RTRIM(value)) NOT LIKE '_%#_%._%' THEN 0
ELSE 1
END
FROM STRING_SPLIT(#EmailList, ';');
--VALIDATION:END
--GET VALID EMAIL
DECLARE #ValidEmailList varchar(max);
SELECT #ValidEmailList = COALESCE(#ValidEmailList + ', ', '') + Email
FROM #EmailTable
WHERE IsValid = 1;
--DO SENDING OF EMAIL USING ValidEmailList
Source: Valid Email Address Check with TSQL
#tontonsevilla Great solution, I just modified it a bit to remove the intermediate table and replace any line breaks that may show up (as they did from copying your solution):
DECLARE #EmailList VARCHAR(MAX);
SET #EmailList
= 'Jane Doe <DoeJ#xdomain.com>; info.support#mssqltips.com;.info#mssqltips.com;
info..#mssqltips.com;info#mssqltips.c;info#support#mssqltips.com;info.support#mssql_tips.com;
+info#mssqltips.com;info Support#mssqltips.com;info#mssql tips.com;NULL;22#mssqltips.com;#mssqltips.com';
DECLARE #ValidEmailList VARCHAR(MAX);
SELECT #ValidEmailList = COALESCE(#ValidEmailList + ';', '') + [x].[Email]
FROM (
SELECT TRIM(REPLACE(REPLACE([value], CHAR(10), ''), CHAR(13), '')) [Email]
FROM STRING_SPLIT(#EmailList, ';')
WHERE LEN([value]) > 4
AND CHARINDEX(' ', TRIM([value])) = 0
AND TRIM([value])NOT LIKE ('%["(),:;<>\]%')
AND SUBSTRING(TRIM([value]), CHARINDEX('#', TRIM([value])), LEN(TRIM([value])))NOT LIKE ('%[!#$%&*+/=?^`_{|]%')
AND LEFT(TRIM([value]), 1)NOT LIKE ('[-_.+]')
AND RIGHT(TRIM([value]), 1)NOT LIKE ('[-_.+]')
AND TRIM([value])NOT LIKE '%[%'
AND TRIM([value])NOT LIKE '%]%'
AND TRIM([value])NOT LIKE '%#%#%'
AND TRIM([value]) LIKE '_%#_%._%'
) [x];
SELECT #ValidEmailList;
How to fetch doctype eg: address or tax rule
I want to fetch the doctype. How do I do this? I want to add a separate column which will give doctype such as sales order, purchase order etc. The first line gives me error what query should be fired. Please help I am new to ERP Next. SELECT AD.ref_doctype AS “Doctype:Link/User:120”, AD.name AS “Doc#:Link/Doctype:120”, AD.owner AS “Created By:Link/User:120”, AD.modified AS “Modified On:Date:120” FROM tabAddress AS AD WHERE DATEDIFF(now(),AD.modified) BETWEEN 1 AND 30 UNION ALL SELECT TR.name AS “Doc#:Link/Doctype:120”, TR.owner AS “Created By:Link/User:120”, TR.modified AS “Modified On:Date:120” FROM tabTax Rule AS TR WHERE DATEDIFF(now(),TR.modified) BETWEEN 1 AND 30 UNION ALL SELECT IT.name AS “Doc#:Link/Doctype:120”, IT.owner AS “Created By:Link/User:120”, IT.modified AS “Modified On:Date:120” FROM tabItem AS IT WHERE DATEDIFF(now(),IT.modified) BETWEEN 1 AND 30
It isn't completely clear to me what you mean by docType field.
Are you wanting a result like this?
Doctype:Link/User:120|Doc#:Link/Doctype:120|Created By:Link/User:120|Modified On:Date:120|
---------------------|---------------------|------------------------|--------------------|
Email Account |Jobs |Administrator | 2019-12-04 06:07:55|
Email Account |Notifications |Administrator | 2019-12-01 05:25:53|
Email Account |Replies |Administrator | 2019-12-01 05:25:53|
Email Account |Sales |Administrator | 2019-12-04 06:07:55|
Email Account |Support |Administrator | 2019-12-04 06:07:55|
Here's the select :
set #docType = "Email Account";
SELECT
#tabDocType AS `Doctype:Link/User:120`,
AD.name AS `Doc#:Link/Doctype:120`,
AD.owner AS `Created By:Link/User:120`,
AD.modified AS `Modified On:Date:120`
FROM `tabEmail Account` AS AD
Note the backticks on the field aliases! All these have different meanings in SQL:
"
'
`
The last one, backtick, is used to refer to database entities. You were trying to use “Doctype:Link/User:120” with double quotes, which declare plain text. Using backtick converts the alias into a db entity which can be referred to from elsewhere.
MariaDb doesn't allow the use of variables as table names directly, but you can do it using prepared statements, like this:
set #docType = "Email Account";
set #tabDocType = CONCAT('tab', #docType);
SET #sql_text = concat('
SELECT
"', #docType, '" AS `Doctype:Link/User:120`
, AD.name AS `Doc#:Link/Doctype:120`
, AD.owner AS `Created By:Link/User:120`
, AD.modified AS `Modified On:Date:120`
FROM `', #tabDocType, '` as AD;
');
PREPARE stmt FROM #sql_text;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
The table name is now also specified by a variable, created from concatenation of 'tab' with the docType declared before.
You get the same result as above but -- you avoid accidentally changing the table name in one place but not in the other when editing some time in the future.
to fetch doctype name you have to give the linked doctype name, For example, select IT.name as "IT No:Link/IT:120"
SSRS reports using prefiltering
I want to run my report for each case associated with a contract for a customer. Right now I am able to run reports on the account level. I have used the filtering view as below: dbo.FilteredAccount AS CRMAF_FilteredAccount on CRMAF_FilteredAccount.accountid = I.CustomerId WHERE A.PartyObjectTypeCode = 8 AND A.PartyIdName IS NOT NULL I want to be able to run this same report against each case I have with the Account associated with a contract. I tried using FilteredIncident view but I'm not sure I am doing it right. Query from comments: SELECT I.CustomerIdName AS 'Customer Name', S.CreatedOn AS 'Task Start Date', S.Subject AS 'Task Performed', A.PartyIdName AS 'Resource', S.Description, cast((S.ActualDurationMinutes / 480) as varchar(5)) +' day(s)' + ', ' + cast((S.ActualDurationMinutes% 480) / 60 as varchar(2))+' hour(s)' as 'Time Spent', S.ActualDurationMinutes AS 'Total Time', S.ActualDurationMinutes AS 'Total' FROM dbo.Incident I JOIN [dbo].[ServiceAppointment] S ON I.IncidentId = S.RegardingObjectId JOIN dbo.ActivityParty A ON S.ActivityId = A.ActivityId
You'll need to add a CaseID in your select list and a parameter to compare it to in the report. So you would add something like this to your WHERE clause: WHERE I.CaseID = #SelectedCase
Invalid length parameter passed to the LEFT or SUBSTRING function
I have the following description: 'Sample Product Maker Product Name XYZ - Size' and I would like to only get the value 'Product Name XYZ' from this. If this were just one row I'd have no issue just using SUBSTRING but I have thousands of records and although the initial value Sample Product Maker is the same for all products the Product Name could be different and I don't want anything after the hyphen.
What I have so far has generated the error in the header of this question.
SELECT i.Itemid,
RTRIM(LTRIM(SUBSTRING(i.ShortDescription, 25, (SUBSTRING(i.ShortDescription, 25, CHARINDEX('-', i.ShortDescription, 25)))))) AS ProductDescriptionAbbrev,
CHARINDEX('-', i.ShortDescription, 0) - 25 as charindexpos
FROM t_items i
I am getting 'Argument data type varchar is invalid for argument 3 of substring function'
As you can see, I am getting the value for the last line the sql statement but when I try and plug that into the SUBSTRING function I get various issues.
Chances are good you have rows where the '-' is missing, which is causing your error.
Try this...
SELECT i.Itemid,
SUBSTRING(i.ShortDescription, 22, CHARINDEX('-', i.ShortDescription+'-', 22)) AS ProductDescriptionAbbrev,
FROM t_items i
You could also strip out the Sample Product Maker text and go from there:
SELECT RTRIM(LEFT(
LTRIM(REPLACE(i.ShortDescription, 'Sample Product Maker', '')),
CHARINDEX('-', LTRIM(REPLACE(i.ShortDescription, 'Sample Product Maker',
'' ))) - 1))
AS ShortDescription
Your first call to SUBSTRING specifies a length of SUBSTRING(i.ShortDescription, 25, CHARINDEX('-', i.ShortDescription, 25)).
You might try:
declare #t_items as Table ( ItemId Int Identity, ShortDescription VarChar(100) )
insert into #t_items ( ShortDescription ) values
( 'Sample Product Maker Product Name XYZ - Size' )
declare #SkipLength as Int = Len( 'Sample Product Maker' )
select ItemId,
RTrim( LTrim( Substring( ShortDescription, #SkipLength + 1, CharIndex( '-', ShortDescription, #SkipLength ) - #SkipLength - 1 ) ) ) as ProductDescriptionAbbrev
from #t_items
The problem is that your outer call to SUBSTRING is being passed a character data type from the inner SUBSTRING call in the third parameter.
+--This call does not return an integer type
SELECT i.Itemid, V
RTRIM(LTRIM(SUBSTRING(i.ShortDescription, 25, (SUBSTRING(i.ShortDescription, 25, CHARINDEX('-', i.ShortDescription, 25)))))) AS ProductDescriptionAbbrev,
CHARINDEX('-', i.ShortDescription, 0) - 25 as charindexpos
FROM t_items i
The third parameter must evaluate to the length that you want. Perhaps you meant LEN(SUBSTRING(...))?
Seems like you want something like this (22, not 25):
SELECT i.Itemid,
RTRIM(LTRIM(SUBSTRING(i.ShortDescription, 22, CHARINDEX('-', i.ShortDescription)-22))) AS ProductDescriptionAbbrev,
CHARINDEX('-', i.ShortDescription)-22 as charindexpos
FROM t_items i
You want:
LEFT(i.ShortDescription, isnull(nullif(CHARINDEX('-', i.ShortDescription),0) - 1, 8000))
Note that a good practice is to wrap charindex(...)'s and patindex(...)'s with nullif(...,0), and then handle the null case if desired (sometimes null is the right result, in this case we want all the text so we isnull(...,8000) for the length we want).
Dynamic pivot - how to obtain column titles parametrically?
I wish to write a Query for SAP B1 (t-sql) that will list all Income and Expenses Items by total and month by month.
I have successfully written a Query using PIVOT, but I do not want the column headings to be hardcoded like: Jan-11, Feb-11, Mar-11 ... Dec-11.
Rather I want the column headings to be parametrically generated, so that if I input:
--------------------------------------
Query - Selection Criteria
--------------------------------------
Posting Date greater or equal 01.09.10
Posting Date smaller or equal 31.08.11
[OK] [Cancel]
the Query will generate the following columns:
Sep-10, Oct-10, Nov-10, ..... Aug-11
I guess DYNAMIC PIVOT can do the trick.
So, I modified one SQL obtained from another forum to suit my purpose, but it does not work. The error message I get is Incorrect Syntax near 20100901.
Could anybody help me locate my error?
Note: In SAP B1, '[%1]' is an input variable
Here's my query:
/*Section 1*/
DECLARE #listCol VARCHAR(2000)
DECLARE #query VARCHAR(4000)
-------------------------------------
/*Section 2*/
SELECT #listCol =
STUFF(
( SELECT DISTINCT '],[' + CONVERT(VARCHAR, MONTH(T0.RefDate), 102)
FROM JDT1
FOR XML PATH(''))
, 1, 2, '') + ']'
------------------------------------
/*Section 3*/
SET #query = '
SELECT * FROM
(
SELECT
T0.Account,
T1.GroupMask,
T1.AcctName,
MONTH(T0.RefDate) as [Month],
(T0.Debit - T0.Credit) as [Amount]
FROM dbo.JDT1 T0
JOIN dbo.OACT T1 ON T0.Account = T1.AcctCode
WHERE
T1.GroupMask IN (4,5,6,7) AND
T0.[Refdate] >= '[%1]' AND
T0.[Refdate] <= '[%2]'
) S
PIVOT
(
Sum(Amount)
FOR [Month] IN ('+#listCol+')
) AS pvt
'
--------------------------------------------
/*Section 4*/
EXECUTE (#query)
I don't know SAP, but a couple of things spring to mind: It looks like you want #listCol to contain a collection of numbers within square brackets, for example [07],[08],[09].... However, your code appears not to put a [ at the start of this string. Try replacing the lines T0.[Refdate] >= '[%1]' AND T0.[Refdate] <= '[%2]' with T0.[Refdate] >= ''[%1]'' AND T0.[Refdate] <= ''[%2]'' (I also added a space before the AND in the first of these two lines while I was editing your question.)