U-SQL Substring out of range exception - substring

We are trying to publish data from a table using U-SQL in ADLA. We are using below code to get substring for FirstName, as we want to restrict the length of FirstName to 50 characters.
SELECT (firstName == null || firstName.Length <= 50) ? firstName : firstName.Substring(0, 50) AS FirstName
But, we are getting below error:
E_RUNTIME_USER_EXPRESSION_EVALUATION >
E_RUNTIME_USER_SUBSTRING_OUT_RANGE
When we tried to perform substring using custom .NET Code, we are not getting exception. The job is completing successfully. We are not getting any error rows.
public static string DoSubString(string firstName)
{
string subFirstName;
try
{
subFirstName = (firstName == null || firstName.Length <= 50) ? firstName : firstName.Substring(0, 50);
}
catch(ArgumentOutOfRangeException ae)
{
subFirstName = string.Format("Argument Out of range Error {0} {1}",firstName,ae.Message);
}
catch(Exception Ex)
{
subFirstName = string.Format("Generic Error {0} {1}",firstName, Ex.Message);
}
return subFirstName;
}
We are not able to find out the error row. When we look into Profile.xml, we are not getting row dump.
Inner exception from user expression: \nCurrent row
dump: "
How to find out the error row causing this exception? How to troubleshoot these kinds of issues ?

For testing I used 10 characters, revise for 50.
#table =
SELECT * FROM
( VALUES
("appleJackss"),
("apple Jacks"),
("appleJacks"),
(" "),
(""),
((string)null)
) AS T(word);
#result =
SELECT //Method 1
CASE
WHEN word.Length <= 10 THEN word
ELSE word.Substring(0, 10)
END AS justTen,
// Method 2
(word.Length <= 10) ? word : word.Substring(0, 10) AS anotherTen
FROM #table;
OUTPUT #result
TO "/Temp/Example1.txt"
USING Outputters.Tsv();

We raised an issue with product group. It seems there is some problem with the substring operation in U-SQL. They asked us to apply the below fix for Substring calculation. We tried it and it is working properly.
SELECT (firstName == null || firstName.Length <= 50) ? firstName : firstName.Substring(0, Math.Min(firstName.Length, 50)) AS FirstName

Related

The column index is out of range: 2, number of columns: 1 error while updating jsonb column

I am trying to update jsonb column in java with mybatis.
Following is my mapper method
#Update("update service_user_assn set external_group = external_group || '{\"service_name\": \"#{service_name}\" }' where user=#{user} " +
" and service_name= (select service_name from services where service_name='Google') " )
public int update(#Param("service_name")String service_name,#Param("user") Integer user);
I am getting the following error while updating the jsonb (external_group) cloumn.
### Error updating database. Cause: org.postgresql.util.PSQLException: The column index is out of range: 2, number of columns: 1.
### The error may involve com.apds.mybatis.mapper.ServiceUserMapper.update-Inline
I am able to update with the same way for non-jsonb columns.
Also if I am putting hardcoded value it's working for jsonb columns.
How to solve this error while updating jsonb column?
You should not enclose #{} in single quotes because it will become part of a literal rather than a placeholder. i.e.
external_group = external_group || '{"service_name": "?"}' where ...
So, there will be only one placeholder in the PreparedStatement and you get the error.
The correct way is to concatenate the #{} in SQL.
You may also need to cast the literal to jsonb type explicitly.
#Update({
"update service_user_assn set",
"external_group = external_group",
"|| ('{\"service_name\": \"' || #{service_name} || '\" }')::jsonb",
"where user=#{user} and",
"service_name= (select service_name from services where service_name='Google')"})
The SQL being executed would look as follows.
external_group = external_group || ('{"service_name": "' || ? || '"}')::jsonb where ...

why my Linqued query is not getting executed

I have this linqued query
var moreThen1dayLeavefed = (from LApp in db.LeaveApplications
join Emp in db.Employees
on LApp.Employee equals Convert.ToInt32(Emp.EmployeeNumber)
join LBrk in db.LeaveBreakups
on LApp.Id equals LBrk.LeaveApplication
where Emp.Team == 8 && LBrk.StartDate.Year == 2015 && LBrk.StartDate.Month == 5
select new { StartDate = LBrk.StartDate.Day, EndDate = LBrk.EndDate.Day, diff = (DbFunctions.DiffDays(LBrk.StartDate, LBrk.EndDate) + 1) }).ToList();
it gives error
LINQ to Entities does not recognize the method 'Int32 ToInt32(System.String)' method, and this method cannot be translated into a store expression.
on line 3, i.e
on LApp.Employee equals Convert.ToInt32(Emp.EmployeeNumber)
as I am converting the string to int during inner join
Just saw your related question. Your EmployeeNumber field seems to be filled with fixed size (5) zero left padded string representation of a number. If that's true, you can use the trick from how to sort varchar column containing numeric values with linq lambdas to Entity to solve the issue.
Just replace
on LApp.Employee equals Convert.ToInt32(Emp.EmployeeNumber)
with
on DbFunctions.Right("0000" + LApp.Employee.ToString(), 5) equals Emp.EmployeeNumber

Order By on specific column

I am trying to get the distinct records
var records = (from entry in db.Table1
select new
{
RowID = entry.RowID,
FirstName = entry.First,
LastName = entry.Last,
Restricted = (entry.Prohibited == null || entry.Prohibited == "False") ? "Restricted" : "Not Restricted"
}).ToList();
Here RowID is an primary key. I want to get the distinct First and Last Name.
For example:
RowID First Last Prohibited ...
1 A B False
2 A B False
3 A B False
4 Z Y True
5 Z Y True
What I am trying to get here is:
RowID First Last Prohibited
1 A B False
4 Z Y True
How can I get it?
var records = (from entry in db.Table1
group entry by new {entry.First, entry.Last} into g
let entry = g.FirstOrDefault()
select new
{
RowID = entry.RowID,
FirstName = entry.First,
LastName = entry.Last,
Restricted = (entry.Prohibited == null || entry.Prohibited == "False") ? "Restricted" : "Not Restricted"
}).ToList();
By grouping the items by the combined first and last name, and then only taking the first item in each group, you effectively ensure that you're getting distinct items based on those values.
Simply you can do in this way:
var records =
(from entry in db.Table1
select new {
FirstName = entry.First,
LastName = entry.Last,
Restricted = (entry.Prohibited == null || entry.Prohibited == "False") ? "Restricted" : "Not Restricted"
}).Distinct();
If you want the rowID also, you can read select-distinct-using-linq.

sphinxSE Setselect

Using SphinxApi , We have using following query for sphinxApi Setselect:
$cl->SetSelect ( "*, IF( IN(watching, 1) OR ordered=1, 1, 0) AS customFilter" );
Using SphinxQL:
select IF( IN(watching, 1) OR ordered=1, 1, 0) AS customFilter from indexname
Is that same thing how to using in SphinxSE ?
Any one can help me out?
SELECT * FROM indexname WHERE query = ';select=*, IF( IN(watching, 1) OR ordered=1, 1, 0) AS customFilter';
Of course customFilter must be defined as a column on the sphinx virtual table, if you want to see the value in your results

T-SQL: returning VARCHAR in a derived column

I am having problems returning a VARCHAR out of a derived column.
Below are extremely simplified code examples.
I have been able to do this before:
SELECT *, message =
CASE
WHEN (status = 0)
THEN 'aaa'
END
FROM products
But when I introduce a Common Table Expression or Derived Table:
WITH CTE_products AS (SELECT * from products)
SELECT *, message =
CASE WHEN (status = 0)
THEN 'aaa'
END
FROM CTE_products
this seems to fail with the following message:
Conversion failed when converting the varchar value 'aaa' to data type int.
When I tweak the line to say:
WITH CTE_products AS (SELECT * from products)
SELECT *, message =
CASE WHEN (status = 0)
THEN '123'
END
FROM CTE_products
It returns correctly.
...
When I remove all the other clauses prior to it, it also works fine returning 'aaa'.
My preference would be to keep this as a single, stand-alone query.
The problem is that the column is an integer dataype and sql server is trying to convert 'aaa' to integer
one way
WITH CTE_products AS (SELECT * from products)
SELECT *, message =
CASE WHEN (status = 0)
THEN 'aaa' else convert(varchar(50),status)
END
FROM CTE_products
I actually ended up finding the answer.
One of my CASE/WHEN clauses used a derived column from the CTE and that ended up causing the confusion.
Before:
WITH CTE_products AS (SELECT *, qty_a + qty_b as qty_total FROM products)
SELECT *, message =
CASE WHEN (status = 0)
THEN 'Status is 0, the total is: ' + qty_total + '!'
END
FROM CTE_products
Corrected:
WITH CTE_products AS (SELECT *, qty_a + qty_b as qty_total FROM products)
SELECT *, message =
CASE WHEN (status = 0)
THEN 'Status is 0, the total is: ' + CAST(qty_total AS VARCHAR) + '!'
END
FROM CTE_products
I ended up removing WHEN/THEN clauses within the CASE statement right afterwards to see if it was a flukey parentheses error when I realized that in the absence of any of the WHEN/THEN clauses that included the derived column from the CTE, it was able to return VARCHAR.