Why is my Hive QL Query that I run in SSMS via Openquery through the Hortonworks ODBC Driver producing an error? - tsql

I set up a connection to a Hive server using the Hortonworks ODBC Driver for Apache Hive. Version info is below:
OS: Windows Server 2012 R2 Standard
Hive: 1.2.1000.2.6.5.4-1
Hadoop: 2.7.3.2.6.5.4-1
Hortonworks ODBC Driver for Apache Hive
ODBC Version: 03.80
Driver Version: 2.1.12.1017
Bitness: 64-bit
Locale: en_US
I can run the queries below using the connector that I configured in Teradata SQL Assistant with no issues. I set up my DSN as a linked server in SSMS. However, when I attempt to run the queries in SSMS using openquery, I have some issues. Info on my SQL Server is below:
Microsoft SQL Server 2016 (SP2-CU3) (KB4458871) - 13.0.5216.0 (X64) Sep 13 2018 22:16:01 Copyright (c) Microsoft Corporation Enterprise Edition (64-bit) on Windows Server 2012 R2 Standard 6.3 <X64> (Build 9600: ) (Hypervisor)
Here is some info on the table that I am querying:
Table Name: instrumentapps_event
Using OPENQUERY, I am capable of querying the Hive DB through SSMS with the following query:
SELECT * FROM OPENQUERY(KMhivehttp, 'select * from dmfwk_gold.instrumentapps_event')
The above query returns the contents of the desired table. However, the query below produces an error:
SELECT * FROM OPENQUERY(KMhivehttp, 'select * from dmfwk_gold.instrumentapps_event WHERE to_date(from_unixtime(UNIX_TIMESTAMP(load_ts,''yyyy/MM/dd''))) >= to_date(''2019-03-01'')')
The error is as follows:
Msg 7355, Level 16, State 1, Line 1
The OLE DB provider "MSDASQL" for linked server "KMhivehttp" supplied inconsistent metadata for a column. The name was changed at execution time.
How can I fix this?

While I'm unsure how to fix the problem that I previously faced (I suspect it to be a bug with the Hortonworks ODBC driver), I did discover a workaround.
Instead of running:
SELECT * FROM OPENQUERY(KMhivehttp,
'SELECT *
FROM dmfwk_gold.instrumentapps_event
WHERE to_date(from_unixtime(UNIX_TIMESTAMP(load_ts,''yyyy/MM/dd''))) >=
to_date(''2019-03-01'')
')
I now use:
SELECT * FROM OPENQUERY(KMhivehttp, 'select * from dmfwk_gold.instrumentapps_event')
WHERE load_ts >= CAST('2019-03-01' AS DATE);
This allows me to avoid any metadata errors.

Related

DBVisualizer displays null on date field holding '0001-01-01'

I issued an SQL statement in DbVis:
select vestdate, name from person where vestdate is not null
And got many results where DbVisualizer showed vestdate as (null)!
After investigating, I discovered that the vestdate was '0001-01-01', so the query correctly returned these records, but DbVisualizer displays them as (null).
I have just switched from windows 8 to windows 10.
It works on windows 8 (displays '0001-01-01'), but not not windows 10 (displays null):
Product: DbVisualizer Pro 11.0.4 [Build #3103]
OS: Windows 8.1
OS Version: 6.3
OS Arch: amd64
Java Version: 1.8.0_252
Java VM: OpenJDK 64-Bit Server VM
Java Vendor: AdoptOpenJDK
Java Home: c:\program files\dbvisualizer\jre
DbVis Home: C:\Program Files\DbVisualizer
User Home: -------
PrefsDir: -------
SessionId: 55
BindDir: -------
Product: DbVisualizer Pro 11.0.5 [Build #3113]
OS: Windows 10
OS Version: 10.0
OS Arch: amd64
Java Version: 1.8.0_252
Java VM: OpenJDK 64-Bit Server VM
Java Vendor: AdoptOpenJDK
Java Home: c:\program files\dbvisualizer\jre
DbVis Home: C:\Program Files\DbVisualizer
User Home: -------
PrefsDir: -------
SessionId: 968
BindDir: -------
Any ideas how to make the program show me the real value, not the interpreted value of null?
The issue is explained in an IBM support document:
Problem
Trying to insert a date value into a date column before 1940 or after 2039 will represent the date as NULL within the respective database table.
Cause
This is caused by a limitation with the IBM i database Toolbox JDBC driver as detailed in the related link:
How does the Toolbox JDBC driver deal with dates before 1940 (or after 2039)?
The IBM i database supports several date formats. The Toolbox JDBC driver uses the date format that is set up as the default on the IBM i system. This default is usually set to "mdy" which only supports dates between 1940 and 2039. You can override the date format by specifying the "date format" property when opening the JDBC connection. The best choice is "iso", which supports a full four-digit date. The easiest way to do this is to add ";date format=iso" to the end of the URL used when connecting to the database.
Resolving The Problem
Appending the ";date format=iso" to the host connection property for the applicable database
via Preferences-> EGL-> SQL Database Connections will then show the respective dates correctly eg:
1939-01-01.
The issue can be fixed in DBVisualizer by doing the following:
Database -> Edit Database Connection(s)...
Select Properties tab
Select Driver Properties
Edit parameters date format and time format to be iso
Apply Changes
Disconnect and reconnect

PSQLEXCEPTION: Unexpected error code "55000" received from data source "FEDSER". Associated text and tokens are "This ResultSet is closed."

I am trying to do the differential data load from db2 to PostgreSQL table through InfoSphere Federation Server.
Followed below steps and got the expeption:
SQL1822N Unexpected error code "55000" received from data source "FEDSER".
Associated text and tokens are "This ResultSet is closed.".
Please find the below steps which I followed:
create wrapper jdbc
DB20000I The SQL command completed successfully.
CREATE SERVER FEDSER TYPE JDBC VERSION '12' WRAPPER JDBC OPTIONS( ADD DRIVER_PACKAGE 'E:\Sandhya\postgresql-8.1-415.jdbc3.jar', URL 'jdbc:postgresql://localhost:5432/SCOPEDB', DRIVER_CLASS 'org.postgresql.Driver', DB2_IUD_ENABLE 'Y', db2_char_blankpadded_comparison 'Y', db2_varchar_blankpadded_comparison 'Y', VARCHAR_NO_TRAILING_BLANKS 'Y', JDBC_LOG 'Y')
DB20000I The SQL command completed successfully.
CREATE USER MAPPING FOR SANAGARW SERVER FEDSER OPTIONS (REMOTE_AUTHID 'postgres',REMOTE_PASSWORD '*****')
DB20000I The SQL command completed successfully
SELECT COUNT(*) FROM "SCOPE".EMPLOYEE
SQL1822N Unexpected error code "55000" received from data source "FEDSER".
Associated text and tokens are "This ResultSet is closed.".
I am using Postgres version 12, Java version "1.8.0_241"
Please help me to resolve this issue. or once connection get created then I can only create the nickname.
Consider using Db2 11.5 instead of InfoSphere Federation Server which went out of support 2017-09-30 https://www.ibm.com/support/lifecycle/#/search?q=InfoSphere%20Federation%20Server
Db2 11.5 includes inbuilt support for PostgreSQL federation in all Db2 editions including Db2 Community Edition
https://www.ibm.com/support/pages/data-source-support-matrix-federation-bundled-db2-luw-v115

Liquibase - DB2 AS/400 - generateChangeLog throwing Exception "SYSCAT type *FILE not found"

I am trying to generate a Liquibase change log by running the command generateChangeLog, but I get the following error:
Starting Liquibase at Fri, 20 Apr 2018 14:26:14 GMT (version 3.6.1
built at 2018-04-11 08:41:04)
Unexpected error running Liquibase:
liquibase.exception.DatabaseException:
com.ibm.as400.access.AS400JDBCSQLSyntaxErrorException: [SQL0204]
REFERENCES in SYSCAT type *FILE not found.
liquibase.exception.LiquibaseException:
liquibase.command.CommandExecutionException:
liquibase.exception.DatabaseException:
liquibase.exception.DatabaseException:
com.ibm.as400.access.AS400JDBCSQLSyntaxErrorException: [SQL0204]
REFERENCES in SYSCAT type *FILE not found.
at
liquibase.integration.commandline.CommandLineUtils.doGenerateChangeLog
(CommandLineUtils.java:279)
at liquibase.integration.commandline.Main.doMigration(Main.java:1043)
at liquibase.integration.commandline.Main.run(Main.java:191)
at liquibase.integration.commandline.Main.main(Main.java:129)
My environment is:
DB2 UDB for AS/400 version 07.01.0000 V7R1m0
AS/400 Toolbox for Java JDBC Driver 10.2
liquibase 3.6.1
Java 8
Maven 3.3.3
syscat.references is not available on Db2 for i. Actually Db2 for i doesn't have a schema named SYSCAT at all.
qsys2.syscstdep seems to be the closest match...
INFORMATION_SCHEMA is the ANSI & ISO standard name for the DB catalog schema. On Db2 for i, it is an alias for QSYS2.
Edit
From the comment by #jmarkmurphy on the OP, it appears there's an Db2 for i Liquibase extension. I don't know for sure, but I'd suspect that installing that would redirect the query to the correct Db2 for i schema & table.

Need query to get function execution stats in SQL Server 2008 R2

Like below query, is there any query which can return the execution status of a function in SQL server using sys.dm_exec_?
SELECT TOP 1
d.object_id,
d.database_id,
OBJECT_NAME(object_id,database_id) 'proc name',
d.cached_time,d.last_execution_time, d.total_elapsed_time,
(d.total_elapsed_time/d.execution_count)/1000 AS [avg_elapsed_time],
d.last_elapsed_time/1000 as last_elapsed_time,
d.execution_count,
*
FROM
sys.dm_exec_procedure_stats AS d
WHERE
OBJECT_NAME(object_id, database_id) = 'ssp_StoredProcedureName'
ORDER BY
d.Last_Execution_Time DESC
You can't get exact function execution stats in versions below SQL2016.But from SQLSERVER 2016,we have sys.dm_exec_function_stats.
Applies to: SQL Server (SQL Server 2016 Community Technology Preview
3.2 (CTP 3.2) through current version), Azure SQL Database, Azure SQL Data Warehouse Public Preview.

Full-text search using Windows Search Service and SQL Server 2008 R2

Currently I'm trying to query the Windows Search Service from a SQL Server 2008 R2 instance (also tested on SQL Server 2012) . Windows Search is being exposed as an OLE DB datasource, giving me several options to query the search index. When configuring a new Linked Server in SQL Server, Management Studio gives me the option to select the Microsoft OLE DB Provider for Search, implying that I should be able to connect to it from SQL Server. It turns out to be a challenge to get this up and running however. Below you'll find the error message I stumbled upon.
OLE DB provider "Search.CollatorDSO" for linked server "TESTSERVER" returned message "Command was not prepared.".
Msg 7399, Level 16, State 1, Line 2
The OLE DB provider "Search.CollatorDSO" for linked server "TESTSERVER" reported an error. Command was not prepared.
Msg 7350, Level 16, State 2, Line 2
Cannot get the column information from OLE DB provider "Search.CollatorDSO" for linked server "TESTSERVER".
Things get even more interesting. Although the Linked Server solution isn't working, I'm able to wrap code that queries Windows Search in a CLR Function (using MSDN: Querying the Index Programmatically) and use if successfully within SQL Server. This is however less desirable, because of the steps needed to set it up (deploying the library, configuring permissions, etc.). I've tried several parameter settings, without any luck. I've also tried enabling some of the Search.CollatorDSO provider options, like allowing the provider to be instantiated as an in-process server. I'm currently using the settings below. For security I'm using the login's current security context.
Provider: Microsoft OLE DB Provider for Search
Data source: (local)
Provider string: Provider=Search.CollatorDSO.1;EXTENDED?PROPERTIES="Application=Windows"
Location: -
Additionally I need to search network drives, can this be done using shared Windows libraries?
I'm aware more people have been struggling with this problem over the last few years. I'm wondering if someone has been able to get this up and running, or could point me in the right direction.
OLEDB Works
Normal ADO/OLEDB components can query the Windows Search service with the connection string:
provider=Search.CollatorDSO.1;EXTENDED PROPERTIES="Application=Windows"
And an example query:
SELECT TOP 100000 "System.ItemName",
"System.ItemNameDisplay",
"System.ItemType",
"System.ItemTypeText",
"System.Search.EntryID",
"System.Search.GatherTime",
"System.Search.HitCount",
"System.Search.Store",
"System.ItemUrl",
"System.Filename",
"System.FileExtension",
"System.ItemFolderPathDisplay",
"System.ItemPathDisplay",
"System.DateModified",
"System.ContentType",
"System.ApplicationName",
"System.KindText",
"System.ParsingName",
"System.SFGAOFlags",
"System.Size",
"System.ThumbnailCacheId"
FROM "SystemIndex"
WHERE CONTAINS(*,'"Contoso*"',1033)
You can try the query directly on SQL Server in SQL Server Management Studio by attempting to run:
SELECT *
FROM OPENROWSET(
'Search.CollatorDSO',
'Application=Windows',
'SELECT TOP 100 "System.ItemName", "System.FileName" FROM SystemIndex');
Which gives the errors:
OLE DB provider "Search.CollatorDSO" for linked server "(null)" returned message "Command was not prepared.".
Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "Search.CollatorDSO" for linked server "(null)" reported an error. Command was not prepared.
Msg 7350, Level 16, State 2, Line 1
Cannot get the column information from OLE DB provider "Search.CollatorDSO" for linked server "(null)".
Bonus Reading
Connect to Windows Search from SQL Server using Linked Server (January 2011)
Link with SQL Server (May 2008 - December 2012)
Trying to access Windows Search from SQL Server: An Appeal (July 2007)
Calling Windows Search from SQL Server 2008 (March 2011)
OLE DB provider "Search.CollatorDSO" returns "Command was not prepared" (April 2014 - Suggests CLR workaround)
Linked Server to Windows Search (February 2011)
MSDN Blogs: Query to the SYSTEMINDEX to read the Microsoft search results fails when using Search.CollatorDSO provider (August 2009 - suggests CLR workaround)
Vista Search (February 2007)
Have a look at this code..It may help
USE [YourDB]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
Create PROC [dbo].[SearchAllTables]
#SearchStr nvarchar(100)
AS
BEGIN
DECLARE #dml nvarchar(max) = N''
IF OBJECT_ID('tempdb.dbo.#Results') IS NOT NULL DROP TABLE dbo.#Results
CREATE TABLE dbo.#Results
([tablename] nvarchar(100),
[ColumnName] nvarchar(100),
[Value] nvarchar(max))
SELECT #dml += ' SELECT ''' + s.name + '.' + t.name + ''' AS [tablename], ''' +
c.name + ''' AS [ColumnName], CAST(' + QUOTENAME(c.name) +
' AS nvarchar(max)) AS [Value] FROM ' + QUOTENAME(s.name) + '.' + QUOTENAME(t.name) +
' (NOLOCK) WHERE CAST(' + QUOTENAME(c.name) + ' AS nvarchar(max)) LIKE ' + '''%' + #SearchStr + '%'''
FROM sys.schemas s JOIN sys.tables t ON s.schema_id = t.schema_id
JOIN sys.columns c ON t.object_id = c.object_id
JOIN sys.types ty ON c.system_type_id = ty.system_type_id AND c .user_type_id = ty .user_type_id
WHERE t.is_ms_shipped = 0 AND ty.name NOT IN ('timestamp', 'image', 'sql_variant')
INSERT dbo.#Results
EXEC sp_executesql #dml
SELECT *
FROM dbo.#Results
END