Failure to find table when using multiple schemas in PostgreSQL - postgresql

WPF PostgreSQL 11.1
Npgsql.PostgresException: '42P01: relation "testme" does not exist'
When attempting to use a PostgreSQL database with multiple schemas, I have defined the following connection strings in the App.config. Note that the only difference is in the SearchPath:
<system.data>
<DbProviderFactories>
<add name="Npgsql Data Provider" invariant="Npgsql" support="FF" description=".Net Framework Data Provider for Postgresql Server" type="Npgsql.NpgsqlFactory, Npgsql, Version=4.0.4.0, Culture=neutral" />
</DbProviderFactories>
</system.data>
<connectionStrings>
<clear />
<add name="localconnection" providerName="Npgsql" connectionString="Server=127.0.0.1;Port=5432;Database=chaos;User Id=postgres;Password=****;Searchpath=nova" />
<add name="phoenixconnection" providerName="Npgsql" connectionString="Server=127.0.0.1;Port=5432;Database=chaos;User Id=postgres;Password=****;SearchPath=phoenix;" />
</connectionStrings>
The Npgsql data provider was installed using NuGet: Runtime Version:
v4.0.30319 Version: 4.0.4.0
In PostgreSQL, in the Phoenix schema:
CREATE TABLE phoenix.testme
(
name text COLLATE pg_catalog."default" NOT NULL
)
WITH (
OIDS = FALSE
)
TABLESPACE pg_default;
ALTER TABLE phoenix.testme
OWNER to postgres;
Using PgAdmin, displaying the testme table works without problem:
select * from phoenix.testme;
I have configured the WCF service using the above connection strings. Using PetaPoco, I write the following script:
public string SayHello()
{
string msg;
using (var db = new chaosDB("phoenixconnection"))
{
var m = db.ExecuteScalar<string>("select version()");
msg = string.Format("Hello from {0}", m);
m = db.ExecuteScalar<string>("select current_schema");
msg = string.Format("{0} Current Schema is {1}", msg, m);
var ss = db.ExecuteScalar<string>("show search_path");
var s = db.Fetch<string>("select * from testme"); <---THIS FAILS!
msg = string.Format("{0} I Am {1}", msg, m);
}
return msg;
}
All works correctly until the "select * from testme" is executed, when I receive the above error. Note: ss from "show search_path" returns correctly with "phoenix"
WHAT AM I DOING WRONG? How do I get this to work??
Any help is most appreciated?

After much head scratching the answer became self-evident. First I had reset the search_path in the database. This did not help. Then I rebuilt the POCO's with PetaPoco and quickly discovered that not only was the new table, "testme", not created, but nor were any POCO's. So, checking, the Database.tt file in PetaPoco showed it to have the wrong ConnectionStringName. Changing the ConnectionStringName to "phoenixconnection" allowed building the POCO's, but again failed to find the "testme" table.
Then the mistake became readily apparent, as stated above, both the "phoenixconnection" and the "localconnection" were pointed to the same port. From previous development, I had PostgreSQL v10.1 running on the same port as the newer PostgreSQL v11.1. Apparently, the first PostgreSQL v10.1 was receiving the connection (and not the newer PostgreSQL v11.1).
Going to services (services.msc) and shutting down v10.1 and running Database.TT now gave the error:
System.InvalidOperationException: Sequence contains more than one matching element
Apparently v10.1 (which I was using for development) only had ONE schema, but v11.1 has multiple schemas. I take the error message to mean that PetaPoco was seeing multiple tables with the same table name--i.e.,it was not distinguishing between schemas.
So, the problem is now solved.
Fix the ports! The older single-schema PostgreSQL v10.1 is kept on port: 5432.
The newer multiple-schema PostgreSQL is kept on port 5433. The v10.1 will be used for the POCO's.
Fix the connection strings in App.config of the WCF so that at run time, the WCF will use the newer v11.1. Once generated, LEAVE THE POCO'S alone and reference them in the WCF file.
Apparently, PetaPoco, can only work with one schema in generating its POCO's, but at runtime will read the connection strings from the App.Config of the WCF to execute its queries, etc. (So in the App.config where Database.TT resides, point PetaPoco to the "development" Database having only a single schema, but in the WCF environment, point the connection string to the new database with multiple schemas. The SearchPath of the connection string IS respected when running through Npgsql).
It would be nice if PetaPoco could generate POCO's specific to a schema in a multi-schema environment, but at the moment, I guess it can't :(
Addendum Note: It turns out that a given instance of PostgreSQL can have multiple DATABASES. So if the connection string for Npgsql is specific to a development database --i.e., a database with only one schema--then during development, PetaPoco works great to create the POCO's. These POCO's can then be directly used in a WCF Service project and uploaded to IIS website. The App.config files of the web site can then be directed to use the run-time database (again in the connection string) to the deployed database. All works well! :)

Related

SocketException thrown when using linq for Npgsql EntityFramework6 model

After much effort getting my MVC5 vb.net app configured with Npgsql and EntityFramework6.Npgsql I can connect to the postgres database fine using the Entity Data Model Wizard in Visual Studio 2017, creating a new connection with Data Provider of PostgreSQL Database, entering my database server details and testing the connection (succeeds). I can also double-click the EDMX file and add tables, etc as expected. This is my first project that talks to a postgreSQL database (normally use MS SQL Server). Everything seems to be happy and fine until I encounter code that is a linq statement that attempts to query the database. My Linq statement is:
Dim l_ListOfPwrCycleCrashEvents As List(Of pwr_cycle_crash_events) = (From item In m_AnalyticsEntities.pwr_cycle_crash_events Where item.serialnumber = "0123445678" Select item).ToList
I have the statement in a try/catch block, but visual studio still reports the following:
Exception thrown:'System.Net.Sockets.SocketException' in System.dll Additional
information: A non-blocking socket operation could not be completed
immediately occurred
I cannot seem to figure out why this statement fails. I am able to use similar statements against a different EntityFramework (MS SQL Server) that is also configured in the application.
The complete method that contains the linq statement:
Public ReadOnly Property iCountOfPwrCycleCrashEvents(
ByVal a_sSerialNumber As String
) As Integer Implements IAnalytics.iCountOfPwrCycleCrashEvents
Get
Try
Dim l_ListOfPwrCycleCrashEvents As List(Of pwr_cycle_crash_events) = (From item In m_AnalyticsEntities.pwr_cycle_crash_events
Where item.serialnumber = a_sSerialNumber
Select item).ToList
If (l_ListOfPwrCycleCrashEvents IsNot Nothing) Then Return l_ListOfPwrCycleCrashEvents.Count
Catch l_Exception2 As System.Net.Sockets.SocketException
Dim askjfhakjh As Integer = 7
Catch l_Exception As Exception
Dim kjhadkjh As Integer = 1
End Try
Return 0
End Get
End Property
I am using Npgsql v4.1.1 and EntityFramework6.Npgsql v6.3.0 in an application that targets .NET Framework 4.5.1.
I can't seem to figure out why the linq statement throws this exception or how to solve the problem. What suggestions do you have?
I figured out the issue. I was using 4.5.1 framework because I thought I had a dependency that would not allow me to move to 4.5.2. I went to Package Manager Console and ran the following command:
Update-Package -ProjectName myProjectName -reinstall
By doing so, I found that I had a package that was needed but could not be loaded because it required the 4.5.2 framework (or higher). I changed the framework of the app to 4.5.2 and issued the above Package Manager command and it executed without errors. I ran the app and the linq statement no longer throws an exception.

Define a database entry in DB2 db2dsdriver.cfg file that can access multiple databases

I currently use the CLI method of registering a node and then a database for that node (with CATALOG NODE / CATALOG DATABASE) to configure my DB2 CLI client to access our database server.
With a single registration of a database I can effectively register the default database but then when I connect in my application using SQLDriverConnect and use the "DATABASE=" option I can connect to other databases available on my server.
I would like to switch to the much easier to manage db2dsdriver.cfg configuration file, however I have been unable to configure it to allow a single to access multiple databases.
Some code to help clarify. My DB2 server instance has two databases defined like:
CREATE DATABASE DB_1 ON /opt/data/DB_1 USING CODESET UTF-8 TERRITORY US COLLATE USING SYSTEM
CREATE DATABASE DB_2 ON /opt/data/DB_2 USING CODESET UTF-8 TERRITORY US COLLATE USING SYSTEM
I register this server with my client CLI using these commands:
CATALOG TCPIP NODE DB_NODE remote example.server.com server 50000
CATALOG DB DB_1 as DB_1 at node DB_NODE
With that setup I can perform the following from my CLI application:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_1",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
or if I want to use the DB_2 database:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_2",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
Note I did not need to change the DSN, merely the "DATABASE" connection option.
Recently I found the db2dsdriver.cfg configuration file which I would rather use. To that end I created this and uncataloged my node and db from the cli:
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<configuration>
<dsncollection>
<dsn alias="DB_1" name="DB_1" host="server.example.com" port="50000"/>
</dsncollection>
<databases>
<database name="DB_1" host="server.example.com" port="50000"/>
<database name="DB_2" host="server.example.com" port="50000"/>
</databases>
</configuration>
I can connect with this:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_1",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
just fine, but now using this to connect to DB_2:
rc = SQLDriverConnect(hdbc, NULL, "DSN=DB_1;UID=dbtest1;PWD=zebco5;DATABASE=DB_2",
SQL_NTS, outStr, 128, &outSize, SQL_DRIVER_NOPROMPT);
results in this error:
SQL0843N The server name does not specify an existing connection. SQLSTATE=08003
Which I understand, but seems like a regression in functionality from the old node/db registration mechanism.
I'm attempting to determine if the functionality I was using is supported with the configuration file and I am doing something wrong or if it just doesn't work that way?
Thank you for your help.

DB2 exception: Cannot create PoolableConnectionFactory SQLCODE=-142,

I keep having the following error in MobileFirst Platform 6.3:
Runtime: org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (DB2 SQL Error: SQLCODE=-142,
SQLSTATE=42612, SQLERRMC=null, DRIVER=4.19.26)
This is my adapter code:
var test2 = WL.Server.createSQLStatement("SELECT * FROM WSDIWC.WBPTRR1");
function getCEID(cnum) {
return WL.Server.invokeSQLStatement({
preparedStatement : test2,
parameters : []
});
}
And adapter XML:
<connectivity>
<connectionPolicy xsi:type="sql:SQLConnectionPolicy">
<!-- Example for using a JNDI data source, replace with actual data source
name -->
<!-- <dataSourceJNDIName>${training-jndi-name}</dataSourceJNDIName> -->
<!-- Example for using MySQL connector, do not forget to put the MySQL
connector library in the project's lib folder -->
<dataSourceDefinition>
<driverClass>com.ibm.db2.jcc.DB2Driver</driverClass>
<url>jdbc:db2://***</url>
<user>**</user>
<password>**</password>
</dataSourceDefinition>
</connectionPolicy>
</connectivity>
I have remove the url, user and password.
Hope you help me out on clarification about the current problem.
I already know that the sql is not accepted since it's just a simple query.
I have also research about z/OS DB2 that it has issue with same error code sqlcode=-142. http://answers.splunk.com/answers/117024/splunk-db-connect-db2.html
While you say that this is a "simple query", the exception error code mentions the following:
-142
THE SQL STATEMENT IS NOT SUPPORTED
Explanation
An SQL statement was detected that is not supported by the database.
The statement might be valid for other IBM® relational database
products or it might be valid in another context. For example,
statements such as VALUES and SIGNAL or RESIGNAL SQLSTATE can be used
only in certain contexts, such as in a trigger body or in an SQL
Procedure.
System action
The statement cannot be processed.
Programmer response
Change the syntax of the SQL statement or remove the statement from
the program.
You should review the DB2 SQL guidelines for how to achieve what you want to achieve, and also explain that in the question if you'd like further assistance. For example, are you sure "WSDIWC.WBPTRR1" is actually available?
I encountered this same problem with JDBC connections to mainframe DB2 in MobileFirst 6.3. Connections to DB2 LUW worked fine. It appears that default pool validationQuery is valid for DB2 LUW but not DB2 z/OS.
You can work around the bug by doing the data source configuration in the Liberty profile server.xml. From the Eclipse Servers view, expand MobileFirst Development Server and edit the Server Configuration. Add the driver and data source there; for example:
<library id="db2jcc">
<fileset dir="whereever" includes="db2jcc4.jar db2jcc_license_cisuz.jar"/>
</library>
<dataSource id="db2" jndiName="jdbc/db2">
<jdbcDriver libraryRef="db2jcc"/>
<properties.db2.jcc databaseName="mydb" portNumber="5021"
serverName="myserver" user="myuser" password="mypw" />
</dataSource>
Then reference it in your adapter XML under connectionPolicy:
<dataSourceJNDIName>jdbc/db2</dataSourceJNDIName>
A benefit of configuring data sources in server.xml (vs the adapter XML) is you have access to all data source, JDBC, and JCC properties. So if the connection pool gives you other problems, you can customize it or switch to another data source type, such as type="javax.sql.DataSource".

Getting DB2 SQL Error while firing a select query using Worklight 6.1.0?

I am trying to connect to DB2 in my local LAN using worklight 6.1.0 and firing a Select Query for lookup of data if exist. But i am getting below error:
{
"errors": [
"Runtime: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=DATABASE_NAME.REGISTRATION, DRIVER=3.58.82.\nPerformed query:\nSELECT * FROM DATABASE_NAME.registration where DATABASE_NAME.registration.Mob_No = ?"
],
"info": [
],
"isSuccessful": false,
"warnings": [
]
}
My SQL adapter configuration looks like below:
<connectionPolicy xsi:type="sql:SQLConnectionPolicy">
<!-- Example for using a JNDI data source, replace with actual data source name -->
<!-- <dataSourceJNDIName>java:/data-source-jndi-name</dataSourceJNDIName> -->
<!-- Example for using MySQL connector, do not forget to put the MySQL connector library in the project's lib folder -->
<dataSourceDefinition>
<driverClass>com.ibm.db2.jcc.DB2Driver</driverClass>
<url>jdbc:db2://172.21.11.129:50000/MOBILEDB</url>
<user>db2admin</user>
<password>Newuser123</password>
</dataSourceDefinition>
</connectionPolicy>
And js file which has procedure looks like:
var selectStatement1 = "SELECT * FROM DATABASE_NAME.registration where DATABASE_NAME.registration.Mob_No = ?";
var procStmt1 = WL.Server.createSQLStatement(selectStatement1);
function registrationLookup(mobile){
WL.Logger.debug("Inside registrationLookup");
return WL.Server.invokeSQLStatement(
{
preparedStatement : procStmt1,
parameters : [mobile]
}
);
}
I did some Research about connecting DB2 with Worklight and came to know that i need to put below data in worklight.properties file.
wl.db.username=db2admin
wl.db.type=DB2
wl.db.password=Newuser123
wl.db.driver=com.ibm.db2.jcc.DB2Driver
But after adding it, i am not able to deploy Adapter and error says 'db2admin' does not exist. So i have skipped this step in the context of current question. But after going through error which i am getting without adding this worklight.properties data it seems to me that 'Object doesn't exist' as per http://www-01.ibm.com/support/docview.wss?uid=swg21613531 or user table does not exist. Any suggestion would be helpful.
NOTE:
My IP address is 172.21.11.125 from where i am invoking Adapter for DB2.
DB2 instance is running on 172.21.11.129 # 50000.
Already Added db2jcc_license_cu_9.5.jar & db2jcc_9.5.jar in server/lib. It had name appended with '_9.5' which i have removed from both jar and kept only db2jcc_license_cu.jar and db2jcc.jar.
The error message is saying that your SQL statement is invalid, and I therefore infer that your connection to the DB is fine.
To diagnose this first run the SQL using DB2 command line or other tools. My guess is you mean
LARSEN.registration
whereas you are saying DATABASE_NAME.registration
Got Answer to my own Question and its really interesting one. Thanks to https://stackoverflow.com/users/2260967/glen-misquith [Glen Misquith]
Problem was with SQL Query framing which is Different what i did with MYSQL.
Adapter's Java Script file:
var selectStatement4 = "UPDATE \"LARSEN\".\"registration\" SET \"LARSEN\".\"registration\".\"Pass\"=?, \"LARSEN\".\"registration\".\"Re_Pass\"=? WHERE \"User_Name\" = ?";
var procStmt5 = WL.Server.createSQLStatement(selectStatement4);
function updatePassword(username,pass,repass){
WL.Logger.debug("Inside updatePassword "+username+" "+pass+" "+repass);
return WL.Server.invokeSQLStatement(
{
preparedStatement : procStmt5,
parameters : [pass,repass,username]
}
);
}
This is quite a Strange thing Slashes need to be used in SQL Statement while Preparing it.
I would really like to understand this behavior of DB2. And also i cant directly write 'Select * from schema.table_name' and precisely write column name from which data needs to be fetched.

can't change connection string (ODP, Oracle, Linq to Entities)

I have a App.config connection string that connects me to my database just fine:
<add name="SFEntities" connectionString="metadata=res://*/Model1.csdl|res://*/Model1.ssdl|res://*/Model1.msl;provider=Oracle.DataAccess.Client;provider connection string="DATA SOURCE=micahs_poc_dev_server.kci;PASSWORD=kierkegaard;PERSIST SECURITY INFO=True;USER ID=KIERKEGAARD"" providerName="System.Data.EntityClient" />
I am using an ODP reference in conjunction with an ADO.NET / Linq to Entities approach.
The trouble comes when I want to connect to a different database (with the same schema) on our network.
<add name="SFEntities" connectionString="metadata=res://*/Model1.csdl|res://*/Model1.ssdl|res://*/Model1.msl;provider=Oracle.DataAccess.Client;provider connection string="DATA SOURCE=QA_ETL_TEST_SERVER.COM;PASSWORD=bigshotpassword;PERSIST SECURITY INFO=True;USER ID=CoolidgeCalvin"" providerName="System.Data.EntityClient" />
When I just change the name of the server I get this error:
ORA-00942: table or view does not exist
I can actually run several lines of code before it throws an exception:
using (SFEntities ctx1 = new SFEntities())
{
var ds = ctx1.Connection.DataSource; // debugger: QA_ETL_TEST_SERVER.COM
var db = ctx1.Connection.Database;
var dstate = ctx1.Connection.State;
var dsite = ctx1.Connection.Site;
ctx1.Connection.Open();
SF_CHANGE_ORDER cotest = new SF_CHANGE_ORDER();
cotest.DELETE_FLAG = "D";
var result = (from cp in ctx1.SF_CLIENT_PROJECT
select cp).ToList(); // <--- error here!
I also tried changing the format to this:
<add name="SFEntities" connectionString="metadata=res://*/Model1.csdl|res://*/Model1.ssdl|res://*/Model1.msl;provider=Oracle.DataAccess.Client;provider connection string="DATA SOURCE=(DESCRIPTION =(ADDRESS=(PROTOCOL=TCP)(HOST=155.32.75.11)(PORT=65550))(ADDRESS=(PROTOCOL=TCP)(HOST=155.32.75.11)(PORT=65550))(CONNECT_DATA=(SERVICE_NAME=QA_ETL_TEST_SERVER.COM)(FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC)(RETRIES=180)(DELAY=5))));PASSWORD=bigshotpassword;PERSIST SECURITY INFO=True;USER ID=CoolidgeCalvin"" providerName="System.Data.EntityClient" />
Which basically mirrors the TNS entry I have for this server. I still get the 'does not exist' error.
If I make the server name something goofy I get a TNS error saying that server isn't recognized. So I am pretty sure it is picking up the server name I want and checking to see it is there.
I don't think it is a table permissions/creation issue because I can query them just fine on both databases from the same machine with the same credentials (and same TNS file).
[note: I changed the server/user/pw in the strings]
[edit: tried bringing up fiddler, but I didn't see any messages pass through]
Ah ... VisualStudio tucks away the schema name in the Model .edmx file. To make it work you have to somehow open the model file (use notepad or search all documents in the project) and manually change the schema name. Boo!