I'm using PostgreSQL 9.5 and Npgsql 2.2.5.0 to connect from an ASP.NET web service written with VB.NET to the PostgreSQL server.
The webservice gets around 15 requests a second.
The connection string I use is:
<connectionStrings>
<add name="PostgreConnectionStringPCMRead"
connectionString="Server=xxx.xxx.xxx.xxx;Port=5432;User
Id=the_user;Password=the_password;Database=example;
CommandTimeout=600;Timeout=30;ConnectionLifeTime=30;
Pooling=False;"/>
</connectionStrings>
When I take a look at the Windows Resource Monitor, I see hundreds of Terminated connections.
I realise that I am using Pooling=False, however when I set Pooling=True, I get errors "Timeout while getting a connection from pool."
With every query I use the following Class (I've left in only the essential code):
Public Class dlNpgSQL
Dim _sqlCommand As NpgsqlCommand
Dim _sqlDataAdapter As NpgsqlDataAdapter
Dim _dataset As DataSet
Public Sub New()
_sqlConnection = New NpgsqlConnection(ConfigurationManager.
ConnectionStrings("connstring").ConnectionString)
End Sub
Public Function GetDataSet(ByVal strQuery As String) As DataSet
_dataset = New DataSet
_sqlDataAdapter = New NpgsqlDataAdapter(strQuery, OpenConnection)
_sqlDataAdapter.Fill(_dataset)
Return _dataset
End Function
Public Function OpenConnection() As NpgsqlConnection
If _sqlConnection.State = ConnectionState.Closed Then
_sqlConnection.Open()
End If
End Function
Public Sub CloseConnection()
If _sqlConnection.State = ConnectionState.Open Then
_sqlConnection.Close()
End If
End Function
End Class
And then the actual query:
Dim ds As DataSet
Dim objDBRead As New dlNpgSQL("connstring")
tmpSQL = "SELECT * FROM table1"
ds = objDBRead.GetDataSet(tmpSQL)
If (objDBRead IsNot Nothing) Then
objDBRead.CloseConnection()
End If
So obviously, I'm opening and closing the connection with each query.
Is this the correct way to do things?
Should I be using pooling, and if yes, why am I getting the errors and how can I fix the issue?
Thanks!
Related
I have copied the example from this post here at
Visual Basic (VB.NET) example code to send SendGrid email by a VB.NET windows app
to get this working in my vb website. I was able to get this to debug by changing the Module to a Class. It would not work as a module for me. I have uploaded it and placed my key into the web.config file. When I click the button at my live site, I don't receive an email. Also, when I check at Sendgrid.com for email stats, everything is zeroed out with no record of receiving any emails to forward. Here is my code:
Imports System
Imports System.Collections.Generic
Imports System.Linq
Imports System.Text
Imports System.Threading.Tasks
Imports System.Configuration
Imports SendGrid
Imports SendGrid.Helpers.Mail
Class sendgrid1
Sub Main(ByVal test As String)
TestSendGrid(test).Wait()
End Sub
Private Async Function TestSendGrid(ByVal test As String) As Task
Try
Dim apiKey = ConfigurationManager.AppSettings("ApiKey")
Dim client = New SendGridClient(apiKey)
Dim msg = New Helpers.Mail.SendGridMessage() With {
.From = New EmailAddress("admin#pacificwestcapital.net", test),
.Subject = "Hello World from the SendGrid VB.NET SDK!",
.PlainTextContent = "Hello, Email!",
.HtmlContent = "<strong>Hello, Email!</strong>"
}
msg.AddTo(New EmailAddress("pacwestcapital#aol.com", "Scot King"))
Dim response = Await client.SendEmailAsync(msg)
Catch ex As Exception
Console.WriteLine(ex.Message)
End Try
End Function
End Class
Partial Class admin_sendgrid
Inherits System.Web.UI.Page
Protected Sub Page_Load(sender As Object, e As EventArgs) Handles Me.Load
End Sub
Protected Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim sg As New sendgrid1
Call sg.Main("Scot King")
End Sub
End Class
Can someone possibly explain why it isn't working?
I've been using npgsql for a few years now.
I've never had an issue until recently when the load on my PostgreSQL server increased dramatically.
At the moment I have hundreds of instances of postgres.exe "Running" and "Terminating" on my server every second.
When I switch on pooling, it doesn't make a difference to the amount of instances being created / terminated and after a few minutes I get an error:
The connection pool has been exhausted, either raise MaxPoolSize (currently 100) or Timeout (currently 15 seconds)
I'm now questioning if my method of using npgsql is correct.
I used nuget to add npgsql into my project and created the following class which is used every time that I run a query:
Public Class dlNpgSQL
Dim _sqlCommand As NpgsqlCommand
Dim _sqlDataAdapter As NpgsqlDataAdapter
Dim _dataset As DataSet
Public Sub New()
_sqlConnection = New NpgsqlConnection(ConfigurationManager.
ConnectionStrings("connstring").ConnectionString)
End Sub
Public Function GetDataSet(ByVal strQuery As String) As DataSet
_dataset = New DataSet
_sqlDataAdapter = New NpgsqlDataAdapter(strQuery, OpenConnection)
_sqlDataAdapter.Fill(_dataset)
Return _dataset
End Function
Public Function OpenConnection() As NpgsqlConnection
If _sqlConnection.State = ConnectionState.Closed Then
_sqlConnection.Open()
End If
End Function
Public Sub CloseConnection()
If _sqlConnection.State = ConnectionState.Open Then
_sqlConnection.Close()
End If
End Function
End Class
I run queries as following:
Dim ds As DataSet
Dim objDB As New dlNpgSQL("connstring")
tmpSQL = "SELECT * FROM table1"
Try
ds = objDB.GetDataSet(tmpSQL)
Catch ex As Exception
If (objDB IsNot Nothing) Then
objDB.CloseConnection()
End If
Return Nothing
Finally
If (objDB IsNot Nothing) Then
objDB.CloseConnection()
End If
End Try
The connection string I use is:
<connectionStrings>
<add name = "connstring" connectionString="Server=localhost;Port=5432;User Id=postgres;Password=password;Database=test_db;CommandTimeout=600;Pooling=True;"/>
</connectionStrings>
So my questions are:
Am I using npgsql in the correct way?
If so, why are so many instances of postgres.exe being created?
Why do I get the error about the pool being exhausted?
Is there a way to see what connections are in the pool? When I check pg_stat_activity, there are no open connections.
I've been battling with this for days - any assistance will be greatly appreciated!
PREFACE: I am using SQL Server 2008 R2 BackEnd and MS Access 2007 for the FrontEnd
I have a Class Module that returns any ADO Recordset I want from the SQL Server. I can then assign this to any form RecordSource property.
The problem is that when I try to edit the fields it says "This form is read-only" in the status bar. I want the form to be editable.
I have two forms
FormEntities
FormEntitiesEdit
The FormEntitiesEdit form does NOT use the Class Module. Rather all the code is in the form itself.
The purpose of the class module is avoid redundancy and just be able to use the Class Module to get a recordset from SQL Server easily.
FIRST HERE IS MY GLOBAL MODULE
'Default error message. 'eh' stands for error handler
Public eh As String
'Global variables for universal use
Public conn As ADODB.Connection
Public rs As ADODB.Recordset
Public com As ADODB.Command
SECOND IS THE CLASS MODULE (Name is cADO).
THIS CLASS MODULE USES THE conn CONNECTION OBJECT ABOVE
Option Explicit
Private Const CONST_LockType = 3
Private Const CONST_CursorType = 1
Private Const CONST_CursorLocationServer = 3
Private Const CONST_CursorLocationClient = 2
Private m_Recordset As ADODB.Recordset
'For Public Recordset function
Private cSQL$
'**********************************************************************
Public Function cGetRecordset(ByRef sql) As ADODB.Recordset
Set m_Recordset = New ADODB.Recordset
cSQL = sql
cOpenRecordset
Set cGetRecordset = m_Recordset
End Function
'**********************************************************************
Public Property Set Recordset(Value As ADODB.Recordset)
'Assigns private variable a property
If Not Value Is Nothing Then Set m_Recordset = Value
End Property
'**********************************************************************
Public Property Get Recordset() As ADODB.Recordset
'Reads the recordset from the private variable and assigns to new object variable
Set Recordset = m_Recordset
End Property
'********************************** PRIVATE SECTION **********************************
Private Sub cOpenRecordset()
On Error GoTo eh
'Ensures that if a recordset is opened from previously that it closes before opening a new one
If m_Recordset.State adStateClosed Then m_Recordset.Close
Set m_Recordset.ActiveConnection = conn
With m_Recordset
.LockType = CONST_LockType
.CursorType = CONST_CursorType
.CursorLocation = CONST_CursorLocationClient
.Source = cSQL
.Open .Source
End With
If Not m_Recordset.EOF Then m_Recordset.MoveFirst
Exit Sub
eh:
eh = "Error # " & Str(Err.Number) & " was generated by " & _
Err.Source & Chr(13) & Err.Description
MsgBox eh, vbCritical, "Open Recordset System"
End Sub
'**********************************************************************
Private Sub cCloseRecordset()
m_Recordset.Close
Set m_Recordset = Nothing
End Sub
'**********************************************************************
Private Sub Class_Terminate()
If Not (m_Recordset Is Nothing) Then Set m_Recordset = Nothing
End Sub
THIRD IS THE CODE BEHIND MY FormEntities FORM (USES THE THE cADO CLASS MODULE)
Option Explicit
Dim db As cADO
'**********************************************************************
Private Sub Form_Current()
LoadTab
End Sub
'**********************************************************************
Private Sub Form_Load()
Set db = New cADO
FetchRecordSource
End Sub
'**********************************************************************
Private Sub FetchRecordSource()
db.cGetRecordset ("SELECT * FROM dbo.Entities")
Set Forms("fEntities").Recordset = db.Recordset
End Sub
FOURTH AND FINALLY IS THE CODE BEHIND THE FormEntitiesEdit FORM (THIS FORM DOES NOT USE THE CLASS MODULE AND I CAN EDIT IT)
Option Compare Database
Option Explicit
Dim rsEntity As New ADODB.Recordset
'**********************************************************************
Private Sub Form_Load()
FetchRecordSource
End Sub
'**********************************************************************
Private Sub FetchRecordSource()
Set rsEntity.ActiveConnection = conn
'Sets the record source for the main form
With rsEntity
.LockType = adLockOptimistic
.CursorType = adOpenKeyset
.CursorLocation = adUseClient
.Source = "SELECT * FROM dbo.Entities"
.Open .Source
End With
Set Forms("fEntitiesEdit").Recordset = rsEntity
End Sub
'**********************************************************************
Private Sub CloseConn()
rsEntity.Close
End Sub
If Access Jet SQL could do the SQL I would bind this form to a Linked Table instead. However I am using a hierarchical (recursive) query which Jet SQL cannot do so I have to bypass the idea of bound forms to Linked Tables.
I have .Allow Edits and .AllowAdditions on the form set to true.
I also tried changing the .LockType on the ADO Recordset to
adOpenKeyset and
adOpenDynamic
My LockType is adLockOptimistic
As you can see the properties are set correctly to be able to edit the recordset I return.
I know this is true because when I use the FormEntitiesEdit form with the same properties it lets me edit the field. But when I use the Class Module to return (using the same properties) it says it's read-only.
This is important because as you can see it is a lot simpler to use the Class Module, just need it to return an editable recordset.
Anyone have ideas? suggestions?
The problem is here in the class' cOpenRecordset() method:
.CursorLocation = CONST_CursorLocationClient
Here is where you assign constant values ...
Private Const CONST_CursorLocationServer = 3
Private Const CONST_CursorLocationClient = 2
Unfortunately, you swapped those two values. Here are the ADODB.CursorLocationEnum constants ...
adUseClient = 3
adUseServer = 2
So your class is effectively doing this ...
.CursorLocation = adUseServer
But you need a client-side cursor if you want the recordset to be editable. I think your class approach may work if you simply redefine the constant, or you will expose a different problem ...
Private Const CONST_CursorLocationClient = 3
FormEntitiesEdit is editable because you're using the correct constant there ...
.CursorLocation = adUseClient
What's the right way to run a EF6 stored procedure (database-first) in async mode?
I read about ToListAsync() but I don't see that available on stored procedure.
Also not sure if there is a different way to call the stored procedure when the actual call returns (#1) an OUT param or a (#2) list of items:
Case #1
using (DBContext db = new DBContext())
{
ObjectParameter result = new ObjectParameter("Result",
typeof(global::System.Boolean));
db.Login("email#email.com", "password", result);
}
Case #2
using (DBContext db = new DBContext())
{
var result = db.Contact_GetList("New York");
}
Thanks for the help
As per this workitem you would need to use SqlQueryAsync. Feel free to upvote the work item on the EF Codeplex site.
To map stored procedures and start using it with out writing any initial code, this is how I did it.
create a new model with a new connection string that will generate the connection string automatically in the web.config file where the connection string is at (if you use a current connection string it may no work when you test the function for the SP on the model browser).
map your table and the stored procedures (you can test the stored procedures in the model browser).
create classes that represents the attributes retrieved by each stored procedure e.g if your stored procedure returns three columns A,B,C, then the class must also have these three columns as attribute with the [key()] on top of the column that is going to be the PK
now create your controller with the class created and a new DbContext
then copy the information in the data context generated for the model and pasted in the new context that you generate when creating the controller.
when you want to use the store procedures they will be ready on the db.context because you paste their code on you new db-context that you create when the controller was crated.
NOTE: I hope this is not confusing but I can use the stored procedures with out typing any code, please ask me if you need sample code or screen shots, your new db-context will not over write after you created
This is the stored procedure I map
'--------------------------------------------------------------------------
' <auto-generated>
' This code was generated from a template.
'
' Manual changes to this file may cause unexpected behavior in your application.
' Manual changes to this file will be overwritten if the code is regenerated.
' </auto-generated>
'-----------------------------------------------------------------------
Imports System
Imports System.Collections.Generic
Partial Public Class phone_CurrentConferences_Result
Public Property AppointmentID As Integer
Public Property AppTitle As String
Public Property DateTime As Nullable(Of Date)
Public Property [Date] As String
Public Property Time As String
Public Property Company As String
Public Property Contact As String
Public Property Phone As String
Public Property Office As String
Public Property Lead_Director As String
Public Property TBD As Nullable(Of Boolean)
Public Property conference As String
End Class
This is the same model with a primary key
Imports System
Imports System.Collections.Generic
Imports System.ComponentModel.DataAnnotations
Public Class Conferences
[Key]
Public Property AppointmentID As Integer
Public Property AppTitle As String
Public Property DateTime As Nullable(Of Date)
Public Property [Date] As String
Public Property Time As String
Public Property Company As String
Public Property Contact As String
Public Property Phone As String
Public Property Office As String
Public Property Lead_Director As String
Public Property TBD As Nullable(Of Boolean)
Public Property conference As String
End Class
This is the context generated by the EF
'--------------------------------------------------------------------------
' <auto-generated>
' This code was generated from a template.
'
' Manual changes to this file may cause unexpected behavior in your application.
' Manual changes to this file will be overwritten if the code is regenerated.
' </auto-generated>
'--------------------------------------------------------------------------
Imports System
Imports System.Data.Entity
Imports System.Data.Entity.Infrastructure
Imports System.Data.Entity.Core.Objects
Imports System.Linq
Partial Public Class DayMasterEntities
Inherits DbContext
Public Sub New()
MyBase.New("name=DayMasterEntities")
End Sub
Protected Overrides Sub OnModelCreating(modelBuilder As DbModelBuilder)
Throw New UnintentionalCodeFirstException()
End Sub
Public Overridable Function phone_CurrentConferences(number As String, [date] As Nullable(Of Date)) As ObjectResult(Of phone_CurrentConferences_Result)
Dim numberParameter As ObjectParameter = If(number IsNot Nothing, New ObjectParameter("number", number), New ObjectParameter("number", GetType(String)))
Dim dateParameter As ObjectParameter = If([date].HasValue, New ObjectParameter("date", [date]), New ObjectParameter("date", GetType(Date)))
Return DirectCast(Me, IObjectContextAdapter).ObjectContext.ExecuteFunction(Of phone_CurrentConferences_Result)("phone_CurrentConferences", numberParameter, dateParameter)
End Function
End Class
SO, when I create the controller I use the model with the <KEY()> and I create my own context that will look like this
Imports System.Data.Entity
Imports System.Data.Entity.Infrastructure
Imports System.Data.Entity.Core.Objects
Namespace Models
Public Class DayMasterContext
Inherits DbContext
' You can add custom code to this file. Changes will not be overwritten.
'
' If you want Entity Framework to drop and regenerate your database
' automatically whenever you change your model schema, please use data migrations.
' For more information refer to the documentation:
' http://msdn.microsoft.com/en-us/data/jj591621.aspx
Public Sub New()
MyBase.New("name=DayMasterEntities")
End Sub
Protected Overrides Sub OnModelCreating(modelBuilder As DbModelBuilder)
Throw New UnintentionalCodeFirstException()
End Sub
Public Property Conferences As System.Data.Entity.DbSet(Of Conferences)
End Class
End Namespace
Then I copy the info in the context generated by the EF to my context
Imports System.Data.Entity
Imports System.Data.Entity.Infrastructure
Imports System.Data.Entity.Core.Objects
Namespace Models
Public Class DayMasterContext
Inherits DbContext
' You can add custom code to this file. Changes will not be overwritten.
'
' If you want Entity Framework to drop and regenerate your database
' automatically whenever you change your model schema, please use data migrations.
' For more information refer to the documentation:
' http://msdn.microsoft.com/en-us/data/jj591621.aspx
Public Sub New()
MyBase.New("name=DayMasterEntities")
End Sub
Protected Overrides Sub OnModelCreating(modelBuilder As DbModelBuilder)
Throw New UnintentionalCodeFirstException()
End Sub
Public Overridable Function phone_CurrentConferences(number As String, [date] As Nullable(Of Date)) As ObjectResult(Of phone_CurrentConferences_Result)
Dim numberParameter As ObjectParameter = If(number IsNot Nothing, New ObjectParameter("number", number), New ObjectParameter("number", GetType(String)))
Dim dateParameter As ObjectParameter = If([date].HasValue, New ObjectParameter("date", [date]), New ObjectParameter("date", GetType(Date)))
Return DirectCast(Me, IObjectContextAdapter).ObjectContext.ExecuteFunction(Of phone_CurrentConferences_Result)("phone_CurrentConferences", numberParameter, dateParameter)
End Function
Public Property Conferences As System.Data.Entity.DbSet(Of Conferences)
End Class
End Namespace
So, now you can use this context to query
entConferences(number As String, [date] As Nullable(Of Date)) As ObjectResult(Of phone_CurrentConferences_Result)
or to get a DBSet(of conferences)
Here is a controller I have created with this technique
Look where I call my stored procedure
Dim conferences = db.phone_CurrentConferences(phoneNumber, currentDate)
Imports System.Data
Imports System.Data.Entity
Imports System.Data.Entity.Infrastructure
Imports System.Linq
Imports System.Net
Imports System.Net.Http
Imports System.Web.Http
Imports System.Web.Http.Description
Imports BIWEBAPI
Imports BIWEBAPI.Models
Namespace Controllers.DayMasterControllers
Public Class ConferencesController
Inherits System.Web.Http.ApiController
Private db As New DayMasterContext
' GET: api/Conferences
Function GetConferences() As IQueryable(Of Conferences)
Return db.Conferences
End Function
' GET: api/Conferences/3053742500
''' <summary>
''' Use to get the current conferences selected by date
''' </summary>
''' <param name="id">phone number and date separated by coma ",""</param>
''' <returns>conferences by date</returns>
''' <remarks></remarks>
<ResponseType(GetType(Conferences))>
Function GetConferences(ByVal id As String) As List(Of Conferences)
Dim conferencelist = New List(Of Conferences)
Dim dateAndPhoneNumber = Split(id, ",")
Dim currentDate = ""
Dim phoneNumber = dateAndPhoneNumber(0)
If dateAndPhoneNumber.Length > 1 Then
currentDate = DateTime.Parse(dateAndPhoneNumber(1))
Else : currentDate = DateTime.Today
End If
Dim conferences = db.phone_CurrentConferences(phoneNumber, currentDate)
For Each conferenceInQuery As Object In conferences
Dim conference = New Conferences()
conference.AppointmentID = conferenceInQuery.AppointmentID
conference.AppTitle = conferenceInQuery.AppTitle
conference.DateTime = conferenceInQuery.DateTime
conference.[Date] = conferenceInQuery.[Date]
conference.Time = conferenceInQuery.Time
conference.Company = conferenceInQuery.Company
conference.Contact = conferenceInQuery.Contact
conference.Phone = conferenceInQuery.Phone
conference.Office = conferenceInQuery.Office
conference.Lead_Director = conferenceInQuery.Lead_Director
conference.TBD = conferenceInQuery.TBD
conference.conference = conferenceInQuery.conference
conferencelist.Add(conference)
Next
Return conferencelist
End Function
' PUT: api/Conferences/5
<ResponseType(GetType(Void))>
Function PutConferences(ByVal id As Integer, ByVal conferences As Conferences) As IHttpActionResult
If Not ModelState.IsValid Then
Return BadRequest(ModelState)
End If
If Not id = conferences.AppointmentID Then
Return BadRequest()
End If
db.Entry(conferences).State = EntityState.Modified
Try
db.SaveChanges()
Catch ex As DbUpdateConcurrencyException
If Not (ConferencesExists(id)) Then
Return NotFound()
Else
Throw
End If
End Try
Return StatusCode(HttpStatusCode.NoContent)
End Function
' POST: api/Conferences
<ResponseType(GetType(Conferences))>
Function PostConferences(ByVal conferences As Conferences) As IHttpActionResult
If Not ModelState.IsValid Then
Return BadRequest(ModelState)
End If
db.Conferences.Add(conferences)
db.SaveChanges()
Return CreatedAtRoute("DefaultApi", New With {.id = conferences.AppointmentID}, conferences)
End Function
' DELETE: api/Conferences/5
<ResponseType(GetType(Conferences))>
Function DeleteConferences(ByVal id As Integer) As IHttpActionResult
Dim conferences As Conferences = db.Conferences.Find(id)
If IsNothing(conferences) Then
Return NotFound()
End If
db.Conferences.Remove(conferences)
db.SaveChanges()
Return Ok(conferences)
End Function
Protected Overrides Sub Dispose(ByVal disposing As Boolean)
If (disposing) Then
db.Dispose()
End If
MyBase.Dispose(disposing)
End Sub
Private Function ConferencesExists(ByVal id As Integer) As Boolean
Return db.Conferences.Count(Function(e) e.AppointmentID = id) > 0
End Function
End Class
End Namespace
I am not sure I am doing any of this correctly. Is it because I am opening two connections? I am closing them regardless of errors. I did try putting in some inner transaction scopes and setting the second one to RequiresNew. The two methods are independent of each other, however, if one fails I need them both to rollback. I may have to modify how I am creating and closing the connections, I feel. Any thoughts? Here is some example code of what I am doing:
Public Sub TransMethod()
Using sTran As New Transactions.TransactionScope
factory1.UpdateMethod(someObject1)
facotry2.insert(someobject2)
sTran.Complete()
End Using
End Sub
Public Class factory1
Public Shared Sub UpdateMethod(obj)
dim someSQLParams....
DataAcces.ExecuteNonQuery(command,someSQLParams)
End Sub
End Class
Public Class factory2
Public Shared Sub Insert(obj)
dim someSQLParams....
DataAcces.ExecuteNonQuery(command,someSQLParams)
End Sub
End Class
Public Function ExecuteNonQuery(ByVal spname As String, _
ByVal ParamArray parameterValues() As Object) As Object
Dim connection As SqlConnection = Nothing
Dim command As SqlCommand = Nothing
Dim res As Object = Nothing
Try
connection = New SqlConnection(_connectionString)
command = New SqlCommand(spname, connection)
command.CommandType = CommandType.StoredProcedure
command.Parameters.AddRange(parameterValues)
connection.Open()
command.ExecuteNonQuery()
res = command.Parameters(command.Parameters.Count - 1).Value
Catch ex As Exception
CreateDataEntry(ex, WriteType.ToFile, spname)
If Not (transaction Is Nothing) Then
transaction.Rollback()
End If
Finally
If Not (connection Is Nothing) AndAlso _
(connection.State = ConnectionState.Open) Then _
connection.Close()
If Not (command Is Nothing) Then command.Dispose()
End Try
Return res
End Function
Whenever you open more than 1 connection using TransactionScope, it changes from a normal transaction against sql server to a distributed transaction, which requires setting up the MSDTC.
That will happen even if the connections have the same connection string, which is one of the "by design" issues. I haven't followed up if it remains the same on .net 3.5+