I am trying to implement Npgsql in our DAL and running into issues under heavy load. the following sample application is a decent representation of just a simple query that under heavy load, throws a 'A command is already in progress' exception. I am assuming this is due to the lack of MARS support so I also tried creating a connection each time with a using statement around each command only to have the performance become unusable. I checked that the username is indexed so that shouldn't be an issue.
Not sure what I am doing wrong here but I need some advice on how to get this performing well.
OS: Docker Container: microsoft/dotnet:2.1.301-sdk
using Npgsql;
using System;
using System.Collections.Generic;
using System.Data.Common;
using System.Linq;
using System.Threading.Tasks;
namespace npgsqlTest
{
class Program
{
static async Task Main(string[] args)
{
DAL dal = new DAL();
dal.Prepare();
var tasks = dal.Users.Select(async user =>
{
Console.WriteLine(await dal.RunTest(user));
});
await Task.WhenAll(tasks);
}
}
public class DAL
{
private static string _ConnectionString;
private NpgsqlConnection _Connection;
public List<string> Users { get; set; } = new List<string>();
public DAL()
{
_ConnectionString = $"Host=192.168.1.1;Username=admin;Port=5432;Password=password;Database=BigDB;";
_Connection = new NpgsqlConnection(_ConnectionString);
_Connection.Open();
}
public void Prepare()
{
string query = "SELECT username FROM usertable;";
using (var cmd = new NpgsqlCommand(query, _Connection))
{
var reader = cmd.ExecuteReader();
using (reader)
{
while (reader.Read())
{
Users.Add(reader[0].ToString());
}
}
}
}
public async Task<string> RunTest(string user)
{
var parameters = new Dictionary<string, Object> { { "username", user } };
var query = $"SELECT name FROM usertable WHERE username = (#username);";
var reader = await QueryAsync(query, parameters);
using (reader)
{
if (reader.HasRows)
{
while (await reader.ReadAsync())
{
var name = reader["name"];
if (!(hash is DBNull))
return (string)name;
}
}
}
return String.Empty;
}
public async Task<DbDataReader> QueryAsync(string query, Dictionary<string, Object> parameters)
{
using (var cmd = new NpgsqlCommand(query, _Connection))
{
foreach (var parameter in parameters)
{
cmd.Parameters.AddWithValue(parameter.Key, parameter.Value == null ? DBNull.Value : parameter.Value);
}
cmd.Prepare();
return await cmd.ExecuteReaderAsync();
}
}
}
}
I am developing a stand alone application, using sql server compact 3.5 sp2 which runs in process. No Database writes involved. Its purely a reporting application. Read many articles about reusing open db connections in case of sql compact(connection pooling) due to its different behavior from sql server.
Quoting the comments from a quiz opened by Erik Ejlskov Jensen Link, where its discussed an open early close late strategy for sql server compact databases. Based on this, with my limited experience I have implemented a not so complex Connection handling+Data access layer. Basically I am unsure if i am writing it in a recommended way. Please could any one point me in the right direction with rooms for improvement in this connection handling approach i have written?
The DbConnection class
public class FkDbConnection
{
private static SqlCeConnection conn;
private static DataTable table;
private static SqlCeCommand cmd;
~FkDbConnection() { conn = null; }
//This will be called when the main winform loads and connection will be open as long as the main form is open
public static string ConnectToDatabase()
{
try {
conn = new SqlCeConnection(ConfigurationManager.ConnectionStrings["Connstr"].ConnectionString);
if (conn.State == ConnectionState.Closed || conn.State == ConnectionState.Broken)
{
conn.Open();
}
return "Connected";
}
catch(SqlCeException e) { return e.Message; }
}
public static void Disconnect()
{
if (conn.State == ConnectionState.Open || conn.State == ConnectionState.Connecting || conn.State == ConnectionState.Fetching)
{
conn.Close();
conn.Dispose();
//conn = null; //does conn have to be set to null?
}
//else the connection might be already closed due to failure in opening it
else if (conn.State == ConnectionState.Closed) {
conn.Dispose();
//conn = null; //does conn have to be set to null?
}
}
/// <summary>
/// Generic Select DataAccess
/// </summary>
/// <param name="sql"> the sql query which needs to be executed by command object </param>
public static DataTable ExecuteSelectCommand(SqlCeCommand comm)
{
if (conn != null && conn.State == ConnectionState.Open)
{
#region block using datareader
using (table = new DataTable())
{
//using statement needed for reader? Its closed below
using (SqlCeDataReader reader = comm.ExecuteReader())
{
table.Load(reader);
reader.Close(); //is it needed?
}
}
#endregion
# region block using dataadpater
//I read DataReader is faster?
//using (SqlCeDataAdapter sda = new SqlCeDataAdapter(cmd))
//{
// using (table = new DataTable())
// {
// sda.Fill(table);
// }
//}
#endregion
//}
}
return table;
}
/// <summary>
/// Get Data
/// </summary>
/// <param name="selectedMPs"> string csv, generated from a list of selected posts(checkboxes) from the UI, which forms the field names used in SELECT </param>
public static DataTable GetDataPostsCars(string selectedMPs)
{
DataTable dt;
//i know this it not secure sql, but will be a separate question to pass column names to select as parameters
string sql = string.Format(
"SELECT " + selectedMPs + " "+
"FROM GdRateFixedPosts");
using (cmd = new SqlCeCommand(sql,conn))
{
cmd.CommandType = CommandType.Text;
//cmd.Parameters.Add("#fromDateTime",DbType.DateTime);
//cmd.Parameters.Add("#toDateTime",DbType.DateTime);
dt = ExecuteSelectCommand(cmd);
}
return dt;
}
}
The Main UI (Form) in which connection opened, for connection to be open through out. 2 other reporting forms are opened from here. Closing main form closes all, at which point connection is closed and disposed.
private void FrmMain_Load(object sender, EventArgs e)
{
string str = FkDbConnection.ConnectToDatabase();
statStDbConnection.Items[0].Text = str;
}
private void FrmMain_FormClosing(object sender, FormClosingEventArgs e)
{
FkDbConnection.Disconnect();
}
Comments, improvements on this connection class much appreciated. See my questions also inline code
Thank you.
Updated classes as per Erik's suggestion. with a correction on ExecuteSelectCommand() and an additional class which will instantiate command objs in "using" and pass data to the UI. I intent to add separate GetDataForFormX() methods since the dynamic sql for each form may differ. Hope this is ok?
Correction to Erik's code:
public static DataTable ExecuteSelectCommand(SqlCeCommand comm)
{
var table = new DataTable();
if (conn != null && conn.State == ConnectionState.Open)
{
comm.Connection = conn;
using (SqlCeDataReader reader = comm.ExecuteReader())
{
table.Load(reader);
}
}
return table;
}
New FkDataAccess class for passing Data to UI
public class FkDataAccess
{
public static DataTable GetDataPostsCars(string selectedMPs)
{
var table = new DataTable();
string sql = string.Format(
"SELECT " + selectedMPs + " " +
"FROM GdRateFixedPosts");
if (FkDbConnection.conn != null && FkDbConnection.conn.State == ConnectionState.Open)
{
using (SqlCeCommand cmd = new SqlCeCommand(sql, FkDbConnection.conn))
{
cmd.CommandType = CommandType.Text;
//cmd.Parameters.Add("#fromDateTime",DbType.DateTime);
table = FkDbConnection.ExecuteSelectCommand(cmd);
}
}
return table;
}
//public static DataTable GetDataXY(string selectedvals)
// and so on
}
Too much code in your data access class, makes it unreadable and hard to maintain
The SqlCeonnection object will be disposed when you close it (and when the app closes)
You cannot dispose the DataTable if you want to use it elsewhere, and it is an completely managed object anyway.
It is a good pattern to limit your classes to a single responsibility
public class FkDbConnection
{
private static SqlCeConnection conn;
~FkDbConnection() { conn = null; }
//This will be called when the main winform loads and connection will be open as long as the main form is open
public static void ConnectToDatabase()
{
// Handle failure to open in the caller
conn = new SqlCeConnection(ConfigurationManager.ConnectionStrings["Connstr"].ConnectionString);
conn.Open();
}
public static void Disconnect()
{
if (conn != null)
{
conn.Close();
}
}
public static DataTable ExecuteSelectCommand(SqlCeCommand comm)
{
var table = new DataTable();
if (conn != null && conn.State == ConnectionState.Open)
{
comm.Connection = conn;
using (SqlCeDataReader reader = comm.ExecuteReader())
{
table.Load(reader);
}
}
return table;
}
private void FrmMain_Load(object sender, EventArgs e)
{
try
{
FkDbConnection.ConnectToDatabase();
statStDbConnection.Items[0].Text = "Connected";
}
catch (Exception ex)
{
//Inform use that we canot proceed, what she can do to remedy, and exit
}
}
private void FrmMain_FormClosing(object sender, FormClosingEventArgs e)
{
FkDbConnection.Disconnect();
}
i am working on ado.net project,, i used to have separate classes for codes ,,,
now i am trying to learn more about local databases but i am stock when i tried to call class that contain SQL commands ,,, please
have a look to my code i commented where i stock
namespace Maintenance
{
public partial class registerReports : Form
{
private void btnSave_Click(object sender, EventArgs e)
{
//the code works if it exists here
/**
conn.Open();
com.Parameters.AddWithValue("#reportID",
tbReportIDox.Text);
com.ExecuteNonQuery();
conn.Close();
**/
}
}
}
class reportTableSQL
{
public void reportTable()
{
string connectionString = connectionString = "Data Source=
..//..//maintenanceDB.sdf";
SqlCeConnection conn = new SqlCeConnection(connectionString);
using (SqlCeCommand com = new SqlCeCommand("INSERT INTO
ReportForm VALUES(#reportID)", conn))
{
// if i call this method from class registerReports : Form
// it doesn't recognise tbReportIDox.Text as
//it isn't exist in this class
/**
conn.Open();
com.Parameters.AddWithValue("#reportID",
tbReportIDox.Text);
com.ExecuteNonQuery();
conn.Close();
**/
}
}
}
thank you
What you can do is to pull tbReportIDox.Text up as a parameters to the method, that is
public void reportTable(string reportName)
{
...
com.Parameters.AddWithValue("#reportID", reportName);
...
}
And then on a button click
private void btnSave_Click(object sender, EventArgs e)
{
// resolve instance of reportTableSQL class
// for example through: new reportTableSQL();
var reportGenerator = new reportTableSQL();
reportGenerator.reportTable(tbReportIDox.Text);
}
I have a generic repository for my Entities, all my entities (generated by the Code generation Item) have a personalized partial that implements an IID interface, at this point all of my entites must have a Int32 Id property.
So, my problems is with the update, here is my code
public class RepositorioPersistencia<T> where T : class
{
public static bool Update(T entity)
{
try
{
using (var ctx = new FisioKinectEntities())
{
// here a get the Entity from the actual context
var currentEntity = ctx.Set<T>().Find(((BLL.Interfaces.IID)entity).Id);
var propertiesFromNewEntity = entity.GetType().GetProperties();
var propertiesFromCurrentEntity = currentEntity.GetType().GetProperties();
for (int i = 0; i < propertiesFromCurrentEntity.Length; i++)
{
//I'am trying to update my current entity with the values of the new entity
//but this code causes an exception
propertiesFromCurrentEntity[i].SetValue(currentEntity, propertiesFromNewEntity[i].GetValue(entity, null), null);
}
ctx.SaveChanges();
return true;
}
}
catch
{
return false;
}
}
}
Someone can help me? this driving me crazy.
You can use the EF API to update the values of an entity as follows.
public static bool Update(T entity)
{
try
{
using (var ctx = new FisioKinectEntities())
{
var currentEntity = ctx.Set<T>().Find(((BLL.Interfaces.IID)entity).Id);
var entry = ctx.Entry(currentEntity);
entry.CurrentValues.SetValues(entity);
ctx.SaveChanges();
return true;
}
}
catch
{
return false;
}
}
May I ask for help with the following?
I am attempting to connect and control three pieces of household electronic equipment by computer through a GlobalCache GC-100 and iTach. As you will see in the following code, I created a class ("GlobalCacheAdapter") that can communicate and control the equipment, and created an instance of the class for each piece of equipment. Although each instance seems to work well with communicating and in controlling each piece of equipment, the *feedback returned from the equipment* seems only to be visible at the defining class level's - "ReaderThreadProc" procedure. Further processing of the feedback is required for each piece of equipment and I am uncertain as to how to forward this feedback at the equipment specific instance-level. I suspect that an instance-specific EventHandler will need to be implemented; however I am not aware as to how to implement this type of instance-specific EventHandler in order to complete processing and update the appropriate controls.
Any help wold be greatly appreciated.
using System;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Windows.Forms;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
// Create three new instances of GlobalCacheAdaptor and connect.
// GC-100 (Elan) 192.168.1.70 4998
// GC-100 (TuneSuite) 192.168.1.70 5000
// GC iTach (Lighting) 192.168.1.71 4999
private GlobalCacheAdaptor elanGlobalCacheAdaptor;
private GlobalCacheAdaptor tuneSuiteGlobalCacheAdaptor;
private GlobalCacheAdaptor lutronGlobalCacheAdaptor;
public Form1()
{
InitializeComponent();
elanGlobalCacheAdaptor = new GlobalCacheAdaptor();
elanGlobalCacheAdaptor.ConnectToDevice(IPAddress.Parse("192.168.1.70"), 4998);
tuneSuiteGlobalCacheAdaptor = new GlobalCacheAdaptor();
tuneSuiteGlobalCacheAdaptor.ConnectToDevice(IPAddress.Parse("192.168.1.70"), 5000);
lutronGlobalCacheAdaptor = new GlobalCacheAdaptor();
lutronGlobalCacheAdaptor.ConnectToDevice(IPAddress.Parse("192.168.1.71"), 4999);
elanTextBox.Text = elanGlobalCacheAdaptor._line;
tuneSuiteTextBox.Text = tuneSuiteGlobalCacheAdaptor._line;
lutronTextBox.Text = lutronGlobalCacheAdaptor._line;
}
private void btnZoneOnOff_Click(object sender, EventArgs e) { elanGlobalCacheAdaptor.SendMessage("sendir,4:3,1,40000,4,1,21,181,21,181,21,181,21,181,21,181,21,181,21,181,21,181,21,181,21,181,21,181,21,800" + Environment.NewLine); }
private void btnSourceInput1_Click(object sender, EventArgs e) { elanGlobalCacheAdaptor.SendMessage("sendir,4:3,1,40000,1,1,20,179,20,179,20,179,20,179,20,179,20,179,20,179,20,278,20,179,20,179,20,179,20,780" + Environment.NewLine); }
private void btnSystemOff_Click(object sender, EventArgs e) { elanGlobalCacheAdaptor.SendMessage("sendir,4:3,1,40000,1,1,20,184,20,184,20,184,20,184,20,184,20,286,20,286,20,286,20,184,20,184,20,184,20,820" + Environment.NewLine); }
private void btnLightOff_Click(object sender, EventArgs e) { lutronGlobalCacheAdaptor.SendMessage("sdl,14,0,0,S2\x0d"); }
private void btnLightOn_Click(object sender, EventArgs e) { lutronGlobalCacheAdaptor.SendMessage("sdl,14,100,0,S2\x0d"); }
private void btnChannel31_Click(object sender, EventArgs e) { tuneSuiteGlobalCacheAdaptor.SendMessage("\xB8\x4D\xB5\x33\x31\x00\x30\x21\xB8\x0D"); }
private void btnChannel30_Click(object sender, EventArgs e) { tuneSuiteGlobalCacheAdaptor.SendMessage("\xB8\x4D\xB5\x33\x30\x00\x30\x21\xB8\x0D"); }
}
}
public class GlobalCacheAdaptor
{
public Socket _multicastListener;
public string _preferredDeviceID;
public IPAddress _deviceAddress;
public Socket _deviceSocket;
public StreamWriter _deviceWriter;
public bool _isConnected;
public int _port;
public IPAddress _address;
public string _line;
public GlobalCacheAdaptor() { }
public static readonly GlobalCacheAdaptor Instance = new GlobalCacheAdaptor();
public bool IsListening { get { return _multicastListener != null; } }
public GlobalCacheAdaptor ConnectToDevice(IPAddress address, int port)
{
if (_deviceSocket != null) _deviceSocket.Close();
try
{
_port = port;
_address = address;
_deviceSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
_deviceSocket.Connect(new IPEndPoint(address, port)); ;
_deviceAddress = address;
var stream = new NetworkStream(_deviceSocket);
var reader = new StreamReader(stream);
var writer = new StreamWriter(stream) { NewLine = "\r", AutoFlush = true };
_deviceWriter = writer;
writer.WriteLine("getdevices");
var readerThread = new Thread(ReaderThreadProc) { IsBackground = true };
readerThread.Start(reader);
_isConnected = true;
return Instance;
}
catch { DisconnectFromDevice(); MessageBox.Show("ConnectToDevice Error."); throw; }
}
public void SendMessage(string message)
{
try
{
var stream = new NetworkStream(_deviceSocket);
var reader = new StreamReader(stream);
var writer = new StreamWriter(stream) { NewLine = "\r", AutoFlush = true };
_deviceWriter = writer;
writer.WriteLine(message);
var readerThread = new Thread(ReaderThreadProc) { IsBackground = true };
readerThread.Start(reader);
}
catch { MessageBox.Show("SendMessage() Error."); }
}
public void DisconnectFromDevice()
{
if (_deviceSocket != null)
{
try { _deviceSocket.Close(); _isConnected = false; }
catch { MessageBox.Show("DisconnectFromDevice Error."); }
_deviceSocket = null;
}
_deviceWriter = null;
_deviceAddress = null;
}
**private void ReaderThreadProc(object state)**
{
var reader = (StreamReader)state;
try
{
while (true)
{
var line = reader.ReadLine();
if (line == null) break;
_line = _line + line + Environment.NewLine;
}
**// Feedback from each piece of equipment is visible here.
// Need to create EventHandler to notify the TextBoxes to update with _line**
}
catch { MessageBox.Show("ReaderThreadProc Error."); }
}
}
From my understanding of the question, you want to do something like this?
You need to know when a GlobalCacheAdapter updates and which one updated in order to update textboxes on a form. My question to you is this - do you actually need to know which updated?
If you declare in your class an event handler like this:
public class GlobalCacheAdaptor
{
public event EventHandler<EventArgs> Updated;
protected virtual void OnUpdated()
{
var handler = Updated;
if (handler != null)
{
handler(this, EventArgs.Empty);
}
}
private void Foo()
{
// When an update is received, raise Updated event
OnUpdated();
}
}
Then in your form subscribe to Updated for all three GlobalCacheHandler instances
public Form1()
{
elanGlobalCacheAdaptor.Updated += (s,e) =>
{
elanTextBox.Text = elanGlobalCacheAdaptor._line;
}
tuneSuiteGlobalCacheAdaptor.Updated += (s,e) =>
{
tuneSuiteTextBox.Text = tuneSuiteGlobalCacheAdaptor._line;
}
lutronGlobalCacheAdaptor.Updated += (s,e) =>
{
lutronTextBox.Text = lutronGlobalCacheAdaptor._line;
}
}
You should be able to update the correct text box when the appropriate cache handler raises the Updated event.
Finally you may need to handle cross-thread interactions. if so, see this article on MSDN, particularly the part "Thread-Safe Calls to a Windows Forms Control"