We are using the awesome & fast OrmLite (ServiceStack) library as our microORM to connect to our PostgreSQL database.
We have TDE encryption enabled in our PostgreSQL database. To ensure that the relevant data is decrypted before we query, we need to execute the following:
Db.ExecuteSql(string.Format("SELECT pgtde_begin_session('{0}');", tdeKey));
and at the end:
Db.ExecuteSql("SELECT pgtde_end_session();");
Instead of inserting these into each of our RequestDto methods, can we instead ensure that these sql statements are executed before and after each call.
You can try using an OrmLite Exec Filter, with something like:
public class PgSqlSecureSessionFilter : OrmLiteExecFilter
{
public override T Exec<T>(IDbConnection db, Func<IDbCommand, T> filter)
{
try
{
db.Execute("SELECT pgtde_begin_session(#tdeKey)", new { tdeKey });
return base.Exec(db, filter);
}
finally {
db.Execute("SELECT pgtde_end_session();");
}
}
}
OrmLiteConfig.ExecFilter = new PgSqlSecureSessionFilter();
Related
I have a question regarding joins in Amazon Dynamo DB. As Amazon Dynamo DB is a NoSQL database and doesn't supports joins. I am looking for an alternate solution for using a join command for Dynamo DB tables. I am using Dynamo DB with Android SDK.
No way to do joins in Dynamo DB.
Dynamo db table's primary key is a composite of partition key & sort
key. You must need partition key to query in table.
It is not like Relational Database, Dynamo DB table is irrespective
of data type. SO it's quite complicate to use joins in it.
Query each table individually & use values of resultant to proceed with other table.
Since DynamoDB is a NOSQL Database It doesn't support RDBMS. So you cannot join tables in the dynamo db. AWS has other databases which support RDBMS like AWS Aurora.
Disclaimer: I work at Rockset- but I def. see that this can help you solve this issue really easily. Yes, you can't do joins on DynamoDB, but you can indirectly do joins using Dyanmodb integrated with Rockset.
Create integration between dynamo db giving Rockset read permissions.
Write your SQL query with JOIN on the editor
Save the SQL query as a RESTFUL endpoint via Query Lambda on Rockset console.
On your android app, making a HTTP request to that endpoint and get your query:
Assuming you imported all the proper libraries:
public class MainActivity extends AppCompatActivity {
private String url = "https://api.rs2.usw2.rockset.com/v1/orgs/self/ws/commons/lambdas/LambdaName/versions/versionNumber";
private String APIKEY = "ApiKey YOUR APIKEY";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
new JSONAsyncTask().execute("url");
}
class JSONAsyncTask extends AsyncTask<String, Void, Boolean> {
#Override
protected void onPreExecute() {
super.onPreExecute();
}
#Override
protected Boolean doInBackground(String... urls) {
try {
HttpPost httpPost = new HttpPost(url);
HttpClient httpclient = new DefaultHttpClient();
httpPost.addHeader("Authorization" , APIKEY);
httpPost.addHeader("Content-Type" , "application/json");
HttpResponse response = httpclient.execute(httpPost);
int status = response.getStatusLine().getStatusCode();
if (status == 200) {
HttpEntity entity = response.getEntity();
String data = EntityUtils.toString(entity);
Log.e("foo", data);
JSONObject jsono = new JSONObject(data);
return true;
} else {
Log.e("foo", "error" + String.valueOf(status));
}
} catch (IOException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}
return false;
}
protected void onPostExecute(Boolean result) {
}
}
}
From there, you'll get your results as a log and then you can do what you want with that data.
While you can't do JOINS directly on DyanmoDB, you can indirectly with Rockset if you're building data-driven applications.
DynamoDb is a NoSQl database and as such you cant do joins.
However from my experience there isn't anything you can't do if you create a correct design of your database.Use a single table and a combination of primary and secondary keys.
Here are the docs https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html
You cannot use joins in DynamoDB, but you can structure your data in a single table using GSI indexes, so that you can query data in most of the possible way.
So before designing database structure, you will need to make a list of all the queries that will need and design DB structure, mainly set indexes, according to that.
I am working on getting back up to speed on Spring, Spring Data and JPA. I was interested in implementing the Specifications piece to enable a more dynamic query.
Say I have the following methods:
protected String containsLowerCase(String searchField) {
return WILDCARD + searchField.toLowerCase() + WILDCARD;
}
protected Specification<T> attributeContains(String attribute, String value) {
return (root, query, cb) -> {
return value == null ? null : cb.like(
cb.lower(root.get(attribute)),
containsLowerCase(value)
);
};
}
#Override
public Specification<Widget> getFilter(WidgetListRequest request) {
return (root, query, cb) -> Specifications.where(
Specifications.where(nameContains(request.getSearch()))
.or(descriptionContains(request.getSearch()))
)
.and(testValBetween(request.getMinTestVal(), request.getMaxTestVal()))
.toPredicate(root, query, cb);
}
private Specification<Widget> nameContains(String name) {
return attributeContains("name", name);
}
And in my service I call like:
WidgetListRequest request = new WidgetListRequest();
request.setSearch(search);
request.setMinTestVal(minTestVal.doubleValue());
request.setMaxTestVal(maxTestVal.doubleValue());
return widgetRepo.findAll(widgetSpec.getFilter(request));
If this was straight Repo call I know it, like save() is safe against SQL injection, and I know that JPQL is safe when you put parameters in. But, as I have tested, the above is NOT safe from SQL injection since it's using a concatenated string for the wildcards. I've seen some examples using the CriteraBuilder and CriteriaQuery to add parameters (ex: this), but I am not sure how that will work when you are using the Specifications on the service-side.
What I would need to do, I think, is somehow return the Specification with ParameterExpressions, but I don't know how to set them once the Specification is returned to the service class, nor how to add the wildcards such that they don't get escaped. How do I escape just the passed in value of containsLowerCase() above?
I want to log SQL Queries, to examine what exactly is my LINQ queries doing to understand how can I improve it.
But I can't find how to logging it.
I did it like this:
Change DbContextConfigurer in EntityFramework project
as:
public static void Configure(DbContextOptionsBuilder<TestDbContext> builder, string connectionString, **ILoggerFactory loggerFactory = null**)
{
**builder.UseLoggerFactory(loggerFactory);**
builder.UseSqlServer(connectionString);
}
Then add in file MyProjectEntityFrameworkCoreModule.cs into PreInitiallize section
public override void PreInitialize()
{
if (!SkipDbContextRegistration)
{
Configuration.Modules.AbpEfCore().AddDbContext<TestDbContext>(options =>
{
if (options.ExistingConnection != null)
{
TestDbContextConfigurer.Configure(options.DbContextOptions, options.ExistingConnection);
}
else
{
**var loggerFactory = IocManager.Resolve<ILoggerFactory>();**
TestDbContextConfigurer.Configure(options.DbContextOptions, options.ConnectionString,**loggerFactory**);
}
});
}
//Uncomment below line to write change logs for the entities below:
//Configuration.EntityHistory.Selectors.Add("TestEntities", typeof(OrganizationUnit), typeof(Role), typeof(Tenant));
}
If you are using MSSQL with EF and intend to troubleshoot how LINQ is being translated into SQL statement. You can try SQL Profiler that comes together with MSSQL Database Engine
With SQL Profiler, you are able to record and trace tsql event in it. With proper isolation of the LINQ call in your application, you can trace the SQL statement received by the Database engine.
https://learn.microsoft.com/en-us/sql/relational-databases/event-classes/tsql-event-category?view=sql-server-2017
When setting up our new architecture I followed a guide which used NHibernate with MsSql2008 configuration.
We are not using MsSql2008, instead using Postgresql. The configuration for this all works great and it saves to the database etc.
I am trying to write a unit test to test the UoW but I can't get the InMemory configuration to work.
The guide that I followed used this following Provider:
public class InMemoryNHibernateConfigurationProvider : NHibernateConfigurationProvider
{
public override Configuration GetDatabaseConfiguration()
{
var databaseDriver = SQLiteConfiguration.Standard.InMemory().ShowSql();
return CreateCoreDatabaseConfiguration(databaseDriver);
}
public static void InitialiseDatabase(Configuration configuration, ISession session)
{
new SchemaExport(configuration).Execute(true, true, false, session.Connection, Console.Out);
}
}
My standard (Non UnitTest) configuration looks like this:
public abstract class NHibernateConfigurationProvider : INHibernateConfigurationProvider
{
public abstract Configuration GetDatabaseConfiguration();
public Configuration CreateCoreDatabaseConfiguration(
IPersistenceConfigurer databaseDriver,
Action<Configuration> databaseBuilder = null)
{
var fluentConfiguration =
Fluently.Configure()
.Database(databaseDriver)
.Mappings(m => m.AutoMappings.Add(AutoMap.AssemblyOf<Organisation>(new DefaultMappingConfiguration())
//.Conventions.AddFromAssemblyOf<IdGenerationConvention>()
.UseOverridesFromAssemblyOf<OrganisationMappingOverride>()));
if (databaseBuilder != null)
{
fluentConfiguration.ExposeConfiguration(databaseBuilder);
}
return fluentConfiguration.BuildConfiguration();
}
}
public class PostgreSQLServerNHibernateConfigurationProvider : NHibernateConfigurationProvider
{
private static readonly string NpgsqlConnectionString = ConfigurationManager.ConnectionStrings["ProdDBConnection"].ConnectionString;
public override Configuration GetDatabaseConfiguration()
{
return CreateCoreDatabaseConfiguration(
PostgreSQLConfiguration.Standard.ConnectionString(NpgsqlConnectionString).
Dialect("NHibernate.Dialect.PostgreSQL82Dialect").ShowSql(),
BuildDatabase);
}
....... // Other Methods etc
}
How do I write a InMemoryConfigurationProvider that tests using PostgresqlConfiguration instead of SqlLiteCOnfiguration. PostgresqlConfiguration does not have an InMemory option.
Do I implement a configuration that creates another database and just drop it on teardown? Or is there perhaps another way of doing it?
Using sqlite works really well and although it does have some differences to SQL-server which we use they are so minor it doesn't matter for testing purposes.
With that said, this is how we setup the tests:
All test-cases where we want to write/read from db extend the SqLiteTestBaseclass. That way they all get access to a session created by the basesetup method, and can setup the daos / repositories as needed.
Using this approach we also always get a fresh new db for each test-case.
Update:
After trying this out a bit more I actually found that you have to modify it a bit to use InMemory (we had previously used sqlite backed by a file on disk instead). So the updated (complete) setup looks like this:
private Configuration _savedConfig;
[SetUp]
public void BaseSetup()
{
FluentConfiguration configuration =
Fluently.Configure()
.Database(SQLiteConfiguration.Standard
.InMemory)
.ExposeConfiguration(
x => x.SetInterceptor(new MultiTenancyInterceptor(ff)))
.Mappings(m => m.FluentMappings.AddFromAssemblyOf<IRepository>())
.Mappings(m => m.FluentMappings.ExportTo("c:\\temp\\mapping"))
.ExposeConfiguration(x => _savedConfig = x) //save the nhibernate configuration for use when creating the schema, in order to be able to use the same connection
.ExposeConfiguration(x => ConfigureEnvers(x))
.ExposeConfiguration(x => ConfigureListeners(x));
ISessionFactory sessionFactory;
try
{
sessionFactory = configuration.BuildSessionFactory();
}
catch (Exception ex)
{
Console.WriteLine(ex.StackTrace);
throw;
}
_session = sessionFactory.OpenSession();
BuildSchema(_savedConfig, _session);
}
private void BuildSchema(Configuration config, ISession session)
{
new SchemaExport(config)
.Execute(false, true, false, session.Connection, null);
}
The reason why you have to jump through all these hoops in order to use the in-memory version of Sqlite is due to the db being tied to the connection. You have to use the same connection that creates the db to populate the schema, thus we have to save the Configuration object so that we can export the schema later when we've created the connection.
See this blogpost for some more details: http://www.tigraine.at/2009/05/29/fluent-nhibernate-gotchas-when-testing-with-an-in-memory-database/
N.B: This only shows the setup of the db. We have some code which also populates the db with standard values (users, customers, masterdata etc) but I've omitted that for brevity.
I'm trying to create my database (code first) and I want to add some data in it when it's created.
public class InitializerWithData : CreateDatabaseIfNotExists<DatabaseContext>
{
protected override void Seed(DatabaseContext ctx)
{
GroupType gt = new GroupType() { Name = "RNC" };
//save
ctx.GroupType.Add(gt);
ctx.SaveChanges();
}
}
public DatabaseContext()
{
Database.SetInitializer<DatabaseContext>(new InitializerWithData());
Database.CreateIfNotExists();
}
As you can see I wrote my custom initializer but the code inside it is never fired though the database does get created.
So how do I solve this?
When you call Database.CreateIfNotExists(), it doesn't trigger the InitializeDatabase of the initializer. Basically it has separated implementation than the initializer.
If you want the Seed method to be fired. You need to execute a code that causes EF to send a query to the database.
First remove this line.
Database.CreateIfNotExists();
Then just execute a query, the least you could have is something like.
using(var db = new DatabaseContext())
{
db.Set<GroupType>().Any();
}
This code will create the database if it doesn't exist and execute the Seed method.