So I've got a SQL query that is called from an API that I'm trying to write an integration test for. I have the method that prepares the data totally working, but I realized that I don't know how to actually execute the query to check that data (and run the test). Here is what the query looks like (slightly redacted to protect confidental data):
SELECT HeaderQuery.[headerid],
kaq.[applicationname],
HeaderQuery.[usersession],
HeaderQuery.[username],
HeaderQuery.[referringurl],
HeaderQuery.[route],
HeaderQuery.[method],
HeaderQuery.[logdate],
HeaderQuery.[logtype],
HeaderQuery.[statuscode],
HeaderQuery.[statusdescription],
DetailQuery.[detailid],
DetailQuery.[name],
DetailQuery.[value]
FROM [DATABASE1].[dbo].[apilogheader] HeaderQuery
LEFT JOIN [DATABASE1].[dbo].[apilogdetails] DetailQuery
ON HeaderQuery.[headerid] = DetailQuery.[headerid]
INNER JOIN [DATABASE2].[dbo].[apps] kaq
ON HeaderQuery.[applicationid] = kaq.[applicationid]
WHERE HeaderQuery.[applicationid] = #applicationid1
AND HeaderQuery.[logdate] >= #logdate2
AND HeaderQuery.[logdate] <= #logdate3
For the sake of the test, and considering I already have the SQL script, I was hoping to be able to just execute that script above (providing the where clause programmatically) using context.Database.SqlQuery<string>(QUERY) but since I have two different contexts, I'm not sure how to do that.
The short answer is no, EF doesn’t support cross database queries. However there are a few things you can try.
You can use two different database contexts (one for each database).
Run your respective queries and then merge / massage the data after
the query returns.
Create a database view and query the view through EF.
Using a SYNONYM
https://rachel53461.wordpress.com/2011/05/22/tricking-ef-to-span-multiple-databases/
If the databases are on the same server, you can try using a
DbCommandInterceptor
I’ve had this requirement before and personally like the view option.
Related
I am currently implementing a Vapor 4 application, which will be used to manage machines.
The user should be able to search for a machine name, which I accomplished by
.filter(Machine.path(for: \Machine.$name), .contains(inverse: false, .anywhere), term)
where term is an arbitrary String provided by the user.
The code itself works as intended, but I was wondering if there is the possibility of a SQL Injection vulnerability (or other attacks).
My Question:
Is SQL Injection (or other attacks) possible and if so, how can I prevent it (please provide a code example)?
Since you are using Fluent, SQL injection is prevented automatically and you are good to go!
Instead of simply constructing a query like this:
SELECT * FROM machines WHERE name = '\(user_provided_name)'
Fluent uses value binding, which is a feature provided by databases to pass values into the query so that the value is escaped and won't be executed if the string contains SQL code. It looks something like this:
SELECT * FROM machines WHERE name = ?
And then the values are passed to the database server (MySQL in this case) with the query, where it automatically replaces the placeholders (?) with the values provided.
A quick comment on your query, if you want, you can import the FluentSQL module and then write your query like this:
.filter(\.$name ~~ term)
If you would rather leave it the way you have it now, that's fine also.
I have next issue:
I have list of names, based on which I want to filter.The problem is that I have not full names(Because I'm receiving them from ui), and I have, for example, this array= ['Joh', 'Michae'].
So, I want to filter based on this array.
I wrote query in PostgreSQL
select * from q_ob_person where name like any (array['%Хомяченко%', '%Вартопуз%']);
And I want to ask how to write JPQL query gor this.
Is there an option to call postgresql function like any from JPQL?
JPA 2.1 allows invocation of any SQL function using
FUNCTION(sqlFuncName, sqlArgs)
So you could likely do something like (note never tried this LIKE ANY you refer to, just play around with it)
FUNCTION("LIKE", FUNCTION("ANY", arrayField))
Obviously by invoking SQL functions specific to a particular RDBMS you lose database independence (in case that's of importance).
I have a stored procedure in my project under sql/my_prod.sql
there I have my function delete_entity
In my entity
#NamedNativeQuery(name = "delete_entity_prod",
query = "{call /sql/delete_entity(:lineId)}",
and I call it
Query query = entityManager.createNamedQuery("delete_entity_prod")
setParameter("lineId",lineId);
I followed this example: http://objectopia.com/2009/06/26/calling-stored-procedures-in-jpa/
but it does not execute the delete and it does not send any error.
I haven't found clear information about this, am I missing something? Maybe I need to load the my_prod.sql first? But how?
JPA 2.1 standardized stored procedure support if you are able to use it, with examples here http://en.wikibooks.org/wiki/Java_Persistence/Advanced_Topics#Stored_Procedures
This is actually they way you create a query.
Query query = entityManager.createNamedQuery("delete_entity_prod")
setParameter("lineId",lineId);
To call it you must execute:
query.executeUpdate();
Of course, the DB must already contain the procedure. So if you have it defined in your SQL file, have a look at Executing SQL Statements from a Text File(this is for MySQL but other database systems use a similar approach to execute scripts)
There is no error shown because query is not executed at any point - just instance of Query is created. Query can be executed by calling executeUpdate:
query.executeUpdate();
Then next problem will arise: Writing some stored procedures to file is not enough - procedures live in database, not in files. So next thing to do is to check that there is correct script to create stored procedure in hands (maybe that is currently content of sql/my_prod.sql) and then use that to create procedure via database client.
All JPA implementations do not support calling stored procedures, but I assume Hibernate is used under the hood, because that is also used in linked tutorial.
It can be the case that current
{call /sql/delete_entity(:lineId)}
is right syntax for calling stored procedure in your database. It looks rather suspicious because of /sql/. If it turns out that this is incorrect syntax, then:
Consult manual for correct syntax
Test via client
Use that as a value of query attribute in NamedNativeQuery annotation.
All that with combination MySQL+Hibernate is explained for example here.
I am currently experimenting with Tableau Extract API to generate some TDE from the tables I have in a PostgreSQL database. I was able to write a code to generate the TDE from single table, but I would like to do this for multiple joined tables. To be more specific, if I have two tables that are inner joined by some field, how would I generate the TDE for this?
I can see that if I am working with small number of tables, I could use a SQL query with JOIN clauses to create a one gigantic table, and generate the TDE from that table.
>> SELECT * FROM table_1 INNER JOIN table_2
INTO new_table_1
ON table_1.id_1 = table_2.id_2;
>> SELECT * FROM new_table_1 INNER JOIN TABLE_3
INTO new_table_2
ON new_table_1.id_1 = table_3.id_3
and then generate the TDE from new_table_2.
However, I have some tables that have over 40 different fields, so this could get messy.
Is this even a possibility with current version of the API?
You can read from as many tables or other sources as you want. Or use complex query with lots of joins, or create a view and read from that. Usually, creating a view is helpful when you have a complex query joining many tables.
The data extract API is totally agnostic about how or where you get the data to feed it -- the whole point is to allow you to grab data from unusual sources that don't have pre-built drivers for Tableau.
Since Tableau has a Postgres driver and can read from it directly, you don't need to write a program with the data extract API at all. You can define your extract with Tableau Desktop. If you need to schedule automated refreshes of the extract, you can use Tableau Server or its tabcmd command.
Many thanks for your replies. I am aware that I could use Tableau Desktop to define my extract. In fact, I have done this many times before. I am just trying to create the extracts using the API, because I need to create some calculated fields, which is near impossible to create using the Tableau Desktop.
At this point, I am hesitant to use JOINs in the SQL query because the resulting table would look too complicated to comprehend (some of these tables also have same field names).
When you say that I could read from multiple tables or sources, does that mean with the Tableau Extract API? At this point, I cannot find anywhere in this API that accommodates multiple sources. For example, I know that when I use multiple tables in the Tableau Desktop, there are icons on the left hand side that tells me that the extract is composed of multiple tables. This just doesn't seem to be happening with the API, which leaves me stranded. Anyways, thank you again for your replies.
Going back to the topic, this is something that I tried few days ago on my python code
try:
tdefile= tde.Extract("extract.tde")
except:
os.remove("extract.tde")
tdefile = tde.Extract("extract.tde")
tableDef = tde.TableDefinition()
# Read each column in table and set the column data types using tableDef.addColumn
# Some code goes here...
for eachTable in tableNames:
tableAdd = tdeFile.addTable(eachTable, tableDef)
# Use SQL query to retrieve bunch_of_rows from eachTable
for some_row in bunch_of_rows:
# Read each row in table, and set the values in each column position of each row
# Some code goes here...
tableAdd.insert(some_row)
some_row.close()
tdefile.close()
When I execute this code, I get the error that eachTable has to be called "Extract".
Of course, this code has its flaws, as there is no where in this code that tells how each table are being joined.
So I am little thrown off here, because it doesn't seem like I can use multiple tables unless I use JOINs to generate one table that contains everything.
I'm using Zend Framework for my website and I'd like to retrieve some data from my PostgreSQL database.
I have a request like :
SELECT DISTINCT ON(e.id) e.*, f.*, g.* FROM e, f, g
WHERE e.id = f.id_e AND f.id = g.id_f
This request works well but I don't know how to convert the DISTINCT ON(e.id) with Zend.
It seems that I can get DISTINCT rows but no distinct columns.
$select->distinct()->from("e")->join("f", "e.id = f.id_e")
->join("g", "f.id = g.id_f");
Any idea on how to make a select with distinct column ?
Thanks for help
You probably can't do this with Zend Framework since distinct on is not part of the SQL standard (end of page in Postgres documentation). Although Postgres supports it, I would assume its not part of Zend Framework because you could in theory configure another database connection which does not offer support.
If you know in advance that you're developing for a specific database (Postgres in this case), you could use manually written statements instead. You'll gain more flexibility within the queries and better performance at the cost of no longer being able to switch databases.
You would then instantiate a Zend_Db_Apdapter for Postgres. There a various methods available to get results for SQL queries which are described in the frameworks documentation starting at section Reading Query Results. If you choose to go this route I'd recommend to create an own subclass of the Zend_Db_Adapter_Pgsql class. This is to be able to convert data types and throw exceptions in case of errors instead of returning ambiguous null values and hiding error causes.