Join database query Vs Handeling Joins in API - postgresql

I am developing an API where I am confused as to what is the efficient way to handle join query.
I want to join 2 tables data and return the response. Either I can query the database with join query and fetch the result and then return the response OR I can fire two separate queries and then I would handle the join in the API on the fly and return the response. Which is the efficient and correct way ?

Databases are pretty much faster than querying and joining as class instances. Always do joins in the database and map them from the code. Also look for any lazy loading if possible. Cause in a situation like below:
#Entity
#Table(name = "USER")
public class UserLazy implements Serializable {
#Id
#GeneratedValue
#Column(name = "USER_ID")
private Long userId;
#OneToMany(fetch = FetchType.LAZY, mappedBy = "user")
private Set<OrderDetail> orderDetail = new HashSet();
// standard setters and getters
// also override equals and hashcode
}
you might not want order details when you want the initial results.

Usually it's more efficient to do the join in the database, but there are some corner cases, mostly due to the fact that application CPU time is cheaper than database CPU time. Here are a few examples that come to mind, with a query like "table A join table B":
B is a small table that rarely changes.
In this case it can be profitable to cache the contents of this table in the application, and not query it at all.
Rows in A are quite large, and many rows of B are selected for each row of A.
This will cause useless network traffic and load as rows from A are duplicated many times in each result row.
Rows in B are quite large, and there are few distinct b_id's in A
Same as above, except this time the same few rows from B are duplicated in the result set.
In the previous two examples, it could be useful to perform the query on table A, then gather a set of unique b_id's from the result, and SELECT FROM b WHERE b_id IN (list).
Data structure and ORMs
If each table contains a different object type, and they have a "belongs to" relationship (like category and product) and you use an ORM which will instantiate objects for each selected row, then perhaps you only want one instance of each category, and not one per selected product. In this case, you could select the products, gather a list of unique category_ids, and select the categories from there. The ORM may even do that for you behind the scene.
Complicated aggregates
Sometimes, you want some stuff, and some aggregates of other stuff related to the first stuff, but it just won't fit in a neat GROUP BY, or you may need several ones.
So basically, usually the join works better in the database, so that should be the default. If you do it in the application, then you should know why you're doing it, and decide it's a good reason. If it is, then fine. I gave a few reasons, based on performance, data model, and SQL constraints, these are only examples of course.

Related

Typed Query for those tables created from relationships (One to Many)

I am little confused, If i have two tables related then I will have a combined table in MySQL, Since we do not have a class in our project , how would my Typed Query look in order to fetch data from table that is create from Relationship( say One to Many).
eg:
#OneToMany(cascade = CascadeType.ALL)
#JoinTable(name = "CustomerBilling",joinColumns = #JoinColumn(name = "Customer_Id"),
inverseJoinColumns = #JoinColumn(name = "Billing_Id"))
private List<Billing> billing = new ArrayList<>();
with the above mentioned code i will have CustomerBilling table , So i would like to get all the records for a particular customer id. in my test (jUnit) file what Typed Query do i need to put ?
TypedQuery<Customer> a = em.createQuery("select b from CustomerBilling b where b.Customer_Id =?1", Customer.class);
This did not work Since CustomerBilling abstrate schema is not present.
Thanks
Prashanth
I am sure you have classes in your project as it is java based :)
Reading between the lines, you mean, that you have 3 tables in DB Customer, Billing and CustomerBilling, but only 2 entities in JPA the Customer and Billing as the CustomerBilling is there only to store the relation. (But your example of typed query tries to get Customer.class as the result so maybe I am wrong).
You don't reference jointable objects directly (unless they are mapped).
You can query by Customer, or by Billing but not by CustomerBilling. If there is no such entity defined in JPA, there is nothing that JPA can query about.
And you shouldn't try to do it, even for unit testing (BTW jpa tests are integration tests not unit tests).
The whole point of mapping the relationships with #OnteToMany and such is to hide, from the program, the actual DB structure. So you should access Billings of a customer, or customer for a billing, not the relation info.
Currently the relation is stored by join table, but it could be change it to join column and whole program logic would stay the same (and so should the tests).
You should test, that if you add bilings to customer and save them, you correctly retrieves them, that if you remove the biling from customer it disappears, but there is no reason to check the content of the join table.
Finally, if you really must, you can use native sql query to access CustomerBilling table
em.createNativeQuery
or
you can change your mappings, and introdude the CustomerBilling entity, and map your OneToMay on Customer to the CustomerBilling instead of Directly Billing.

Do canonical LINQ queries guard against N+1

With lazy loading used by default, I know that you should call .Include() on your Entity Framework entities to pull in associated entities you want in your queries to reduce the number of calls to the db if you're calling LINQ methods on your entities. If you don't, you run the risk of repeated database calls for each row (the N+1 problem)
Can someone confirm that if I write a canonical LINQ query, with the joins defined explicitly, that we guard against N+1?
from x in _context.tblOrder
join y in _context.tblCustomer equals y.id = x.customerId
select x
Is there any way N+1 could creep in when we're loading in all the required entities with joins?
EDIT
As background, someone asked how junior developers could guard against N+1. I mentioned the simplest way would be to write out your queries and define your joins, I want confirmation that was I indicated was 100% accurate.
If what you are really asking is
Will this query hit the database once?
Then the answer is yes. LINQ to EF translates your expression to raw SQL and only when you evaluate the query will it send anything to the database e.g. ToList()/foreach/for etc. and once that query is sent nothing else is unless you explicitly tell it otherwise.
Your LINQ statement could be simplified using a Lambda expression e.g.
_context.tblOrder.Include("Customer").ToList();
This would give you all the order details, including all related customer details, in a single database trip.
Just because you specify tables in a join doesn't mean that you can't run into a n+1 issue when you iterate over the values. Consider the following extension to your query:
var query = from o in Orders
join c in Customers on o.CustomerID equals c.CustomerID
select o;
foreach (var o in query)
{
Console.WriteLine(String.Format("{0}: {1}", o.OrderDate, o.Employee.FirstName));
}
In this case, each time you navigate through the order's Employee object, the employee is fetched from the database for that order. If you wanted to avoid the issue, you could project the values you want in the select clause:
var query = from o in Orders
join c in Customers on o.CustomerID equals c.CustomerID
select new {o.OrderDate, o.Employee.FirstName};
foreach (var o in query)
{
Console.WriteLine(String.Format("{0}: {1}", o.OrderDate, o.FirstName));
}
Note, in this case, you don't even need the join as you can just use the navigation properties instead. Of course, if you don't allow navigation properties in your entities and rely only on joins, you can avoid the n+1 situation, but that is not a very OOP way of solving the problem.
I think you would be safe guaranteeing against n+1 if you only return anonymous types from your queries, but that would be rather restrictive as well.
The best option is to make sure to profile your application's generated SQL and know precisely when and why you are hitting the database. I discuss some of the profilers available at http://www.thinqlinq.com/Post.aspx/Title/LINQ-to-Database-Performance-hints.

getting data out of secondary tables with oneToMany annotations using JPA

I have a query in which I am attempting to get data out of a table. I have foreign keys pointing to other tables and I want that data back as well except I want it in collections of the entities that I will be returning...
try
{
System.out.println("testing 1..2..3");
listQL = emf.createNamedQuery("getQuickLaunch").getResultList();
System.out.println("What is the size of this list: number "+listQL.size());
qLaunchArr = listQL.toArray(new QuickLaunch[listQL.size()]);
}
That works at getting the initial table, but the other collections don't seem to be filling. At least in the test view window that I have... (I am using WID)...
Here are my JPA's...
#OneToMany(mappedBy="quickLaunchId", cascade=CascadeType.ALL)
private List<QuickLaunchPerm> quickLaunchPermCollection;
#OneToMany(mappedBy="quickLaunchId", cascade=CascadeType.ALL)//fetch=FetchType.EAGER)
private List<QuickLaunchProvider> quickLaunchProviderCollection;
#OneToMany(mappedBy="quickLaunchId")//, cascade=CascadeType.ALL)//fetch=FetchType.EAGER)
private List<QuickLaunchDistlist> quickLaunchDistlistCollection;
Would I need a named query that actually joins the tables that I am calling???
That is what I am thinking I will need.
Thanks.
UDPATE: Named query in question
#NamedQuery(name="getQuickLaunch", query = "SELECT q FROM QuickLaunch q")
So would i need to add an inner join (or left join) for every other collection that I want to pull back?

returning collections using linking tables and JPA

After struggling for days attempting to get back collections that are linked to a table via a foreign key, I just realized that the tables I am linking to are actually LINKING tables to other tables with the actual data (chock one up for normalized tables).
I am still struggling to get collections out of ManyToOne annotated variables with references to foreign keys, but is there a way I can pull the data back from the table that actually contains the information? Has anyone run into an instance of this?
UPDATE: AS per request I will be posting some code instances... This would be my named query in the entity that I will be calling...
#NamedQuery(name="getQuickLaunchWithCollections", query = "SELECT q FROM QuickLaunch q LEFT JOIN FETCH q.quickLaunchDistlistCollection LEFT JOIN FETCH q.quickLaunchPermCollection LEFT JOIN FETCH q.quickLaunchProviderCollection")})
These would be the collections that I am looking to fill...
#OneToMany(mappedBy="quickLaunchId", fetch=FetchType.EAGER)
private List<QuickLaunchPerm> quickLaunchPermCollection;
#OneToMany(mappedBy="quickLaunchId", fetch=FetchType.EAGER)
private List<QuickLaunchProvider> quickLaunchProviderCollection;
#OneToMany(mappedBy="quickLaunchId", fetch=FetchType.EAGER)
private List<QuickLaunchDistlist> quickLaunchDistlistCollection;
As you can see, I have the fetch type set to eager. So technically, I should be getting some data back? But in actuality those are just linking tables the data that I actually want to pull back. I will need to figure out how to get that data back eventually.
This is how I am calling that named query...
listQL = emf.createNamedQuery("getQuickLaunchWithCollections").getResultList();
Alright, it appears as though LEFT JOIN FETCH
is causing my runtime to throw an expception of some kind. It is pretty unclear as to what it is. But I feel as though I am getting no where with that technique. I am going to try something slightly different.
I would suggest simplifying example, to face the problem, since you are going worldwide now.
Specifying mappedBy="quickLaunchId" attribute, you are saying, that QuickLaunchPerm entity has QuickLaunch as its property named "quickLaunchId". Is this true?
If it is not, then you need to define it in QuickLaunchPerm:
#ManyToOne(fetch = FetchType.EAGER)
#JoinColumn(name = "QUICK_LAUNCH_ID")
private QuickLaunch quickLaunchId;
//getters setters

JPA/Eclipselink - Multpile entities in single table

I'm using Eclipselink to map my tables to entities.
I have one big database table (actually it's view) with columns like groupId, groupName, categoryId, categoryName etc. I know it's redundand, but we're trying to minimize queries and it's dynamically created view.
The question is: How to map such table to several entities like Group, Category etc?
You would probably be better off mapping to the real tables and use query optimization to reduce your queries (such as join fetching and batch fetching)
See,
http://java-persistence-performance.blogspot.com/2010/08/batch-fetching-optimizing-object-graph.html
If you really want to have several class map to the same table, you will need to have one Entity and make the rest Embeddables.
See,
http://en.wikibooks.org/wiki/Java_Persistence/Embeddables