DynamoDB Swift4 complex query - swift

I am trying to make a complex query in swift to get data from DynamoDB.
I am able to get all information by using the userID. However there are times that I may not know the entirety of the userID and need to make a more complex query.
For instance, if I know the first name and the last name, and the user id format is "firstname:lastname:email", I need to be able to query all userID's that include the first and last name, then add a where for another column.
I am very new to dynamo and want to accomplish something like the sql query below.
SQL example:
SELECT * FROM mytable
WHERE column2 LIKE '%OtherInformation%'
AND (column1 LIKE '%lastname%' OR column1 LIKE '%firstname%')
Here is the code I have in swift4 for getting the userID if I know it exaclty, not entirely sure how to modify this for complex queries.
func queryDBForUser(Fname: String, Lname: String) {
let userId = Fname + "." + Lname + ":" + (UIDevice.current.identifierForVendor?.uuidString)!
self.UserId = userId
let objectMapper = AWSDynamoDBObjectMapper.default()
let queryExpression = AWSDynamoDBQueryExpression()
queryExpression.keyConditionExpression = "#userId = :userId"
queryExpression.expressionAttributeNames = ["#userId": "userId",]
queryExpression.expressionAttributeValues = [":userId": userId,]
objectMapper.query(CheckaraUsers.self, expression: queryExpression, completionHandler: {(response: AWSDynamoDBPaginatedOutput? ,error: Error?) -> Void in
if let error = error {
print("Amazon DynamoDB Error: \(error)")
return
}
I have also tried many variations along the lines of the following code, with no luck:
queryExpression.keyConditionExpression = "#FirstName = :firstName and #LastName = :lastName,"
queryExpression.expressionAttributeNames = ["#FirstName": "FirstName" , "#LastName": "LastName"]
queryExpression.expressionAttributeValues = [":FirstName": Fname,":LastName": Lname]
Any help would be greatly appreciated, thanks in advance!

You won't be able to do this with a DynamoDB query. When you query a table (or index) in DynamoDB you must always specify the complete primary key. In your case that would mean the full value of "firstname:lastname:email".
You could sort of do this with a DynamoDB scan and a filter expression, but that will look at every item in your table, so it could be slow and expensive. Amazon will charge you for the read capacity necessary to look at every item in the table.
So if you really wanted to, the filter expression for the scan operation would be something like:
"contains (#FirstName, :firstName) and contains (#LastName, : lastName)"
Note that contains looks for an exact substring match, so if you want case insensitive matches (like ILIKE in SQL) it won't work.
If you need to do these types of queries then you need to evaluate whether or not DynamoDB is the right choice for you. DynamoDB is a NoSQL key/value store basically. It trades limited querying functionality for scalability and performance. If you are coming at DynamoDB from a SQL background and are expecting to be able to do freeform queries of anything in your table, you will be disappointed.

Got the query working by adding a secondary index to my DynamoDB table, although this is not what I initially wanted, it still works as now I can query for a value that exists in both columns I needed, without doing a table scan and filtering after.
query code:
queryExpression.indexName = "Index-Name" queryExpression.keyConditionExpression = "#LastName = :LastName and #otherValue = :otherValue"
queryExpression.expressionAttributeNames = ["#LastName": "LastName" , "#otherValue": "otherValue"]
queryExpression.expressionAttributeValues = [":LastName": Lname,":otherValue": self.otherValue!]

Related

Rewrite raw PostgreSQL query to use SelectQueryBuilder from TypeORM

I've written this query that fetches user ids (that's for now, cos I actually need way more fields from the user table as well as from another table called image that is related the user table).
The problem with this query is that it returns a plain object and I need an entity object, I mean I know I could just deserialise it to whatever model I need, but the thing also is that I normally deserialise entity to a required response model. Also, I would like to avoid making
a couple of requests: one fetching user ids and the other fetching right entity objects by those ids using queryBuilder.
So, it seems that one possible solution would be to rewrite this query to make use of queryBuilder straight away.
const matchedUsers = await this.usersRepository.query(
`
SELECT id FROM users
WHERE id IN (
SELECT "usersId" FROM locations_available_fighters_users
WHERE "locationsId" IN (
SELECT "locationsId" FROM locations_available_fighters_users
WHERE "usersId" = ${ me.getId() }
)
) AND is_active IS TRUE
AND id != ${ me.getId() }
AND weight = '${ me.getWeight() }'
AND gender = '${ me.getGender() }'
AND role_name = '${ me.getRoleName() }';
`
);
If the problems you're trying to solve are:
Use a single query
Get the returned data as entity objects rather than raw results
Use QueryBuilder to avoid writing raw queries
I believe what you should be looking into is typeorm subqueries.
For the query you posted in the question, you can try something like below using QueryBuilder:
// Subquery for your inner most subquery,
const locationsQb = connection.getRepository(LocationsAvailableFightersUser)
.createQueryBuilder("lafu_1")
.select("lafu_1.locationsId")
.where("lafu_1.usersId = :usersID", { usersID: me.getId() });
// Subquery for your middle subquery,
const usersQb = connection.getRepository(LocationsAvailableFightersUser)
.createQueryBuilder("lafu_2")
.select("lafu_2.usersId")
.where("lafu_2.locationsId IN (" + locationsQb.getQuery() + ")");
// Query for retrieving `User` entities as you needed,
const matchedUsers = await connection.getRepository(User)
.createQueryBuilder("user")
.where("user.id IN (" + usersQb.getQuery() + ")")
.setParameters(locationsQb.getParameters())
.getMany();
Here I assumed that,
LocationsAvailableFightersUser is the entity for locations_available_fighters_users table
User is the entity for users table
Hope this helps you. Cheers 🍻 !!!

linq nested query conditional order by

I have a parent query with sub query return child ,
what i need to sort child query according to one property in parent.
here is sudo code:
from menu in db.Menus
orderby menu.Order
select new
{
Title= menu.Title,
OrderNumber = menu.Order,
data = (from menuItem in menu.Items
let g = Guid.NewGuid()
orderby g
select new
{
id = worker.ID,
Title = worker.JobTitle
})
.Take(4)
};
that works ok,But what i need to sort some menu in random(NewGuid) and sort others with their item priority some thing like it:
let g = Guid.NewGuid()
orderby menu.ISRandom ? g: menuItem.Order
But it give error about mismatch guid and int.What's the soloution?
second:How can i replace take(4) with take(menu.size)?
thank's
You can solve the ordering problem by adding ToString():
let g = Guid.NewGuid().ToString()
orderbymenu.ISRandom ? g : menuItem.Order.ToString()
Using Take(someProperty) isn't allowed in an EF LINQ query. This is because the take is translated into a TOP(x) clause, which can't possibly refer to a column in the SQL result. You can only do this afterwards, after pulling the results into memory without Take (or taking some reasonable fixed maximum).
It seems you want to read four records from menu.Items at random.
I think you'd be way better off just reading all the items, and then doing the random selection later in code in memory.

How insert item on top table

How insert item on top table in PostgreSQL? That it is possible? In the table I have only two fields as text. First is primary key.
CREATE TABLE news_table (
title text not null primary key,
url text not null
);
I need a simple query for the program in java.
OK, this is my code:
get("/getnews", (request, response) -> {
List<News> getNews = newsService.getNews();
List<News> getAllNews = newsService.getAllNews();
try (Connection connection = DB.sql2o.open()) {
String sql = "INSERT INTO news_table(title, url) VALUES (:title, :url)";
for (News news : getNews) {
if (!getAllNews.contains(news)) {
connection.createQuery(sql, true)
.addParameter("title", news.getTitle())
.addParameter("url", news.getUrl())
.executeUpdate()
.getKey();
}
}
}
return newsService.getNews();
}, json());
The problem is that as it calls getnews method for the second time this new news adds at the end of the table, and there is no extant hronologi news. How this resolve? I use Sql2o + sparkjava.
Probably already I know. I need to reverse the List getnews before I will must contains object getnews and getallnews?
There is no start or end in a table. If you want to sort your data, just use an ORDER BY in your SELECT statements. Without ORDER BY, there is no order.
Relational theory, the mathematical foundation of relational databases, lays down certain conditions that relations (represented in real databases as tables) must obey. One of them is that they have no ordering (i.e., the rows will neither be stored nor retrieved in any particular order, since they are treated as a mathematical set). It's therefore completely under the control of the RDBMS where a new row is entered into a table.
Hence there is no way to ensure a particular ordering of the data without using an ORDER BY clause when you retrieve the data.

Count in jpa without getting result [duplicate]

I like the idea of Named Queries in JPA for static queries I'm going to do, but I often want to get the count result for the query as well as a result list from some subset of the query. I'd rather not write two nearly identical NamedQueries. Ideally, what I'd like to have is something like:
#NamedQuery(name = "getAccounts", query = "SELECT a FROM Account")
.
.
Query q = em.createNamedQuery("getAccounts");
List r = q.setFirstResult(s).setMaxResults(m).getResultList();
int count = q.getCount();
So let's say m is 10, s is 0 and there are 400 rows in Account. I would expect r to have a list of 10 items in it, but I'd want to know there are 400 rows total. I could write a second #NamedQuery:
#NamedQuery(name = "getAccountCount", query = "SELECT COUNT(a) FROM Account")
but it seems a DRY violation to do that if I'm always just going to want the count. In this simple case it is easy to keep the two in sync, but if the query changes, it seems less than ideal that I have to update both #NamedQueries to keep the values in line.
A common use case here would be fetching some subset of the items, but needing some way of indicating total count ("Displaying 1-10 of 400").
So the solution I ended up using was to create two #NamedQuerys, one for the result set and one for the count, but capturing the base query in a static string to maintain DRY and ensure that both queries remain consistent. So for the above, I'd have something like:
#NamedQuery(name = "getAccounts", query = "SELECT a" + accountQuery)
#NamedQuery(name = "getAccounts.count", query = "SELECT COUNT(a)" + accountQuery)
.
static final String accountQuery = " FROM Account";
.
Query q = em.createNamedQuery("getAccounts");
List r = q.setFirstResult(s).setMaxResults(m).getResultList();
int count = ((Long)em.createNamedQuery("getAccounts.count").getSingleResult()).intValue();
Obviously, with this example, the query body is trivial and this is overkill. But with much more complex queries, you end up with a single definition of the query body and can ensure you have the two queries in sync. You also get the advantage that the queries are precompiled and at least with Eclipselink, you get validation at startup time instead of when you call the query.
By doing consistent naming between the two queries, it is possible to wrap the body of the code to run both sets just by basing the base name of the query.
Using setFirstResult/setMaxResults do not return a subset of a result set, the query hasn't even been run when you call these methods, they affect the generated SELECT query that will be executed when calling getResultList. If you want to get the total records count, you'll have to SELECT COUNT your entities in a separate query (typically before to paginate).
For a complete example, check out Pagination of Data Sets in a Sample Application using JSF, Catalog Facade Stateless Session, and Java Persistence APIs.
oh well you can use introspection to get named queries annotations like:
String getNamedQueryCode(Class<? extends Object> clazz, String namedQueryKey) {
NamedQueries namedQueriesAnnotation = clazz.getAnnotation(NamedQueries.class);
NamedQuery[] namedQueryAnnotations = namedQueriesAnnotation.value();
String code = null;
for (NamedQuery namedQuery : namedQueryAnnotations) {
if (namedQuery.name().equals(namedQueryKey)) {
code = namedQuery.query();
break;
}
}
if (code == null) {
if (clazz.getSuperclass().getAnnotation(MappedSuperclass.class) != null) {
code = getNamedQueryCode(clazz.getSuperclass(), namedQueryKey);
}
}
//if not found
return code;
}

Entity Framework Timeout

I have been trying to figure out how to optimize the following query for the past few days and just not having much luck. Right now my test db is returning about 300 records with very little nested data, but it's taking 4-5 seconds to run and the SQL being generated by LINQ is awfully long (too long to include here). Any suggestions would be very much appreciated.
To sum up this query, I'm trying to return a somewhat flattened "snapshot" of a client list with current status. A Party contains one or more Clients who have Roles (ASPNET Role Provider), Journal is returning the last 1 journal entry of all the clients in a Party, same goes for Task, and LastLoginDate, hence the OrderBy and FirstOrDefault functions.
Guid userID = 'some user ID'
var parties = Parties.Where(p => p.BrokerID == userID).Select(p => new
{
ID = p.ID,
Title = p.Title,
Goal = p.Goal,
Groups = p.Groups,
IsBuyer = p.Clients.Any(c => c.RolesInUser.Any(r => r.Role.LoweredName == "buyer")),
IsSeller = p.Clients.Any(c => c.RolesInUser.Any(r => r.Role.LoweredName == "seller")),
Journal = p.Clients.SelectMany(c => c.Journals).OrderByDescending(j => j.OccuredOn).Select(j=> new
{
ID = j.ID,
Title = j.Title,
OccurredOn = j.OccuredOn,
SubCatTitle = j.JournalSubcategory.Title
}).FirstOrDefault(),
LastLoginDate = p.Clients.OrderByDescending(c=>c.LastLoginDate).Select(c=>c.LastLoginDate).FirstOrDefault(),
MarketingPlanCount = p.Clients.SelectMany(c => c.MarketingPlans).Count(),
Task = p.Tasks.Where(t=>t.DueDate != null && t.DueDate > DateTime.Now).OrderBy(t=>t.DueDate).Select(t=> new
{
ID = t.TaskID,
DueDate = t.DueDate,
Title = t.Title
}).FirstOrDefault(),
Clients = p.Clients.Select(c => new
{
ID = c.ID,
FirstName = c.FirstName,
MiddleName = c.MiddleName,
LastName = c.LastName,
Email = c.Email,
LastLogin = c.LastLoginDate
})
}).OrderBy(p => p.Title).ToList()
I think posting the SQL could give us some clues, as small things like the order of OrderBy coming before or after the projection could make a big difference.
But regardless, try extracting the Clients in a seperate query, this will simplify your query probably. And then include other tables like Journal and Tasks before projecting and see how this affects your query:
//am not sure what the exact query would be, and project it using ToList()
var clients = GetClientsForParty();
var parties = Parties.Include("Journal").Include("Tasks")
.Where(p=>p.BrokerID == userID).Select( p => {
....
//then use the in-memory clients
IsBuyer = clients.Any(c => c.RolesInUser.Any(r => r.Role.LoweredName == "buyer")),
...
}
)
In all cases, install EF profiler and have a look at how your query is affected. EF can be quiet surprising. Something like putting OrderBy before the projection, the same for all these FirstOrDefault or SingleOrDefault, they can all have a big effect.
And go back to the basics, if you are searching on LoweredRoleName, then make sure it is indexed so that the query is fast (even though that could be useless since EF could end up not making use of the covering index since it is querying so many other columns).
Also, since this is query is to view data (you will not alter data), don't forget to turn off Entity tracking, that will give you some performance boost as well.
And last, don't forget that you could always write your SQL query directly and project to your a ViewModel rather than anonymous type (which I see as a good practice anyhow) so create a class called PartyViewModel that includes the flatten view you are after, and use it with your hand-crafted SQL
//use your optimized SQL query that you write or even call a stored procedure
db.Database.SQLQuery("select * from .... join .... on");
I am writing a blog post about these issues around EF. The post is still not finished, but all in all, just be patient, use some of these tricks and observe their effect (and measure it) and you will reach what you want.