We use Java Datanucleus 5.0.2 with JDO.
We get occasional exception while several threads are retrieving the same information and NO thread is changing this particular "linkedObject" reference.
NOTE: The object is retrieved using the fetch plan and logs show that.
It is hard to write a test case for this to fail as it is a race condition. But nevertheless I want to ask if someone has experienced it?
Caused by: javax.jdo.JDODetachedFieldAccessException: You have just attempted to access field "linkedObject" yet this field was not detached when you detached the object. Either dont access this field, or detach it when detaching the object.
at com.company.BaseClass.dnGetlinkedObject(BaseClass.java)
at com.company.BaseClass.getLinkedObject(BaseClass.java:71)
...
I can see three threads calling this method
private static <T> T getUniqueQueryJDO(final PersistenceManager pm, final JDOQLQuery query) throws PersistenceException {
try {
final javax.jdo.Query jdoQuery = setUpJDOQuery(pm, query);
jdoQuery.setUnique(true);
T result = null;
final T queryResult = (T) jdoQuery.executeWithMap(query.getMapValues());
if (queryResult != null) {
result = pm.detachCopy(queryResult);
}
jdoQuery.closeAll();
return result;
}
and one of them fails randomly
BaseClass.java
#PersistentDomainObject
#PersistenceCapable(table = "BaseClass", detachable = TRUE)
#Inheritance(strategy = InheritanceStrategy.SUPERCLASS_TABLE)
#FetchGroups(
...
#FetchGroup(name = FETCH_LINKED_OBJECT, members = {#Persistent(name = "linkedObject")})})
...
public class BaseClass {
...
public static final String FETCH_LINKED_CLASS = "FETCH_NAME";
...
#Persistent(defaultFetchGroup = FALSE, columns = {#Column(name = "linkedObjectId", allowsNull = FALSE)}, nullValue = NullValue.EXCEPTION)
private LinkedClass linkedObject;
...
public LinkedClass getLinkedObject() {
return linkedObject;
}
}
Related
I call this method in few threads and write to sessionId parameter the same value each time:
public SessionEntity getOrCreateSession(String sessionId) {
Optional<SessionEntity> optSession = sessionRepository.findByUuid(sessionId);
if (optSession.isPresent()) {
return optSession.get();
}
SessionEntity session = create();
SessionEntity savedSession = sessionRepository.save(session);
}
I also have not constraints on field sessionId in db. After preccessing of few threads I have many records in db with the same sessionId. And I try to use #Lock annotation. I put it on repository method like this:
#Repository
public interface SessionRepository extends JpaRepository<SessionEntity, Long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
Optional<SessionEntity> findByUuid(String uuid);
}
But it doesn't help, I keep getting many records in db with the save sessionId. This my test for creating few threads and calling method:
var uuid = UUID.randomUUID();
ExecutorService executor = Executors.newFixedThreadPool(100);
List<Callable<SessionEntity>> list = new ArrayList<>();
for (int i = 0; i < 100; i++) {
list.add(() ->
sessionService.getOrCreateClientSession(uuid));
}
executor.invokeAll(list);
How should I use annotation #Lock for my case to have only one record in DB?
I seem to be unable to store a simple object to cosmos db?
this is the database model.
public class HbModel
{
public Guid id { get; set; }
public string FormName { get; set; }
public Dictionary<string, object> Form { get; set; }
}
and this is how I store the data into the database
private static void SeedData(HbModelContext dbContext)
{
var cosmosClient = dbContext.Database.GetCosmosClient();
cosmosClient.ClientOptions.AllowBulkExecution = true;
if (dbContext.Set<HbModel>().FirstOrDefault() == null)
{
// No items could be picked hence try seeding.
var container = cosmosClient.GetContainer("hb", "hb_forms");
HbModel first = new HbModel()
{
Id = Guid.NewGuid(),//Guid.Parse(x["guid"] as string),
FormName = "asda",//x["name"] as string,
Form = new Dictionary<string, object>() //
}
string partitionKey = await GetPartitionKey(container.Database, container.Id);
var response = await container.CreateItemAsync(first, new PartitionKey(partitionKey));
}
else
{
Console.WriteLine("Already have data");
}
}
private static async Task<string> GetPartitionKey(Database database, string containerName)
{
var query = new QueryDefinition("select * from c where c.id = #id")
.WithParameter("#id", containerName);
using var iterator = database.GetContainerQueryIterator<ContainerProperties>(query);
while (iterator.HasMoreResults)
{
foreach (var container in await iterator.ReadNextAsync())
{
return container.PartitionKeyPath;
}
}
return null;
}
but when creating the item I get this error message
A host error has occurred during startup operation '3b06df1f-000c-4223-a374-ca1dc48d59d1'.
[2022-07-11T15:02:12.071Z] Microsoft.Azure.Cosmos.Client: Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 24bac0ba-f1f7-411f-bc57-3f91110c4528; Reason: ();.
Value cannot be null. (Parameter 'provider')
no idea why it fails?
the data should not be formatted incorreclty?
It also fails in case there is data in the dictionary.
What is going wrong?
There are several things wrong with the attached code.
You are enabling Bulk but you are not following the Bulk pattern
cosmosClient.ClientOptions.AllowBulkExecution = true is being set, but you are not parallelizing work. If you are going to use Bulk, make sure you are following the documentation and creating lists of concurrent Tasks. Reference: https://learn.microsoft.com/azure/cosmos-db/sql/tutorial-sql-api-dotnet-bulk-import#step-6-populate-a-list-of-concurrent-tasks. Otherwise don't use Bulk.
You are blocking threads.
The call to container.CreateItemAsync(first, new PartitionKey("/__partitionKey")).Result; is a blocking call, this can lead you to deadlocks. When using async operations (such as CreateItemAsync) please use the async/await pattern. Reference: https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#avoid-using-taskresult-and-taskwait
The PartitionKey parameter should be the value not the definition.
On the call container.CreateItemAsync(first, new PartitionKey("/__partitionKey")) the Partition Key (second parameter) should be the value. Assuming your container has a Partition Key Definition of /__partitionKey then your documents should have a __partitionKey property and you should pass the Value in this parameter of such property in the current document. Reference: https://learn.microsoft.com/azure/cosmos-db/sql/troubleshoot-bad-request#wrong-partition-key-value
Optionally, if your documents do not contain such a value, just remove the parameter from the call:
container.CreateItemAsync(first)
Be advised though that this solution will not scale, you need to design your database with Partitioning in mind: https://learn.microsoft.com/azure/cosmos-db/partitioning-overview#choose-partitionkey
Missing id
The model has Id but Cosmos DB requires id, make sure the content of the document contains id when serialized.
We are using ASP.NET Zero and are running into issues with parallel processing from an AppService. We know requests must be transactional, but unfortunately we need to break out to slow running APIs for numerous calls, so we have to do parallel processing.
As expected, we are running into a DbContext contingency issue on the second database call we make:
System.InvalidOperationException: A second operation started on this context
before a previous operation completed. This is usually caused by different
threads using the same instance of DbContext, however instance members are
not guaranteed to be thread safe. This could also be caused by a nested query
being evaluated on the client, if this is the case rewrite the query avoiding
nested invocations.
We read that a new UOW is required, so we tried using both the method attribute and the explicit UowManager, but neither of the two worked.
We also tried creating instances of the referenced AppServices using the IocResolver, but we are still not able to get a unique DbContext per thread (please see below).
public List<InvoiceDto> CreateInvoices(List<InvoiceTemplateLineItemDto> templateLineItems)
{
List<InvoiceDto> invoices = new InvoiceDto[templateLineItems.Count].ToList();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(templateLineItems, async (templateLineItem) =>
{
try
{
XAppService xAppService = _iocResolver.Resolve<XAppService>();
InvoiceDto invoice = await xAppService
.CreateInvoiceInvoiceItem();
invoices.Insert(templateLineItems.IndexOf(templateLineItem), invoice);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) throw new AggregateException(exceptions);
return invoices;
}
How can we ensure that a new DbContext is availble per thread?
I was able to replicate and resolve the problem with a generic version of ABP. I'm still experiencing the problem in my original solution, which is far more complex. I'll have to do some more digging to determine why it is failing there.
For others that come across this problem, which is exactly the same issue as reference here, you can simply disable the UnitOfWork through an attribute as illustrated in the code below.
public class InvoiceAppService : ApplicationService
{
private readonly InvoiceItemAppService _invoiceItemAppService;
public InvoiceAppService(InvoiceItemAppService invoiceItemAppService)
{
_invoiceItemAppService = invoiceItemAppService;
}
// Just add this attribute
[UnitOfWork(IsDisabled = true)]
public InvoiceDto GetInvoice(List<int> invoiceItemIds)
{
_invoiceItemAppService.Initialize();
ConcurrentQueue<InvoiceItemDto> invoiceItems =
new ConcurrentQueue<InvoiceItemDto>();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(invoiceItemIds, (invoiceItemId) =>
{
try
{
InvoiceItemDto invoiceItemDto =
_invoiceItemAppService.CreateAsync(invoiceItemId).Result;
invoiceItems.Enqueue(invoiceItemDto);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) {
AggregateException ex = new AggregateException(exceptions);
Logger.Error("Unable to get invoice", ex);
throw ex;
}
return new InvoiceDto {
Date = DateTime.Now,
InvoiceItems = invoiceItems.ToArray()
};
}
}
public class InvoiceItemAppService : ApplicationService
{
private readonly IRepository<InvoiceItem> _invoiceItemRepository;
private readonly IRepository<Token> _tokenRepository;
private readonly IRepository<Credential> _credentialRepository;
private Token _token;
private Credential _credential;
public InvoiceItemAppService(IRepository<InvoiceItem> invoiceItemRepository,
IRepository<Token> tokenRepository,
IRepository<Credential> credentialRepository)
{
_invoiceItemRepository = invoiceItemRepository;
_tokenRepository = tokenRepository;
_credentialRepository = credentialRepository;
}
public void Initialize()
{
_token = _tokenRepository.FirstOrDefault(x => x.Id == 1);
_credential = _credentialRepository.FirstOrDefault(x => x.Id == 1);
}
// Create an invoice item using info from an external API and some db records
public async Task<InvoiceItemDto> CreateAsync(int id)
{
// Get db record
InvoiceItem invoiceItem = await _invoiceItemRepository.GetAsync(id);
// Get price
decimal price = await GetPriceAsync(invoiceItem.Description);
return new InvoiceItemDto {
Id = id,
Description = invoiceItem.Description,
Amount = price
};
}
private async Task<decimal> GetPriceAsync(string description)
{
// Simulate a slow API call to get price using description
// We use the token and credentials here in the real deal
await Task.Delay(5000);
return 100.00M;
}
}
I am learning JPA, I found out that we have some functions which is already present in Jparepository like save,saveAll,find, findAll etc. but there is nothing like update,
I come across one scenario where I need to update the table, if the value is already present otherwise I need to insert the record in table.
I created
#Repository
public interface ProductInfoRepository
extends JpaRepository<ProductInfoTable, String>
{
Optional<ProductInfoTable> findByProductName(String productname);
}
public class ProductServiceImpl
implements ProductService
{
#Autowired
private ProductInfoRepository productRepository;
#Override
public ResponseMessage saveProductDetail(ProductInfo productInfo)
{
Optional<ProductInfoTable> productInfoinTable =
productRepository.findByProductName(productInfo.getProductName());
ProductInfoTable productInfoDetail;
Integer quantity = productInfo.getQuantity();
if (productInfoinTable.isPresent())
{
quantity += productInfoinTable.get().getQuantity();
}
productInfoDetail =
new ProductInfoTable(productInfo.getProductName(), quantity + productInfo.getQuantity(),
productInfo.getImage());
productRepository.save(productInfoDetail);
return new ResponseMessage("product saved successfully");
}
}
as you can see, I can save the record if the record is new, but when I am trying to save the record which is already present in table it is giving me error related to primarykeyviolation which is obvious. I checked somewhat, we can do the update by creating the entitymanager object or jpa query but what if I dont want to use both of them. is there any other way we can do so ?
update I also added the instance of EntityManager and trying to merge the code
#Override
public ResponseMessage saveProductDetail(ProductInfo productInfo)
{
Optional<ProductInfoTable> productInfoinTable =
productRepository.findByProductName(productInfo.getProductName());
ProductInfoTable productInfoDetail;
Integer price = productInfo.getPrice();
if (productInfoinTable.isPresent())
{
price = productInfoinTable.get().getPrice();
}
productInfoDetail =
new ProductInfoTable(productInfo.getProductName(), price, productInfo.getImage());
em.merge(productInfoDetail);
return new ResponseMessage("product saved successfully");
but no error, no execution of update statements in log, any possible reasons for that ?
}
I suspect you need code like this to solve the problem
public ResponseMessage saveProductDetail(ProductInfo productInfo)
{
Optional<ProductInfoTable> productInfoinTable =
productRepository.findByProductName(productInfo.getProductName());
final ProductInfoTable productInfoDetail;
if (productInfoinTable.isPresent()) {
// to edit
productInfoDetail = productInfoinTable.get();
Integer quantity = productInfoDetail.getQuantity() + productInfo.getQuantity();
productInfoDetail.setQuantity(quantity);
} else {
// to create new
productInfoDetail = new ProductInfoTable(productInfo.getProductName(),
productInfo.getQuantity(), productInfo.getImage());
}
productRepository.save(productInfoDetail);
return new ResponseMessage("product saved successfully");
}
I have method called selectAll which has to return records selected from database, but when I want to return them as String it doesn't allow it and gives me:
Local variable S defined in an enclosing scope must be final or effectively final.
I tried to add final to my String but it stills the same.
Here is my selectAll() method:
public String selectAll(String table, String order) {
String S;
S = "";
this.getCnx().getConnection(res -> {
if (res.succeeded()) {
SQLConnection connection = res.result();
connection.queryWithParams(ReqSql.SELECT_ALL, new JsonArray().add(table).add(order), (ar) -> {
if (ar.failed()) {
this.sendError(500, response);
} else {
JsonArray arr = new JsonArray();
ar.result().getRows().forEach(arr::add);
S = ar.result().getRows().toString();
response.putHeader("content-type", "application/json").end(arr.encode());
}
});
}
});
}
public AsyncSQLClient getCnx(){
JsonObject mySQLClientConfig = new JsonObject()
.put("host", "localhost")
.put("database", "test")
.put("username", "test")
.put("password", "test")
.put("port", 3306);
return MySQLClient.createShared(vertx, mySQLClientConfig);
}
And I create another class for my requests:
public class ReqSql {
public static final String SELECT_ALL = "SELECT * FROM ? ORDER BY ? ASC";
}
Regards.
Problem is that you need to declare local variables as final if you want to use them in a Lambda expression. But you cannot assign a value to a final variable. So this will not work:
final String S = "";
this.getCnx().getConnection(res -> {
//...
S = ar.result().getRows().toString();
//...
});
Vert.x is highly asynchronous. That means that most operations, like getConnection(), will infect return immediately but their results, in this case SQLConnection, will be available to the Handler to a later point in time.
If you try to make a asynchronous result available in the main program flow that wouldn't work. What every you want to do with S is probably wrong. So if you don't need S afterwards I would convert S to a local variable.
I suggest you read about Future in the documentation. A Future is a placeholder for results of asynchronous calls. Vert.x is full of asynchronous calls. With Future you can do something like this:
Future<SQLConnection> connection = Future.future();
this.getCnx().getConnection(res -> {
if (res.succeeded()) {
logger.info("Got a connection.");
connection.complete(res.result());
} else {
connection.fail(res.cause());
}
});
You specify a Handler on a Future for retrieving the asynchronous result in a much more readable manner than callbacks/Lambdas:
connection.setHandler(res -> {
...
})