I'm trying to optimize my application to perform at maximum speed. I intended on having two threads each executing a batch request of sales receipts additions. I also intended on having two parallel threads each with a batch request of customer additions. I was wondering whether this is possible or would the API lock the sales receipt/customer table in QuickBooks thus only allowing one thread to perform.
From my research I know that there a three types of entities (Name list, transaction and supporting entities). So what are the causes of locks on these entities, ie what scenario's will cause a lock? Is there any documentation on this matter I couldn't seem to find any?
Thanks
Lock is applicable for Name entities(Vendor, Customer and Employee ). While creating a new name entity, service ensures that an unique name is getting inserted in cloud. So, it puts a lock across all names of these 3 entities.
You can try this scenario using a decent payload.
public static void main(String args[]) {
PropertyConfigurator
.configure("log4j.properties");
Config.setProperty(Config.SERIALIZATION_REQUEST_FORMAT, "xml");
Config.setProperty(Config.SERIALIZATION_RESPONSE_FORMAT, "xml");
final Context platformContext = getPlatformContext("QBO");
final QBOV3ProdTest qbov3ProdTest = new QBOV3ProdTest(platformContext);
Thread customerThread = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < 15; i++) {
qbov3ProdTest.addCustomer();
}
}
});
customerThread.start();
Thread vendorThread = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < 15; i++) {
qbov3ProdTest.addVendor();
}
}
});
vendorThread.start();
}
private void addCustomer() {
Customer customer = new Customer();
customer.setDisplayName("TestCustomer-" + staticCount++);
try {
this.service.add(customer);
} catch (FMSException e) {
e.printStackTrace();
}
}
private void addVendor() {
Vendor vendor = new Vendor();
vendor.setDisplayName("TestVendor-" + staticCount++);
try {
this.service.add(vendor);
} catch (FMSException e) {
e.printStackTrace();
}
}
Service doesn't return a proper response. Wherever it fails, service returns 401. Please let me know if you can reproduce this behaviour while trying this use-case in your test QBO account.
Thanks
This is not exactly a DB locking rule but because of the way we are saving data to our cache for Names lists.
We do not allow users to update these entities in a multi-threaded manner:
Account,
Department,
Item,
Class,
Customer,
Employee,
Vendor,
PaymentMethod,
Terms.
The above has been confirmed by our engineering team.
Related
We are using ASP.NET Zero and are running into issues with parallel processing from an AppService. We know requests must be transactional, but unfortunately we need to break out to slow running APIs for numerous calls, so we have to do parallel processing.
As expected, we are running into a DbContext contingency issue on the second database call we make:
System.InvalidOperationException: A second operation started on this context
before a previous operation completed. This is usually caused by different
threads using the same instance of DbContext, however instance members are
not guaranteed to be thread safe. This could also be caused by a nested query
being evaluated on the client, if this is the case rewrite the query avoiding
nested invocations.
We read that a new UOW is required, so we tried using both the method attribute and the explicit UowManager, but neither of the two worked.
We also tried creating instances of the referenced AppServices using the IocResolver, but we are still not able to get a unique DbContext per thread (please see below).
public List<InvoiceDto> CreateInvoices(List<InvoiceTemplateLineItemDto> templateLineItems)
{
List<InvoiceDto> invoices = new InvoiceDto[templateLineItems.Count].ToList();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(templateLineItems, async (templateLineItem) =>
{
try
{
XAppService xAppService = _iocResolver.Resolve<XAppService>();
InvoiceDto invoice = await xAppService
.CreateInvoiceInvoiceItem();
invoices.Insert(templateLineItems.IndexOf(templateLineItem), invoice);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) throw new AggregateException(exceptions);
return invoices;
}
How can we ensure that a new DbContext is availble per thread?
I was able to replicate and resolve the problem with a generic version of ABP. I'm still experiencing the problem in my original solution, which is far more complex. I'll have to do some more digging to determine why it is failing there.
For others that come across this problem, which is exactly the same issue as reference here, you can simply disable the UnitOfWork through an attribute as illustrated in the code below.
public class InvoiceAppService : ApplicationService
{
private readonly InvoiceItemAppService _invoiceItemAppService;
public InvoiceAppService(InvoiceItemAppService invoiceItemAppService)
{
_invoiceItemAppService = invoiceItemAppService;
}
// Just add this attribute
[UnitOfWork(IsDisabled = true)]
public InvoiceDto GetInvoice(List<int> invoiceItemIds)
{
_invoiceItemAppService.Initialize();
ConcurrentQueue<InvoiceItemDto> invoiceItems =
new ConcurrentQueue<InvoiceItemDto>();
ConcurrentQueue<Exception> exceptions = new ConcurrentQueue<Exception>();
Parallel.ForEach(invoiceItemIds, (invoiceItemId) =>
{
try
{
InvoiceItemDto invoiceItemDto =
_invoiceItemAppService.CreateAsync(invoiceItemId).Result;
invoiceItems.Enqueue(invoiceItemDto);
}
catch (Exception e)
{
exceptions.Enqueue(e);
}
});
if (exceptions.Count > 0) {
AggregateException ex = new AggregateException(exceptions);
Logger.Error("Unable to get invoice", ex);
throw ex;
}
return new InvoiceDto {
Date = DateTime.Now,
InvoiceItems = invoiceItems.ToArray()
};
}
}
public class InvoiceItemAppService : ApplicationService
{
private readonly IRepository<InvoiceItem> _invoiceItemRepository;
private readonly IRepository<Token> _tokenRepository;
private readonly IRepository<Credential> _credentialRepository;
private Token _token;
private Credential _credential;
public InvoiceItemAppService(IRepository<InvoiceItem> invoiceItemRepository,
IRepository<Token> tokenRepository,
IRepository<Credential> credentialRepository)
{
_invoiceItemRepository = invoiceItemRepository;
_tokenRepository = tokenRepository;
_credentialRepository = credentialRepository;
}
public void Initialize()
{
_token = _tokenRepository.FirstOrDefault(x => x.Id == 1);
_credential = _credentialRepository.FirstOrDefault(x => x.Id == 1);
}
// Create an invoice item using info from an external API and some db records
public async Task<InvoiceItemDto> CreateAsync(int id)
{
// Get db record
InvoiceItem invoiceItem = await _invoiceItemRepository.GetAsync(id);
// Get price
decimal price = await GetPriceAsync(invoiceItem.Description);
return new InvoiceItemDto {
Id = id,
Description = invoiceItem.Description,
Amount = price
};
}
private async Task<decimal> GetPriceAsync(string description)
{
// Simulate a slow API call to get price using description
// We use the token and credentials here in the real deal
await Task.Delay(5000);
return 100.00M;
}
}
There's more than 15 items in my azure database table called Events.
I've tried to run most of the commands found on
https://learn.microsoft.com/en-us/azure/app-service-mobile/app-service-mobile-android-how-to-use-client-library such as :
List<Events> results = eventsTable.execute().get()
and
List<Events> results = eventsTable.select("Events").execute().get();
and
List<Events> results = eventsTable.top(20).execute().get();
to return all the row items in the table. The queries seem to run on the last row of the table only and returns the last row or nothing at all when query is executed.
Though the ToDoItem Quickstart from Azure works perfectly with all the queries - which is odd.
Here's some of the code
ArrayList<Events> events = new ArrayLists<Events>();
private void EventsFromTable() {
AsyncTask<Void, Void, Void> task = new AsyncTask<Void, Void, Void>(){
#Override
protected Void doInBackground(Void... params) {
try {
final List<Events> results = EventsTable.execute().get();
runOnUiThread(new Runnable() {
#Override
public void run() {
for (Events event : results) {
Events ev = new Events(event.getName(), event.getVenue(), event.getDate());
events.add(ev);
System.out.println("size is " +events.size());
<======This returns "size is 1"======>
}
}
});
} catch (final Exception e){
createAndShowDialogFromTask(e, "Error");
}
return null;
}
};
runAsyncTask(task);
}
Might any one know what the matter is?
Thanks
According to your code, the variable events seems to be a public shared instance of ArraryList in your Android app, so I don't know whether exists the case which multiple threads access it concurrently. The implementation of ArrayList class is not synchronized, please see here.
So please use the code below instead of the code ArrayList<Events> events = new ArrayLists<Events>(); when you shared the variable between UI thread and data async task thread.
List<Events> events = Collections.synchronizedList(new ArrayLists<Events>());
And I think it's better for copying data retrieved from table via addAll method, not add method for each, as the code below.
#Override
public void run() {
events.addAll(results);
}
We currently use MS Dynamics CRM V4 and are in the process of upgrading to 2015. I've got the task of updating some of our plugins.
1 of the things I've come across that is a little confusing is whether I still need to do some sort of check on the stage of the pipeline to determine if it's a parent or child. As I understand it the parent and child pipelines have been merged into 1 as of 2011, so how should the following code be altered?
public CrmServiceProxy(IPluginExecutionContext context, Guid userId)
{
if (context.InvocationSource == MessageInvocationSource.Parent)
{
iCrmService = context.CreateCrmService(userId);
}
else
{
try
{
RegistryKey regkey = Registry.LocalMachine.OpenSubKey("SOFTWARE\\Microsoft\\MSCRM");
string crmUrl = regkey.GetValue("ServerUrl").ToString();
string crmServiceUrl = string.Concat(crmUrl, "/2007/crmservice.asmx");
crmService = CreateCrmService(crmServiceUrl, context, userId);
}
catch (Exception)
{
throw new InvalidPluginExecutionException("Unable to create CrmServiceProxy - the service URL cannot be read from the Registry");
}
}
}
I've made a start like this:
private readonly IOrganizationService iCrmService;
private IOrganizationServiceFactory serviceFactory;
public CrmServiceProxy(IServiceProvider serviceProvider, Guid userId)
{
IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
if (context.Stage == 10) //10 should indicate it's the parent
{
iCrmService = serviceFactory.CreateOrganizationService(context.UserId);
}
else
{
try
{
RegistryKey regkey = Registry.LocalMachine.OpenSubKey("SOFTWARE\\Microsoft\\MSCRM");
string crmUrl = regkey.GetValue("ServerUrl").ToString();
string crmServiceUrl = string.Concat(crmUrl, "/2007/crmservice.asmx");
iCrmService = serviceFactory.CreateOrganizationService(crmServiceUrl, context, userId); //doesn't work, just something I was trying
}
catch (Exception)
{
throw new InvalidPluginExecutionException("Unable to create CrmServiceProxy - the service URL cannot be read from the Registry");
}
}
}
So, I understand that previously in V4 you needed to use CrmService for child pipelines and ICrmService for parent ones, hence the if statement to determine which pipeline it came from. However, do I still need to do this kind of check or can I just do away with the whole if statement and just create the service using ICrmService?
Actually Parent and Child pipelines were not merged. They still exist in Dynamics CRM 2011. I guess Microsoft simplified the model to prevent confusion. Also, in CRM 4.0 the child pipeline did not support free access to the CRM service; hence the different ways of creating ICrmService instances.
E.g. when an AssignRequest is issued, the following plugin steps are invoked:
Validate Assign
PreOperate Assign
PreOperate Update
(Platform operations)
PostOperate Update
PostOperate Assign
Async Update
Async Assign
Steps 3 and 5 are actually child pipeline steps; they both have a parent context belonging to the Assign message.
You need to rewrite also the class declaration, here an example you can use:
namespace PluginNamespace
{
public class MyPluginClass : IPlugin
{
public void Execute(IServiceProvider serviceProvider)
{
IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);
// ...
}
}
}
I want to implement synchronization in salesforce using apex.
My Requirement is that, i have to update a field from the future method.
I have a field in database with name Counter_c and i am calling a method that method will call 3 future method. Those future method will try to increment value of counter_c by 1. But field is not getting update.
Here is my code:
public void generateReports() {
ReportHistory__c repHst = new ReportHistory__c();
repHst.Counter__c = 0;
insert repHst;
generateReport1(repHst.Id);
generateReport2(repHst.Id);
generateReport3(repHst.Id);
}
#future
public static void generateReport1(Id id) {
List<ReportHistory__c> lstRep = [select Counter__c rom ReportHistory__c where Id = :id];
if(!lstRep.isEmpty()) {
++lstRep[0].Counter__c;
}
update lstRep;
}
#future
public static void generateReport2(Id id) {
List<ReportHistory__c> lstRep = [select Counter__c rom ReportHistory__c where Id = :id];
if(!lstRep.isEmpty()) {
++lstRep[0].Counter__c;
}
update lstRep;
}
#future
public static void generateReport3(Id id) {
List<ReportHistory__c> lstRep = [select Counter__c rom ReportHistory__c where Id = :id];
if(!lstRep.isEmpty()) {
++lstRep[0].Counter__c;
}
update lstRep;
}
After executing above code i want Counter__c should be 3. But its still 0 or sometime its 1.
Please help me, if there is any way so that i can control future call so that every future call should update value of Counter__c by 1.
Thanks,
Vivek
From your example code it appears the lstRep.isEmpty() check is missing the negation.
E.g. Add a ! in the if condition.
#future
public static void generateReport1(Id id) {
List<ReportHistory__c> lstRep = [select Counter__c rom ReportHistory__c where Id = :id];
if(!lstRep.isEmpty()) {
lstRep[0].Counter__c++;
}
update lstRep;
}
It is also possible that your future methods are being pulled off the Queue and processed by different Application Servers in parallel. If this is the case your have a concurrency issue. Have a read of Asynchronous Processing in Force.com. You could try adding the FOR UPDATE keyword to your SOQL query, but this may cause some of your future methods to timeout.
I have a problem when running multiple impersonations of users in EWS, when I want to recieve notifications on each of the impersonated persons calendars (possible 100 persons).
Currently I have an outlook account who have rights to impersonate all other users, and all the ExchangeService-objects get this accounts credentials
Short version is, that when I try to bind to an appointment via the unique ID it works as long as I only have one thread running. When I start a new thread containing a new Exchangeservice with its own subscription I dont recieve any response on the Appointment.Bind()-request.
When I run two instances of my program with only 1 thread in each it works fine, but as soon as I start a new thread with a new ExchangeService the Appointment.Bind() doesnt give any response.
The weird part about this is, that it worked fine two weeks ago, but suddenly it stopped working and I didnt change my code.
I have created a quick demo of my problem:
class Program
{
static void Main(string[] args)
{
var x = new OutlookListener("user1#server.com");
var y = new OutlookListener("user2#server.com");
new Thread(x.Start).Start();
new Thread(y.Start).Start();
while (true)
{
}
}
}
class OutlookListener
{
private ExchangeService _ExchangeService;
private AutoResetEvent _Signal;
public OutlookListener(string emailToImp)
{
_ExchangeService = new ExchangeService(ExchangeVersion.Exchange2010_SP1)
{
Credentials = new NetworkCredential("superuser#server.com", "password"),
Url = new Uri("exchangeUrl"),
ImpersonatedUserId = new ImpersonatedUserId(ConnectingIdType.SmtpAddress, emailToImp)
};
}
public void Start()
{
var subscription = _ExchangeService.SubscribeToStreamingNotifications(new FolderId[] { WellKnownFolderName.Calendar },
EventType.Created);
var connection = CreateStreamingSubscription(_ExchangeService, subscription);
Console.Out.WriteLine("Subscription created.");
_Signal = new AutoResetEvent(false);
_Signal.WaitOne();
subscription.Unsubscribe();
connection.Close();
}
private StreamingSubscriptionConnection CreateStreamingSubscription(ExchangeService service, StreamingSubscription subscription)
{
var connection = new StreamingSubscriptionConnection(service, 30);
connection.AddSubscription(subscription);
connection.OnNotificationEvent += OnNotificationEvent;
connection.OnSubscriptionError += OnSubscriptionError;
connection.OnDisconnect += OnDisconnect;
connection.Open();
return connection;
}
private void OnNotificationEvent(object sender, NotificationEventArgs args)
{
// Extract the item ids for all NewMail Events in the list.
var newMails = from e in args.Events.OfType<ItemEvent>()
where e.EventType == EventType.Created
select e.ItemId;
foreach (var newMail in newMails)
{
var appointment= Appointment.Bind(_ExchangeService, newMail); //This is where I dont get a response!
Console.WriteLine(appointment.Subject);
}
}
private void OnSubscriptionError(object sender, SubscriptionErrorEventArgs args)
{
}
private void OnDisconnect(object sender, SubscriptionErrorEventArgs args)
{
}
}
Any suggestions?
I have had the same issue and found that my EWS solution was limited by two factors.
The System.Net.ServicePointManager.DefaultConnectionLimit is by default set to 2, which I've changed to 20 which i beleive to match the throttling policy of Exchange Online.
Second the ConnectionGroupName property on the ExchangeService object can be used to pool connections into different relevant groups which have a limit of concurrent connections cohernet with the DefaultConnectionLimit property.
A way to override the settings is to set the ConnectionGroupName property to a uniquevalue for each ExchangeService object you create.
ExchangeService exchangeService = new ExchangeService()
{
ConnectionGroupName = Guid.NewGuid().ToString()
};
Why do you need multiple threads ?
In my case , I have created a dictionary of Services based on the smtpaddress for each email I want to impersonate, and I subscribe to them all. All can happen in one thread, and all notification from any user will be handled in the OnNotificationEvent .
[THIS CODE IS JUST TO SHOW THE LOGIC AND IS NOT COMPLETE FOR FULL COMPILATION AND RUN]
var service = new ExchangeService(exchangeVersion);
var serviceCred = ((System.Net.NetworkCredential)(((WebCredentials)(Services.First().Value.Credentials)).Credentials));
service.Credentials = new WebCredentials(serviceCred.UserName, serviceCred.Password);
service.AutodiscoverUrl(userSmtp, RedirectionUrlValidationCallback);
service.ImpersonatedUserId = new ImpersonatedUserId(ConnectingIdType.SmtpAddress, userSmtp);
Services.Add(userSmtp, service);
Note that Services.First().Value is the service that can impersonate all the other users, and here it is cloned as the number of the user.
After that Subscriptions for the all services (note that now each service is impersonating different user)
foreach (var service in Services.Values)
{
SubscribeToService(service);
}
and the definition for SubscribeToService is as follow
private void SubscribeToService(ExchangeService service)
{
if (service.ImpersonatedUserId == null)
return;
if (service.Url == null)
return;
var serviceName = service.ImpersonatedUserId.Id;
var streamingSubscription =
service.SubscribeToStreamingNotifications(new FolderId[] { WellKnownFolderName.DeletedItems, WellKnownFolderName.Calendar },
EventType.FreeBusyChanged, EventType.Moved, EventType.Created, EventType.Modified);
if (!Connections.ContainsKey(service.Url))
{
Connections.Add(service.Url, new StreamingSubscriptionConnection(service, 30));
}
var connection = Connections[service.Url];
CloseConnection(connection);
if (!_subscriptions.ContainsKey(serviceName))
{
_subscriptions.Add(serviceName, streamingSubscription);
connection.AddSubscription(streamingSubscription);
}
}
}
All of this can happen in one single thread, and I hope my answer will help you
Cheers