Azure Mobile app service & EF Table Column Restriction? - entity-framework

I am using an Azure Mobile app service with Odata and Entity Framework. I have table that has around 90 columns and it doesn't work unless i comment out some of the columns in the data model. It doesn't matter what columns I comment out a query won't work if with all the columns.
The azure mobile service debugger tells me:
"An unhandled exception of type 'System.StackOverflowException' occurred in EntityFramework.dll" The odata query works fine and returns 200 ok but then the EF talking to the SQL (it's an azure sql server) has a freakout and never returns anything... until i remove some columns from the datamodel.
This is just with the datamodels and tablecontrollers defined out of the box and i'm sending queries with Postman. I have 20 other tables in this service that work perfectly. I can't find a size limit or anything base on number of columns you can use with this. Any help is awesome.
Postman gives me:
"502 - Web server received an invalid response while acting as a gateway or proxy server" but that is too generic to trace. looked in all the azure logs that are possible and nothing. just said there was an error

I would recommend you Enable diagnostics logging for your mobile app to collection the detailed error message. Also, you could modify your TableController and log the detailed SQL statement as follows:
protected override void Initialize(HttpControllerContext controllerContext)
{
base.Initialize(controllerContext);
var context = new MobileServiceContext();
//add this line
context.Database.Log += (msg) =>
{
System.Diagnostics.Trace.TraceInformation($"{msg}{Environment.NewLine}");
};
DomainManager = new EntityDomainManager<Message>(context, Request);
}
Also, you could execute your relevant SQL statement via SSMS and check the query execution plan to narrow this issue. Moreover, for your mobile client, you could only retrieve the specific columns from your remote table. Here the code snippet for C#:
var query =
from r in table.CreateQuery()
select new { r.Text, r.Complete, r.UpdatedAt, r.Version };

After 8 months of randomly trying I actually got it working!
Instead of returning the IQueryable directly from the helper that's built into the AzureMobileServiceServer namespace convert it to an IEnumerable, then re-select everything then change it back to an IQueryable.
No idea why this works but it negates the stackoverflow problem that is happening otherwise. I think this adds a bit more time when syncing offline but if it works I'll take it.
public IQueryable<ItemLibrary> GetAllItemLibrarys()
{
// return Query(); // !! this is the original from the table controller
return Query().AsEnumerable()
.Select(s => s).AsQueryable(); // !! replace with this
}

Related

SAP CDS Odata URL in ADF

I am new to azure data factory (ADF) and trying to create a dataset from an Odata source. The only problem here is that the Odata URL was developed in SAP CDS and so has custom query options as shown below:
"http://XXXXXXX/ZC_XXX_TU_SR_ACTIVITY_CDS/ZC_XXX_TU_SR_Activity(p_warehouse='E065',p_from=datetimeoffset'2021-06-01T00:01:01',p_to=datetimeoffset'2021-08-11T23:01:01')/Set"
When choosing the path I expect only one path in the options but I get 2 - ZC_XXX_TU_SR_Activity and ZC_XXX_TU_SR_ActivitySet so I am unsure of which one to use even though I have tried both
When writing the query, I have tried:
?(p_warehouse='E065',p_from=datetimeoffset'2021-06-01T00:01:01',p_to=datetimeoffset'2021-08-11T23:01:01')/Set
?(p_warehouse='E065'&p_from=datetimeoffset'2021-06-01T00:01:01'&p_to=datetimeoffset'2021-08-11T23:01:01')/Set
?(p_warehouse=%27E065%27&p_from=datetimeoffset%272021-06-01T00:01:01%27&p_to=datetimeoffset%272021-08-11T23:01:01%27)/Set
I have also tried to use all 3 options without the '?', "()" and the '/Set' but I am still getting errors.
I get this error:
"query (p_warehouse='E065',p_from=datetimeoffset'2021-06-01T00:01:01',p_to=datetimeoffset'2021-08-11T23:01:01')/Set failed with status code InternalServerError and message SY/530An exception was raised."
I have run out of ideas now and don't know what else to do. Please help. Thanks!
Note: The OData connector copies data from the combined URL: [URL specified in linked service]/[path specified in dataset]?[query specified in copy activity source].
Here, I could see that you have the root path as http://XXXXXXX/ZC_XXX_TU_SR_ACTIVITY_CDS and the resource path as ZC_XXX_TU_SR_Activity or ZC_XXX_TU_SR_ActivitySet.
So, there is an issue passing the query in :
System Query Option :
System Query Options are query string options that a client may use to
alter the amount and order of data returned by an OData service for
the URL-identified resource. All System Query Options have a “$”
character before their names.
Custom Query Option:
Because the character "$" is reserved for system query options, custom
query options MUST NOT begin with it. A custom query option can start
with the “#” character, however this can cause custom query options to
clash with function parameter values supplied via Parameter Aliases.
This URL addresses, for example, give a ‘secURLtytoken' through a
custom query option.
This is for more information: URL Conventions (OData Version 3.0)

Sequelize TypeScript returns a wrong number of items when calling findAll() on model factory

I'm using Sequelize to query my items from the local Postgres database, then I found something really scary.
I just query all database entries of a specific model using:
model.findAll({attributes: ["columnName"]}).then((data) => {
console.log(data.length); // logged: 800
});
Everything looks good, but then I went and bind the Postgres datasource directly from my IDE, opened a console from there and ran the same query to validate the overall number of items returned from Sequelize:
select count(*) from tableOfModel; # returned: 600
I got a shock from that moment as the number of items I was getting was only a total of 600 items and was different from what I got with Sequelize.
What might be wrong here?
I found where the error was: a collaborator changed the database configuration without telling me so, so the database was remotely hosted on server, so everything was okay with Sequelize. I should have noticed it at first :).

Getting translated records in CommandController

I've searched and debugged the last couple of days how to obtain the
translated version of a DomainModel object in a CommandController in Typo3 v8.7.
In Typo3 4.5/4.7 I've done the following:
- input: DomainModel in default language
- build a query that finds the record with the l10n_parent matching the
given domain model
- obtain a new domain model with the desired sys_language_uid
Unfortunately this does not work in Typo3 v8.7 any more. I always get
the domain model for the default language.
I've traced this down to the method Typo3DbBackend::doLanguageAndWorkspaceOverlay
called via Typo3DbBackend::getObjectDataByQuery
The query returns the correct (translated) row (seen in the debugger and also the mysql query log), but then the variable
$row gets overwritten in doLanguageAndWorkspaceOverlay no matter how I
set the querySettings setLanguageOverlayMode and setLanguageMode.
So what is the correct way to get a translated domain model in a
CommandController?
UPDATE:
I think I'm a step further. If I add ->setQueryLanguage(1) to the query settings, doLanguageAndWorkspaceOverlay() tries to fetch the translated record for language = 1. But in order to succeed I need to trick the FrontendGroupRestriction class by setting $GLOBALS['TSFE']->gr_list = "0,-2";.
The array returned by doLanguageAndWorkspaceOverlay() now contains all translated entries, except the uid, which is still the uid from the record in the main language. The uid of the translated record is stored in _LOCALIZED_UID.
Now my problem now is that I still get the record in the main laguage because DataMapper->mapSingleRow() (called via DataMapper->map()) has some kind of object cache and thus returns the object in the default language (because the uid is still the one of the record in the main language).
All this seems a little hackish. So again my question: what is the correct way to get a translated domain model in a CommandController?
thanks,
mika
p.s.: I've set up the second language in the backend and creating a translated record works just fine. My question is just how to I get an existing translated record in a CommandController.
alternative solution:
based on the solution above, I decided that I can do almost everything
on my own. So what I do now is
i) create an independend querybuilder for the according table:
$queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)->getQueryBuilderForTable($tableName);`
ii) select the record with the desired l10n_parent and sys_language_uid
$query = $queryBuilder->select('*')
->from($tableName)
->where($queryBuilder->expr()->eq('sys_language_uid', $langId))
->andWhere($queryBuilder->expr()->eq('l10n_parent', $parentUid))
->execute();
iii) fetch all records into an array
$rows = $query->fetchAll();
iv) invoke the DataMapper manually to get the object
$dataMapper = $this->objectManager->get(DataMapper::class);
$translated = $dataMapper->map($className, $rows);
I know it has nothing to do with the ModelRepository any more, but it
works quite fine for now...
that's all folks
My solution to the issue described above:
To avoid invoking the DataMapper as part of the query from
Typo3DbBackend, I used a raw query (argument for ->execute()) and get
back an array, that already went through language overlay etc.
BUT: in
the array the '_LOCALIZED_UID' is still available. So I overwrite the
uid with the value from '_LOCALIZED_UID' and invoke the DataMapper
manually. Quite cumbersome and very hackish to overcome the Typo3
backend shortcomings...

javax.persistence.cache.retrieveMode and javax.persistence.cache.retrieveMode does not work with NamedQuery when used with Eclipse Link ORM

Query Hints not working in Eclipse Link 2.3.2/2.6.1 when used to fetch data from second level Cache
Used Hints,
#QueryHint(name = "javax.persistence.cache.retrieveMode", value = "USE"),
#QueryHint(name = "javax.persistence.cache.storeMode ", value = "USE")
Tried with below options.
1. Added JPA Hints to Named query itself
#NamedQuery(
name = TestEntity.FIND_BY_CODE,
query = "select t from Test t where t.code = :code",
hints = {
#QueryHint(name = "javax.persistence.cache.retrieveMode", value = "USE"),
#QueryHint(name = "javax.persistence.cache.storeMode ", value = "USE") })
2. Adding hints to the Entity Manager Itself after injecting it
em.setProperty("javax.persistence.cache.retrieveMode", CacheRetrieveMode.USE);
em.setProperty("javax.persistence.cache.storeMode", CacheRetrieveMode.USE);
3. Added JPA hints at the time of Query execution
em.createNamedQuery(TestEntity.FIND_BY_CODE,
AlertCategoryType.class).setHint("javax.persistence.cache.retrieveMode", CacheRetrieveMode.USE)
.setHint("javax.persistence.cache.storeMode", CacheStoreMode.USE)
.setParameter("code", code).getSingleResult();
None of the above usage of hints worked. Then i tried debugging on three different options what i found is,
the Data Base Query which formed after setting these hints is passing hints as key/value pair below.
eclipselink.query.hints => {javax.persistence.cache.retrieveMode=USE, javax.persistence.cache.storeMode=USE}
Where eclipselink.query.hints is the key even when we have set the JPA hints. This is something we don't have control over to change this.
But when i pass Eclipse Link provided hints as below , It started working as expected and the Results are fetched from Cache and not from the DB.
eclipselink.query.hints => {eclipselink.query-results-cache.size=500, eclipselink.query-results-cache=true}
It means That When we use Eclipse Link it only recognizes Eclipse Link provided hints according to the key[ above shown] we see in Query.
Please suggest any work around to get the JPA Hints working
Environment I'm using is
Eclispe Link 2.3.2/2.6.1
Runnin fin serve Glassfish 4.1[payara]
Java8/JEE7
The query hint you state is working (eclipselink.query-results-cache) is completely unrelated - it creates a new cache for the query results so that next time you execute that same query, the results are already there, so it does not need to execute the query again. This is outside (above an beyond) the second level cache.
The settings you refer to as not working affect the second level cache. Without more information, I'm going to state they are likely working as expected. Just because your query goes to the database does not mean the cache isn't being used. Caching entities is very different than caching the results to a query. If the query results are not cached, unless you have in-memory querying enabled, most read-all type queries must go to the database to determine what entities need to be built and returned. EclipseLink will then use those results to check the cache- if the entities already exist, they are returned as is - this avoids the overhead of rebuilding entities from the data.
You can check if your entity has been cached by using a em.find() or read query that uses the ID value. The cache is indexed by ID, so it will not need to go to the database to figure out which entities you want.

Entity Framework Function Import returns incorrect Column Information

I am trying to import a function spa_SessionLoad which should be returning a particular row from my Session management table. When I click on the 'Get Column Information' button, it instead pulls the info from a different stored procedure (spa_InventoryUpdate), which returns a single string.
I'm working with a sql 2008 database, using database-first methods.
I've built a few other function imports, and they all seem to be correct.
I've verified the code on the SQL Server returns a row when the sproc is executed.
The culprit seems to be that my 'Load' function contains some cleanup code to deal with abandoned sessions, which ultimately calls the spa_InventoryUpdate procedure. The import tool seems to be getting confused with the return of that EXEC call, instead of the one for the 'Load' .
At any rate, how can I get the function imported, if its pulling the wrong return info?
Thanks
It turns out that if you know the actual return type you can just specify it in the UI. In my case, I knew it was really returning a single "session" row, so I selected my "session" EF object, and everything worked as required.