I have a view using a CTE that exceeds the maximum recursion, so I need to select from it using the hint
OPTION (MAXRECURSION 3650)
Is that possible? I do not seem to be able to find any information on it, other than the fact that is not working in the Source Query - any documentation on what you can do as far as SQL queries would be greatly appreciated.
Error message:
at Source 'Calendar': shaded.msdataflow.com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near the keyword 'option'.
Source Query:
select * from dbo.ReportingCalendar option (maxrecursion 3650)
The above query is correct and runs on the SQL Server.
I refer to this documentation, but there isn't information about keyword 'option'. I also test it with data flow, got the same error with you. So it seems not support this keyword.
As an alternative, you can use copy activity, it supports 'option'. You can copy data from your SQL database to Azure data lake gen2( or someplace else that data flow support as source), then use that as source in data flow and do some transform.
Related
I've been following the tutorial here:
Tutorial
I can get the odata uri parsed together just fine and get a json repsonse from azure devops that looks exactly like I expect. However when I take that same uri and use it as the odata source in Power Bi, I get the error:
Details: "OData: The property 'PartiallySuccessfulRate' does not exist on type 'Microsoft.VisualStudio.Services.Analytics.Model.PipelineRun'. Make sure to only use property names that are defined by the type or mark the type as open type."
If I remove them, the query works fine in powerbi.
Is there a way to make powerbi accept the computed columns? Or do I have to do the calculation in powerbi?
I would rather to these small calculations in power bi. I use most of the time Odata query for Dynamics as well. My main purpose of Oata query is to fetch only required data and not like millions of records.
Once this purpose is solved, I let powerbi do some calculations for me.
In this way it is easier for my Team to collaborate as well so that they can update/change as easily as they can.
I'm using Copy Data.
When there is some data error. I would export them to a blob.
But in this case, the Pipelines's status is still Succeeded. I want to set it to false. Is it possible?
When there is some data error.
It depends on what error you mentioned here.
1.If you mean it's common incompatibility or mismatch error, ADF supports built-in feature named Fault tolerance in Copy Activity which supports below 3 scenarios:
Incompatibility between the source data type and the sink native
type.
Mismatch in the number of columns between the source and the sink.
Primary key violation when writing to SQL Server/Azure SQL
Database/Azure Cosmos DB.
If you configure to log the incompatible rows, you can find the log file at this path: https://[your-blob-account].blob.core.windows.net/[path-if-configured]/[copy-activity-run-id]/[auto-generated-GUID].csv.
If you want to abort the job as soon as any error occurs,you could set as below:
Please see this case: Fault tolerance and log the incompatible rows in Azure Blob storage
2.If you are talking about your own logic for the data error,may some business logic. I'm afraid that ADF can't detect that for you, though it's also a common requirement I think. However,you could follow this case (How to control data failures in Azure Data Factory Pipelines?) to do a workaround. The main idea is using custom activity to divert the bad rows before the execution of copy activity. In custom activity, you could upload the bad rows into Azure Blob Storage with .net SDK as you want.
Update:
Since you want to log all incompatible rows and enforce the job failed at the same time, I'm afraid that it can not be implemented in the copy activity directly.
However, I came up with an idea that you could use If Condition activity after Copy Activity to judge if the output contains rowsSkipped. If so, output False,then you will know there are some skip data so that you could check them in the blob storage.
Domo calls one of the DataFlow types a "MySQL DataFlow". MySQL has a function called JSON_EXTRACT. How can I use this on a field that contains some JSON encoded data? I receive back the error:
The database reported a syntax error. FUNCTION json_extract does not exist
I know this is a somewhat old question, but I think the answer is that Domo uses an older version of MySQL (5.6) that does not support the JSON_EXTRACT function (5.7 and later).
You can check the version by running this statement in a MySQL DataFlow transformation step:
SELECT VERSION();
When I run that on the Domo instance I use at work I see 5.6.28-76.1-56
If you go to the JSON Functions page of the MySQL documentation and try to select any version before 5.7 from the dropdown in the upper right of the page you will see a message saying "The page you are viewing does not exist in version X.X"
We have an Firebird database for a (very crappy) application, and the app's front end, but nothing in between (i.e. no source code).
There is a field in the database that is stored as -2086008209 but in the front-end represents as 63997.
Examples:
Database Front-End
758038959 44093
1532056691 61409
28401112 65866
-712038758 40712
936488434 43872
-688079579 48567
1796491935 39437
1178382500 30006
1419373703 66069
1996421588 48454
890825339 46313
-820234748 45206
What kind of storage is this? The aim for us here is to access the application's back-end data and bypass the front-end GUI alltogether, so I need to know how to decode this field in order to get appropriate values from it. It is stored as a int in FireBird (I don't know if FireBird has signed/unsigned ints, but this is showing as signed when we select it).
This is the definition of the field:
It is not, as far as I can tell, de-normalised. The generator GEN_CONTACTS_ID has 66241 against it, which at a glance looks accurate.
I work on with an application that stores bitmaps in integers (just don't ask), if you express them in that form do you something useful or consistant
My impression is that the problem is in the front end. If what is stored in the DB is -2086008209, then what is stored in the DB is -2086008209. To understand better how the application is manipulating the data, try storing other numbers in the DB and see how they are displayed.
Did you come to this realization through logging SQL? If you havent, you may serve yourself well by using the Firebird Trace API to get that SQL: http://www.firebirdfaq.org/faq95/. An easier tool to parse the Trace API is this commercial product: http://www.upscene.com/products.fbtm.index.php.
I've used these tools and other techniques (triggers etc,.) to find what an application is using/changing in the Database.
Of course, if the SQL statement is select * from table, then these tools would not help much.
Whenever I try to apply filter to an attribute, which has ValueSelection= Dropdown, the dropdown is not populated and error message "The requested list could not be retrieved because the query is not valid or a connection could not be made to the data source" is shown instead.
If I set up ValueSelection=List I am getting a different error message:
An attempt has been made to use a semantic query extension associated with the data extension 'SQL' that is not registered for this report server.
(Microsoft.ReportingServices.SemanticQueryEngine)
This happens within BIDS environment and was observed both in SQL 2005 and SQL 2008.
I've already studied articles, which discussed the similiar problem, but neither of them applied to my case. The user account in data source has all necessary rights, data could be retrieved without any problem (for example if i try "Explore data" in data source view). The SQL profiler shows that no query is being sent to SQL Server when there is an attempt to populate dropdown. So nothing is wrong with the query, it is simply never executed.
Your connection is not working. Try to test you connection by trying a simple table and query output.
This will enable you to test the connection before trying anything advanced.
Got this problem and in my case it was caused by wrong connection string in Data Source - instead of just having a SQL Server name like "SOMESQLSERVER_MACHINE" I had for some reason "SOMESQLSERVER_MACHINE.our.corp.domain". It had to be the same, but then I realized that the domain is wrong, after removing it all works like a charm again. That said: it's always good idea to start with detailed checks on your basic settings.
Otherwise this could be a problem with permissions to the folders on Report Manager.