I am using Oracle express for the database comparisons (Teradata and Adaptive Server IQ)
When i am trying to connect a database and try to display all the columns using select function, all the columns except date columns are getting displayed.
can i know a solution for this?
Related
When I aggregate values in Google Data Studio with a date dimension on a PostgreSQL Connector, I see buggy behaviour. The symptom is that performing COUNT(DISTINCT) returns the same value as COUNT():
My theory is that it has something to do with the aggregation on the data occurring after the count has already happened. If I attempt the exact same aggregation on the same data in an exported CSV instead of directly from a PostgreSQL Connector Data Source, the issue does not reproduce:
My PostgreSQL Connector is connecting to Amazon Redshift (jdbc:postgresql://*******.eu-west-1.redshift.amazonaws.com) with the following custom query:
SELECT
userid,
submissionid,
date
FROM mytable
Workaround
If I stop using the default date field for the Date Dimension and aggregate my own dates directly in within the SQL query (date_byweek), the COUNT(DISTINCT) aggregation works as expected:
SELECT
userid,
submissionid,
to_char(date,'YYYY-IW') as date_byweek
FROM mytable
While this workaround solves my immediate problem, it sucks because I miss out on all the date functionality provided by Data Studio (Hierarchy Drill Down, Date Range filtering, etc.). Not to mention reducing my confidence at what else may be "buggy" within the product 😞
How to Reproduce
If you'd like to re-create the issue, using the following data as a PostgreSQL Data Source should suffice:
> SELECT * FROM mytable
userid submissionid
-------- -------------
1 1
2 2
1 3
1 4
3 5
> COUNT(DISTINCT userid) -- ERROR: Returns 5 when data source is PostgreSQL
> COUNT(DISTINCT userid) -- EXPECTED: Returns 3 when data source is CSV (exported from same PostgreSQL query above)
I'm happy to report that as of Sep 17 2020, there's a workaround.
DataStudio added the DATETIME_TRUNC function (see here https://support.google.com/datastudio/answer/9729685?), that allows you to add a custom field that truncs the original date to whatever granularity you want, without causing the distinct bug.
Attempting to set the display granularity in the report still causes the bug (i.e., you'll still set Oct 1 2020 12:00:00 instead of Oct 2020).
This can be solved by creating a SECOND custom field, which just returns the first, and then you can add IT to the report, change the display granularity, and everything will work OK.
I have the same issue with MySQL Connector. But my problem is solved, when I change date field format in DB from DATETIME (YYYY-MM-DD HH:MM:SS) to INT (Unixtimestamp). After connection this table to the Googe Datastudio I set type for this field as Date (YYYYMMDD) and all works, as expected. Hope, this may help you :)
In this Google forum there is a curious solution by Damien Choizit that involves combining your data source with itself. It works well for me.
https://support.google.com/datastudio/thread/13600719?hl=en&msgid=39060607
It says:
I figured out a solution in my case: I used a Blend Data joining twice the same data source with corresponding join key(s), then I specified a data range dimension only on the left side and selected the columns I wanted to CTD aggregate as "dimensions" (and not metric!) on the right side.
Halo,
im importing tables from MS-Access into R via
channel <- odbDriverConnect(...)
foo_import <-sqlFetch(channel,"foo")
now Im using the RSQLite for SQL Querys and other stuff
dbWriteTable(conn, "foo_table", foo_import)
R_Table <- dbReadTable(conn, "foo_table")
the Problem is, that at the sqlFetch step columns including data are still correct, tested with class(foo_import$Datetime) which gives POSIXct POSIXt
After using dbWriteTable and testing the Format with class(R_Table$Datetime)
which gives numeric
The Dates are just numbers
with as.POSIXct(R_Table$Datetime, origin= "1970-01-01")
I can transform the column manually. Are they other options to Change the format automatically to a date Format ?
We are using Microsoft Reporting for generating a daily report. I want to add another column to one of the tables we have. Initially, I had set this up correctly and the report worked fine. However, due to technicalities I have to use a different table (with exact the same data) so I edited the query and once I do that I star getting "#Error" in the cell calues of my column.
The cell expression:
=Lookup(Fields!fldFlight.Value, Fields!OutboundFlightNumber.Value, Fields!OnTime.Value, "DataSet")
I use the following query to form DataSet:
SELECT
turnarounds_staging.OutboundFlightNumber
,turnarounds_staging.VisitDatabaseID AS [turnarounds_staging VisitDatabaseID]
,turnarounds_staging.STDDate
,events_staging.VisitDatabaseID AS [events_staging VisitDatabaseID]
,events_staging.OnTime
,events_staging.Event
FROM
turnarounds_staging
LEFT OUTER JOIN events_staging
ON turnarounds_staging.VisitDatabaseID = events_staging.VisitDatabaseID
WHERE
events_staging.Event ='PDC'AND
turnarounds_staging.STDDate= #Date
Where #Date is a parameter indicating yesterday.
If I change the query to the original table (identical). It works fine.
Any ideas why this happens when turnarounds_staging is identical to the original table?
I'm making a database for a website and it needs to take the users input of a date but I think I can do that bit. The part I'm struggling to do is create a table with the date item. What do I need to tell it to do when creating the table I've gotcreate table Results (GameID integer,Date ???????, primary key(GameID)
I just need to to know what I set date to as its not an integer real or text.
I'm using python 3.2
SQLite does not have a built-in date type.
SQLite can store dates as a string or a number.
I have a FoxPro data source, and the destination is SQL Server 2008.
On the FoxPro side, I have a column with the Date data type. That's a width of 8, min value is 0001-01-01 and max is 9999-12-31. On the SQL Server side, I have a datetime. Also a width of 8, min value is 1753-01-01 and max is 9999-12-31.
In my SSIS Data Flow task, I have an OLE DB Source component that reads in the FoxPro table. The columns are mapped as DT_DBDate in both External and Output Columns. The OLE DB Destination to the SQL Server table takes that columns and flows it to a DT_DBTIMESTAMP. I'm sure that DT_DBTIMESTAMP can handle these date ranges adequately, and reasonably sure about DT_DBDATE (but am having trouble finding it in MS documentation).
The Problem
When I execute the task, I've been having trouble with it failing and complaining of "Invalid date format" and then "Conversion failed because the data value overflowed the specific type" when it encounters values from around the early 1900s, to values over 2050. I'm not exactly sure where I've gone wrong.
Bonus Question
How can I handle overflows in my data flow task?
you might want to do a simple VFP query looking for the records that are beyond the range such as a VFP query of
select * from yourtable
where YourDate < date( 1753, 1, 1 )
or YourDate > date( 9999, 12, 31 )
look at those records for problems...
Additionally, look at other columns that may be causing your boundary issues
I think you need to decide what do you want to do with your data
If you want to load it as it is, you need to change field data type so it will be able to hold the data.
Or if you any wish to correct wrong data you would need to validate and transform it.
If you willing to spend some money please consider using Advanced ETL Processor.
It works with Text, XML, Excel, Access, DBF, Foxpro, ODBC, OLE DB, MS Sql Server, Oracle, MySql, PostgreSQL, Firebird, Interbase, SQLite, POP3, SMTP, File System, FTP, SSL and Unicode.