We are using Datastore export feature for exporting datastore entities and importing them in Big Query table. It works perfectly fine.
The only problem is that the entities imported in big query tables do not have 'id' column but '__key__.id'.
We are using a hack(which is unnecessary overhead) for now get 'id' column generated by following query written to target table.
select key.id As id ,* from projectId.datasetId.ImportedTable
Actually, this transfer has be automated via cron jobs and there is no scope for such hack there. so wanted to check if there is any way that imported table by itself, expose ID column.
Can the way we generate entities and choose its Primary column strategy play any role here?
As per documentation, when importing data to BigQuery from Datastore, BigQuery creates a RECORD type for each entity unique key. As this is automated it can't be modified the way is it done.
Maybe a good possibility will be updating the entities to add an extra property and copy the ID so this way the column will be named ID instead of key.id once the import is done to BigQuery.
As this feature is not yet included in the export service between Datastore and BigQuery, you can create a Feature Request in Google Cloud Platform's public issue tracker.
Related
I have a table established in PowerApps for Teams. It has a lookup column referring to another table and functions. I have external data that I wish to populate this table with. When using the Dataflow created by the Import function offered by the UI, I cannot put data into this one column. The column is not available for import. All the other data does get pushed into the table as desired. Even one of the Choice columns receives its data.
How would I enable this lookup column to show up and receive data? The solutions I have found have been for the full PowerApps suite for which I don't have a license. They used alternative keys, but I cannot find a means to enable alternate keys in this stripped down version.
I have used data from an excel file both in OneDrive and external.
I have used the data from a SharePoint list to find the same result.
I have attempted to use and Alternate key in the receiving table, but it doesn't seem to be an option for the PowerApps for Teams.
I have formatted the data to be imported to match the lookup options for the receiving table.
My company is building a factory warehouse managing system with springboot and spring-data-ldap. The old system only supports warehouse with 3 tiers like this:
warehouse (contains) storagearea(s) (contains) storagelocation(s)
This kind of structure is way too strict for customers since their warehouses are uniquely different. We are thinking of building a system that users are able to config their own warehouse tiers without we hard coding the relations in java.
Now I am migrating mysql table schema of warehouse, storagearea and storagelocation, into LDAP. But it seems we have to define new objectClasses in ApacheDS since the attributes are not defined in OpenLdap.
Creating a new attribute type requires an OID, which is a unique number that has to be acquired by applying online from some website.
Is it possible to build a system like this?
Is there any way of obtaining OID or randomly create one?
That is possible. You need to aquire an OID from the IANA (Here is the link to the form: http://pen.iana.org/pen/PenApplication.page) and then you can create your own LDAP Schema entities.
And as long as the customer can create their tree with your schema entries there is nothing stopping you. And when you need to refactor your schema: do it.
You can find more on OIDs at https://ldapwiki.com/wiki/OID or o wikipedia.
I am trying to pull data from Oracle RDBMS and move it to OrientDB using teleporter. My relational database have multiple columns and have E-R relationships maintained. I have two questions :
My objective is to get only few columns ( that holds unique identity and foreign key relations ) and not all bulky column data. Is there any configuration using which I could do so. Today include and exclude only works at full DB table level.
Another objective is to keep my graph db sync with these selected table-column data which I pushed in previous run. Additional data which comes to RDBMS I would want in my graph db too.
You can enjoy this feature, and more others, in orientdb 3.0 through a JSON configuration, but there is not any documentation about it yet. Currently in 2.2.x you can just configure relationships and edges as described here:
http://orientdb.com/docs/2.2.x/Teleporter-Import-Configuration.html
In the next 2 weeks all these features will be available also in 2.2.x and well documented in order to make the comprehension of the config very easy.
At the moment you can adopt the following workaround:
import all the columns for each table in the correspondent vertex as usual.
drop the properties you are not interested in after each sync. You could write down a script where you call the teleporter execution and then delete the properties you don't care about from the schema.
I will update here when the alignment with 3.0 and the doc will be complete.
I have a database on SQL Azure which has an identity primary.
After using SQL Server Import and Export Wizard, I transferred the data to my SQL Server 2008 R2 database.
My ASP.NET Application runs fine and reads the data. But When I try to insert a value in a table 'User', it gives me an error:
Cannot insert null in column 'UserId'.
The reason being that it is not able to generate the identity value.
How can I overcome this issue?
PS: I tried Generating the scripts from SQL Azure, but the SQL file is 500MB in size and my host does not allow that big a script to run.
Edit: using Entity Framework for data access. The UserId field has an IDENTITY property (1,1).
Edit Tried to create the schema from SQLAzure Migration tool and then used the import/export data to copy the data.
But the wizard does not maintain the relations amongst the rows.
The data import/export wizard doesn't preserve the whole structure of your database objects.
i.e. it will only copy the data, not the whole structure of the table that the data fits into - including identity and key definitions.
You could import the data, and then manually set all the primary keys and default fields to match your desired database definition, or you could connect to your Azure instance and use the generate script option to generate your schema in the 2008 database prior to copying.
But the real answer is that you should be using the Copy Database Wizard to accomplish this, which works fine with Azure. It was designed for this scenario.
The issue was the wizard was trying to insert primary key values, which is disabled by default. And without inserting the primary keys, the relationships can't be maintained, thus the whole issue.
To resolve this issue and do a foolproof migration, ensure that the new schema maintains all the identity columns.
When selecting the source and destination tables, for the specific tables, click on "Edit Mappings" and Check the "Enable identity insert" check box to enable insertion of primary key values, which keep the structure and relations intact.
We have an application that creates new tables at runtime, but always with the same table schema. The only thing that varies from one of these tables to the next is the table name. Is it possible to access these tables using Entity Framework, specifying which table to access by name?
Entity Framework is not designed for DDL, it's an ORM tool for data access. You would want to use a simple ADO.NET query to create/drop the table.
Creating and dropping tables for every user session will make your log file grow very big very fast. I would consider carefully the reasons you think this is necessary. If the data is temporary, why not save the Session ID in each row and truncate the table on a daily basis?
UPDATE:
No, not really. The Entity Data Model is not dynamic, it's a static XML document that describes the structure of the database. If you want to interact with a table with a dynamic name, you're going to have to stick to "classic" ADO.NET.
With Linq to SQL I guess it would be possible with a stored procedure taking the table Name as a parameter.
A nice post about SP in L2SQL: http://weblogs.asp.net/scottgu/archive/2007/08/16/linq-to-sql-part-6-retrieving-data-using-stored-procedures.aspx
I don't know if that feature exists in EF.