is there a possibility to in bulk import documents and their metadata in alfresco. In fact what I want is upload a bunch of documents and inject their metadata from a xml file.
thanks in advance
The link that Abbas pointed you to is the best option. The Bulk File System Import Tool supports bulk importing content as well as metadata.
Write a script that exports your spreadsheet into the format the BFSIT expects. Then upload your content and each of the content's metadata descriptor files (generated from your spreadsheet) to the server. Finally, run the import.
If instead what you are trying to do is not import files and metadata but instead you just want to set metadata from your spreadsheet on a bunch of existing content that is already in the repository, then what you can do is write a script that reads your spreadsheet and uses something like Python cmislib or OpenCMIS (both are from Apache Chemistry) to set that metadata on objects in the repository in bulk.
You can also use CMIS to upload files, but the BFSIT is much more efficient.
Related
I have an AS400 with an IBM DB2 database and I need to create a Format Description File (FDF) for each table in the DB. I can create the FDF file using the IBM Export tool but it will only create one file at a time which will take several days to complete. I have not found a way to create the files systematically using a tool or query. Is this possible or should this be done using scripting?
First of all, to correct a misunderstanding...
A Format Description File has nothing at all to do with the format of a Db2 table. It actually describes the format of the data in a stream file that you are uploading into the Db2 table. Sure you can turn on an option during the download from Db2 to create the FDF file, but it's still actually describing the data in the stream file you've just downloaded the data into. You can use the resulting FDF file to upload a modified version of the downloaded data or as the starting point for creating an FDF file that matches the actual data you want to upload.
Which explain why there's no built-in way to create an appropriate FDF file for every table on the system.
I question why you think you actually to generate an FDF file for every table.
As I recall, the format of the FDF (or it's newer variant FDFX) is pretty simple; it shouldn't be all that difficult to generate if you really wanted to. But I don't have one handy at the moment, and my Google-FU has failed me.
I am new to firebase and bigquery (and all gcp services) and am trying to use the extension inside of firebase that connects my firestore db to BigQuery. I seem to have set it up correctly, but it only shows data from the time I installed the extension going forward (no older data). I would like to see all old data and any new data in my firestore DB. How can this be done?
From the documentation of the Stream Collections to BigQuery extension:
If you create, update, delete, or import a document in the specified collection, this extension sends that update to BigQuery. You can then run queries on this mirrored dataset.
So (as Doug commented too) it does not export the existing contents of the collection, but rather only streams the changes from the moment you install the extension.
There's a section on backfilling your BigQuery dataset that says:
This extension only sends the content of documents that have been changed -- it does not export your full dataset of existing documents into BigQuery. So, to backfill your BigQuery dataset with all the documents in your collection, you can run the import script provided by this extension.
Important: Run the import script over the entire collection after installing this extension, otherwise all writes to your database during the import might be lost.
I would like to use data factory to regularly download 500000 json files from a web API and store them in a blob storage container. Then I need to parse the json files to extract some values from each file and store these values together with an ID (part of filename) in a database. I can do this using a ForEach activity and run a custom activity for each file, but this is very slow, so I would prefer some batch activity which could run the same parsing code on each file. Is there some way to do this?
If your source json files have same schema, you can leverage the Copy Activity which can parse those files in a single run. But if possible, I would suggest to split those files into different sub folder (e.g. 1000 files per folder), so that each copy run needs less time and ease the management.
Refer to this doc for more details: https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview
What is the best way to import data from few csv files in Spring Batch? I mean one csv file responds to one table in database.
I created one batch configuration class for each table and every table has its own job and step.
Is there any solution to do this in more elegant way?
There's a variety of ways you could tackle the problem, but the simplest job would look something like:
FlatFileItemWriter reader with a DelmitedLineTokenizer and BeanWrapperFieldSetMapper to read the file
Processor if you need to do any additional validation/filtering/transformation
JDBCBatchItemWriter to insert/update the target table
Here's an example that includes more information around specific dependencies, config, etc. The example uses context file config rather than annotation-based, but it should be sufficient to show you the way.
A more complex solution might be a single job with a partitioned step that scans the input folder for files and, leveraging reference table/schema information, creates a reader/writer step for each file that it finds.
You also may want to consider what to do with the files once you're done... Delete them? Compress them?
I would like to import an Excel .xls file workbook into Powerbuilder. The file has 2 sheets and these sheets must be imported into 2 differenct db tables.
Any assistance is kindly appreciated.
Thanks
John.
First thing, there's nothing automagic, along the lines of a one-line solution that you could get for other file formats. There's a manual method, there's a scripting approach, and you can probably merge the two as a third option.
For a manual method, you can go into Excel and export your data as something that will import into a DataWindow. You don't mention your PowerBuilder version, but the file format for importing from Excel that comes to mind is CSV, which was added in PB9.
For a scripting approach, you can use OLE (assuming Excel is installed on the client machine) and access data however you want with the scripting engine, moving it into PowerBuilder in whatever format you want.
To mix the methods, you could use OLE to export the file to a couple of CSVs, then dw.FileImport() the data in.
Good luck,
Terry.
Postscript: Sybase has examples of OLE access, and examples of using ODBC, a solution I had neglected before.
If you give names to the areas with the data in Excel and then setup ODBC connections that point to them, you can access them like a database table from within PowerBuilder.