As mentioned in the title, I want to transfer data, such as PDF files, from a document management system (DMS) to a directory on my server.
I'm unsure how I should approach the problem however.
I thought to use ETL Talend, but I don't think it offer components to deal with my problem.
Alternatively, I was wondering if FileZilla could help me with my problem.
Related
I have two PCs that want to share tensorflow models "hdf5 format" in a federated learning manner via a PostgresSQL database.
The models will be trained locally on both machines, and then transferred to the database along with the training history. The transfer will be done for multiple cycles in a specific schedule.
I searched online for solutions to transfer the files via PostgresSQL database, but all solutions suggest a tabulated data transfer, e.g. csv file data, not arbitrary file extensions, like hdf5.
Can anyone help me, even with a roadmap, for the solution?
If any tutorials or examples for similar scenarios would be suggested, that would be also appreciated.
Thanks for your help in advance!
I am trying to write a custom FoxPro XBase database driver for Unix via Perl. I am currently able to perform CRUD on my .dbf and .cdx indices. The point of sale software my store uses has a FoxPro based file system. The master plan behind the idea is to have a proxy between the POS and my Magento web store. Allowing real time updates between the brick and mortar store and the ecommerce solution.
My question is fairly simple and straight forward. Whats the best way to dump my .cdx file contents so i can quickly look at the indices on the .dbf file. I am currently using XBases indexdump, but this is a slow and tedious process. The dump can be platform independent and can be purchasable software if needed. I am trying to accomplish this picture from MSDN
This from MSDN might help - it's not really a documented or discussed thing though, as someone is doing what you're doing there is no need to ever delve into the CDX structure.
I've tried to google this for a couple of days and I am still pretty confused, so I thought I would try here.
I have an iPhone app that uses Core Data with an sqlite database. I am trying to implement a simple backup/restore of the database with Dropbox.
I downloaded the Dropbox SDK, and I have everything running fine as far as linking, uploading and downloading my .sqlite file.
However, I don't want users to have access to the actual .sqlite file for security purposes. I have been seeing JSON on these boards for some time now so I decided to look into it. I think it is exactly what I need.
Easier said than done. I have never worked with Java and have never implemented anything like JSON before, so I have had to try to figure out where to start.
I understand basically what is going on, but I'm having a heck of a time figuring out how to do it. I think I found a way to get the Core Data model into JSON format (and I do use the term 'think' loosely here). But then what - what exactly do I upload to dropbox? Do I somehow combine the model (in JSON format) and the database? What gets uploaded to Dropbox? I'm sorry if this seems obvious to most, it really is not obvious to me, and I have looked.
I am willing to do the work, but it just seems like I could go in 90 directions without some basic guidance and a start. I am not trying to do anything fancy as far as determining data that has been changed, etc. - just want to backup/restore the whole database. I just need some basic explanation and to be pointed in the right direction. A simple core-data sample project would be tremendous.
I'm not an experienced programmer, but I am a fast learner. Just break it down easy...
Thanks in advance.
JPK
Thanks Andrew. I didn't want to 'give away' the database structure of my app, but i can now see that the json string wouldn't be much better than the sqlite file in that area. I am a teach-myself programmer (stay at home mom) so this is all pretty new to me. Maybe I want encryption? But is that allowed for iPhone apps anyway? I recall them asking about encryption when I have uploaded binaries.... I know that iCloud is coming out soon, and I do plan to implement that as well, but with the limited amount of data that can be synced for free, I want to be able to do a simple backup as well. Many of my users have asked for it - a backup in addition to that of iTunes, which really is not a great backup since you can't restore data for just one app (you would have to restore for all apps on the iDevice). Hmmm... Any suggestions as to how to upload the file in such a way that it is not easily readable? Is encryption the way to go in this situation?
JPK,
I think you're crossing multiple streams here.
JSON is a data transfer format. IOW, it has almost nothing to do with the architecture of what you are attempting. You will almost certainly use JSON to communicate with Dropbox.
I have a question: why do you think that the user won't have access to any data you send to Dropbox? I suspect that you are probably wrong. The user will have access to everything.
You are doing to an awful lot of work. You already have the .sqlite file being persisted on Dropbox. If you are doing this to make a backup, that data is being backed up in many other venues. In other words, your task is likely moot and unworthy of your time.
I use JSON, REST networks and Core Data daily. If you have a specific question, I am happy to answer it.
Andrew
Instead of saving the whole database, save out a plist file to Dropbox that you can rebuild a database from. That's assuming you have a lot of extra stuff in your database you do not want the user to see, otherwise just back up the DB as-is if it's all user generated data.
You could also encrypt it but why? Adding encryption only means you have to answer "yes" that you use encryption and may not be able to sell to specific companies, should you choose to encrypt - it's not forbidden.
I'm refactoring an older online catalog system where the various messages sent to users - things like 'Thank you for registering with our store', 'Thank you for your order', 'Your order has shipped' - were all stored as text fields in the database.
It's worked well enough and it's nice to have the CMS subsystem be able to simply pull a db field into a browser-based editor for updates. But a browser-based editor is also a downside if I want a richer editor to handle more complex layout functions.
Should I give consideration to moving the messages to the filesystem and just store a URI in the db?
thx
It sounds like you're already considering it! :)
Well, doing that will add complexity won't it, because information will have to be looked up. You'll have to look in the database anyway to find where the file is, so what's the point. It'll probably go wonky over time, unless you're very careful, you'll end up with URI's in the database where the file is missing, or files without a URI in the database.
I'm assuming your richer editor works on files, and that's why you're considering switching to file based?
How about, when you get the record out of the db, just extract the data to temporary file then. Then edit it in the rich editor, then when you've finished and saved it, write it back to the db.
To be honest, I don't really know enough to answer this. What editor are you considering using?
I am working on the developement of a application that will perform online backup of the files and folder in the PC, automatically or manually. Currently, I was keeping only the latest version of the file at the server.Now, I have to implement the versioning so that only the changes can be transfered to the online server and user must be able to download any of the available version of the file at Backup Server.
I need to perform Deduplication for this. Guys, though I am able to perform it using the fixed block size but facing an overhead of transferring the file having CRC information with each version backup.
I have never worked on such technology , so lacks in experience. I am eager to know is there any feasible method to embedd this functionality in the application without much pain. Is any third party tool would help to perform same thing? Please let me know?
Note: I am using FTP protocol to transfer the data.
There's a program called dump that does something similar, but it operates on filesystem blocks rather than files. rsync also may be of interest.
You will need to keep track of a large number of blocks with multiple versions and how they fit into the various versions of the original files, so you will need some kind of database to track this information, and an efficient way to query it to determine which blocks in a given file need to be transferred. Also note that adding something to the beginning of a file will cause all your blocks to be "new" if you use a naive blocking and diff scheme.
To do this well will be very complex. I highly recommend you thoroughly research already-available solutions, and if you decide you need to write your own, consider the benefits of their designs carefully.