Copying images in from anti-scraper websites. Google Docs handles it easily - anyone know how? - wysiwyg

I've been playing around with making a draftjs plugin that lets the user paste in mixed text&image content from websites and have images auto-uploaded to the server. I've quickly come to the realization that it's not easy, simply because of how many different sites use different kinds of counter-measures for copy/pasting images. Standard image tags in page content are no problem - easily grab the src and handle the file upload from the url. However, many sites use all kinds of trickery to make this a pain. For example, some will only serve small thumbnails, requiring a GET request on the image with a hash key in order to retrieve a larger version. Others somehow seem to corrupt the image so that it's unreadable by the time it's been retrieved. Others still play with weird embed tags to mess with draftjs' image blocks.
But then I open up a Google Docs file, and find that when I copy any images into that from a website, there's never any troubles whatsoever. All the problematic websites that I'm finding myself having to write specific methods for retrieving from seem to be handled by Google Docs with ease.
Am I using completely the wrong approach by trying to retrieve images from a url? Does Google use a far superior approach (yes, I presume) - in which case, does anyone have any idea what that approach might be?

Related

How to upload files and attachments to the sobject record using REST API?

Salesforce has two different UIs and in accordance with it, it has the possibility to store attached files differently.
Two files were uploaded via the classic UI and they are marked as 'attachments'. Other files were uploaded through the new UI and they are marked as 'files'.
I want to upload all of these files using REST API. I cannot find the proper documentation. Can somebody help me with this?
That's not 100% true. In SF Classic UI you were able to upload Files too. It's "just" about knowing the right API name of the table and you'll find lots of examples online.
Attachment and Document objects have exactly same API names, you can view their definitions in SOAP API definition or in REST API explorer (there was something which you can still see in screenshot in here, seems to be down now, maybe they're moving it to another area in documentation...)
The Files (incl. "Chatter Files") are stored in ContentDocument and ContentVersion object. The name is unexpected because long time ago SF purchased another company's product and it was called "Salesforce Content". In beginning it was bit of mess, now it's better integrated into whole platform but still some things lurk like File folders can be called Libraries sometimes in documentation but actual API name is ContentWorkspace. The entity relationship diagram can help a bit: https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_erd_content.htm
ContentDocument is a header to which many places in SF link (imagine file wasting space on disk only once but being cross-linked from multiple records). It can have at least 1 version and if you need to update the document - you'd upload new version but all links in org wouldn't change, they'd still link to header.
So, how to use it?
REST API guide: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/dome_sobject_insert_update_blob.htm
or maybe Chatter API guide (you tagged it with chatter so chances are you already use it): https://developer.salesforce.com/docs/atlas.en-us.chatterapi.meta/chatterapi/connect_resources_files.htm
some of my answers here might help (shameless plug). They're about upload and reading data too and one is even about data loader... but you might experiment with exporting files first, get familiar with structure before you load?
https://stackoverflow.com/a/48668673/313628
https://stackoverflow.com/a/56268939/313628
https://stackoverflow.com/a/60284736/313628

Filepicker.io - image conversions preventing video uploads

We're currently working with Filepicker.io to allow users the ability to upload both images and videos. It appears that if we specify image conversions in the Javascript API options, video uploads don't process and instead get stuck at 99.30%. If I remove the 'conversions' option, video uploads process without issue. Is it not possible to specify image conversion options and accept both type of uploads? If so, this should really be specified in the docs.
I attached a JSFiddle with the code in question. http://jsfiddle.net/BYkD4/
It might be an issue on our end, taking a look now. For large files (+1Mb) we split the file into chunks, upload them in parallel, and then reassemble them on the server side. We use browser progress up to the 90% mark, after which we have to "best guess" what the server-side progress looks like, for now at least. That's the reason why it's hanging at 99.30% - it may actually be able to complete if you give it enough time.
In any case, looking into it
Edit: looks like this was an issue on our end. Fix deployed, everything should be working fine. Sorry about the issue

tinymce file browsers multiple file source

I am doing some updates to a site I have developed over the last few years. It has grown rather erratically (I tried to plan ahead, but with this site it has taken some odd turns).
Anyway, the site has a community blog ( blog.domain.com - used to be domainblog.com) ) and users with personal areas ( user1.domain.com, user2.domain.com, etc ).
The personal areas have standard page content that the user can use, or add snippets of text to partially customize. Now the owner wants the users to be able to create their own content.
Everything is done up to using a file browser.
I need a browser that will allow me to do the following:
the browser needs to be able to browse the common files at blog.domain.com/files and the user files at user_x.domain.com/files
the browser will also need to be able to differentiate between the two and generate the appropriate image url.
of course, the browser access to the user files will need to be dynamic and only show those files particular to the user (along with the common files)
I also need to be able to set a file size for images
the admin area is in a different directory than either the blog or the user subdomains.
general directory structure
--webdir--
|--client --
|--clientsite--
|--blog (blog.domain.com)
|--sites--
|--main site (domain.com)
|--admin (admin.domain.com)
|--users--
|--user1 (user1.domain.com)
|--user2 (user2.domain.com)
...etc.
I have tried several different browsers and using symlinks but the browsers don't seem to be able to follow them. I am also having trouble even setting them to use a directory that isn't the default.
what browser would you recommend? what would I need to customize to make it work.
TIA
ok, since I have not had any responses to this question, I guess I will have to do a work around and then see about writing a custom file browser down the road.

Examples and/or 3RD Party Libraries to Optimize Display of PDFs on iPad

I have read numerous posts here (and experienced first hand) some of the limitations of using UIWebView to display PDF files on iPad/iPhone.
Are there any 3rd party libraries that handle PDF page rendering/caching/optimization? Don't need to handle adding annotations or markups to the PDF; just want to minimize delays in opening the document and moving between pages in PDFs.
Thanks.
Instead of using UIWebView use CoreGraphics. The best library I have seen for implementing PDF's with this is the Leaves library.
You will need to get familiar with the CGPDF calls and if you are not experienced with CoreGraphics it can be a bit tricky at first but the framework I suggested makes things pretty simple to get started and do basic things like displaying a local document or document on a server.

Product idea/approach : Folder based disk organization

Sweet..I bought myself a 1TB portable harddrives this week. Don't you just love how much data you could store on one of these disks? The fact that I could store my bluray rips on to my portable harddisk and that my lg lcd tv can do HD rips right from the drive - that's amazing practicality right there! However, life it seems, is never so simple. I have 100s of movies unorganized in one huge folder, which is exactly what I needed to annoy myself while browsing the same on my tv to play a single movie. That got me thinking...
What if I had an automated way to organize movies into folders such that my folder-browsing-on-a-lcd-tv-or-a-comp would make my life a little easy?
I started thinking about this... I browsed a little in this context and I realized that if only I could "tag my movies somehow and create folders on-the-fly based on tags using hardlinks", I would have addressed my problem. I googled a bit to find software that works in the above fashion, only to find none.
A few more days of serious thought (as you know by now.. I think a lot.. and I guess this question is starting to sound like a blog rant/post of sorts...), in the interest of humanity, I thought I should come up with a generic way to address this: What if someone wanted to organize photos... organize music.. organize software?!
Turned my grey cells off for a while and here is an approach I came up with to solving my what-if scenario.
Tag / Group tag individual files (rely on a slick GUI to do it fast and do it good) - Adobe Flex/Eclipse RCP to do this?
Create hardlinks to each of the tagged files.
The first point is self-explanatory. The second (coz I am talking windows here), refers to making use of mklink.exe.
Consider a scenario where I have 2 movie files: I have a movie file "Transformers.avi" tagged as "english, action, bluray, sci-fi, imdb-top-50, must-watch-with-kids" and another movie file "The Specialist.avi" tagged as "english, bluray, thriller, adult". Here are a few of the possible locations I want to see my Transformers to be found:
[root directory]->all-tags->english
[root directory]->all-tags->bluray
[root directory]->all-tags->english->all-tags->bluray
[root directory]->all-tags->bluray->all-tags->action
[root direcotry]->all-tags->english->all-tags->action->bluray->all-tags->imdb-top-50
Given that windows has a limit of 1024 hardlinks to a single file, I probably would be allowed 7 unique tags per file. Each sub-folder will have an "all-tags" folder. Having it named "all-tags" makes it more accessible when order by name.
I believe this approach when automated to let you configure tags you want and where the hardlinks are created for you, helps you organize stuff effectively.
I don't know if there are better things out there. I would like your inputs on this approach and other possible ideas. I would like to gather inputs here and release something to sourceforge for everyone to use in a couple of weeks. I am sure, I can count on your positive response as always.
I believe hardlinks are not a good approach. Reason? A standalone player won't play them, and I wouldn't like a program who's made for tagging to tell me to stop making so many tags because of a Windows limitation on hardlinks (remembering each tag will increment the number of links exponentially).
Plus, "help" is not a good tag.
And I've had an idea once that I'm still planning to make some day to sort my own files - put the files in a big storage each below a GUID foldername (filename untouched) and store metadata in a sqlite database to be used by a smart file browser.
I was considering doing something similar to this with music for detecting duplicate songs and auto-organize funcationality.
For your application, I wouldn't recommend using any shell programs through Java. Exception handling becomes difficult, and your application becomes bound by the shell interface and implementation (i.e. windows versions or installations affect your application behavior).
I would use a database with a few tables: Files, Tags, and an association table.
The Files table would list the physical location of each file, the filename, and a unique identifier. This way, you can maintain information about each file without having to modify it for every tag association.
The Tags table would list each tag, and any metadata you want to store for each tag.
A third table, maybe 'FileTags' would store the assocation between tags and files. When adding tags to the stack, you would add a statement to the WHERE clause, and the list of files with all of the tags would be returned. This structure would also allow open your codebase up to other designs, such as include/exclude (autocomplete with X buttons), or possibly search.
If implemented in Java, your app would be platform independent, and would allow a very large number of tags and files. You can then use the system default application for opening the media file, and the user can make the selection in their native OS.
Reiser4?
...
(I mean nevermind Hans, but the tech...)
[disclaimer: Not a hacker. I know nothing of programming/coding, never mind filesystems & databases. I can barely code decent HTML even, if at all. Hey y'all! :D]
[footnote: does plain HTML5 work here? Too lazy to close my tags hehe :p]