read a websites html code, then select an element and then make an action on that element - flutter

I want to know how Read a websites html code. Then i want to take a specific Element from the html code and make a request to make an action to the Element(e.g. Button,Textfield)
For example i take a websites URL, read the html code,take the element a button and then press the button from my Flutter App.

Web Crawlers/Scrapers only works on mobile clients (or website with CORS enable).
You need to add the following libraries http and html inside your pubspec.yaml:
html: ^0.14.0+3
http: ^0.12.0+4
// latest versions by Feb 18th 2020
to use it, create an async method to fetch the URL you want to "read":
Future<Response> _initiate(String url) async {
Response response = await get(url);
return response;
}
and then, use it this way, reading the specific div, a, img, etc you need:
_initiate(url).then((response) {
var document = parse(response.body);
var items = document.querySelector('div.groupHomeHeader-banner');
var split = items.attributes.values.toList()[1].split('(')[1];
var image = split.substring(0, split.indexOf(')'));
var name = document.querySelector('a.groupHomeHeader-groupNameLink');
});
Every Element has different attributes. For example, an InputElement has the attribute 'click()' and creates the desire effect.
Example of this will be:
var openDoc = document.querySelector('div.pdfInput') as InputElement;
openDoc.click();

Related

How to use the nextHandler functionality as shown in the Infinite Ajax Scroll JSON example

I’m hoping to be able to use Infinite Ajax Scroll for a project I’m working on.
I’ve been looking at the Infinite Scroll JSON example here (https://infiniteajaxscroll.com/examples/json/) and I’m finding it difficult to understand how it works. I was wondering if there is any further documentation or code examples on how to use a JS or jQuery handler as shown in the example.
Ultimately what I want to do is load my container "items" using my own ajax function and then have Infinite Ajax Scroll display them. I want to do this because my container "items" are not located at URLs, but are saved as Wordpress transients.
Any help I could get with this would be very much appreciated.
Thanks,
David.
Thank you for your question. The docs on using the nextHandler could indeed use improvement. Regardless, I'll try to explain how it works.
Normally IAS uses a selector to find the url of the next page. Then it loads the page and extracts the elements and appends them to the DOM. If you use the nextHandler, you will completely bypass this behavior. That means you will have to fetch data (in this case JSON) yourself and also insert new elements in the DOM.
Here is an example with comments to explain what it does.
First, let's assume our movie(1..n).json has the following format:
[
{
Title: 'item1',
Plot: 'description1'
}, {
Title: 'item2',
Plot: 'description2'
}
]
Now the implementation of the nextHandler:
import InfiniteAjaxScroll from "#webcreate/infinite-ajax-scroll";
function nextHandler(pageIndex) {
// we use the fetch api to load the next page. Any http client library will do, like axios.
return fetch("./static/movies"+pageIndex+".json")
// use the response as json
.then((response) => response.json())
// process the actual json data
.then((jsonData) => {
// create an array to store our html elements in memory
let elements = [];
// we loop over the items in our json and create an html element for each item
jsonData.forEach(function (item) {
const template = `<div class="item">
<h1>${item.Title}</h1>
<p>${item.Plot}</p>
</div>`;
const element = document.createElement("div");
element.innerHTML = template.trim();
elements.push(element.firstChild);
});
// now use IAS's append method to insert the elements
// it's import that we return the append result, as it's an promise
return this.append(elements);
})
// page 3 returns a 404, returning false here indicates there are no more pages to load
.catch(() => false);
}
window.ias = new InfiniteAjaxScroll(".container", {
item: ".item",
next: nextHandler,
pagination: false
});
I also prepared an interactive demo on Codesandbox:
https://codesandbox.io/s/serene-einstein-f73em

Express [413 too large] with QuillJS image

I am trying to use QuillJS to let the user write a rich text, and then store it as JSON to display later on. There are 2 of these rich text areas in a single form, and may include images. QuillJS encodes images as base64 strings, and my POST request results in 413 by Express.
I have tried to change the limits by adding express json parameters, even trying extreme numbers.
// app.js
//----------------------------------------------------
// Middlewares
//----------------------------------------------------
app.use(express.json({limit: '2000mb'}));
app.use(express.urlencoded({extended: true, limit:'2000mb'}));
Even this did not help and I think it is not logical to let these parameters with such values.
I tried with json and urlencoded enctypes. When I tried to post with multipart/form, req.body was empty.
// My html page (pugJS)
form(enctype='application/x-www-form-urlencoded', action='/editor/page',
method='POST', onsubmit='return addContent()')
.form-control
label Content-1
div#toolbar
div#editor
input#content(name='content', type='text', hidden)
addContent() function that runs before form submit simply changes input#content's value with JSON.stringify(#editor.getContents())
I want to be able to store two quill content in a single database row, to display later.
A better approach to this would be to overwrite the image upload function and then save the image in Amazon S3 or some cloud server. Then you paste it inside the editor as <img src="http://uploaded-image-url"> This would solve your problem of maximum memory issue.
I fixed my problem few hours before #argo mentioned and I did it that way. So I wanted to post little bit of detail to the solution. I have been also guided by a github issue but can't seem to find the link again, in case I find it I will edit the post and add it.
// Quill - EN content
var quillEn = new Quill('#editor-en', {
modules: {
toolbar: toolbarOptions
},
theme: 'snow'
});
// set custom image handler
quillEn.getModule('toolbar').addHandler('image', () => {
selectLocalImage(quillEn);
});
// create fake input to upload image to quill
function selectLocalImage(editor) {
const input = document.createElement('input');
input.setAttribute('type', 'file');
input.setAttribute('accept', 'image/png, image/jpeg')
input.click();
// Listen upload local image and save to server
input.onchange = () => {
const file = input.files[0];
saveImageToServer(editor, file);
};
}
// upload image to server
function saveImageToServer(editor, file) {
const fd = new FormData();
fd.append('image', file);
const xhr = new XMLHttpRequest();
xhr.open('POST', '/api/page/upload_image', true);
xhr.onload = () => {
if (xhr.status === 200) {
// this is callback data: url
const url = JSON.parse(xhr.responseText).data;
insertToEditor(editor, url);
}
};
xhr.send(fd);
}
// manipulate quill to replace b64 image with uploaded image
function insertToEditor(editor, url) {
// push image url to rich editor.
const range = editor.getSelection();
editor.insertEmbed(range.index, 'image', url.toString());
}
In the backend where you POST image, you must return json as { data: FullUrlToImg } with 200 response, if you want to change your status to 201 or something else, don't forget to update it in saveImageToServer function.
So to summarize, you set custom image handler for your quill editor, you post the image to server as soon as user chooses to insert, then you replace the URL with your uploaded image in the editor.
Thanks.

How to download mongo collections as file using iron-router (and ground-db)? [duplicate]

I'm playing with the idea of making a completely JavaScript-based zip/unzip utility that anyone can access from a browser. They can just drag their zip directly into the browser and it'll let them download all the files within. They can also create new zip files by dragging individual files in.
I know it'd be better to do it serverside, but this project is just for a bit of fun.
Dragging files into the browser should be easy enough if I take advantage of the various methods available. (Gmail style)
Encoding/decoding should hopefully be fine. I've seen some as3 zip libraries so I'm sure I should be fine with that.
My issue is downloading the files at the end.
window.location = 'data:jpg/image;base64,/9j/4AAQSkZJR....'
this works fine in Firefox but not in Chrome.
I can embed the files as images just fine in chrome using <img src="data:jpg/image;ba.." />, but the files won't necessarily be images. They could be any format.
Can anyone think of another solution or some kind of workaround?
If you also want to give a suggested name to the file (instead of the default 'download') you can use the following in Chrome, Firefox and some IE versions:
function downloadURI(uri, name) {
var link = document.createElement("a");
link.download = name;
link.href = uri;
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
delete link;
}
And the following example shows it's use:
downloadURI("data:text/html,HelloWorld!", "helloWorld.txt");
function download(dataurl, filename) {
const link = document.createElement("a");
link.href = dataurl;
link.download = filename;
link.click();
}
download("data:text/html,HelloWorld!", "helloWorld.txt");
or:
function download(url, filename) {
fetch(url)
.then(response => response.blob())
.then(blob => {
const link = document.createElement("a");
link.href = URL.createObjectURL(blob);
link.download = filename;
link.click();
})
.catch(console.error);
}
download("https://get.geojs.io/v1/ip/geo.json","geoip.json")
download("data:text/html,HelloWorld!", "helloWorld.txt");
Ideas:
Try a <a href="data:...." target="_blank"> (Untested)
Use downloadify instead of data URLs (would work for IE as well)
Want to share my experience and help someone stuck on the downloads not working in Firefox and updated answer to 2014.
The below snippet will work in both firefox and chrome and it will accept a filename:
// Construct the <a> element
var link = document.createElement("a");
link.download = thefilename;
// Construct the uri
var uri = 'data:text/csv;charset=utf-8;base64,' + someb64data
link.href = uri;
document.body.appendChild(link);
link.click();
// Cleanup the DOM
document.body.removeChild(link);
Here is a pure JavaScript solution I tested working in Firefox and Chrome but not in Internet Explorer:
function downloadDataUrlFromJavascript(filename, dataUrl) {
// Construct the 'a' element
var link = document.createElement("a");
link.download = filename;
link.target = "_blank";
// Construct the URI
link.href = dataUrl;
document.body.appendChild(link);
link.click();
// Cleanup the DOM
document.body.removeChild(link);
delete link;
}
Cross-browser solutions found up until now:
downloadify -> Requires Flash
databounce -> Tested in IE 10 and 11, and doesn't work for me. Requires a servlet and some customization. (Incorrectly detects navigator. I had to set IE in compatibility mode to test, default charset in servlet, JavaScript options object with correct servlet path for absolute paths...) For non-IE browsers, it opens the file in the same window.
download.js -> http://danml.com/download.html Another library similar but not tested. Claims to be pure JavaScript, not requiring servlet nor Flash, but doesn't work on IE <= 9.
There are several solutions but they depend on HTML5 and haven't been implemented completely in some browsers yet. Examples below were tested in Chrome and Firefox (partly works).
Canvas example with save to file support. Just set your document.location.href to the data URI.
Anchor download example. It uses <a href="your-data-uri" download="filename.txt"> to specify file name.
Combining answers from #owencm and #Chazt3n, this function will allow download of text from IE11, Firefox, and Chrome. (Sorry, I don't have access to Safari or Opera, but please add a comment if you try and it works.)
initiate_user_download = function(file_name, mime_type, text) {
// Anything but IE works here
if (undefined === window.navigator.msSaveOrOpenBlob) {
var e = document.createElement('a');
var href = 'data:' + mime_type + ';charset=utf-8,' + encodeURIComponent(text);
e.setAttribute('href', href);
e.setAttribute('download', file_name);
document.body.appendChild(e);
e.click();
document.body.removeChild(e);
}
// IE-specific code
else {
var charCodeArr = new Array(text.length);
for (var i = 0; i < text.length; ++i) {
var charCode = text.charCodeAt(i);
charCodeArr[i] = charCode;
}
var blob = new Blob([new Uint8Array(charCodeArr)], {type: mime_type});
window.navigator.msSaveOrOpenBlob(blob, file_name);
}
}
// Example:
initiate_user_download('data.csv', 'text/csv', 'Sample,Data,Here\n1,2,3\n');
This can be solved 100% entirely with HTML alone. Just set the href attribute to "data:(mimetypeheader),(url)". For instance...
<a
href="data:video/mp4,http://www.example.com/video.mp4"
target="_blank"
download="video.mp4"
>Download Video</a>
Working example: JSFiddle Demo.
Because we use a Data URL, we are allowed to set the mimetype which indicates the type of data to download. Documentation:
Data URLs are composed of four parts: a prefix (data:), a MIME type indicating the type of data, an optional base64 token if non-textual, and the data itself. (Source: MDN Web Docs: Data URLs.)
Components:
<a ...> : The link tag.
href="data:video/mp4,http://www.example.com/video.mp4" : Here we are setting the link to the a data: with a header preconfigured to video/mp4. This is followed by the header mimetype. I.E., for a .txt file, it would would be text/plain. And then a comma separates it from the link we want to download.
target="_blank" : This indicates a new tab should be opened, it's not essential, but it helps guide the browser to the desired behavior.
download: This is the name of the file you're downloading.
If you only need to actually have a download action, like if you bind it to some button that will generate the URL on the fly when clicked (in Vue or React for example), you can do something as easy as this:
const link = document.createElement('a')
link.href = url
link.click()
In my case, the file is already properly named but you can set it thanks to filename if needed.
For anyone having issues in IE:
dataURItoBlob = function(dataURI) {
var binary = atob(dataURI.split(',')[1]);
var array = [];
for(var i = 0; i < binary.length; i++) {
array.push(binary.charCodeAt(i));
}
return new Blob([new Uint8Array(array)], {type: 'image/png'});
}
var blob = dataURItoBlob(uri);
window.navigator.msSaveOrOpenBlob(blob, "my-image.png");
This code was originally provided by #Yetti on this answer (separate question).
Your problem essentially boils down to "not all browsers will support this".
You could try a workaround and serve the unzipped files from a Flash object, but then you'd lose the JS-only purity (anyway, I'm not sure whether you currently can "drag files into browser" without some sort of Flash workaround - is that a HTML5 feature maybe?)
Coming late to the party, if you'd like to use a function without using the DOM, here it goes, since the DOM might not even be available for whatever reason.
It should be applicable in any Browser which has the fetch API.
Just test it here:
// declare the function
function downloadAsDataURL (url) {
return new Promise((resolve, reject) => {
fetch(url)
.then(res => res.blob())
.then(blob => {
const reader = new FileReader()
reader.readAsDataURL(blob)
reader.onloadend = () => resolve(reader.result)
reader.onerror = err => reject(err)
})
.catch(err => reject(err))
})
}
// simply use it like this
downloadAsDataURL ('https://cdn-icons-png.flaticon.com/512/3404/3404134.png')
.then((res) => {
console.log(res)
})
.catch((err) => {
console.error(err)
})
export const downloadAs = async (url: string, name: string) => {
const blob = await axios.get(url, {
headers: {
'Content-Type': 'application/octet-stream',
},
responseType: 'blob',
});
const a = document.createElement('a');
const href = window.URL.createObjectURL(blob.data);
a.href = href;
a.download = name;
a.click();
};
You can use a clean code solution, inform your url in a constant, and set it as param of open method instead in object window.
const url = "file url here"
window.open(url)

Unexpected behavior for Facebook Sharing [duplicate]

This question already has an answer here:
Facebook ignoring OG image on first share
(1 answer)
Closed 6 years ago.
First of all hi and thanks in advance to anyone who can help with this because I've been going crazy over this for weeks now.
So I've got a website which lists gif taken from my mobile application (which are then stored on AWS and my visitors ( I haven't found a use for me to have users) can share these gifs on facebook using the facebook sdk.
The problem appears when I try sharing an image for the first time
This is what the share dialog shows the first time I click on my sharing button:
http://i.stack.imgur.com/lNVNF.png
and then I close and reclick the same button and now it works:
http://i.stack.imgur.com/YsDUm.png
Now I've been trying to find a way to make this work on the first sharing attempt but to no avail.
I am using meteor in combination with biasport:facebook-sdk and Amazon S3 for the hosting of my files.
Edit here is the code used:
FRONT SIDE
HTML
<div class="facebook share">
<img src="/gallery/fb.png">
</div>
Javascript
Template.*templateName*.events({
'click .facebook': function(e){
e.preventDefault();
e.stopPropagation();
// this is in a modal so I store the data I need
// (events have photos which in turn contain a url to the gif
var url = Session.get('event').photos[Session.get("id")].url;
FB.ui({
method: 'share',
href: url
});
}
SERVER SIDE
JAVASCRIPT
if(Meteor.isClient) {
window.fbAsyncInit = function() {
FB.init({
appId : 'APP_ID',
status : true,
xfbml : true,
version : 'v2.5'
});
};
}
Edit: I found a manual solution using exec future and curl
so first I added a call to a meteor method on the share that updates the facebook crawler
JAVASCRIPT
Template.*templateName*.events({
'click .facebook': function(e){
e.preventDefault();
e.stopPropagation();
// this is in a modal so I store the data I need
// (events have photos which in turn contain a url to the gif
var url = Session.get('event').photos[Session.get("id")].url;
Meteor.call('updateCrawler', url, function(){
FB.ui({
method: 'share',
href: url
});
});
}
Then I defined my meteor method as such
JAVASCRIPT
Meteor.methods({
updateCrawler: function(url){
var future = new Future();
cmd = 'curl -X POST -F "id=' + url + '" -F "scrape=true" -F "access_token={my_access_token}" "https://graph.facebook.com"';
exec(cmd, function(error){
if (error){
console.log(error);
}
future.return();
});
future.wait();
}
});
it's ugly but since I'd have to wait for the crawler to update and it works I'll leave this here for future use for someone maybe
Edit2:
I did not use og tags at all since I was simply sharing a url to aws directly and not a url to my website
I worked around this problem by calling the Facebook API direct from the server to make it scrape the og data by requesting info on the page. First time round it doesn't have the image cached but second time it does so this workaround does the initial call before sharing.
Use an access token for your facebook app and call the below in an ajax call and await the response before opening share dialog. Replace Google address with your own uri encoded address https://graph.facebook.com/v2.5/?id=http%3A%2F%2Fwww.google.co.uk&access_token=xxxxx
EDIT:
As per comments, here is my server side method for calling this which I use when posts etc are inserted to make the initial call and prompt a scrape from fb:
var getTheOGInfo = function (link)
{
if (!link || link.slice(0, 4).toLowerCase() != "http"){
throw new Meteor.Error("og-info-bad-url", "Function requires an unencoded fully qualified url");
return false;
}
var url = "https://graph.facebook.com/v2.5/{{{{id}}}}?access_token={{{{token}}}}&fields=og_object{id,description,title,type,updated_time,url,image},id,share";
var token = Meteor.settings.private.fb.token;
if (!token){
throw new Meteor.Error("og-info-no-token", "Function requires a facebook token in Meteor.settings.private.fb.token");
return false;
}
var link_id = encodeURIComponent(link);
url = url.replace('{{{{token}}}}', token).replace('{{{{id}}}}', link_id);
var result = HTTP.get(url, {timeout:1000});
return result;
}
Or for your purposes you may not want anything that might be blocking so you could change the last two lines to be aynchronous:
var result = HTTP.get(url, {timeout:1000});
return result;
//Replace with non blocking
HTTP.get(url, {timeout:1000}, function(err, result){console.log('something asynchronous', err, result);});
return true;

How to manipulate forms with Mootools

I'm trying to manipulate forms with Mootools. My purpose is to inject the response content of a form into a div element named result.
Here a code that works, but it replaces the content of the result div. This is not what I want : I want to ADD the form response content to the result div existing content. I just can't find on the web how to do this, and I've tried many things that are not working ... Please help
window.addEvent('domready', function() {
$('myform').addEvent('submit', function(e) {
e.stop();
var result = $('result').empty();
this.set('send',{
url: this.get('action'),
data: this,
onSuccess: function() {
result.set("html", this.response.text);
}
}).send();
});
});
If it's only text you want to add, just remove the empty method, and replace result.set() with result.appendText().
If you need to append an element tree, repeat the first step, and do:
onSuccess: function(){
Elements.from(this.response.text).inject(result);
}
Btw. It's all in the documentation - http://mootools.net/docs/core/Element/Element