I am trying to download HTML content from any URL through webbrowser control in C#.net.
I choose webrowser to handle Javascript issues. I am using webbrowser control without placing
it on the form. It works great for one url, but when I call it for multiple urls I am unable
to download the page.
Here is the code
GetWebpage()
{
System.Windows.Forms.WebBrowser wb = new System.Windows.Forms.WebBrowser();
wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wb_DocumentCompleted);
wb.Navigate(sURI, false);
bDocumentLoaded = false;
while (!bDocumentLoaded)
{
Application.DoEvents();
Thread.Sleep(100);
}
sHTML = wb.DocumentText;
bDocumentLoaded = false;
}
Event:
private void wb_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
System.Windows.Forms.WebBrowser webBrowser1;
webBrowser1 = sender as WebBrowser;
string strTit = webBrowser1.DocumentTitle;
string str = webBrowser1.DocumentText;
bDocumentLoaded = true;
}
Cheers,
Karthik
You can use webclient object to fetch data from some url.
Try using Downloading String
public static void DownloadString (string address)
{
WebClient client = new WebClient ();
string reply = client.DownloadString (address);
Console.WriteLine (reply);
}
You can also use ASYC method of same downloading string.
I think your problem is that some sites are detecting specific browsertype and then they are returning HTML
Try setting the HeaderProperty of WebClient Object this is a list of HttpWebRequest Object
For Example
myWebClient.Headers.Add("Content-Type","application/x-www-form-urlencoded");
Modify the useragent of HTTPWEBRequest then add to headers.
HTTPWEBRequest.UserAgent=".NET Framework Test Client";
You can check more information about this in MSDN Link
I might recommend using the mshtml and SHDocVW libraries and using approach found in the answer here:
Unable to to locate and click a submit button using mshtml.HTMLInputElement
Related
I am using Unity 5.5.2f1 pro and facebook's SDK v 7.9.4
I have a script which after login (managed in a previous scene) sends an API request to FB asking for friends, name and email and sends that info as a POST to a php website.
code:
[Serializable]
public struct FBData {
public string first_name;
public string email;
public string friends;
public string id;}
public class UserManagement : MonoBehaviour {
string urlSaveUserData="some php website";
public Text testTxt;
FBData parsedData;
// Use this for initialization
void Start () {
//Check if it's the first time the user is opening the app.
if (UserInfo.FIRST_TIME) {
//update text (only used for testing, should be removed in production.)
testTxt.text = "Your user id is: " + UserInfo.ID;
//Perform FB.API call to get User Data.
getUserData ();
//Save in SQL table. (won't get here if line in getUserData() is active)
StartCoroutine ("saveUserData");
} else {
//do something else.
}
note: Since this is meant for iOS I have to test it on a device so I'm using text in the screen to display info (think of it as a badly implemented print statement).
The problem: In my callback function for FB.API I write in the text Gameobject (aka testTxt) the parsed information from the response which is saved in the Custom UserInfo clss. It display's correctly but the code gets stuck there. It doesn't continue to the next function. HOWEVER, if I delete/comment that line and don't display anything in the text field. The codes does continue to the POST function BUT the information from the API call is not passed, i.e my custom class is empty (leading me to believe the callback function is not called at all).
public void getUserData(){
string query = "me?fields=first_name,email,friends";
FB.API (query, HttpMethod.GET, Apicallback, new Dictionary<string, string> ());
}
private void Apicallback(IGraphResult result){
//Parse Graph response into a specific class created for this result.
parsedData = JsonUtility.FromJson<FBData>(result.RawResult);
//Pass each field into UserInfo class.
UserInfo.EMAIL = parsedData.email;
UserInfo.FRIENDS = parsedData.friends;
UserInfo.NAME = parsedData.first_name;
UserInfo.FACEBOOKID = parsedData.id;
/*problem area, if I comment line below, then previous information is apparently not stored. If left as is then testTxt displays correct information but code gets stuck there. */
testTxt.text = "This is the info from USerInfoInside the APICallback: " + UserInfo.EMAIL + UserInfo.FRIENDS + UserInfo.FACEBOOKID;
}
The function below is to send info to php website, is there for illustrative purposes:
code:
public IEnumerator saveUserData() {
//get user info (this information is EMPTY if line in getUserData() is commented.
parsedData.id = UserInfo.FACEBOOKID;
parsedData.friends = UserInfo.FRIENDS;
parsedData.first_name = UserInfo.NAME;
parsedData.email = UserInfo.EMAIL;
//translate data into json
string JsonBodyData = JsonUtility.ToJson (parsedData);
//Custom web request (POST method doesnt seem to work very well, documentation example sends empty form)
var w = new UnityWebRequest(urlSaveUserData, "POST");
byte[] bodyRaw = new System.Text.UTF8Encoding().GetBytes(JsonBodyData);
w.uploadHandler = (UploadHandler) new UploadHandlerRaw(bodyRaw);
w.downloadHandler = (DownloadHandler) new DownloadHandlerBuffer();
w.SetRequestHeader("Content-Type", "application/json");
yield return w.Send();
//work with received data...}
Im stuck here any help is appreciated. Thanks!
Be sure to use EscapeURL when using strings directly for JSON or HTTP POST and GET methods. The lack of this treatment tends to screw things over, particulary in iOS platforms.
From what I can see, this code
string query = "me?fields=first_name,email,friends";
should instead be escaped as
string query = WWW.EscapeURL("me?fields=first_name,email,friends");
so characters like "?" won't get encoded as an URL symbol.
I'm assuming you don't need to do that for your illustrative example, because UnityWebRequest already escapes your POST request strings internally, but I can't fully confirm that.
I have made a REST API and I want to use it using my Xamarin.iOS application.
Basically I want to call the API from my Xamarin application by sending some arguments to one of my API's function.
I tried the resources available at Xamarin's official website, but I a newbie so I cannot understand how it was done.
The REST API is hosted locally by the network I am using. It is not hosted at a static IP.
Kindly guide me.
You don't really need a fancy plugin if you just want to hit Web Endpoints. I simply use the basic WebRequest API.
var request = WebRequest.CreateHttp(YOUR_URL_HERE);
request.Method = "GET";
request.ContentType = "application/JSON";
request.BeginGetResponse(ResponseComplete, request);
... and then your response method can be something along the lines of...
protected void ResponseComplete(IAsyncResult result)
{
try
{
var request = result.AsyncState as HttpWebRequest;
if (request != null)
{
Debug.WriteLine("Completed query: " + request.RequestUri);
using (var streamReader = new StreamReader(response.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
Debug.WriteLine("Query Result: " + result);
}
}
}
}
... and if you need to post data you can add request.BeginGetRequestStream(PostData, request); before request.BeginGetResponse(ResponseComplete, request); and make your GetRequestStream handling method something along the lines of...
protected void PostData(IAsyncResult result)
{
var request = result.AsyncState as HttpWebRequest;
if (request != null)
{
using (var postStream = request.EndGetRequestStream(result))
{
var json = JsonConvert.SerializeObject(DATA_TO_POST);
Debug.WriteLine("Posting data: " + json);
var byteArray = Encoding.UTF8.GetBytes(json);
postStream.Write(byteArray, 0, byteArray.Length);
}
}
}
I would recommend Refit, you can install it as a NuGet package. Its pritty simple to use.
Refit allows us to define an interface that describes the API that we're calling, and the Refit framework handles making the call to the service and deserializing the return.
Have a look at this great blog post on how to set it up and other packages that might help you out. http://arteksoftware.com/resilient-network-services-with-xamarin/
I have used RestSharp before but Refit is alot easier to get running.
This question already has answers here:
Is it possible to download a file from server to my device having MGWT&GwtPhoneGap app?
(2 answers)
Closed 8 years ago.
I am using gwt canvas .
I am having 74kb string(image) data which I want to pass it to servlet. So that servlet process that data and throw the contents to the browser . In this way it will prompt user to download it.
From client side I am using RequestBuilder to call a servlet ,set request data to it ,data is large so I am using post request. It is hitting servlet also throwing contents on the browser but not showing anything downloading.
The current url having canvas .I thinks that's why it is not downloading anything this conclusion is because If am opening that servlet directly using http://localhost:8080/servlet then its downloading it property (In this case i am not proving any content from client side) but for the url having canvas its giving problem.
So is there any way where i can open a url in new tab and can call servlet using post request in gwt.
You can use a Form Panel for do the upload, the Form Panel will use a hidden iframe,
FormPanel form = new FormPanel();
form.setMethod(FormPanel.METHOD_POST);
form.setAction("/downloadServlet");
FlowPanel hiddenPanel = new FlowPanel();
hiddenPanel.add(new Hidden("name1", "value"));
hiddenPanel.add(new Hidden("name2", "value"));
form.setWidget(hiddenPanel);
RootPanel.get().add(form);
form.submit();
The content return by the servlet if you put the correct header will be downloaded by the user navigator.
public class ServletDownloadDemo extends HttpServlet{
private static final int BYTES_DOWNLOAD = 1024;
#Override
public void doPost(HttpServletRequest request,
HttpServletResponse response) throws IOException{
//Get Parameters
String name1 = request.getParameter("name1");
String name2 = request.getParameter("name2");
response.setContentType("text/plain");
response.setHeader("Content-Disposition",
"attachment;filename=downloadname.txt");
ServletContext ctx = getServletContext();
InputStream is = ctx.getResourceAsStream("/testing.txt");
int read=0;
byte[] bytes = new byte[BYTES_DOWNLOAD];
OutputStream os = response.getOutputStream();
while((read = is.read(bytes))!= -1){
os.write(bytes, 0, read);
}
os.flush();
os.close();
}
}
I am using MonoTouch to build an iPhone app. In the app I am making Web Requests to pull back information from the web services running on our server.
This is my method to build the request:
public static HttpWebRequest CreateRequest(string serviceUrl, string methodName, JsonObject methodArgs)
{
string body = "";
body = methodArgs.ToString();
HttpWebRequest request = WebRequest.Create(serviceUrl) as HttpWebRequest;
request.ContentLength = body.Length; // Set type to POST
request.Method = "POST";
request.ContentType = "text/json";
request.Headers.Add("X-JSON-RPC", methodName);
StreamWriter strm = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
strm.Write(body);
strm.Close();
return request;
}
Then I call it like this:
var request = CreateRequest(URL, METHOD_NAME, args);
request.BeginGetResponse (new AsyncCallback(ProcessResponse), request);
And ProcessResponse looks like this:
private void ProcessResponse(IAsyncResult result)
{
try
{
HttpWebRequest request = (HttpWebRequest)result.AsyncState;
using (HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result)) // this is where the exception gets thrown
{
using (StreamReader strm = new System.IO.StreamReader(response.GetResponseStream()))
{
JsonValue value = JsonObject.Load(strm);
// do stuff...
strm.Close();
} // using
response.Close();
} // using
Busy = false;
}
catch(Exception e)
{
Console.Error.WriteLine (e.Message);
}
}
There is another question about this issue for Monodroid and the answer there suggested explicitly closing the output stream. I tried this but it doesn't solve the problem. I am still getting a lot of ReadDone2 errors occurring.
My workaround at the moment involves just re-submitting the Web Request if an error occurs and the second attempt seems to work in most cases. These errors only happen when I am testing on the phone itself and never occur when using the Simulator.
Whenever possible try to use WebClient since it will deal automatically with a lot of details (including streams). It also makes it easier to make your request async which is often helpful for not blocking the UI.
E.g. WebClient.UploadDataAsync looks like a good replacement for the above. You will get the data, when received from the UploadDataCompleted event (sample here).
Also are you sure your request is always and only using System.Text.Encoding.ASCII ? using System.Text.Encoding.UTF8 is often usedm, by default, since it will represent more characters.
UPDATE: If you send or receive large amount to byte[] (or string) then you should look at using OpenWriteAsync method and OpenWriteCompleted event.
This is a bug in Mono, please see https://bugzilla.xamarin.com/show_bug.cgi?id=19673
this is what I have so far:
void xmppConnection_OnReadXml(object sender, string xml)
{
if (xml.Contains(XmlTags.PhotoOpen))
{
int startIndex = xml.IndexOf(XmlTags.PhotoOpen) + XmlTags.PhotoOpen.Length;
int length = xml.IndexOf(XmlTags.PhotoClose) - startIndex;
string photoHash = xml.Substring(startIndex, length);
}
}
I guess I can't undo the hash, but I want to the get a person's avatar/photo. How do I achieve this?
You need to handle the VCard events and responses from XMPP connection:
private void vcardToolStripMenuItem_Click(object sender, EventArgs e)
{
RosterNode node = rosterControl.SelectedItem();
if (node != null)
{
frmVcard f = new frmVcard(node.RosterItem.Jid, XmppCon);
f.Show();
}
}
The above is from the miniclient solution example from the AGSXMPP download. Note, it happens when a user request a VCARD for a user. You can initiate that request whenever you want, however.
private void VcardResult(object sender, IQ iq, object data)
{
if (InvokeRequired)
{
// Windows Forms are not Thread Safe, we need to invoke this :(
// We're not in the UI thread, so we need to call BeginInvoke
BeginInvoke(new IqCB(VcardResult), new object[] { sender, iq, data });
return;
}
if (iq.Type == IqType.result)
{
Vcard vcard = iq.Vcard;
if (vcard!=null)
{
txtFullname.Text = vcard.Fullname;
txtNickname.Text = vcard.Nickname;
txtBirthday.Text = vcard.Birthday.ToString();
txtDescription.Text = vcard.Description;
Photo photo = vcard.Photo;
if (photo != null)
picPhoto.Image = vcard.Photo.Image;
}
}
}
That is what happens when someone requests the VCARD information from XMPP and the IQ type matches the proper data. You can thenpull the photo from vcard.Photo.
You trigger the pull with:
VcardIq viq = new VcardIq(IqType.get, new Jid(jid.Bare));
con.IqGrabber.SendIq(viq, new IqCB(VcardResult), null);
The first line there is the request to the XMPP server, that the VCARD form uses to request user information.
The second line there, sets up another grabber (callback of sorts), that the form uses to wait for the information to arrive, and then parse out the necessary information. IN this case, the grabber is in a new form, so that the main application doesn't have to worry about parsing that information.
You can look at the entire source by extracting the AGSXMPP zip file to your local drive, and looking in the Samples\VS2008\miniclient folder.
You can click link:http://forum.ag-software.de/thread/192-How-to-save-vcard-data