can i get Chrome Developer Tools Network Panel'data by communicate with chrome remote debugging port ？
it can be saved as har formate file,but i want to save it by some script.
You can right click on the Sources panel wheren you see the network data, and save the entire view as a HAR file, or a single file as a HAR file. And then you can import that exact data into chrome which will give you everything you had back from that specific data load.
I assume you want to get your web app performance information back from your clients.
window.performance has a lot of information about how your page loaded and all timings for document ready and DOMContentLoaded and many other stuff in very high resolution.
Anyway that is not as detailed as Network panel in developer tools and if you really want to get that information back you should ask your user to grab that information manually and send it to you or ask them to install a chrome extension that does the job for you.
Related
I need to upload a file to OneDrive, via the command line. This will be done through a batch file which is distributed to end users.
From searching on Stack Overflow, I find questions like this one which say that you need to register an app and create an app password, using Azure. I don't have the necessary permissions to do this in the organization where I work, nor can I do anything that requires an admin account. So I can't any install software - I have to use what comes with Windows 10. I can't use VBA either as that's blocked.
I've managed to download files from OneDrive without anything like that, using the process described here:
Open the URL in either of the browser.
Open Developer options using Ctrl+Shift+I.
Go to Network tab.
Now click on download. Saving file isn’t required. We only need the network activity while browser requests the file from the server.
A new entry will appear which would look like “download.aspx?…”.
Right click on that and Copy → Copy as cURL.
Paste the copied content directly in the terminal and append ‘--output file.extension’ to save the content in file.extension since
terminal isn’t capable of showing binary data.
Example:
curl https://xyz.sharepoint.com/personal/someting/_layouts/15/download.aspx?UniqueId=cefb6082%2D696e%2D4f23%2D8c7a%2
…. some long text ….
cCtHR3NuTy82bWFtN1JBRXNlV2ZmekZOdWp3cFRsNTdJdjE2c2syZmxQamhGWnMwdkFBeXZlNWx2UkxDTkJic2hycGNGazVSTnJGUnY1Y1d0WjF5SDJMWHBqTjRmcUNUUWJxVnZYb1JjRG1WbEtjK0VIVWx2clBDQWNyZldid1R3PT08L1NQPg==;
cucg=1’ --compressed --output file.extension
I tried to do something similar after clicking 'upload' on the browser, but didn't find anything useful when trying to filter the requests.
I found these two questions but there is no keyboard shortcut to upload, AFAICT. Also the end user will be uploading a file to a folder I've shared with them from my OneDrive. Opening Chrome or Edge as a minimised window is fine, but I can't just shove a window in their face which automatically clicks on things - they won't like that.
It's just occurred to me that I might be able to use an office application to Save As the file to the necessary onedrive folder, where the keyboard shortcuts are pretty stable, but have no idea how to achieve that via the command line.
The best and more secure way to accomplish this goal I think is going to be with the Rest API for OneDrive.
(Small Files <4MB)
https://learn.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_put_content?view=odsp-graph-online
(Large files)
https://learn.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_createuploadsession?view=odsp-graph-online
You still need a Azure AD App Registration (which your admin should be able to configure for you), to provide API access to services in Azure. Coding with the API is going to be far easier and less complicated, not to mention more versatile.
I’m trying to use chrome devtools to see what network requests are.
But for some links, a new tab will be created for downloading a file and once the file is downloaded the tab is immediately closed.
There is no time to for me to inspect what the network requests are involved in the new tab. Is there a way to force the download in the original window so that I can still see the network activity?
As this answer suggest, yo may want to use chrome net export using chrome://net-export/
How it works?
You open a new tab and enter chrome://net-export/
Press the start logging to disk button and select a file
Do whatever
Press the stop recording button and inspect the file (should be formatted to be readable)
How to reproduce?
function popup() {
window.open('https://google.com', '_blank')
}
<button onclick="popup()">
click me
</button>
You will get WAY more information than you wished for, so - be patient when going over all the traffic details and also - make your recording as targeted and short as possible
Enjoy
EDIT
#Nathan raises a fair point in the comment - this method is not visual. a tool that may help to visualize the data is netlog viewer
Use the link, press the choose file button and upload your json file
In the left menu select events - this will display all events in a big table
Filter table by using URL_REQUEST or
Click each item to inspect and get detailed information (such as: url, headers, method, etc.)
There are other cool tools there (such as timeline) but it is different from chrome dev tools. This solution is just another set of tools for developers, that's all
I'm using R/RStudio on CentOS platform. I need to see if the binary log file (/var/log/messages) contains further information regarding a graphics device plotting issue, but I cannot read it from within RStudio.
I cannot find any information on the community support troubleshooting pages. The Help Menu -> Diagnostics -> ... does not indicate a 'Show Log Files' option. Can someone advise how this file can be accessed, preferably from within RStudio?
By default /var/log/messages is not readable by all users, so the first thing you need to do is make it accessible to whatever user you're logged in as in RStudio. This question has details and ideas:
https://serverfault.com/questions/258827/what-is-the-most-secure-way-to-allow-a-user-read-access-to-a-log-file
Once your user account has permissions to /var/log/messages, you can use something like the following to show it in RStudio:
> readLines("/var/log/messages")
(You could also try file.edit("/var/log/messages") to open it in an editor buffer, but that is less likely to succeed.)
Network analysis by Chrome when page loads
I would like to export this data to Microsoft Excel so that I will have a list of similar data when loaded at different times. Loading a page one time doesn't really tell me much especially if I want to compare pages.
if you right click on any of the rows you can export the item or the entire data set as HAR which appears to be a JSON format.
It shouldn't be terribly difficult to script up something to transform that to a csv if you really need it in excel, but if you're already scripting you might as well just use the script to ask your questions of the data.
If anyone knows how to drive the "load page, export data" part of the process from the command line I'd be quite interested in hearing how
from Chrome 76, you have Import/Export buttons.
I was trying to copy the size data measured from Chrome Network and stumbled on this post. I just found an easier way to "export" the data out to excel which is to copy the table and paste to excel.
The trick is click Control + A (select all) and once the entire table will be highlighted, paste it to Microsoft Excel. The only issue is if there are too many fields, not all rows are copied and you might have to copy and paste several times.
UPDATED: I found that copying the data only works when I turn off the filter options (the funnel-looking button above the table). – bendur
Right-click and export as HAR, then view it using Jan Odvarko's HAR Viewer
This helps in visualising the already captured HAR logs.
I came across the same problem, and found that easier way is to undock the developer tool's video to a separate window! (Using the right hand top corner toolbar button of developer tools window)
and in the new window , simply say select all and copy and paste to excel!!
In Chrome, in the Developer Tools, under Network, in the Name column, right-click and select "Save as HAR with content". Then open a new tab, go to https://toolbox.googleapps.com/apps/har_analyzer/ and open the saved HAR file.
Note that ≪Copy all as HAR≫ does not contain response body.
You can get response body via ≪Save as HAR with Content≫, but it breaks if you have any more than a trivial amount of logs (I tried once with only 8k requests and it doesn't work.) To solve this, you can script an output yourself using _request.contentData().
When there's too many logs, even _request.contentData() and ≪Copy response≫ would fail, hopefully they would fix this problem. Until then, inspecting any more than a trivial amount of network logs cannot be properly done with Chrome Network Inspector and its best to use another tool.
You can use fiddler web debugger to import the HAR and then it is very easy from their on... Ctrl+A (select all) then Ctrl+c (copy summary) then paste in excel and have fun
I don't see an export or save as option.
I filtered out all the unwanted requests using -.css -.js -.woff then right clicked on one of the requests then Copy > Copy all as HAR
Then pasted the content into a text editor and saved it.
I had same issue for which I came here. With some trials, I figured out for copying multiple pages of chrome data as in the question I zoomed out till I got all the data in one page, that is, without scroll, with very small font size. Now copy and paste that in excel which copies all the records and in normal font.
This is good for few pages of data I think.
In more modern versions of Chrome you can just drag a .har file into the network tab of Chrome Dev Tools to load it.
To get this in excel or csv format- right click the folder and select "copy response"- paste to excel and use text to columns.
You can try use Haiphen, which is a chrome extension that allows you to analyze network traffic and what API calls a web application is making.
So the website constantly changes the data that it displays, and I want to get that data every several seconds and log it in a spreadsheet. The problem is in order to get to the page, I have to have a cookie which I get when I log in. Unfortunately I only know how to program in MATLAB. MATLAB has a function for this, urlread, but it doesn't deal with cookies. What can I do to get to that page? Can anyone help me with this? Point me into a direction where a programing noob like me can succeed please.
You could use wget to download content while using HTTP cookies. I will be using StackOverflow.com as example target. Here are the steps to follow:
1) Obtain the wget command tool. For Mac or Linux, I think it is already available. On Windows, you can get it from the GnuWin32 project or from one of the many other ports (Cygwin, MinGW/MSYS, etc..).
2) Next we need to obtain an authenticated cookie by logging into the website in question. You can use your preferred browser for this.
In Internet Explorer, you can produce it using "File menu > Import and Export > Export Cookies". In Firefox, I used the Cookie Exporter extension to export cookies to text file. For Chrome, there should be similar extensions
Obviously you only need to do this step once, as long as the cookies have not yet expired!
3) Once you locate the cookie file exported, we can use wget to fetch the web page and provide it with this cookie. This of course can be performed from inside MATLAB using the SYSTEM function:
%# fetch page and save it to disk
url = 'http://stackoverflow.com/';
cmd = ['wget --cookies=on --load-cookies=./cookies.txt ' url];
system(cmd, '-echo');
%# process page: I am simply viewing it using embedded browser
web( ['file:///' strrep(fullfile(pwd,'index.html'),'\','/')] )
Parsing the web page is a whole other topic that I will not go into. Once you get the data you seek, you can interact with Excel spreadsheets using the XLSREAD and XLSWRITE functions.
4) Finally you can write this in a function, and make it execute on regular intervals using the TIMER function
Try using the java.net.* classes.
You should be able to use them directly in the MATLAB workspace, as described here: http://www.mathworks.co.uk/help/techdoc/matlab_external/f4863.html
Matlab has built-in functions for web downloading. For http sites, there is webread.m and websave.m. For FTPs, there is mget.m