Using Puppeteer, how to get Chrome DevTools' "Network" tab's timing information? - google-chrome-devtools

Question

Below is a screenshot of me accessing https://www.ted.com and inspecting the Google Chrome DevTools' "Network" tab, and viewing the Timing data for the root & child requests.
How can I get all of the above timing data programatically using Puppeteer? Ideally, it would look like this JSON structure:
{
name: "www.ted.com",
queueTime: 0,
startedTime: 1.93,
stalledTime: 4.59,
dnsLookupTime: 10.67,
initialConnectionTime: <the number of milliseconds>,
...
},
{
name: <the next child request>,
...
}

Answer 1

You want to check out HAR (HTTP Archive) files, which is what you would create by saving the data via Chrome.
Quotation what a HAR file is (source):
The HAR file format is an evolving standard and the information contained within it is both flexible and extensible. You can expect a HAR file to include a breakdown of timings including:
How long it takes to fetch DNS information
How long each object takes to be requested
How long it takes to connect to the server
How long it takes to transfer assets from the server to the browser of each object
The data is stored as a JSON document and extracting meaning from the
low-level data is not always easy. But with practice a HAR file can
quickly help you identify the key performance problems with a web
page, letting you efficiently target your development efforts at areas
of your site that will deliver the greatest results.
There are libraries like puppeteer-har and chrome-har which can create HAR files by using puppeteer.
Code sample (for puppeteer-har, quote from the page)
const har = new PuppeteerHar(page);
await har.start({ path: 'results.har' });
await page.goto('http://example.com');
await har.stop();

Answer 2

HAR files are a good option, but if want something a bit more custom you can use Puppeteer to record request timing data by navigating to the page you wish to analyze, and tapping into the Chrome DevTools Protocol.
(async function() {
// launch in headless mode & create a new page
const browser = await pptr.launch({
headless: true,
});
const page = await browser.newPage();
// attach cdp session to page
const client = await page.target().createCDPSession();
await client.send('Debugger.enable');
await client.send('Debugger.setAsyncCallStackDepth', { maxDepth: 32 });
// enable network
await client.send('Network.enable');
// attach callback to network response event
await client.on('Network.responseReceived', (params) => {
const { response: { timing } } = params;
/*
* See: https://chromedevtools.github.io/devtools-protocol
* /tot/Network/#type-ResourceTiming for complete list of
* timing data available under 'timing'
*/
});
await page.goto('https://www.ted.com/', {
waitUntil: 'networkidle2',
});
// cleanup
await browser.close();
})();
For your case, you can listen on the Network.responseReceived event, and parse out the responseTime parameter, nested within the response property on the response object provided in the event listener callback. Their documentation on the interfaces are quite good. I'll list them below:
Chrome DevTools Protocol Docs
Data you can expect to receive from every Network.responseReceived event callback: Network.responseReceived.
More specific response-related data, in the response property: Network.Response.
And, finally, the nested request timing data you are looking for, under timing: Network.ResourceTiming.
You may also want to check out the Network.requestWillBeSent interface. You will be able to match up requests and responses by requestId.
From here, you can capture more data than you could ever need about the page you're visiting. You can also format however you wish obviously.

Answer 3

Currently, you can also get this information without the HAR file.
Using performance.getEntriesByType("resource")
// Obtain PerformanceEntry objects for resources
const performanceTiming = JSON.parse(
await page.evaluate(() =>
JSON.stringify(performance.getEntriesByType("resource"))
)
);
// Optionally filter resource results to find your specifics - ex. filters on URL
const imageRequests = performanceTiming.filter((e) =>
e.name.endsWith("/images")
);
console.log("Image Requests " , imageRequests)