Getting all styles with devtool-protocol in puppeteer - google-chrome-devtools

I'm trying to get all styles for all nodes on page and for that i want to use CSS.getMatchedStylesForNode from devtool-protocol, but its only working for one node. If loop through an array of nodes i get a lot of warning in console(code below) and nothing is returned. What i'm doing wrong ?
warning in console:
(node:5724) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 11): Error: Protocol error (CSS.getMatchedStylesForNode): Target closed.
my code
'use strict';
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://example.com');
await page._client.send('DOM.enable');
await page._client.send('CSS.enable');
const doc = await page._client.send('DOM.getDocument');
const nodes = await page._client.send('DOM.querySelectorAll', {
nodeId: doc.root.nodeId,
selector: '*'
});
const styleForSingleNode = await page._client.send('CSS.getMatchedStylesForNode', {nodeId: 3});
const stylesForNodes = nodes.nodeIds.map(async (id) => {
return await page._client.send('CSS.getMatchedStylesForNode', {nodeId: id});
});
console.log(JSON.stringify(stylesForNodes));
console.log(JSON.stringify(styleForSingleNode));
await browser.close();
})();
Puppeteer version: 0.13.0
Platform: Window 10
Node: 8.9.3

Works using for of loop
const stylesForNodes = []
for (id of nodes.nodeIds) {
stylesForNodes.push(await page._client.send('CSS.getMatchedStylesForNode', {nodeId: id}));
}

Related

One time authentication in playwright is giving issues

I tried to have a one time authentication using session and use the same for all the tests in the spec file.
While trying to run my test , sometimes i get this below error which im unable to underdstand or fix. Any help on this would be appreciated.
browser.newContext: Cookie should have a valid expires, only -1 or a positive number for the unix timestamp in seconds is allowed
at C:\Users\v.shivarama.krishnan\source\repos\PlaywrightDemo\node_modules\#playwright\test\lib\index.js:595:23
at Object.context [as fn] (C:\Users\v.shivarama.krishnan\source\repos\PlaywrightDemo\node_modules\#playwright\test\lib\index.js:642:15)
Spec.ts
import { chromium, test, expect } from "#playwright/test";
test.describe('Launch Browser', () => {
await context.storageState({ path: 'storage/admin.json' });
await page.goto('abc.com');
await expect(page.locator('#ebiLink')).toBeVisible();
const texts = await page.locator('#ebiLink').textContent();
console.log("text of ebi is " + texts);
await page.goto('abc.com');
await expect(page.locator('text= Detailed Interfaces ')).toBeVisible();
await page.waitForSelector('#searchTab');
await page.waitForSelector('#InterfaceCard');
await page.locator('#searchTab').type('VISHW-I7939');
await page.locator("button[type='button']").click();
await page.locator('#InterfaceCard').first().click();
await expect(page.locator('#ngb-nav-0')).toBeVisible();
const interfaceID = await page.locator("//span[#class='value-text']").first().allInnerTexts();
console.log('interface id is' + interfaceID);
const dp = await page.waitForSelector('role=listbox');
await page.locator('role=listbox').click();
const listcount = await page.locator('role=option').count();
await page.locator('role=option').nth(1).click();
await expect(page.locator('#ngb-nav-0')).toBeVisible();
});
test('Move to shells screen', async ({ page, context }) => {
await context.storageState({ path: 'storage/admin.json' });
await page.goto('abc.com');
await expect(page.locator('#ListHeader')).toBeVisible();
const shells = await page.locator('#ListHeader').textContent();
console.log('Text of shells header is '+shells);
});
});
global-setup.ts (for one time login and getting the session)
import { Browser, chromium, FullConfig } from '#playwright/test'
async function globalSetup(config: FullConfig) {
const browser = await chromium.launch({
headless: false
});
await saveStorage(browser, 'Admin', 'User', 'storage/admin.json')
await browser.close()
}
async function saveStorage(browser: Browser, firstName: string, lastName: string, saveStoragePath: string) {
const page = await browser.newPage()
await page.goto('abc.com');
await page.waitForSelector("//input[#type='email']", { state: 'visible' });
await page.locator("//input[#type='email']").type('ABC#com');
await page.locator("//input[#type='submit']").click();
await page.locator("//input[#type='password']").type('(&^%');
await page.locator("//input[#type='submit']").click();
await page.locator('#idSIButton9').click();
await page.context().storageState({ path: saveStoragePath })
}
export default globalSetup
Have you registered global-setup.ts script in the Playwright configuration file: like below?
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
globalSetup: require.resolve('./global-setup'),
};
export default config;
again you don't have to write code to use the session-storage at each test level, you can use - use attribute of Playwright configuration as below:
// playwright.config.ts
import { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
globalSetup: require.resolve('./global-setup'),
use: {
// Tell all tests to load signed-in state from 'storageState.json'.
storageState: 'storageState.json'
}
};
export default config;
Seems like you are trying to use the same context in both tests, could that be a problem?
can you please try with isolated context and page for each tests?
Also please check if it make sense to use session storage at test level instead of context-
test.use({ storageState: './storage/admin.json' })
Update about the tests-
General structure of tests would be -
test.describe('New Todo', () => {
test('Test 1', async ({context, page }) => {});
test('Test 2', async ({context, page }) => {});
});
I looked into the source code of playwright and found these two lines which show the error message you see.
assert(!(c.expires && c.expires < 0 && c.expires !== -1), 'Cookie should have a valid expires, only -1 or a positive number for the unix timestamp in seconds is allowed');
assert(!(c.expires && c.expires > 0 && c.expires > kMaxCookieExpiresDateInSeconds), 'Cookie should have a valid expires, only -1 or a positive number for the unix timestamp in seconds is allowed');
The kMaxCookieExpiresDateInSeconds is defined as 253402300799.
So basically the cookie that you captured could breach one of above rules. In my case it's that the expiry of a cookie is greater than this figure :).
refer to source code - https://github.com/microsoft/playwright/blob/5fd6ce4de0ece202690875595aa8ea18e91d2326/packages/playwright-core/src/server/network.ts#L53

access document.documentElement from puppeteer

I can get access to the entire HTML for any URL by opening dev-tools and typing:
document.documentElement
I am trying to replicate the same behavior using puppeteer, however, the snippet below returns {}
const puppeteer = require('puppeteer'); // v 1.1.0
const iPhone = puppeteer.devices['Pixel 2 XL'];
async function start(canonical_url) {
const browserURL = 'http://127.0.0.1:9222';
const browser = await puppeteer.connect({browserURL});
const page = await browser.newPage();
await page.emulate(iPhone);
await page.goto(canonical_url, {
waitUntil: 'networkidle2',
});
const data = await page.evaluate(() => document.documentElement);
console.log(data);
}
returns:
{}
Any idea on what I could be doing wrong here?

Cannot read property innerText of null for valid selector using playwright

This script is supposed to retrieve the innerText of a DOM element, the elements selector is
('div[class=QzVHcLdwl2CEuEMpTUFaj]') and I've hand-tested the selector and called the getSharePrice function in the REPL which also works.
const { chromium } = require('playwright');
const util = require('util');
const setTimeoutPromise = util.promisify(setTimeout);
(async () => {
const userDataDir = 'path'
const browser = await chromium.launchPersistentContext(userDataDir, {headless: false });
const page = await browser.newPage();
await page.goto('https://robinhood.com', {timeout: 60000, waitUntil: 'domcontentloaded'});
await getSharePrice(page)
await setTimeoutPromise(1000);
await browser.close();
})();
async function getSharePrice(page) {
const price = await page.evaluate(() => {
return {
price: document.querySelector('div[class=QzVHcLdwl2CEuEMpTUFaj]').innerText.replace(/\D/g,'')
}
});
console.log(price)
}
for some reason, I am getting a (node:59324) UnhandledPromiseRejectionWarning: Error: Evaluation failed: TypeError: Cannot read property 'innerText' of null error, not sure why.
The only thing that I could come up with is that the element hasn't been loaded yet, causing it to evaluate to null which is why innerText can't be called.
adding await page.waitForSelector('div[class=QzVHcLdwl2CEuEMpTUFaj]') before my evaluate block fixed this. Looks like the issue was caused by the element not being loaded yet

mongodb jest error ReferenceError: You are trying to `import` a file after the Jest environment has been torn down

I am trying to test basic mongoDB operation using Jest.
it('should insert a record into collection', async () => {
const users = await db.collection('mariochar');
const mockUser = { name: 'marioChar3'};
await users.insertOne(mockUser);
const insertedUser = await users.findOne({name: 'marioChar3'});
expect(insertedUser).toEqual(mockUser);
});
If I use the default Jest suggested template as above, tests are passing correctly and records can also be seen in the database.
However, since I was learning from this youtube tutorial, I tried to test the Schema directly as shown in the video like this:
it('should insert a mario into collection', async () => {
const MarioCharSchema = new Schema({
name: String,
weight: Number
})
const MarioChar = await mongoose.model('mariochar', MarioCharSchema)
const char = await new MarioChar({ name: "Mario" })
char.save()
console.log(char.isNew)
expect(char.isNew).toBeTruthy()
});
I get the error: ReferenceError: You are trying toimporta file after the Jest environment has been torn down.
After viewing few posts in stackoverflow, I added the fake timer and I am posting the complete code below. It did not get rid of the error.
My second question is that even though Jest shows the above error and also npm ERR! Test failed. See above for more details., it shows the test as passed. Is there a way to report it as failed instead since it is easy to mistake it as passed when there will be a lot of tests?
const { MongoClient } = require('mongodb');
const mongoose = require('mongoose')
const Schema = mongoose.Schema
jest.useFakeTimers()
describe('insert', () => {
let connection;
let db;
beforeAll(async () => {
jest.useFakeTimers()
connection = await MongoClient.connect("mongodb://localhost/testaroo", {
useNewUrlParser: true,
useUnifiedTopology: true
});
db = await connection.db();
});
afterAll(async () => {
await connection.close();
});
it('should insert a mario into collection', async () => {
const MarioCharSchema = new Schema({
name: String,
weight: Number
})
const MarioChar = await mongoose.model('mariochar', MarioCharSchema)
const char = await new MarioChar({ name: "Mario" })
char.save()
expect(char.isNew).toBeTruthy()
});
});
jest.config.js
module.exports = {
"preset": '#shelf/jest-mongodb',
"timers": "fake",
"testEnvironment": "node"
};
package.json
...
"dependencies": {
"mongoose": "^5.9.18"
},
"devDependencies": {
"#shelf/jest-mongodb": "^1.1.5",
"jest": "^26.0.1"
}
...
full error:
ReferenceError: You are trying to `import` a file after the Jest environment has been torn down.
at model.Document.$__getAllSubdocs (node_modules/mongoose/lib/document.js:2890:37)
at _getPathsToValidate (node_modules/mongoose/lib/document.js:2159:23)
at model.Document.$__validate (node_modules/mongoose/lib/document.js:2314:23)
at node_modules/kareem/index.js:369:33
at node_modules/#jest/fake-timers/build/legacyFakeTimers.js:496:18
PASS test/saving.test.js

Puppeteer Generate PDF from multiple HTML strings

I am using Puppeteer to generate PDF files from HTML strings.
Reading the documentation, I found two ways of generating the PDF files:
First, passing an url and call the goto method as follows:
page.goto('https://example.com');
page.pdf({format: 'A4'});
The second one, which is my case, calling the method setContent as follows:
page.setContent('<p>Hello, world!</p>');
page.pdf({format: 'A4'});
The thing is that I have 3 different HTML strings that are sent from the client and I want to generate a single PDF file with 3 pages (in case I have 3 HTML strings).
I wonder if there exists a way of doing this with Puppeteer? I accept other suggestions, but I need to use chrome-headless.
I was able to do this by doing the following:
Generate 3 different PDFs with puppeteer. You have the option of saving the file locally or to store it in a variable.
I saved the files locally, because all the PDF Merge plugins that I found only accept URLs and they don't accept buffers for instance. After generating synchronously the PDFs locally, I merged them using PDF Easy Merge.
The code is like this:
const page1 = '<h1>HTML from page1</h1>';
const page2 = '<h1>HTML from page2</h1>';
const page3 = '<h1>HTML from page3</h1>';
const browser = await puppeteer.launch();
const tab = await browser.newPage();
await tab.setContent(page1);
await tab.pdf({ path: './page1.pdf' });
await tab.setContent(page2);
await tab.pdf({ path: './page2.pdf' });
await tab.setContent(page3);
await tab.pdf({ path: './page3.pdf' });
await browser.close();
pdfMerge([
'./page1.pdf',
'./page2.pdf',
'./page3.pdf',
],
path.join(__dirname, `./mergedFile.pdf`), async (err) => {
if (err) return console.log(err);
console.log('Successfully merged!');
})
I was able to generate multiple PDF from multiple URLs from below code:
package.json
{
............
............
"dependencies": {
"puppeteer": "^1.1.1",
"easy-pdf-merge": "0.1.3"
}
..............
..............
}
index.js
const puppeteer = require('puppeteer');
const merge = require('easy-pdf-merge');
var pdfUrls = ["http://www.google.com","http://www.yahoo.com"];
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
var pdfFiles=[];
for(var i=0; i<pdfUrls.length; i++){
await page.goto(pdfUrls[i], {waitUntil: 'networkidle2'});
var pdfFileName = 'sample'+(i+1)+'.pdf';
pdfFiles.push(pdfFileName);
await page.pdf({path: pdfFileName, format: 'A4'});
}
await browser.close();
await mergeMultiplePDF(pdfFiles);
})();
const mergeMultiplePDF = (pdfFiles) => {
return new Promise((resolve, reject) => {
merge(pdfFiles,'samplefinal.pdf',function(err){
if(err){
console.log(err);
reject(err)
}
console.log('Success');
resolve()
});
});
};
RUN Command: node index.js
pdf-merger-js is another option. page.setContent should work just the same as a drop-in replacement for page.goto below:
const PDFMerger = require("pdf-merger-js"); // 3.4.0
const puppeteer = require("puppeteer"); // 14.1.1
const urls = [
"https://news.ycombinator.com",
"https://en.wikipedia.org",
"https://www.example.com",
// ...
];
const filename = "merged.pdf";
let browser;
(async () => {
browser = await puppeteer.launch();
const [page] = await browser.pages();
const merger = new PDFMerger();
for (const url of urls) {
await page.goto(url);
merger.add(await page.pdf());
}
await merger.save(filename);
})()
.catch(err => console.error(err))
.finally(() => browser?.close())
;