I have an app that is attempting to webscrape GHIN.com for golfers' handicaps. After prompting the user with a WKWebView page for login, the user opens the "followed" golfers page in order for the app to scrape the web page for handicap data. I am able to retrieve HTML from the webView using:
webView.evaluateJavaScript("document.documentElement.outerHTML.toString()", completionHandler: { (html: Any?, error: Error?) in
self.HTML = html as! String
print(self.HTML)
})
This code is called whenever the URL is detected to have changed (as the user logs in and navigates to the correct page). However, the HTML saved from that process includes JS code seen near the end of the string that is to be executed during the building of the page (entire HTML):
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="shortcut icon" href="/favicon.ico" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png" />
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png" />
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png" />
<link rel="manifest" href="/site.webmanifest" />
<link rel="mask-icon" href="/safari-pinned-tab.svg" color="#5bbad5" />
<meta name="msapplication-TileColor" content="#da532c" />
<meta name="theme-color" content="#ffffff" />
<meta name="apple-itunes-app" content="app-id=491796218" />
<meta name="google-play-app" content="app-id=com.advancedmobile.android.ghin" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<meta name="theme-color" content="#000000" />
<link rel="manifest" href="/manifest.json" />
<title>GHIN - United States Golf Association Service</title>
<link href="https://fonts.googleapis.com/icon?family=Material+Icons+Outlined" rel="stylesheet" />
<link href="/static/css/2.c7bb94bc.chunk.css" rel="stylesheet" />
<link href="/static/css/main.6b12f9df.chunk.css" rel="stylesheet" />
<script async="" src="//www.googletagservices.com/tag/js/gpt.js"></script>
<script src="https://securepubads.g.doubleclick.net/gpt/pubads_impl_2020082401.js?21067294" async=""></script>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root">
<div></div>
<div class="s-navigation" id="header" style="top: 0px;">
<header>
<div class="header__container">
<div class="menu-ham__container">
<button class="menu-ham__trigger"><span></span><span></span><span></span></button>
<div class="menu-ham__content">
<div class="menu__links__wrapper">
<ul class="menu__links">
<li class="hide-on-tablet-and-above">
<a class="menu__link" href="/post-score"><span>Post Score</span></a>
</li>
<li class="hide-on-tablet-and-above">
<a class="menu__link is-active" href="/golfer-lookup" aria-current="page"><span>Golfer Lookup</span></a>
</li>
<li class="hide-on-tablet-and-above">
<a class="menu__link" href="/stats"><span>Stats</span></a>
</li>
<li class="hide-on-tablet-and-above">
<a class="menu__link" href="/handicap-calculator/select-course"><span>Handicap Calculator</span></a>
</li>
<li class="hide-on-tablet-and-above">
<a class="menu__link" href="/profile-handicap-history"><span>Handicap History</span></a>
</li>
<li class="order-1">
<a class="menu__link" href="/my-card"><span>My Card</span></a>
</li>
<li class="order-2">
<a class="menu__link" target="_blank" href="https://www.usga.org/content/usga/home-page/handicapping.html"><span>WHS Changes</span></a>
</li>
<li class="order-3">
<a class="menu__link" href="/ghin-associations"><span>GHIN Associations</span></a>
</li>
<li class="order-4">
<a class="menu__link" href="/about-us"><span>About Ghin</span></a>
</li>
<li class="order-5">
<a class="menu__link" href="/contact-us"><span>Contact Us</span></a>
</li>
<li class="order-6">
<a class="menu__link" target="_blank" href="http://www.usga.org/handicapping/ghin-faqs.html"><span>FAQs</span></a>
</li>
</ul>
<ul class="menu__links hide-on-tablet-and-above">
<li>
<a class="menu__link" href="/?logout"><span>Log Out</span></a>
</li>
</ul>
</div>
</div>
</div>
<span role="button" tabindex="0" class="usga_logo">
golf handicap <br />
information network
</span>
<div class="user-menu__container">
<button class="user-menu__trigger"><span>username</span><i class="material-icons-outlined more">expand_more</i><i class="material-icons-outlined less">expand_less</i></button>
<div class="user-menu__content">
<ul class="menu__links compact">
<li>
<a class="menu__link" href="/?logout"><span>Log Out</span></a>
</li>
</ul>
</div>
</div>
</div>
</header>
<nav class="navigation">
<div class="container">
<ul class="nav__links">
<li><a class="nav__link" href="/post-score">Post Score</a></li>
<li><a class="nav__link is-active" href="/golfer-lookup" aria-current="page">Golfer Lookup</a></li>
<li><a class="nav__link" href="/stats">Stats</a></li>
<li><a class="nav__link" href="/handicap-calculator">Handicap Calculator</a></li>
<li><a class="nav__link" href="/profile-handicap-history">Handicap History</a></li>
</ul>
</div>
</nav>
</div>
<div class="s-navigation-placeholder" style="margin-top: 0px;"></div>
<div class="container has-adds">
<section class="golfer_lookup_section">
<div id="div-gpt-ad-1539376483266-0" class="promo_google_banner"><div style="width: 728px; height: 90px;"></div></div>
<div class="section_header">
<h3>Golfer Lookup</h3>
<ul class="tabs rounded">
<li>Following</li>
<li>My Clubs</li>
<li>All Golfers</li>
</ul>
</div>
<div class="panel">
<div style="display: flex; justify-content: center; align-items: center; padding-top: 30px; padding-bottom: 15px;"><div class="loader"></div></div>
</div>
</section>
</div>
<footer>
<div class="footer__container">
<div class="information__container">
<div class="partners">
<h4 class="partners__title">Usga Partners</h4>
<div class="partners__list">
<a class="partner__item american-express" href="https://global.americanexpress.com/entertainment/home/" target="_blank" rel="noopener noreferrer">
<img alt="logo_american_express" class="partner__logo" src="/img/logos/logo_american_express.png" />
</a>
<a class="partner__item cisco" href="https://www.cisco.com/" target="_blank" rel="noopener noreferrer"><img alt="logo_cisco" class="partner__logo fit-size" src="/img/logos/logo_cisco.svg" /></a>
<a
class="partner__item delloite"
href="https://www2.deloitte.com/us/en/pages/consulting/articles/united-states-golf-association.html?id=us:2pm:3pt:lookagain:awa:cons:060116:usga"
target="_blank"
rel="noopener noreferrer"
>
<img alt="logo_delloite" class="partner__logo fit-size" src="/img/logos/logo_delloite.svg" />
</a>
<a class="partner__item lexus" href="https://www.lexus.com/" target="_blank" rel="noopener noreferrer"><img alt="logo_lexus" class="partner__logo lexus" src="/img/logos/logo_lexus.png" /></a>
<a class="partner__item rolex" href="https://www.rolex.com/world-of-rolex/golf.html" target="_blank" rel="noopener noreferrer">
<img alt="logo_rolex" class="partner__logo" src="/img/logos/logo_rolex.svg" />
</a>
</div>
</div>
<div class="mobile__app">
<h4 class="app__title">Ghin Mobile App</h4>
<div class="app__details">
<span class="app__logo"></span>
<ul class="app__stores">
<li><a class="store app-store" rel="noopener noreferrer" target="_blank" href="https://itunes.apple.com/us/app/ghin-mobile/id491796218?mt=8"> </a></li>
<li><a class="store google-play" rel="noopener noreferrer" target="_blank" href="https://play.google.com/store/apps/details?id=com.advancedmobile.android.ghin&hl=en_US"> </a></li>
</ul>
</div>
</div>
</div>
<ul class="footer__links">
<li><a class="footer__link" href="https://www.usga.org/content/usga/home-page/handicapping.html" target="_blank" rel="noopener noreferrer">WHS CHANGES</a></li>
<li><a class="footer__link" href="http://www.usga.org/handicapping/ghin-faqs.html" target="_blank" rel="noopener noreferrer">FAQs</a></li>
<li><a class="footer__link" href="/contact-us">CONTACT US</a></li>
<li><a class="footer__link" href="https://www.usga.org/content/usga/home-page/Handicap-ghin/ghin-privacy-policy.html" target="_blank" rel="noopener noreferrer">PRIVACY POLICY</a></li>
<li><a class="footer__link" href="https://www.usga.org/content/usga/home-page/Handicap-ghin/ghin-terms-of-service.html" target="_blank" rel="noopener noreferrer">TERMS OF SERVICE</a></li>
</ul>
<div class="copyright__container"><p>© 2020 United States Golf Association. All Rights Reserved.</p></div>
</div>
</footer>
<button id="back-top-button" class="hidden" title="Go to top"></button>
</div>
<script async="" src="https://www.google-analytics.com/analytics.js"></script>
<script>
!(function (e) {
function r(r) {
for (var n, i, l = r[0], f = r[1], a = r[2], c = 0, s = []; c < l.length; c++) (i = l[c]), Object.prototype.hasOwnProperty.call(o, i) && o[i] && s.push(o[i][0]), (o[i] = 0);
for (n in f) Object.prototype.hasOwnProperty.call(f, n) && (e[n] = f[n]);
for (p && p(r); s.length; ) s.shift()();
return u.push.apply(u, a || []), t();
}
function t() {
for (var e, r = 0; r < u.length; r++) {
for (var t = u[r], n = !0, l = 1; l < t.length; l++) {
var f = t[l];
0 !== o[f] && (n = !1);
}
n && (u.splice(r--, 1), (e = i((i.s = t[0]))));
}
return e;
}
var n = {},
o = { 1: 0 },
u = [];
function i(r) {
if (n[r]) return n[r].exports;
var t = (n[r] = { i: r, l: !1, exports: {} });
return e[r].call(t.exports, t, t.exports, i), (t.l = !0), t.exports;
}
(i.m = e),
(i.c = n),
(i.d = function (e, r, t) {
i.o(e, r) || Object.defineProperty(e, r, { enumerable: !0, get: t });
}),
(i.r = function (e) {
"undefined" != typeof Symbol && Symbol.toStringTag && Object.defineProperty(e, Symbol.toStringTag, { value: "Module" }), Object.defineProperty(e, "__esModule", { value: !0 });
}),
(i.t = function (e, r) {
if ((1 & r && (e = i(e)), 8 & r)) return e;
if (4 & r && "object" == typeof e && e && e.__esModule) return e;
var t = Object.create(null);
if ((i.r(t), Object.defineProperty(t, "default", { enumerable: !0, value: e }), 2 & r && "string" != typeof e))
for (var n in e)
i.d(
t,
n,
function (r) {
return e[r];
}.bind(null, n)
);
return t;
}),
(i.n = function (e) {
var r =
e && e.__esModule
? function () {
return e.default;
}
: function () {
return e;
};
return i.d(r, "a", r), r;
}),
(i.o = function (e, r) {
return Object.prototype.hasOwnProperty.call(e, r);
}),
(i.p = "/");
var l = (this.webpackJsonpghin = this.webpackJsonpghin || []),
f = l.push.bind(l);
(l.push = r), (l = l.slice());
for (var a = 0; a < l.length; a++) r(l[a]);
var p = f;
t();
})([]);
</script>
<script src="/static/js/2.ee61cc35.chunk.js"></script>
<script src="/static/js/main.7596f1a6.chunk.js"></script>
</body>
I need to access the HTML that is ultimately displayed by the webView (following execution of the JS code?). Is there a term for that specific part of the HTML rendering? I'm looking to pull user data from the page. It is displayed in the app (WKWebView), but is not present in the HTML saved using the above code.
I want to display only those pages in the header which have child pages inside them. Using the following code
<nav class="wsmenu clearfix" data-sly-use.nav="in.gov.odishatourism.core.models.HeaderNavigation">
<ul class="wsmenu-list" >
<li aria-haspopup="true" data-sly-repeat="${nav.root.listChildren}" ><span class="wsmenu-click" ><i class="wsmenu-arrow fa fa-angle-down"></i></span>
<a class="" href="javascript:void(0);" data-sly-test="!${item.hasChild}">${item.title}</a>
<div class="wsmegamenu clearfix ">
<div class="container paddingmenu">
<div class="row">
<div class="menuimgdiv" data-sly-repeat.subcategory="${item.listChildren}">
<h3 class="title" data-sly-test.subcat="${subcategory.title}" >${subcat}</h3>
<ul>
<li data-sly-repeat.page="${subcategory.listChildren}"><a class="" href="${page.path # extension='html'}">${page.title}</a></li>
</ul>
</div>
</div>
</div>
</div>
</li>
<li aria-haspopup="true"> Dept of Tourism</li>
</ul>
<a class="wsdownmenu-animated-arrow"><span></span></a>
<div class="wsdownmenu-text">Navigation</div>
</nav>
gives the following output
But this code -
<nav class="wsmenu clearfix" data-sly-use.nav="in.gov.odishatourism.core.models.HeaderNavigation">
<ul class="wsmenu-list" >
<li aria-haspopup="true" data-sly-repeat="${nav.root.listChildren}" ><span class="wsmenu-click" ><i class="wsmenu-arrow fa fa-angle-down"></i></span>
<a class="" href="javascript:void(0);" data-sly-test="${item.hasChild}">${item.title}</a>
<div class="wsmegamenu clearfix ">
<div class="container paddingmenu">
<div class="row">
<div class="menuimgdiv" data-sly-repeat.subcategory="${item.listChildren}">
<h3 class="title" data-sly-test.subcat="${subcategory.title}" >${subcat}</h3>
<ul>
<li data-sly-repeat.page="${subcategory.listChildren}"><a class="" href="${page.path # extension='html'}">${page.title}</a></li>
</ul>
</div>
</div>
</div>
</div>
</li>
<li aria-haspopup="true"> Dept of Tourism</li>
</ul>
<a class="wsdownmenu-animated-arrow"><span></span></a>
<div class="wsdownmenu-text">Navigation</div>
</nav>
gives the output as shown below
The code
<a class="" href="javascript:void(0);" data-sly-test="${item.hasChild}">${item.title}</a>
is not working properly.
My site structure is as shown below
And the actual desired output is
You cannot use ${item.hasChild} in your test condition as the hasChild() method of the Page API requires you to pass a parameter. AFAIK, HTL doesn't support invocation of parameterized methods.
Since there is no direct API available to check if a page has child pages or not, you may need to do the following to validate if the page has child pages
<a class="" href="javascript:void(0);" data-sly-test="${item.listChildren && item.listChildren.hasNext}">${item.title}</a>
However, I would prefer building the entire navigation tree using Sling models or WCM Use API and not invoking so many methods in the HTL. That would make the code easier to maintain, change and test. YMMV
I want to display names of every child pages of the parent of the current page except the current page in the component in aem. But my output is coming including the current page name.
<section id="touractivities">
<div class="container">
<div class="row">
<div class="col-sm-12">
<ul class="row buttontab">
<li class="col-md col-sm-4 col-6" data-sly-repeat="${CurrentPage.parent.listChildren}">
<div class="btnbggradient">
<a href="${item.path}.html" class="shadow">
<i class="icon-${item.name}"></i>
<h5 >${item.title}</h5>
</a>
</div>
</li>
</ul>
</div>
</div>
</div></section>
my aem structure:
required output:
output im getting:
The code is correct except that the bird watching li should not come while i am in bird watching page.
You can switch from data-sly-repeat to data-sly-list (so you can move it to the ul element). You can then use data-sly-test on the li element and check each item path against the current resource/page path:
<li data-sly-test=“${item.path != resource.path}” ...
<section id="touractivities">
<div class="container">
<div class="row">
<div class="col-sm-12">
<ul class="row buttontab" data-sly-list="${CurrentPage.parent.listChildren}">
<li class="col-md col-sm-4 col-6" data-sly-test="${item.path != currentPage.path}" >
<div class="btnbggradient">
<a href="${item.path}.html" class="shadow">
<i class="icon-${item.name}"></i>
<h5 >${item.title}</h5>
</a>
</div>
</li>
</ul>
</div>
</div>
</div></section>
I have the following structure HTML code
<ul id="list_abcxyz">
<li class="label even-row">
<a class="label-link lia-link-navigation" rel="tag" id="link_c19bf51ba9103c" href="test.html">Label One<wbr></a>
<span class="label-count">(1)</span>
</li>
<li class="label odd-row">
<a class="label-link lia-link-navigation" rel="tag" id="link_c19bf51ba9103c_0" href="test2.html">Label Two<wbr></a>
<span class="label-count">(1)</span>
</li>
<li class="label even-row">
<a class="label-link lia-link-navigation" rel="tag" id="link_c19bf51ba9103c_1" href="test3.html">Label Three<wbr></a>
<span class="label-count">(1)</span>
</li>
How to convert it to a select drop down? I have tried following the guides in community. But those didn't work. I guess because of extra span tag. But I need that span tag as well.
How can we convert it to select option drop down using JS ?
Thank you.
Something like this should get you started. If you were running into trouble with the <span> then you can use the textContent property to get all text without the html tags included, then remove all multiple spaces with a replace() and trim().
var myform = document.getElementById('myform'),
items = document.getElementById('list_abcxyz').getElementsByTagName('li'),
select = document.createElement('select'),
len = items.length,
option;
for(var i = 0; i < len; i++) {
option = document.createElement('option');
var label = items[i].textContent.replace(/\s\s+/g," ").trim(),
link = items[i].getElementsByTagName('a')[0].href;
option.textContent = label;
option.value = link;
select.appendChild(option);
}
myform.appendChild(select);
select.addEventListener('change',function(evt){
var selectedLink = this.options[this.selectedIndex].value;
console.log(selectedLink);
location.href = selectedLink;
});
<ul id="list_abcxyz">
<li class="label even-row">
<a class="label-link lia-link-navigation" rel="tag" id="link_c19bf51ba9103c" href="test.html">Label One<wbr></a>
<span class="label-count">(1)</span>
</li>
<li class="label odd-row">
<a class="label-link lia-link-navigation" rel="tag" id="link_c19bf51ba9103c_0" href="test2.html">Label Two<wbr></a>
<span class="label-count">(1)</span>
</li>
<li class="label even-row">
<a class="label-link lia-link-navigation" rel="tag" id="link_c19bf51ba9103c_1" href="test3.html">Label Three<wbr></a>
<span class="label-count">(1)</span>
</li>
</ul>
<form id="myform">
</form>
I am trying jsTree for the first time, and have the problem that after calling jstree(), the text in my <li> elements vanishes. After the call, my tree looks like this:
<div id="assessment-treeview" class="jstree jstree-0 jstree-focused jstree-classic">
<ul>
<li class="jstree-last jstree-open"><ins class="jstree-icon"> </ins>
<ul style="">
<li class="jstree-last jstree-open"><ins class="jstree-icon"> </ins>
<ul style="">
<li class="jstree-last jstree-leaf"><ins class="jstree-icon"> </ins></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
Before the call it looks like this:
<div id="assessment-treeview">
<ul>
<li>Specific Outcomes
<ul>
<li>[section: name not available for import]
<ul>
<li>[outcome: name not available for import] </li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
The jstree call looks like this:
$("#assessment-treeview").jstree({ "themes": { "theme": "classic" } });
What could be causing this?
jsTree doesn't like pure text nodes inside the 'li' tag. When I put the item text into a tag like 'span', everything worked fine again.
From the question code above:
<li><span>Specific Outcomes</span>
<ul>
.
.
</ul>
</li>