Finding html element with class using lxml - class

Question

I've searched everywhere and what I most found was doc.xpath('//element[#class="classname"]'), but this does not work no matter what I try.
code I'm using
import lxml.html
def check():
data = urlopen('url').read();
return str(data);
doc = lxml.html.document_fromstring(check())
el = doc.xpath("//div[#class='test']")
print(el)
It simply prints an empty list.
Edit:
How odd. I used google as a test page and it works fine there, but it doesn't work on the page I was using (youtube)
Here's the exact code I'm using.
import lxml.html
from urllib.request import urlopen
import sys
def check():
data = urlopen('http://www.youtube.com/user/TopGear').read(); #TopGear as a test
return data.decode('utf-8', 'ignore');
doc = lxml.html.document_fromstring(check())
el = doc.xpath("//div[#class='channel']")
print(el)

Answer 1

The TopGear page that you use for testing doesn't have any <div class="channel"> elements. But this works (for example):
el = doc.xpath("//div[#class='channel-title-container']")
Or this:
el = doc.xpath("//div[#class='a yb xr']")
To find <div> elements with a class attribute that contains the string channel, you could use
el = doc.xpath("//div[contains(#class, 'channel')]")

Answer 2

You can use lxml.cssselect to simplify class and id request: http://lxml.de/dev/cssselect.html

Answer 3

HTML uses classes (a lot), which makes them convenient to hook XPath queries. However XPath has no knowledge/support of CSS classes (or even space-separated lists) which makes classes a pain in the ass to check: the canonically correct way to look for elements having a specific class is:
//*[contains(concat(' ', normalize-space(#class), ' '), '$className')]
In your case this is
el = doc.xpath(
"//div[contains(concat(' ', normalize-space(#class), ' '), 'channel')]"
)
# print(el)
# [<Element div at 0x7fa44e31ccc8>, <Element div at 0x7fa44e31c278>, <Element div at 0x7fa44e31cdb8>]
or use own XPath function hasclass(*classes)
def _hasaclass(context, *cls):
return "your implementation ..."
xpath_utils = etree.FunctionNamespace(None)
xpath_utils['hasaclass'] = _hasaclass
el = doc.xpath("//div[hasaclass('channel')]")