How to tokenize a text in lucene 3.0.3 - lucene.net

Question

I need to translate this code from lucene.net 2.3.2 to 3.0.3.
This code works fine in 2.3.2 but in 3.0.3 the method tokenStream.Next() does not return a token but a boolean. What I need to understand is where to read the token object when next() returns true.
Analyzer analyzer = new StandardAnalyzer();
StringReader stringReader = new StringReader("the house id red");
TokenStream tokenStream = analyzer.TokenStream("", stringReader);
Token token = tokenStream.Next();
while (token != null)
{
System.Diagnostics.Debug.Write(token.TermText());
token = tokenStream.Next();
}

Answer 1

The TokenStreams in Lucene 3+ can now represent a whole set of values (called attributes) for each position in the stream. So you need to modify your example to be attribute aware...
var analyzer = new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30);
var stringReader = new StringReader("the house id red");
var tokenStream = analyzer.TokenStream(string.Empty, stringReader);
var termAtt = tokenStream.GetAttribute<ITermAttribute>();
while (tokenStream.IncrementToken())
{
Console.WriteLine(termAtt.Term);
}