Lucene.NET lifetime management - lucene.net

Question

Let's assume that I have a basic understanding of adding and searching documents.
What would be the best practice for managing instances of IndexWriter and IndexReader?
Currently, my application creates a singleton instance of an IndexWriter. When ever I need to do a search, I just create an IndexSearcher from the IndexWriter by using the following
var searcher = new IndexSearcher(writer.GetReader())
I am doing this because creating a new IndexReader causes the index to get loaded into memory, and then waits for the GC to reallocate the memory. This was causing out of memory errors.
Is this current implementation considered ideal? This implementation has solved the memory issue, but there is an issue with the write.lock file always existing (because IndexWriter is always instantied and opened). Here is the stack trace of the errors I get in the app.
Lock obtain timed out:
NativeFSLock#C:\inetpub\wwwroot\htdocs_beta\App_Data\products3\write.lock:
System.IO.IOException: The process
cannot access the file
'C:\inetpub\wwwroot\htdocs_beta\App_Data\products3\write.lock'
because it is being used by another
process. at
System.IO.__Error.WinIOError(Int32
errorCode, String maybeFullPath) at
System.IO.FileStream.Init(String path,
FileMode mode, FileAccess access,
Int32 rights, Boolean useRights,
FileShare share, Int32 bufferSize,
FileOptions options,
SECURITY_ATTRIBUTES secAttrs, String
msgPath, Boolean bFromProxy, Boolean
useLongPath) at
System.IO.FileStream..ctor(String
path, FileMode mode, FileAccess
access) at
Lucene.Net.Store.NativeFSLock.Obtain()
I'm thinking maybe it would be best to create a singleton instance of IndexSearcher for searching, and then create an IndexWriter as needed in memory. That way, the write.lock file will be created/deleted when updating the index. The only issue I see with this is that the IndexSearcher instance will become outdated, I would need to have a task running that reloads the IndexSearcher if the index has been updates.
What do you think?
How do you handle a large index with live updating?

Answer 1

You should use only one index writer, to avoid your locking issues. Have a look at : Lucene.Net writing/reading synchronization