Details
-
Task
-
Resolution: Done
-
Major
-
3.8.1.Final, 3.8.x-prod-ip6.1
-
None
Description
Lucene memory usage should be investigated when indexing nodes that have binary properties full text search is enabled and text extraction is not enabled. In this particular case, there is no reason for ModeShape to attempt reading any binary content since the only meaningful (and required by the JCR spec) information is the length of the property.
If text extraction is enabled, then the binary content will have to be read by the text extractor(s) and there is no way around this
If OOM exceptions in the form of:
0:00:43,906 ERROR [org.hibernate.search.exception.impl.LogErrorHandler] (Hibernate Search sync consumer thread for index nodeinfo) HSEARCH000058: Exception occurred java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit Primary Failure: Entity org.modeshape.jcr.query.lucene.basic.NodeInfo Id d7d2e6b7505d64aabe99db-872d-4a93-b68d-8ae175a3a971 Work Type org.hibernate.search.backend.AddLuceneWork : java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:3358) [lucene-core-3.6.2.redhat-8.jar:3.6.2.redhat-8] at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3485) [lucene-core-3.6.2.redhat-8.jar:3.6.2.redhat-8] at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3467) [lucene-core-3.6.2.redhat-8.jar:3.6.2.redhat-8] at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3451) [lucene-core-3.6.2.redhat-8.jar:3.6.2.redhat-8] at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.commitIndexWriter(IndexWriterHolder.java:158) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.commitIndexWriter(IndexWriterHolder.java:171) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.PerChangeSetCommitPolicy.onChangeSetApplied(PerChangeSetCommitPolicy.java:46) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.afterTransactionApplied(AbstractWorkspaceImpl.java:107) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:124) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:67) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:162) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:152) [hibernate-search-engine-4.6.0.Final-redhat-2.jar:4.6.0.Final-redhat-2] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_75]
occur, there are a couple of workarounds that can be used:
1. make the node which holds the jcr:data property not queryable. For example:
[nt:nqresource] > nt:resource noquery
2. disable full search search from the JSON configuration:
"query" : { "enabled" : true, "enableFullTextSearch" : false, }
