Lucene search as default search?

7 messages Options
Embed this post
Permalink
vmassol

Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
Hi,

Should we make the lucene search the default search or not?
In the default XAR the Lucene search is marked as experimental. This  
has been the case for a long time now.

I think we need to do something about this. Do you think the Lucene  
search is working well enough to be set as the default?

WDYT?

Thanks
-Vincent
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Denis Gervalle

Re: Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
Hi Vincent,

22 opens issues on the lucene plugins seems to me a lot to put it as a  
default.
Of course, I'd really like to see most of them solved, but some seems  
to me blocker issue to put the lucene plugins as the default:
XPLUCENE-5
XPLUCENE-8
XPLUCENE-13
XPLUCENE-30
XPLUCENE-33
XPLUCENE-34
XPLUCENE-35
XPLUCENE-37
XPLUCENE-40

If search seems powerful (which the XWIki search is not), but is  
unreliable, I think that you will disappoint more than a less powerful  
but reliable solution (which the XWiki search is).

WDYT ?

Denis

On Oct 26, 2009, at 21:00, Vincent Massol wrote:

> Hi,
>
> Should we make the lucene search the default search or not?
> In the default XAR the Lucene search is marked as experimental. This
> has been the case for a long time now.
>
> I think we need to do something about this. Do you think the Lucene
> search is working well enough to be set as the default?
>
> WDYT?
>
> Thanks
> -Vincent
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Guillaume Lerouge

Re: Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
Hi,

On Tue, Oct 27, 2009 at 6:33 PM, Denis Gervalle <[hidden email]> wrote:

> Hi Vincent,
>
> 22 opens issues on the lucene plugins seems to me a lot to put it as a
> default.
> Of course, I'd really like to see most of them solved, but some seems
> to me blocker issue to put the lucene plugins as the default:
> XPLUCENE-5
> XPLUCENE-8
> XPLUCENE-13
> XPLUCENE-30
> XPLUCENE-33
> XPLUCENE-34
> XPLUCENE-35
> XPLUCENE-37
> XPLUCENE-40
>
> If search seems powerful (which the XWIki search is not), but is
> unreliable, I think that you will disappoint more than a less powerful
> but reliable solution (which the XWiki search is).
>

I agree with Denis. We need to fix important bugs that make it unstable
before putting it as the default search.

I'd really like if a developer could spend some time working on the Lucene
Plugin in the course of the 2.1 roadmap but so far it looks like we'll all
be busy on other things...

Guillaume

WDYT ?

>
> Denis
>
> On Oct 26, 2009, at 21:00, Vincent Massol wrote:
>
> > Hi,
> >
> > Should we make the lucene search the default search or not?
> > In the default XAR the Lucene search is marked as experimental. This
> > has been the case for a long time now.
> >
> > I think we need to do something about this. Do you think the Lucene
> > search is working well enough to be set as the default?
> >
> > WDYT?
> >
> > Thanks
> > -Vincent
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Guillaume Lerouge
Product Manager - XWiki SAS
Skype: wikibc
Twitter: glerouge
http://guillaumelerouge.com/
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Pascal Voitot

Re: Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
In reply to this post by Denis Gervalle
Hello,

I already use it as default (since 1.9.x) as I have a solution with too many
data (mainly attached documents) to index and needed a more powerful
indexation engine. Lucene works quite good for me but you are right, those
open issues might be ennoying... I'm not expert enough to choose...

Pascal

On Tue, Oct 27, 2009 at 6:33 PM, Denis Gervalle <[hidden email]> wrote:

> Hi Vincent,
>
> 22 opens issues on the lucene plugins seems to me a lot to put it as a
> default.
> Of course, I'd really like to see most of them solved, but some seems
> to me blocker issue to put the lucene plugins as the default:
> XPLUCENE-5
> XPLUCENE-8
> XPLUCENE-13
> XPLUCENE-30
> XPLUCENE-33
> XPLUCENE-34
> XPLUCENE-35
> XPLUCENE-37
> XPLUCENE-40
>
> If search seems powerful (which the XWIki search is not), but is
> unreliable, I think that you will disappoint more than a less powerful
> but reliable solution (which the XWiki search is).
>
> WDYT ?
>
> Denis
>
> On Oct 26, 2009, at 21:00, Vincent Massol wrote:
>
> > Hi,
> >
> > Should we make the lucene search the default search or not?
> > In the default XAR the Lucene search is marked as experimental. This
> > has been the case for a long time now.
> >
> > I think we need to do something about this. Do you think the Lucene
> > search is working well enough to be set as the default?
> >
> > WDYT?
> >
> > Thanks
> > -Vincent
> > _______________________________________________
> > devs mailing list
> > [hidden email]
> > http://lists.xwiki.org/mailman/listinfo/devs
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
vmassol

Re: Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
In reply to this post by Denis Gervalle

On Oct 27, 2009, at 6:33 PM, Denis Gervalle wrote:

> Hi Vincent,
>
> 22 opens issues on the lucene plugins seems to me a lot to put it as a
> default.

We have over 50 for the wysiwyg editor, but it doesn't mean it  
shouldn't be used as the default ;)

> Of course, I'd really like to see most of them solved, but some seems
> to me blocker issue to put the lucene plugins as the default:
> XPLUCENE-5
> XPLUCENE-8
> XPLUCENE-13
> XPLUCENE-30
> XPLUCENE-33
> XPLUCENE-34
> XPLUCENE-35
> XPLUCENE-37
> XPLUCENE-40
>
> If search seems powerful (which the XWIki search is not), but is
> unreliable, I think that you will disappoint more than a less powerful
> but reliable solution (which the XWiki search is).
>
> WDYT ?

Yes I agree. Let's postpone till we fix some of these then.

Thanks
-Vincent

>
> Denis
>
> On Oct 26, 2009, at 21:00, Vincent Massol wrote:
>
>> Hi,
>>
>> Should we make the lucene search the default search or not?
>> In the default XAR the Lucene search is marked as experimental. This
>> has been the case for a long time now.
>>
>> I think we need to do something about this. Do you think the Lucene
>> search is working well enough to be set as the default?
>>
>> WDYT?
>>
>> Thanks
>> -Vincent
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Andreas Jonsson

Re: Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
In reply to this post by Guillaume Lerouge
Hi,

I've did some work on the Lucene plugin that should fix XPLUCENE-5,
XPLUCENE-38 and XPLUCENE-40 and posted a patch here:
http://jira.xwiki.org/jira/browse/XPLUCENE-5.  There are more things
that needs to be done.  I have some comments and questions:

* Why are there special "index" classes
  (IndexData/DocumentData/ObjectData/AttachmentData) that are stored
  in the queue?  Why not just store XWikiDocuments and (or
  DocumentModelBridge) and XWikiAttachments directly?  This would seem
  to save memory (the documents must be loaded to be indexed anyway)
  and reduce the load on the database (when the documents are already
  loaded).  Maybe the idea was that by looking up the document as late
  as possible we will get a fresh version of it, and duplicated
  requests for updating a document are filtered out in the queue.  But
  to me this seem to be a rare situation, and the way this was
  implemented, there was a race condition so the document was likely
  to be added twice anyway.

* There are some code to support multiple indices, but it doesn't seem
  to be used and I doubt that it works.

* The code contains lots of "catch (Exception e)" clauses.

* Also, how threads are synchronized should be thought of more
  carefully.  It seems that the keyword "synchronized" have been
  thrown in at various places "just in case".

* To complete the upgrade to Lucene 2.9.0 the deprecated methods and
  classes should be replaced.  For instance the now deprecated Hits
  class should be replaced with TopDocuments.

* The methods IndexWriter::isLocked and IndexWriter::unlock in lucene
  seems unreliable.  This is an obstacle for XPLUCENE-33.

Best regards,

Andreas Jonsson


Guillaume Lerouge wrote:

> Hi,
>
> On Tue, Oct 27, 2009 at 6:33 PM, Denis Gervalle <[hidden email]> wrote:
>
>  
>> Hi Vincent,
>>
>> 22 opens issues on the lucene plugins seems to me a lot to put it as a
>> default.
>> Of course, I'd really like to see most of them solved, but some seems
>> to me blocker issue to put the lucene plugins as the default:
>> XPLUCENE-5
>> XPLUCENE-8
>> XPLUCENE-13
>> XPLUCENE-30
>> XPLUCENE-33
>> XPLUCENE-34
>> XPLUCENE-35
>> XPLUCENE-37
>> XPLUCENE-40
>>
>> If search seems powerful (which the XWIki search is not), but is
>> unreliable, I think that you will disappoint more than a less powerful
>> but reliable solution (which the XWiki search is).
>>
>>    
>
> I agree with Denis. We need to fix important bugs that make it unstable
> before putting it as the default search.
>
> I'd really like if a developer could spend some time working on the Lucene
> Plugin in the course of the 2.1 roadmap but so far it looks like we'll all
> be busy on other things...
>
> Guillaume
>
> WDYT ?
>  
>> Denis
>>
>> On Oct 26, 2009, at 21:00, Vincent Massol wrote:
>>
>>    
>>> Hi,
>>>
>>> Should we make the lucene search the default search or not?
>>> In the default XAR the Lucene search is marked as experimental. This
>>> has been the case for a long time now.
>>>
>>> I think we need to do something about this. Do you think the Lucene
>>> search is working well enough to be set as the default?
>>>
>>> WDYT?
>>>
>>> Thanks
>>> -Vincent
>>> _______________________________________________
>>> devs mailing list
>>> [hidden email]
>>> http://lists.xwiki.org/mailman/listinfo/devs
>>>      
>> _______________________________________________
>> devs mailing list
>> [hidden email]
>> http://lists.xwiki.org/mailman/listinfo/devs
>>
>>    
>
>
>
>  

_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs
Guillaume Lerouge

Re: Lucene search as default search?

Reply Threaded More More options
Print post
Permalink
Hi Andreas,

On Mon, Nov 9, 2009 at 3:00 PM, Andreas Jonsson <[hidden email]> wrote:

> Hi,
>
> I've did some work on the Lucene plugin that should fix XPLUCENE-5,
> XPLUCENE-38 and XPLUCENE-40 and posted a patch here:
> http://jira.xwiki.org/jira/browse/XPLUCENE-5.


Thanks for your contribution! The Lucene plugin definitely needs some love
and your help is great.

Guillaume


> There are more things
> that needs to be done.  I have some comments and questions:
>
> * Why are there special "index" classes
>  (IndexData/DocumentData/ObjectData/AttachmentData) that are stored
>  in the queue?  Why not just store XWikiDocuments and (or
>  DocumentModelBridge) and XWikiAttachments directly?  This would seem
>  to save memory (the documents must be loaded to be indexed anyway)
>  and reduce the load on the database (when the documents are already
>  loaded).  Maybe the idea was that by looking up the document as late
>  as possible we will get a fresh version of it, and duplicated
>  requests for updating a document are filtered out in the queue.  But
>  to me this seem to be a rare situation, and the way this was
>  implemented, there was a race condition so the document was likely
>  to be added twice anyway.
>
> * There are some code to support multiple indices, but it doesn't seem
>  to be used and I doubt that it works.
>
> * The code contains lots of "catch (Exception e)" clauses.
>
> * Also, how threads are synchronized should be thought of more
>  carefully.  It seems that the keyword "synchronized" have been
>  thrown in at various places "just in case".
>
> * To complete the upgrade to Lucene 2.9.0 the deprecated methods and
>  classes should be replaced.  For instance the now deprecated Hits
>  class should be replaced with TopDocuments.
>
> * The methods IndexWriter::isLocked and IndexWriter::unlock in lucene
>  seems unreliable.  This is an obstacle for XPLUCENE-33.
>
> Best regards,
>
> Andreas Jonsson
>
>
> Guillaume Lerouge wrote:
> > Hi,
> >
> > On Tue, Oct 27, 2009 at 6:33 PM, Denis Gervalle <[hidden email]> wrote:
> >
> >
> >> Hi Vincent,
> >>
> >> 22 opens issues on the lucene plugins seems to me a lot to put it as a
> >> default.
> >> Of course, I'd really like to see most of them solved, but some seems
> >> to me blocker issue to put the lucene plugins as the default:
> >> XPLUCENE-5
> >> XPLUCENE-8
> >> XPLUCENE-13
> >> XPLUCENE-30
> >> XPLUCENE-33
> >> XPLUCENE-34
> >> XPLUCENE-35
> >> XPLUCENE-37
> >> XPLUCENE-40
> >>
> >> If search seems powerful (which the XWIki search is not), but is
> >> unreliable, I think that you will disappoint more than a less powerful
> >> but reliable solution (which the XWiki search is).
> >>
> >>
> >
> > I agree with Denis. We need to fix important bugs that make it unstable
> > before putting it as the default search.
> >
> > I'd really like if a developer could spend some time working on the
> Lucene
> > Plugin in the course of the 2.1 roadmap but so far it looks like we'll
> all
> > be busy on other things...
> >
> > Guillaume
> >
> > WDYT ?
> >
> >> Denis
> >>
> >> On Oct 26, 2009, at 21:00, Vincent Massol wrote:
> >>
> >>
> >>> Hi,
> >>>
> >>> Should we make the lucene search the default search or not?
> >>> In the default XAR the Lucene search is marked as experimental. This
> >>> has been the case for a long time now.
> >>>
> >>> I think we need to do something about this. Do you think the Lucene
> >>> search is working well enough to be set as the default?
> >>>
> >>> WDYT?
> >>>
> >>> Thanks
> >>> -Vincent
> >>> _______________________________________________
> >>> devs mailing list
> >>> [hidden email]
> >>> http://lists.xwiki.org/mailman/listinfo/devs
> >>>
> >> _______________________________________________
> >> devs mailing list
> >> [hidden email]
> >> http://lists.xwiki.org/mailman/listinfo/devs
> >>
> >>
> >
> >
> >
> >
>
> _______________________________________________
> devs mailing list
> [hidden email]
> http://lists.xwiki.org/mailman/listinfo/devs
>



--
Guillaume Lerouge
Product Manager - XWiki SAS
Skype: wikibc
Twitter: glerouge
http://guillaumelerouge.com/
_______________________________________________
devs mailing list
[hidden email]
http://lists.xwiki.org/mailman/listinfo/devs