Why mutable_properties is a IUserEnumerationPlugin?

7 messages Options
Embed this post
Permalink
Vincent Fretin

Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
Hello,

The optimization fix introduced in Products.PlonePAS 3.10 (Plone
3.3.1) broke completely my products.
The changelog entry is the following:
- Performance fix for searching in the mutable properties plugin:
  when only searching on user id do not walk over all properties,
  but only test if the user id is known. This fixes
  http://dev.plone.org/old/plone/ticket/9361
  [toutpt]

The thing is that when you try to do
portal_membership.getMemberById("agroupid")
which iterate on all IUserEnumerationPlugin (source_users and
mutable_properties)
source_users return nothing, that's fine
mutable_properties returns the properties of the group! We don't want
that! It should return nothing like in the PlonePAS 3.9.

But there is another thing that bother me, in PlonePAS 3.9 the
enumerationUsers function in plugins/property.py (mutable_properties)
will always return nothing, because
data dictionary doesn't contain an id key, so testMemberData will
return False right away for each principal in the storage.

So I'm asking, why mutable_properties is a IUserEnumerationPlugin? I
completely deactivated it on my Plone site right now to fix the
problem.
But if this enumerateUsers function returns always nothing, we should
get rid of it.

Please tell me if I'm missed something.

Vincent Fretin
Ecreall
Site : http://vincentfretin.ecreall.com

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Gilles Lenfant

Re: Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)
Le 5 oct. 09 à 12:10, Vincent Fretin a écrit :

Hello,


[...]

So I'm asking, why mutable_properties is a IUserEnumerationPlugin? I
completely deactivated it on my Plone site right now to fix the
problem.
But if this enumerateUsers function returns always nothing, we should
get rid of it.

Please tell me if I'm missed something.

Perhaps ;)

You may search users based on their properties. The "enumerateUsers" docstring in "IUserEnumerationPlugin" says :

"""
...
        o If 'exact_match' is False, then 'id' and / or login may be
          treated by the plugin as "contains" searches (more complicated
          searches may be supported by some plugins using other keyword
          arguments).
...
"""

And that's what the ZODBMutablePropertyProvider does, searching in fullname and email attribute.

Anyway, there's still an issue :


Cheers
-- 
Gilles Lenfant


Vincent Fretin
Ecreall
Site : http://vincentfretin.ecreall.com

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers


------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Wichert Akkerman

Re: Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
In reply to this post by Vincent Fretin
On 10/5/09 12:10 , Vincent Fretin wrote:
> So I'm asking, why mutable_properties is a IUserEnumerationPlugin? I
> completely deactivated it on my Plone site right now to fix the
> problem.

So you can search for users by properties.

Wichert.

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
In reply to this post by Vincent Fretin
Vincent Fretin wrote:
> Hello,
>
> The optimization fix introduced in Products.PlonePAS 3.10 (Plone
> 3.3.1) broke completely my products.

That is a problem, probably enough to revert that fix, unless your
products were doing something insane.

Can you elaborate on what your product does?

> The changelog entry is the following:
> - Performance fix for searching in the mutable properties plugin:
>   when only searching on user id do not walk over all properties,
>   but only test if the user id is known. This fixes
>   http://dev.plone.org/old/plone/ticket/9361

s/old//

>   [toutpt]

That bug seems important to fix, though, if the benchmarks are to be
believed!

> The thing is that when you try to do
> portal_membership.getMemberById("agroupid")

Is this really kosher? I think you should use portal_groups instead, or
just acl_users.getGroupById(). I'm not sure getMemberById() is meant to
work on group principals.

I'd guess that the results of "undefined". What actually calls this, and
why should it return None?

> which iterate on all IUserEnumerationPlugin (source_users and
> mutable_properties)
> source_users return nothing, that's fine
> mutable_properties returns the properties of the group! We don't want
> that! It should return nothing like in the PlonePAS 3.9.

Mmm.... why? If there are properties, shouldn't we get the results?

Bear in mind that at this point, you're deep enough into PAS to be
dealing in "principals", not "users" and "groups" as separate entities.

> But there is another thing that bother me, in PlonePAS 3.9 the
> enumerationUsers function in plugins/property.py (mutable_properties)
> will always return nothing, because
> data dictionary doesn't contain an id key, so testMemberData will
> return False right away for each principal in the storage.

Always returning None definitely sounds like a bug. :)

> So I'm asking, why mutable_properties is a IUserEnumerationPlugin? I
> completely deactivated it on my Plone site right now to fix the
> problem.

That's been answered: so that you can search by properties. It sounds
like it's buggy, though.

> But if this enumerateUsers function returns always nothing, we should
> get rid of it.

It should return something, obviously. I'm not sure if getMemberById()
should return groups, though. My guess is it shouldn't, so maybe we need
to change the way it's implemented?

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Vincent Fretin

Re: Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
Thank you for your replies.

On Mon, Oct 5, 2009 at 1:15 PM, Martin Aspeli <[hidden email]> wrote:

> Vincent Fretin wrote:
>> Hello,
>>
>> The optimization fix introduced in Products.PlonePAS 3.10 (Plone
>> 3.3.1) broke completely my products.
>
> That is a problem, probably enough to revert that fix, unless your
> products were doing something insane.
>
> Can you elaborate on what your product does?

Well I use context.get_local_roles() to retrieve a list of principals
(it can be a username or groupname)
then I need to retrieve properties from the user or group, so I use
getToolByName(self, 'portal_membership').getMemberById
and if this return None, then I try
getGroupById = getToolByName(self, 'portal_groups').getGroupById


>
>> The changelog entry is the following:
>> - Performance fix for searching in the mutable properties plugin:
>>   when only searching on user id do not walk over all properties,
>>   but only test if the user id is known. This fixes
>>   http://dev.plone.org/old/plone/ticket/9361
>
> s/old//

I'll modify it.

>
>>   [toutpt]
>
> That bug seems important to fix, though, if the benchmarks are to be
> believed!
>
>> The thing is that when you try to do
>> portal_membership.getMemberById("agroupid")
>
> Is this really kosher? I think you should use portal_groups instead, or
> just acl_users.getGroupById(). I'm not sure getMemberById() is meant to
> work on group principals.
I know. See the above note of my use case.


> I'd guess that the results of "undefined". What actually calls this, and
> why should it return None?
I think it should return empty tuple because we are in a User
enumeration plugin and not a Group Enumeration plugin.

>
>> which iterate on all IUserEnumerationPlugin (source_users and
>> mutable_properties)
>> source_users return nothing, that's fine
>> mutable_properties returns the properties of the group! We don't want
>> that! It should return nothing like in the PlonePAS 3.9.
>
> Mmm.... why? If there are properties, shouldn't we get the results?
Well you're right, this plugin should return properties for user or
group matching criteria from kw parameter, like before.
And return () if you do a search by id or login, like before.

>
> Bear in mind that at this point, you're deep enough into PAS to be
> dealing in "principals", not "users" and "groups" as separate entities.
Well, you have a differentiation in PAS, you have User and Group
Enumeration plugin.
but mutable_properties is special, the storage contains users and
groups properties.

>
>> But there is another thing that bother me, in PlonePAS 3.9 the
>> enumerationUsers function in plugins/property.py (mutable_properties)
>> will always return nothing, because
>> data dictionary doesn't contain an id key, so testMemberData will
>> return False right away for each principal in the storage.
>
> Always returning None definitely sounds like a bug. :)

Actually this plugin is only useful in Plone user search form. If you
search 'vincent' in the form, you get the following parameters:
id = None
login = None
exact_match = False
kw = {'fullname': 'vincent'}

Here the plugin will return something because id and login are None.
Having id and login in the criteria doesn't make sense because you
don't have thoses keys in the data dictionary and so testMemberData
will always return False.
I propose to revert the optimization fix in PlonePAS 3.10 and do the
following optimization fix:


--- /tmp/products.plonepas-3.9/Products/PlonePAS/plugins/property.py 2009-10-05
11:04:42.346879367 +0200
+++ plugins/property.py 2009-10-05 13:40:47.923638276 +0200
@@ -252,11 +252,10 @@
         """
         plugin_id = self.getId()

+        if not kw:
+            return ()
+
         criteria=copy.copy(kw)
-        if id is not None:
-            criteria["id"]=id
-        if login is not None:
-            criteria["login"]=login

         users=[ (user,data) for (user,data) in self._storage.items()
                     if self.testMemberData(data, criteria, exact_match)]

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Martin Aspeli

Re: Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
Vincent Fretin wrote:

> Well I use context.get_local_roles() to retrieve a list of principals
> (it can be a username or groupname)
> then I need to retrieve properties from the user or group, so I use
> getToolByName(self, 'portal_membership').getMemberById
> and if this return None, then I try
> getGroupById = getToolByName(self, 'portal_groups').getGroupById

Right. This sounds reasonable. I think we should have tests that provide
getMemberById() always returns None for a group id. But I also think we
need to fix that at the CMF/Plone layer if it breaks, not by making the
more generic mutable_properties only work for members.

>> I'd guess that the results of "undefined". What actually calls this, and
>> why should it return None?
> I think it should return empty tuple because we are in a User
> enumeration plugin and not a Group Enumeration plugin.

Mmm... maybe. But then presumably mutable_properties would also need to
be an IGroupEnumeration plugin?

>>> which iterate on all IUserEnumerationPlugin (source_users and
>>> mutable_properties)
>>> source_users return nothing, that's fine
>>> mutable_properties returns the properties of the group! We don't want
>>> that! It should return nothing like in the PlonePAS 3.9.
>> Mmm.... why? If there are properties, shouldn't we get the results?
> Well you're right, this plugin should return properties for user or
> group matching criteria from kw parameter, like before.
> And return () if you do a search by id or login, like before.

It feels weird if it's to work on groups sometimes but not others, though.

>> Bear in mind that at this point, you're deep enough into PAS to be
>> dealing in "principals", not "users" and "groups" as separate entities.
> Well, you have a differentiation in PAS, you have User and Group
> Enumeration plugin.
> but mutable_properties is special, the storage contains users and
> groups properties.

True. So maybe the user enumeration API needs to return users only, and
the group enumeration UI needs to return groups only? I don't know the
full consequences of that, though. I think Wichert needs to chime in. :)

>>> But there is another thing that bother me, in PlonePAS 3.9 the
>>> enumerationUsers function in plugins/property.py (mutable_properties)
>>> will always return nothing, because
>>> data dictionary doesn't contain an id key, so testMemberData will
>>> return False right away for each principal in the storage.
>> Always returning None definitely sounds like a bug. :)
>
> Actually this plugin is only useful in Plone user search form. If you
> search 'vincent' in the form, you get the following parameters:
> id = None
> login = None
> exact_match = False
> kw = {'fullname': 'vincent'}
>
> Here the plugin will return something because id and login are None.
> Having id and login in the criteria doesn't make sense because you
> don't have thoses keys in the data dictionary and so testMemberData
> will always return False.
> I propose to revert the optimization fix in PlonePAS 3.10 and do the
> following optimization fix:
>
>
> --- /tmp/products.plonepas-3.9/Products/PlonePAS/plugins/property.py 2009-10-05
> 11:04:42.346879367 +0200
> +++ plugins/property.py 2009-10-05 13:40:47.923638276 +0200
> @@ -252,11 +252,10 @@
>          """
>          plugin_id = self.getId()
>
> +        if not kw:
> +            return ()
> +
>          criteria=copy.copy(kw)
> -        if id is not None:
> -            criteria["id"]=id
> -        if login is not None:
> -            criteria["login"]=login
>
>          users=[ (user,data) for (user,data) in self._storage.items()
>                      if self.testMemberData(data, criteria, exact_match)]

I don't know the code well enough to know if that's a valid fix,
although it does look like a very different type of "fix".

Wichert? :)

Martin


--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers
Vincent Fretin

Re: Why mutable_properties is a IUserEnumerationPlugin?

Reply Threaded More More options
Print post
Permalink
On Mon, Oct 5, 2009 at 3:06 PM, Martin Aspeli <[hidden email]> wrote:
>>> Bear in mind that at this point, you're deep enough into PAS to be
>>> dealing in "principals", not "users" and "groups" as separate entities.
>> Well, you have a differentiation in PAS, you have User and Group
>> Enumeration plugin.
>> but mutable_properties is special, the storage contains users and
>> groups properties.
>
> True. So maybe the user enumeration API needs to return users only, and
> the group enumeration UI needs to return groups only?
It's already separated for the source_users (IUserEnumerationPlugin)
and source_groups (IGroupEnumerationPlugin).
mutable_properties IUserEnumerationPlugin is only used in user search
for fullname or email as far as I know. You won't have groups in the
result because group don't have a fullname key.


> I don't know the
> full consequences of that, though. I think Wichert needs to chime in. :)
>
>>>> But there is another thing that bother me, in PlonePAS 3.9 the
>>>> enumerationUsers function in plugins/property.py (mutable_properties)
>>>> will always return nothing, because
>>>> data dictionary doesn't contain an id key, so testMemberData will
>>>> return False right away for each principal in the storage.
>>> Always returning None definitely sounds like a bug. :)
>>
>> Actually this plugin is only useful in Plone user search form. If you
>> search 'vincent' in the form, you get the following parameters:
>> id = None
>> login = None
>> exact_match = False
>> kw = {'fullname': 'vincent'}
>>
>> Here the plugin will return something because id and login are None.
>> Having id and login in the criteria doesn't make sense because you
>> don't have thoses keys in the data dictionary and so testMemberData
>> will always return False.
>> I propose to revert the optimization fix in PlonePAS 3.10 and do the
>> following optimization fix:
>>
>>
>> --- /tmp/products.plonepas-3.9/Products/PlonePAS/plugins/property.py  2009-10-05
>> 11:04:42.346879367 +0200
>> +++ plugins/property.py       2009-10-05 13:40:47.923638276 +0200
>> @@ -252,11 +252,10 @@
>>          """
>>          plugin_id = self.getId()
>>
>> +        if not kw:
>> +            return ()
>> +
>>          criteria=copy.copy(kw)
>> -        if id is not None:
>> -            criteria["id"]=id
>> -        if login is not None:
>> -            criteria["login"]=login
>>
>>          users=[ (user,data) for (user,data) in self._storage.items()
>>                      if self.testMemberData(data, criteria, exact_match)]
>
> I don't know the code well enough to know if that's a valid fix,
> although it does look like a very different type of "fix".

I optimized it, instead of looping to all principals to return finally
(), we return () right away. :-)

I released a new version of PlonePAS 3.11 with this fix.

Vincent

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers