Chaining searches with joins on the same table

6 messages Options
Embed this post
Permalink
A Murka

Chaining searches with joins on the same table

Reply Threaded More More options
Print post
Permalink
Here's my issue:  I have a table of 'images', which has_many
'ratings', each of which belongs_to a 'user'. I want to get:
1. Aggregate sum of all ratings for each object, as well as
2. the logged-in user's rating for each object.

I have extended ResultSet::Image to have 'with_scores' and
'with_scores_for_user' methods, each of which works individually. They
both join on the ratings table, though, and when I chain them as in
$images->with_scores->with_scores_for_user($user), it only produces
ONE join instead of the two that are necessary.

Here's the code for the methods:

package PicDB::ResultSet::Image;

use strict;
use warnings;
use base 'PicDB::ResultSet';

sub with_scores {
       my ($self) = @_;

       return $self->search({},
               {
                       join => ['ratings'],
                       '+select' => [
                               \'SUM(CASE WHEN ratings.score = 1 THEN
1 ELSE 0 END) AS upvotes',
                               \'SUM(CASE WHEN ratings.score = -1 THEN
1 ELSE 0 END) AS downvotes',
                               \'SUM(ratings.score) AS total_score',
                               \'COUNT(*) AS num_votes',
                       ],
                       '+as' => [ 'upvotes', 'downvotes',
'total_score', 'num_votes' ],
                       group_by => ['me.id'],
               }
       );
}
sub with_scores_for_user {
       my ($self, $user) = @_;
       return $self;

       return $self->search({'ratings.user_id' => $user->id},
               {
                       join => ['ratings'],
                       '+select' => ['ratings.score AS user_score'],
                       '+as' => ['user_score'],
               }
       );
}

1;


Here is the output using DBIC_TRACE=1 for
$images->with_scores->with_scores_for_user($user).
The ->with_scores_for_user($user) call adds the <bracketed> sections:

SELECT me.id, me.srcurl, me.caption, me.user_id, me.ip,
me.when_uploaded, me.views,
SUM(CASE WHEN ratings.score = 1 THEN 1 ELSE 0 END) AS upvotes,
SUM(CASE WHEN ratings.score = -1 THEN 1 ELSE 0 END) AS downvotes,
SUM(ratings.score) AS total_score,
COUNT(*) AS num_votes,
< ratings.score AS user_score >
FROM image me LEFT JOIN rating ratings ON ratings.image_id = me.id
< WHERE ( ratings.user_id = ? ) >
GROUP BY me.id ORDER BY when_uploaded DESC LIMIT 12: '7'

As you can see, this query should have joined on 'ratings' -twice-,
and the condition should have been WHERE (ratings_2.user_id = ?). I
feel that DBIC should somehow recognize that chaining the two queries
should create two joins.

Do I just have to create a method that does both joins at once? Is a
better way to implement these aggergates?

Thanks,
-amurka

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Peter Rabbitson-2

Re: Chaining searches with joins on the same table

Reply Threaded More More options
Print post
Permalink
On Fri, Jul 10, 2009 at 12:08:56AM -0400, A Murka wrote:

> Here's my issue:  I have a table of 'images', which has_many
> 'ratings', each of which belongs_to a 'user'. I want to get:
> 1. Aggregate sum of all ratings for each object, as well as
> 2. the logged-in user's rating for each object.
>
> I have extended ResultSet::Image to have 'with_scores' and
> 'with_scores_for_user' methods, each of which works individually. They
> both join on the ratings table, though, and when I chain them as in
> $images->with_scores->with_scores_for_user($user), it only produces
> ONE join instead of the two that are necessary.
>
> Here's the code for the methods:
>
> <snipped chained join example>
>
>
> Here is the output using DBIC_TRACE=1 for
> $images->with_scores->with_scores_for_user($user).
> The ->with_scores_for_user($user) call adds the <bracketed> sections:
>
> SELECT me.id, me.srcurl, me.caption, me.user_id, me.ip,
> me.when_uploaded, me.views,
> SUM(CASE WHEN ratings.score = 1 THEN 1 ELSE 0 END) AS upvotes,
> SUM(CASE WHEN ratings.score = -1 THEN 1 ELSE 0 END) AS downvotes,
> SUM(ratings.score) AS total_score,
> COUNT(*) AS num_votes,
> < ratings.score AS user_score >
> FROM image me LEFT JOIN rating ratings ON ratings.image_id = me.id
> < WHERE ( ratings.user_id = ? ) >
> GROUP BY me.id ORDER BY when_uploaded DESC LIMIT 12: '7'
>
> As you can see, this query should have joined on 'ratings' -twice-,
> and the condition should have been WHERE (ratings_2.user_id = ?). I
> feel that DBIC should somehow recognize that chaining the two queries
> should create two joins.
>

The join part seems like a bug to me... possibly an untested regression
I introduced while playing with the join handling. Please submit a test
patch, adding a couple of tests demonstrating this behavior against [1]
I'd say a good place would be to augment the last test block on line
122 of t/90join_torture.t

Now the WHERE is a diferent story - the arguments are not (and probably
never will be) adjusted. Remember that in a chained search you might
(and are in fact more likely to) be adding more WHERE conditions to your
first join instead of what came in next. DBIC does the right thing and
does not attempt to guess for you. The WHERE condition adjstment is
solely your responsibility.

Cheers

[1] http://dev.catalyst.perl.org/repos/bast/DBIx-Class/0.08/trunk/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Rob Kinyon

Re: Chaining searches with joins on the same table

Reply Threaded More More options
Print post
Permalink
In reply to this post by A Murka
On Fri, Jul 10, 2009 at 00:08, A Murka<[hidden email]> wrote:
[snip]
> Do I just have to create a method that does both joins at once? Is a
> better way to implement these aggergates?

There is such a thing as putting too much into one query. These
queries do very different things. So, just do two queries. Simplify
your life.

--
Thanks,
Rob Kinyon

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
A Murka

Re: Chaining searches with joins on the same table

Reply Threaded More More options
Print post
Permalink
On Fri, Jul 10, 2009 at 9:59 AM, Rob Kinyon<[hidden email]> wrote:
> On Fri, Jul 10, 2009 at 00:08, A Murka<[hidden email]> wrote:
> [snip]
>> Do I just have to create a method that does both joins at once? Is a
>> better way to implement these aggergates?
>
> There is such a thing as putting too much into one query. These
> queries do very different things. So, just do two queries. Simplify
> your life.

The trouble with that is that it would mean making an extra query for
each image as it's displayed: one to select all the images, then one
for each to select the user's rating for it. On a thumbnail page with
many images, this would mean many extra queries.

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Rob Kinyon

Re: Chaining searches with joins on the same table

Reply Threaded More More options
Print post
Permalink
On Fri, Jul 10, 2009 at 11:39, A Murka<[hidden email]> wrote:

> On Fri, Jul 10, 2009 at 9:59 AM, Rob Kinyon<[hidden email]> wrote:
>> On Fri, Jul 10, 2009 at 00:08, A Murka<[hidden email]> wrote:
>> [snip]
>>> Do I just have to create a method that does both joins at once? Is a
>>> better way to implement these aggergates?
>>
>> There is such a thing as putting too much into one query. These
>> queries do very different things. So, just do two queries. Simplify
>> your life.
>
> The trouble with that is that it would mean making an extra query for
> each image as it's displayed: one to select all the images, then one
> for each to select the user's rating for it. On a thumbnail page with
> many images, this would mean many extra queries.

Two queries. One to get all the images, then one to get all the user
ratings for each image. Use the new subquery interface.

my $rs_all_images = resultset('Image')->get_all_images;
my $rs_user_ratings = resultset('UserRatings')->search({
    user_id => $user->id,
    image_id => [ $rs_all_images->get_column('image_id')->as_query ],
})

Rob

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Peter Rabbitson-2

Re: Chaining searches with joins on the same table

Reply Threaded More More options
Print post
Permalink
In reply to this post by A Murka
A Murka wrote:

> Here's my issue:  I have a table of 'images', which has_many
> 'ratings', each of which belongs_to a 'user'. I want to get:
> 1. Aggregate sum of all ratings for each object, as well as
> 2. the logged-in user's rating for each object.
>
> I have extended ResultSet::Image to have 'with_scores' and
> 'with_scores_for_user' methods, each of which works individually. They
> both join on the ratings table, though, and when I chain them as in
> $images->with_scores->with_scores_for_user($user), it only produces
> ONE join instead of the two that are necessary.
>
> Here's the code for the methods:
>
> package PicDB::ResultSet::Image;
>
> use strict;
> use warnings;
> use base 'PicDB::ResultSet';
>
> sub with_scores {
>        my ($self) = @_;
>
>        return $self->search({},
>                {
>                        join => ['ratings'],
>                        '+select' => [
>                                \'SUM(CASE WHEN ratings.score = 1 THEN
> 1 ELSE 0 END) AS upvotes',
>                                \'SUM(CASE WHEN ratings.score = -1 THEN
> 1 ELSE 0 END) AS downvotes',
>                                \'SUM(ratings.score) AS total_score',
>                                \'COUNT(*) AS num_votes',
>                        ],
>                        '+as' => [ 'upvotes', 'downvotes',
> 'total_score', 'num_votes' ],
>                        group_by => ['me.id'],
>                }
>        );
> }
> sub with_scores_for_user {
>        my ($self, $user) = @_;
>        return $self;
>
>        return $self->search({'ratings.user_id' => $user->id},
>                {
>                        join => ['ratings'],
>                        '+select' => ['ratings.score AS user_score'],
>                        '+as' => ['user_score'],
>                }
>        );
> }
>
> 1;
>
>
> Here is the output using DBIC_TRACE=1 for
> $images->with_scores->with_scores_for_user($user).
> The ->with_scores_for_user($user) call adds the <bracketed> sections:
>
> SELECT me.id, me.srcurl, me.caption, me.user_id, me.ip,
> me.when_uploaded, me.views,
> SUM(CASE WHEN ratings.score = 1 THEN 1 ELSE 0 END) AS upvotes,
> SUM(CASE WHEN ratings.score = -1 THEN 1 ELSE 0 END) AS downvotes,
> SUM(ratings.score) AS total_score,
> COUNT(*) AS num_votes,
> < ratings.score AS user_score >
> FROM image me LEFT JOIN rating ratings ON ratings.image_id = me.id
> < WHERE ( ratings.user_id = ? ) >
> GROUP BY me.id ORDER BY when_uploaded DESC LIMIT 12: '7'
>
> As you can see, this query should have joined on 'ratings' -twice-,
> and the condition should have been WHERE (ratings_2.user_id = ?). I
> feel that DBIC should somehow recognize that chaining the two queries
> should create two joins.
>

So any chance I can get a test for this?

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...