Inflation and Deflation

7 messages Options
Embed this post
Permalink
Jose Luis Martinez

Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
Hi,

   I've uploaded DBIx-Class-Inflator-Serializers to CPAN
(http://search.cpan.org/~jlmartin/DBIx-Class-Inflator-Serializers-0.01/).
I had to write a JSON inflator/deflator for a project, and thought that
it would be nice to have the inflators pre-packaged. The intention is to
provide a suite of well proven and reusable inflators and deflators to
complement DBIx::Class.

What's been improved over the inflators in the FAQ:

  - If the column definition has a size attribute, and the serialized
data structure is longer than the size: an exception is thrown.
  - Deserialization errors throw an exception when there is an error for
all modules (this can happen if you don't specify size and your DB
truncates the value when it gets stored. Also not all deserialzation
modules throw exceptions on error).
  - use JSON::Any (it's already required by DBIC 0.08103)

Hope you like it. I'd love to get feedback from the DBIx community to
get this package to a 1.00 state.

Jose Luis Martinez
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
fREW Schmidt

Re: Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
On Mon, Jun 8, 2009 at 4:52 PM, Jose Luis Martinez <[hidden email]> wrote:
Hi,

 I've uploaded DBIx-Class-Inflator-Serializers to CPAN (http://search.cpan.org/~jlmartin/DBIx-Class-Inflator-Serializers-0.01/). I had to write a JSON inflator/deflator for a project, and thought that it would be nice to have the inflators pre-packaged. The intention is to provide a suite of well proven and reusable inflators and deflators to complement DBIx::Class.

What's been improved over the inflators in the FAQ:

 - If the column definition has a size attribute, and the serialized data structure is longer than the size: an exception is thrown.
 - Deserialization errors throw an exception when there is an error for all modules (this can happen if you don't specify size and your DB truncates the value when it gets stored. Also not all deserialzation modules throw exceptions on error).
 - use JSON::Any (it's already required by DBIC 0.08103)

Hope you like it. I'd love to get feedback from the DBIx community to get this package to a 1.00 state.

Very cool!  you may want to consider allowing the use of a TO_JSON method as an alias to serialized.  At least for JSON stuff that's the standard.  I will probably try this out tomorrow or the next day and I'll let you know how it goes.


--
fREW Schmidt
http://blog.afoolishmanifesto.com

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Jose Luis Martinez

Re: Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
fREW Schmidt escribió:
>
> Very cool!  you may want to consider allowing the use of a TO_JSON
> method as an alias to serialized.  At least for JSON stuff that's the
> standard.  I will probably try this out tomorrow or the next day and
> I'll let you know how it goes.

Thanks! You choose the method, as the column "serialized" is the column
name that stores data structures in JSON format.

> fREW Schmidt
> http://blog.afoolishmanifesto.com

Reading through your blog, I found this entry:
http://blog.afoolishmanifesto.com/archives/739

Maybe you're confusing my module with something that can give you a JSON
string from a DBIC row object. I have a couple of ideas for this one:

package DBIx::Class::Serializer::ToJSON;

use JSON::Any;

sub TO_JSON {
   my ($self, @cols) = @_;
   #if called without columns to pass to JSON, use all of them
   @cols = keys %{ $self->get_columns };

   return JSON::Any->objToJson({ map { $_ => $self->$_ } @cols });
}

1;

Then you load Serializer::ToJSON in the load_components of your DBIC
Classes, and you get for free a TO_JSON method for all the row objects
you apply it to. Maybe that is what you where looking for? (note: I've
written this code as it was coming out... no testing)

Another idea is to take ResultClass::HashRefInflator and convert it into
a ResultClass::JSONInflator... But that seems like an overkill, maybe.

Jose Luis Martinez
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Matt S Trout

Re: Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jose Luis Martinez
On Mon, Jun 08, 2009 at 11:52:47PM +0200, Jose Luis Martinez wrote:
> Hi,
>
>   I've uploaded DBIx-Class-Inflator-Serializers to CPAN
> (http://search.cpan.org/~jlmartin/DBIx-Class-Inflator-Serializers-0.01/).

Ok. Great idea. Glad you made the effort to write and release it. Now for
the part where I tear the details to pieces, please hold onto your chair.

First, please delete the current copy from CPAN, it's in the wrong namespace.

Column inflators go in DBIx::Class::InflateColumn, not ::Inflator:: - stealing
random DBIC subnamespaces when there's already an accepted convention isn't
very polite and perhaps more importantly will make your code much harder to
find on CPAN.

Second, the architecture's silly. What you wanted to write was a single
DBIx::Class::InflateColumn::Serializer that uses a driver model like
DBIx::Class::EncodedColumn and DBIx::Class::UUIDColumns do.

It also seems like if you do that then the length check could be generalised.

Third, your code is full of things like -

    if (defined $info->{'size'}){
       $freezer = get_bounded_column_freezer($info->{'size'});
    } else {
       $freezer = \&unbounded_freezer;
    }

which is catastrophically bad since you've now added methods to the class
that can't actually be called as methods because they're really subs, and
can't be overriden by subclasses either.

Within OO code, please only use method calls and document them so people
can override them - otherwise your code isn't usefully pre-packaged because
people will have to copy and paste it in order to extend it ...

> Hope you like it. I'd love to get feedback from the DBIx community to
> get this package to a 1.00 state.

There's no such thing as "the DBIx community". DBIx is the namespace for
DBI extensions, DBIx::Class is one project in that namespace.

Just like DBix::Class::InflateColumn:: is the namespace for DBIx::Class
inflators, and there are many projects in that namespace.

Plus yours in the wrong one until you delete it and fix it :)

All that said, I think the -idea- is great - just the name, architecture
and implementation need some work :)

I look forward to looking at a tarball or repository of a rewritten version
so we can try and give feedback on the new one -before- it goes to CPAN ...

(also, I'd be happy to see this as a "DBIC team supported project" - in
which case could we offer you subversion or git hosting for it? :)

--
        Matt S Trout         Catalyst and DBIx::Class consultancy with a clue
     Technical Director      and a commit bit: http://shadowcat.co.uk/catalyst/
 Shadowcat Systems Limited
  mst (@) shadowcat.co.uk        http://shadowcat.co.uk/blog/matt-s-trout/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Matt S Trout

Re: Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jose Luis Martinez
On Thu, Jun 11, 2009 at 12:28:05AM +0200, Jose Luis Martinez wrote:

> fREW Schmidt escribió:
> >
> >Very cool!  you may want to consider allowing the use of a TO_JSON
> >method as an alias to serialized.  At least for JSON stuff that's the
> >standard.  I will probably try this out tomorrow or the next day and
> >I'll let you know how it goes.
>
> Thanks! You choose the method, as the column "serialized" is the column
> name that stores data structures in JSON format.
>
> >fREW Schmidt
> >http://blog.afoolishmanifesto.com
>
> Reading through your blog, I found this entry:
> http://blog.afoolishmanifesto.com/archives/739
>
> Maybe you're confusing my module with something that can give you a JSON
> string from a DBIC row object. I have a couple of ideas for this one:
>
> package DBIx::Class::Serializer::ToJSON;
>
> use JSON::Any;
>
> sub TO_JSON {
>   my ($self, @cols) = @_;
>   #if called without columns to pass to JSON, use all of them
>   @cols = keys %{ $self->get_columns };
>
>   return JSON::Any->objToJson({ map { $_ => $self->$_ } @cols });
> }

I'd suggest delegating to an _data_for_json method here - that way
people can subclass it to include rels or forcibly exclude specific
columns or whatever ...

--
        Matt S Trout         Catalyst and DBIx::Class consultancy with a clue
     Technical Director      and a commit bit: http://shadowcat.co.uk/catalyst/
 Shadowcat Systems Limited
  mst (@) shadowcat.co.uk        http://shadowcat.co.uk/blog/matt-s-trout/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Jose Luis Martinez

Re: Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
In reply to this post by Matt S Trout
Matt S Trout escribió:
> First, please delete the current copy from CPAN, it's in the wrong namespace.
> Column inflators go in DBIx::Class::InflateColumn, not ::Inflator:: - stealing
> random DBIC subnamespaces when there's already an accepted convention isn't
> very polite and perhaps more importantly will make your code much harder to
> find on CPAN.

No problem. Sorry for the namespace invasion. I've already scheduled the
deletion in PAUSE.

> Second, the architecture's silly. What you wanted to write was a single
> DBIx::Class::InflateColumn::Serializer that uses a driver model like
> DBIx::Class::EncodedColumn and DBIx::Class::UUIDColumns do.

+1. Since I started out with a JSON inflator for one of my projects, I
extended the behaviour to other serializers, thinking of usefulness for
other people (not so much in elegance of the code :p).

> Third, your code is full of things like -
>
>     if (defined $info->{'size'}){
>        $freezer = get_bounded_column_freezer($info->{'size'});
>     } else {
>        $freezer = \&unbounded_freezer;
>     }
>
> which is catastrophically bad since you've now added methods to the class
> that can't actually be called as methods because they're really subs, and
> can't be overriden by subclasses either.

Let's see if you like it better like this:

method select_freezer
   - selects the get_xxx_freezer to call depending on the properties of
the column (provided just in case someone wants to override the method
for selecting freezers).
   - get_bounded_column_freezer and unbounded_freezer get transformed
into $self->get_bounded_column_freezer, and $self->unbounded_freezer.
Both will return subs that let the column get unfrozen. Are you OK with
returning subs?

> Within OO code, please only use method calls and document them so people
> can override them - otherwise your code isn't usefully pre-packaged because
> people will have to copy and paste it in order to extend it ...

Didn't want to document them, since I consider they are internal to the
working of the inflators. maybe I should change them to sub
_get_xxx_freezer? Maybe document select_freezer so people can override
that one?

>> Hope you like it. I'd love to get feedback from the DBIx community to
>> get this package to a 1.00 state.
>
> There's no such thing as "the DBIx community". DBIx is the namespace for
> DBI extensions, DBIx::Class is one project in that namespace.

s/DBIx/DBIC/ (fast typing. null review).

> All that said, I think the -idea- is great - just the name, architecture
> and implementation need some work :)

That's what I was looking for... 0.01 exposes an idea. The feedback gets
done what the community wants/needs/thinks (not just what I
want/need/think :))

> I look forward to looking at a tarball or repository of a rewritten version
> so we can try and give feedback on the new one -before- it goes to CPAN ...
>
> (also, I'd be happy to see this as a "DBIC team supported project" - in
> which case could we offer you subversion or git hosting for it? :)
>
+1 for a SVN repo.

Thanks for the opinion and the pointers,

Jose Luis Martinez
[hidden email]

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
fREW Schmidt

Re: Inflation and Deflation

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jose Luis Martinez

Maybe you're confusing my module with something that can give you a JSON string from a DBIC row object. I have a couple of ideas for this one:

package DBIx::Class::Serializer::ToJSON;

use JSON::Any;

sub TO_JSON {
 my ($self, @cols) = @_;
 #if called without columns to pass to JSON, use all of them
 @cols = keys %{ $self->get_columns };

 return JSON::Any->objToJson({ map { $_ => $self->$_ } @cols });
}

1;

Then you load Serializer::ToJSON in the load_components of your DBIC Classes, and you get for free a TO_JSON method for all the row objects you apply it to. Maybe that is what you where looking for? (note: I've written this code as it was coming out... no testing)

Another idea is to take ResultClass::HashRefInflator and convert it into a ResultClass::JSONInflator... But that seems like an overkill, maybe.

Eh, I have always thought that what I'd like to do is this:

__PACKAGE->add_columns(
   username {
      json => 1,
   },
   street_address {
      json => 1,
   });

And then the TO_JSON method would just return:
{ username => $self->username, street_address => $self->street_address }

You can set the JSON class to automatically call the TO_JSON method, which is pretty convenient in these cases.

Anyway, I'm probably *not* really talking about an inflator as the given items keep their same methods.

--
fREW Schmidt
http://blog.afoolishmanifesto.com

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...