DBIx::Class, another data validation approach

26 messages Options
Embed this post
Permalink
1 2
Ulrich Kautz

DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
Hello all

sometime ago i wrote a data validation module for validation right in
the model. I didnt published it yet, and now i searched the list and
found DBIx::Class::Validation.

However, i thought about using this one instead, but the migration would
be to extensive for me and i think it does not fit my needs.. Hope this
isnt too confusing; in the attachment is my idea of validation
(DBIx::Class::DataValidator).

Any tests, critics and ideas are welcome. I would upload it to CPAN as
soon as is find time to clean the docs and write some more tests (the
included should work).

Greets from Berlin
Ulrich Kautz


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...

DBIx-Class-DataValidator-0.01.tar.gz (17K) Download Attachment
Rob Kinyon

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
On Tue, Jul 21, 2009 at 09:41, Ulrich Kautz<[hidden email]> wrote:

> Hello all
>
> sometime ago i wrote a data validation module for validation right in
> the model. I didnt published it yet, and now i searched the list and
> found DBIx::Class::Validation.
>
> However, i thought about using this one instead, but the migration would
> be to extensive for me and i think it does not fit my needs.. Hope this
> isnt too confusing; in the attachment is my idea of validation
> (DBIx::Class::DataValidator).
>
> Any tests, critics and ideas are welcome. I would upload it to CPAN as
> soon as is find time to clean the docs and write some more tests (the
> included should work).

Instead of dropping a blob of code, why don't you put together a
description of what yours does that DBIC::Validation doesn't? Maybe a
pros and cons for each module. If you want someone to use something
other than the established standard, you have to make the case. You
might well be right that yours is better for certain situations, but
I'm not going to take the effort to figure that out - that's your job.
:)

Rob

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
Rob Kinyon schrieb:

> On Tue, Jul 21, 2009 at 09:41, Ulrich Kautz<[hidden email]> wrote:
>  
>> Hello all
>>
>> sometime ago i wrote a data validation module for validation right in
>> the model. I didnt published it yet, and now i searched the list and
>> found DBIx::Class::Validation.
>>
>> However, i thought about using this one instead, but the migration would
>> be to extensive for me and i think it does not fit my needs.. Hope this
>> isnt too confusing; in the attachment is my idea of validation
>> (DBIx::Class::DataValidator).
>>
>> Any tests, critics and ideas are welcome. I would upload it to CPAN as
>> soon as is find time to clean the docs and write some more tests (the
>> included should work).
>>    
>
> Instead of dropping a blob of code, why don't you put together a
> description of what yours does that DBIC::Validation doesn't? Maybe a
> pros and cons for each module. If you want someone to use something
> other than the established standard, you have to make the case. You
> might well be right that yours is better for certain situations, but
> I'm not going to take the effort to figure that out - that's your job.
> :)
>
> Rob
>  
Actually, i didnt want to make a competition out of this, i just though
it might come in handy for someone with the same requirements as i have.
But you are right, i should mention at least why DBIx::Class::Validation
is not sufficient for me.

These pros and cons are from my perspective. Eg you can surely achieve
inheritance with DBIx::Class::Validation within certain circumstances
via outsourcing your validator profiles in external modules and merge
them back in the model.. but on profile level, this is far to imprecise
for _my_ purpose. And in opposite, you can argument that inheritance on
model level is not a good approach for your kind of application, and so
on ..


DBIx::Class::Validation

PRO:
+ Very simple to use, a single module, no complex configuration (besides
the validation profiles)..
+ Supports multiple validation modules (besides FormData::Validator)

CONTRA:
- Very "static", only one validation context (aka one filter profile)
for each model is possible (or validation becomes again part of another
layer, where you provide the profile definition)
- Half way solution, columns and validation definition still separated
- DRY philosophy not achieved, because inheritance of validation not given
- No deflate / inflate awareness


DBIx::Class::DataValidator

PRO:
+ Multiple mergable validators supported for multiple contexts at
runtime (create, update, customs.. )
+ Brings columns and validation together at definition time.
+ Better DRY support via named validation methods and column classes
(inheritable)
+ Deflate / inflate aware

CONTRA:
- More complex usage, consists of two modules (ResultSet and Component),
more settings (configuration) makes it less lighweight
- Bound to Data::FormValidator



Ulrich

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Andy Armstrong

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
On 21 Jul 2009, at 16:29, Ulrich Kautz wrote:

> DBIx::Class::DataValidator
>
> PRO:
> + Multiple mergable validators supported for multiple contexts at
> runtime (create, update, customs.. )
> + Brings columns and validation together at definition time.
> + Better DRY support via named validation methods and column classes
> (inheritable)
> + Deflate / inflate aware
>
> CONTRA:
> - More complex usage, consists of two modules (ResultSet and  
> Component),
> more settings (configuration) makes it less lighweight
> - Bound to Data::FormValidator


Sounds like just what I'm looking for. Are you going to release it?

--
Andy Armstrong, Hexten




_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Rob Kinyon

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
On Tue, Jul 21, 2009 at 11:51, Andy Armstrong<[hidden email]> wrote:

> On 21 Jul 2009, at 16:29, Ulrich Kautz wrote:
>>
>> DBIx::Class::DataValidator
>>
>> PRO:
>> + Multiple mergable validators supported for multiple contexts at
>> runtime (create, update, customs.. )
>> + Brings columns and validation together at definition time.
>> + Better DRY support via named validation methods and column classes
>> (inheritable)
>> + Deflate / inflate aware
>>
>> CONTRA:
>> - More complex usage, consists of two modules (ResultSet and Component),
>> more settings (configuration) makes it less lighweight
>> - Bound to Data::FormValidator

First off, I think this is better than DBIC::Validation. It is a
better fit to the ResultSet/Row separation that DBIC is all about.

That said, there's a few things that I'd like to see be done:
1) With one setting, I should be able to turn on validation for all my
columns that have a type set for them. So, if I have a column that is
an integer, that should just turn on for me.
2) I should be able to chain validations. So, something like
"validation => [ 'foo', 'bar' ]". Look at how Moose does this.
3) Different DBMSes have different types. So, maybe producer-specific
validations would be helpful (q.v. #1).

This is also starting to get very very close to custom types. There's
been discussion of this for 0.09. I think getting you involved would
be very helpful.

Rob

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
fREW Schmidt

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by Andy Armstrong


On Tue, Jul 21, 2009 at 10:51 AM, Andy Armstrong <[hidden email]> wrote:
On 21 Jul 2009, at 16:29, Ulrich Kautz wrote:
DBIx::Class::DataValidator

PRO:
+ Multiple mergable validators supported for multiple contexts at
runtime (create, update, customs.. )
+ Brings columns and validation together at definition time.
+ Better DRY support via named validation methods and column classes
(inheritable)
+ Deflate / inflate aware

CONTRA:
- More complex usage, consists of two modules (ResultSet and Component),
more settings (configuration) makes it less lighweight
- Bound to Data::FormValidator


Sounds like just what I'm looking for. Are you going to release it?

Agreed.
--
fREW Schmidt
http://blog.afoolishmanifesto.com

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by Rob Kinyon
Rob Kinyon schrieb:

> On Tue, Jul 21, 2009 at 11:51, Andy Armstrong<[hidden email]> wrote:
>
>> On 21 Jul 2009, at 16:29, Ulrich Kautz wrote:
>>> DBIx::Class::DataValidator
>>>
>>> PRO: + Multiple mergable validators supported for multiple
>>> contexts at runtime (create, update, customs.. ) + Brings columns
>>> and validation together at definition time. + Better DRY support
>>> via named validation methods and column classes (inheritable) +
>>> Deflate / inflate aware
>>>
>>> CONTRA: - More complex usage, consists of two modules (ResultSet
>>> and Component), more settings (configuration) makes it less
>>> lighweight - Bound to Data::FormValidator


Thanks for the input. Now i have some questions ;)

>
> First off, I think this is better than DBIC::Validation. It is a
> better fit to the ResultSet/Row separation that DBIC is all about.
>
> That said, there's a few things that I'd like to see be done: 1) With
> one setting, I should be able to turn on validation for all my
> columns that have a type set for them. So, if I have a column that is
>  an integer, that should just turn on for me.

Ok, on which level ? For a single model, for all models with columns of a certain type globaly (see 3) or at runtime (eg before the actual update/create.. something like $object->validates( $data_ref, { exclude_types => [ qw/ integer / ] } ) ?
What use-cases can you imagine for this (for my better understanding of the requirements) ? For my use, the decision which validator(s) or none at all to use was sufficient, so far, but that doesnt mean anything ..

> 2) I should be able to chain validations. So, something like
> "validation => [ 'foo', 'bar' ]". Look at how Moose does this.

I'm not quite sure wheter this is not already implemented. Right now, you can provide multiple validators for (update|update_all|create)_validated and the validates method itself. These will by applied one after another, thus chained. Eg:
$object->validates( $data_ref, qw/ validator1 validator2 / );
That's what you meant, or am i missing the point ?

> 3) Different DBMSes have different types. So, maybe producer-specific
>  validations would be helpful (q.v. #1).

Sounds very intersting. I could imagine validation-set-modules, one for every DBMS, like DBIx::Class::DataValidator::MySQLTypes. Then one should be able to enable/disable a subset of validation types.. eg
__PACKAGE__->mysql_validation_enable( qw/ :all / );
__PACKAGE__->mysql_validation_disable( qw/ date datetime / );
If you do this in your super-model (from which all inherit) this would apply globally.
If you switch the DBMS now, you simple load another module like DBIx::Class::DataValidator::PostgresTypes or so and all the special columns for this DBMS will be validated.

> This is also starting to get very very close to custom types. There's
>  been discussion of this for 0.09. I think getting you involved would
>  be very helpful.

Sure, if i can.


Ulrich

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Rob Kinyon

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
On Tue, Jul 21, 2009 at 13:35, Ulrich Kautz<[hidden email]> wrote:
>> That said, there's a few things that I'd like to see be done: 1) With
>> one setting, I should be able to turn on validation for all my
>> columns that have a type set for them. So, if I have a column that is
>>  an integer, that should just turn on for me.
>
> Ok, on which level ? For a single model, for all models with columns of a certain type globaly (see 3) or at runtime (eg before the actual update/create.. something like $object->validates( $data_ref, { exclude_types => [ qw/ integer / ] } ) ?
> What use-cases can you imagine for this (for my better understanding of the requirements) ? For my use, the decision which validator(s) or none at all to use was sufficient, so far, but that doesnt mean anything ..

Frankly, I am of the opinion that if you have DataValidation as one of
your components, any of your columns with a type should at least
validate against that type. This would seriously help with SQLite
usage.

>> 2) I should be able to chain validations. So, something like
>> "validation => [ 'foo', 'bar' ]". Look at how Moose does this.
>
> I'm not quite sure wheter this is not already implemented. Right now, you can provide multiple validators for (update|update_all|create)_validated and the validates method itself. These will by applied one after another, thus chained. Eg:
> $object->validates( $data_ref, qw/ validator1 validator2 / );
> That's what you meant, or am i missing the point ?

That works.

>> 3) Different DBMSes have different types. So, maybe producer-specific
>>  validations would be helpful (q.v. #1).
>
> Sounds very intersting. I could imagine validation-set-modules, one for every DBMS, like DBIx::Class::DataValidator::MySQLTypes. Then one should be able to enable/disable a subset of validation types.. eg
> __PACKAGE__->mysql_validation_enable( qw/ :all / );
> __PACKAGE__->mysql_validation_disable( qw/ date datetime / );
> If you do this in your super-model (from which all inherit) this would apply globally.
> If you switch the DBMS now, you simple load another module like DBIx::Class::DataValidator::PostgresTypes or so and all the special columns for this DBMS will be validated.

Bah. Autoload that crap based on the storage_type I already told DBIC
about in my DSN. Don't make me hardcode anything.

Rob

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
Rob Kinyon schrieb:

> On Tue, Jul 21, 2009 at 13:35, Ulrich Kautz<[hidden email]>
> wrote:
>>> That said, there's a few things that I'd like to see be done: 1)
>>> With one setting, I should be able to turn on validation for all
>>> my columns that have a type set for them. So, if I have a column
>>> that is an integer, that should just turn on for me.
>> Ok, on which level ? For a single model, for all models with
>> columns of a certain type globaly (see 3) or at runtime (eg before
>> the actual update/create.. something like $object->validates(
>> $data_ref, { exclude_types => [ qw/ integer / ] } ) ? What
>> use-cases can you imagine for this (for my better understanding of
>> the requirements) ? For my use, the decision which validator(s) or
>> none at all to use was sufficient, so far, but that doesnt mean
>> anything ..
>
> Frankly, I am of the opinion that if you have DataValidation as one
> of your components, any of your columns with a type should at least
> validate against that type. This would seriously help with SQLite
> usage.
>
>>> 2) I should be able to chain validations. So, something like
>>> "validation => [ 'foo', 'bar' ]". Look at how Moose does this.
>> I'm not quite sure wheter this is not already implemented. Right
>> now, you can provide multiple validators for
>> (update|update_all|create)_validated and the validates method
>> itself. These will by applied one after another, thus chained. Eg:
>> $object->validates( $data_ref, qw/ validator1 validator2 / );
>> That's what you meant, or am i missing the point ?
>
> That works.
>
>>> 3) Different DBMSes have different types. So, maybe
>>> producer-specific validations would be helpful (q.v. #1).
>> Sounds very intersting. I could imagine validation-set-modules, one
>> for every DBMS, like DBIx::Class::DataValidator::MySQLTypes. Then
>> one should be able to enable/disable a subset of validation types..
>> eg __PACKAGE__->mysql_validation_enable( qw/ :all / );
>> __PACKAGE__->mysql_validation_disable( qw/ date datetime / ); If
>> you do this in your super-model (from which all inherit) this would
>> apply globally. If you switch the DBMS now, you simple load another
>> module like DBIx::Class::DataValidator::PostgresTypes or so and all
>> the special columns for this DBMS will be validated.
>
> Bah. Autoload that crap based on the storage_type I already told DBIC
>  about in my DSN. Don't make me hardcode anything.
>
> Rob
>

Alright then, i'll look into it at the weekend. The validation via DBMS determiniation looks like a goode idea. Thanks.

Ulrich

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by fREW Schmidt
fREW Schmidt schrieb:

> On Tue, Jul 21, 2009 at 10:51 AM, Andy Armstrong <[hidden email]> wrote:
>
>> On 21 Jul 2009, at 16:29, Ulrich Kautz wrote:
>>
>>> DBIx::Class::DataValidator
>>>
>>> PRO:
>>> + Multiple mergable validators supported for multiple contexts at
>>> runtime (create, update, customs.. )
>>> + Brings columns and validation together at definition time.
>>> + Better DRY support via named validation methods and column classes
>>> (inheritable)
>>> + Deflate / inflate aware
>>>
>>> CONTRA:
>>> - More complex usage, consists of two modules (ResultSet and Component),
>>> more settings (configuration) makes it less lighweight
>>> - Bound to Data::FormValidator
>>>
>>
>> Sounds like just what I'm looking for. Are you going to release it?
>>
>
> Agreed.
>

I'll rethink something Rob mentioned (especially the column-type based validation for DBMS), finish the docs, write some more tests, then i'll post it on CPAN.. something like two  weeks, i guess.

greets
Ulrich

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Andy Armstrong

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
On 21 Jul 2009, at 18:56, Ulrich Kautz wrote:
> I'll rethink something Rob mentioned (especially the column-type  
> based validation for DBMS), finish the docs, write some more tests,  
> then i'll post it on CPAN.. something like two  weeks, i guess.


Thanks Ulrich :)

--
Andy Armstrong, Hexten




_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Eden Cardim

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by Rob Kinyon
On Tue, Jul 21, 2009 at 1:10 PM, Rob Kinyon<[hidden email]> wrote:
> First off, I think this is better than DBIC::Validation. It is a
> better fit to the ResultSet/Row separation that DBIC is all about.
>
> That said, there's a few things that I'd like to see be done:
> 1) With one setting, I should be able to turn on validation for all my
> columns that have a type set for them. So, if I have a column that is
> an integer, that should just turn on for me.
> 2) I should be able to chain validations. So, something like
> "validation => [ 'foo', 'bar' ]". Look at how Moose does this.

Once DBIC migrates to Moose we should get all of this relatively transparently.

> 3) Different DBMSes have different types. So, maybe producer-specific
> validations would be helpful (q.v. #1).

For Moose I'd make the storage class load up the adequate
TypeConstraints/Coercions for each backend.

--
   Eden Cardim       Need help with your Catalyst or DBIx::Class project?
  Code Monkey                    http://www.shadowcat.co.uk/catalyst/
 Shadowcat Systems Ltd.  Want a managed development or deployment platform?
http://edenc.vox.com/            http://www.shadowcat.co.uk/servers/

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Alexander Hartmaier

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by Ulrich Kautz
Am Dienstag, den 21.07.2009, 17:29 +0200 schrieb Ulrich Kautz:

> DBIx::Class::Validation
>
> PRO:
> + Very simple to use, a single module, no complex configuration (besides
> the validation profiles)..
> + Supports multiple validation modules (besides FormData::Validator)
>
> CONTRA:
> - Very "static", only one validation context (aka one filter profile)
> for each model is possible (or validation becomes again part of another
> layer, where you provide the profile definition)
> - Half way solution, columns and validation definition still separated
> - DRY philosophy not achieved, because inheritance of validation not given
> - No deflate / inflate awareness
>
I overcome some of your contra arguments by auto-generating the FVS
profile out of the column metadata.

Because DBIC 0.09 will have validation out-of-the-box because of Moose I
haven't invested any more work into making my module any better.

The DBMS validates data types, length, undef and foreign keys anyway, so
maybe using it as validator (parse its error messages) is a possibility?

>
> DBIx::Class::DataValidator
>
> PRO:
> + Multiple mergable validators supported for multiple contexts at
> runtime (create, update, customs.. )
> + Brings columns and validation together at definition time.
> + Better DRY support via named validation methods and column classes
> (inheritable)
> + Deflate / inflate aware
>
> CONTRA:
> - More complex usage, consists of two modules (ResultSet and Component),
> more settings (configuration) makes it less lighweight
> - Bound to Data::FormValidator
>
>
>
> Ulrich

--
BR Alex


*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH   Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by Ulrich Kautz

>>>> That said, there's a few things that I'd like to see be done:
>>>> 1) With one setting, I should be able to turn on validation for
>>>> all my columns that have a type set for them. So, if I have a
>>>> column that is an integer, that should just turn on for me.
>>> Ok, on which level ? For a single model, for all models with
>>> columns of a certain type globaly (see 3) or at runtime (eg
>>> before the actual update/create.. something like
>>> $object->validates( $data_ref, { exclude_types => [ qw/ integer /
>>> ] } ) ? What use-cases can you imagine for this (for my better
>>> understanding of the requirements) ? For my use, the decision
>>> which validator(s) or none at all to use was sufficient, so far,
>>> but that doesnt mean anything ..
>> Frankly, I am of the opinion that if you have DataValidation as one
>>  of your components, any of your columns with a type should at
>> least validate against that type. This would seriously help with
>> SQLite usage.
>>
>>>> 2) I should be able to chain validations. So, something like
>>>> "validation => [ 'foo', 'bar' ]". Look at how Moose does this.
>>> I'm not quite sure wheter this is not already implemented. Right
>>> now, you can provide multiple validators for
>>> (update|update_all|create)_validated and the validates method
>>> itself. These will by applied one after another, thus chained.
>>> Eg: $object->validates( $data_ref, qw/ validator1 validator2 / );
>>>  That's what you meant, or am i missing the point ?
>> That works.
>>
>>>> 3) Different DBMSes have different types. So, maybe
>>>> producer-specific validations would be helpful (q.v. #1).
>>> Sounds very intersting. I could imagine validation-set-modules,
>>> one for every DBMS, like DBIx::Class::DataValidator::MySQLTypes.
>>> Then one should be able to enable/disable a subset of validation
>>> types.. eg __PACKAGE__->mysql_validation_enable( qw/ :all / );
>>> __PACKAGE__->mysql_validation_disable( qw/ date datetime / ); If
>>> you do this in your super-model (from which all inherit) this
>>> would apply globally. If you switch the DBMS now, you simple load
>>> another module like DBIx::Class::DataValidator::PostgresTypes or
>>> so and all the special columns for this DBMS will be validated.
>> Bah. Autoload that crap based on the storage_type I already told
>> DBIC about in my DSN. Don't make me hardcode anything.
>>
>> Rob
>>
>
> Alright then, i'll look into it at the weekend. The validation via
> DBMS determiniation looks like a goode idea. Thanks.
>
Ok, i've updated the modules concerning your ideas. A db-type aware validation would be auto-loaded and applied, if not disabled. But, as Eden wrote, the DBMS stuff will be added in DBIC 0.9 anyway, so i kept it most simplistic and modular, that it can be removed later on and doesnt make me to waste much time now.
Further, i'm not sure "how much validation" will be part of 0.9 and if this module (even without the DBMS stuff) is not totally obsolete when DBIC 0.9 releases. However, i need validation (especially for profiles) right now and cant wait for the release. Could'nt find any any date, though.

I've updated the documentation to explain the ideas and possibilities of this module. Following i've pasted an excerpt, which should explain most of it briefly and doesnt make me write it twice:

***********************
[...] a short introduction of the multiple approaches and methods you can follow to achieve validation.

Most of them could be combined and are not "either-or".

* Validation profiles
   These are ment to be used in most complex validation scenarios. You can validate the data with multiple profiles (one after another, chained)
   with one call.

* Inline column validation
   Directly in the column definition via add_column they are most simple and fast. Also the definition of the columns is not apart from the defi‐
   nition of the validation.

   They are limited because they have to fit always (each save) and cannot be used, as validation profiles, only in certain circumstances.

   You can easily use them in addition to validation profiles or exclusive. In the first case they’ll run before the validation profiles.

* DBMS specific validation
   This is a kind of auto-validation for basic datatypes of your column. They are DBMS dependent. So Oracle DB might have some types which Post‐
   greSQL has not and vice versa. Also the dates for PostgreSQL DBs and for MySQL DBs look different.

   They normally apply per default if you dont deactivate them manually (see below). They are overwritten by your custom inline column
   validation.

   They can be understand as a kind of default inline column validation.

* Named validation methods
   Those could be used either in inline column validation or validation profiles alike.  This is simply an approach to outsource often used vali‐
   dation methods or regular expressions to reuse them in multiple models and/or for multiple columns.

* Validation column classes
   Another approach to reduce code in your models by pre-defining.  This includes more then the basic datatypes (you can say each integer column
   is an integer-class column) by allowing custom and complex validations.

   They can also used for non validation purposes (just dont use the validation key). Named validation methods are of course usable.

   Probably you should define those classes in a super model from which all your other models inherit to get the most benefit out of it.
***********************

Please read the POD in DBIx::Class::DataValidator for more info..

Despite what i wrote above about 0.9, i think i could provide DBMS specific validation for MySQL and PostgreSQL before i release this to CPAN (but in sepparate packages, due to dependencies), but first of all i would like to hear what you all think  about the usefullness at all.

Nice weekend to everybody!
Ulrich


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...

DBIx-Class-DataValidator-0.02.tar.gz (32K) Download Attachment
fREW Schmidt

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
On Sat, Jul 25, 2009 at 10:14 AM, Ulrich Kautz <[hidden email]> wrote:

>>>> That said, there's a few things that I'd like to see be done:
>>>> 1) With one setting, I should be able to turn on validation for
>>>> all my columns that have a type set for them. So, if I have a
>>>> column that is an integer, that should just turn on for me.
>>> Ok, on which level ? For a single model, for all models with
>>> columns of a certain type globaly (see 3) or at runtime (eg
>>> before the actual update/create.. something like
>>> $object->validates( $data_ref, { exclude_types => [ qw/ integer /
>>> ] } ) ? What use-cases can you imagine for this (for my better
>>> understanding of the requirements) ? For my use, the decision
>>> which validator(s) or none at all to use was sufficient, so far,
>>> but that doesnt mean anything ..
>> Frankly, I am of the opinion that if you have DataValidation as one
>>  of your components, any of your columns with a type should at
>> least validate against that type. This would seriously help with
>> SQLite usage.
>>
>>>> 2) I should be able to chain validations. So, something like
>>>> "validation => [ 'foo', 'bar' ]". Look at how Moose does this.
>>> I'm not quite sure wheter this is not already implemented. Right
>>> now, you can provide multiple validators for
>>> (update|update_all|create)_validated and the validates method
>>> itself. These will by applied one after another, thus chained.
>>> Eg: $object->validates( $data_ref, qw/ validator1 validator2 / );
>>>  That's what you meant, or am i missing the point ?
>> That works.
>>
>>>> 3) Different DBMSes have different types. So, maybe
>>>> producer-specific validations would be helpful (q.v. #1).
>>> Sounds very intersting. I could imagine validation-set-modules,
>>> one for every DBMS, like DBIx::Class::DataValidator::MySQLTypes.
>>> Then one should be able to enable/disable a subset of validation
>>> types.. eg __PACKAGE__->mysql_validation_enable( qw/ :all / );
>>> __PACKAGE__->mysql_validation_disable( qw/ date datetime / ); If
>>> you do this in your super-model (from which all inherit) this
>>> would apply globally. If you switch the DBMS now, you simple load
>>> another module like DBIx::Class::DataValidator::PostgresTypes or
>>> so and all the special columns for this DBMS will be validated.
>> Bah. Autoload that crap based on the storage_type I already told
>> DBIC about in my DSN. Don't make me hardcode anything.
>>
>> Rob
>>
>
> Alright then, i'll look into it at the weekend. The validation via
> DBMS determiniation looks like a goode idea. Thanks.
>

Ok, i've updated the modules concerning your ideas. A db-type aware validation would be auto-loaded and applied, if not disabled. But, as Eden wrote, the DBMS stuff will be added in DBIC 0.9 anyway, so i kept it most simplistic and modular, that it can be removed later on and doesnt make me to waste much time now.
Further, i'm not sure "how much validation" will be part of 0.9 and if this module (even without the DBMS stuff) is not totally obsolete when DBIC 0.9 releases. However, i need validation (especially for profiles) right now and cant wait for the release. Could'nt find any any date, though.

I've updated the documentation to explain the ideas and possibilities of this module. Following i've pasted an excerpt, which should explain most of it briefly and doesnt make me write it twice:

***********************
[...] a short introduction of the multiple approaches and methods you can follow to achieve validation.

Most of them could be combined and are not "either-or".

* Validation profiles
  These are ment to be used in most complex validation scenarios. You can validate the data with multiple profiles (one after another, chained)
  with one call.

* Inline column validation
  Directly in the column definition via add_column they are most simple and fast. Also the definition of the columns is not apart from the defi‐
  nition of the validation.

  They are limited because they have to fit always (each save) and cannot be used, as validation profiles, only in certain circumstances.

I'm not completely sure what you mean by this.  Are you trying to say that column validation is used for both inserts and updates no matter what profile you are using?
 

  You can easily use them in addition to validation profiles or exclusive. In the first case they’ll run before the validation profiles.

* DBMS specific validation
  This is a kind of auto-validation for basic datatypes of your column. They are DBMS dependent. So Oracle DB might have some types which Post‐
  greSQL has not and vice versa. Also the dates for PostgreSQL DBs and for MySQL DBs look different.

  They normally apply per default if you dont deactivate them manually (see below). They are overwritten by your custom inline column
  validation.

  They can be understand as a kind of default inline column validation.

* Named validation methods
  Those could be used either in inline column validation or validation profiles alike.  This is simply an approach to outsource often used vali‐
  dation methods or regular expressions to reuse them in multiple models and/or for multiple columns.

* Validation column classes
  Another approach to reduce code in your models by pre-defining.  This includes more then the basic datatypes (you can say each integer column
  is an integer-class column) by allowing custom and complex validations.

  They can also used for non validation purposes (just dont use the validation key). Named validation methods are of course usable.

  Probably you should define those classes in a super model from which all your other models inherit to get the most benefit out of it.
***********************

Please read the POD in DBIx::Class::DataValidator for more info..

Despite what i wrote above about 0.9, i think i could provide DBMS specific validation for MySQL and PostgreSQL before i release this to CPAN (but in sepparate packages, due to dependencies), but first of all i would like to hear what you all think  about the usefullness at all.

Nice weekend to everybody!
Ulrich

Thanks for spending your time on this!  I looked through your tests and briefly read some code and I think this will do quite nicely.  Just a couple questions: does it only validate when you call create_validated or can you have create validate by default?  Same for update and the $rs version of update.
--
fREW Schmidt
http://blog.afoolishmanifesto.com

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
--%<--

>> [...] a short introduction of the multiple approaches and methods
>> you can follow to achieve validation.
>>
>> Most of them could be combined and are not "either-or".
>>
>> * Validation profiles These are ment to be used in most complex
>> validation scenarios. You can validate the data with multiple
>> profiles (one after another, chained) with one call.
>>
>> * Inline column validation Directly in the column definition via
>> add_column they are most simple and fast. Also the definition of
>> the columns is not apart from the defi‐ nition of the validation.
>>
>> They are limited because they have to fit always (each save) and
>> cannot be used, as validation profiles, only in certain
>> circumstances.
>>
>
> I'm not completely sure what you mean by this.  Are you trying to say
> that column validation is used for both inserts and updates no matter
> what profile you are using?

Yes, column validation always applies! In opposite to profiles you can't trigger them on or off, especially there is no trigger to switch only certain column validations on or off (as you can do with profile validation). Excuse my poor english, little bit out of practice, i'll try to write it more clearly..


--%<--

>>
>> Please read the POD in DBIx::Class::DataValidator for more info..
>>
>> Despite what i wrote above about 0.9, i think i could provide DBMS
>> specific validation for MySQL and PostgreSQL before i release this
>> to CPAN (but in sepparate packages, due to dependencies), but first
>> of all i would like to hear what you all think  about the
>> usefullness at all.
>>
>> Nice weekend to everybody! Ulrich
>
>
> Thanks for spending your time on this!  I looked through your tests
> and briefly read some code and I think this will do quite nicely.
> Just a couple questions: does it only validate when you call
> create_validated or can you have create validate by default?  Same
> for update and the $rs version of update.

No problem, actually i'm using this - so i do benefit myself ;)

Yes, validation is only performed on (update|update_all|create)_validated. Maybe this is a based on the development history. I merged a (Catalyst) app, where i use this, slowy from controller validation to model validation. However, i think its more readable if you still use both (validation and no validation) - especially for the profile validation (at which you provide the profile names as 2++ arguments), but i can implement an "enable_native_validation"-switch, or so. A possibility to use the "non-validation"-methods should always exist.


Ulrich


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
fREW Schmidt

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink


On Sat, Jul 25, 2009 at 1:31 PM, Ulrich Kautz <[hidden email]> wrote:
--%<--
>> [...] a short introduction of the multiple approaches and methods
>> you can follow to achieve validation.
>>
>> Most of them could be combined and are not "either-or".
>>
>> * Validation profiles These are ment to be used in most complex
>> validation scenarios. You can validate the data with multiple
>> profiles (one after another, chained) with one call.
>>
>> * Inline column validation Directly in the column definition via
>> add_column they are most simple and fast. Also the definition of
>> the columns is not apart from the defi‐ nition of the validation.
>>
>> They are limited because they have to fit always (each save) and
>> cannot be used, as validation profiles, only in certain
>> circumstances.
>>
>
> I'm not completely sure what you mean by this.  Are you trying to say
> that column validation is used for both inserts and updates no matter
> what profile you are using?

Yes, column validation always applies! In opposite to profiles you can't trigger them on or off, especially there is no trigger to switch only certain column validations on or off (as you can do with profile validation). Excuse my poor english, little bit out of practice, i'll try to write it more clearly..

Well, at some point I'm sure that once this is in a public repository you could convince some of us to do some doc for you, that way you can focus on what you're already good at :-)
 


--%<--
>>
>> Please read the POD in DBIx::Class::DataValidator for more info..
>>
>> Despite what i wrote above about 0.9, i think i could provide DBMS
>> specific validation for MySQL and PostgreSQL before i release this
>> to CPAN (but in sepparate packages, due to dependencies), but first
>> of all i would like to hear what you all think  about the
>> usefullness at all.
>>
>> Nice weekend to everybody! Ulrich
>
>
> Thanks for spending your time on this!  I looked through your tests
> and briefly read some code and I think this will do quite nicely.
> Just a couple questions: does it only validate when you call
> create_validated or can you have create validate by default?  Same
> for update and the $rs version of update.

No problem, actually i'm using this - so i do benefit myself ;)

Yes, validation is only performed on (update|update_all|create)_validated. Maybe this is a based on the development history. I merged a (Catalyst) app, where i use this, slowy from controller validation to model validation. However, i think its more readable if you still use both (validation and no validation) - especially for the profile validation (at which you provide the profile names as 2++ arguments), but i can implement an "enable_native_validation"-switch, or so. A possibility to use the "non-validation"-methods should always exist.

I think that sounds great.  enable_native_validation probably wouldn't complicate things significantly and having  *_non_validated (or something like that) methods would allow you to bypass it if you really need to.  If you'd like some help with doc, tests, or smallish features I'd be glad to throw some time into this. 

 
--
fREW Schmidt
http://blog.afoolishmanifesto.com

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Ulrich Kautz

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink

>> --%<--
>>>> [...] a short introduction of the multiple approaches and
>>>> methods you can follow to achieve validation.
>>>>
>>>> Most of them could be combined and are not "either-or".
>>>>
>>>> * Validation profiles These are ment to be used in most complex
>>>>  validation scenarios. You can validate the data with multiple
>>>> profiles (one after another, chained) with one call.
>>>>
>>>> * Inline column validation Directly in the column definition
>>>> via add_column they are most simple and fast. Also the
>>>> definition of the columns is not apart from the defi‐ nition of
>>>> the validation.
>>>>
>>>> They are limited because they have to fit always (each save)
>>>> and cannot be used, as validation profiles, only in certain
>>>> circumstances.
>>>>
>>> I'm not completely sure what you mean by this.  Are you trying to
>>> say that column validation is used for both inserts and updates
>>> no matter what profile you are using?
>> Yes, column validation always applies! In opposite to profiles you
>> can't trigger them on or off, especially there is no trigger to
>> switch only certain column validations on or off (as you can do
>> with profile validation). Excuse my poor english, little bit out of
>> practice, i'll try to write it more clearly..
>>
>
> Well, at some point I'm sure that once this is in a public repository
> you could convince some of us to do some doc for you, that way you
> can focus on what you're already good at :-)
Yeah, that would be nice :)

>>
>> --%<--
>>>> Please read the POD in DBIx::Class::DataValidator for more
>>>> info..
>>>>
>>>> Despite what i wrote above about 0.9, i think i could provide
>>>> DBMS specific validation for MySQL and PostgreSQL before i
>>>> release this to CPAN (but in sepparate packages, due to
>>>> dependencies), but first of all i would like to hear what you
>>>> all think  about the usefullness at all.
>>>>
>>>> Nice weekend to everybody! Ulrich
>>>
>>> Thanks for spending your time on this!  I looked through your
>>> tests and briefly read some code and I think this will do quite
>>> nicely. Just a couple questions: does it only validate when you
>>> call create_validated or can you have create validate by default?
>>> Same for update and the $rs version of update.
>> No problem, actually i'm using this - so i do benefit myself ;)
>>
>> Yes, validation is only performed on
>> (update|update_all|create)_validated. Maybe this is a based on the
>> development history. I merged a (Catalyst) app, where i use this,
>> slowy from controller validation to model validation. However, i
>> think its more readable if you still use both (validation and no
>> validation) - especially for the profile validation (at which you
>> provide the profile names as 2++ arguments), but i can implement an
>>  "enable_native_validation"-switch, or so. A possibility to use the
>>  "non-validation"-methods should always exist.
>>
>
> I think that sounds great.  enable_native_validation probably
> wouldn't complicate things significantly and having  *_non_validated
> (or something like that) methods would allow you to bypass it if you
> really need to.  If you'd like some help with doc, tests, or smallish
> features I'd be glad to throw some time into this.
You are of course more then welcome, if you'd like to help. Actually i think, i've spent a lot more time in documenting then acutally writing the code and the tests - and yet, its no good. For now, IMO the v0.02 version is not yet tested enough to upload it to CPAN. I could provide a svn repo (maybe on some free hoster like assembla.com or on my own machines). Then you can add your code additions and hopefully run and write some tests. How does this sound ?

I've attached the updated module with the native-validtion thingy.


Ulrich


_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...

DBIx-Class-DataValidator-0.02.tar.bz2 (20K) Download Attachment
Rob Kinyon

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
In reply to this post by Alexander Hartmaier
On Thu, Jul 23, 2009 at 08:28, Alexander
Hartmaier<[hidden email]> wrote:
> The DBMS validates data types, length, undef and foreign keys anyway, so
> maybe using it as validator (parse its error messages) is a possibility?

The problem is that many DBMSes do -NOT- validate things sanely -
SQLite and MySQL being the biggest offenders, though every DBMS has
its problems. Oracle, for example, in 9.2.0.3 had an issue where
certain invalid datetime entries were accepted. Furthermore, part of
the benefit of an ORM is that it is DBMS-independent - I want to know
that the same value restrictions are going to be maintained regardless
of backend.

And, finally, the biggest win for validation is things that DBMSes
cannot do. For example, a SMALLINT that, for business reasons, has to
be within 20 of another column. How do you express that constraint in
a DBMS-independent fashion?

I, for one, cannot wait for our new Moose overlords!

Rob

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
Alexander Hartmaier

Re: DBIx::Class, another data validation approach

Reply Threaded More More options
Print post
Permalink
Am Dienstag, den 28.07.2009, 00:29 +0200 schrieb Rob Kinyon:

> On Thu, Jul 23, 2009 at 08:28, Alexander
> Hartmaier<[hidden email]> wrote:
> > The DBMS validates data types, length, undef and foreign keys anyway, so
> > maybe using it as validator (parse its error messages) is a possibility?
>
> The problem is that many DBMSes do -NOT- validate things sanely -
> SQLite and MySQL being the biggest offenders, though every DBMS has
> its problems. Oracle, for example, in 9.2.0.3 had an issue where
> certain invalid datetime entries were accepted. Furthermore, part of
> the benefit of an ORM is that it is DBMS-independent - I want to know
> that the same value restrictions are going to be maintained regardless
> of backend.
>
> And, finally, the biggest win for validation is things that DBMSes
> cannot do. For example, a SMALLINT that, for business reasons, has to
> be within 20 of another column. How do you express that constraint in
> a DBMS-independent fashion?
>
> I, for one, cannot wait for our new Moose overlords!
>
> Rob
I didn't mean that we don't need business logic or extended validation,
but use the basic validation the db does anyway instead of cloning it
which could lead to different checks and a performance penalty (i had to
disable validation for an app that inserts lots of data because it
wasn't fast enough).

If a specific db (version) is buggy we can warn the user and suggest an
upgrade.

--
BR Alex


*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
T-Systems Austria GesmbH   Rennweg 97-99, 1030 Wien
Handelsgericht Wien, FN 79340b
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*
Notice: This e-mail contains information that is confidential and may be privileged.
If you are not the intended recipient, please notify the sender and then
delete this e-mail immediately.
*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*"*

_______________________________________________
List: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbix-class
IRC: irc.perl.org#dbix-class
SVN: http://dev.catalyst.perl.org/repos/bast/DBIx-Class/
Searchable Archive: http://www.grokbase.com/group/dbix-class@...
1 2