spatial restriction for queries

[Martin Kofahl]

Hello!
I'm trying to complete enhancement #3116 (spatial oblogation per layer basis). Queries should be limited to the are a user is allowed to access. Because of the number of query functions, I need some help to figure out where the best place for checking access rights is. I'd manage this using msGEOSWithin().

For vector data all query function seem to depend on msLayerWhichShapes. If so, could MapServer 'simply' filter the list of shapes to be queried?

For raster data I'm still looking if there's a single core function where the check could be done.

On the other hand, I could extend the msQuery functions in mapquery.c in order to check the incoming selectshape or - for attribute search etc. - removing some items from the resultset afterwards.

Can you give me some advice? Thanks!
Martin
_______________________________________________
mapserver-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapserver-dev

[Tamas Szekeres]

Martin,

I think you should flash out further information about your objectives
there are a couple of things not obvious for me with this proposal;

1. I'm not sure where are the user identities stored when restricting
their access to regions. It seems like the mapfile would contain
constant restrictions independent from the users. How the users are
assigned to regions?

2. If we are thinking about static mapfiles (different mapfile for
each user) it seems like this problem is running out of the scope of
mapserver. You may pre-define those mapfiles by setting up different
spatial filters for each regions. The upper level application (ie. a
WEB application) would identify the user any refer to different
mapfiles accordingly.

3. It looks like you'd implement to render only a subset of the
features per-layer within a spatial query. How this option is
different as what the QUERYMAP setting with STYLE=selected would
normally provide? (http://mapserver.org/mapfile/querymap.html) the
query maps could be drawn by the cgi mapserver or by using a custom
script (mapscript) within your application.


Best regards,

Tamas





2009/10/2 Martin Kofahl <[hidden email]>:

> Hello!
> I'm trying to complete enhancement #3116 (spatial oblogation per layer basis). Queries should be limited to the are a user is allowed to access. Because of the number of query functions, I need some help to figure out where the best place for checking access rights is. I'd manage this using msGEOSWithin().
>
> For vector data all query function seem to depend on msLayerWhichShapes. If so, could MapServer 'simply' filter the list of shapes to be queried?
>
> For raster data I'm still looking if there's a single core function where the check could be done.
>
> On the other hand, I could extend the msQuery functions in mapquery.c in order to check the incoming selectshape or - for attribute search etc. - removing some items from the resultset afterwards.
>
> Can you give me some advice? Thanks!
> Martin
> _______________________________________________
> mapserver-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>

_______________________________________________
mapserver-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapserver-dev

[Martin Kofahl]

Hi Tamas,
my intention is to have a simple access control mechanism (so a clipping
mechanism per layer based on user credentials or other id-like data)
inside MapServer. For now this plan is limited to raster data, so for
wms and browse mode. Currently, I have an access control system (a kind
of cascading wms) in front of MapServer which increases response times
noticeable.


What I have done so far:

* user credentials: In order to use the ID of a user accessing MapServer
(no, I don't want to have a single mapfile for each potential user) I
found it handy to have access to environment variables in the DATA
statement such as remote_ip, remote_user or other things one could set
before running MapServer. This is done in
http://trac.osgeo.org/mapserver/ticket/3122 and - from my point of view
- only needs some syntax discussion and a solution to validate such
variables as it's done for url substitution. This will then allow
interesting solutions: drawing/querying only a subset of a sql
datasource by using a joined credentials-table in the where clause
(actually, I haven't tried this way round).

* clipping mechanism: As #3122 only works for sql data sources and I
have a couple of raster data sources which have to be clipped based on
user-id, I use a special layer which holds the area a user have access
to (using #3122). This layer is referenced by the real data-layer
similar to a tile-index layer. The negative of the referenced layer is
drawn other the already rendered data-layer. The patch is enclosed to
http://trac.osgeo.org/mapserver/ticket/3116. Contrary to #3122 this
solution has the advantage that it works for all data sources. Up to now
it's not perfect, but it works as expected.

What I am missing: you can still query a clipped area. And this is where
my question was aimed at. _I think_, I'll have to check if the
query-shape(s) is/are inside the polygons of the referenced layer used
for clipping. But there are so many different msQuery.. functions; if
avoidable, I don't want to alter them all and currently looking for the
some 'core functions'.


In reply to your 2nd question: The reason why - as far I know - I cannot
use different mapfiles for each spatial restriction is that I have tiff
files as data source and polygons for restrictions. And personally, I
don't feel it's a good idea to have tons of similar mapfiles for each
combination of about 20 areas.

Querymaps: I must say I didn't use this feature for now. But it's only
for drawing pre-selected shapes (so vector data sources only), right?
How could querymaps be used in wms mode (not only highlighting but
clipping out the rest)? Can I use it in order to limit queries to given
polygons? Reading the docs, I think querymaps have another target.


What is your (and other MapServer devs') opinion about a having a
spatial access control system inside MapServer?

Kind regards! Martin


Tamas Szekeres wrote:

> Martin,
>
> I think you should flash out further information about your objectives
> there are a couple of things not obvious for me with this proposal;
>
> 1. I'm not sure where are the user identities stored when restricting
> their access to regions. It seems like the mapfile would contain
> constant restrictions independent from the users. How the users are
> assigned to regions?
>
> 2. If we are thinking about static mapfiles (different mapfile for
> each user) it seems like this problem is running out of the scope of
> mapserver. You may pre-define those mapfiles by setting up different
> spatial filters for each regions. The upper level application (ie. a
> WEB application) would identify the user any refer to different
> mapfiles accordingly.
>
> 3. It looks like you'd implement to render only a subset of the
> features per-layer within a spatial query. How this option is
> different as what the QUERYMAP setting with STYLE=selected would
> normally provide? (http://mapserver.org/mapfile/querymap.html) the
> query maps could be drawn by the cgi mapserver or by using a custom
> script (mapscript) within your application.
>
>
> Best regards,
>
> Tamas
>
>
>
>
>
> 2009/10/2 Martin Kofahl <[hidden email]>:
>  
>> Hello!
>> I'm trying to complete enhancement #3116 (spatial oblogation per layer basis). Queries should be limited to the are a user is allowed to access. Because of the number of query functions, I need some help to figure out where the best place for checking access rights is. I'd manage this using msGEOSWithin().
>>
>> For vector data all query function seem to depend on msLayerWhichShapes. If so, could MapServer 'simply' filter the list of shapes to be queried?
>>
>> For raster data I'm still looking if there's a single core function where the check could be done.
>>
>> On the other hand, I could extend the msQuery functions in mapquery.c in order to check the incoming selectshape or - for attribute search etc. - removing some items from the resultset afterwards.
>>
>> Can you give me some advice? Thanks!
>> Martin
>> _______________________________________________
>> mapserver-dev mailing list
>> [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
>>    

_______________________________________________
mapserver-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapserver-dev

[Tamas Szekeres]

2009/10/3 Martin Kofahl <[hidden email]>

What is your (and other MapServer devs') opinion about a having a spatial access control system inside MapServer?

Martin,

I admit I haven't studied all aspects of this proposal, but it looks like as it would be an example of an "extension layer" addressed by one of my previous RFCs,
http://mapserver.org/development/rfc/ms-rfc-22a.html

(Note: The title of this RFC doesn't completely reflect the proposed concept, the feature cache would only be just one simple usage example of that. I should probably rename this RFC to something like: "Support to implement extension layers").

In this case the the layer would be defined in a cascaded fashion in the map file. The outer layer would be able override the vtable functions of the inner layer, therefore a subset of the features could also be hidden according to a custom logic when displaying the whole layer. I this regard, it wouldn't be required to define 2 sibling layers (which is quite annoying here), the outer layer would contain the select shape, and the vtable functions could eventually implement reading the env variables as required.

In case if this common infrastructure would exist in MapServer, then you'd only require to write your custom layer data provider (ie. the vtable functions of a new layer type) which could eventially be a plugin (independent from the mapserver codebase).

By all means, my preference here would be to establish a common infrastucture where various special requirements could easily be implemented without affecting the core funtionality too much. Your intents seems to be just a simple use case of the need to extend/override the functionality of the existing layer data sources which should be handled in a common way.

At that time when this proposal have been created (along with working patches, see the corresponding ticket) I couldn't gather any community support to make it realized in the development version. At this time I'm not too keen to spend more efforts on this, however I would personally support any effort to implement a concept which could be extensible such way. 


Best regards,

Tamas



 

_______________________________________________
mapserver-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapserver-dev

[Martin Kofahl]

Hi Tamas,
this is a very interesting rfc. I'm still reading through your patches in order to follow the examples as I haven't heard of vtables yet. Can you please clarify some things for me?

- Any filterings are probably done in mapfilter.c, currently for MS_SHAPE_POINT/LINE/POLYGON. Does this has to be extended for all other types including raster?

- There's a mail on the list saying that performance is quite slow. Have you done any measurements?

- There's no vote on this rfc. Does it make sense to work on it, so how are the chances that your concept will go into the development version.

- Can you give an abstract of work which is to be done?

Thank you!
Martin

-------- Original-Nachricht --------
> Datum: Sun, 4 Oct 2009 20:48:08 +0200
> Von: Tamas Szekeres <[hidden email]>
> An: Martin Kofahl <[hidden email]>
> CC: [hidden email]
> Betreff: Re: [mapserver-dev] spatial restriction for queries

> 2009/10/3 Martin Kofahl <[hidden email]>
>
> > What is your (and other MapServer devs') opinion about a having a
> spatial
> > access control system inside MapServer?
> >
> >
> Martin,
>
> I admit I haven't studied all aspects of this proposal, but it looks like
> as
> it would be an example of an "extension layer" addressed by one of my
> previous RFCs,
> http://mapserver.org/development/rfc/ms-rfc-22a.html
>
> (Note: The title of this RFC doesn't completely reflect the proposed
> concept, the feature cache would only be just one simple usage example of
> that. I should probably rename this RFC to something like: "Support to
> implement extension layers").
>
> In this case the the layer would be defined in a cascaded fashion in the
> map
> file. The outer layer would be able override the vtable functions of the
> inner layer, therefore a subset of the features could also be hidden
> according to a custom logic when displaying the whole layer. I this
> regard,
> it wouldn't be required to define 2 sibling layers (which is quite
> annoying
> here), the outer layer would contain the select shape, and the vtable
> functions could eventually implement reading the env variables as
> required.
>
> In case if this common infrastructure would exist in MapServer, then you'd
> only require to write your custom layer data provider (ie. the vtable
> functions of a new layer type) which could eventially be a plugin
> (independent from the mapserver codebase).
>
> By all means, my preference here would be to establish a common
> infrastucture where various special requirements could easily be
> implemented
> without affecting the core funtionality too much. Your intents seems to be
> just a simple use case of the need to extend/override the functionality of
> the existing layer data sources which should be handled in a common way.
>
> At that time when this proposal have been created (along with working
> patches, see the corresponding ticket) I couldn't gather any community
> support to make it realized in the development version. At this time I'm
> not
> too keen to spend more efforts on this, however I would personally support
> any effort to implement a concept which could be extensible such way.
>
>
> Best regards,
>
> Tamas

_______________________________________________
mapserver-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapserver-dev

[Tamas Szekeres]

2009/10/9 Martin Kofahl <[hidden email]>

Hi Tamas,
this is a very interesting rfc. I'm still reading through your patches in order to follow the examples as I haven't heard of vtables yet. Can you please clarify some things for me?

- Any filterings are probably done in mapfilter.c, currently for MS_SHAPE_POINT/LINE/POLYGON. Does this has to be extended for all other types including raster?

Martin,

The concept of RFC22  (http://mapserver.org/development/rfc/ms-rfc-22a.html) is related only to the 'data source' vtable, and we could only filter vector data (ie. shapes) by using this approach. With regards to the raster data this could provide an option only to filter the tiles by the tile indexes (when using tiled rasters).

If you're about to exclude arbitrary regions then you should probably plug in to the renderer vtable (rendererVTable) which is a pretty new concept in MapServer. I could imagine an intermediary renderer which would be capable to modify the image rendered by another renderer. However this option is not covered by RFC22 at all.

 

- There's a mail on the list saying that performance is quite slow. Have you done any measurements?

I'm not aware of much issues with the performance of this concept itself. I guess you're pointing to the performance issue of the 2-phase query behaviour of MapServer which would be one of the issues addressed by RFC22. I must mention that this problem has already been addressed by MapServer in another way (see: http://mapserver.org/development/rfc/ms-rfc-52.html)

- There's no vote on this rfc. Does it make sense to work on it, so how are the chances that your concept will go into the development version.

Every significant change would normally require an RFC and should be approved by the MapServer PSC in a voting process. I would normally prefer to issue a vote if I had at least some of the PSC members support the concept, but it seems this one has lack of this support due to the presumed complexity of these additions, which would be considered to violate the designated objective of "MapServer is not a full-featured GIS system, nor does it aspire to be."  (http://mapserver.org/)

 

- Can you give an abstract of work which is to be done?

I think we should normally require:

1. An RFC to describe the required changes to satisfy with your needs to be able to discuss on
2. The RFC should be negotiated on the mapserver-dev list
3. Issue a vote on the RFC
4. If the voting is passed then implement the changes and modify the doc accordingly

I currently don't have enough motivation to proceed with this RFC, as currently I have no such requirements in my project actually. However I would still support having such feature in MapServer (it should probably be extended to the renderer vtables as well) if someone would feel enough power to do some polishing on that.


Best regards,

Tamas




_______________________________________________
mapserver-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapserver-dev