OSGeo.org › OSGeo Software and Data Projects › Feature Data Objects (FDO) › FDO Internals

SDF feature reader

16 messages

Haris Kurtagic

SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Badreddine Karoui

RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic
Sent: Wednesday, August 13, 2008 2:29 PM
To: FDO Internals Mail List
Subject: [fdo-internals] SDF feature reader

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Gavin Cramer

RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

Yes. The particular problem that you are seeing can also happen with String values, since the pointer returned by GetString is to a buffer owned by the reader. Per-connection threading for any provider (that supports this kind of threading, as determined by FdoIConnectionCapabilities::GetThreadCapability) is the way to go.

Gavin

From: [hidden email] [mailto:[hidden email]] On Behalf Of Badreddine Karoui
Sent: Wednesday, August 13, 2008 2:18 PM
To: FDO Internals Mail List
Subject: [fdo-internals] RE: SDF feature reader

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Haris Kurtagic

RE: RE: SDF feature reader

Reply Threaded

In reply to this post by Badreddine Karoui

Some javascript/style in this post has been disabled (why?)

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

From: [hidden email] [mailto:[hidden email]] On Behalf Of Badreddine Karoui
Sent: Wednesday, August 13, 2008 8:18 PM
To: FDO Internals Mail List
Subject: [fdo-internals] RE: SDF feature reader

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Badreddine Karoui

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

It was not obvious that the readers were on the same thread. In that case it’s a defect. Make sure to copy the geometry before ReadNext is called. It’s possible the second reader(ReadNext) is invalidating the geometry.

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic
Sent: Wednesday, August 13, 2008 2:39 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Haris Kurtagic

RE: RE: SDF feature reader

Reply Threaded

In reply to this post by Gavin Cramer

Some javascript/style in this post has been disabled (why?)

I haven’t looked that carefully into GetString but it looks that BinaryReader do some string buffering.

Anyhow, I am talking now only about single thread use with multiple feature readers.

Threading is not issue for that case.

Also, I would like to understand why SDF provider is not thread safe and to look into possibilities

I think that sqlite version used with SDF allows thread safe access.

Thank you for your help,

Haris

From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin Cramer
Sent: Wednesday, August 13, 2008 8:23 PM
To: FDO Internals Mail List
Subject: [fdo-internals] RE: SDF feature reader

Gavin

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Haris Kurtagic

RE: RE: SDF feature reader

Reply Threaded

In reply to this post by Badreddine Karoui

Some javascript/style in this post has been disabled (why?)

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

From: [hidden email] [mailto:[hidden email]] On Behalf Of Badreddine Karoui
Sent: Wednesday, August 13, 2008 8:36 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Badreddine Karoui

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

I had a quick look and I can see that the SDFSimpleFeatureReader::GetGeometry(FdoString*, FdoInt32*) is returing the pointer from the internal buffer. This may have been done to avoid paying the penalty of copying the geometry. The second oveload, GetGeometry(FdoString*) is returing a copy and should not have the same issue.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic
Sent: Wednesday, August 13, 2008 3:12 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Gavin Cramer

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

So, the code is behaving according to the specification. There should be a similar statement for GetString, as well as any other accessor that returns a pointer to non-reference-counted arrays, but I do not see one. Anyway, an FDO client should expect the above behaviour from any FDO provider, even if it actually does some internal copying.

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

From: [hidden email] [mailto:[hidden email]] On Behalf Of Badreddine Karoui
Sent: Wednesday, August 13, 2008 4:14 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Haris Kurtagic

RE: RE: SDF feature reader

Reply Threaded

In reply to this post by Badreddine Karoui

Some javascript/style in this post has been disabled (why?)

Ok, I read that .h many times :)

I would appreciate if you can help me to understand and discuss why sdf provider is not FdoThreadCapability_PerCommandThreaded.

Thank you,

Haris

From: [hidden email] [mailto:[hidden email]] On Behalf Of Badreddine Karoui
Sent: Wednesday, August 13, 2008 10:14 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Traian Stanev

RE: RE: SDF feature reader

Reply Threaded

In reply to this post by Gavin Cramer

Some javascript/style in this post has been disabled (why?)

This behavior is really a tradeoff between performance and full thread safety. SDF is near the performance end of the spectrum. I figure, if someone desires full

thread safety, they can use a real database.

Traian

From: [hidden email] [mailto:[hidden email]] On Behalf Of Gavin Cramer
Sent: Wednesday, August 13, 2008 4:27 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Haris Kurtagic

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

Hm, but problem is not thread related.

Having 2 feature readers for same SDF connection inside one thread will create invalid data.

Haris

From: [hidden email] [mailto:[hidden email]] On Behalf Of Traian Stanev
Sent: Wednesday, August 13, 2008 10:32 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

This behavior is really a tradeoff between performance and full thread safety. SDF is near the performance end of the spectrum. I figure, if someone desires full

thread safety, they can use a real database.

Traian

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Traian Stanev

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

Yes, this is an intentional limitation, for the same reason (performance).

Traian

From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic
Sent: Wednesday, August 13, 2008 4:53 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

Hm, but problem is not thread related.

Having 2 feature readers for same SDF connection inside one thread will create invalid data.

Haris

This behavior is really a tradeoff between performance and full thread safety. SDF is near the performance end of the spectrum. I figure, if someone desires full

thread safety, they can use a real database.

Traian

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Badreddine Karoui

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

There is probably a way to fix this issue and only make a copy if 2 readers are used.

Badreddine

From: [hidden email] [mailto:[hidden email]] On Behalf Of Traian Stanev
Sent: Wednesday, August 13, 2008 4:39 PM
To: 'FDO Internals Mail List'
Subject: RE: [fdo-internals] RE: SDF feature reader

Yes, this is an intentional limitation, for the same reason (performance).

Traian

Hm, but problem is not thread related.

Having 2 feature readers for same SDF connection inside one thread will create invalid data.

Haris

This behavior is really a tradeoff between performance and full thread safety. SDF is near the performance end of the spectrum. I figure, if someone desires full

thread safety, they can use a real database.

Traian

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Haris Kurtagic

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

if i interpret code correctly there was allready attempt to solve issue with multiple readers with “Tag” in DbData and call to function RefreshData before every Get.. in feature reader.

I would like to ask is why only one sqlite cursor is used for one sqlite table for all readers. That looks like potential source of problem ?

Create new sqlite cursor for every reader ?

Haris

From: [hidden email] [mailto:[hidden email]] On Behalf Of Badreddine Karoui
Sent: Wednesday, August 13, 2008 10:47 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

There is probably a way to fix this issue and only make a copy if 2 readers are used.

Badreddine

Yes, this is an intentional limitation, for the same reason (performance).

Traian

Hm, but problem is not thread related.

Having 2 feature readers for same SDF connection inside one thread will create invalid data.

Haris

This behavior is really a tradeoff between performance and full thread safety. SDF is near the performance end of the spectrum. I figure, if someone desires full

thread safety, they can use a real database.

Traian

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals

Badreddine Karoui

RE: RE: SDF feature reader

Reply Threaded

Some javascript/style in this post has been disabled (why?)

I think the reason is the fact that the SDF provider keeps an update buffer for every table. All readers are using the same cursor to avoid flushing the buffer everytime a query is executed. Writing and reading at the same time is a common use case.

Badreddine

From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic
Sent: Wednesday, August 13, 2008 5:16 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader

if i interpret code correctly there was allready attempt to solve issue with multiple readers with “Tag” in DbData and call to function RefreshData before every Get.. in feature reader.

I would like to ask is why only one sqlite cursor is used for one sqlite table for all readers. That looks like potential source of problem ?

Create new sqlite cursor for every reader ?

Haris

There is probably a way to fix this issue and only make a copy if 2 readers are used.

Badreddine

Yes, this is an intentional limitation, for the same reason (performance).

Traian

Hm, but problem is not thread related.

Having 2 feature readers for same SDF connection inside one thread will create invalid data.

Haris

This behavior is really a tradeoff between performance and full thread safety. SDF is near the performance end of the spectrum. I figure, if someone desires full

thread safety, they can use a real database.

Traian

GetGeometry (the one returning an FdoByte *) has documentation that says the following:

“The array's memory area is only guaranteed to be valid until a call to ReadNext() or Close(), or the disposal of this reader object.”

Regardless, it looks like the behaviour that Haris sees is a defect. Different readers should not be sharing an internal cursor.

Gavin

The free threaded provider would use FdoThreadCapability_MultiThreaded defined in https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabilities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

We may have to be extra carefull in case we decide to fix this issue to make sure the reader performance is not impacted.

Badreddine

it looks that feature readers are sharing same sqlite cursor with one data buffer. (BTW: Application is sending this readers to be used by another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can’t be shared across threads, right ? ( I thought that as thread safe )

I would like to understand reasons for that behavior and if there are rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

The SDF provider is thread safe but not free threaded, it’s per-connection threaded.

Badreddine

I was saying for two feature readers inside same thread will result in error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

The SDF provider is per connection threaded which means only one thread can access a reader at a time. You can have many threads using their own connections/readers at the same time.

Badreddine

Hi,

I have issue when when more than one feature reader is created against same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean feature readers.

This will cause that data returned from feature reader with function GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not “thread safe” ? if so, what are main reasons for it ?

Thank you,

Haris

_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals