MG Object stream serialization/deserialization
|
|
Haris Kurtagic
|
Hi ,
I have lot of problems while trying to communicate with different versions of MapGuide. I am developing application which should communicate with MG using MG common libraries: MapGuideCommon, PlatformBase,... Problem is that I can't find two version of MG which will work one against other and work equally. For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch against current branch version. Also for example I can't use any of those versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while current 2.1 branch will throw exception InvalidStreamHeader. At least one of problems is that ClassDefinition in one version is serializing SchemaName in another it is not. So those versions can't be used one against each other. I have two questions: 1. I really can't' find consistency in our svn branches and trunk how and when is changing how MG objects are serialized/desterilized. I hope somebody can light me up little bit when/ why are this changes done. 2. We don't have any versioning of serialization process of objects. I think that could be issue ( hopefully one day we could imagine farm of MapGuide's ). Can we add versions of objects to serialization process ? I would appreciate any help/ideas how to solve this mess, how to have one application communicating with different MG versions. Thank you, Haris _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
UV-2
|
I very much support Haris ideas.
Versioning of serialized objects as well as the API calls and DLL versions is something that is state-of-the-art in the IT field. Those pieces of extra information help to work efficiently as it helps to avoid a number frustrating build & deployment scenarios. A seperate external text document containing configuration information tends to be outdated most of the time and arriving too late. One issue is the kind of implicit dependency on a certain FDO release. For the releases this piece of information is written down somewhere.... however I dont understand why the DLL loaders dont check those versions???? The versioning of serialized objects is another related issue. THose are approaches to write robuster code. A lot of such configuration data should be made part of the CI system. This is how those issues are addressed in successful IT projects as it makes the dependencies a part of the build which shows up as a TAG in the repository. Leaving this as loose as it is in the moment causes pain and more pain for too many people. I would call this a part of the evolutionary step from inhouse development to open source! In a small local team such formalisms can be avoided by asking the correct team member sitting near. However this does not scale well and in respect to the community those issues should be addresses more thoroughly at some point hopefully soon. UV Haris Kurtagic wrote: > Hi , > > > > I have lot of problems while trying to communicate with different versions > of MapGuide. I am developing application which should communicate with MG > using MG common libraries: MapGuideCommon, PlatformBase,... > > > > Problem is that I can't find two version of MG which will work one against > other and work equally. > > For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch > against current branch version. Also for example I can't use any of those > versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while > current 2.1 branch will throw exception InvalidStreamHeader. > > > > At least one of problems is that ClassDefinition in one version is > serializing SchemaName in another it is not. So those versions can't be used > one against each other. > > > > I have two questions: > > > > 1. I really can't' find consistency in our svn branches and trunk how and > when is changing how MG objects are serialized/desterilized. I hope somebody > can light me up little bit when/ why are this changes done. > > > > 2. We don't have any versioning of serialization process of objects. I think > that could be issue ( hopefully one day we could imagine farm of MapGuide's > ). > > Can we add versions of objects to serialization process ? > > > > I would appreciate any help/ideas how to solve this mess, how to have one > application communicating with different MG versions. > > > > Thank you, > > Haris > > _______________________________________________ > mapguide-internals mailing list > [hidden email] > http://lists.osgeo.org/mailman/listinfo/mapguide-internals > > _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
Kenneth Skovhede, GEOGRAF A/S
|
I agree that the API should be fully backwards compatible.
It's a real pain that the client must know in advance what version of the API to use. This is especially true for .Net which has a less-than-easy method for version selection on dll's. The HTTP API is fully version capable, but the native API (aka WebAPI) is not. The reason for this is that the native API is actually wrapping C++ classes. Wrapping these classes reproduce some functionality from the MapGuide server on the client (buffering, etc.). Since these C++ classes are not clearly defined, they change between releases without notice, breaking the API compatibility. If there was a version field, the serializer could take care of this, and select a proper default value for missing fields. Unfortunately, there is no serializer, only custom code inside the classes that do this, making it fairly difficult to maintain. I belive that the root issue is that the API is too deeply nested in the server. I would propose that the API sends binary versions of the same requests that the HTTP API sends. It would then be a matter of implementing a light interface that resembles the current http request parser, and reuses the same functions. I hear speed concerns incoming already :) The overhead for the HTTP API is significant, both because of the HTTP protocol itself, but also due to the http parameter parsing and the xml input/output. Both these issues could be resolved by using a binary format of the messages, and allowing the server to accept these binary messages directly (like the WebAPI does now). The binary format should not be a repetition of the current system, but rather a generic format, something similar to "binary JSON", so the actual class layout and message format are independant. Such a seperation is also much safer, as it is easier to unittest the binary parser, and the messages could be allowed over http, unlike the current WebAPI. Regards, Kenneth Skovhede, GEOGRAF A/S UV skrev: > I very much support Haris ideas. > Versioning of serialized objects as well as the API calls and DLL > versions is something that is state-of-the-art in the IT field. Those > pieces of extra information help to work efficiently as it helps to > avoid a number frustrating build & deployment scenarios. > > A seperate external text document containing configuration information > tends to be outdated most of the time and arriving too late. > > One issue is the kind of implicit dependency on a certain FDO release. > For the releases this piece of information is written down somewhere.... > however I dont understand why the DLL loaders dont check those > versions???? > > The versioning of serialized objects is another related issue. THose > are approaches to write robuster code. > > A lot of such configuration data should be made part of the CI system. > This is how those issues are addressed in successful IT projects as it > makes the dependencies > a part of the build which shows up as a TAG in the repository. > > Leaving this as loose as it is in the moment causes pain and more pain > for too many people. > I would call this a part of the evolutionary step from inhouse > development to open source! > In a small local team such formalisms can be avoided by asking the > correct team member sitting near. However this does not scale well and > in respect to the community those issues should be addresses > more thoroughly at some point hopefully soon. > > UV > > > > > Haris Kurtagic wrote: >> Hi , >> >> >> >> I have lot of problems while trying to communicate with different >> versions >> of MapGuide. I am developing application which should communicate >> with MG >> using MG common libraries: MapGuideCommon, PlatformBase,... >> >> >> >> Problem is that I can't find two version of MG which will work one >> against >> other and work equally. >> >> For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch >> against current branch version. Also for example I can't use any of >> those >> versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. >> 2010 while >> current 2.1 branch will throw exception InvalidStreamHeader. >> >> >> >> At least one of problems is that ClassDefinition in one version is >> serializing SchemaName in another it is not. So those versions can't >> be used >> one against each other. >> >> >> >> I have two questions: >> >> >> >> 1. I really can't' find consistency in our svn branches and trunk >> how and >> when is changing how MG objects are serialized/desterilized. I hope >> somebody >> can light me up little bit when/ why are this changes done. >> >> >> >> 2. We don't have any versioning of serialization process of objects. >> I think >> that could be issue ( hopefully one day we could imagine farm of >> MapGuide's >> ). >> >> Can we add versions of objects to serialization process ? >> >> >> >> I would appreciate any help/ideas how to solve this mess, how to have >> one >> application communicating with different MG versions. >> >> >> >> Thank you, >> >> Haris >> >> _______________________________________________ >> mapguide-internals mailing list >> [hidden email] >> http://lists.osgeo.org/mailman/listinfo/mapguide-internals >> >> > > _______________________________________________ > mapguide-internals mailing list > [hidden email] > http://lists.osgeo.org/mailman/listinfo/mapguide-internals mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
|
Haris Kurtagic
|
In reply to this post
by Haris Kurtagic
Hi Trevor,
I believe I found when ClassDefinition serialization has changed. Changeset 3905 <http://trac.osgeo.org/mapguide/changeset/3905> for trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp <http://trac.osgeo.org/mapguide/browser/trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp?rev=3905> Timestammp Author trevorwekel Ticket <http://trac.osgeo.org/mapguide/ticket/1002> #1002 Batch Processing for Get Identity Properties http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBase /Services/ClassDefinition.cpp@3905 <http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBas e/Services/ClassDefinition.cpp@3905&new=trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp@3905> &new=trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp@3905 It feels bit strange that in previous versions schema name from class def. was omitted ( but that is how 2.1 Beta and MG Ent 2010 works ). This change was really for that ticket "Batch Processing for Get Identity Properties" or ? I would like to be able to communicate with MG 2.0.2, MG 2.1 MG Ent. 2010 with one application using standard MG libraries. Any idea If that would be possible ? Haris From: Haris Kurtagic [mailto:[hidden email]] Sent: Tuesday, November 03, 2009 11:47 PM To: 'MapGuide Internals Mail List' Subject: MG Object stream serialization/deserialization Hi , I have lot of problems while trying to communicate with different versions of MapGuide. I am developing application which should communicate with MG using MG common libraries: MapGuideCommon, PlatformBase,... Problem is that I can't find two version of MG which will work one against other and work equally. For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch against current branch version. Also for example I can't use any of those versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while current 2.1 branch will throw exception InvalidStreamHeader. At least one of problems is that ClassDefinition in one version is serializing SchemaName in another it is not. So those versions can't be used one against each other. I have two questions: 1. I really can't' find consistency in our svn branches and trunk how and when is changing how MG objects are serialized/desterilized. I hope somebody can light me up little bit when/ why are this changes done. 2. We don't have any versioning of serialization process of objects. I think that could be issue ( hopefully one day we could imagine farm of MapGuide's ). Can we add versions of objects to serialization process ? I would appreciate any help/ideas how to solve this mess, how to have one application communicating with different MG versions. Thank you, Haris _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
Trevor Wekel
|
In reply to this post
by Kenneth Skovhede, GEOGRAF A/S
Hey guys,
The programmatic (web extensions) API which is based on wrapped C++ objects is much more involved than the HTTP API. As far as I know, there are about 100 methods exposed in the HTTP API. The web extensions API consists of approximately 130 classes and 1400 method calls. It is much richer and much more complicated. The problem with backward compatibility is what should the API do if objects change. If you add a new data member to a C++ class which gets passed around in the API, how do you deal with it? Probably the only solution is to default the new data member to something "reasonable" if an old API signature is used. However, this is avoided if the object definitions on the web extensions and the server are in sync. That way you know what you are getting instead of some side effect of "somewhat" incompatible API versions and objects. I agree, backward compatibility would make it easier from a deployment perspective but it is a non-trivial piece of work to maintain backward compatibility. I'm sure Sun and Microsoft do a very good job of maintaining backward compatibility but MapGuide does not have hundreds of developers and testers working on the project. If someone can get backward compatibility buy in from all the contributors, I may be willing to support such an initiative. I believe this would mean that every RFC moving forward would have to explicitly consider and address coding and testing of any backward compatibility issues. Trevor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Kenneth Skovhede, GEOGRAF A/S Sent: November 4, 2009 1:12 AM To: MapGuide Internals Mail List Subject: Re: [mapguide-internals] MG Object stream serialization/deserialization I agree that the API should be fully backwards compatible. It's a real pain that the client must know in advance what version of the API to use. This is especially true for .Net which has a less-than-easy method for version selection on dll's. The HTTP API is fully version capable, but the native API (aka WebAPI) is not. The reason for this is that the native API is actually wrapping C++ classes. Wrapping these classes reproduce some functionality from the MapGuide server on the client (buffering, etc.). Since these C++ classes are not clearly defined, they change between releases without notice, breaking the API compatibility. If there was a version field, the serializer could take care of this, and select a proper default value for missing fields. Unfortunately, there is no serializer, only custom code inside the classes that do this, making it fairly difficult to maintain. I belive that the root issue is that the API is too deeply nested in the server. I would propose that the API sends binary versions of the same requests that the HTTP API sends. It would then be a matter of implementing a light interface that resembles the current http request parser, and reuses the same functions. I hear speed concerns incoming already :) The overhead for the HTTP API is significant, both because of the HTTP protocol itself, but also due to the http parameter parsing and the xml input/output. Both these issues could be resolved by using a binary format of the messages, and allowing the server to accept these binary messages directly (like the WebAPI does now). The binary format should not be a repetition of the current system, but rather a generic format, something similar to "binary JSON", so the actual class layout and message format are independant. Such a seperation is also much safer, as it is easier to unittest the binary parser, and the messages could be allowed over http, unlike the current WebAPI. Regards, Kenneth Skovhede, GEOGRAF A/S UV skrev: > I very much support Haris ideas. > Versioning of serialized objects as well as the API calls and DLL > versions is something that is state-of-the-art in the IT field. Those > pieces of extra information help to work efficiently as it helps to > avoid a number frustrating build & deployment scenarios. > > A seperate external text document containing configuration information > tends to be outdated most of the time and arriving too late. > > One issue is the kind of implicit dependency on a certain FDO release. > For the releases this piece of information is written down somewhere.... > however I dont understand why the DLL loaders dont check those > versions???? > > The versioning of serialized objects is another related issue. THose > are approaches to write robuster code. > > A lot of such configuration data should be made part of the CI system. > This is how those issues are addressed in successful IT projects as it > makes the dependencies > a part of the build which shows up as a TAG in the repository. > > Leaving this as loose as it is in the moment causes pain and more pain > for too many people. > I would call this a part of the evolutionary step from inhouse > development to open source! > In a small local team such formalisms can be avoided by asking the > correct team member sitting near. However this does not scale well and > in respect to the community those issues should be addresses > more thoroughly at some point hopefully soon. > > UV > > > > > Haris Kurtagic wrote: >> Hi , >> >> >> >> I have lot of problems while trying to communicate with different >> versions >> of MapGuide. I am developing application which should communicate >> with MG >> using MG common libraries: MapGuideCommon, PlatformBase,... >> >> >> >> Problem is that I can't find two version of MG which will work one >> against >> other and work equally. >> >> For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch >> against current branch version. Also for example I can't use any of >> those >> versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. >> 2010 while >> current 2.1 branch will throw exception InvalidStreamHeader. >> >> >> >> At least one of problems is that ClassDefinition in one version is >> serializing SchemaName in another it is not. So those versions can't >> be used >> one against each other. >> >> >> >> I have two questions: >> >> >> >> 1. I really can't' find consistency in our svn branches and trunk >> how and >> when is changing how MG objects are serialized/desterilized. I hope >> somebody >> can light me up little bit when/ why are this changes done. >> >> >> >> 2. We don't have any versioning of serialization process of objects. >> I think >> that could be issue ( hopefully one day we could imagine farm of >> MapGuide's >> ). >> >> Can we add versions of objects to serialization process ? >> >> >> >> I would appreciate any help/ideas how to solve this mess, how to have >> one >> application communicating with different MG versions. >> >> >> >> Thank you, >> >> Haris >> >> _______________________________________________ >> mapguide-internals mailing list >> [hidden email] >> http://lists.osgeo.org/mailman/listinfo/mapguide-internals >> >> > > _______________________________________________ > mapguide-internals mailing list > [hidden email] > http://lists.osgeo.org/mailman/listinfo/mapguide-internals mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
|
Trevor Wekel
|
In reply to this post
by Haris Kurtagic
Hi Haris,
Yes. That submission did add a member variable to MgClassDefinition in order to support schema search related optimizations with some Fdo providers. Up to this point, the web extensions API has never supported backward compatibility from a binary perspective. You cannot successfully mix and match different web extensions and servers. As you have noticed, any change in object signature breaks binary compatibility. I believe it will be a huge amount of work to successfully code and test for backward binary compatibility on an API this large. If the PSC really wants to go down this road, I may be willing to support the decision but that means every contributor will be responsible for maintaining this with every submission. The permutations and combinations on the testing effort could be painful to say the least. It's hard enough to keep one API version stable. A mixed and matched the web extensions and server should probably be considered a whole other API. Sincerely, Trevor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic Sent: November 4, 2009 3:41 AM To: 'MapGuide Internals Mail List' Subject: [mapguide-internals] RE: MG Object stream serialization/deserialization Hi Trevor, I believe I found when ClassDefinition serialization has changed. Changeset 3905 <http://trac.osgeo.org/mapguide/changeset/3905> for trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp <http://trac.osgeo.org/mapguide/browser/trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp?rev=3905> Timestammp Author trevorwekel Ticket <http://trac.osgeo.org/mapguide/ticket/1002> #1002 Batch Processing for Get Identity Properties http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBase /Services/ClassDefinition.cpp@3905 <http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBas e/Services/ClassDefinition.cpp@3905&new=trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp@3905> &new=trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp@3905 It feels bit strange that in previous versions schema name from class def. was omitted ( but that is how 2.1 Beta and MG Ent 2010 works ). This change was really for that ticket "Batch Processing for Get Identity Properties" or ? I would like to be able to communicate with MG 2.0.2, MG 2.1 MG Ent. 2010 with one application using standard MG libraries. Any idea If that would be possible ? Haris From: Haris Kurtagic [mailto:[hidden email]] Sent: Tuesday, November 03, 2009 11:47 PM To: 'MapGuide Internals Mail List' Subject: MG Object stream serialization/deserialization Hi , I have lot of problems while trying to communicate with different versions of MapGuide. I am developing application which should communicate with MG using MG common libraries: MapGuideCommon, PlatformBase,... Problem is that I can't find two version of MG which will work one against other and work equally. For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch against current branch version. Also for example I can't use any of those versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while current 2.1 branch will throw exception InvalidStreamHeader. At least one of problems is that ClassDefinition in one version is serializing SchemaName in another it is not. So those versions can't be used one against each other. I have two questions: 1. I really can't' find consistency in our svn branches and trunk how and when is changing how MG objects are serialized/desterilized. I hope somebody can light me up little bit when/ why are this changes done. 2. We don't have any versioning of serialization process of objects. I think that could be issue ( hopefully one day we could imagine farm of MapGuide's ). Can we add versions of objects to serialization process ? I would appreciate any help/ideas how to solve this mess, how to have one application communicating with different MG versions. Thank you, Haris _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
|
Haris Kurtagic
|
Current:
If I interpret correctly your email, you think there is no possibility to implement application communicating to diff. version of MG using current common libraries ? Future: Regarding versioning of serialization of objects I have experience with two different large scale application (many objects). In one case versioning is done "per stream" in another versioning it is "per object". I prefer "per stream", which means that we use incrementally one version number for any change on any object serialization/deserialization. In current MG stream packet there is stream version which is unused. Of 'course in order to work developers has to be carefully when changing objects to increase global version number and add handling of previous versions. Regarding "default values" or handling previous versions, if there would be such changes that it is impossible to came up with "default" value for changed member we can still throw exception "invalid stream.." as it does now. Haris -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Trevor Wekel Sent: Wednesday, November 04, 2009 12:38 PM To: MapGuide Internals Mail List Subject: RE: [mapguide-internals] RE: MG Object stream serialization/deserialization Hi Haris, Yes. That submission did add a member variable to MgClassDefinition in order to support schema search related optimizations with some Fdo providers. Up to this point, the web extensions API has never supported backward compatibility from a binary perspective. You cannot successfully mix and match different web extensions and servers. As you have noticed, any change in object signature breaks binary compatibility. I believe it will be a huge amount of work to successfully code and test for backward binary compatibility on an API this large. If the PSC really wants to go down this road, I may be willing to support the decision but that means every contributor will be responsible for maintaining this with every submission. The permutations and combinations on the testing effort could be painful to say the least. It's hard enough to keep one API version stable. A mixed and matched the web extensions and server should probably be considered a whole other API. Sincerely, Trevor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic Sent: November 4, 2009 3:41 AM To: 'MapGuide Internals Mail List' Subject: [mapguide-internals] RE: MG Object stream serialization/deserialization Hi Trevor, I believe I found when ClassDefinition serialization has changed. Changeset 3905 <http://trac.osgeo.org/mapguide/changeset/3905> for trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp <http://trac.osgeo.org/mapguide/browser/trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp?rev=3905> Timestammp Author trevorwekel Ticket <http://trac.osgeo.org/mapguide/ticket/1002> #1002 Batch Processing for Get Identity Properties http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBase /Services/ClassDefinition.cpp@3905 <http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBas e/Services/ClassDefinition.cpp@3905&new=trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp@3905> &new=trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp@3905 It feels bit strange that in previous versions schema name from class def. was omitted ( but that is how 2.1 Beta and MG Ent 2010 works ). This change was really for that ticket "Batch Processing for Get Identity Properties" or ? I would like to be able to communicate with MG 2.0.2, MG 2.1 MG Ent. 2010 with one application using standard MG libraries. Any idea If that would be possible ? Haris From: Haris Kurtagic [mailto:[hidden email]] Sent: Tuesday, November 03, 2009 11:47 PM To: 'MapGuide Internals Mail List' Subject: MG Object stream serialization/deserialization Hi , I have lot of problems while trying to communicate with different versions of MapGuide. I am developing application which should communicate with MG using MG common libraries: MapGuideCommon, PlatformBase,... Problem is that I can't find two version of MG which will work one against other and work equally. For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch against current branch version. Also for example I can't use any of those versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while current 2.1 branch will throw exception InvalidStreamHeader. At least one of problems is that ClassDefinition in one version is serializing SchemaName in another it is not. So those versions can't be used one against each other. I have two questions: 1. I really can't' find consistency in our svn branches and trunk how and when is changing how MG objects are serialized/desterilized. I hope somebody can light me up little bit when/ why are this changes done. 2. We don't have any versioning of serialization process of objects. I think that could be issue ( hopefully one day we could imagine farm of MapGuide's ). Can we add versions of objects to serialization process ? I would appreciate any help/ideas how to solve this mess, how to have one application communicating with different MG versions. Thank you, Haris _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
|
Trevor Wekel
|
Hi Haris,
Current: It may be possible to write code in a latest version server to account for API differences. However, I believe it would be prohibitively expensive to do so. Future: I think we would need to sit down and figure out a plan of attack for this. One possible suggestion would be to add a version number as the first serialized parameter for every object serialized over TCP/IP. This would allow us to handle minor API differences (ie. minor changes to objects but identical method signatures) in a straightforward way. The objects could handle forward/backward compatibility themselves. We would be forced to increment the global stream version if we did this. For significant API changes, the Server would have to be specifically coded to handle both versions of the TCP/IP operation. From what I recall, this should be possible with the existing infrastructure. However, it may "pollute" the API over a period of time if we have to keep old C++ signatures around. Instead, it might be possible and more straightforward to implement all of the version handling in the "Op" logic and force old TCP/IP signatures to use new C++ API methods. Theoretically, ensuring exactly the same behaviour when using new C++ API methods for old TCP/IP signatures could be tricky if there are a lot of changes underneath the covers. This would be an excellent topic for discussion at the PSC meeting tomorrow, at least as far as saying whether or not we actually want to pursue this. As I mentioned before, the testing effort required to validate API compatibility could be painful. The existing (open source) unit tests do not cover the API sufficiently to validate full compatibility. I am sure internal Autodesk QA resources would have a better idea of the number of man years required to do something like this. Developing a comprehensive test suite for 130 objects and 1400 method signatures seems like a multiple man year effort to me. Thanks, Trevor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic Sent: November 4, 2009 5:33 AM To: 'MapGuide Internals Mail List' Subject: RE: [mapguide-internals] RE: MG Object stream serialization/deserialization Current: If I interpret correctly your email, you think there is no possibility to implement application communicating to diff. version of MG using current common libraries ? Future: Regarding versioning of serialization of objects I have experience with two different large scale application (many objects). In one case versioning is done "per stream" in another versioning it is "per object". I prefer "per stream", which means that we use incrementally one version number for any change on any object serialization/deserialization. In current MG stream packet there is stream version which is unused. Of 'course in order to work developers has to be carefully when changing objects to increase global version number and add handling of previous versions. Regarding "default values" or handling previous versions, if there would be such changes that it is impossible to came up with "default" value for changed member we can still throw exception "invalid stream.." as it does now. Haris -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Trevor Wekel Sent: Wednesday, November 04, 2009 12:38 PM To: MapGuide Internals Mail List Subject: RE: [mapguide-internals] RE: MG Object stream serialization/deserialization Hi Haris, Yes. That submission did add a member variable to MgClassDefinition in order to support schema search related optimizations with some Fdo providers. Up to this point, the web extensions API has never supported backward compatibility from a binary perspective. You cannot successfully mix and match different web extensions and servers. As you have noticed, any change in object signature breaks binary compatibility. I believe it will be a huge amount of work to successfully code and test for backward binary compatibility on an API this large. If the PSC really wants to go down this road, I may be willing to support the decision but that means every contributor will be responsible for maintaining this with every submission. The permutations and combinations on the testing effort could be painful to say the least. It's hard enough to keep one API version stable. A mixed and matched the web extensions and server should probably be considered a whole other API. Sincerely, Trevor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Haris Kurtagic Sent: November 4, 2009 3:41 AM To: 'MapGuide Internals Mail List' Subject: [mapguide-internals] RE: MG Object stream serialization/deserialization Hi Trevor, I believe I found when ClassDefinition serialization has changed. Changeset 3905 <http://trac.osgeo.org/mapguide/changeset/3905> for trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp <http://trac.osgeo.org/mapguide/browser/trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp?rev=3905> Timestammp Author trevorwekel Ticket <http://trac.osgeo.org/mapguide/ticket/1002> #1002 Batch Processing for Get Identity Properties http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBase /Services/ClassDefinition.cpp@3905 <http://trac.osgeo.org/mapguide/changeset?old=trunk/MgDev/Common/PlatformBas e/Services/ClassDefinition.cpp@3905&new=trunk/MgDev/Common/PlatformBase/Serv ices/ClassDefinition.cpp@3905> &new=trunk/MgDev/Common/PlatformBase/Services/ClassDefinition.cpp@3905 It feels bit strange that in previous versions schema name from class def. was omitted ( but that is how 2.1 Beta and MG Ent 2010 works ). This change was really for that ticket "Batch Processing for Get Identity Properties" or ? I would like to be able to communicate with MG 2.0.2, MG 2.1 MG Ent. 2010 with one application using standard MG libraries. Any idea If that would be possible ? Haris From: Haris Kurtagic [mailto:[hidden email]] Sent: Tuesday, November 03, 2009 11:47 PM To: 'MapGuide Internals Mail List' Subject: MG Object stream serialization/deserialization Hi , I have lot of problems while trying to communicate with different versions of MapGuide. I am developing application which should communicate with MG using MG common libraries: MapGuideCommon, PlatformBase,... Problem is that I can't find two version of MG which will work one against other and work equally. For example I can't use MG OS Beta 2.1.0.3891 version from 2.1 branch against current branch version. Also for example I can't use any of those versions against MG Enterprise 2010. 2.1.0.3891 will halt MG Ent. 2010 while current 2.1 branch will throw exception InvalidStreamHeader. At least one of problems is that ClassDefinition in one version is serializing SchemaName in another it is not. So those versions can't be used one against each other. I have two questions: 1. I really can't' find consistency in our svn branches and trunk how and when is changing how MG objects are serialized/desterilized. I hope somebody can light me up little bit when/ why are this changes done. 2. We don't have any versioning of serialization process of objects. I think that could be issue ( hopefully one day we could imagine farm of MapGuide's ). Can we add versions of objects to serialization process ? I would appreciate any help/ideas how to solve this mess, how to have one application communicating with different MG versions. Thank you, Haris _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals _______________________________________________ mapguide-internals mailing list [hidden email] http://lists.osgeo.org/mailman/listinfo/mapguide-internals |
||||||||||||||||
| Free Embeddable Forum Powered by Nabble | Help |
