Java 5 64-bit: memory hog?

[Ben Caradoc-Davies]

I have been encountering heap size problems running unit tests on a
64-bit JVM (yes, gt-app-schema). I might have expected a 64-bit JVM to
consume twice the heap of a 32-bit JVM at worst (some suggest 40% to 60%
more is typical), but now I have unit tests that pass with 512M heap on
32-bit and run out of memory with 1024M on 64-bit. Something is wrong. I
expect that there will be gc differences (server versus client), but
this behaviour was unexpected.

Has anyone else run into problems with memory leaks or other such
problems on 64-bit JDK? app-schema does a lot of XML/XSD processing,
could this be implicated?

I am using jdk1.5.0_18 amd64 on Linux.

--
Ben Caradoc-Davies <[hidden email]>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Geotools-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[aaime]

Ben Caradoc-Davies ha scritto:

> I have been encountering heap size problems running unit tests on a
> 64-bit JVM (yes, gt-app-schema). I might have expected a 64-bit JVM to
> consume twice the heap of a 32-bit JVM at worst (some suggest 40% to 60%
> more is typical), but now I have unit tests that pass with 512M heap on
> 32-bit and run out of memory with 1024M on 64-bit. Something is wrong. I
> expect that there will be gc differences (server versus client), but
> this behaviour was unexpected.
>
> Has anyone else run into problems with memory leaks or other such
> problems on 64-bit JDK? app-schema does a lot of XML/XSD processing,
> could this be implicated?

I don't have infos about the JDK per se, but reading through the release
notes of Galileo I remember seeing a report of optimizations
in the EMF memory consumption. E.g., from this presentation:
http://www.eclipse.org/project-slides/Galileo/EMF_Core_Galileo_Review.pdf
see page 3:
Minimal footprint EObject implementation
* New, optimally memory-efficient base EObject
implementation available for use in generated models
* Adopted in Ecore to reduce footprint of metadata

Given we use EMF bindings this _might_ be relevant

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Geotools-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Ben Caradoc-Davies]

Andrea Aime wrote:
> Given we use EMF bindings this _might_ be relevant

Thanks, Andrea. I am sure it is relevant. We use an ancient EMF, and I
am pointing the finger squarely at it.

--
Ben Caradoc-Davies <[hidden email]>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Geotools-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Gabriel Roldan]

Hi Ben,
Ben Caradoc-Davies wrote:
> Andrea Aime wrote:
>> Given we use EMF bindings this _might_ be relevant
>
> Thanks, Andrea. I am sure it is relevant. We use an ancient EMF, and I
> am pointing the finger squarely at it.
you're right at pointing a finger at it. I have a suggestion for you.
Besides upgrading to a newer EMF, the problem with app schema is startup
time. Once you got the schemas parsed you're good to go with less memory.
If I remember well, EMFAppSchemaReader does a two step process: get all
the schemas as EMF XSD, then parse all the types to GeoTools
FeatureTypes/Descriptors.
That's indeed a problem: parse _all_ the types. What would be
interesting is to parse only the tree of types _needed_ by the
FeatureTypeMappings. That is, start with the target root Feature element
and parse down as needed, instead of parsing the whole schemas.
Makes sense?

Cheers,
Gabriel
>


--
Gabriel Roldan
OpenGeo - http://opengeo.org
Expert service straight from the developers.


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Geotools-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Justin Deoliveira]

Ben Caradoc-Davies wrote:
> Andrea Aime wrote:
>> Given we use EMF bindings this _might_ be relevant
>
> Thanks, Andrea. I am sure it is relevant. We use an ancient EMF, and I
> am pointing the finger squarely at it.
>

Indeed, faster EMF would be nice. I will throw it on the TODO list of
things to look at doing. Thankfully EMF seems to be pretty good about
backwards compatibility so hopefully it should be painless.


--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Geotools-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Ben Caradoc-Davies]

Gabriel Roldan wrote:

> you're right at pointing a finger at it. I have a suggestion for you.
> Besides upgrading to a newer EMF, the problem with app schema is startup
> time. Once you got the schemas parsed you're good to go with less memory.
> If I remember well, EMFAppSchemaReader does a two step process: get all
> the schemas as EMF XSD, then parse all the types to GeoTools
> FeatureTypes/Descriptors.
> That's indeed a problem: parse _all_ the types. What would be
> interesting is to parse only the tree of types _needed_ by the
> FeatureTypeMappings. That is, start with the target root Feature element
> and parse down as needed, instead of parsing the whole schemas.
> Makes sense?

Gabriel, that is a very sensible suggestion. We will have a look and see
how easy it is to do.

Kind regards,

--
Ben Caradoc-Davies <[hidden email]>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Geotools-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geotools-devel