eqOSG performance

[bhulliger]

Hi all,

We are using Equalizer on two different setups:

1. 1 Node with 4 graphics cards (nvidia quadro with 2 outputs each)
2. 6 nodes with 1 graphics card each.

We use Ubuntu on all machines. The setup is meant to run a 3 sided cave. We have 1 window per pipe and 1 channel per window. Each channel is assigned to one wall.

We implemented a stress test application to test the two different setups. The application contains 3 stanford dragons.

If we start the second configuration with multiple nodes, we get a result of about 65 fps.

If we start the first configuration, the framerate is about 17 fps only. The strange thing is, that the framerate increases if we remove pipe(s) from the configuration. It seems as if the whole scenegraph is rendered on each pipe, which is not really the purpose of Equalizer I think.

Did anyone experience similar problems?

The configuration file for setup 1 is posted below.

Cheers,
Brigitte

global {
        EQ_CONFIG_FATTR_EYE_BASE 0.025 # set the eye distance between the left and the right eye.
}

server {
        config {
                # the appNode represents the node, on which the application runs.
                appNode {
                        pipe { device 0 window { channel { name "fwall-leye" } } }
                        pipe { device 1 window { channel { name "rwall-leye" } } }
                        pipe { device 3 window { channel { name "lwall-leye" } } }
                        pipe { device 5 window { channel { name "fwall-reye" } } }
                        pipe { device 6 window { channel { name "lwall-reye" } } }
                        pipe { device 7 window { channel { name "rwall-reye" } } }
                }
                compound {
                        compound {
                                wall {
                                        bottom_left [ -1.25 -1  1 ]
                                        bottom_right [ -1.25 -1 -1 ]
                                        top_left [ -1.25  1  1 ]
                                }
                                compound {
                                        channel "lwall-leye" # channel to be displayed
                                        eye [ LEFT ] # eye pass
                                }
                                compound {
                                        channel "lwall-reye"
                                        eye [ RIGHT ]
                                }
                        }
                        compound {
                                wall {
                                        bottom_left [ -1.25 -1 -1 ]
                                        bottom_right [  1.25 -1 -1 ]
                                        top_left [ -1.25  1 -1 ]
                                }
                                compound {
                                        channel "fwall-leye"
                                        eye [ LEFT ]
                                }
                                compound {
                                        channel "fwall-reye"
                                        eye [ RIGHT ]
                                }
                        }
                        compound {
                                wall {
                                        bottom_left [  1.25 -1 -1 ]
                                        bottom_right [  1.25 -1  1 ]
                                        top_left [  1.25  1 -1 ]
                                }
                                compound {
                                        channel "rwall-leye"
                                        eye [ LEFT ]
                                }
                                compound {
                                        channel "rwall-reye"
                                        eye [ RIGHT ]
                                }
                        }
                }
        }
}

[Stefan Eilemann]

On 3. Jun 2009, at 14:49, bhulliger wrote:

> If we start the first configuration, the framerate is about 17 fps  
> only. The
> strange thing is, that the framerate increases if we remove pipe(s)  
> from the
> configuration. It seems as if the whole scenegraph is rendered on  
> each pipe,
> which is not really the purpose of Equalizer I think.
>
> Did anyone experience similar problems?

This can have multiple causes:

- CPU/Memory contention: use the task manager (or better tool) to  
analyze this
- Serialization of the draw calls: Observe the statistics
- Bad multi-threading driver support: try four nodes on the local  
machine, try 185.x driver
- Lock contention in osg render traversal: imo unlikely

Frustum culling seems to work since you get >60Hz with multiple  
machines.


HTH,

Stefan.


_______________________________________________
eq-dev mailing list
[hidden email]
http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev
http://www.equalizergraphics.com

[Jonas Walti]

Some javascript/style in this post has been disabled (why?)

Stefan Eilemann schrieb:

On 3. Jun 2009, at 14:49, bhulliger wrote:

  

If we start the first configuration, the framerate is about 17 fps  
only. The
strange thing is, that the framerate increases if we remove pipe(s)  
from the
configuration. It seems as if the whole scenegraph is rendered on  
each pipe,
which is not really the purpose of Equalizer I think.

Did anyone experience similar problems?
    

This can have multiple causes:

- CPU/Memory contention: use the task manager (or better tool) to  
analyze this
  

Ubuntu reports well spread CPU usage (around 50%-60% for all 4 cores with a simple OSG model but fully 100% with eVolve. This results into 1-2 FPS with eVolve)

- Serialization of the draw calls: Observe the statistics
  

What exaclty do you mean with "Serialisation"? The statistics report long green boxes and sometimes a bigger red box.

- Bad multi-threading driver support: try four nodes on the local  
machine, try 185.x driver
  

W'll try this l8er...

- Lock contention in osg render traversal: imo unlikely
  

eVolve is slow as well, probably not an osg problem

Furthermore, the Quadros look a bit bored while rendering. The GPU-Temp does never hit the 70°C border (btw: does anyone know a good nvidia GPU measurement tool for linux?). Maybe this is because we do not use the SLI tech. Any experience with SLI/Equalizer out there?

Thank you!

Jonas Walti & Crew




_______________________________________________
eq-dev mailing list
[hidden email]
http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev
http://www.equalizergraphics.com

[Stefan Eilemann]

On 3. Jun 2009, at 23:18, Jonas Walti wrote:

>> - Serialization of the draw calls: Observe the statistics
>>
> What exaclty do you mean with "Serialisation"? The statistics report  
> long green boxes and sometimes a bigger red box.

Some kind of de-parallelization of the render threads, either in the  
driver, OSG or somewhere else. I guess this is the case since you get  
slower with each pipe added.

What is the big red box? Can you find out from the doc, or post a  
screenshot? The only red box should be for Config::finishFrame, since  
you're not doing compositing or using a swap barrier.

>> - Bad multi-threading driver support: try four nodes on the local
>> machine, try 185.x driver
>>
> W'll try this l8er...

The multi-process test is done quickly and will point in the direction  
of the driver, if it improves performance.

>> - Lock contention in osg render traversal: imo unlikely
>>
> eVolve is slow as well, probably not an osg problem
>
> Furthermore, the Quadros look a bit bored while rendering. The GPU-
> Temp does never hit the 70°C border (btw: does anyone know a good  
> nvidia GPU measurement tool for linux?).

> Ubuntu reports well spread CPU usage (around 50%-60% for all 4 cores  
> with a simple OSG model but fully 100% with eVolve. This results  
> into 1-2 FPS with eVolve)

This sounds like a driver issue. eVolve shouldn't be CPU-bound, and  
the GPU's should all be busy.

Max - do you have any experience with eVolve on Multi-GPU systems?

> Maybe this is because we do not use the SLI tech. Any experience  
> with SLI/Equalizer out there?

SLI is like a driver-based Equalizer (oversimplified). It abstracts  
multiple GPU's into one. You can use it with Eq, as if you use one  
GPU. Typically this approach will give you less performance, since  
you're approaching the problem at a lower level.


HTH,

Stefan.


_______________________________________________
eq-dev mailing list
[hidden email]
http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev
http://www.equalizergraphics.com