okl4_pd_attr_setconstructrootpd function problem

[crisvlad]

Hi,

 I'm trying to compile the pds_zone example inside /example folder, in order to see how a shared-memory mechanism should be implemented in OKL4 v3. and I get:
examples/pds_zones/src/main.c:113: warning: implicit declaration of function 'okl4_pd_attr_setconstructrootpd'
examples/pds_zones/src/main.c:113: warning: nested extern declaration of 'okl4_pd_attr_setconstructrootpd'

 As I have seen on this forum (there is a message posted by Alexandre Courbot) and from documentation I have (section B-21.34 in 'OKL4 Library Reference Manual',  Document Number:  OK 40292:2008 (revision 2), Software Version: 3.0, Date: October 13, 2008 which is marked as Draft)  the function above appears in there but has absolutely no reference below its name.

 I've tried to bypass this function, and I imagined that inside it some fields of the okl4_pd_attr structure should be initialized, namely those that are not touched by okl4_pd_attr_init(). Thus I put some more lines into okl4_pd_attr_init() function (in ./okl4/src/micro/pd_attr_init.c):
 attr->virtmem_pool = NULL;
 attr->kclistid_pool = NULL;
 attr->kspaceid_pool = NULL;

 Additionally, I reversed the call order between  get_root_weaved_objects() and create_root_pd(). If I didn't do that, the following message should appear:
Assertion failed: attr->virtmem_pool, function okl4_pd_create, file libs/okl4/src/micro/pd_create.c, line 409
This says that some field inside attr structure (of type okl4_pd_attr_t) is not initialized, so remains NULL.

 Now, after all these changes my error is somewhere inside the okl4_zone_create() in ./okl4/src/micro/zone_create.c:
Assertion failed: attr->range.size % OKL4_ZONE_ALIGNMENT == 0, function okl4_zone_create, file libs/okl4/src/micro/zone_create.c, line 115

 My questions are:
 - is the example pds_zone an official one (therefore can be questioned upon, so supported)?
 - why am I getting that assertion failed (what would the size of that zone be?)

 Any other comment related to this shared memory technique would be very useful.

[Hannes Payer]

Hi,

pds_zones would be a very useful example. I also tried to get it correct but
like crisvlad I am running into several problems. Has anyone a solution for
the problems crisvlad mentioned? Disabling the assertions does nor really help ;-)

-Hannes

crisvlad wrote:

Hi,

 I'm trying to compile the pds_zone example inside /example folder, in order to see how a shared-memory mechanism should be implemented in OKL4 v3. and I get:
examples/pds_zones/src/main.c:113: warning: implicit declaration of function 'okl4_pd_attr_setconstructrootpd'
examples/pds_zones/src/main.c:113: warning: nested extern declaration of 'okl4_pd_attr_setconstructrootpd'

 As I have seen on this forum (there is a message posted by Alexandre Courbot) and from documentation I have (section B-21.34 in 'OKL4 Library Reference Manual',  Document Number:  OK 40292:2008 (revision 2), Software Version: 3.0, Date: October 13, 2008 which is marked as Draft)  the function above appears in there but has absolutely no reference below its name.

 I've tried to bypass this function, and I imagined that inside it some fields of the okl4_pd_attr structure should be initialized, namely those that are not touched by okl4_pd_attr_init(). Thus I put some more lines into okl4_pd_attr_init() function (in ./okl4/src/micro/pd_attr_init.c):
 attr->virtmem_pool = NULL;
 attr->kclistid_pool = NULL;
 attr->kspaceid_pool = NULL;

 Additionally, I reversed the call order between  get_root_weaved_objects() and create_root_pd(). If I didn't do that, the following message should appear:
Assertion failed: attr->virtmem_pool, function okl4_pd_create, file libs/okl4/src/micro/pd_create.c, line 409
This says that some field inside attr structure (of type okl4_pd_attr_t) is not initialized, so remains NULL.

 Now, after all these changes my error is somewhere inside the okl4_zone_create() in ./okl4/src/micro/zone_create.c:
Assertion failed: attr->range.size % OKL4_ZONE_ALIGNMENT == 0, function okl4_zone_create, file libs/okl4/src/micro/zone_create.c, line 115

 My questions are:
 - is the example pds_zone an official one (therefore can be questioned upon, so supported)?
 - why am I getting that assertion failed (what would the size of that zone be?)

 Any other comment related to this shared memory technique would be very useful.

[crisvlad]

Hello again,
  I did more work for that pds_zone example in /examples/pds_zones .  My notes follow:

1.  The memory pools (kspace, kspaceid_pools, range and max_pds) are all statically configured/initiated using okl4_cell.py file. The compilation is done with respect to this static configuration. We are able to okl4_env_get() in main.c, by using get_root_weaved_objects() function.

2.  The okl4_pd_attr_init() function in /libs/okl4/src/micro/pd_attr_init.c states that: 'By default we do not use pools'. And that is why all the protection domain's pointer-type attributes are set to NULL. In my opinion this is not entirely true since, as we read in the section 'A-2.2 Resource Pools Interface' from the 'OKL4 Library Reference Manual' the resources pool interface consists both of kernel identifier pools interface and the memory pools interface. Furthermore, in create_elf_segment_zone() there is a call to okl4_zone_create() in which the following assertion appears:
   assert(XOR(attr->kspaceid.raw != ~0UL, attr->kspaceid_pool));

with the following comment:
  /* The attributes must contain either a kspace or a space id pool, but not both. */

 So at least one attribute - either kspace or kspaceid_pool - must be previously set. Since okl4_pd_attr_init() doesn't use pools from default and okl4_zone_create() wants them, the question could be if that missing function okl4_setconstructrootpd() had to deal with this issue?

3. For every new allocation done within a zone (that translates to a okl4_zone_create() call), the range.base and range.size's values generate alignment assertion errors inside okl4_zone_create(). I did some (temporary!) changes by forcing proper values to those variables according to the macros' definition, in order to pass over those messages. For the range.size I set its value to 1MB (0x100000) since we have (in zone.h file inside /libs/okl4/include):
  #ifdef ARM_SHARED_DOMAINS
    #define OKL4_ZONE_ALIGNMENT 0x100000

and for the range.base I used 0x80100000, which is divisible (a multiple of) 0x100000.
 For the moment I think that the computation mechanism involving those variables has some insider inadvertences. For example, being known that the address space starts at 0x80000000 (range.base:80000000) I get a values such as:
  range.base:cfff2000

inside okl4_memsec_getrange(), which is very strange despite the fact that every page_size is 0x2000 long (4k, which is allright). Should these errors have the same cause as above (that famous missing function)?

 I must say once again that the values I've chosen are there just to see how far this example goes(!). The real values that I get do not pass the assertions' filters, namely:
 computed size(before item_setrange()): 94208<->17000, chosen size: 1048576<->100000,
 computed max_vaddr:2147577856<->80017000, computed min_vaddr:2147483648<->80000000

4. In the instructions:
  text = create_memsec_from_elf_segment("MAIN_RX");
  data = create_memsec_from_elf_segment("MAIN_RW");
we can see in main.c of the pds_example I cannot find out which MAIN_RX and MAIN_RW are. I've searched them in:
 ./tools/pyelf/weaver/cell_environment.py
and
 ./tools/pyelf/weaver/okl4_cell.py
but they are not there...

By 'computed' values I meant the values as they get computed in the original example.


5. Following is a short cross-references for the functions that we have in this example. The calls represented are made either to the okl4 library set or to system calls (by '->' I mean 'calls to'):
 - create_root_pd() -> okl4_pd_create() -> {allocate_pd_slack_memory, construct_pd_from_pools(), construct_kthread_pool(), create_utcb_memsec()};
 - create_memsec_from_elf_segment() -> okl4_env_get_segment() -> okl4_env_get();
 - create_elf_segment_zone() -> okl4_zone_create() -> L4_SpaceControl();
 - create_thread_in_pd() -> okl4_pd_thread_create() -> okl4_kthread_create() -> L4_ThreadControl();
 - create_memsec_from_pools() -> okl4_virtmem_pool_alloc() -> okl4_range_allocator_alloc() -> anonymous_allocation() / named_allocation();
 - okl4_physmem_segpool_alloc() -> okl4_range_allocator_alloc() ->  anonymous_allocation() / named_allocation();
 - create_zone() -> okl4_zone_create() -> L4_SpaceControl();
 - attach_zone_to_pd() -> okl4_pd_zone_attach() -> do_attach_zone();
 - create_thread_in_pd() -> okl4_pd_thread_create() -> {okl4_bitmap_allocator_alloc() -> named_allocation()/ anonymous_allocation(), okl4_kthread_create() -> okn_syscall_thread_create()};
 - okl4_pd_createcap() -> {okl4_kclist_kcap_allocany(), okl4_kclist_kcap_copy(), okl4_kclist_kcap_free};
 - L4_Set_Priority() -> L4_Schedule();
 - okl4_pd_thread_start() -> okl4_kthread_start() -> okn_syscall_futex_signal();
 - okl4_pd_deletecap() ->
 - okl4_pd_thread_delete() -> {okl4_kthread_delete(), okl4_utcb_free(), okl4_kclist_kcap_free()}
 - okl4_virtmem_pool_free() -> okl4_range_allocator_free();
 - okl4_physmem_segpool_free() -> okl4_range_allocator_free();
 - okl4_zone_memsec_detach() -> {_okl4_mem_container_list_remove(), _okl4_kspace_unmap_memsec()};
 - destroy_zone() -> okl4_zone_delete();
 - destroy_memsec() -> okl4_memsec_destroy()

 I use for compiling the following command:
 : ./tools/build.py machine=gumstix project=examples example=pds_zones TOOLCHAIN=gnu_arm_eabi_toolchain PYFREEZE=false pistachio.TOOLCHAIN=gnu_arm_toolchain pistachio.enter_kdb=true kdb_serial=True BUILD_DIR=build_pds_zones
 
and for the simulation:
  skyeye -c tools/sim_config/gumstix.skyeye -e build_pds_zones/images/image.boot

 I would like someone to point to my eventually mistakes I did in my line of thoughts above.
 I also need some insides/details related to the 'root PD' concept and where from does it borrow memory to allocate as it needs?


6.  Some output that I get follows:

big_endian is false.
arch: arm
cpu info: xscale, pxa25x, 69052100, fffffff0, 2
mach info: name pxa_lubbock, mach_init addr 0x8061210
uart_mod:0, desc_in:, desc_out:, converter:
SKYEYE: use xscale mmu ops
start addr is set to 0xa0000000 by exec file.

OKL4 - (provider: Open Kernel Labs) built on Oct  8 2009 19:48:30 using gcc version 3.4.4.
--- KD# System started (press 'g' to continue)  ---
> go
Hello, World
Before get_root_weaved_objects()
root_virtmem_pool:8001779c
root_physseg_pool:80017838
root_kspace_pool:800171a4
root_kclist_pool:800171f4
root_kspace:8001786c
root_kclist:800170e4
root_kthread:80017184
After get_root_weaved_objects()
Before create_root_pd()
attr->default_pager.raw:4294967292, fffffffc | ~3UL:fffffffc
After create_root_pd()
Created root PD object
Before create_pd_from_root_pools()
attr->default_pager.raw:4294967292, fffffffc | ~3UL:fffffffc
After create_pd_from_root_pools()
Child pd 0 created successfully
Before create_pd_from_root_pools()
attr->default_pager.raw:4294967292, fffffffc | ~3UL:fffffffc
After create_pd_from_root_pools()
Child pd 1 created successfully
Created a memsection object for the text segment
text segment.pagesize:4096<->1000
Created a memsection object for the data segment
data segment.pagesize:4096<->1000
okl4_memsec_getrange,733 -> range.base:80000000, range.size:d000
okl4_memsec_getrange,733 -> range.base:80014000, range.size:3000
text_range(before item_setrange()): 53248<->d000, text.page_size:1000, text->total_size:0
data_rangee(before item_setrange()): 12288<->3000, data.page_size:1000, data->total_size:0
text_base(before item_setrange()): 2147483648<->80000000, text_size:d000
data_base(before item_setrange()): 2147565568<->80014000, data_size:3000
computed size(before item_setrange()): 94208<->17000, chosen size: 1048576<->100000,
computed max_vaddr:2147577856<->80017000, computed min_vaddr:2147483648<->80000000
create_elf_segment_zone,range_base:80000000, range_size:1048576, range_total_size:0, range_next(pointer):0
zone_attr.range.total_size:0, range size: 1048576
sizeof(elf_zone):4, elf_zone's dimension: 228, max_pds:10
elf_zone's start address: 80101108
before okl4_zone_create()
start okl4_zone_create()
okl4_zone_create,115 -> OKL4_ZONE_ALIGNMENT:1048576<->100000
ARM_SHARED_DOMAINS:1, kspaceid.raw:ffffffff, kspaceid_pool:800171a4
end okl4_zone_create()
after okl4_zone_create()
Created a zone for the ELF segments, attached both memsections
main,542 -> Attached the zone containing the ELF segments to both PDs
create_memsec_from_pools,296 -> Allocated virtual memory from the root pool
create_memsec_from_pools, 301 -> Allocated physical memory from the root pool
create_memsec_from_pools,311 -> ms's base:80101218
main,548 -> Created new static memsection:80101218
okl4_memsec_getrange,733 -> range.base:cfff2000, range.size:2000
main,558 -> base value:80100000, shared memory buffer size: 1048576
start okl4_zone_create()
okl4_zone_create,115 -> OKL4_ZONE_ALIGNMENT:1048576<->100000
ARM_SHARED_DOMAINS:1, kspaceid.raw:ffffffff, kspaceid_pool:800171a4
end okl4_zone_create()
Created a zone for the shared buffer
Assertion failed: !error, function main, file examples/pds_zones/src/main.c, line 564.
--- KD# User: L4_Rootserver abort()ed ---
>

crisvlad wrote:

Hi,

 I'm trying to compile the pds_zone example inside /example folder, in order to see how a shared-memory mechanism should be implemented in OKL4 v3. and I get:
examples/pds_zones/src/main.c:113: warning: implicit declaration of function 'okl4_pd_attr_setconstructrootpd'
examples/pds_zones/src/main.c:113: warning: nested extern declaration of 'okl4_pd_attr_setconstructrootpd'
....

 I've tried to bypass this function, and I imagined that inside it some fields of the okl4_pd_attr structure should be initialized, namely those that are not touched by okl4_pd_attr_init(). Thus I put some more lines into okl4_pd_attr_init() function (in ./okl4/src/micro/pd_attr_init.c):
 attr->virtmem_pool = NULL;
 attr->kclistid_pool = NULL;
 attr->kspaceid_pool = NULL;

 Additionally, I reversed the call order between  get_root_weaved_objects() and create_root_pd(). If I didn't do that, the following message should appear:
Assertion failed: attr->virtmem_pool, function okl4_pd_create, file libs/okl4/src/micro/pd_create.c, line 409
This says that some field inside the 'attr' structure (of type okl4_pd_attr_t) is not initialized, so remains NULL.
....

Regards,
Vlad.

[crisvlad]

Hello there,

Finally the pds_zones example works for gumstix (pxa machines).
The changes that I've made both to this example's main.c and some other file consists of:
=> For the main.c file within: /examples/pds_zones/src:
 -> The get_root_weaved_objects() and create_root_pd() and must be called in this order and not in the order we initially have;
 -> In create_root_pd() function:
    > The root protection-domain has to be privileged, so its privileged attribute must be 1;
    > The fields virtmem_pool, kclistid_pool and kspaceid_pool in the attr variable of type okl4_pd_attr_t must be initialised, so they all have to be equal to: root_virtmem_pool,
    root_kclist_pool and root_kspace_pool respectively:
        attr.virtmem_pool = root_virtmem_pool;
        attr.kclistid_pool = root_kclist_pool;
        attr.kspaceid_pool = root_kspace_pool;
    Note: Based on these observations, I could say that the missing library function okl4_pd_attr_setconstructrootpd() should deal with these last two steps inside create_root_pd() function.

    > The alignment for the children threads' stack is little-endian ALIGNED(32) (I've also tried ALIGNED(8) and I didn't have problems);
    > After calling okl4_zone_attr_setmaxpds() we must say something like:
        zone_attr.kspaceid_pool = root_kspace_pool;
      otherwise we get an assertion error in: libs/okl4/src/micro/zone_create.c, line 137:
        Assertion failed: XOR(attr->kspaceid.raw != ~0UL, attr->kspaceid_pool)

=> For an external file:
    > The OKL4_ZONE_ALIGNMENT alignment macro inside: /libs/okl4/include/micro/zone.h must be set to either 0x1000 or 0x2000; this example didn't work for this macro set either to 0x10000 (64KB) or to the default 0x100000 (1MB alignment). If this is not the case, based on the computation for the virtual-address' range and base the zone must cover, we will have an assertion error like:
       Assertion failed: attr->range.base % OKL4_ZONE_ALIGNMENT == 0, function okl4_zone_create, file libs/okl4/src/micro/zone_create.c, line 119.

The following macros must be set (my compilation was done using KERNEL=micro and BUILD=debug):
   OKL4_KERNEL_MICRO, ENDIAN_LITTLE and ARM_SHARED_DOMAINS.

In the pds_zones example, when the threads in every protection-domain are started, we do not have access to the serial server inside them. At least this was my conclusion since no messages
could be seen at the stdout (whereas every thread's entry routine uses printf and all the other printfs in the main.c could be seen at the output). For this issue I used some improvisation, by constructing a small printing function based on L4_KDB_PrintChar().

My compilation and simulation commands were:
>> Compilation:
./tools/build.py machine=gumstix2 project=examples example=pds_zones TOOLCHAIN=gnu_arm_eabi_toolchain PYFREEZE=false pistachio.TOOLCHAIN=gnu_arm_toolchain pistachio.enter_kdb=true kdb_serial=True BUILD_DIR=build_pds_zones

>> Simulation:
skyeye -c tools/sim_config/gumstix.skyeye -e build_pds_zones/images/image.boot

It all works now. Of course, the external modification mentioned above must be reverted for some other architectures or eventually guarded somehow in order not to modify it repeatedly.
If there are any comments/observations please make them publicly available.

Regards,
Vlad.

crisvlad wrote:

Hi,

 I'm trying to compile the pds_zone example inside /example folder, in order to see how a shared-memory mechanism should be implemented in OKL4 v3. and I get:
examples/pds_zones/src/main.c:113: warning: implicit declaration of function 'okl4_pd_attr_setconstructrootpd'
examples/pds_zones/src/main.c:113: warning: nested extern declaration of 'okl4_pd_attr_setconstructrootpd'

 As I have seen on this forum (there is a message posted by Alexandre Courbot) and from documentation I have (section B-21.34 in 'OKL4 Library Reference Manual',  Document Number:  OK 40292:2008 (revision 2), Software Version: 3.0, Date: October 13, 2008 which is marked as Draft)  the function above appears in there but has absolutely no reference below its name.

 I've tried to bypass this function, and I imagined that inside it some fields of the okl4_pd_attr structure should be initialized, namely those that are not touched by okl4_pd_attr_init(). Thus I put some more lines into okl4_pd_attr_init() function (in ./okl4/src/micro/pd_attr_init.c):
 attr->virtmem_pool = NULL;
 attr->kclistid_pool = NULL;
 attr->kspaceid_pool = NULL;

 Additionally, I reversed the call order between  get_root_weaved_objects() and create_root_pd(). If I didn't do that, the following message should appear:
Assertion failed: attr->virtmem_pool, function okl4_pd_create, file libs/okl4/src/micro/pd_create.c, line 409
This says that some field inside attr structure (of type okl4_pd_attr_t) is not initialized, so remains NULL.

 Now, after all these changes my error is somewhere inside the okl4_zone_create() in ./okl4/src/micro/zone_create.c:
Assertion failed: attr->range.size % OKL4_ZONE_ALIGNMENT == 0, function okl4_zone_create, file libs/okl4/src/micro/zone_create.c, line 115

 My questions are:
 - is the example pds_zone an official one (therefore can be questioned upon, so supported)?
 - why am I getting that assertion failed (what would the size of that zone be?)

 Any other comment related to this shared memory technique would be very useful.