Purpose of `map` in `tw_lptype` for built-in mappings

[gkaf89]

Why map in tw_lptype must be defined manually in built-in mappings? In custom mappings the map field is required to assign LPs to their PE. In built-in mappings the map field must also be specified even though it seems that there enough information to set it automatically.

For instance, for the built-in mappings a function like the following should provide the PE mapping automatically:

tw_peid map(tw_lp id) {
    return g_tw_lp[id]->pe->id;
}

Am I missing something, or it's just that the configuration of the map field has not yet been automated for built-in mappings?

[helq]

The interface exposed by tw_lp is unfortunately outdated (sadly, the tw_lp API is frozen forever). As you have noticed, map should return the PE id for a given LP id. We do not need to set this function for every LP, it would be sufficient to set this function once as a global variable somewhere. Yet, we don't do it. So, you have to set it multiple times.

Please check the working 2-LP example to see how we handle the map function for a LINEAR mapping: https://github.com/ROSS-org/ross-model-example

This is the function you would need for LINEAR mapping to work:

tw_peid map(tw_lpid gid) {
   return (tw_peid)gid / g_tw_nlp;
}

You are correct. The configuration for the map field has not been implemented for built-in mappings. It is indeed a bit embarrassing that the model developer has to provide the map function for a mode that is said to be supported by default! Please open an issue here in github if we should include the function above in ROSS.

Now, on a subtle issue with your proposed code. When running ROSS in multiple cores (aka, PEs), we will allocate some LPs to each PE. Each PE is its own MPI rank, and thus, g_tw_lp holds only LPs accessible locally to that PE. (Why is there a pointer to a PE within the LP? This is an unfortunate result of years of features being added and removed. That pointer and g_tw_pe are the same, always. One PE per MPI rank.)

[gkaf89]

I was under the impression that all variables with prefix g_ have the same value on all MPI processes, but it seems that this is not the case! Thank you for the explanation, it was very clear.

I tried following the instruction in the section Mapping Setup Function of the documentation and there are 2 issues.

    // set up the single KP
    tw_kp_onpe(0, g_tw_pe);

• The g_tw_nkp variable must be set to the exact number of kernel processes setup with tw_kp_onpe, otherwise some kernel processes are left NULL.

    // set up the LPs
    for (lp_index = 0; lp_idex < lps_on_pe; lp_index++) {
        // map LP to KP
        tw_lp_onkp(g_tw_lp[lp_index], g_tw_kp[0]);

        // calculate LP's GID
        tw_lpid lp_gid = g_tw_mynode + (lp_index * tw_nnodes());

        // map LP to PE
        tw_lp_onpe(lp_index, g_tw_pe, lp_gid);
    }

• The tw_lp_onkp must be called after tw_lp_onpe, as otherwise the g_tw_lp entries are null.

Regarding the the interface exposed by tw_lp, it would be nice if it was reworked at some point. There are many variables whose names could be misinterpreted, the interface is mixing indexes and pointers, and it would be nice if the array with pointers to local processes was hidden from the interface. But I understand that this would require a lot of work.

Regarding the provision of mapping function for default mappings, I have a few questions before opening a ticket. In the function,

tw_peid map(tw_lpid gid) {
   return (tw_peid)gid / g_tw_nlp;
}

does the variable g_tw_nlp contain the number of local LPs and is gid the global ID of the LP?

[helq]

Yes, that is correct g_tw_nlp contains the number of local LPs (to a PE), and gid is the global id