How to get the value on halo regions?

[mirabelle]

Hi,

I want to add an online data assimilation code by using WRFV3.7.1. So that I need to design a new halo region by getting the value on halo regions .

I have tried to print value on memory domain and calculate domain, but I can only get value on calculate domain, such as by print psfc(its:ite, jts:jte). The value on halo region is totally zero,such as print psfc(ims:its,jms:jts).

Can you tell me how to get the information on halo regions ?
Is there any subroutine in WRF can access other processor and grab the information ?

 

[kwerner]

Hi,
I first would like to apologize for the delay in response. Our team took a couple of weeks off for the holidays and are just now getting caught up with everything.

If you haven't already, take a look at this presentation, which may have some useful advice regarding halos.

 

[mirabelle]

Thanks for your reply.

This document tell us what is halo, but doesn't tell us which subroutine control these process.

It seems that WRF exchange information through some subroutine, such like "rsl_comm_iter_init"、“RSL_LITE_INIT_EXCH”、“RSL_LITE_PACK”、“RSL_LITE_EXCH_Y”、“RSL_LITE_EXCH_X”.

Does that means the MPI lib is rewritten by C and packed in RSL_LITE* subroutine ?

But there is no notes for RSL_LITE* subrourine.

Is there any document can help us understand these subroutine?

 

[davegill]

The halo regions tend to be filled in only for fields that are required for horizontal differencing. For example, the prognostic dynamics variables have halo regions that are filled with valid data. There are some fields that typically do not have any need for halo data, such as fields computed from inside the physics schemes. In those routines, the computations are almost entirely done in the vertical, with no "neighbor" requirements.

If there is a variable that you would like to use halo information from a neighboring grid cell, it is fairly easy.

1. The variable must be declared "state" in the Registry. An "i1" variable may not be used for a communication. If you need to promote a variable to a state from an i1 type, there are probably a few changes to the files in the dyn_em directory that will be required. The state variables are all contained inside for the DDT grid, and tend to get passed into routines in that way. The i1 variables must be explicitly passed into routines, for example first_rk_step_part1.

2. Look in Registry/Registry.EM_COMMON for the keyword "halo". Here's an example:

halo      HALO_EM_PHYS_HCW   dyn_em 8:hailcast_wup_mask, hailcast_wdur

This gives an 8-point stencil for two variables: hailcast_wup_mask and hailcast_wdur. If you think of a square cut into nxn pieces (where n is odd), then a 3x3 box (excluding the center point) is a communication of one grid cell in each of the i- and j-directions. So an 8-point stencil represents a 3x3 box. A 5x5 box (two values in each of the i- and j-directions) would be a 24-point stencil. A 7x7 box (three values in the halo for each i- and j-direction) is a 48-point stencil. Name the communication (in this example, it is HALO_EM_PHYS_HCW) something that is a mnemonic for you, such as HALO_EM_DA.

3. Inside the dyn_em/solve_em.F (almost always) you will find the associated strings that are included in the source code to "do" the actual communication at that point. For example, look for HALO_EM_A. Note that all halos are surrounded by an ifdef:

#ifdef DM_PARALLEL
#    include "HALO_EM_A.inc"
#endif

After that call, then the variables specified in the Registry for that communication will have updated halo regions, to the depth that you requested. Also note, that every halo communication that is used must be mentioned at the top of the routine ("halo_em_a_sub" is a subroutine associated with the communication that was defined as "HALO_EM_A"):

   USE module_comm_dm, ONLY : &
                  halo_em_a_sub,halo_em_b_sub,halo_em_c2_sub,halo_em_chem_e_3_sub          &