fail to run init_atmosphere model

ntmanhvn181 · Jul 15, 2019

Dear All,
I try to run test case on MPAS model by using real datas which were downloaded in this link (https://mpas-dev.github.io/). But when I run ./init_atmosphere_model but it fails. Please help me how to fix this problem, the error was introduced in this following code:

Code:

----------------------------------------------------------------------
Beginning MPAS-init_atmosphere Error Log File for task       0 of       1
    Opened at 2019/07/16 11:05:44
----------------------------------------------------------------------

CRITICAL ERROR: Error while reading namelist record data_sources.
Logging complete.  Closing file at 2019/07/16 11:05:44

mgduda · Jul 16, 2019

I was able to successfully use your namelist.init_atmosphere and streams.init_atmosphere files (after changing the path to the geographical datasets), so it is interesting that the init_atmosphere_model program would have problems in reading the "data_sources" namelist record. Are you using MPAS v7.0 or an earlier version of the model? The namelist options have changed between MPAS v6.x and MPAS v7.0.

ntmanhvn181 · Jul 16, 2019

Dear Mr.mgduda,
Currently, I have been using MPAS version 6.3 with this case. I have tried to change some option on &preproc_stages but it still error.
Please help me to fig this problem.

mgduda · Jul 16, 2019

If possible, the easiest solution might be to update to MPAS v7.0. There should be no additional libraries needed compared with MPAS v6.x, so if you have already been able to compile MPAS v6.x, compiling MPAS v7.0 should work without problem.

However, if you do need to use MPAS v6.3, you can begin with the namelist.init_atmosphere file that should have been created in the MPAS-Model/default_inputs/ directory and edit it as described in the MPAS v6.3 User's Guide. MPAS v7.0 added several more options in the namelist.init_atmosphere file that will not be recognized by MPAS v6.3.

ntmanhvn181 · Jul 16, 2019

Thank you for your suggestion to update to MPAS v7.0. I compile MPAS model v7.0 and run ./init_atmosphere and it works. However, I display file log.err and it shows this following information:

Code:

Beginning MPAS-init_atmosphere Error Log File for task       0 of      30
    Opened at 2019/07/17 09:34:10
----------------------------------------------------------------------

ERROR: Could not open block decomposition file for 30 blocks.
CRITICAL ERROR: Filename: x1.40962.graph.info.part.30
Logging complete.  Closing file at 2019/07/17 09:34:10

In addition, I show the log.out file, it stills run:

Code:

 /home/hiepn/share/geog/modis_landuse_20class_30s/39601-40800.07201-08400
 /home/hiepn/share/geog/modis_landuse_20class_30s/40801-42000.07201-08400
 /home/hiepn/share/geog/modis_landuse_20class_30s/42001-43200.07201-08400
 /home/hiepn/share/geog/modis_landuse_20class_30s/00001-01200.08401-09600
 /home/hiepn/share/geog/modis_landuse_20class_30s/01201-02400.08401-09600
 /home/hiepn/share/geog/modis_landuse_20class_30s/02401-03600.08401-09600
 /home/hiepn/share/geog/modis_landuse_20class_30s/03601-04800.08401-09600
 /home/hiepn/share/geog/modis_landuse_20class_30s/04801-06000.08401-09600
 /home/hiepn/share/geog/modis_landuse_20class_30s/06001-07200.08401-09600
 /home/hiepn/share/geog/modis_landuse_20class_30s/07201-08400.08401-09600

mcurry · Jul 17, 2019

The model failed because you tried to run on 30 mpi tasks, but did not have a graph partition file that was partitioned for 30 tasks (i.e. x1.40962.graph.info.part.30 in the run directory). It looks like you just installed metis on your HPC, so you can create one on your own using metis and the x1.40962.graph.info file via the command:

Code:

gpmetis -minconn -contig -niter=200 x1.40962.graph.info.part 30

Let me know if that works or if you have any problems!

ntmanhvn181 · Jul 17, 2019

Dear mcurry,
I run your command in my HPC but it shows:

Code:

-bash: gpmetis: command not found

Please help me why it fails, eventhough I had installed metis on my HPC.

mcurry · Jul 18, 2019

The gpmetis command was not found because its directory is not within your PATH environment variable. You can either append/prepend it to PATH or you can launch it directly from the location you specified using the --prefix command when installing metis. I.E:

Code:

/your/path/to/metis-5.1.0/build/Linux-X86_64/programs/gpmetis

My metis installation location is in my home directory, so I can launch gpmetis by running:

Code:

~/metis-5.1.0/build/Linux-X86_64/programs/gpmetis -minconn -contig -niter=200 x1.40962.graph.info.part 30

Hope that helps!

ntmanhvn181 · Jul 18, 2019

Thank for your useful suggestion to launch the path of gpmetis to use this command (gpmetis -minconn -contig -niter=200 x1.40962.graph.info.part 30).
I insert my path and it works. However when I run "mpiexec -n 32 ./init_atmosphere_model", the log.err file shows:

Code:

ERROR: Writing to stream 'output' would clobber file 'x1.40962.static.nc',
ERROR:     but clobber_mode is set to 'never_modify'.

The output files are introduced in the attachments. Plus, the namelist and streams file are included as well.
Please help me explain about it.

mcurry · Jul 19, 2019

Glad to hear you are able to get metis working.

By default, steams are set to not overwrite data that maybe present from previous runs (to not lose any data). Please see this post in a previous topic on how to resolve this error.

mgduda · Jul 19, 2019

@ntmanhvn181 If you have already processed the static fields as described in Section 7.2.1 of the MPAS User's Guide, you will need to edit the filenames in the "input" and "output" streams before interpolating initial conditions as described in Section 7.2.2. The streams.init_atmosphere file that you attached has

Code:

<immutable_stream name="input"
                  type="input"
                  filename_template="x1.40962.grid.nc"
                  input_interval="initial_only" />

<immutable_stream name="output"
                  type="output"
                  filename_template="x1.40962.static.nc"
                  packages="initial_conds"
                  output_interval="initial_only" />

but the "input" and "output" streams probably should look like the following

Code:

<immutable_stream name="input"
                  type="input"
                  filename_template="x1.40962.static.nc"
                  input_interval="initial_only" />

<immutable_stream name="output"
                  type="output"
                  filename_template="x1.40962.init.nc"
                  packages="initial_conds"
                  output_interval="initial_only" />

ntmanhvn181 · Jul 19, 2019

Dear mgduda and mcury,
Thank you for your suggestion. I edit the streams.init_atmosphere like this following code and it works without error

Code:

<immutable_stream name="output"
                  type="output"
                  filename_template="x1.40962.init.nc"
                  clobber_mode="overwrite"
                  packages="initial_conds"
                  output_interval="initial_only" />

mgduda · Jul 22, 2019

Did you also change the filename in the "input" stream to "x1.40962.static.nc"? Alternatively, have you set

Code:

    config_static_interp = true
    config_native_gwd_static = true

in you namelist.init_atmosphere file?

ntmanhvn181 · Jul 22, 2019

Dear mgduda,
I follow with your suggestion and set up my namelist with these option:

Code:

config_static_interp = true
config_native_gwd_static = true

and I also have changed streams.init_atmosphere file into:

Code:

<immutable_stream name="input"
                  type="input"
                  filename_template="x1.40962.static.nc"
                  input_interval="initial_only" />

After I run ./init_atmosphere_model, it occurs this following problem

Code:

[cloud@agpc 120km_uniform]$ ./init_atmosphere_model >log.run
*** glibc detected *** ./init_atmosphere_model: malloc(): smallbin double linked list corrupted: 0x00000000050fe760 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3d81875e5e]
/lib64/libc.so.6[0x3d8187a528]
/lib64/libc.so.6(__libc_malloc+0x5c)[0x3d8187ab1c]
./init_atmosphere_model(for__get_vm+0x6d)[0xc0195d]
./init_atmosphere_model(for__acquire_lun+0xe6a)[0xbd347a]
./init_atmosphere_model(for_write_int_fmt+0x9c)[0xc02b2c]
./init_atmosphere_model[0x4f5522]
./init_atmosphere_model[0x4c4af9]
./init_atmosphere_model[0x469af9]
./init_atmosphere_model[0x42bae7]
./init_atmosphere_model[0x42ba6e]
./init_atmosphere_model[0x42ba1e]
/lib64/libc.so.6(__libc_start_main+0x100)[0x3d8181ed20]
./init_atmosphere_model[0x42b929]
======= Memory map: ========
00400000-00ef6000 r-xp 00000000 510:481a2 144115950751440242             /work/users/cloud/user/manh/MPAS/MPAS_initatmosphere/120km_uniform/init_atmosphere_model
010f6000-011d2000 rw-p 00af6000 510:481a2 144115950751440242             /work/users/cloud/user/manh/MPAS/MPAS_initatmosphere/120km_uniform/init_atmosphere_model
011d2000-01367000 rw-p 00000000 00:00 0 
0268e000-2aaf8000 rw-p 00000000 00:00 0                                  [heap]
3d81400000-3d81420000 r-xp 00000000 09:00 262251                         /lib64/ld-2.12.so
3d81620000-3d81621000 r--p 00020000 09:00 262251                         /lib64/ld-2.12.so
3d81621000-3d81622000 rw-p 00021000 09:00 262251                         /lib64/ld-2.12.so
3d81622000-3d81623000 rw-p 00000000 00:00 0 
3d81800000-3d8198b000 r-xp 00000000 09:00 262309                         /lib64/libc-2.12.so
3d8198b000-3d81b8a000 ---p 0018b000 09:00 262309                         /lib64/libc-2.12.so
3d81b8a000-3d81b8e000 r--p 0018a000 09:00 262309                         /lib64/libc-2.12.so
3d81b8e000-3d81b90000 rw-p 0018e000 09:00 262309                         /lib64/libc-2.12.so
3d81b90000-3d81b94000 rw-p 00000000 00:00 0 
3d81c00000-3d81c83000 r-xp 00000000 09:00 262327                         /lib64/libm-2.12.so
3d81c83000-3d81e82000 ---p 00083000 09:00 262327                         /lib64/libm-2.12.so
3d81e82000-3d81e83000 r--p 00082000 09:00 262327                         /lib64/libm-2.12.so
3d81e83000-3d81e84000 rw-p 00083000 09:00 262327                         /lib64/libm-2.12.so
3d82000000-3d82017000 r-xp 00000000 09:00 262311                         /lib64/libpthread-2.12.so
3d82017000-3d82217000 ---p 00017000 09:00 262311                         /lib64/libpthread-2.12.so
3d82217000-3d82218000 r--p 00017000 09:00 262311                         /lib64/libpthread-2.12.so
3d82218000-3d82219000 rw-p 00018000 09:00 262311                         /lib64/libpthread-2.12.so
3d82219000-3d8221d000 rw-p 00000000 00:00 0 
3d82400000-3d82402000 r-xp 00000000 09:00 262319                         /lib64/libdl-2.12.so
3d82402000-3d82602000 ---p 00002000 09:00 262319                         /lib64/libdl-2.12.so
3d82602000-3d82603000 r--p 00002000 09:00 262319                         /lib64/libdl-2.12.so
3d82603000-3d82604000 rw-p 00003000 09:00 262319                         /lib64/libdl-2.12.so
3d82800000-3d82807000 r-xp 00000000 09:00 262313                         /lib64/librt-2.12.so
3d82807000-3d82a06000 ---p 00007000 09:00 262313                         /lib64/librt-2.12.so
3d82a06000-3d82a07000 r--p 00006000 09:00 262313                         /lib64/librt-2.12.so
3d82a07000-3d82a08000 rw-p 00007000 09:00 262313                         /lib64/librt-2.12.so
3d83000000-3d8301d000 r-xp 00000000 09:00 262334                         /lib64/libselinux.so.1
3d8301d000-3d8321c000 ---p 0001d000 09:00 262334                         /lib64/libselinux.so.1
3d8321c000-3d8321d000 r--p 0001c000 09:00 262334                         /lib64/libselinux.so.1
3d8321d000-3d8321e000 rw-p 0001d000 09:00 262334                         /lib64/libselinux.so.1
3d8321e000-3d8321f000 rw-p 00000000 00:00 0 
3d83800000-3d83816000 r-xp 00000000 09:00 262332                         /lib64/libresolv-2.12.so
3d83816000-3d83a16000 ---p 00016000 09:00 262332                         /lib64/libresolv-2.12.so
3d83a16000-3d83a17000 r--p 00016000 09:00 262332                         /lib64/libresolv-2.12.so
3d83a17000-3d83a18000 rw-p 00017000 09:00 262332                         /lib64/libresolv-2.12.so
3d83a18000-3d83a1a000 rw-p 00000000 00:00 0 
3d84c00000-3d84c16000 r-xp 00000000 09:00 262364                         /lib64/libgcc_s-4.4.7-20120601.so.1
3d84c16000-3d84e15000 ---p 00016000 09:00 262364                         /lib64/libgcc_s-4.4.7-20120601.so.1
3d84e15000-3d84e16000 rw-p 00015000 09:00 262364                         /lib64/libgcc_s-4.4.7-20120601.so.1
3d86400000-3d86432000 r-xp 00000000 09:00 262676                         /lib64/libidn.so.11.6.1
3d86432000-3d86631000 ---p 00032000 09:00 262676                         /lib64/libidn.so.11.6.1
3d86631000-3d86632000 rw-p 00031000 09:00 262676                         /lib64/libidn.so.11.6.1
3d86800000-3d86827000 r-xp 00000000 09:00 5794346                        /usr/lib64/libssh2.so.1.0.1
3d86827000-3d86a26000 ---p 00027000 09:00 5794346                        /usr/lib64/libssh2.so.1.0.1
3d86a26000-3d86a28000 rw-p 00026000 09:00 5794346                        /usr/lib64/libssh2.so.1.0.1
3d87000000-3d87052000 r-xp 00000000 09:00 5794347                        /usr/lib64/libcurl.so.4.1.1
3d87052000-3d87251000 ---p 00052000 09:00 5794347                        /usr/lib64/libcurl.so.4.1.1
3d87251000-3d87254000 rw-p 00051000 09:00 5794347                        /usr/lib64/libcurl.so.4.1.1
3d88000000-3d8800e000 r-xp 00000000 09:00 262677                         /lib64/liblber-2.4.so.2.10.2
3d8800e000-3d8820d000 ---p 0000e000 09:00 262677                         /lib64/liblber-2.4.so.2.10.2
3d8820d000-3d8820e000 r--p 0000d000 09:00 262677                         /lib64/liblber-2.4.so.2.10.2
3d8820e000-3d8820f000 rw-p 0000e000 09:00 262677                         /lib64/liblber-2.4.so.2.10.2
3d88400000-3d88403000 r-xp 00000000 09:00 262340                         /lib64/libcom_err.so.2.1
3d88403000-3d88602000 ---p 00003000 09:00 262340                         /lib64/libcom_err.so.2.1
3d88602000-3d88603000 rw-p 00002000 09:00 262340                         /lib64/libcom_err.so.2.1
3d88c00000-3d88c4d000 r-xp 00000000 09:00 262678                         /lib64/libldap-2.4.so.2.10.2
3d88c4d000-3d88e4c000 ---p 0004d000 09:00 262678                         /lib64/libldap-2.4.so.2.10.2Aborted

mgduda · Jul 25, 2019

I should have stated this more clearly, but you will need to either set

Code:

    config_static_interp = true
    config_native_gwd_static = true

to cause the init_atmosphere_model program to interpolate static (time-invariant) fields, or set

Code:

<immutable_stream name="input"
                  type="input"
                  filename_template="x1.40962.static.nc"
                  input_interval="initial_only" />

in your streams.init_atmosphere file to cause the init_atmosphere_model program to read static (time-invariant) fields from the input file. But, you should not do both, since the interpolation of static fields first expands the radius of the sphere by a factor of 6371229. By doing both, you will have created a sphere with radius 6371229*6371229 m.

ntmanhvn181 · Jul 29, 2019

Dear mgduda,
I follow with your suggestion and I set my streams.init_atmosphere file by this option:

Code:

<streams>
<immutable_stream name="input"
                  type="input"
                  filename_template="x1.40962.grid.nc"
                  input_interval="initial_only" />

<immutable_stream name="output"
                  type="output"
                  filename_template="x1.40962.init.nc"
                  packages="initial_conds"
                  output_interval="initial_only" />

and I set my namelist_init_atmosphere:

Code:

&preproc_stages
    config_static_interp = false
    config_native_gwd_static = false
    config_vertical_grid = true
    config_met_interp = true
    config_input_sst = false
    config_frac_seaice = true

Then model works without error. Thank for your help.

mgduda · Jul 31, 2019

If your last post is accurate, then I think you have taken neither of the options needed to correctly produce real-data initial conditions. The "x1.40962.grid.nc" file defines the MPAS mesh on a unit sphere (a sphere with radius 1.0 m). The size of the sphere is expanded to one with radius 6371229 m when "static" fields are interpolated. By setting

Code:

<immutable_stream name="input"
                  type="input"
                  filename_template="x1.40962.grid.nc"
                  input_interval="initial_only" />

you have read as input to the init_atmosphere_model program a unit-sphere mesh, and by setting

Code:

&preproc_stages
    config_static_interp = false
    config_native_gwd_static = false
    config_vertical_grid = true
    config_met_interp = true
    config_input_sst = false
    config_frac_seaice = true

you have skipped the step during which the sphere radius is expanded to 6371229 m.

So, what I suspect you have are initial conditions on a unit sphere with no topography or land use information. You can verify this by running, e.g.,

Code:

ncdump -v ter x1.40962.init.nc

and

Code:

ncdump -v dcEdge x1.40962.init.nc

.

fail to run init_atmosphere model

ntmanhvn181

Member

Attachments

mgduda

Administrator

ntmanhvn181

Member

mgduda

Administrator

ntmanhvn181

Member

mcurry

New member

ntmanhvn181

Member

mcurry

New member

ntmanhvn181

Member

Attachments

mcurry

New member

mgduda

Administrator

ntmanhvn181

Member

mgduda

Administrator

ntmanhvn181

Member

mgduda

Administrator

ntmanhvn181

Member

mgduda

Administrator