How to run WRF_VPRM in WRF4.3?

[zhengzich]

Hi guys, I'm currently learning to use WRF_VPRM.
I carefully read the WRF_GHG manual and the WRF-CHEM user guide 4.4.
But I have a question now, I use WRF-CHEM version 4.3, I hope to be able to run WRF-VPRM with chem_opt=16, use VPRM to provide fluxes and combine with other lists to calculate changes in CO 2 concentration.

Do I need to just use WRF4.3, or do I need to take the WRF-GHG code and combine it with WRF4.3?

I know this question may be a little silly, but it's important to me and I look forward to your answers.

 

[callewaerts]

I assume you refer to Beck et al (2011) when talking about the WRF-GHG manual? This is unfortunally an outdated document as since then (2012) the VPRM module was implemented in the WRF-Chem code. So generally, you don't need to adjust the model code.

The main task that you need to do in order to 'activate' the VPRM module is provide the correct input fields.
WRF-Chem expects a file (for every domain and time step) containing VEGFRA_VPRM, EVI, EVI_MIN, EVI_MAX, LSWI, LSWI_MIN and LSWI_MAX, which then should be refered to by using auxinput15 in the namelist.

Additionally, make sure that you set bio_emiss_opt=16 or 17 in the namelist.

 

[zengchang fan]

I assume you refer to Beck et al (2011) when talking about the WRF-GHG manual? This is unfortunally an outdated document as since then (2012) the VPRM module was implemented in the WRF-Chem code. So generally, you don't need to adjust the model code.

The main task that you need to do in order to 'activate' the VPRM module is provide the correct input fields.
WRF-Chem expects a file (for every domain and time step) containing VEGFRA_VPRM, EVI, EVI_MIN, EVI_MAX, LSWI, LSWI_MIN and LSWI_MAX, which then should be refered to by using auxinput15 in the namelist.

Additionally, make sure that you set bio_emiss_opt=16 or 17 in the namelist.

Hello, I just started learning WRF-VPRM, and I would like to ask you how to generate files containing VEGFRA_VPRM, EVI, EVI_MIN, EVI_MAX, LSWI, LSWI_MIN and LSWI_MAX data.

I am going to try VPRM_Preprocessor, but LPDAAC no longer provides the Linux version of MRT; then I tried pyVPRM, but I also encountered some problems; so I would like to ask if you have any recommended methods for generating VPRM module data.

Your help is very important to me, and I look forward to your reply.

 

[callewaerts]

Dear zengchang fan

I personally only have experience with the (old) VPRM Preprocessor as it was installed together with the other required tools a few years ago on the computing system that I use. I would assume the best approach nowadays is to try using pyVPRM, which I have not tried yet myself.
Maybe you can try contacting the creators of the tool? Or you can be more specific about the problems you encounter there and hope that some other users on this forum can help you with that. Good luck!

 

[zengchang fan]

Dear zengchang fan

I personally only have experience with the (old) VPRM Preprocessor as it was installed together with the other required tools a few years ago on the computing system that I use. I would assume the best approach nowadays is to try using pyVPRM, which I have not tried yet myself.
Maybe you can try contacting the creators of the tool? Or you can be more specific about the problems you encounter there and hope that some other users on this forum can help you with that. Good luck!

Dear callewaerts

Thanks for your reply, it will help me to clarify the focus of the next step.
Specifically, in the process of using pyVPRM, the packages in the environment have errors one after another, but I am trying to solve these problems one by one.
Thanks again for your advice, I wish you all the best!

 

[zhengzich]

Dear callewaerts

Thanks for your reply, it will help me to clarify the focus of the next step.
Specifically, in the process of using pyVPRM, the packages in the environment have errors one after another, but I am trying to solve these problems one by one.
Thanks again for your advice, I wish you all the best!

I haven't used py processing, but VPRMpreproc does require some specific debugging. In this calculation process, it is mainly necessary to calculate a percentage based on the local class in order to enter the WRF, which is a more critical step.

 

[zengchang fan]

I haven't used py processing, but VPRMpreproc does require some specific debugging. In this calculation process, it is mainly necessary to calculate a percentage based on the local class in order to enter the WRF, which is a more critical step.

pyVPRM also needs debugging, but more problems occur in the packages in the environment. Thank you for sharing your experience, I will pay attention to the debugging of VPRM parameters.

 

[liwenjia]

Hello, I just started learning WRF-VPRM, and I would like to ask you how to generate files containing VEGFRA_VPRM, EVI, EVI_MIN, EVI_MAX, LSWI, LSWI_MIN and LSWI_MAX data.

I am going to try VPRM_Preprocessor, but LPDAAC no longer provides the Linux version of MRT; then I tried pyVPRM, but I also encountered some problems; so I would like to ask if you have any recommended methods for generating VPRM module data.

Your help is very important to me, and I look forward to your reply.

你好，我也在研究VPRM，请问可以加个联系方式交流一下吗

 

[richard_n413]

i've been trying to use PyVPRM too, but have got stuck. The examples read in NetCDF files from MODIS, but the files downloaded from MODIS are in HDF4, which causes it not to succeed. Has anyone else encountered this, and how have they solved it?

 

[liwenjia]

i've been trying to use PyVPRM too, but have got stuck. The examples read in NetCDF files from MODIS, but the files downloaded from MODIS are in HDF4, which causes it not to succeed. Has anyone else encountered this, and how have they solved it?

There is no need to convert the files into Netcdf. You can simply use the HDF files that you download from the MODIS server. And if you are doing wrf-vprm, we can communicate about that!!!

 

[richard-newton]

I have now run PyVPRM successfully, with some help from the author (Theo Glauch) also; I noticed he has edited the readme to include this information too. For additional reference, I had to edit the vprm_preprocessor_new.py file on l.68 to target *.hdf files rather than *.nc files, and also install gdal and rasterio in my conda installation, which I hadn't done.

 

[liwenjia]

I have now run PyVPRM successfully, with some help from the author (Theo Glauch) also; I noticed he has edited the readme to include this information too. For additional reference, I had to edit the vprm_preprocessor_new.py file on l.68 to target *.hdf files rather than *.nc files, and also install gdal and rasterio in my conda installation, which I hadn't done.

Hello, I had a problem when running pyvprm, I don't know if you have encountered. Looking forward to your reply, Thank you!!!

 

[richard-newton]

Does the script generate output? The warnings you're getting normally occur if there are NaN values in your arrays, which Python should handle without failing.

 

[liwenjia]

Does the script generate output? The warnings you're getting normally occur if there are NaN values in your arrays, which Python should handle without failing.

Yes, I generate output, but I don't know if it has impact. Because my GEE is very small.

 

[zhengzich]

Hello, I had a problem when running pyvprm, I don't know if you have encountered. Looking forward to your reply, Thank you!!!

In fact, I have also appeared about this situation and discussed it with the author (Theo Glauch) , but it may be that I provided too little information andTheo Glauch did not know what caused it.

However, even with this problem, it is still possible to obtain the result file in the end. As for whether there is an impact on the subsequent WRF_VPRM simulation, I also want to know.

 

[zzzyy]

I have now run PyVPRM successfully, with some help from the author (Theo Glauch) also; I noticed he has edited the readme to include this information too. For additional reference, I had to edit the vprm_preprocessor_new.py file on l.68 to target *.hdf files rather than *.nc files, and also install gdal and rasterio in my conda installation, which I hadn't done.

Hi, I had a problem when running pyvprm, I don't know if you have encountered. Looking forward to your reply, Thank you!!!

 

[zengchang fan]

Hi, I had a problem when running pyvprm, I don't know if you have encountered. Looking forward to your reply, Thank you!!!

Hi, I had this problem too, and I solved the problem according to his suggestion：

and took took me quite a lot of trial an error to figure out the correct conda environment:

name: pyvprm
channels:
- conda-forge
- defaults
dependencies:
- python=3.12
- esmpy
- esmf
- netcdf4
- dask
- numpy
- rioxarray
- rasterio
- proj-data

 

[richard-newton]

Hi, I had a problem when running pyvprm, I don't know if you have encountered. Looking forward to your reply, Thank you!!!

This seems to be a problem with rasterio not recognizing hdf files. My first guess is that you need to install h5netcdf into your Python / Conda environment first. Second, if your modis files have the NetCDF extension ".nc", do the fix as above.

 

[zzzyy]

Hi, I had this problem too, and I solved the problem according to his suggestion：

Thanks for your reply, according to your tips, I was able to run the process again. However, I have encountered a new issue. I tried adding CRS information to the MODIS data（HDF4), but it seems to have no effect. Have you encountered a similar issue before? How should I proceed from here? Could I kindly ask for your assistance in reviewing this? The issue is as follows:

2024-12-09 22:22:45.873 | INFO | pyVPRM.sat_managers.base_manager:add_tile:254 - Do reprojections
2024-12-09 22:22:46.135 | INFO | pyVPRM.sat_managers.base_manager:add_tile:261 - Merge
Traceback (most recent call last):
File "/home/test/anaconda3/WRF-GHG-Prepy-main/vprm_input/pyVPRM_Final/wrf_preprocessor/vprm_preprocessor_new1.py", line 168, in <module>
vprm_inst.add_vprm_insts(insts[1:])
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/pyVPRM/VPRM.py", line 1107, in add_vprm_insts
self.sat_imgs.add_tile(
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/pyVPRM/sat_managers/base_manager.py", line 262, in add_tile
self.sat_img = merge.merge_datasets(to_merge)
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/rioxarray/merge.py", line 254, in merge_datasets
merged_data[data_var] = merge_arrays(
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/rioxarray/merge.py", line 169, in merge_arrays
merged_data, merged_transform = _rio_merge(
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/rasterio/merge.py", line 252, in merge
first_crs = first.crs
AttributeError: 'RasterioDatasetDuck' object has no attribute 'crs'

 

[zengchang fan]

Thanks for your reply, according to your tips, I was able to run the process again. However, I have encountered a new issue. I tried adding CRS information to the MODIS data（HDF4), but it seems to have no effect. Have you encountered a similar issue before? How should I proceed from here? Could I kindly ask for your assistance in reviewing this? The issue is as follows:

2024-12-09 22:22:45.873 | INFO | pyVPRM.sat_managers.base_manager:add_tile:254 - Do reprojections
2024-12-09 22:22:46.135 | INFO | pyVPRM.sat_managers.base_manager:add_tile:261 - Merge
Traceback (most recent call last):
File "/home/test/anaconda3/WRF-GHG-Prepy-main/vprm_input/pyVPRM_Final/wrf_preprocessor/vprm_preprocessor_new1.py", line 168, in <module>
vprm_inst.add_vprm_insts(insts[1:])
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/pyVPRM/VPRM.py", line 1107, in add_vprm_insts
self.sat_imgs.add_tile(
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/pyVPRM/sat_managers/base_manager.py", line 262, in add_tile
self.sat_img = merge.merge_datasets(to_merge)
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/rioxarray/merge.py", line 254, in merge_datasets
merged_data[data_var] = merge_arrays(
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/rioxarray/merge.py", line 169, in merge_arrays
merged_data, merged_transform = _rio_merge(
File "/home/test/anaconda3/envs/vprm/lib/python3.10/site-packages/rasterio/merge.py", line 252, in merge
first_crs = first.crs
AttributeError: 'RasterioDatasetDuck' object has no attribute 'crs'

Sorry, I didn't encountered a similar issue before. But I think this problem may not come from MODIS data. Generally, MODIS data can be used directly in pyVPRM. By comparing with my running log, this code should occur after all MODIS tile data in a chunk has been read and processed, and tile merging is in progress. So I thought maybe we could try these methods to solve this problem:
1. Check super computer's memory.
2. Clean files in /wrf_preprocessor/out after a failed run.
3. Check rasterio and rioxarray installed with conda, and I think the most likely cause is a version conflict in the environment package.
4. Install pyVPRM in python 3.12 is recommended.
These are based on my previous experience in building pyPVRM. I hope it can help you.