On the second domain - Forced exit from Rosenbrock due to the following error: Step size too small

[Zuhayr Shahid]

Dear community members,

I hope this message finds you well. I have been trying to run a WRF-CHEM simulation over the western tropical South America with a nested domain. My target is to simulate black carbon transport due to anthropogenic, biogenic and biomass burning emissions. So, I have the following files: wrffirechemi_d0*, wrffchemi_d0*, wrffbiochemi_d0*, exo_coldens_d0*, wrf_season_wes_usgs_d0* (phot_opt = 3). I am using chem_opt = 112 (MOZCART). I also used MOZBC to modify my wrfinputs (applied chemical boundary condition from CAM-CHEM).

When I try to run with two domains with all these chemistry options, the model run doesn't start. It gives me the following error message: Forced exit from Rosenbrock due to the following error:

 Step size too small

 T=  4.800000000000008E-028 and H=  4.800000000000008E-028

 I tried both increasing and decreasing the chemdt. I am attaching my namelist.input here. I also tried using ndown (Run the Model for a One-way Nested Domain Using Ndown — WRF Model documentation) so that I could run the second domain separately, but I'm still falling into the same error. However, it is possible to run with only the first domain (sample picture attached).

I checked the user guide and it said that chem_opt = 112 can use phot_opt 3 or 4. However, WRF-Chem | Atmospheric Chemistry Observations & Modeling says that New TUV photolysis code (phot_opt = 4). This option should be used for all MOZART schemes and replaces phot_opt = 3.   

So, is using phot_opt = 3 causing this problem for the second domain? My outer domain is 12km, inner domain is 4 km, and ERA5 as the boundary condition. Time step is 48s for the outer domain.

The run with chem OFF runs smooth using both the domains.

Here are my chem options:

&chem
 kemit                               = 1
 chem_opt                            = 112, 112,
 chemdt                              = 2, 2,
 emiss_opt                           = 8, 8,
 bio_emiss_opt      = 3, 3,
 bioemdt                             = 30, 30,
 ne_area                             = 97,
 dust_opt      = 3,
 seas_opt      = 1,
 phot_opt                            = 3, 3,
 photdt                              = 30, 30,  
 aer_op_opt      = 1, 1,
 opt_pars_out      = 0,
 aer_ra_feedback      = 0, 0,
 chem_conv_tr      = 0, 0,
 biomass_burn_opt                    = 2, 2,
 scale_fire_emiss                    = .true., .true.,
 plumerisefire_frq                   = 10, 10,
 have_bcs_chem                       = .true., .true.,
 / 

I would really benefit a lot from any insights to this issue from the experts and scientists of the WRF-Chem community!

Best regards,

Zuhayr Ishmam

University at Albany, SUNY

[Zuhayr Shahid]

I also remember that for MOZBC I did  do_ic = .true. and do_bc = .true. for both the domains, whereas I think for the second domain maybe it should be  do_ic = .true. and do_bc = .false. as there was no wrfbdy_d02. However, MOZBC ran without any error. Can it be also a possible source of the Rosenbrock error?

[Gabriele Pfister]

Hi Zuhayr, 

the source of the error could be multiple but yes, you definitely should use chem_opt=4 and I suggest setting chemdt the same as your model time step to start with for both d01 and d02. In your settings it seems you use the same chemdt for d01 and d02 but you do need to set the model and chem time steps for each resolution.  

I am not sure how far into the simulation you get before you get the error and what your other namelist settings are, but in general you do want to generate the wrfinput_d02 dor met and chem IC. And yes, for d02 you would set do_bc = .false. 

Gabi

--

================================
Gabriele Pfister

Atmospheric Chemistry Observations & Modeling (ACOM) Lab

NSF National Center for Atmospheric Research (NCAR)

Email: pfi...@ucar.edu

Work Phone: +1 303 497 2915

Web: https://staff.ucar.edu/users/pfister

[Mary Barth]

Hi there,

Could you review the settings of your time steps?

A chem_dt = 120 s is not a multiple of the meteorology dt = 48 s. So, chemistry is not being calculated at each chemdt.  It would be cleanest to use meteorology dt = 60 s, but if that causes CFL errors, then use dt = 30 s.

(and Gabi meant, use phot_opt = 4)

- Mary

^--^--^--^--^--^--^--^--^--^--^--^--^--^--^-^--^--^--^-^--^

Mary Barth 

Senior Scientist   

phone: 303-497-8186    email: bar...@ucar.edu

NSF - National Center for Atmospheric Research

P.O. Box 3000

Boulder, CO 80307

Staff web page

SIMA Web Site

MUSICA Web Site

DC3 Web Site 

^--^--^--^--^--^--^--^--^--^--^--^--^--^--^-^--^--^--^-^--^

[Zuhayr Shahid]

Hi Gabi and Mary,

 

Thank you so much for your valuable insights. So, I started running the simulation with phot_opt = 4 (downloaded the additional required file and updated some other namelist settings in the chem section as needed by phot_opt = 4). This time for MOZBC, I did do_bc = .false. for d02.

 

For the meteorological time step, I set time_step = 30s.

 

For chem_dt = 1.5, 0.5. Am I selecting the chem_dt in the right way?

 

[1.5 minutes = 90s being a multiple of the time_step = 30s. For chemdt of the second domain, I used 0.5 now as my domains are 12 and 4 km (3:1 ratio)]

 

But still, I am falling into the same Rosenbrock error Step size too small

 T=  4.000000000000006E-029 and H=  4.000000000000006E-029

 

The simulation does not really start as far as the progress in the time step is concerned when using two domains. And I also noticed that the job that I submitted doesn’t really stop in Derecho unless I delete the job. I found out the Rosenbrock error by looking at rsl.out.0067. I have been using 80 cores.

 

I know this is a difficult error to debug, but looking at the namelist options could you please identify any vital component that I might be missing out? Thank you very much!

 

Please note that with the same setting simulations apparently start running and progress when I set max_dom = 1.

 

 

Here is the chem section of the namelist when using two domains. The other portions of the namelist are also copied here.

 

&chem

 kemit                               = 1

 chem_opt                            = 112, 112,

 chemdt                              = 1.5, 0.5,  

 emiss_opt                           = 8, 8,

 bio_emiss_opt                                               = 3, 3,

 bioemdt                             = 30, 30,

 ne_area                             = 97,

 dust_opt                                            = 3,

 seas_opt                                            = 1,

 phot_opt                            = 4, 4,

 photdt                              = 30, 30, 

 aer_op_opt                                       = 1, 1,

 opt_pars_out                                   = 0,

 aer_ra_feedback                             = 0, 0,

 chem_conv_tr                                                = 0, 0,

 biomass_burn_opt                    = 2, 2,

 scale_fire_emiss                    = .true., .true.,

 plumerisefire_frq                   = 10, 10,

 have_bcs_chem                       = .true., .true.,

 is_full_tuv                         = .false., .false.,

 has_o3_exo_coldens                  = .false.,

 pht_cldfrc_opt                      = 1, 1,

 cld_od_opt                          = 1, 1, 

 /

 

 

 

 

&time_control

 run_days                            = 0,

 run_hours                           = 30,

 run_minutes                         = 0,

 run_seconds                         = 0,

 start_year                          = 2019, 2019,

 start_month                         = 09, 09,  

 start_day                           = 18, 18,  

 start_hour                          = 00, 00,   

 end_year                            = 2019, 2019,

 end_month                           = 09, 09,   

 end_day                             = 19, 19,   

 end_hour                            = 06, 06,   

 interval_seconds                    = 21600

 input_from_file                     = .true., .true.,

 history_interval                    = 360, 360, 

 frames_per_outfile                  = 1, 1,

 restart                             = .false.,

 restart_interval                    = 1440,

 io_form_history                     = 2,

 io_form_restart                     = 2,

 io_form_input                       = 2,

 io_form_boundary                    = 2,

 output_diagnostics                  = 0,

 nwp_diagnostics                     = 0,

 auxhist3_outname                    = "wrfxtrm_d<domain>_<date>"

 auxhist3_interval                   = 0, 0,

 auxinput5_inname                    = 'wrfchemi_<hr>z_d<domain>',

 auxinput6_inname                    = 'wrfbiochemi_d<domain>',

 auxinput7_inname                    = 'wrffirechemi_d<domain>_<date>',

 io_form_auxinput5                   = 2,

 io_form_auxinput6                   = 2,

 io_form_auxinput7                   = 2,

 auxinput5_interval_m                = 60, 60,

 auxinput7_interval_m                = 60, 60,

 frames_per_auxinput5                = 12,12,

 frames_per_auxinput6                = 1 ,1,

 frames_per_auxinput7                = 1, 1,

 debug_level                         = 0,

 force_use_old_data                  = .true.,

 

 /

 

 &domains

 time_step                           = 30,

 time_step_fract_num                 = 0,

 time_step_fract_den                 = 1,

 max_dom                             = 2,

 e_we                                = 204, 412,   

 e_sn                                = 192, 289,   

 e_vert                              = 61, 61,   

 dzbot                               = 30.

 dzstretch_s                         = 1.11,

 dzstretch_u                         = 1.10,

 p_top_requested                     = 5000,

 num_metgrid_levels                  = 138,

 num_metgrid_soil_levels             = 4,

 dx                                  = 12000,

 dy                                  = 12000,

 grid_id                             = 1, 2    

 parent_id                           = 1, 1,    

 i_parent_start                      = 1, 21,     

 j_parent_start                      = 1, 46,    

 parent_grid_ratio                   = 1, 3,   

 parent_time_step_ratio              = 1, 3,    

 feedback                            = 0,

 smooth_option                       = 0,

 /

 

 &physics

 mp_physics                          = 8, 8,   

 cu_physics                          = 16, 0,   

 ra_lw_physics                       = 4, 4,   

 ra_sw_physics                       = 4, 4,   

 bl_pbl_physics                      = 1, 1,    

 sf_sfclay_physics                   = 1, 1,    

 sf_surface_physics                  = 4, 4,   

 radt                                = 4,   

 bldt                                = 0, 0,     

 cudt                                = 0, 0,     

 icloud                              = 1,

 num_land_cat                        = 21,

 sf_urban_physics                    = 0, 0,    

 fractional_seaice                   = 1,

 num_soil_layers                     = 4,

 do_radar_ref                        = 0,

 cu_rad_feedback                            = .false., .false.,

 /

 

 &fdda

 /

 

 &dynamics

 hybrid_opt                          = 2,

 w_damping                           = 0,

 diff_opt                            = 1,     

 km_opt                              = 1,    

 diff_6th_opt                        = 0,    

 diff_6th_factor                     = 0.12,  

 base_temp                           = 290.

 damp_opt                            = 3,

 zdamp                               = 5000., 

 dampcoef                            = 0.2,   

 khdif                               = 0,     

 kvdif                               = 0,     

 non_hydrostatic                     = .true.,

 moist_adv_opt                       = 1,    

 scalar_adv_opt                      = 1,    

 gwd_opt                             = 1,    

 /

 

 &bdy_control

 spec_bdy_width                      = 5,

 specified                           = .true.

 /

 

 &grib2

 /

 

 &namelist_quilt

 nio_tasks_per_group = 0,

 nio_groups = 1,

 /