Problem with creating an hydrologic enforced DEM with WBT

[Thomas Wanderer]

Hi. I have a problem creating a proper hydrologic enforced DEM (From a SRTM DEM) in a workflow that has the following steps:

The sequence is to first apply a filter to the SRTM DEM to smoothen it, then removing single cell pits and afterwards use either the breach_depressions or fill_depressions tools to fill remaining depressions in my DEM. I optionally burn streams from OSM in my prepared DEM and then create a D8 pointer grid (d8_pointer) and then try to delineate for instance the streams based on it.

My study area is in the Swiss alps, so I have a pretty well defined orographic situation with streams from the hillslopes and a confluence in the valleys. Unfortunately I am struggling to get a proper result that reflects this situation. In the following image one can see that the delineated streams from hillslopes follow downhill, but never join in the valley (Blue = OSM stream).

The reason is a DEM with no clear flow direction in the valleys due to man small depression patches in my DEM, which I thought to get rid of with either the breaching or the filling tool. Both do not result in a DEM from which I can generate a proper d8 pointer grid.

I'm using the filling tool for instance with no max depth parameter, so I would expect the depressions in my valley floors getting filled but they do not. Any input is appreciated. Maybe my approach is flawed or I use the wrong sequence of DEM preparations?

Thanks already for any hints.

[Thomas Wanderer]

I have made a comparison between the ArcGIS Hydro Tools fill and the WBT fill tool.

ArcHydro Fill:

WBT Fill:

The results look pretty similar with the only difference, that ArcGIS uses integers (therefore creating  plain level) values and WBT tool has small floating increments (I guess to avoid no flow areas). The latter behaviour leads in the WBT D8 pointer grid to an unclear flow direction. As an example look at the following area of the DEM displaying a larger valley floor (WBT fill and d8 pointer grid)

 

[Jean-François Bourdon]

How exactly are you burning your OSM streams? In you process, you doesn't seem to do any filling/breaching after your stream burning so if you simply set all raster values below the streams lines as the same low value, you will endup with undetermined flow direction (what your the map on your first post seems to indicate).

Have you tried your same workflow but without burning the OSM streams?

[Thomas Wanderer]

The stream burning is optional as I mentioned. The above Fill example happens even before it, so its output was not affected by any stream burning at all. My problem is the Filling or Breaching on the DEM. It results in a very noisy DEM with no clear flow direction.

[Whitebox Geospatial]

Hello, 

What specific tools in WhiteboxTools are you using for your filling and breaching? For Breaching, I would suggest using the BreachDepressionsLeastCost as opposed to the BreachDepression algorithm. Overall parameter setting for both breaching algorithm is not as straight forward as other tools and requires some user experimentation. 

Best, 

Anthony

[Thomas Wanderer]

I am already using the  BreachDepressionsLeastCost as it was advertised in the WBT tool documentation. I use the following parameters:

wbt.breach_depressions_least_cost(
  dem_raster,
  out_file,
  5,
  max_cost=None,
  min_dist=True,
  flat_increment=None,
  fill=True
)

My model (see graph above) actually is parametrizable, so I can set if I want to use normal filling (fill_depressions) or (breach_depressions) or (breach_depressions_least_cost). I tried all of them already. But none of my results is usable. Looking at the derived D8 pointer from my valley bottom I always get noisy results. My expectation would be that using one of the WBT breaching/filling tools would remove exactly that by filling the area or establishing a distinct breaching path and keeping a small increment to avoid no flow areas. In contrast to the WBT tools I get this D8 pointer grid from the  ArcHydro filled DEM. I have a flat no flow area, but at least I have no noisy undirected pointer grid.

In all my examples, the only preparation steps where filtering the DEM, removing single Cell pits and then filling/breaching with the mentioned WBT tools.

WBT D8 pointer grid based on DEM filled with ArcHydro:

I am not sure if I need to filter the DEM stronger to remove artefacts, but in general I was hoping to achieve something similar like with ArcHydro

Thanks for all the feedback so far.

[Hugh Allan]

Hey Tom,

Without your exact data and code/workflow for this exact example, I can't be 100% sure.

Because I can't see it in your workflow image, my understanding is you'll need to calculate flow accumulation after you get your d8 pointer grid. The pointer grid tells you which way each grid flows, and the flow accumulation tells you how many cells (or area etc) flows into each cell. Ultimately, streams are then extracted based on a minimum threshold for number of cells in the flow accumulation grid, using the extract streams function (also not specifically in your workflow?).

I have followed this example https://vt-hydroinformatics.github.io/rgeoraster.html (which is run in R), that hopefully you can interpret into your platform based on the names of the functions. It steps through all the stages of extracting streams and watersheds, right from the initial DEM.

If you have done all the steps in the link above, my next guess would be that the filling or breaching values/distances are insufficient - maybe try increasing the values at these stages of the workflow? Likewise for the thresholds when extracting streams - if this value is too high you won't get any streams extracted (but it doesn't look like this is the problem based on your image).

As for burning the OSM streams into the DEM - this is not something that I'm familiar with, so unless there's something special about your situation, I don't see why it should be required. As in the link above, streams can be extracted without needing to do this. I might be wrong as my knowledge is very infant with all this, and I'm happy to be enlightened otherwise.

Hope this helps mate,

Hugh

[Thomas Wanderer]

Thank you Hugh,

I have checked it and actually do the same as in your R based instructions. All my attempts and playing a bit with parameters lead to the same problem, that I do not get a proper filled or breached DEM (as seen in this DEM area) and therefore the flow direction and flow accumulation in WBT do not yield in a correct stream network. 

Doing the same steps in ArcHydro tools, works without an issue: I have attached my SRTM DEM to this message, to test with WBT tools:

[Whitebox Geospatial]

Hello all, 

Thank you for everyone's comments/insight. It is great to see these discussions in the google group!

Regarding twnovelt's last post... I don't think I would go as far as saying 

" that I do not get a proper filled or breached DEM (as seen in this DEM area) and therefore the flow direction and flow accumulation in WBT do not yield in a correct stream network". 

Many of the algorithms that exist in WhiteboxTools are implementations of already existing algorithms. In addition, how WhiteboxTools has implemented an algorithm may be different than how ArcGIS implements an algorithm. Just because an output from an ArcGIS tool has a more visually appealing outcome for ones' needs... does not mean that other implementations are wrong. In addition, it cannot be assumed that a filled D8pointer will look the same as a breached D8 pointer. In general those are fundamentally different algorithms, and they provide different solutions.  After taking some more time to explore the issue you were having, I have come up with a couple alternatives in WhiteboxTools that you are free to use/follow. 

In this example, I have attached what I believe is that same location in the DEM that you have highlighted. In this image, I followed the exact same workflow that you implemented. I first filtered the DEM using the FeaturePreservingSmoothing tool, I filled single cell pits using the FillSingleCellPits in WhiteboxTools and then I filled the DEM using FillDepressionsWangAndLiu in WhiteboxTools which uses Wang and Liu (2006) filling method. This is one of the most common filling techniques, even though it may not be the most efficient algorithm. This is the output I was able to produce. In addition, I tested this same DEM with the Planchon and Darboux 2002 algorithm and it produced the same result. 

 

In this example, I followed your same workflow as above (i.e. FeaturePreserving smoothing) but instead of filling I breached this DEM. specifically I used the LeastCostDepressionBreaching tool. I tried the parameters you used at first i.e. search distance = 5. I found this parameter set to not be satisfactory and I had to parametrize the tool more through experimentation with the other parameters. I was able to produce this result. In this image, there is a clear flow line that extends through the lake .In this breaching solution, I would not expect to see a lake with a uniform flow direction over the entire lake like the image you posted from the Arc HydroTools as this is a breaching technique.

 

I am able to share these parameters/workflows with you, please feel free to email us at sup...@whiteboxgeo.com. 

Best, 

Anthony 

[Thomas Wanderer]

Hi Anthony,

thanks a lot for your reply and the work you put into testing my case. That is really appreciated and the general responsiveness around WBT gives me a positive feeling about using it. On Thursday, 2 hours before leaving early into my weekend I have actually also given the  FillDepressionsWangAndLiu tool a try and discovered that it produces  similar results as ArcGIS and therefore put me on track again with my work. The documentation in general advices to use the Breaching LCP and this would also better fit my problem, but I was not successful as you saw. Of course I would be very interested to understand what parameters you used to achieve the above result where one can clearly see a flow line.

Best,

Thomas