Re: [fds-smv post:6638] Re: Validation of FDS Sprinkler Activation Times Pred...

[FPESC...@aol.com]

I have been reviewing the data from Tests P-1 through P-5 of the UL/NFPRF fire tests from 1998.  A quick summary of the test data is as follows:

 

First Sprinkler Activation Time; Second Sprinkler Activation Time; Maximum Near Ceiling Temperature

 

Test P-1:  76 sec.; 134 sec.; 454F

Test P-2:  100 sec.; 108 sec.; 437F

Test P-3:  67 sec.; 72 sec.; 416F

Test P-4:  93 sec.; 94 sec.; 347F

Test P-5:  74 sec.; 75 sec.; 353F

 

With all 5 fire tests using the same storage array, ignition source and ignition location with respect to the sprinklers, the apparent randomness of the data above would make it seem that it is virtually impossible to make reasonably accurate predictions as to the operation times of even the first activating sprinkler.  (Of course, the location of the roof vent directly over the fire in Test P-2 is the plausible explanation for Test P-2.)  If predicting the activating time of the first activating sprinkler is difficult, predicting the activating times of multiple sprinklers in any sort of reasonably accurate way would seem impossible.

 

Similarly, predicting the activating times of a smoke/heat vent in a building protected by standard spray spray (control-mode) sprinklers in any sort of reasonably accurate fashion would also appear to be impossible.

 

I would appreciate any thoughts or comments on the above, particularly in light of Dr. McGrattan's comments from February 17, 2009 (below).

 

Richard Schulte

Schulte & Associates

Building Code Consultants

Chicago/New Orleans

fpesc...@aol.com

 

 

In a message dated 2/17/2009 5:03:49 P.M. Central Standard Time, mcgr...@gmail.com writes:

The purpose of the FDS Validation Guide is to present comparisons of
FDS predictions against full-scale measurements. We work very hard to
present the data in a form that enables those who use FDS, or those
who are thinking of using it, to decide for themselves if the model is
appropriate for a given application. We do not believe that our role
is to say whether or not the model is appropriate because we cannot be
sure about what the application could potentially be or what the
required level of accuracy should be. We prefer that people use their
own judgment to decide what is the best tool for the job. That is
essentially what you are doing. You are making an argument that the
model is not sufficiently accurate to predict multiple sprinkler
activation. We do not want to make such a statement because we don't
know exactly what you intend to use the model for, and furthermore,
there is no consensus metric in fire protection engineering by which a
model is considered validated or not for a particular application. We
prefer to do the technical work in developing the model and
quantifying its accuracy as we have done in the Validation Guide. We
prefer to leave the decision about validation up to you. We even
provide you with this forum by which you and others can discuss the
merits of the model for this and other applications. We make the
source code available for those who want to check the model
themselves, or publish their results in the open literature. We feel
that an open discussion of model strengths and weaknesses is healthy,
and we do everything we can to promote it.

In that spirit, let me point out the second plot in Figure 6.2.
Throughout the Validation Guide, there are scatterplots similar to
those shown here, except all the other scatterplots have off-diagonal
lines that represent the estimated experimental uncertainty. All large
scale fire experiments have a considerable amount of uncertainty in
the reported heat release rate, environmental conditions, sprinkler
characteristics (like droplet size, RTI, etc), and various other
parameters that are input into the fire model. Because of the
complexity of the experiments and simulations of fires in large
warehouse-type facilities, especially those involving multiple
sprinkler activations, we do not have a good way (yet) of quantifying
the experimental uncertainty. It might be as hard to do that as to
predict the experimental results themselves. So rather than try to
quantify the experimental uncertainty, we have added the second plot
in Figure 6.2. In the UL/NFPRF test series, Phase I, there were 22
experiments, all involving a heptane spray burner and a heat release
rate of approximately 4.4 MW. Of those 22 tests, there were three
replicate tests (Tests 1 and 8, Tests 4 and 7, and Tests 9 and 10).
These were not designed as replicates, but in each case, a vent was
either closed for the duration or did not activate, making the two
tests essentially the same. The second plot in Fig. 6.2 compares the
measured activation times for the sprinklers in one test against the
measured activation times in the other (replicate) test. This is only
comparing one experiment against another. This has nothing to do with
FDS. For example, in Test 8, four sprinklers activated at about 4.5
min after ignition whereas in Test 1, these same four sprinklers
activated after about 2 min. There was even a sprinkler that activated
after 6 min in Test 8 and after about 2.25 min in Test 1.

This information tells us something about the reproducibility of large
scale sprinkler experiments. It is not an indictment of the testing
lab, UL, because this sort of behavior is not surprising for those who
do this sort of testing. I observed these experiments, and I noted
that following the first activation, there was a considerable effect
on the fire because these sprinklers release about 1 gallon of water
per second. The burner was placed exactly between four sprinklers each
test, and because there is some variability in the activation
temperature of a real sprinkler, there was usually one sprinkler that
activated a few seconds before the others, which caused the fire, the
plume, and the subsequent activations to trend in a particular
direction. FDS has no such bias -- the sprinklers in these
calculations were programmed to activate at exactly 74 C (165 F). I
suppose that we could build in a random component to the activation
temperature to mimic reality, but we worry that this would simply add
an additional uncertainty to an already complicated problem. We prefer
that the model produce a result that, on average, compares favorably
with a number of replicate tests. The fact that FDS sometimes over-
predicts and sometimes under-predicts the number of activations is a
good thing. Our goal is to predict the total number of activations and
the average activation time of each "ring" of sprinklers. We are less
concerned about one or two outliers because we know that there is a
randomness to this kind of experiment that simply cannot be
predicted.

This kind of information is part of what goes into deciding if the
model is appropriate for your purpose. It is my job to provide you
with as much information as I can so that you can make an informed
judgment. But it is not my place to tell you that the model is right
for you. You decide. Ask me questions about the data if something is
not clear. But I hope you understand that I simply cannot make a
blanket statement like "FDS is validated for predicting multiple
sprinkler activations." You have made an argument above that it is
not, and you have every right to that opinion.




On Feb 17, 4:00 pm, FPESCHU...@aol.com wrote:
> ...
>
> read more »
>
> Figure 6.2 of the FDS Validation Guide does indeed present  data comparing
> predicted activation times to actual activation times.  The  data provided in
> Figure 6.2 appears to me to indicate that the FDS doesn't do a  very good job of
> predicting sprinkler activation times.  For example, one  point in Figure 6.2
> indicates that the predicted activation time of one  sprinkler is 150
> seconds, while the actual activation time is 360  seconds.
>
> If you enlarge Figure 6.2, the differences in the predicted  sprinkler
> activation times versus actual activating times becomes much more  apparent.  (The
> size of Figure 6.2 influences your perception of the  meaning of the Figure.)
>
> Other points of note regarding Figure 6.2 is that the  tests were all
> conducted with the same sprinkler, the same sprinkler  operating pressure, same
> sprinkler temperature rating, the same sprinkler RTI  and only 5 of the tests
> utilized fires in high-piled combustible arrays,  rather than heptane spray burners.
>
> More convincing evidence that the capability of the FDS was  validated would
> have a been a figure which isolates the FDS  predictions for the 5 tests with
> high-piled combustible arrays.  (See  Dr. Craig Beyler's comments regarding
> this in my previous post.)  It should also be noted that the paragraph in the
> Validation  Guide which immediately precedes Figure 6.2 indicates that the
> median droplet  size greatly influences the number of sprinklers which activate.
>
> It should also be noted that the none of the data shown in  Figure 6.2
> involved tests with dry pipe sprinkler systems.
>
> A detailed analysis of the data presented in the Validation  Guide indicates
> that the FDS typically under-predicts the activation time of the  first
> sprinkler to operate.  Of the 21 tests on which data is provided, the  FDS
> under-predicts the activation time of the first sprinkler in 17 of the  tests.  (It
> should be noted that sprinklers were not used in one of  the 22 tests.  Hence,
> data from only 21 of the tests is pertinent to the  discussion.)  If I recall
> correctly, this same observation regarding the  prediction of activation times
> of the first operating sprinkler is  included in NUREG-1824.
>
> In 3 of the tests, the activation time of the first sprinkler  is
> over-predicted however.  Hence, it cannot be stated that the FDS  consistently
> under-predicts the activation time of the first sprinkler to  operate.
>
> With respect to the prediction of the number of sprinklers  which activated,
> the data shows 6 exact predictions, 10 over-predictions of the  number of
> sprinklers which activated, and 5 under-predictions.  Of the 21  tests, the FDS
> was only able to predict the number of sprinklers which activate  in the range
> of + 1 sprinkler in 12 of the 21 tests.  In other  words, 42.8 percent of the
> tests (9 of 21), the FDS either over- or  under-predicted the number of
> operating sprinklers by more than one  sprinkler.
>
> Again, it should be noted that all of the data provided  utilized
> heptane-spray burners with the same sprinkler, same pressure etc.,  etc., etc.
>
> In my mind, the data provided in the Validation Guide does not  in any way
> constitute of validation of the capabilities of the FDS to accurately  predict
> the activation time of multiple sprinklers or the number of sprinklers  which
> activate.  In other words, the information provided in the Validation  Guide is
> simply data, but not a validation, and certainly not a validation  without
> any reservations or caveats as maintained by at least one expert in the  use of
> the FDS.
>
> Given the above, it is my opinion that the FDS Development  Team should be
> able to easily respond to the questions about the validation of  the FDS
> regarding sprinkler activation times and number of sprinklers which  activate.  In my
> mind, the one outstanding question is why the FDS  Development Team appears
> to be so shy about responding to the questions  posed.
>
> In one of Clint Eastwood's movies, Eastwood said "A man has  got to know his
> limits."  Users of the FDS should know the limits of the  validation of the
> FDS, particularly when a highly-respected expert maintains  that there are no
> limitations on the validation of the FDS with respect to  accurately predicting
> the activation times of multiple sprinklers and accurately  predicting the
> number of sprinklers which will operate.
>
> Is that highly-respected expert wrong about the validation of  the FDS for
> the purpose of predicting sprinkler activation times and number of  sprinkler
> activations or that expert correct?  Billions of construction  dollars are
> riding on the answer to that question.
>

 

---

2009 3 Free CREDIT SCORES: See Your 3 Credit Scores from All 3 Bureaus FREE!