Bangalore Days Index

[Valiant Puga]

Iam trying to understand the difference between log files and log indexes in terms of retention. We have a management server setup for Logging&Status, SmartEvent Viewer, and Correlation Unit. Log indexing is enabled and set to alert when space is below 20% and delete old files below 22%. Index files is set to delete older than 7 days. The folder /opt/CPrt-R80/log_indexes shows 7 days worth of index files. However the folder /opt/CPsuite-R80/fw1/log shows log files going back several months. We have cleared out log files from this folder before. With indexing set to delete older than 7 days we can only go back 7 days on reports but the log files themselves go back past 7 days. I am trying to understand the difference between the log files themselves and the indexing retention.

Logs are stored as the files under $FWDIR/logs, this is a part of MGMT Log Server functionality. There is no automatic built-in mechanism to remove old log files. The only option you have is to start removing older logs when disk space utilization reaches a certain threshold. I have highlighted this part with the blue rectangle.

Log indexing is done by an indexing engine, and the indexes are stored to $RTDIR/log_indexes. You can set the maximum depth of indexing, which is important for Event Analysis performance and stability. The indexer has a built in retention option, and older indexes are routinely removed.

The main reason not to remove logs automatically is simple. You may want to keep your security logs to maintain ability of investigating past breaches and other security incidents. In some cases compliance regulations require keeping up to 2 years of logs available.

So to manage log retention I would advise you to run a cron task with a script that performs backup and removal of older logs. There are quite a few publicly available samples of such a script. On of example is here: Log Backup/Archive Script

When running reports and searching logs from SmartDashboard is it only going to look at the index log files which in our case log index retention is 7 days? If so, how would you then search the log files past the 7 days?

I believe that is correct, you will be able to run reports for 7 days worth of logs as both indexes and log files must be present to be seen in R80x. In order to go back further you would need to re index those log files which is detailed in the R80 admin guide and sk111766. Once the setting is made you will observe higher CPU/RAM consumption until all those log files have been indexed (could take hours and even days depending on your resources/amount of logs).

Index files allow you to see/search logs for a period, for which you have these index files (a week in your case). But you still can open separate log files and check some traffic there, but you would be able to see/search only in this specific file (for example, for a day if you set up a midnight log file switch).

I am in a situation where i can only find logs going back less than a week. Considering that Indexes are configured to be deleted at 30 days and old log files when disk space is below 5 GB. Disk space is currently at 20 GB and there are no signs it was ever below this.

The Standardized Precipitation Index (SPI) is a widely used index to characterize meteorological drought on a range of timescales. On short timescales, the SPI is closely related to soil moisture, while at longer timescales, the SPI can be related to groundwater and reservoir storage. The SPI can be compared across regions with markedly different climates. It quantifies observed precipitation as a standardized departure from a selected probability distribution function that models the raw precipitation data. The raw precipitation data are typically fitted to a gamma or a Pearson Type III distribution, and then transformed to a normal distribution. The SPI values can be interpreted as the number of standard deviations by which the observed anomaly deviates from the long-term mean. The SPI can be created for differing periods of 1-to-36 months, using monthly input data. For the operational community, the SPI has been recognized as the standard index that should be available worldwide for quantifying and reporting meteorological drought. Concerns have been raised about the utility of the SPI as a measure of changes in drought associated with climate change, as it does not deal with changes in evapotranspiration. Alternative indices that deal with evapotranspiration have been proposed (see SPEI).

Uses precipitation only; can characterize drought or abnormal wetness at different time scales which correspond with the time availability of different water resources (e.g. soil moisture, snowpack, groundwater, river discharge and reservoir storage)

As a measure of water supply only, the SPI does not account for evapotranspiration, and this limits its ability to capture the effect of increased temperatures (associated with climate change) on moisture demand and availability

The SPI addresses this challenge by comparing the precipitation total for the chosen interval against a cumulative probability distribution for the precipitation data (for the identical interval). For example, what is statistical interpretation of the one-month precipitation total (e.g., 29 mm), compared to all known one-month totals? Obviously, the geographic location and time of year are important restrictions; if the precipitation was during April 2005 in southwestern Idaho, its magnitude should only be judged against April data from other years, in southwestern Idaho. Thus, it is necessary to view the drought according to the climatological norms for the location and season.

The SPI can compute drought intensity over any desired interval, e.g., one month, five months or 200 days. However, one of the most powerful features of the SPI is its intrinsic ability to simultaneously assess drought over a suite of timescales. For example, the precipitation totals for one, three, six, 12, 18, 24, 36, 48, and 60 month durations are routinely used by researchers to compute the SPI for the same respective intervals. A plot of SPI values from southwestern Idaho, for two different timescales, is shown in the first figure.

Precipitation is known to follow an asymmetric frequency distribution, with the bulk of the occurrences at low values, and a rapidly decreasing likelihood of larger precipitation totals. There are a number of such positively-skewed analytical distributions, six of which were analyzed for SPI computations by Guttman (1999). The distribution for the SPI adopted by McKee et al. (1993), as well as the NDMC, is the incomplete gamma distribution. SPI algorithms analyze the input data to optimally estimate two key coefficients which govern the transformation, and the observed precipitation data are transformed to Gaussian (normal) equivalents. The transformed precipitation data are then used to compute the dimensionless SPI value, defined as the standardized anomaly of the precipitation:

SPI values for five month and sixty month timescales, in Idaho climate zone 5 (Southwestern Valleys). The sixty month SPI clearly tracks the long-term drought pattern. The SPI parameters for the data transformation were constructed using 111 years of observations. Contributed by J. Keyantash

SPI labels and their relationship to the normal curve. The intensity implied by each label corresponds to the degree of removal from mean conditions (i.e., SPI=0). The percentages printed within the regions bounded by the dashed lines indicate the probability for SPI values to fall within that region only; overall cumulative probabilities require summing the probabilities from the SPI datum of interest through a tail of the curve. For example, SPI values of -1.5 or lower occur in only six percent (2% + 4%) of cases. The sum of all of the stated values is less than 100% due to rounding. Contributed by J. Keyantash.

External Affairs Minister S Jaishankar has dismissed the World Happiness Index 2023, which ranked India among the least happy countries. Interacting with students during BJYM Yuva Samvada at the RV Dental College in Bengaluru, Jaishankar cited an example of his friend from Singapore who once said Indians looked happier than Europeans.

"You should actually tell everybody to come to Bengaluru - you can see which is the world's happiest place, particularly on a Friday night," the foreign minister said while replying to a question on the World Happiness Index 2023.

"I don't know how these guys make up those indexes. A Singaporean friend of mine told me - every time he travels out, he says 'I just have to go to some place and look at the faces of some people, that is my happiness index'. His sense was people in Europe did not look that happy. People, to him in India, looked very happy. I'm not necessarily saying I will make an index out of it, but these are all mind games that people play," the minister said.

Last week, filmmaker Vivek Agnihotri rejected the rankings, calling it 'crap'. He said questions asked to measure one's happiness were Western. He said that Western countries would fare badly if people there are asked questions like, how many days they ate with their family and if they can depend on their family for a lifetime.

3a8082e126