#FeatureFriday: Checking and Changing Polling Cycles and Retention

Welcome to “Feature Friday”, a series of (typically short) videos which explain a feature, function, or technique.

One of the biggest challenges when maintaining a monitoring system is handling the sheer volume of data. While your initial thought may be “keep it all”, the reality is that this has implications with regard to everything from display speed (because data has to be culled from the database and loaded onto the screen and if the data set is large, the query time will be long) to storage (leading to the question “if you were the IT pro who had everything, where would you put it all?”).

The secret to effectively managing this issue lies in understanding and being able to tune polling cycles and retention times, which is what I discuss with my fellow SolarWinds Head Geeks Kong Yang, Patrick Hubbard in the video below.

For more insights into monitoring, as well as random silliness, you can follow me on Twitter (@LeonAdato) or find me on the SolarWinds THWACK.com forums (@adatole)

2 thoughts on “#FeatureFriday: Checking and Changing Polling Cycles and Retention

  1. I encountered an issue that made me consider data retention a couple of weeks ago. I was configuring some reports for granular information within solarwinds that is by default purged every 7 days and it was (unluckily for me) a weird week in the UK where we had two public holidays, one on a friday and one on a monday, this meant that four days had already elapsed.

    The detailed statistics I wanted for my reports, were generated on the previous Wednesday and Thursday and my reports were generated late on the next Thursday. So as I was generating the reports I was noticing my data was disappearing.

    The solution for me was to set the detailed statistics data retention to two weeks instead of one and now I can generate reports from a week ago without worrying my data is going to be incomplete.

    Unfortunately most of the data from the period I wanted was purged before I altered the setting. So I learned that detailed statistics retention is important and should always be considered especially at holiday times.

    1. Another option – especially for volitile data like you mention, is to leverage your backups – create a mirror database for the tables like this that do NOT have a deletion period (or which delete on a much slower rate) so you can dumpster dive through them as needed.

      Because the main culprit of the “7 day retention” window is syslog and trap data, I also STRONGLY suggest setting up a syslog and trap “filtration” layer to handle this type of message separately. I write about it in detail here: https://thwack.solarwinds.com/community/solarwinds-community/geek-speak_tht/blog/2017/06/13/kiwi-syslog-95-no-longer-trapped

Comments are closed.