# How to reduce Application Insights telemetry data amount

**URL:** <https://www.virtocommerce.org/t/how-to-reduce-application-insights-telemetry-data-amount/271>\
**Category:** hosting\
**Tags:** azure, platform-v3\
**Created:** [November 17, 2020, 10:14am UTC](https://www.virtocommerce.org/t/how-to-reduce-application-insights-telemetry-data-amount/271 "2020-11-17T10:14:06Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![akak1977](https://yyz2.discourse-cdn.com/flex032/user_avatar/www.virtocommerce.org/akak1977/32/239_2.png) [@akak1977](https://www.virtocommerce.org/u/akak1977)\
**Post date:** [November 17, 2020, 10:14am UTC](https://www.virtocommerce.org/t/how-to-reduce-application-insights-telemetry-data-amount/271/1 "2020-11-17T10:14:06Z")

</div>

There is the following problem with “overlogging” in Application Insights: VM or AppService resources drain. It’s very specific to your system, depending on modules, types of loads, etc., but too much telemetry can really slow your system.  
You should be aware if you see something, like:

1. Number of threads to service non-proportionally high for your system load (you have a couple or triple clients per second, but thousands of threads);
2. You see CPU burning;
3. If you have a dump, you discovered a lot of AI-specific active calls.

The solution is to reduce the amount of telemetry data passing by your system to Application Insights.  
The platform, starting from v3.32, contains an ability to tune Application Insights telemetry by:

1. Setting up standard telemetry sampling processors;
2. Additional SQL-dependency ignoring processor.

## Standard telemetry sampling processors

There are two standard sampling processors in Application Insights: adaptive and fixed. To learn the difference and detailed settings info take the topic [Sampling in Application Insights](https://docs.microsoft.com/en-us/azure/azure-monitor/app/sampling).  
Unfortunately, `ApplicationInsights.config` not supported with [ASP.NET](http://ASP.NET) Core, but we implemented equivalent settings in the platform.  
Look at the section `VirtoCommerce:ApplicationInsights:SamplingOptions` in appsettings.json:

> ```
> "SamplingOptions": {
> "Processor": "Adaptive",
> "Adaptive": {
> "MaxTelemetryItemsPerSecond": "5",
> "InitialSamplingPercentage": "100",
> "MinSamplingPercentage": "0.1",
> "MaxSamplingPercentage": "100",
> "EvaluationInterval": "00:00:15",
> "SamplingPercentageDecreaseTimeout": "00:02:00",
> "SamplingPercentageIncreaseTimeout": "00:15:00",
> "MovingAverageRatio": "0.25"
> },
> "Fixed": {
> "SamplingPercentage": 100
> },
> "IncludedTypes": "Dependency;Event;Exception;PageView;Request;Trace",
> "ExcludedTypes": ""
> }
> 
> ```

You can choose the sampling processor in the `Processor` key between `Adaptive` and `Fixed` and tune each of them under sections `Adaptive` and `Fixed` respectively.

## SQL-dependency ignoring processor

We implemented this additional processor to have an ability to ignore SQL-related dependencies, containing specified substrings. If one of the strings found in the dependency SQL statement it will be not sampled by Application Insights.  
To enable SQL telemetry in dependencies set the value for key `VirtoCommerce:ApplicationInsights:EnableSqlCommandTextInstrumentation` to `true`. To setup the set of ignoring substrings, change the array value under key `VirtoCommerce:ApplicationInsights:IgnoreSqlTelemetryOptions:QueryIgnoreSubstrings`.  
Full example of `VirtoCommerce:ApplicationInsights` section:

> ```
> "ApplicationInsights": {
> "SamplingOptions": {
> "Processor": "Adaptive",
> "Adaptive": {
> "MaxTelemetryItemsPerSecond": "5",
> "InitialSamplingPercentage": "100",
> "MinSamplingPercentage": "0.1",
> "MaxSamplingPercentage": "100",
> "EvaluationInterval": "00:00:15",
> "SamplingPercentageDecreaseTimeout": "00:02:00",
> "SamplingPercentageIncreaseTimeout": "00:15:00",
> "MovingAverageRatio": "0.25"
> },
> "Fixed": {
> "SamplingPercentage": 100
> },
> "IncludedTypes": "Dependency;Event;Exception;PageView;Request;Trace",
> "ExcludedTypes": ""
> },
> "EnableSqlCommandTextInstrumentation": true,
> "IgnoreSqlTelemetryOptions": {
> "QueryIgnoreSubstrings": ["[HangFire].", "sp_getapplock", "sp_releaseapplock" ]
> }
> }
> 
> ```

Upd.: Also, the platform, starting from v3.34, can [configure every available standard Application Insights setting, thru platform appsettings.json](https://docs.microsoft.com/en-us/azure/azure-monitor/app/asp-net-core#configuration-recommendation-for-microsoftapplicationinsightsaspnetcore-sdk-2150--above).