ToStringWithData leading to high memory/CPU

[abko-auto]

We had incident where our service in production was using CPU more than 60%. Upon closer look at memory, lot of allocation from ToStringWithData.

exception.ToString was active and exception had _innerException set so ToString was active ~6%.

Mostly this logging I saw in Microsoft.Azure.Documents.GoneAndRetryWithRetryPolicy!ShouldRetryAsync function. There are lot of DefaultTrace.TraceError statements, I doubt they are collected if default listeners are removed.

Can we have configuration flag for verbose logging - If enabled emit these errors else not. This is consuming more than %10 of cpu + allocating those many string objects putting pressure on GC.

[ealsur]

See #3703