Decrease Variance of LogNormal to Converge on Expected Value Sooner#301
Decrease Variance of LogNormal to Converge on Expected Value Sooner#301carlaKC wants to merge 3 commits intobitcoin-dev-project:mainfrom
Conversation
Previously we created our normal distribution with a very large sigma, which would result in our samples taking a long time to converge on our expected payment amount (around 100,000 samples required to get close with previous values). This commit changes our variance to still depend on payment channel size, but do so more gently. The values picked in this PR are somewhat reverse engineered - we ran a few experiments to find acceptable variance ranges, took a look at typical channel sizes in lightning and then worked backwards to find a relation between channel size and acceptable variance that would fit.
Speed up this test by using our sim clock that can speed up the simulation. The downside of this clock is that we may stop one payment over/under if we try to match exactly to the number of payments that we expect in a given period of time. Instead, we generate our set of expected payment and then run the simulation for much longer than we need to get this expected list, so we should always get at least this subset. We then just assert that the payments we get exactly match this first set.
|
Review note: I don't expect folks to go through the math of this one, Clara and I did it offline before this came up and she's been over the code. General review of code, any under/over flows that we could have with this math, test coverage etc is still very appreciated! |
|
@elnosh has pointed out that when you run this patch for ~6 months it drops down to around 20% payment successes (vs main which is around ~70%), at least on the test topologies we have. Perhaps this is a more realistic simulator, as network traffic isn't perfectly circular and channels do deplete, but it doesn't make for a very useful simulator. Going to spend some more time on design here to figure out what the best option is. |
|
Not actively working on this, warrants further investigation IMO. |
2 participants
See rationale here.
This PR makes a fix to the way we pick our
LogNormaldistribution's parameters which results in our getting to an average of ourexpected_payment_amountmuch sooner.As this changes the way we pick payments, the hard-coded ordering of some of our tests has to change.
While I'm here, I also make some changes to long-running unit tests to speed them up a bit.