Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
519185f
add .gitignore
sebulino May 13, 2022
cc1e862
introduce register payment method on OracleLightningNetwork.py
sebulino May 13, 2022
4832253
adding settlement_payment in OracleLightningNetwork
sebulino May 13, 2022
afa6a71
rewriting setter for `actual_liquidity` using properties (python way)
sebulino May 13, 2022
4ecadee
adjust channels in both directions when payment is made
sebulino May 16, 2022
a131eaf
Settlement of onions with partial payments only after all onions were…
sebulino May 16, 2022
0b0e868
deletion of unused import
sebulino May 16, 2022
8dffa28
shell for a Payment Class
sebulino May 17, 2022
8df2c57
Revert "shell for a Payment Class"
sebulino May 17, 2022
c03fe39
shell for a Payment Class for issue #20
sebulino May 17, 2022
7170150
shell for as Attempt Class for issue #20
sebulino May 17, 2022
16bad26
Indroduction of Payment and Attempt class, refactoring.
sebulino May 20, 2022
8727759
Introduction of PaymentClass and AttemptClass, refactoring.
sebulino May 20, 2022
f6dad46
error fixing, semantic changes and refinement of Payment ant Attempt …
sebulino May 26, 2022
40967ff
adding types in methods for Attempt and Payment Class and amending re…
sebulino May 26, 2022
5903cd2
initializes fee and probability w/ none (instead of -1) #20
sebulino May 29, 2022
c1c61a0
adjustment of relative paths for modules in import
sebulino May 29, 2022
54e905c
extending .gitignore
sebulino May 29, 2022
cf25aa0
Merge branch 'renepickhardt:main' into main
sebulino May 29, 2022
cf9c4f1
Merge branch 'payments_attempts'
sebulino May 29, 2022
9d56f65
Merge remote-tracking branch 'origin/main'
sebulino May 29, 2022
3e69225
correcting tipo for gitignore
sebulino May 29, 2022
bf14533
init for documentation
sebulino May 29, 2022
97d01cb
configuration updated
sebulino May 29, 2022
c8bfeb8
correction in filter function for Attempt.Status #24
sebulino May 29, 2022
fd232b5
changing imports back to initial call
sebulino Jun 18, 2022
0730a91
fixing issues from partial review
sebulino Jun 18, 2022
f0bda8b
deleting docs folder and moving _estimate_payment_statistics into att…
sebulino Jun 21, 2022
29990a2
deleting docs folder and moving _estimate_payment_statistics into att…
sebulino Jun 21, 2022
081d756
removed doc folder
sebulino Jun 21, 2022
c404105
adding CHANGELOG
sebulino Jun 21, 2022
3c850fe
Cleaning up after Attempt and Payment PR review
sebulino Jun 21, 2022
111a691
changed filter function in payment to generator
sebulino Jun 22, 2022
c320189
added setter to OracleChannel and fixed imports in init.py
renepickhardt May 30, 2022
de25111
Fix typos in documentation
nassersaazi Jun 19, 2022
40fb9be
rewriting setter for `actual_liquidity` using properties (python way)
sebulino May 13, 2022
54ccdc3
Indroduction of Payment and Attempt class, refactoring.
sebulino May 20, 2022
a40e61a
Introduction of PaymentClass and AttemptClass, refactoring.
sebulino May 20, 2022
1a05a21
adjustment of relative paths for modules in import
sebulino May 29, 2022
d494c0d
changing imports back to initial call
sebulino Jun 18, 2022
fc1e8d5
another rebasing hassle ;)
sebulino Jun 22, 2022
8541a28
Merge branch 'main' into payments_attempts
renepickhardt Jun 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
.DS_Store
__pycache__
.idea
pickhardtpayments/pickhardtpayments.egg-info
pickhardtpayments/__pycache__

# not including large file with Channel Graph
*.json
examples/.json

pickhardt_pay.log
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

<!--
TODO: Insert version codename, and username of the contributor that named the release.
-->
## [Unreleased]

## [0.1.0] - 2022-06-21
### Added
- introduction of an Attempt Class and a Payment Class ([#28])
- introduction of an AttemptStatus to describe the state of the Attempt ([#28])
- settle_payment in OracleLightingNetwork is added ([#28])
- logging added in SyncSimulatedPaymentSession ([#28])

### Changed
- calculation of fees and probabilities is moved from SyncSimulatedPaymentSession to Attempt class ([#28])

### Deprecated

### Removed

### Fixed

### EXPERIMENTAL
File renamed without changes.
2 changes: 1 addition & 1 deletion examples/basicexample.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,4 @@
C_OTTO = "027ce055380348d7812d2ae7745701c9f93e70c1adeb2657f053f91df4f2843c71"
tested_amount = 10_000_000 #10 million sats

payment_session.pickhardt_pay(RENE,C_OTTO, tested_amount,mu=0,base=0)
payment_session.pickhardt_pay(RENE, C_OTTO, tested_amount, mu=0, base=0)
136 changes: 136 additions & 0 deletions pickhardtpayments/Attempt.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
from enum import Enum

from Channel import Channel
from pickhardtpayments import UncertaintyChannel


class AttemptStatus(Enum):
PLANNED = 1
INFLIGHT = 2
ARRIVED = 4
FAILED = 8
SETTLED = 16


class Attempt:
"""
An Attempt describes a path (a list of channels) of an amount from sender to receiver.

# TODO Describe life cycle of an Attempt

When sending an amount of sats from sender to receiver, a payment is usually split up and sent across
several paths, to increase the probability of being successfully delivered. Each of these paths is referred to as an
Attempt.
An Attempt consists of a list of Channels (class:Channel) and the amount in sats to be sent through this path.
Comment thread
sebulino marked this conversation as resolved.
When an Attempt is instantiated, the given amount is allocated to the inflight amount in the channels of the
path and the AttemptStatus is set to PLANNED.

:param path: a list of Channel objects from sender to receiver
:type path: list[Channel]
:param amount: the amount to be transferred from source to destination
:type amount: int
"""

def __init__(self, path: list[UncertaintyChannel], amount: int = 0):
"""Constructor method
"""
if amount >= 0:
self._amount = amount
else:
raise ValueError("amount for payment attempts needs to be positive")

i = 1
valid_path = True
while i < len(path):
valid_path = valid_path and (path[i - 1].dest == path[i].src)
i += 1
if valid_path:
self._path = path

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the difference to self._paths from line 32?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the real one ;)


channel: UncertaintyChannel
self._routing_fee = 0
self._probability = 1
for channel in path:
self._routing_fee += channel.routing_cost_msat(amount)
self._probability *= channel.success_probability(amount)
# When Attempt is created, all amounts are set inflight. Needs to be updated with AttemptStatus change!
# This is to correctly compute conditional probabilities of non-disjoint paths in the same set of paths
# channel.in_flight(amount)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far so good and makes sense

channel.allocate_amount(amount)
Comment thread
sebulino marked this conversation as resolved.
self._status = AttemptStatus.PLANNED

def __str__(self):
description = "Path with {} channels to deliver {} sats and status {}.".format(len(self._path),
self._amount, self._status.name)
if self._routing_fee and self._routing_fee > 0:
description += "\nsuccess probability of {:6.2f}% , fee of {:8.3f} sat and a ppm of {:5} ".format(
self._probability * 100, self._routing_fee/1000, int(self._routing_fee * 1000 / self._amount))
return description

@property
def path(self) -> list[Channel]:
"""Returns the path of the attempt.

:return: the list of Channels that the path consists of
:rtype: list[Channel]
"""
return self._path

@property
def amount(self) -> int:
"""Returns the amount of the attempt.

:return: the amount that was tried to send in this Attempt
:rtype: int
"""
return self._amount

@property
def status(self) -> AttemptStatus:
"""Returns the status of the attempt.

:return: returns the state of the attempt
:rtype: AttemptStatus
"""
return self._status

@status.setter
def status(self, value: AttemptStatus):
"""Sets the status of the attempt.

A flag to describe if the path failed, succeeded or was used for settlement (see enum SettlementStatus)

:param value: Current state of the Attempt
:type value: AttemptStatus
"""
if not self._status == value:
# remove allocated amounts when Attempt status changes from PLANNED
if self._status == AttemptStatus.PLANNED and not value == AttemptStatus.INFLIGHT:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not inflight?

I am wondering if we should have a setter API or if we can only inc status by 1.

That being said I see the idea. While i think it is -given the design of the code- the only reasonable option i think it demonstrates that I messed something up. (It feels really wrong to do the amounts tweeking here)

for channel in self._path:
channel.allocate_amount(-self._amount)

if self._status == AttemptStatus.INFLIGHT and value == AttemptStatus.ARRIVED:
# TODO write amount from inflight to min_liquidity/max_liquidity
# for channel in self._path:
# channel.allocate_amount(-self._amount)
pass

self._status = value

@property
def routing_fee(self) -> int:
"""Returns the accrued routing fee in msat requested for this path.

:return: accrued routing fees for this attempt in msat
:rtype: int
"""
return self._routing_fee

@property
def probability(self) -> float:
"""Returns estimated success probability before the attempt

:return: estimated success probability before the attempt
:rtype: float
"""
return self._probability

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not entirely sure as the attempt class does not know the uncertainty network but we could compute this on the fly for the path (as the path contains such information)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is btw a strong attempt against it as non disjoint paths will change probability computation.

@sebulino sebulino May 20, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this is to collect statistical a priori information:
In _estimate_payment_statistics this is added when calling uncertainty_network.get_features_of_candidate_path. it's tied to this instance of attempt, so another Attempt on the same graph will have its own (then perhaps different probability). It should be purely descriptive and is not used for further calculation.

@sebulino sebulino May 20, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bias in probabilities in UncertaintyGraph?

When collecting possible paths in a round/"payment loop", all path candidates are assumed to be sent and settled successfully.
This is my assumption, because we iterate over each attempt:

  • UncertaintyChannel.success_probability freshly calculates probabilities from current channel balances, and
  • then allocate_amount_on_path is updated before the loop commences with the next possibly not disjoint path, that then gets its probabilities from the just before adjusted inflight amount.

Or am I reading it wrong?

Ist this something we can live with?
What would be possible ways to correct for this (like trying to immediately figure out if the onion would fail or not)?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This indeed the really messy book keeping that I introduced. I am happy to find a better solution to this! I need to allocate the amounts for planned onions to have probability correction of the next path correct. (Ask me for an example if you need one!) I remove all of the allocated HTLCs then before I do the actual sending / probing. It is really the most ugly part of my code :(

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just started to look at the code. I think this is btw exactly the difference between PLANNED and INFLIGHT while internally I increase the inflight counter I am actually in the planning phase and to compute the probabilities of planned paths I need to take into account that I already plan to allocate something sats to an edge (without knowing if that will work yet) So maybe we just need to move over the logic from the Enum to the state of a channel where we also have planned sats before we go and send onions and switch those to inflight sats. in this way I would note have to remove them before testing onions. does that make any sense? I am actually not sure myself if this would be any good.

4 changes: 2 additions & 2 deletions pickhardtpayments/Channel.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
class ChannelFields():
class ChannelFields:
"""
These are the values describing public data about channels that is either available
via gossip or via the Bitcoin Blockchain. Their format is taken from the core lighting
Expand All @@ -21,7 +21,7 @@ class ChannelFields():
SHORT_CHANNEL_ID = 'short_channel_id'


class Channel():
class Channel:
"""
Stores the public available information of a channel.

Expand Down
4 changes: 2 additions & 2 deletions pickhardtpayments/ChannelGraph.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from .Channel import Channel


class ChannelGraph():
class ChannelGraph:
"""
Represents the public information about the Lightning Network that we see from Gossip and the
Bitcoin Blockchain.
Expand All @@ -15,7 +15,7 @@ class ChannelGraph():

def _get_channel_json(self, filename: str):
"""
extracts the dictionary from the file that contains lightnig-cli listchannels json string
extracts the dictionary from the file that contains lightning-cli listchannels json string
"""
f = open(filename)
return json.load(f)["channels"]
Expand Down
18 changes: 13 additions & 5 deletions pickhardtpayments/OracleChannel.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ def __init__(self, channel: Channel, actual_liquidity: int = None):
self._actual_liquidity = random.randint(0, self.capacity)

def __str__(self):
return super().__str__()+" actual Liquidity: {}".format(self.actual_liquidity)
return super().__str__() + " actual Liquidity: {}".format(self.actual_liquidity)

@property
def actual_liquidity(self):
Expand All @@ -30,15 +30,23 @@ def actual_liquidity(self):
return self._actual_liquidity

@actual_liquidity.setter
def actual_liquidity(self,amt):
self._actual_liquidity = amt
def actual_liquidity(self, amt: int):
"""Sets the liquidity of a channel in the Oracle Network

:param amt: amount to be assigned to channel liquidity
:type amt: int
"""
if 0 <= amt <= self.capacity:
self._actual_liquidity = amt
else:
raise ValueError(f"Liquidity for channel {self.short_channel_id} cannot be set. Amount {amt} is negative or higher than capacity")



def can_forward(self, amt: int):
"""
check if the oracle channel can forward a certain amount
"""
if amt <= self.actual_liquidity:
return True
else:
return False
return False
34 changes: 30 additions & 4 deletions pickhardtpayments/OracleLightningNetwork.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
from typing import List

from .ChannelGraph import ChannelGraph
from .OracleChannel import OracleChannel
import networkx as nx
Expand All @@ -13,7 +15,7 @@ def __init__(self, channel_graph: ChannelGraph):
for src, dest, short_channel_id, channel in channel_graph.network.edges(data="channel", keys=True):
oracle_channel = None

# If Channel in oposite direction already exists with liquidity information match the channel
# If Channel in opposite direction already exists with liquidity information match the channel
if self._network.has_edge(dest, src):
if short_channel_id in self._network[dest][src]:
capacity = channel.capacity
Expand All @@ -35,14 +37,18 @@ def network(self):
return self._network

def send_onion(self, path, amt):
"""

:rtype: object
"""
for channel in path:
oracle_channel = self.get_channel(
channel.src, channel.dest, channel.short_channel_id)
success_of_probe = oracle_channel.can_forward(
channel.in_flight+amt)
channel.in_flight + amt)
# print(channel,amt,success_of_probe)
channel.update_knowledge(amt, success_of_probe)
if success_of_probe == False:
if not success_of_probe:
return False, channel
return True, None

Expand All @@ -55,7 +61,7 @@ def theoretical_maximum_payable_amount(self, source: str, destination: str, base
"""
test_network = nx.DiGraph()
for src, dest, channel in self.network.edges(data="channel"):
#liqudity = 0
# liquidity = 0
# for channel in channels:
if channel.base_fee > base_fee:
continue
Expand All @@ -71,3 +77,23 @@ def theoretical_maximum_payable_amount(self, source: str, destination: str, base

mincut, _ = nx.minimum_cut(test_network, source, destination)
return mincut

def settle_payment(self, path: List[OracleChannel], payment_amount: int):

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is from the other PR #18 ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

"""
receives a List of channels and payment amount and adjusts the balances of the channels along the path.

settle_payment should only be called after all send_onions for a payment terminated successfully!
# TODO testing
"""
for channel in path:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if the path is not connected? Maybe we really should have a Path class.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i changed it so that path from an Attempt is passed to settle_payment. At initialisation of the Attempt with the path, the connectedness is checked.
Can still be refactored to a Path Class later on.

settlement_channel = self.get_channel(channel.src, channel.dest, channel.short_channel_id)
return_settlement_channel = self.get_channel(channel.dest, channel.src, channel.short_channel_id)
if settlement_channel.actual_liquidity > payment_amount:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you do both in one if? if for some reason the data was not consistant than we don't catch it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if understand - do you refer to the fact that an exception might leave an incomplete rebalancing of the channels and it ends up not being atomic?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am talking abou the fact that you reduce the liquidity in one channel and increase in the other one. But maybe the state of the channels was not connected anymore and now we would get liquidity beyond capacity. (negative won't work because of the if. So I was wondering if we need two sperate ifs or at least both conditions in the same if)

# decrease channel balance in sending channel by amount
settlement_channel.actual_liquidity = settlement_channel.actual_liquidity - payment_amount
# increase channel balance in the other direction by amount
return_settlement_channel.actual_liquidity = return_settlement_channel.actual_liquidity + payment_amount
else:
raise Exception("""Channel liquidity on Channel {} is lower than payment amount.
\nPayment cannot settle.""".format(channel.short_channel_id))
return 0
Loading