Pages:
Author

Topic: Important Lighting Network reading- for everyone! (Read 1261 times)

BEX
newbie
Activity: 25
Merit: 3
The only problem lies in the fact that the channels are only defined between two persons. In order to reach a person with whom you do not share a channel, you will need to use intermediate nodes or “hubs” (Bob in our example). However, these nodes could be considered by the regulatory agencies as “money transmitters” and as such be subject to requirements such as the minimum capitalization, KYC checks (…) which cannot be filled by anyone.
More here...https://www.blockchains-expert.com/en/lightning-network/
legendary
Activity: 2352
Merit: 6089
bitcoindata.science
I think it’s important that everyone who supports bitcoin be on the same page about how the Lightning Network works and why bitcoin’s future is still as bright as ever. This short read was written up by a buddy in a slack channel and I thought it was important to pass along. It’s easy reading for anyone! 10 minutes of your time and if you’re not fully educated on the Lighting Network basics..you will be!

https://medium.com/@melik_87377/lightning-network-enables-unicast-transactions-in-bitcoin-lightning-is-bitcoins-tcp-ip-stack-8ec1d42c14f5

I was researching about lightning network and found this topic. Very nice article.

I made a summary and translated it, and posted in my local board. Thanks for sharing.

https://bitcointalksearch.org/topic/m.32927586
legendary
Activity: 2282
Merit: 3014
NP guys!  But the smart mo fo who typed this bad boy up deserves all credit. I've been looking for articles like this that teach the non-coder/programmer (as badly as I'd die to be) and they aren't easy to find.

sr. member
Activity: 602
Merit: 265
Very good read, and the analogies helped, as most of us aren't deep inclined. Maybe if a short comic-like/graphic video was created, we can get a larger pool of people to understand this upcoming feature of bitcoin, and re-affirm their belief that the future is good for bitcoin. I'm really looking forward to the Launch of Bitcoin's Lightening Network, and the Raiden Network of Ethereum.
newbie
Activity: 154
Merit: 0
Man, I want to say thanks for such info, because I couldn't find something usefull about that. It's really interesting and I think it has the future.
sr. member
Activity: 257
Merit: 343
If I understood the docs correctly, the big difference between routing in IP networks and in Lightning is the used model: in Lightning I see source routing used, so the sender first creates the route, packs it in layers (hence onion), and then sends the package. In IP networks there is ROUTERS all over the network, that do this job. So in the IP world you can send a package with a destination address (encapsulated in the header field) to the next router, and he has these lookup tables, and forwards accordingly.
In a lightning node would have to ask "the network" first, then create the route (based on fees), and then send the package over this pre-defined route. So the node has it's own "lookup table". It might be called differently though. I still have an issue to understand the "ask the network", so leave it open for discussion.

There was some text from the devs on the mailing list:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2015-December/000384.html

Quote
Surely you mean the 2500 Series? (Ooops showing my age again.)
yes, and yes  Grin, me too!
hero member
Activity: 2576
Merit: 883
Freebitco.in Support https://bit.ly/2I9BVS2
More to the point (and my original question), are the Lightning devs doing so—and if so, how?

I'm afraid I can't answer this bit as I don't know.

So—how would you take all this networking knowledge, and apply it to routing and network topology in Lightning?  
......

I see plenty of speculation about what LN will look like, topology-wise.  Yet much of that depends not only on what potential links are available, but also on how nodes use them.  I don’t see how a simple look-up table would suffice.  If you are connected to A, B, and C, and you want to reach Z, that’s not an easy problem.  It’s not easy in the first instance; and however it’s answered now, I expect that could be fertile ground for optimization in the future.

The lesson I would take from IP routing protocols would the introduction of 'areas' that allowed changes that occur in certain places need only be propagated within that area and have no effect on the network topology as a whole. The IP routing table in Internet routers is a simple lookup table, but these protocols keep the updating of that table to a minimum.
copper member
Activity: 630
Merit: 2614
If you don’t do PGP, you don’t do crypto!
Thanks for the networking discussion, folks.  I want to underscore this twice:

I see so many debates about the possible limitations of Lightning Network that miss this. It's not an end product set in stone, it will constantly evolve to address any issues that arise. That's not so easy to do in blockchain where much was set in stone in the genesis block.

So—how would you take all this networking knowledge, and apply it to routing and network topology in Lightning?  More to the point (and my original question), are the Lightning devs doing so—and if so, how?

(Aside, or perhaps not:  Shifting analogies around and down to the link layer, per OP’s article, we no longer have only a broadcast network as with old Ethernet hubs.  There is a reason I thought of Spanning Tree first.)

I see plenty of speculation about what LN will look like, topology-wise.  Yet much of that depends not only on what potential links are available, but also on how nodes use them.  I don’t see how a simple look-up table would suffice.  If you are connected to A, B, and C, and you want to reach Z, that’s not an easy problem.  It’s not easy in the first instance; and however it’s answered now, I expect that could be fertile ground for optimization in the future.
legendary
Activity: 3542
Merit: 1965
Leading Crypto Sports Betting & Casino Platform
While you guys dissect the technical aspect of this document, I would just like to thank the author for a very enlightened piece of art. The main point of this article in my humble opinion, was to highlight the fact that we are still in the earliest phase of this experiment. We are building the "Internet of Money", piece by piece and if second layer applications are necessary to do that, then so be it.

The point is, Block size increases can only take us to a specific level, before it becomes destructive to the network. We have to rely on second layer applications to take us to the next level.
hero member
Activity: 2576
Merit: 883
Freebitco.in Support https://bit.ly/2I9BVS2
Remember good old CISCO 27xx or 29xx models with configs for RIPv1 and v2, and then open shortest path first (OSPF)? These were the internal routing protocols. Then there was the bigger networks in the late 80s and they needed a professional base layer (border gateway protocol, BGP and IGRP/EGRP, or EGP). And of course a DNS. btw: why is DNSsec not used today?

Surely you mean the 2500 Series? (Ooops showing my age again.) This is a good demonstration of how easy it is solve scaling issues at layer 3. When the limitations of RIP became apparent it was easy to simply add additional routing protocols to solve the problems. For example, EIGRP is a Cisco proprietary protocol that can be used in areas of the network using exclusively Cisco equipment and then BGP handles connecting to other areas that may not be.

I see so many debates about the possible limitations of Lightning Network that miss this. It's not an end product set in stone, it will constantly evolve to address any issues that arise. That's not so easy to do in blockchain where much was set in stone in the genesis block.


Spanning tree was the protection layer in bridges, to prevent loops in larger networks, with the "routing" of MAC addresses (bridge = layer 2 device). Yup, no routing at layer 4 here ...

Pedantic Note: STP allowed layer 2 topologies to be designed with redundancy built in by temporarily blocking loops and then opening them when a failure elsewhere eliminated the loop.
sr. member
Activity: 257
Merit: 343
...
Network routing is one of those arcane specialties filled with scary-smart people who know all the minutiae of complex systems which most developers are barely aware exist.  (How does the Internet really work?  How do all those little packets know where to go?  Magic!)  I am not in that specialty, and I’m not familiar with its research literature
I'm also not a specialist, but understand enough on the basics. So no, not magic, and I bet you know as well :-)
IP protocol is well defined, with routing information (or destination) in the header of each packet. In the very early days a router would ask his neighbor, do you have a route for network x? (neighbor discovery protocol). Someone to whom I was connected would answer accordingly (or not, then "PATH NOT FOUND"). And the question of routing optimization started quite early. There was this ATM network model at the same time (goal to unify networks with telephony and IP in 53 byte packets!)... Remember good old CISCO 27xx or 29xx models with configs for RIPv1 and v2, and then open shortest path first (OSPF)? These were the internal routing protocols. Then there was the bigger networks in the late 80s and they needed a professional base layer (border gateway protocol, BGP and IGRP/EGRP, or EGP). And of course a DNS. btw: why is DNSsec not used today?

Quote
... what first occured to me was that Spanning Tree Protocol might somehow be applicable. Of course, that’s not an Internet routing protocol; but it is the standard staple for organizing the network topology on LANs.
Spanning tree was the protection layer in bridges, to prevent loops in larger networks, with the "routing" of MAC addresses (bridge = layer 2 device). Yup, no routing at layer 4 here ...

Quote
The general question is:  Given a global set of nodes which form and remove links between each other unpredictably, how does each node organize its own view of potential routes and choose optimal paths?
Lookup tables?
I took a look into the routing in Lightning - you come to deal with SPINX, HMACs, and of course onion routings. I think the difficulty layer of lightning routing stems from the fact, that the node only knows the predecessor and the successor of a route. Nothing else. Not the origin, not the final destination, not the amount (but then, how to know, that the channel supports the requested value transfer?), and for sure in a way bullet proof, that the node cannot benefit from forwarding the package to a different target. There I find this table in the data structure of 20 entries @65 bytes (hops_data), which makes me think on what it is used for? Is there a max of 20 hops?
Also the flare white paper (http://bitfury.com/content/5-white-papers-research/whitepaper_flare_an_approach_to_routing_in_lightning_network_7_7_2016.pdf) says:
Quote
Hence, it is in the interest of the sender to optimize fees and make the final decision on which route to choose to the recipient (otherwise the sender gets potentially unpredictable expenses, as other nodes are not incentivized to optimize for the cheapest path).
and
Quote
Source routing leads to the requirement that the sender node should be able to collect information on fees and available channel capacity to pick the best route (as well as knowing which nodes are currently online). Thus, an overlay mechanism should exist to enable requesting information about a channel from any of its owners.
Interesting, that would mean no dynamic routing?

I don't understand (yet) how routing achieves a dynamic management, when it is pre-defined from the beginning. And this would mean, it does not require IP routing specialists? There's work to be done!
newbie
Activity: 5
Merit: 0
So my question is it possible to make your own coin that can be traded that could be used on the lightning network and if so how would one go about doing this?
copper member
Activity: 630
Merit: 2614
If you don’t do PGP, you don’t do crypto!
...
I will make explicit a specific question I earlier implied:  Are the Lightning engineers availing themselves of the fine research literature on network routing protocols and routing algorithms?  If that could be answered off-hand by anybody who’s been following Lightning development much more closely than I have, I’d be much obliged.

Thanks.

When following older threads, there is lot of discussion on onion routing. But here in the forum I haven‘t seen engineers discussing the research literature.

There has been a short discussion, but it did not get the desired attention it deserves... https://bitcointalk.org/index.php?topic=2573055.msg26369895;topicseen#msg26369895

The routing itself is described here:
https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md

Thanks, pebwindkraft.  Of course, onion routing for privacy is a subject near and near and dear to my heart.  (The name of the Tor network started as an acronym for “The Onion Router”; it was later declard simply “Tor”.)  As a general concept, it is a source-routing method which conceals prior hops from each next-hop, including the destination; and I am glad to see the active development of an onion-routing implementation for Lightning Network.  But that’s not the type of routing problem I had in mind, when I thought about the IP network analogy.

Network routing is one of those arcane specialties filled with scary-smart people who know all the minutiae of complex systems which most developers are barely aware exist.  (How does the Internet really work?  How do all those little packets know where to go?  Magic!)  I am not in that specialty, and I’m not familiar with its research literature—but I know it has such a thing, and a quick search found some handy starting points:

https://www.caida.org/research/routing/
(CAIDA is an important site, by the way...)

https://www.academia.edu/Documents/in/Routing_protocols

When OP’s link launched a train of thought about Lightning and networking, what first occured to me was that Spanning Tree Protocol might somehow be applicable.  Of course, that’s not an Internet routing protocol; but it is the standard staple for organizing the network topology on LANs.  Then, I thought of Internet routing.  Also, self-organizing mesh networks.

The general question is:  Given a global set of nodes which form and remove links between each other unpredictably, how does each node organize its own view of potential routes and choose optimal paths?

I think there are a few network characteristics of Lightning which are sui generis:  The problem of optimal (or even possible) routes involves monetary calculations, both as to fees charged, and as to availability of funds on each channel which provides a potential hop.  Indeed, network “cost” is monetary, rather than usual measures of network bandwidth and latency.  Otherwise, however, it mostly sounds like just the sort of problem for which a network routing specialist would have applicable existing expertise.  Whence my question.  Lightning development must already encompass some answers to these questions; and I expect that routing optimization may be a significant area of evolution and improvement as Lightning grows and matures.

And I read, Acinq/Eclair is using the flare routing engine. Haven‘t found the spec yet. If someone has?

A quick search found only some PDFs pertaining to a financial order routing system, on a site which won’t load for me (probably blocks Tor).  I have no idea if those be relevant.  But come to think of it, there’s another pre-existing field which is likely to have applicable existing knowledge.


I have a lot of reading to catch up on.  Thanks for the discussions and ideas you are all sharing. I won't be able to understand half of it, but still great.

Hey, thanks for posting the link in OP!  That started an interesting discussion—one in which I myself have mostly questions, not answers.
legendary
Activity: 2282
Merit: 3014
I have a lot of reading to catch up on.  Thanks for the discussions and ideas you are all sharing. I won't be able to understand half of it, but still great.

The good news - LN is in place and starting to do it's thing!! I paid .33 cents for a roughly $250 transaction.  Much better!

The bad new- we're all getting spanked right now.

However I see a great buying opp and the LN makes me that much more confident about it.  

sr. member
Activity: 257
Merit: 343
...
I will make explicit a specific question I earlier implied:  Are the Lightning engineers availing themselves of the fine research literature on network routing protocols and routing algorithms?  If that could be answered off-hand by anybody who’s been following Lightning development much more closely than I have, I’d be much obliged.

Thanks.

When following older threads, there is lot of discussion on onion routing. But here in the forum I haven‘t seen engineers discussing the research literature.

There has been a short discussion, but it did not get the desired attention it deserves... https://bitcointalk.org/index.php?topic=2573055.msg26369895;topicseen#msg26369895

The routing itself is described here:
https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md

And I read, Acinq/Eclair is using the flare routing engine. Haven‘t found the spec yet. If someone has?
full member
Activity: 1022
Merit: 127
Make a difference, make it better.
Even to someone with limited knowledge of network engineering and specs, it seems very simple to understand and informative. Based on what was explained it definitely is in a long way to advance o better system of Bitcoin and cryptocurrencies. The way it is designed now is falling the real purpose of the crypto ideal that was having a decentralized, secure way of fast and cheap transactions on the peer to peer basis. Definitely, there is ample space for evolution and so we should head for a better world community based on the blockchain.
copper member
Activity: 630
Merit: 2614
If you don’t do PGP, you don’t do crypto!
My core idea is that the Singularity is unavoidable, but I never figured out how machines would do it.  I think crypto is the answer.

I already told people: the long version is free, for the short version, you'll have to pay.  I didn't have time to be short.

It is not my "inability".  It is my lack of desire to waste time on being succinct.  I can be, but it takes time and effort I don't want to spend on a forum.  I already spend too much time on it, I cannot spend 2 or 3 times more.

Have you ever read, say, an exposition of general relativity ?  How many pages do you have to acquire, follow explanations, fill in gaps the author left, think through what the author is saying, not being quite sure that you're with him, before you actually start understanding the argument ?  Compared to that level of difficulty, "working through my walls of text" is leisure in a blink of an eye.  People not capable of doing this, can probably not reason on a sophisticated enough level to even start being useful.

[...]

If you tell me that the few people capable of seeing that, are elsewhere, then one must conclude that the amount of brain power here is too low to be of any sensible use in the development of any form of advanced argument.  That's also a possibility of course.

Ynqvrf naq tragyrzra, jr unir n trahvar xbbx.

dinofelis, I admit that my brain is no match for your “advanced argument”.  Indeed, I am certain that none of the regulars on this forum has a brain capable of operating on the level of yours.  I grant you all thanks you are owed for your having been so magnanimous as to grace us with your presence.  Please, do not waste further time here.  Go forth to seek the company of like-brained people.  Just remember to give us a wave (a particular wave) when you are accepting the award to you of the Fields Medal, or whatever; and please tell the Singularity to go easy on poor, be(k)nighted old nullius, nobody’s man.

Now, this started as a most excellent thread on the topic of the Lightning Network.  I know, I admit, it is a characteristic of our brains that we need thoughts concisely organized and focused.  Be that as it may.  Does anybody have anything further to say about the Lightning Network and the metaphor of unicast networking?

I will make explicit a specific question I earlier implied:  Are the Lightning engineers availing themselves of the fine research literature on network routing protocols and routing algorithms?  If that could be answered off-hand by anybody who’s been following Lightning development much more closely than I have, I’d be much obliged.

Thanks.
hero member
Activity: 2576
Merit: 883
Freebitco.in Support https://bit.ly/2I9BVS2
Have you ever read, say, an exposition of general relativity ?.......

You seem to have misunderstood the function of a forum. It is a place to meet and discuss issues not to publish a thesis.

If you tell me that the few people capable of seeing that, are elsewhere, then one must conclude that the amount of brain power here is too low to be of any sensible use in the development of any form of advanced argument.  That's also a possibility of course.

You also don't seem to understand the shortcomings of your writings. It's not that the brain power here is too low to understand them it is that they ramble on and on and rarely actually get to a point. I'm trying to help you get more out of the forum by being clear and concise. That way you will actually interact far more.

I want to find out if my thinking is correct, if there are intelligent counter arguments to my thinking.

If this is your aim I am sure that would be a far better way to proceed.

It is not my "inability".  It is my lack of desire to waste time on being succinct.  I can be, but it takes time and effort I don't want to spend on a forum.

I understand it can be difficult for some people, writing clearly, concisely and debating are skills that have to be learned. With practice, it will come much more easily to you. In the long run, you will benefit greatly from making the effort to acquire this skill and you'll find it eventually will save you much more time as once you get the hang of it writing concisely is far less time consuming than rambling on the way you do now.
hero member
Activity: 770
Merit: 629
On the contrary to the rest of your statement, I think many of the most knowledgeable people here have got far more important things to do with there time than trying to decipher your "text walls" to see if there is anything vaguely resembling coherent thinking hidden in there somewhere.

Have you ever read, say, an exposition of general relativity ?  How many pages do you have to acquire, follow explanations, fill in gaps the author left, think through what the author is saying, not being quite sure that you're with him, before you actually start understanding the argument ?  Compared to that level of difficulty, "working through my walls of text" is leisure in a blink of an eye.  People not capable of doing this, can probably not reason on a sophisticated enough level to even start being useful.  Usually, in texts like that, the problem is rather that the text is too concise, and that one has to fill in too many gaps.  I err probably on the other side, I'm too verbose, too explicit, too much in simple details that could be filled in, in what I say.

I'll ask you: how many lines of explanation would you need to understand, from scratch, say, Pollard's rho attack on a Diffie-Hellman key exchange ?  Suppose that this was an unknown thing, and that someone posted this here for the first time, somewhat hesitant maybe in the fluidity of his wordings.  Would you also complain that there are "walls of text" if someone would try to give an argument explaining how it could be done in a page or two ?  Do you think that your comments would be of any use ?

If you tell me that the few people capable of seeing that, are elsewhere, then one must conclude that the amount of brain power here is too low to be of any sensible use in the development of any form of advanced argument.  That's also a possibility of course.
hero member
Activity: 2576
Merit: 883
Freebitco.in Support https://bit.ly/2I9BVS2
I certainly don't want to blog, because I have nothing to "tell the world"

You couldn't come across as someone trying to do that more if you were trying.

On the contrary to the rest of your statement, I think many of the most knowledgeable people here have got far more important things to do with there time than trying to decipher your "text walls" to see if there is anything vaguely resembling coherent thinking hidden in there somewhere.
Pages:
Jump to: