Topic: Important Lighting Network reading- for everyone! (Read 1220 times)

The only problem lies in the fact that the channels are only defined between two persons. In order to reach a person with whom you do not share a channel, you will need to use intermediate nodes or “hubs” (Bob in our example). However, these nodes could be considered by the regulatory agencies as “money transmitters” and as such be subject to requirements such as the minimum capitalization, KYC checks (…) which cannot be filled by anyone.
More here...https://www.blockchains-expert.com/en/lightning-network/

I was researching about lightning network and found this topic. Very nice article.

I made a summary and translated it, and posted in my local board. Thanks for sharing.

https://bitcointalksearch.org/topic/m.32927586

NP guys!  But the smart mo fo who typed this bad boy up deserves all credit. I've been looking for articles like this that teach the non-coder/programmer (as badly as I'd die to be) and they aren't easy to find.

Very good read, and the analogies helped, as most of us aren't deep inclined. Maybe if a short comic-like/graphic video was created, we can get a larger pool of people to understand this upcoming feature of bitcoin, and re-affirm their belief that the future is good for bitcoin. I'm really looking forward to the Launch of Bitcoin's Lightening Network, and the Raiden Network of Ethereum.

Man, I want to say thanks for such info, because I couldn't find something usefull about that. It's really interesting and I think it has the future.

If I understood the docs correctly, the big difference between routing in IP networks and in Lightning is the used model: in Lightning I see source routing used, so the sender first creates the route, packs it in layers (hence onion), and then sends the package. In IP networks there is ROUTERS all over the network, that do this job. So in the IP world you can send a package with a destination address (encapsulated in the header field) to the next router, and he has these lookup tables, and forwards accordingly.
In a lightning node would have to ask "the network" first, then create the route (based on fees), and then send the package over this pre-defined route. So the node has it's own "lookup table". It might be called differently though. I still have an issue to understand the "ask the network", so leave it open for discussion.

There was some text from the devs on the mailing list:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2015-December/000384.html

yes, and yes  , me too!

I'm afraid I can't answer this bit as I don't know.


The lesson I would take from IP routing protocols would the introduction of 'areas' that allowed changes that occur in certain places need only be propagated within that area and have no effect on the network topology as a whole. The IP routing table in Internet routers is a simple lookup table, but these protocols keep the updating of that table to a minimum.

Thanks for the networking discussion, folks.  I want to underscore this twice:


So—how would you take all this networking knowledge, and apply it to routing and network topology in Lightning?  More to the point (and my original question), are the Lightning devs doing so—and if so, how?

(Aside, or perhaps not:  Shifting analogies around and down to the link layer, per OP’s article, we no longer have only a broadcast network as with old Ethernet hubs.  There is a reason I thought of Spanning Tree first.)

I see plenty of speculation about what LN will look like, topology-wise.  Yet much of that depends not only on what potential links are available, but also on how nodes use them.  I don’t see how a simple look-up table would suffice.  If you are connected to A, B, and C, and you want to reach Z, that’s not an easy problem.  It’s not easy in the first instance; and however it’s answered now, I expect that could be fertile ground for optimization in the future.

While you guys dissect the technical aspect of this document, I would just like to thank the author for a very enlightened piece of art. The main point of this article in my humble opinion, was to highlight the fact that we are still in the earliest phase of this experiment. We are building the "Internet of Money", piece by piece and if second layer applications are necessary to do that, then so be it.

The point is, Block size increases can only take us to a specific level, before it becomes destructive to the network. We have to rely on second layer applications to take us to the next level.

Surely you mean the 2500 Series? (Ooops showing my age again.) This is a good demonstration of how easy it is solve scaling issues at layer 3. When the limitations of RIP became apparent it was easy to simply add additional routing protocols to solve the problems. For example, EIGRP is a Cisco proprietary protocol that can be used in areas of the network using exclusively Cisco equipment and then BGP handles connecting to other areas that may not be.

I see so many debates about the possible limitations of Lightning Network that miss this. It's not an end product set in stone, it will constantly evolve to address any issues that arise. That's not so easy to do in blockchain where much was set in stone in the genesis block.



Pedantic Note: STP allowed layer 2 topologies to be designed with redundancy built in by temporarily blocking loops and then opening them when a failure elsewhere eliminated the loop.

I'm also not a specialist, but understand enough on the basics. So no, not magic, and I bet you know as well :-)
IP protocol is well defined, with routing information (or destination) in the header of each packet. In the very early days a router would ask his neighbor, do you have a route for network x? (neighbor discovery protocol). Someone to whom I was connected would answer accordingly (or not, then "PATH NOT FOUND"). And the question of routing optimization started quite early. There was this ATM network model at the same time (goal to unify networks with telephony and IP in 53 byte packets!)... Remember good old CISCO 27xx or 29xx models with configs for RIPv1 and v2, and then open shortest path first (OSPF)? These were the internal routing protocols. Then there was the bigger networks in the late 80s and they needed a professional base layer (border gateway protocol, BGP and IGRP/EGRP, or EGP). And of course a DNS. btw: why is DNSsec not used today?

Spanning tree was the protection layer in bridges, to prevent loops in larger networks, with the "routing" of MAC addresses (bridge = layer 2 device). Yup, no routing at layer 4 here ...

Lookup tables?
I took a look into the routing in Lightning - you come to deal with SPINX, HMACs, and of course onion routings. I think the difficulty layer of lightning routing stems from the fact, that the node only knows the predecessor and the successor of a route. Nothing else. Not the origin, not the final destination, not the amount (but then, how to know, that the channel supports the requested value transfer?), and for sure in a way bullet proof, that the node cannot benefit from forwarding the package to a different target. There I find this table in the data structure of 20 entries @65 bytes (hops_data), which makes me think on what it is used for? Is there a max of 20 hops?
Also the flare white paper (http://bitfury.com/content/5-white-papers-research/whitepaper_flare_an_approach_to_routing_in_lightning_network_7_7_2016.pdf) says:
and
Interesting, that would mean no dynamic routing?

I don't understand (yet) how routing achieves a dynamic management, when it is pre-defined from the beginning. And this would mean, it does not require IP routing specialists? There's work to be done!

So my question is it possible to make your own coin that can be traded that could be used on the lightning network and if so how would one go about doing this?

Thanks, pebwindkraft.  Of course, onion routing for privacy is a subject near and near and dear to my heart.  (The name of the Tor network started as an acronym for “The Onion Router”; it was later declard simply “Tor”.)  As a general concept, it is a source-routing method which conceals prior hops from each next-hop, including the destination; and I am glad to see the active development of an onion-routing implementation for Lightning Network.  But that’s not the type of routing problem I had in mind, when I thought about the IP network analogy.

Network routing is one of those arcane specialties filled with scary-smart people who know all the minutiae of complex systems which most developers are barely aware exist.  (How does the Internet really work?  How do all those little packets know where to go?  Magic!)  I am not in that specialty, and I’m not familiar with its research literature—but I know it has such a thing, and a quick search found some handy starting points:

https://www.caida.org/research/routing/
(CAIDA is an important site, by the way...)

https://www.academia.edu/Documents/in/Routing_protocols

When OP’s link launched a train of thought about Lightning and networking, what first occured to me was that Spanning Tree Protocol might somehow be applicable.  Of course, that’s not an Internet routing protocol; but it is the standard staple for organizing the network topology on LANs.  Then, I thought of Internet routing.  Also, self-organizing mesh networks.

The general question is:  Given a global set of nodes which form and remove links between each other unpredictably, how does each node organize its own view of potential routes and choose optimal paths?

I think there are a few network characteristics of Lightning which are sui generis:  The problem of optimal (or even possible) routes involves monetary calculations, both as to fees charged, and as to availability of funds on each channel which provides a potential hop.  Indeed, network “cost” is monetary, rather than usual measures of network bandwidth and latency.  Otherwise, however, it mostly sounds like just the sort of problem for which a network routing specialist would have applicable existing expertise.  Whence my question.  Lightning development must already encompass some answers to these questions; and I expect that routing optimization may be a significant area of evolution and improvement as Lightning grows and matures.


A quick search found only some PDFs pertaining to a financial order routing system, on a site which won’t load for me (probably blocks Tor).  I have no idea if those be relevant.  But come to think of it, there’s another pre-existing field which is likely to have applicable existing knowledge.

---

Hey, thanks for posting the link in OP!  That started an interesting discussion—one in which I myself have mostly questions, not answers.

I have a lot of reading to catch up on.  Thanks for the discussions and ideas you are all sharing. I won't be able to understand half of it, but still great.

The good news - LN is in place and starting to do it's thing!! I paid .33 cents for a roughly $250 transaction.  Much better!

The bad new- we're all getting spanked right now.

However I see a great buying opp and the LN makes me that much more confident about it.  

When following older threads, there is lot of discussion on onion routing. But here in the forum I haven‘t seen engineers discussing the research literature.

There has been a short discussion, but it did not get the desired attention it deserves... https://bitcointalk.org/index.php?topic=2573055.msg26369895;topicseen#msg26369895

The routing itself is described here:
https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md

And I read, Acinq/Eclair is using the flare routing engine. Haven‘t found the spec yet. If someone has?

Even to someone with limited knowledge of network engineering and specs, it seems very simple to understand and informative. Based on what was explained it definitely is in a long way to advance o better system of Bitcoin and cryptocurrencies. The way it is designed now is falling the real purpose of the crypto ideal that was having a decentralized, secure way of fast and cheap transactions on the peer to peer basis. Definitely, there is ample space for evolution and so we should head for a better world community based on the blockchain.

Ynqvrf naq tragyrzra, jr unir n trahvar xbbx.

dinofelis, I admit that my brain is no match for your “advanced argument”.  Indeed, I am certain that none of the regulars on this forum has a brain capable of operating on the level of yours.  I grant you all thanks you are owed for your having been so magnanimous as to grace us with your presence.  Please, do not waste further time here.  Go forth to seek the company of like-brained people.  Just remember to give us a wave (a particular wave) when you are accepting the award to you of the Fields Medal, or whatever; and please tell the Singularity to go easy on poor, be(k)nighted old nullius, nobody’s man.

Now, this started as a most excellent thread on the topic of the Lightning Network.  I know, I admit, it is a characteristic of our brains that we need thoughts concisely organized and focused.  Be that as it may.  Does anybody have anything further to say about the Lightning Network and the metaphor of unicast networking?

I will make explicit a specific question I earlier implied:  Are the Lightning engineers availing themselves of the fine research literature on network routing protocols and routing algorithms?  If that could be answered off-hand by anybody who’s been following Lightning development much more closely than I have, I’d be much obliged.

Thanks.

You seem to have misunderstood the function of a forum. It is a place to meet and discuss issues not to publish a thesis.


You also don't seem to understand the shortcomings of your writings. It's not that the brain power here is too low to understand them it is that they ramble on and on and rarely actually get to a point. I'm trying to help you get more out of the forum by being clear and concise. That way you will actually interact far more.


If this is your aim I am sure that would be a far better way to proceed.


I understand it can be difficult for some people, writing clearly, concisely and debating are skills that have to be learned. With practice, it will come much more easily to you. In the long run, you will benefit greatly from making the effort to acquire this skill and you'll find it eventually will save you much more time as once you get the hang of it writing concisely is far less time consuming than rambling on the way you do now.

Have you ever read, say, an exposition of general relativity ?  How many pages do you have to acquire, follow explanations, fill in gaps the author left, think through what the author is saying, not being quite sure that you're with him, before you actually start understanding the argument ?  Compared to that level of difficulty, "working through my walls of text" is leisure in a blink of an eye.  People not capable of doing this, can probably not reason on a sophisticated enough level to even start being useful.  Usually, in texts like that, the problem is rather that the text is too concise, and that one has to fill in too many gaps.  I err probably on the other side, I'm too verbose, too explicit, too much in simple details that could be filled in, in what I say.

I'll ask you: how many lines of explanation would you need to understand, from scratch, say, Pollard's rho attack on a Diffie-Hellman key exchange ?  Suppose that this was an unknown thing, and that someone posted this here for the first time, somewhat hesitant maybe in the fluidity of his wordings.  Would you also complain that there are "walls of text" if someone would try to give an argument explaining how it could be done in a page or two ?  Do you think that your comments would be of any use ?

If you tell me that the few people capable of seeing that, are elsewhere, then one must conclude that the amount of brain power here is too low to be of any sensible use in the development of any form of advanced argument.  That's also a possibility of course.

You couldn't come across as someone trying to do that more if you were trying.

On the contrary to the rest of your statement, I think many of the most knowledgeable people here have got far more important things to do with there time than trying to decipher your "text walls" to see if there is anything vaguely resembling coherent thinking hidden in there somewhere.