legendary
Activity: 1176
Merit: 1134
Been coding pretty intensely to create a general framework that allows all the specific usecases that I have thought about to be handled.
For people who are complaining of the delays, then I apologize that I am slow to get things completed. It is not from lack of hours though as I have put in about 5000 hours of work in the last 10 months. Maybe I am just a slow coder, or maybe this is not such an easy thing to do. I searched for things I can cut and paste to solve it, but there didnt seem to be any such thing.
However, things are inching toward the Tuesday. The framework described below is in testing with an InstantDEX/MGW/ramchains/Lchains agents and a set of decentralized load balanced servers. I am not doing the GUI or the Windows port or the installer, so I cannot speak to when all that will be done. I do feel that I am bugfixes away from a completed framework and then it will be onto specific agents for the pending usecases, eg. teleport, telepathy, etc.
The following is the basic, simplified explanation. You also need to know about the SuperNET agents described elsewhere as they are what plugs into all of this. ultimately agents will be able to be written in any of a dozen different languages and it wont have to know anything about networking, crypto, blockchains, etc. but it can take advantage of them.
you send in JSON, it returns JSON
this should be the key to understanding
now the cgi path has messy HTML handling that is needed to allow browsers/curl/GUI to be able to construct the JSON and push it into the blackbox. From HTML there are all sorts of restrictions so making a single localhost HTML to access two different ports, is actually an issue. the SuperNET cgi allows a single HTML to interface to any number of RPC interfaces in addition to the SuperNET API.
the API is pretty directly invoked via ./BitcoinDarkd SuperNET '{...}'
where the '{...}' is the JSON
so you dont have to understand HOW it is doing, but just what. like driving a car
so if something is broken, then what does that mean?
either the blackbox is broken (JSON -> return JSON is wrong) or the JSON being pushed into the blackbox is wrong
if that is the case, it could be the cgi glue layer
or the GUI
let me know if any questions. people often confuse the difficulty of getting it to work with the difficulty of using it
[GUI] <-> (cgi) <-> {API JSON blackbox}
curl bypasses the GUI and injects into the (cgi)
./BitcoinDarkd bypasses the (cgi) and injects into the {API JSON blackbox}
*** advanced stuff follows ***
now this {API JSON blackbox} is where the SuperNET agents are
the (cgi) actually spawns a new nanomsg connection with a special thread to accept the requests from the cgi, this is fully multithreaded
the SuperNET commandline goes through a parser and the BitcoinDarkd path also goes through a bit of processing
but they all end up in the same place called process_user_json
this processes the user's JSON
now things do indeed get a bit complicated....
I am pushing more and more things through the "busdata" path as that allows for modular authentication, encryption, and other privacy things
but it is still possible (not sure if I wont deprecate this) to directly invoke an agent by not having the "busdata" agent specified
let us ignore this as I expect to use the busdata for as much as possible as it makes for the same processing whether going out to the network or staying within the local node
the busdata path behaves a bit differently if you are running a relay node or not. if you are not, then it converts the user json into a binary format with whatever authentication that is specified and issues a load balanced call to the relays. one thing to note is that the client can specify a "broadcast":"allrelays" or "allnodes"
if it is a relay node, it broadcasts to all relays and then processes the request locally
back to the client path... it ends up at a specific relay node that processes it locally, but if the "broadcast" is specified, then it is broadcast to allrelays or allnodes
notice that in the case of "broadcast":"allrelays", this is having the same state as when the originating node was a relay
the relays are receiving the busdata packet and if it is an "allrelays" one they process it locally, if it is a "allnodes" global broadcast, they currently ignore it
the reason for this is that for something like InstantDEX where your node wants to broadcast its placebid globally, it just needs to get to all the other nodes and not the relays themselves (assuming the relay node is not used as a normal node), so it is like a "doughnut", with a hole in the middle. "allrelays" sends to the middle and "allnodes" to all nodes, but the middle is probably ignoring it. havent figure out whether it is worth to adding a flag to the global broadcast to tell the relays to process locally
now, this "process locally", you are asking what that means
it means to decode the binary data, authenticate it and then route it to the correct place
this can be one of several places, plus there are also some control things like registering a service provider
if the {user json} was a request for a specific service provider, then the relay will send the request to a random node that has previously registered and then it waits for the response and then routes the data back to the original node. In the event that the random relay that received the request does not have any registered nodes for that service, it does an allrelays broadcast, in hopes that some other relay has such a service. havent automated the sending back of the response from the failover path back to the original caller yet
keep in mind this is all happening within a second or so
if the request is for a specific agent on that node, then it is much simpler and it sends a message to that specific agent, gets the response and sends it back
so the above is the simplified explanation of the {API JSON blackbox}
James