Update:Finished some small things, but mostly design work
There are some minor changes
- UPNP support for Skycoin networking is done. Can tunnel through firewalls now.
- Networking will probably be switched from TCP/IP to UDP
Exchange:This is ready to go almost, but have to finish
Other:The GUI for interaction with networking and meshnet configuration is extremely frustrating. I am trying to find some way to take a shortcut or make this easier.
- I need a cross platform pseudo terminal and a metacircular evaluator for golang. Very frustrating. There is nothing like ncurses that is cross platform.
- I feel like this will require writing an application in opengl on top of SDL. This is annoying because the code base currently cross-compiles 100% and this will break that
I feel like I am rewriting a bunch of software from the 70s and 80s, such as ncurses and pseudo terminal multiplexing... I think if I just keep coding, that it will work out.
There is a new programming technique called CODA that is extremely powerful. TCP/ip implementation is about 20,000 lines in C. In Alan Kay's CODA type environment, TCP/ip type flow control can be implemented in 150 lines.
He was able to implement a full operating system from scratch, with document editor, email and a webbrowser, in 20,000 lines of code
The mesh network requires something like this and may have to implement a simple scripting language for this.
After two weeks, I figured out how to implement it in very few lines but still has to be implemented and debugged.
This is extremely occult tier mathematics.
- The programs in this language, do not have a representation as a text file
- The syntax looks exactly like golang
- All programs are deterministic
- The programs have a very elegant algebraic representation
- there does not appear to be a difference between local program objects and program objects on a remote machine
- the programs accept only length prefixed bytes as input and emit length prefixed bytes
- it is recursive (turtles all the way down), in that you can instantiate a "world" or program context and within that context, instantiate three "worlds" or program contexts (own computers) and network them (top level passing messages between them) and run them. The top level programming context appears to be recursive.
- the programs are constructed by applying a series of operators on an object starting in a null state
- the programs themselves are program objects
- the meaning of the program or action of a statement upon the program state is not fixed, but defined in terms of another program object
- it is impossible to write software that is not open source in this language
When we use the computer, we do not interact with software objects. We interact mediated through the computer but not on "computer objects", but rather interact with twitter, facebook, youtube mediated by the computer
object, tool, interaction
There is a triangle like this, with
- the object on the left (representation, visual thing, object)
- the interaction/goal or outcome on the right (pragmatics, what you are doing, the operation or action)
- the tool or action at the top (the intermediating thing, the tool that immediate between the object, the reification of the action)
An object starts as a null state, then a finite series of actions are applied to it
- a twitter feed starts empty and then "create tweet" actions are applied in series
- a coin starts with an unspent output set, then transaction actions are applied, that destroy unspent outputs and create new unspent outputs
- a text file is opened and begins empty, then a series of key press or "insert characters", "remove characters" actions are applied
- a wikipedia page starts as a blank text file and then a series of "apply diff" actions are carried out upon the state
The state of the object and the sequence of reifed of the actions to the current states are dual. They are isomorphic and equivalent.
- starting with the null object and applying the actions in series to the state will yield the same result as transmitting the current state itself
This may seem abstract, but the reason it is important is that the nodes act like agents
- they have a goal (such as multiplexing packets across multiple connections)
- they have multiple things or actions they can attempt in order to possibly achieve the goal
- they have to choose a sequence of actions to accomplish the goal
- There are procedural or pragmatic aspects of actions (what an action does) (the result of applying the action to the program state)
- There are declarative aspects of actions (when actions ca be applied, serialization for transmission over network)
This is needed for constructing a program, which can be set with a goal and which can attempt a series of actions to accomplish that goal.
- "retransmit messages from a list of messages, until a receipt confirmation has been received from the destination"
- "find a list of peer who each have data item that hashes to H" (DHT lookup)
Downloading a file is
- "find a list of peers who each have data item that hashes to H"
- "from the list of peers make download requests for chunks of file, until all chunks are received"
- there is a series of attempts to retrieve peers for the hash (DHT or super node)
- for each peer, there is attempt to connect to them (may fail)
-- for each connection attempt, there is an attempt to find a path to the peer (may fail)
- for each connection, there is an attempt to make the chunk download request (may fail)
This type of program is not procedural or declarative, but uses both and is defined in terms of behaviors.
- if file is not downloaded, find peers to who have the file which hashes to X
- if peer is known, but are not connected, then attempt to connect to the peer
- if connected to peer, requests chunks which have not been downloaded
- if file is done downloading, terminate
The language actually looks identical to golang, except that there is a meta-syntactic operator for creating higher order structures like behaviors, attributes, goals and higher order concepts for implementing things like this.
The language has everything stripped out, so I may be able to implement it in a few thousand lines. I am trying to figure out
- should I do this later
- should I do prototype first before writing this
The largest problem is that I dont have a pseudo-terminal that is cross platform and I need a meta-circular interpreter for driving the terminal gui widgets and the best way to do that is an early version of the scripting language.
The scripting language has added security benefit of being memory safe. In theory, you can strip out the whole operating system and just have a program for the networking drivers and another program for disc.
I should stop thinking about it and just code.
Pseudo Terminal:I want to be able to quickly open a box, type text into into, hit enter, run a command. Write a program, select a data object, apply operations to the data object, drag boxes around the screen, connect to foreign server, implement a chat program in a few lines that connects to a server and has a chat room, display graphs etc...
I find it frustrating that it takes more than five minutes to do any of those things. I have data and program state and I just need a quick prototype gui for displaying it and letting the user interact with it. Nothing does that well. It is very frustrating especially cross platform. Just getting a cross platform eval loop, for a Bitcoin exchange on the command line, is hell.
I tried to use golang as the scripting language, but golang does not have a metacircular evaluator yet. I cannot do something like iPython Notebook in golang yet and embed it in the terminal.
For the meshnet gui you end up
- writing an opengl application usind SDL to abstract mouse wheel, clicking, key presses and API for events like setting characters and window resizing
- writing a scripting language in golang
- rewriting the gui front end to use the scripting language and the application backend
- writing a terminal multi-plexer and full environment
- writing a framework for making meteor/Angular.js type terminal applications and layout
- rewriting most of unix as it existed in the 70/80s
However, what this gives you, is a very clean, very self contained environment that can be dropped anywhere and is security sandboxed.
- an environment that can be compiled down and run in the browser
- an environment that can run on bare metal on a Raspberry PI
- an environment that can be run on Windows, OSX, Linux and has the same interface and feel
- less than 20,000 lines of code and no room for backdoors
I think for IoT there is an argument towards standardization and stripping out everything. Getting rid of groups, users, permissions, rings, ... and just reducing linux down to nothing and minimizing the number of concepts or things that exist. Pushing everything into user space. Replacing files with software objects and enforcing everything as being a software object.
Containerization, L4, COLA, cloud computing, IoT, Golang, Urbit, docker, Tox, etc.... These are all part of a cycle where we go back and reevaluate the pile of crap we are building upon.
This may be necessary in the long term, especially given the aggressiveness of the exploits being placed into Java, PHP, OpenSSL, SystemD. The governments have decided that they are going to ruthlessly attack the open source community and drag everything down into the mud they can.
This is especially, frustrating, because if you are using crypto or applications for secure communication, that is detected and then your computer can automatically be flagged for automated root-kitting. The attack cycle is being automated end-to-end, with automated targeting and data ex-filtration.
- start from day 1 with memory safety
- start from day 1 with completely open source drivers and no binary blobs
- start from day 1 with deterministic builds
- start from day 1 with containerization and application sand boxing
- start from day 1 with everything in user space
The whole internet is in the process of being refactored and rewritten. Many of the existing protocols and encryption methods will just be abandoned by necessity, as soon as alternatives become available.
- IP addresses replaced by public key hashes
- static files referenced by hash (content addressable memory)
- alternatives to domain and CA that do not require centralized third parties
- all software in memory safe language, that is not backdoored
- software defined networking, making interception and censorship impossible
- default encryption end-to-end for all applications
- compartmentalization of application data and sandboxing (app store type installation, uninstallation, data synchronization between devices)
- improved process management (a single well defined hierarchy of processes in memory)
- abstraction of file system to eliminate differences between remote and local files
- self-describing data APIs (reflection upon APIs exposed by a processes and hardware) with single standard for data exchange between processes
- elimination of difference of difference between local and remote software objects and resources
- containerization, no need for super-users, ability to do everything in user space, recursive encapsulation of workspaces, sandboxes and processes ("turtles all the way down")
- one language for both client side and server side (arbitrarily, or something that looks more like golang and is not javascript).
- new method of GUI layout, inspired by DOM/CSS (simplification) but which is not DOM/CSS, that is used for both the operating system UI, applications and web-applications (no distinction between desktop applications and remote applications).
- elimination of difference between desktop applications, mobile applications and web-applications
- universal platform between mobile, tablet and desktop (and there is something above desktop that does not exist yet, corresponding to large scale ubiquitous, ambient and volumetric compute surfaces)
All of the recent software projects are moving in this direction, incrementally and piecemeal. So there is an emerging platform, that aspires to universality and simplifies or eliminates everything complex or varied about the existing computer. There are simplifications required and then new programming abstractions required, to deal with systems where a single person has 300 devices, including light bulbs, speakers, microphones, thermostats, tablets, coffee makers, drones, robots, servers, cell phones, laptops, storage drives, networking nodes, etc...
You have systems of systems, that need to be able to interact cooperatively, to achieve goals or maintain states or take autonomous action to optimize criteria. The Skycoin meshnet is one of the first system of this type (aside from mega-projects like the global information grid, internet and military, manufacturing prototype systems).
I am still trying to figure out how, I can avoid needing to write a pseudo terminal or metacircular evaluator and a new scripting language for what I need. Or if I can get away with a bare minimum and only minimum amount of work to implement it.
- i need a pseudo terminal that is cross platform and can handle mouse scroll
- I need a meta-circular evaluator
- I need ability to embed a compute context in a another computing context (the context is a first class object)
- need to be able to construct the program as a series of operations applied on a program object
- I need abstraction of the inputs the program responds to, restricted to length prefixed messages (simple, algebraic universal CSP construct)
- I need golang like procedural functions, structs, module structure (works very well, very simple, easy to implement)
- I need "behavior" type programming like in CODA (easily implemented with meta-syntactic construct on top of base language) (helps for networking operations and drastically reduces lines of code required)
- library for Meteor type object synchronization (helps for exchange and writing chat programs)
- Ability to call remote API/RPC on remote objects in syntax similar to local
- small library for simple pseudo-terminal GUIs
I am trying to find the minimum set of what needs to be implemented for very basic application, such as the meshnet gui. There is a basic application, that can be grown or extended later.