I will try to git a shot on how the problem i ran through and what solution i picked in the past years
I have read your whole convo on the git, but your post are often scattered on different places and thread, so i don't know where it's better to answer it
I hope it doesn't bother you, if you prefer to keep your thread clean on your idea and project, no problem !
I will probably end up writing more detailed and clear paper on this, but i don't have such document for the moment.
The first thing that got me into thinking around this, is the feature in AS3 and also javascript to parse arbitrary json / xml data to runtime class. Like json key/value pairs are called an object , and can be loaded as an object in javascript or AS3, same goes with xml.
My thinking got that the main problem of C and C++ stem from that all the type need to be known at compilation time, and are hard coded by the compiler in the executable, and all the code to access those data is hardcoded to fit with how the compiler want to represent this data internally, and there is no way to know for sure how it is going to end up, and that on many level it's a bad idea, and as it has been said in the discussion on the git, and also from other discussion i seen on other boards, it's generally not a so good idea now day to bet on compiler to be too complex. With the cpu who already have instruction pipelines , can do lot of things with reordering, optimizing cache, pipeline things, it's very hard to know exactly what is on to happen at runtime on silicon level and it's generally better to leave optimization as much as possible in runtime where all the machine state are determined in the course of execution, rather than try to pre order, or pre optimize things through language and compiler who can never take in account execution temporality, and runtime states.
I saw the discussion went on rust too, and there was a very good discussion on this topic on devos too, with a guy who was firm supporter of rust to develop os, but another person explained that in fact the whole concept of rust unsafe code make it actually virtually useless, specially if you want to see the broad picture at system level, it's impossible to have an operating system made only out of safe code, and from the moment you have 'unsafe code' it mean the whole rest of the program is vulnerable to issues in those unsafe zone, which make the real usefulness of rust rather low all together. Maybe for some purpose it can still make memory handling more easier in certain case, but if the application has to make use of any unsafe code, and ultimately there will always be unsafe code executed somewhere in the whole software stack to the cpu, so it's more fake impression of security, or maybe a better way to organize data and code to avoid certain mistake, but certainly not silver bullet.
So the way i went with my things, is to have a whole system of software typed variable ( actually json - like tree of variable), that are totally handled via software instead of relying on compiler built in types and operators, like this the big advantage is that there can be a whole layer above any memory / variable access, with all the runtime information on states of everything and still retaining C in language for maximum portability and performance.
In the system, anytime the application want to use a new variable, or object, or json tree, it create a new typed software variable in the tree, and then get a reference counted pointer to it, in sort to totally eliminate pointer ownership, and it make it very easy to pass a reference to complex json like tree structure with a simple pointer to the root node in the variable tree, and the callee can determine all the content and type at runtime, even if the class or structure or type have never been defined in the language and the compiler have no idea of it.
It's possible to keep track for every of such variable in which function they have been allocated and when, the number of reference to it, as all the access to the variable are made through reference pointer, can relocate the data in memory completely transparently for the application, for doing either temporal optimization, or defragmenting the memory, or to fit with the promise thing in the context of distributed application when you don't have to necessarily know where an object or data is located to be able to access it.
With this system, as all variable access are put out of the hand of compiler specific type, using the system of reference pointers, it become very easy to optimize data access, even through different thread, based on runtime state and information, also in case of threading, or complex interaction, all the data can marked and either 'locked' or 'duplicated' more or less automatically from the moment they are marked as such by the application, and it make it very easy to handle many cases in runtime, rather than trying to have a somehow deterministic approach based on compile time information.
And also it deal with the main problem of C dealing with dynamic data, and it can become very easily bothering when a C program has to manipulate many list of different structure with dynamically allocated size, and direct memory pointer, but with this system, it become totally easy to create as many list of as many different object type with zero bother and zero memory leak.
On the os level, it mean an application can get all the data and states from drivers and kernel level and output them in json like tree, without having to even know anything about the kernel code at all, just knowing the key name used to store the information the application need, i already made a full device explorer rooting from pci bus up to usb device with a single unique UI controller who just display the tree of data coming from bus devices.
I'm also working on system to make all io/dsp like function more transparent, and have good management of io asynch event (network, usb, audio), but that's also another part of the system, it's not that useful all together for blockchain because actually there is not so much many thing that can benefit for parallelisation, as its lot of very sequential operation, so it's not something i'm probably going to dig too deep yet, it's also why i made the raytracing demo to have a toy to play with multi threading and parallelism, but the system of reference pointer and runtime data definition make it also much easier to handle at runtime.
I hope it's not bothering you, and is a least a bit interesting
Anyway as i said i'm going to write more extensive information on this probably in the next month on my website, it's just quick shot at trying to put it out there, if you have any comment or critic don't hesitate !
