Author

Topic: [ANN][XEL] Elastic Project - The Decentralized Supercomputer - page 233. (Read 450523 times)

legendary
Activity: 1260
Merit: 1168
@coralreefer: Extended the pull request! The first functions are now being compiled ;-)

Nice work EK, I think have have all your changes to this point merged (plus I uploaded a lot of cleanup changes to xel_miner.c & miner.h this morning).

One minor question.  Visual Studio is complaining about the rotate functions like.

uint64_t rotl64(uint64_t x, unsigned int n)
{
   const unsigned int mask = (CHAR_BIT * sizeof(x) - 1);
   n &= mask;  // avoid undef behaviour with NDEBUG.  0 overhead for most types / compilers
   return (x << n) | (x >> ((-n)&mask));
}

with the error "unary minus operator applied to unsigned type, result still unsigned".

I switched it from being unsigned and it runs fine now, but not sure that's what you intended.

Strange, here it works flawlessly.
I have taken it from John Regehr's blog post on rotation functions in C (the branchless one) and adapted it accordingly.

http://blog.regehr.org/archives/1063

I think it would be best to check if the VC compiler generates the correct assembly with your change. Something like this:

Code:
rotl32:
        movl    %edi, %eax
        movb    %sil, %cl
        roll    %cl, %eax
        ret

Or at least, in the worst case, sth like this:

Code:
rotl32:
        movl    %esi, %ecx
        movl    %edi, %eax
        shll    %cl, %eax
        negl    %ecx
        shrl    %cl, %edi
        orl     %eax, %edi
        movl    %edi, %eax
        ret
sr. member
Activity: 464
Merit: 260
@coralreefer: Extended the pull request! The first functions are now being compiled ;-)

Nice work EK, I think have have all your changes to this point merged (plus I uploaded a lot of cleanup changes to xel_miner.c & miner.h this morning).

One minor question.  Visual Studio is complaining about the rotate functions like.

uint64_t rotl64(uint64_t x, unsigned int n)
{
   const unsigned int mask = (CHAR_BIT * sizeof(x) - 1);
   n &= mask;  // avoid undef behaviour with NDEBUG.  0 overhead for most types / compilers
   return (x << n) | (x >> ((-n)&mask));
}

with the error "unary minus operator applied to unsigned type, result still unsigned".

I switched it from being unsigned and it runs fine now, but not sure that's what you intended.
legendary
Activity: 1260
Merit: 1168
@coralreefer: Extended the pull request! The first functions are now being compiled ;-)
More to come very soon!

Code:
beavis@methusalem ~/Development/elastic-c-miner (git)-[master] % ./xel_miner -k 19fafc1fa028af61d4bb603e1f9f06eca0bea765114115ad503b532588fbc83d --test-compiler fuck.spl
** xel_miner 1.0 **
[19:13:37] DEBUG: Loading Test File
m[1]=1;
m[2]=1;
m[3]=1;
m[4]=1;
m[5]=1;
m[6]=1;
verify 1==1;

[19:13:37] DEBUG: Running ElasticPL Parser
[19:13:37] DEBUG: Running ElasticPL C-Compiler
mem[1] = 1;
mem[2] = 1;
mem[3] = 1;
mem[4] = 1;
mem[5] = 1;
mem[6] = 1;
return ((1) == (1));


[19:13:37] DEBUG: Delete ElasticPL VM
legendary
Activity: 1260
Merit: 1168
Btw, you got a pull request in your git! My recent changes and the preparation for the "compile" function, in addition to the "interpret" function, as well as an updated mangle_state ;-)

If you like, you can take a took at it ... just reject if "too messy".
legendary
Activity: 1260
Merit: 1168
EK, I have been working in Visual Studio on Windows 10 and when I run your ElasticPL example above it parses fine for me.  I'm assuming you are using a different compiler / OS.  Therefore, I'm thinking the bug is due to packing / alignment of the data in the vm_ast structure.  My guess is the statement:

    vm_ast = malloc(vm_ast_cnt * (sizeof(ast) / sizeof(vm_ast[0])));

is not valid.  Do you know of a more correct way to get the correct amount of memory required for this allocataion?

Thanks for pointing me at this part of the code ...

As we have
Code:
ast **vm_ast;		// Final AST List For VM

... my first guess would be, that this might be correct here:

Code:
vm_ast = calloc(vm_ast_cnt, sizeof(ast*));
memcpy(vm_ast, stack_exp, vm_ast_cnt * sizeof(ast*));

At least, with this change the whole thing runs.  Wink
sr. member
Activity: 464
Merit: 260
EK, I have been working in Visual Studio on Windows 10 and when I run your ElasticPL example above it parses fine for me.  I'm assuming you are using a different compiler / OS.  Therefore, I'm thinking the bug is due to packing / alignment of the data in the vm_ast structure.  My guess is the statement:

    vm_ast = malloc(vm_ast_cnt * (sizeof(ast) / sizeof(vm_ast[0])));

is not valid.  Do you know of a more correct way to get the correct amount of memory required for this allocataion?
sr. member
Activity: 464
Merit: 260

Code:
m[1]=1;
m[2]=1;
m[3]=1;
m[4]=1;
m[5]=1;
m[6]=1;
verify 1==1;

Parses fine until the 3rd line, then uninitialized values appear in VM_AST. The longer I make this "=1" slide, the later the uninitialized values appear!
You will see this trace in valgrind!


I'll try to address this issue when I incorporate the other changes to the parser.  I also struggled getting the C version of mangled_state to match the java version.  I'll try out your suggestion and let you how it goes.
legendary
Activity: 1260
Merit: 1168
@coralreefer: Also, the C mangle_state differs from the Java mangle_state.
The reason is that the rotation / shift operator works different in both languages  Huh

Minimal Example:

C

Code:
int main(void) {
// your code goes here
long long l=-4611686018427387903L;
l = l >> 3 | l << (64-3);
 printf("%lld\n",l);}
 

Code:
stdout
-576460752303423488

Java:

Code:
/* package whatever; // don't place package name! */

import java.util.*;
import java.lang.*;
import java.io.*;

/* Name of the class has to be "Main" only if the class is public. */
class Ideone
{
public static void main (String[] args) throws java.lang.Exception
{
// your code goes here
long l=-4611686018427387903L;
l = Long.rotateRight(l,3);
 
System.out.println(l);
}
}

Code:
4035225266123964416

 Huh Huh Huh
 Huh Huh Huh


Edit, to match java behaviour we should go this way:

Code:
#include 
#include
#include

uint64_t rotl64 (uint64_t x, unsigned int n)
{
  const unsigned int mask = (CHAR_BIT*sizeof(x)-1);
  n &= mask;
  return (x<>( (-n)&mask ));
}
uint64_t rotr64 (uint64_t x, unsigned int n)
{
  const unsigned int mask = (CHAR_BIT*sizeof(x)-1);
  n &= mask;
  return (x>>n) | (x<<( (-n)&mask ));
}
int main(void) {
// your code goes here
long long l=-4611686018427387903L;
l = rotr64(l,3);
        printf("%lld\n",l);
}

Not sure if this behaviour applies to the 64bit types only, or if we should revise 32 bit rotation as well!



EDIT


Same applies to 32 bit  Undecided
-1234>>>3 in C (using our present method) is -155
while it correctly should be -536871067 (Java does that).

Let's use:
Code:
uint64_t rotl64 (uint64_t x, unsigned int n)
{
  const unsigned int mask = (CHAR_BIT*sizeof(x)-1);
  n &= mask;  // avoid undef behaviour with NDEBUG.  0 overhead for most types / compilers
  return (x<>( (-n)&mask ));
}
uint64_t rotr64 (uint64_t x, unsigned int n)
{
  const unsigned int mask = (CHAR_BIT*sizeof(x)-1);
  n &= mask;  // avoid undef behaviour with NDEBUG.  0 overhead for most types / compilers
  return (x>>n) | (x<<( (-n)&mask ));
}
uint64_t rotl32 (uint32_t x, unsigned int n)
{
  const unsigned int mask = (CHAR_BIT*sizeof(x)-1);
  n &= mask;  // avoid undef behaviour with NDEBUG.  0 overhead for most types / compilers
  return (x<>( (-n)&mask ));
}
uint64_t rotr32 (uint32_t x, unsigned int n)
{
  const unsigned int mask = (CHAR_BIT*sizeof(x)-1);
  n &= mask;  // avoid undef behaviour with NDEBUG.  0 overhead for most types / compilers
  return (x>>n) | (x<<( (-n)&mask ));
}
for rotation
legendary
Activity: 1260
Merit: 1168
EK, I have posted my miner here:  https://github.com/sprocket-fpga/xel_miner

Please keep in mind that this miner is in the earliest stages of development and still has lots of issues.

Edit:  The AST is stored in the array vm_ast.  I'm assuming this is what you'd need to traverse for the C code interpreter.

vm_ast seems to be crippled in some way.

Especially this part here causes vm_ast to have uninitialized values since vm_ast_cnt is sometimes wrong.

Code:
// Copy Parsed Statements Into VM Array
vm_ast_cnt = stack_exp_idx + 1;
vm_ast = malloc(vm_ast_cnt * (sizeof(ast) / sizeof(vm_ast[0])));
memcpy(vm_ast, stack_exp, vm_ast_cnt * (sizeof(ast) / sizeof(vm_ast[0])));

This program for example:
Code:
m[1]=1;
m[2]=1;
m[3]=1;
m[4]=1;
m[5]=1;
m[6]=1;
verify 1==1;

Parses fine until the 3rd line, then uninitialized values appear in VM_AST. The longer I make this "=1" slide, the later the uninitialized values appear!
You will see this trace in valgrind!

Code:
[12:29:26] DEBUG: Running ElasticPL Parser
--------------------------------
Type: 3, m[] 1
Type: 2, 1
Type: 6, =
--------------------------------
Type: 3, m[] 2
Type: 2, 1
Type: 6, =
--------------------------------
Type: 3, m[] 3
Type: 2, 1
Type: 6, =
--------------------------------
Type: 3, m[] 4
Type: 2, 1
Type: 6, =
--------------------------------
==22460== Thread 3:
==22460== Conditional jump or move depends on uninitialised value(s)
==22460==    at 0x40AA8D: dump_vm_ast (ElasticPL.c:93)
==22460==    by 0x40A9F6: create_epl_vm (ElasticPL.c:69)
==22460==    by 0x405391: test_vm_thread (xel_miner.c:483)
==22460==    by 0x52B4453: start_thread (in /usr/lib/libpthread-2.24.so)
==22460==    by 0x5B2C7DE: clone (in /usr/lib/libc-2.24.so)
==22460==
==22460== Use of uninitialised value of size 8
==22460==    at 0x40AA97: dump_vm_ast (ElasticPL.c:94)
==22460==    by 0x40A9F6: create_epl_vm (ElasticPL.c:69)
==22460==    by 0x405391: test_vm_thread (xel_miner.c:483)
==22460==    by 0x52B4453: start_thread (in /usr/lib/libpthread-2.24.so)
==22460==    by 0x5B2C7DE: clone (in /usr/lib/libc-2.24.so)
==22460==
==22460== Invalid read of size 8
==22460==    at 0x40AA97: dump_vm_ast (ElasticPL.c:94)
==22460==    by 0x40A9F6: create_epl_vm (ElasticPL.c:69)
==22460==    by 0x405391: test_vm_thread (xel_miner.c:483)
==22460==    by 0x52B4453: start_thread (in /usr/lib/libpthread-2.24.so)
==22460==    by 0x5B2C7DE: clone (in /usr/lib/libc-2.24.so)
==22460==  Address 0x8ece58 is not stack'd, malloc'd or (recently) free'd

hero member
Activity: 994
Merit: 513
Guys, I really like your discussions and appreciate all your contribution, but don't you think you should really start focusing on finalising a mainnet candidate that will form a basic foundation for the future? You know, the discussions can last forever without results, and we want to make decisions and move XEL forward, don't we? Then, once mainnet is launched, you could develop a roadmap for development.

Well, if you read this page carefully, I think you'll find that this is exactly what the last few posts were about.
Apart from that, please stop using the word "you" so much Wink If you want a roadmap, I'm afraid, you have to suggest one.
hero member
Activity: 1022
Merit: 507
Guys, I really like your discussions and appreciate all your contribution, but don't you think you should really start focusing on finalising a mainnet candidate that will form a basic foundation for the future? You know, the discussions can last forever without results, and we want to make decisions and move XEL forward, don't we? Then, once mainnet is launched, you could develop a roadmap for development.
full member
Activity: 235
Merit: 100
The ideal would be to use your AST parser to create C code on the fly, which is then compiled and linked into a library that then again is used to execute the logic inside your miner ;-)

Then it would be better not to compile to C code, but JIT compile using LLVM http://llvm.org/

Minus one intermediate step.
hero member
Activity: 994
Merit: 513
just want to show my gratitude as an investor for that the devs are working really hard together with the community to fix all the bugs and take their time to release an good working product/service.

Keep up the good work and just take the time what is needed in order to release a solid version Smiley

Just to make sure there are no misunderstandings in the future: There are no "devs". There are just some guys pushing the project forward.

Ok, here is a random question that popped up in my head on the way home:
Do you think it is dangerous or holds abusive potential, if a job is completely or nearly completely fulfilled by a single miner?

Oh, and to add to this:
My thought is that a reputation system may have merits, but it should not be part of this first release.  There are more critical issues to address and I think we need to focus on the known issues like difficulty retargetting, etc before tackling enhancements.

I second that, including the question regarding the possible "two job types" proposal. It's nice to keep options open, but we should work on Elastics key functionality.

At the moment, I'd like to point my energy in a direction that is useful. I can sit there, thinking about decentralized reputation systems all day and night, if we don't plan to implement it after all, it is wasted time. If we want to move Elastic forward, focus is key. Should we come to a point, where we start thinking about a rep. system again, I'll hook up my brain rigs and point my thinking power in that direction (it's late, I write bad metaphors…).

Got some more stuff, but I need to go to bed and think about it…
legendary
Activity: 1148
Merit: 1000
just want to show my gratitude as an investor for that the devs are working really hard together with the community to fix all the bugs and take their time to release an good working product/service.

Keep up the good work and just take the time what is needed in order to release a solid version Smiley
legendary
Activity: 1260
Merit: 1168
My thought is that a reputation system may have merits, but it should not be part of this first release.  There are more critical issues to address and I think we need to focus on the known issues like difficulty retargetting, etc before tackling enhancements.

I second that, including the question regarding the possible "two job types" proposal. It's nice to keep options open, but we should work on Elastics key functionality.

We at least could use the wiki to keep track of all the great ideas and future works ... it would be a pit to lose track of them ;-)
hero member
Activity: 994
Merit: 513
My thought is that a reputation system may have merits, but it should not be part of this first release.  There are more critical issues to address and I think we need to focus on the known issues like difficulty retargetting, etc before tackling enhancements.

I second that, including the question regarding the possible "two job types" proposal. It's nice to keep options open, but we should work on Elastics key functionality.
sr. member
Activity: 464
Merit: 260
My thought is that a reputation system may have merits, but it should not be part of this first release.  There are more critical issues to address and I think we need to focus on the known issues like difficulty retargetting, etc before tackling enhancements.
hero member
Activity: 994
Merit: 513
Quote
At the moment, I think Evil-Knievels word holds the most weight here, I'd go his way and abandon my pursues of finding a working reputation system.

No, this should be not the case! I am just working shifts here, I am no authority! I do what the majority wants to ... so let's discuss all ideas that come up! I am still thinking about your "second approach" though ;-) I will comment soon!

Hehe, this is pretty much exactly what I was talking about Smiley
Look, you have shown expertise. We could ask all future XEL holders which way to go and I bet a considerable amount would say "umm, what Evil-Knievel says."
I agree, that we should discuss options, but it isn't really happening, is it?
I remember something written regarding the DAO (before the "accident") that pretty much said "If I need heart surgery, I go to a surgeon. I don't ask 100 random people how they would operate." This is a similar situation. We could still ask the people, but they will probably say "go to a surgeon" Wink
ImI
legendary
Activity: 1946
Merit: 1019
I am with EK here as i have followed some of the discussions Golem is having in finding a valid and not manipulable reputational system. They seem to have put lots of thought into it, including using lots of scientific articles in that certain field, and havent come up with a solution so far. That leads me to believe if we can avoid it we should try to, imho.

If the community is really undecided in which way to go we can always use EK's nice voting system and vote about a certain issue. Of course not after having a good discussion before..
legendary
Activity: 1260
Merit: 1168
Quote
At the moment, I think Evil-Knievels word holds the most weight here, I'd go his way and abandon my pursues of finding a working reputation system.

No, this should be not the case! I am just working shifts here, I am no authority! I do what the majority wants to ... so let's discuss all ideas that come up! I am still thinking about your "second approach" though ;-) I will comment soon!
Jump to: