Topic: HZ/NXT: assetparser.py >40 columns per asset | shareholders.py inner circle PM (Read 7602 times)

AssetGraphs.py is born!

Two new threads related to my nxthacks participation

"AssetGraphs.py is born!" explains backgrounds, and paints possible futures for my project.

"AssetGraphs to Open Source - help me design a new asset" does what it says  ;-)

Thanks!

Out of the 48hours, I slept 2 times only 4, and was coding > 24 hours:

• 1052 lines of Python:
  api.py = 1 completely new module for abstracted access to the nxt API in python.
  api_testing.py = accessing & testing the above; generating the Top44 of issuers, etc.
  network.py = 1 completely new module for getting, selecting, formatting API results and transforming them into a network structure.
  I used none of my existing modules, but built everything anew. (import urllib2, urllib, json, pprint, logging, time, threading, random, threading, pickle)
  
  
• ... which created the 9 network.json files as an input for ...
  
  
• a JavaScript that I needed to understand & adapt (thx to d3 = a cool viz tool); plus
  9 webpages to hold (stepwise adapted & extended) copies of that script, that went into ...
  
  
• 11 webpages, 1 css & 1 navigation (adapted template from dreamweaver), and 1 screenshot,
  to then present the whole thing.

And I have just discovered the voting. Great! -->

I suggest to add that to your signature for 4 days? Thx!


Retweet if you like me :-)

Short overview for TweetPic:


... read more interesting details now at     2020.fm/assetparser


.

New much improved version of
assetparser.py

changelog:

* asset holders, accounts:
** before only the workarounds for HZ were implemented, now all also works on NXT again
** funfacts accounts: number of asset positions, asset holding accounts
** HZ: pickle/unpickle all accounts (for faster debugging)

* assets:
** NXT: genesis account block time --> time calculations
** funfacts assets: simple asset shareholder analysis (mean median max, >10, >100, >1000)

* trades:
** Nxt version: sellers & buyers are given by API (faster & easier than HZ workaround of getting all order txs)
** HZ: extracted order ids can now also contain bidOrders, or freely choosable (good for 'shareholders.py')
** progress: show a "*" per downloaded asset
** funfacts: simple number-of-trades analysis (mean median max, >10, >100, >1000)
** analyzeAllTrades can easily handle 100k trades

* architecture:
** all asset-specific analysis is already or will go into shareholders.py
** most api calls now in an own module apiclient.py

* observables - what cannot be calculated for HZ now also include as empty columns, to allow easier platform mixing
* altprices.py - usd as a base, due to missing data
* logging - better and more consistent logging output


Future:

Open for suggestions.


Sample output:
NXT assetparser.py

Sample output
HZ assetparser.py
This is more complicated, because the old API of HZ forced me to invent some clever work-arounds. But then it results in almost the same information:


Open for suggestions.


Availability
For now:

For your own (private!) detailed analysis ...

Buy snapshot spreadsheet tables of the 2 whole asset markets analysis, of these sizes:
* HZ: 147 rows(assets), 42 columns(observables) with 4146 non-empty cells (table 67% filled)
* NXT: 572 rows(assets), 42 columns(observables) with 18249 non-empty cells (table 76% filled)

Buy snapshot rows of selected (e.g. of your own) assets:
* 1 row(asset) with 42 columns(observables)

Tell me what you are interested in, and make an offer how much this is worth to you.


Money talks

Inviting all ideas how to monetize this in the future.

If you already found this interesting, informative, inspiring ... consider to GIVEBACK.

Thanks!

Lootz from supernetradio (my 1 hour interview about AltFolio is online on soundcloud),
and me chatted a bit about AltFolio, and shareholders.py, and assetparser.py, in a slack.

Here is a transcript (without all the uploaded pictures).

 

... it doesn't have a GUI yet, and so far can be run only by me. But could already be booked as a service, actually.

Though, only for experienced users/admins, no newbie support (or yes - if you pay well & upfront).
Makes most sense if you have a web server where you can password-protect an area /shareholders/innercircle

More info above in these FIVE consecutive posts, and on my webpage.

Yes, I know.

That makes sense.

In Nxt there is / will soon be a paid plugins system.


Well, what you can always do is "follow the money", i.e. when he moves his funds somewhere else, to traverse the blockchain, hopping from one account to the next one. That way you can try to find out as much as you can.  That is mainly done manually, because you need to assess the situation with your intelligence, and your own pattern recognition system.



Agreed.

I am happy that my program runs not only on one platform.

I don't know, I would like it best if it would be just integrated into the wallet. Maybe you should join the HZ team. I would also like functions/ideas of your other programs integrated into the HZ wallet. But these are just thoughts and wishes, I don't know what is possible and what you are willing to do. (And I don't really have to know it.)


And some use cases, too.


As an honest person, you always loose all bad and illegal opportunities. Robbing is easier than sowing, harvesting and working. But the good way is the stony way...  

My involvement in GlideSec was only short, I was just starting and testing, as the only way in cryptos seems to test it for myself, as many people talk a lot of contradictory things here and I don't know what I should believe. So, I just received my first (and only) dividend, calculated it and it looked great, so I bought some more shares. Then suddenly (for me) there was this idea with the GlideCoin (completely unnessesary for me) and everything stopped.


The developer/asset issuer could explain himself to calm the 'mob', but he didn't, instead he made false promises or gave no information at all (and seems to have left, nobody knows). If it is no scam, it is a very bad and deceiving behavior. But it's likely scam or he changed his mind and took everything he could get out of it, and left.


This might never be 100% clear. Only the developer or some 'detective work' could clear it up.


Well, the only thing I did so far was looking at the shareholder distribution. I don't know if more will be necessary or promising. We'll see if there are any more ideas. Personally, I did not loose too much money to put more into it, but I would like to help and to prevent such cases. And to get an explanation (and maybe my money back and/or some punishment for the scammer, if he is one). But maybe some big investors could be willing to put some money into investigation of the case. Do you have any ideas for investigation? Does your software help there?


Well, bigger and better is if you make it useable for both.


You should still have my email address. Additionally, I don't like to connect my wallet with my e-mail address. I know that HZ is not an anonymous cryptocurrency system, but anyway, for the moment I prefer to stay "semi-anonymous".

Of course, by anyone. It just needs a nonempty balance, and -right now unfortunately still- the private key of the sender.

It is still a quickNdirty script, which will not leave my machine in the current state, and can only be triggered by myself.

But it can become a product on its own.
For that, I will have to research local tx signing (to protect the sender's private key from me)
- and sourcecode protection! And perhaps GUI programming even. Or make it a webservice?


Exactly. Let us better not even go there ;-)

I am actually considering to hard-code the restriction that an asset issuer can only send these messages to the shareholders of his own asset. That would -in one easy decision- render many abuse cases impossible.


--> https://bitcointalksearch.org/topic/anyone-wants-to-do-something-about-glidesec-asset-and-glidecoin-issue-1044199 --> Interesting.

> "GlideSec Asset and GlideCoin issue (maybe Scam)?"
GlideSec. aha. To me, that thing looked fishy, from day one.  

Or at least like something that I want to stay away from. If someone promises superhigh returns without risks, he is either a genius, or a liar / scammer - or scamming other people to get their money, to then distribute it to its own crownies. Myself, I do not want to be part of such schemes. But perhaps I am wrong, and loosing out on a lot of lucrative opportunities?



So ... what if it is not fraud?
Perhaps it is just an angry mob - which is unhappy about their disappointed expectations to get rich immediately?  

However, once the case is 100% clear, and decided beyond any doubts, come back to me. Then ideally with an already collected crowdfunded bounty, to pay me. Not much, but worth my work.

It does, indeed.

This last function at the moment yet only in HZ, because I have been focussing on my own asset shareholders right now; but with a bit of extra coding, no problem - it would run on Nxt, for sure.

It is a bigger market.


De nada. It is a thank you from me to my shareholders. Every shareholder got one from me.
Not everyone has answered back, though.

EDIT:
So: Thx! Please inform me by PM me so that I can connect email & address. Thanks!
Actually, I changed my mind about that. Why bother with unencrypted email if we can have encrypted messaging. If someone gives me an email address, of course I will use it - but I might not continue to explicitly ask for it.
Still, it would be nice to know for which of the addresses I can tick the box, so why not ... simply send me a HZ PM. Thanks.

Nice and useful idea.

Can it also be used by asset holders?

This would open interesting possibilities of use and abuse...

An example of a use would be the gathering, communication and coordination of defrauded users, as in this probable fraud case (but it's a NXT asset, where the function of listing all asset holders is easier than in current Horizon):

https://bitcointalksearch.org/topic/anyone-wants-to-do-something-about-glidesec-asset-and-glidecoin-issue-1044199

Does your program function in NXT also? NXT is a better market.

(BTW thank you for the friendly password)

... it doesn't have a GUI yet, and so far can be run only by me. But could already be booked as a service, actually.

Though, only for experienced users/admins, no newbie support (or yes - if you pay well & upfront).
Makes most sense if you have a web server where you can password-protect an area /shareholders/innercircle

More info above in these FIVE consecutive posts, and on my webpage.

Announce it on twitter (example), facebook, bitcointalk ...



Click on the pic to the to my real posting.

Webpage /shareholders, open for everyone

Clicking on inner circle ...

Wrong password --> unauthorized

Correct password --> access to protected area

A beautiful newsletter is waiting there ...


Cool, ain't it?

This is how it looks to the recipient = shareholder.

Shareholders.py is born

I built a new tool in a crazy 17 hours session, to reach out to my 17 shareholders,
celebrating the 1 month birthday of AAssetHZ.

The following is the program output, it is self explaining.

(The asset 'beautiful' (id: 5903523947573024709) is a test asset that I have issued only for this purpose. Do not buy it.)



--------------------------------------------------
Sending messages to all shareholders of
'beautiful' (assetId = 5903523947573024709)
--------------------------------------------------
from sending account  NHZ-VJ23-6K9N-33Z2-4DHQV

THEREALDEAL = True

Step 1: Parsing the blockchain for all shareholders
[03:48:09.869]  nhz:
[03:48:10.901]  API-downloading all assets: 122 assets. DONE.
[03:48:12.897]  API-downloading all account ids from NHZ block explorer: 4525 accounts. DONE.
[03:48:30.819]  API-Downloading 4525 accounts (async, MAXTHREADS=1000): 4525 threads finished. DONE.
[03:48:30.848]  2124 accounts are holding (median 200.2 mean 470808.9) HZ, and 708 are holding assets.
[03:48:31.005]  Analyzing 708 shareholders, with 1308 asset balances. DONE.

Asset 5903523947573024709 has 4 shareholders: (asset, accountRS, amount, countOfDifferentAssets, HZ-balance, name, alias)
5903523947573024709   NHZ-YPPJ-5S6B-NKYK-B5FVX    0.0001   1       128.0                          altsheetstestaccount04
5903523947573024709   NHZ-4BHN-8FBZ-TVJS-DMGF6    3.7188   1         4.0                          altsheetstestaccount02
5903523947573024709   NHZ-VJ23-6K9N-33Z2-4DHQV   34.1394   1        35.0                          altsheetstestaccount03
5903523947573024709   NHZ-QEQG-G5N8-8ASD-FYHFR    4.1417   1         2.0                          altsheetstestaccount01

all addresses:
[u'NHZ-YPPJ-5S6B-NKYK-B5FVX', u'NHZ-4BHN-8FBZ-TVJS-DMGF6', u'NHZ-VJ23-6K9N-33Z2-4DHQV', u'NHZ-QEQG-G5N8-8ASD-FYHFR']

with public keys:
{u'NHZ-4BHN-8FBZ-TVJS-DMGF6': u'cfa2d418cb99f9740c37843262fae2ca27c3490c6f6d57dafd494ba748e1e71c', u'NHZ-YPPJ-5S6B-NKYK-B5FVX': u'83a16fee6e5b88e6fa6d9a7912b2882aadb361e22d6a3f5020c89b4e38a2c875', u'NHZ-VJ23-6K9N-33Z2-4DHQV': u'2f57ed957453207b926c8466944856d9e32f89142f327d9e25fb9dbf9788f512', u'NHZ-QEQG-G5N8-8ASD-FYHFR': u'63f26ee1f2389f5fe08da039644dde51fc02ac23419d51aa445f7900f00d0355'}


Step 2: Making passwords, compiling the messages from template, checking lengths.

Have a quick look on the passwords, so they are not obscene :-)
['5xepazwg', 'laqmaxcs', 'hw9kce6q', 'ou34fjiv']

Sample message
--------------------------------------------------

Hello shareholder!

You own shares of asset 'beautiful' (id 5903523947573024709).

Good news for you -->

Visit the new shareholder inner circle at
http://altsheets.ddns.net/shareholders

Your personal credentials are:
  username: NHZ-4BHN-8FBZ-TVJS-DMGF6
  password: 5xepazwg

Enjoy!
       your happy AltSheetsDev :-)

--------------------------------------------------


Step 3: Checking balance
The sending account NHZ-VJ23-6K9N-33Z2-4DHQV has a balance of 35.01 HZ.


Step 4: Sending encrypted messages
only continue here if you are totally sure:

Please type  yes  :yes

------------------
start sending. Please be patient:
NHZ-4BHN-8FBZ-TVJS-DMGF6  success=True txID=17542049851144901313
NHZ-YPPJ-5S6B-NKYK-B5FVX  success=True txID=17826415227643272475
NHZ-VJ23-6K9N-33Z2-4DHQV  success=True txID=7726216962759025843
NHZ-QEQG-G5N8-8ASD-FYHFR  success=True txID=6829710997944664049
ready.

Step 5: Linux commands to set the password for each shareholder. Do this now:
sudo htpasswd -b /path/to/your/.htpasswd NHZ-YPPJ-5S6B-NKYK-B5FVX laqmaxcs
sudo htpasswd -b /path/to/your/.htpasswd NHZ-4BHN-8FBZ-TVJS-DMGF6 5xepazwg
sudo htpasswd -b /path/to/your/.htpasswd NHZ-VJ23-6K9N-33Z2-4DHQV hw9kce6q
sudo htpasswd -b /path/to/your/.htpasswd NHZ-QEQG-G5N8-8ASD-FYHFR ou34fjiv

Step 6: Who enabled you to do this?
Please give a reward to AltSheetsDev now.
How much HZ you want to send? (0 to skip):  50000
Thanks. Sending now (please be patient):
donation success=False txID=Not enough funds

Step 7:
Test the webaccess, that it really works. Monitor the webserver logfiles if you can.
Use all ways to inform your shareholders, otherwise they might not open their wallet inbox.
(twitter, bitcointalk, facebook, etc) Example: http://altsheets.ddns.net/img/HZ-PM.png
Check your own wallet NHZ-VJ23-6K9N-33Z2-4DHQV for incoming answers.
Please give me feedback how this works out for you. Thanks!
Also: Consider to buy my 'AAssetHZ' to support me, and earn dividends.
Enjoy your day!

--------------------------------------------------



This tool is interesting for every
Asset issuer:
Do you want to buy this service from me?

PM me an offer.

Thanks tyz, for your interest.

First of all: This is work in progress. And myself, I have not even really started to show the whole information - in the preview you see 12 columns, while the newest version already produces 43 columns, for each asset. Not all of them hyper-essential, but they are there now.

No.

The thing is that I issued this asset myself, and with selling it, I have got some money from investors - who will want to get it back long-term, in the form of dividends. So I need to find solutions how all of my work can be monetized.

Releasing my working hours that went into AltFolio and assetparser as open source would for sure help the community - but not me nor my shareholders. 

As much as I personally like free flow of information and collaboration - I have to think about money, always. You see my dilemma?


And I would like you to use it. Happy that you ask.

Which way can we find to solve this?

Have you released the source code somewhere? Is it free to change? I would like to use and extend it a little bit to show more information.