NuDB: A Key/Value Store For Decentralized Systems

samson

legendary

Activity: 2097

Merit: 1070

Quote from: misterbigg on April 11, 2015, 05:26:44 PM

No answer?

Nothing specific, I just find it interesting. Certain useful datasets can be key/value for something I'm working on.

I'm using sqlite with simple (but potentially large) recordsets based on a unique key at the moment so I'm open to new interesting things.

tacotime

legendary

Activity: 1484

Merit: 1005

These types of DBs are popular lately, for example, btcwallet now uses boltdb, while Monero uses LMDB. They're all kind of similar: full ACID semantics w/ MVCC, data stored on disk in B+ trees, async read/update with scheduling and transitory state patches for reads, multiple namespaces, nested buckets, etc.

AFAIK, bitcore is moving towards adopting more DB agnostic architecture, so in the future you can plug and play whatever crazy DB type you want.

misterbigg

legendary

Activity: 1064

Merit: 1001

No answer?

misterbigg

legendary

Activity: 1064

Merit: 1001

Quote from: samson on March 30, 2015, 05:44:45 AM

Well it looks interesting, how does it compare to sqlite for a few million records ?

I'm not sure a comparison against SQLite is meaningful. SQLite offers indexes and structured data, NuDB is only a key/value storage system - you can only look up values by their key and nothing else.

Beyond that, you need to be more specific. A few million records of what? Inserts? Fetches?

Or are you asking about the performance of the database after a few million records are already present? Because NuDB does not slow down at all, regardless of how large the database gets. It offers constant performance.

samson

legendary

Activity: 2097

Merit: 1070

Well it looks interesting, how does it compare to sqlite for a few million records ?

misterbigg

legendary

Activity: 1064

Merit: 1001

I developed this embedded database for Ripple:

NuDB: A Key/Value Store For Decentralized Systems
https://github.com/vinniefalco/Beast/tree/master/beast/nudb

Fetch performance is outstanding, and remains constant no matter how big the database. It uses an on-disk hash table so the database needs to be on an SSD drive. It has journaling so it resists corruption on power failure. It supports POSIX and Win32 native APIs.

LevelDB, and RocksDB turned out to be bad solutions for us because as the database grows the performance goes down. And those databases require increasing amounts of memory. So NuDB is designed specifically for our use case.

Could Bitcoin or other related software benefit from this? If there is sufficient interest I might put it in its own repository. The code is headers-only (like boost), except for the unit tests.

Topic: NuDB: A Key/Value Store For Decentralized Systems (Read 1743 times)