Pages:
Author

Topic: What DB do you use at your end? - page 2. (Read 3654 times)

hero member
Activity: 1204
Merit: 531
Metaverse 👾 Cyberweapons
November 27, 2016, 06:50:31 AM
#14
Yes, I used MS SQL for a project www.blockchainsql.io.  450GB database.  If you optimise right, queries can be lightning fast.

Cool. Is that your project or did you build a project that used the mentioned project as something like an API?

At a 450-GB DB I think queries could be optimized up to less than 1 minute speed with your home gaming PC.

Correct me if I misunderstood something, but would not that require a MUCH bigger size of HDD?

As deletion (and in Bitcoin rollbacks) are infrequent operations it wouldn't practically be as significant an increase as you are imagining (but it would use more space for the translation log).

In addition, the log writing and reading speed would significantly increase that would result in slower DBs, would not it? At large DBs, speed is essential because some queries could take up to 20 or more minutes even without the mentioned idea.

Normal RDBMS "queries" do not involve the log at all so it would not have any speed effect upon read operations (it would only slow down "delete" operations).


It depends on the use-case then because in some DBs there are frequent delete operations. But I agree that most DBs are not like that so the possibility is worth to take into consideration. The question is how much it would increase the necessary space and the issue of SRP because even though rollbacks are infrequent operations, when they need them they usually need them ASAP.
legendary
Activity: 1890
Merit: 1078
Ian Knowles - CIYAM Lead Developer
November 27, 2016, 01:20:01 AM
#13
Correct me if I misunderstood something, but would not that require a MUCH bigger size of HDD?

As deletion (and in Bitcoin rollbacks) are infrequent operations it wouldn't practically be as significant an increase as you are imagining (but it would use more space for the translation log).

In addition, the log writing and reading speed would significantly increase that would result in slower DBs, would not it? At large DBs, speed is essential because some queries could take up to 20 or more minutes even without the mentioned idea.

Normal RDBMS "queries" do not involve the log at all so it would not have any speed effect upon read operations (it would only slow down "delete" operations).
hero member
Activity: 765
Merit: 503
November 27, 2016, 12:40:53 AM
#12
Eg:  total chain size

select sum(block.size) as totalsize from block
hero member
Activity: 765
Merit: 503
November 27, 2016, 12:39:08 AM
#11
How should a rollback happen in your view?

A real "rollback" would require that the transaction log would have kept the full details of any deleted record (no major DB in the world does that AFAIA).

So if you want to "reverse history" it would be easy enough to recover "deleted records" using the transaction log.

Hope that makes sense.


Correct me if I misunderstood something, but would not that require a MUCH bigger size of HDD? In addition, the log writing and reading speed would significantly increase that would result in slower DBs, would not it? At large DBs, speed is essential because some queries could take up to 20 or more minutes even without the mentioned idea.

Yes, I used MS SQL for a project www.blockchainsql.io.  450GB database.  If you optimise right, queries can be lightning fast.
hero member
Activity: 1204
Merit: 531
Metaverse 👾 Cyberweapons
November 26, 2016, 05:34:53 PM
#10
How should a rollback happen in your view?

A real "rollback" would require that the transaction log would have kept the full details of any deleted record (no major DB in the world does that AFAIA).

So if you want to "reverse history" it would be easy enough to recover "deleted records" using the transaction log.

Hope that makes sense.


Correct me if I misunderstood something, but would not that require a MUCH bigger size of HDD? In addition, the log writing and reading speed would significantly increase that would result in slower DBs, would not it? At large DBs, speed is essential because some queries could take up to 20 or more minutes even without the mentioned idea.
legendary
Activity: 1890
Merit: 1078
Ian Knowles - CIYAM Lead Developer
November 21, 2016, 03:53:12 PM
#9
How should a rollback happen in your view?

A real "rollback" would require that the transaction log would have kept the full details of any deleted record (no major DB in the world does that AFAIA).

So if you want to "reverse history" it would be easy enough to recover "deleted records" using the transaction log.

Hope that makes sense.
hero member
Activity: 1204
Merit: 531
Metaverse 👾 Cyberweapons
November 21, 2016, 03:02:05 PM
#8
I have worked with many RDBMS engines (including DB/2 and Oracle) and they are all very powerful things.

There is really no reason to prefer one over the other in regards to what you are wanting to do.

The only thing that no current RDBMS handles well is actual "rollback" in terms of a "re-org" (normally a DB "rollback" is actually just restoring a backup and then "rolling forward" the newer txs from the log).

AFAIA no modern DB can do this (although I do have a design for exactly this).


With that argument you have raised my curiosity.

Would you mind, in the essence at least, elaborating your design for me?

How should a rollback happen in your view?
legendary
Activity: 1890
Merit: 1078
Ian Knowles - CIYAM Lead Developer
November 20, 2016, 02:50:40 PM
#7
I have worked with many RDBMS engines (including DB/2 and Oracle) and they are all very powerful things.

There is really no reason to prefer one over the other in regards to what you are wanting to do.

The only thing that no current RDBMS handles well is actual "rollback" in terms of a "re-org" (normally a DB "rollback" is actually just restoring a backup and then "rolling forward" the newer txs from the log).

AFAIA no modern DB can do this (although I do have a design for exactly this).
legendary
Activity: 1568
Merit: 1031
November 20, 2016, 02:38:35 PM
#6
So, most people use MySQL here, but also recommend SQLite and Realm.io. I am missing Oracle, though. In my story, I was using Oracle for several months up to now and it has nice features like highly customizable locking configuration. Although I used and agree with the excellence of the listed other DBs, why am I so lonely with my Oracle as a favorit?

Oracle is considered as one of the most powerful RDBMS, don't under estimate it. It's mostly the companies that use Oracle and here why as for the differences you could check this article from Udemy or Oracle documentation
hero member
Activity: 1204
Merit: 531
Metaverse 👾 Cyberweapons
November 19, 2016, 05:47:46 PM
#5
So, most people use MySQL here, but also recommend SQLite and Realm.io. I am missing Oracle, though. In my story, I was using Oracle for several months up to now and it has nice features like highly customizable locking configuration. Although I used and agree with the excellence of the listed other DBs, why am I so lonely with my Oracle as a favorit?
legendary
Activity: 1568
Merit: 1031
November 18, 2016, 05:04:24 AM
#4
I was researching the ways how people make queries in the blockchain and I got the result that many create custom DBs on their ends to support their ways. If you use an additional DB at your end (other than core's leveldb) to store blockchain data, what DB do you use?


Both MSSQL Server (If you are using Windows to host it) and MySQL should be great choices . SQLite or Realm.io should be considered as an alternative for MSSQL or MySQL If you are building a mobile app.
legendary
Activity: 1890
Merit: 1078
Ian Knowles - CIYAM Lead Developer
November 18, 2016, 04:57:50 AM
#3

Whilst it is a great small SQL DB it isn't suitable for multiple users due to its use of global locking (unless they have completely reworked that in recent years).

I would favour using something like MySQL if wanting to have something that will scale hugely and handle multiple concurrent users.
full member
Activity: 219
Merit: 102
November 18, 2016, 04:53:08 AM
#2
hero member
Activity: 1204
Merit: 531
Metaverse 👾 Cyberweapons
November 17, 2016, 04:13:45 PM
#1
I was researching the ways how people make queries in the blockchain and I got the result that many create custom DBs on their ends to support their ways. If you use an additional DB at your end (other than core's leveldb) to store blockchain data, what DB do you use?
Pages:
Jump to: