Page 1 of 1

Nameindex.dat - Berkleydb - anyone familiar want to help?

Posted: Tue Jan 07, 2014 4:38 pm
by snailbrain
p.s. not sure if this is the right place to post, or could even try bitcointalk dev board (although can be hostile if not related to btc :D)
any help could save a lot of time

Here is the problem :

If the client is forcefully terminated, blkindex.dat and nameindex.dat may get out-of-sync. After that, the chain will stick at the current block.
To fix, one has to delete nameindex.dat (it will then be automatically recreated on startup via rescan).

There are two ad-hoc solutions for this:
1) get rid of nameindex.dat and store it's contents in blkindex.dat, so the two will always be synced
2) store last block number in nameindex.dat and rescan last N blocks on startup (like done in wallet.dat)

However there should exist a simpler solution based on BDB transactions. CNameDB class has some functionality for this. The constructor accepts the parameter

CDB& parent

which is meant to make changes to two separate databases atomically. It is used like this:

Code: Select all

    txdb.TxnBegin();
    ...
    // Process block
    ...
    {
        // Save names from block to name db
        CNameDB dbName("cr+", txdb);
        dbName.TxnBegin();
        ...
        dbName.TxnCommit();
    }
    ...
    if (accept_block)
        txdb.TxnCommit();
    else
        txdb.TxnAbort(); 
It is supposed that the last commit/abort must keep CNameDB in sync too, but for some reason this functionality is broken.
Someone knowledgeful in BDB can probably spot the problem quite easily. There are three places to check:
1) is parent DB functionality implemented properly?
2) is it used properly (as per the code above)?
3) is it used everywhere it should be used (i.e. each time something is written to NameDB)?

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Posted: Wed Jan 15, 2014 9:41 am
by biolizard89
libcoin doesn't use BDB, so I think the best way forward is to do the libcoin rebase and make this issue irrelevant.

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Posted: Wed Jan 15, 2014 12:34 pm
by snailbrain
yep.. but.. is that going to be a long time?

if someone is familiar with bdb, thecoder thinks they can solve in "minutes"..
so if anyone can help, it will still be "good"...

libcoin (the person) is busy atm.. but his last update (last week)
....busy...... I will soon start to look into adding some of the missing functionality asap, could probably need some help re the gui you have added, but let me have a closer look at it first.
Also, i'm not sure if the consensus has agreed on the rebase/re-implementation on libcoin.. has it?
i'm ok with it.

Eitherway if anyone can see the problem, please let us know :)

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Posted: Tue Mar 25, 2014 4:12 pm
by phelix
I guess this issue is still open. Snailbrain, should we throw a small bounty at it?

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Posted: Wed Mar 26, 2014 4:10 pm
by snailbrain
phelix wrote:I guess this issue is still open. Snailbrain, should we throw a small bounty at it?
I'm not sure how far a long domob is with libcoin, but i think it will make the problem obsolete as biolizard said.
I think Libcoin still uses bdb with the wallet (so i read once) but shouldn't be related?

The pool that has been having the issue is Bitminter. It was happening randomly (but not often, possibly every few months).

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Posted: Thu Mar 27, 2014 6:49 am
by domob
snailbrain wrote:
phelix wrote:I guess this issue is still open. Snailbrain, should we throw a small bounty at it?
I'm not sure how far a long domob is with libcoin, but i think it will make the problem obsolete as biolizard said.
I think Libcoin still uses bdb with the wallet (so i read once) but shouldn't be related?

The pool that has been having the issue is Bitminter. It was happening randomly (but not often, possibly every few months).
I've made some progress (name_show is working basically), but am waiting on some answers from Michael at the moment. (I'm not sure where to get the information from to show txid and address of old entries in name_history - not sure if that data is even somewhere available in libcoin.)

Libcoin uses SQLite for the blockchain, but yes, it still uses BDB for the wallet (but I think you can also use a SQLite database for the wallet in case compatibility is not necessary).