Nameindex.dat - Berkleydb - anyone familiar want to help?

Post Reply
snailbrain
Posts: 309
Joined: Tue Jul 19, 2011 9:33 pm

Nameindex.dat - Berkleydb - anyone familiar want to help?

Post by snailbrain »

p.s. not sure if this is the right place to post, or could even try bitcointalk dev board (although can be hostile if not related to btc :D)
any help could save a lot of time

Here is the problem :

If the client is forcefully terminated, blkindex.dat and nameindex.dat may get out-of-sync. After that, the chain will stick at the current block.
To fix, one has to delete nameindex.dat (it will then be automatically recreated on startup via rescan).

There are two ad-hoc solutions for this:
1) get rid of nameindex.dat and store it's contents in blkindex.dat, so the two will always be synced
2) store last block number in nameindex.dat and rescan last N blocks on startup (like done in wallet.dat)

However there should exist a simpler solution based on BDB transactions. CNameDB class has some functionality for this. The constructor accepts the parameter

CDB& parent

which is meant to make changes to two separate databases atomically. It is used like this:

Code: Select all

    txdb.TxnBegin();
    ...
    // Process block
    ...
    {
        // Save names from block to name db
        CNameDB dbName("cr+", txdb);
        dbName.TxnBegin();
        ...
        dbName.TxnCommit();
    }
    ...
    if (accept_block)
        txdb.TxnCommit();
    else
        txdb.TxnAbort(); 
It is supposed that the last commit/abort must keep CNameDB in sync too, but for some reason this functionality is broken.
Someone knowledgeful in BDB can probably spot the problem quite easily. There are three places to check:
1) is parent DB functionality implemented properly?
2) is it used properly (as per the code above)?
3) is it used everywhere it should be used (i.e. each time something is written to NameDB)?

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Post by biolizard89 »

libcoin doesn't use BDB, so I think the best way forward is to do the libcoin rebase and make this issue irrelevant.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

snailbrain
Posts: 309
Joined: Tue Jul 19, 2011 9:33 pm

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Post by snailbrain »

yep.. but.. is that going to be a long time?

if someone is familiar with bdb, thecoder thinks they can solve in "minutes"..
so if anyone can help, it will still be "good"...

libcoin (the person) is busy atm.. but his last update (last week)
....busy...... I will soon start to look into adding some of the missing functionality asap, could probably need some help re the gui you have added, but let me have a closer look at it first.
Also, i'm not sure if the consensus has agreed on the rebase/re-implementation on libcoin.. has it?
i'm ok with it.

Eitherway if anyone can see the problem, please let us know :)

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Post by phelix »

I guess this issue is still open. Snailbrain, should we throw a small bounty at it?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

snailbrain
Posts: 309
Joined: Tue Jul 19, 2011 9:33 pm

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Post by snailbrain »

phelix wrote:I guess this issue is still open. Snailbrain, should we throw a small bounty at it?
I'm not sure how far a long domob is with libcoin, but i think it will make the problem obsolete as biolizard said.
I think Libcoin still uses bdb with the wallet (so i read once) but shouldn't be related?

The pool that has been having the issue is Bitminter. It was happening randomly (but not often, possibly every few months).

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Nameindex.dat - Berkleydb - anyone familiar want to help

Post by domob »

snailbrain wrote:
phelix wrote:I guess this issue is still open. Snailbrain, should we throw a small bounty at it?
I'm not sure how far a long domob is with libcoin, but i think it will make the problem obsolete as biolizard said.
I think Libcoin still uses bdb with the wallet (so i read once) but shouldn't be related?

The pool that has been having the issue is Bitminter. It was happening randomly (but not often, possibly every few months).
I've made some progress (name_show is working basically), but am waiting on some answers from Michael at the moment. (I'm not sure where to get the information from to show txid and address of old entries in name_history - not sure if that data is even somewhere available in libcoin.)

Libcoin uses SQLite for the blockchain, but yes, it still uses BDB for the wallet (but I think you can also use a SQLite database for the wallet in case compatibility is not necessary).
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

Post Reply