Module Request: Keep a Namespace in Sync
Forum rules
Warning !
Avoid using binary softwares from untrusted users.
Prefer compiling it yourself and verify sources.
Warning !
Avoid using binary softwares from untrusted users.
Prefer compiling it yourself and verify sources.
Re: Module Request: Keep a Namespace in Sync
I wrote a patch for Namecoin Core which adds a name_sync command, which is used to implement namesync.
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync
Re: Module Request: Keep a Namespace in Sync
Hit! Once again I am impressed by the thorough specification and general sophistication of your work. Sorry this was wasted on me earlier, I am mostly looking at the things on the forum with their own topic.hla wrote:I wrote a patch for Namecoin Core which adds a name_sync command, which is used to implement namesync.
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync
I was thinking about a solution without changes to the client but your solution looks more efficient and elegant. What are the advantages vs. a polling/blocknotify solution besides not having to parse blocks (which is great)?
The changes are almost completely within the namecoin specific files of the codebase so should we merge this into Namecoin Core? Domob?
Digging out previous discussions:
Any progress on that?# from namecoin-dev meeting notes, 2015 02 21
Hugo proposes a name_sync RPC command for allowing external databases to sync with Namecoin Core's name database.
https://github.com/hlandau/namesync.t/b ... me_sync.md
Daniel suggests that this would potentially be useful for Bitcoin as well, maybe merge upstream.
Daniel mentions that Bitcoin Core supports a notify command feature for new blocks
Jeremy suggests that NMControl should add support for this, so that the name cache refresh rate is more intelligent than every 30 minutes.
Jeremy suggests a namenotify command
Hugo says that a transactional method is more robust than the current Bitcoin Core notify commands.
Hugo will write a proposal for Bitcoin Core.
Re: Module Request: Keep a Namespace in Sync
Sorry, I never attended to any of that. :/
I will confess I have a dislike of polling probably to an almost irrational degree. It just exudes inefficiency to me. No programmer could defend putting a sleep() statement in a program to fix a bug occurring due to a race condition except as the most desparate, last-minute hack... polling is basically the network equivalent. I don't see an issue with my long-polling system though, even if long polling is a bit of a hack from an HTTP perspective. It's not the first RPC command to use it.
I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.
The principal weakness of name_sync I think, besides the fixed rollback capacity (not really an issue since we're assuming there will never be more than 12 blocks rolled back anyway) is that after a rollback, as mentioned in the spec, the name database is patched with current names rather than names for the current synced height. This is a pretty small weakness, and given how quickly namesync syncs (as far as I can remember), it's basically an academic consideration, I think.
I will confess I have a dislike of polling probably to an almost irrational degree. It just exudes inefficiency to me. No programmer could defend putting a sleep() statement in a program to fix a bug occurring due to a race condition except as the most desparate, last-minute hack... polling is basically the network equivalent. I don't see an issue with my long-polling system though, even if long polling is a bit of a hack from an HTTP perspective. It's not the first RPC command to use it.
I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.
The principal weakness of name_sync I think, besides the fixed rollback capacity (not really an issue since we're assuming there will never be more than 12 blocks rolled back anyway) is that after a rollback, as mentioned in the spec, the name database is patched with current names rather than names for the current synced height. This is a pretty small weakness, and given how quickly namesync syncs (as far as I can remember), it's basically an academic consideration, I think.
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Module Request: Keep a Namespace in Sync
I think there have been some changes to upstream Bitcoin's notify stuff, including ZeroMQ support. See https://github.com/bitcoin/bitcoin/blob ... doc/zmq.mdhla wrote:I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.
This seems to address your efficiency and security concerns. Do you have concerns about using Bitcoin's ZeroMQ for name notifications?
Re: Module Request: Keep a Namespace in Sync
That seems like a one-way notification interface, not a two-way synchronization interface. So it can't really satisfy the use case of name_sync, since it can't know the state of the synchronized database and thus know what to send. It would only work if the synchronization daemon was attached to namecoind from the beginning, as it synchronized, etc. and the mechanism never failed.
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Module Request: Keep a Namespace in Sync
Well, if the goal is to stay synchronized to the name database, the client could use name_scan to get the current state, and then use ZeroMQ to stay synced after that.hla wrote:That seems like a one-way notification interface, not a two-way synchronization interface. So it can't really satisfy the use case of name_sync, since it can't know the state of the synchronized database and thus know what to send. It would only work if the synchronization daemon was attached to namecoind from the beginning, as it synchronized, etc. and the mechanism never failed.
That said, I agree with Daniel that while an explicit sync interface might have some useful use cases, it should probably be merged upstream in Bitcoin.
Re: Module Request: Keep a Namespace in Sync
How does that work with block rollbacks? And you're introducing more moving parts there, making the whole thing more fragile.
The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...
The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Module Request: Keep a Namespace in Sync
I have no objection to Bitcoin merging a sync patch. It would be useful for some use cases. Feel free to talk to the Bitcoin Core devs about it.hla wrote:How does that work with block rollbacks? And you're introducing more moving parts there, making the whole thing more fragile.
The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...
Re: Module Request: Keep a Namespace in Sync
That makes no sense. The patch only exports name data. It's useless for a blockchain without names.
Re: Module Request: Keep a Namespace in Sync
This is true, but it would be useful for UTXO's, for instance.hla wrote:That makes no sense. The patch only exports name data. It's useless for a blockchain without names.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/