Namecoin API Server Interface Specification

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Namecoin API Server Interface Specification

Post by phelix »

continued from
https://nf.bit/viewtopic.php?f=18&t=1486
and https://nf.bit/viewtopic.php?p=10680

Suggestions so far have been:

Simple: http://dns.dnschain.net/id/greg

More powerful: https://nameid.org/?action=name_show&name=foobar


IMHO it should resemble namecoind / nmcontrol
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Namecoin API Server Interface Specification

Post by indolering »

I think we should stick to namespace/name. If you want to get into more powerful queries (like full-text searching or faceted search) you will need to implement API limits. I don't think we have the capacity to handle that yet.

However, I think we should use Mashape for the API explorer (which we can embed in the website, mashup with Webshell, and use their client libraries). In the future, if we want to, we can also leverage their platform to handle authenticated requests. You can see a speech.is (d/ only) version here.
DNS is much more than a key->value datastore.

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: Namecoin API Server Interface Specification

Post by phelix »

indolering wrote:I think we should stick to namespace/name. If you want to get into more powerful queries (like full-text searching or faceted search) you will need to implement API limits. I don't think we have the capacity to handle that yet.

However, I think we should use Mashape for the API explorer (which we can embed in the website, mashup with Webshell, and use their client libraries). In the future, if we want to, we can also leverage their platform to handle authenticated requests. You can see a speech.is (d/ only) version here.
yeah. the simple one certainly is a reasonable start.

We should also specify the output.

Code: Select all

{
"name" : "d/nf",
"value" : "{\"map\":{\"\":\"94.23.252.190\"},\"fingerprint\":[\"14:7D:31:8D:52:CD:43:61:32:91:F1:81:1B:C5:B9:CB:7B:25:4C:71\"]}",
"txid" : "11bad911bfacf726932be4a2d8680d9cce6b1c60e0722c79d6bd8653cd7132c5",
"address" : "N7CKB8hEP2YXZ6Zy5xaf8wbHmXLxB2kHk1",
"expires_in" : 31451
}
I suggest json. It must include the "expires_in". Probably it would be nice if it would simply resemble the namecoind output. Anything else?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Namecoin API Server Interface Specification

Post by indolering »

phelix wrote: We should also specify the output.
I think we should push all non-name/value information to meta objects. CouchDB calls these attachments and it enables different MIME types and reduces the amount of stuff that is returned by default.

For example, suppose we want to add cryptographic signing to the records, we would need to store the format and algorithms used to compress the value:

Code: Select all

{
  "name"       : "d/nf",
  "value"      : {
    "map"        : {
      "": "94.23.252.190"
    },
    "fingerprint": ["14:7D:31:8D:52:CD:43:61:32:91:F1:81:1B:C5:B9:CB:7B:25:4C:71"]
  }
  "attachments": {
    "metadata": {
      "txid"      : "11bad911bfacf726932be4a2d8680d9cce6b1c60e0722c79d6bd8653cd7132c5",
      "address"   : "N7CKB8hEP2YXZ6Zy5xaf8wbHmXLxB2kHk1",
      "expires_in": 31451
    },
    "security": {
      "MIME"  : "Some future JSON signing standard",
      "sha256"  : "eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJIUzI1NiJ9",
       ....
    }
  }
}
phelix wrote: It must include the "expires_in". Probably it would be nice if it would simply resemble the namecoind output. Anything else?
I personally think that returning expires_in is a mistake, unless you know the blockcount when the record was retrieved it doesn't tell you much of anything. I think there is a discussion to be had on the mechanics of expiration in general, but if we mandate returning anything in regards to expiration it should be the expiration block itself, not the number of blocks to it.
DNS is much more than a key->value datastore.

sugarpuff
Posts: 110
Joined: Tue Oct 22, 2013 10:17 pm

Re: Namecoin API Server Interface Specification

Post by sugarpuff »

(Thanks indolering for pointing me to this thread.)

As I said to indolering, I'm very appreciative of the idea of collaborating on a technical specification / "standard" of sorts for this, even though I'm skeptical on the outset of reaching any consensus. Thank you phelix, however, for attempting it.

One guiding principle that I try to follow in all of my work is KISS, and all related concepts and corollaries around the idea that "simple is better", and "unnecessary complexity is exactly that." It's against the fire of that benchmark that I hope we can evaluate all proposals.

That said, I want to point out that when I did /[namespace]/[key] I didn't plan on stopping there. It was always my intention to make the API as powerful as possible, while at the same time ensuring that it contains zero extraneous elements.

Let me illustrate what I mean by that by taking this API as an example (slightly modified for accuracy from the original example):

/?action=name_show&name=d%2Ffoobar

That can be represented (as already acknowledged) by a simple: /d/foobar

Why is this better? From the perspective of the API user, it's better simply because it's simpler and much easier to use. The original involves so many extraneous and unnecessary concepts for using it:

- Knowing the RPC name of the namecoind method (why should they care? No reason.)
- Calling the URL with GET parameters and having to form that query string
- URL encoding the request

Complexity with no advantages over simply asking for /d/foobar.

Say we want to do something more complicated, that cannot be expressed with a simple request for /d/foobar.

Say we want to create a new name, the equivalent of calling:

Code: Select all

name_new <name>
Followed by the corresponding:

Code: Select all

name_firstupdate <name> <rand> [<tx>] <value>
Here the namecoind RPC API introduces complexity that we don't need to expose to the users of DNSChain's (or indolering's solution.. not sure what it's called) API.

Notice that we can spare having to make two API calls and simply make it *one* API call. Notice also that there is no reason whatsoever for our users to have to worry about the values of <rand> or <tx>, so there's no reason for us to complicate our API or place that burden on them.

What's the minimum API? The minimum info is:

1. An action. (In this case, a way to indicate that we're creating a new name)
2. The key. (Let's use d/foobar)
3. The value. (Let's just label this as <json-string>)

HTTP already provides us with the means to express (1) because it has, exactly for this purpose, its own verbs.

Thus, the API could be as simple as:

Code: Select all

POST /d/foobar 
<json-string>
Simply create a POST request with the resource URL as the key, and the JSON as the body of the POST message.

The response can be some kind of unique key that confirms that the transaction happened, and that we can use later on to check the status of our registration.

The server, meanwhile, handles *for us* all of the ugly details that are of no importance to us:

1. It calls `name_new`
2. It waits the minimum amount of time necessary to then call `name_firstupdate` with the <json-string> that was passed in, and with the values for <rand> and <tx>.

That's *might* be all that's needed. To create a simple-yet-powerful API we have to start with the simplest possible API, and build it up in the smallest increments necessary to facilitate more powerful features. To do that we need a more complex use-case to explore.

Incremental, minimal complexity

If further complexity is required, we can implement it by building off of this solid and simple foundation. People who want the complexity will be exposed to it, and those who don't want it never have to see or deal with it.

Say we want our server to be notified the *second* that `name_firstupdate` succeeds. For this we'll need:

1. An endpoint for the DNSChain server to call back.
2. (Optional) format of the response with JSON being the default.

We can easily implement this on top of our existing example by taking a page from Twitter's wonderful API, where they have to deal with similar situations. It requires just one small modification:

Code: Select all

POST /d/foobar[.json|.xml]
json=<json-string-urlencoded>&callback=<URL-of-our-callback>
Two notes:

1. The [.json|.xml] indicates an optional extension to specify the format that the callback will receive
2. The one small modification is not that actually, as (1) is backwards compatible with the API. The one modification was the use of URL-encoded parameters to pass in the JSON, as well as all our other parameters (in this case, a callback URL).

More on responses

The discussion above about the response format is also interesting and needs to be address.

Again: what is the simplest, most elegant, and yet powerful way of presenting the response data? What are the various possible approaches?

Above we saw one possible approach: 100% JSON with a mandatory top-level format.

Another possible approach is to store some of that meta data (like response signature) in the HTTP response headers.

Which approach is the best? I can't tell offhand, it would need to be analyzed in detail with a tabulated pros & cons chart.

Some of the pros of ignoring HTTP headers and doing 100% JSON:

- Arguably simpler.
- Arguably more flexible and allows for meta-data to contain large chunks of data (what's the size limit on an HTTP response header?)

Some of the cons of 100% JSON:

- Less readable for human developers / more unwieldy. Headers allow us to focus on the response data itself, instead of the meta data that we (humans) do not care about (that's for the machine to care about).
- Possibly could result in more arguments among standards bodies about what the format actually should be (therefore hinting at a mandatory "version" key in the top-level).

The pros and cons for HTTP headers are basically the inverse of the above. Of course, I'm probably missing some, so please feel free to think about this and add your own! :)

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Namecoin API Server Interface Specification

Post by indolering »

For as much as Sug and I disagree, we have nearly the same opinion on most API issues. I think this discussion is scoped to a web-facing API, which leaves out any commands which may write information. However, while the name_new example is a write command, it still largely applies: they only asked for the record, don't give them anything they didn't ask for.

Namecoind's interface is actually rather poorly constructed, we really should be building a layer of abstraction around it's weirdness if not fixing the API outright. That might even be a discussion worth having before we spend a bunch of energy reimplementing it in Libcoin....

The only information where this specific KISS might not apply is for verification related information. I like the Secure Messaging standard, but whatever the standard, I firmly believe we should have some sort of public key info as a mandatory part of the spec for any non-SPV/UTXO/SCIP clients.
sugarpuff wrote:
Some of the pros of ignoring HTTP headers and doing 100% JSON:

- Arguably simpler.
- Arguably more flexible and allows for meta-data to contain large chunks of data (what's the size limit on an HTTP response header?)

Some of the cons of 100% JSON:

- Less readable for human developers / more unwieldy. Headers allow us to focus on the response data itself, instead of the meta data that we (humans) do not care about (that's for the machine to care about).
- Possibly could result in more arguments among standards bodies about what the format actually should be (therefore hinting at a mandatory "version" key in the top-level).

The pros and cons for HTTP headers are basically the inverse of the above. Of course, I'm probably missing some, so please feel free to think about this and add your own! :)
I think the problem is the assumption that the client software always have access to those HTTP headers. If you want to cache a request locally, then you will have to store whatever you put into the HTTP headers locally as well. You might as well just stuff it into the returned JSON.

The one certainty with any specification is the need for non-standard extensions. We will need information that won't fit into the headers, so why not just come up with a standard place to put it, like an "attachment" catch-all? "inline" attachments would give users a place to stick non-standard information into their domain record. We can define the stuff we need (like a signature attachment) and leave the rest of the namespace to whatever people come up with. Then we are free to cherry pick the best ideas for future versions of the standard.
DNS is much more than a key->value datastore.

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: Namecoin API Server Interface Specification

Post by phelix »

It seems that if we can agree on a specification for name_show we will have achieved something so let's stick to that for a start.
I think we should push all non-name/value information to meta objects. CouchDB calls these attachments and it enables different MIME types and reduces the amount of stuff that is returned by default.
This sounds and looks much more complicated...
I personally think that returning expires_in is a mistake, unless you know the blockcount when the record was retrieved it doesn't tell you much of anything. I think there is a discussion to be had on the mechanics of expiration in general, but if we mandate returning anything in regards to expiration it should be the expiration block itself, not the number of blocks to it.
I expected the expires_in to be up to date. :) I think this goes along well with sugarpuff's design philosophy.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Namecoin API Server Interface Specification

Post by domob »

phelix wrote:It seems that if we can agree on a specification for name_show we will have achieved something so let's stick to that for a start.
I think we should push all non-name/value information to meta objects. CouchDB calls these attachments and it enables different MIME types and reduces the amount of stuff that is returned by default.
This sounds and looks much more complicated...
I agree. I think that the current name_show format is fine also for the API. Especially since if the format stays the same, client applications can easily connect both to the API server and a local namecoind, depending on user preference. (I know that the changes required to support two slightly different JSON formats are also minor, but why not make them exactly the same?) If we really feel that the current format is not good enough, we should discuss changes to name_show, too. (But I don't see the need to do this, TBH.)

What I'd be more interested in is defining a format for querying the API - i. e., which HTTP POST/GET requests to send to some API URL to retrieve the name_show output as result. (And what the server responds to for errors like "name doesn't exist".) If this was defined, I could easily implement this in PHP also on nameid.org and in client applications (like Bitmessage or other things, even FreeSpeechMe could optionally use this). When that is done, Namecoin will be easily usable for people who don't require full security but just want to play around a little.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

sugarpuff
Posts: 110
Joined: Tue Oct 22, 2013 10:17 pm

Re: Namecoin API Server Interface Specification

Post by sugarpuff »

domob wrote:What I'd be more interested in is defining a format for querying the API - i. e., which HTTP POST/GET requests to send to some API URL to retrieve the name_show output as result. (And what the server responds to for errors like "name doesn't exist".) If this was defined, I could easily implement this in PHP also on nameid.org and in client applications (like Bitmessage or other things, even FreeSpeechMe could optionally use this). When that is done, Namecoin will be easily usable for people who don't require full security but just want to play around a little.
Isn't this part of what's being discussed (in addition to the response format)?
domob wrote:I think that the current name_show format is fine also for the API. Especially since if the format stays the same, client applications can easily connect both to the API server and a local namecoind, depending on user preference. (I know that the changes required to support two slightly different JSON formats are also minor, but why not make them exactly the same?) If we really feel that the current format is not good enough, we should discuss changes to name_show, too. (But I don't see the need to do this, TBH.)
Chalk that up to one of the pros of using headers for signatures (or modifying namecoind to include them).
indolering wrote:I think the problem is the assumption that the client software always have access to those HTTP headers. If you want to cache a request locally, then you will have to store whatever you put into the HTTP headers locally as well. You might as well just stuff it into the returned JSON.
That's a great point indolering! That's definitely a pro to having the JSON format include the signature (and other attachments), either by modifying namecoind or having the blockchain proxy-API server do it.

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Namecoin API Server Interface Specification

Post by indolering »

phelix wrote:It seems that if we can agree on a specification for name_show we will have achieved something so let's stick to that for a start.
I think we should push all non-name/value information to meta objects. CouchDB calls these attachments and it enables different MIME types and reduces the amount of stuff that is returned by default.
This sounds and looks much more complicated...
I think the extra information returned makes this more complex, not the format in which it is packaged. It should just be key->value.
phelix wrote:
I personally think that returning expires_in is a mistake, unless you know the blockcount when the record was retrieved it doesn't tell you much of anything. I think there is a discussion to be had on the mechanics of expiration in general, but if we mandate returning anything in regards to expiration it should be the expiration block itself, not the number of blocks to it.
I expected the expires_in to be up to date. :) I think this goes along well with sugarpuff's design philosophy.
The Tao, which is the philosophy with which sugarpuff has expressed a preference, is said to embody nothingness. From the application's perspective, the expires_in value is the opposite of simplicity: if a site has expired then it shouldn't return anything : )

When an application looks up the record it received using name_show in its cache, it will see that the record expires in X blocks from when the record was originally placed into the cache. Client applications without access to the entire blockchain stored locally (such as lite-weight DNS clients) have NO use for this number.

Even for blockchain aware applications, the count of blocks until the record expires from some date in the past is almost entirely useless information! It will still need to call Namecoind to ask if it has expired. At best, including the expiration block saves you a few calls to getblockcount, but not much more than that.

Creating a usable expiration date is what lead me to suggest a "grace period" during which the domain would still be renewable but would stop resolving in client applications. This would give us the ability to give applications a hard date as to when it should stop resolving that was related to but not dependent upon the absolute block-count.
domob wrote: I agree. I think that the current name_show format is fine also for the API. Especially since if the format stays the same, client applications can easily connect both to the API server and a local namecoind, depending on user preference. (I know that the changes required to support two slightly different JSON formats are also minor, but why not make them exactly the same?) If we really feel that the current format is not good enough, we should discuss changes to name_show, too. (But I don't see the need to do this, TBH.)
Well, the REST API already breaks compatibility with name_show as we are abstracting the hard URL (namespace/name). So we might as well have this discussion : )

I will freely admit to being naïve when it comes to the core namecoin implementation, however, from a client-side perspective, the namecoind API is an absolute bear. It's overly verbose, it's VERY command-line centric, and it exposes a bunch of low-level, blockchain related information that should be abstracted away at the client level. Take name_filter:

Code: Select all

namecoind name_filter [regexp] [maxage=36000] [from=0] [nb=0] [stat] 
This kind-of makes sense from a command line point of view, if you can remember what "nb" means. But here is the equivalent JSON-RPC based function call:

Code: Select all

name_filter(regexp, 36000, 0, 0, 'stat')
As an application programmer, I don't care what the expiration date is, I just want valid, non-expired domains returned! But this format forces me to A: keep up to date with the correct expiration length and B: rewrite my function if that date ever changes. Then there is 'stat' which should be a boolean value, if not an entirely separate function.

Measures of complexity are dependent upon the context in which they are measured. In the context I am most familiar with, the blocks until a domain expires, the transaction ID, and the Namecoin wallet address the domain belongs to -catches breath- is of no value to anyone except programs which are interested in data specific to the blockchain. name_show should be replaced with something like lookup(name, namespace) as the interface for record lookups. Within the actual class, it should be replaced with dns.lookup(name) or id.lookip(name). If the application is trusting a third party (and not pulling the data off of the blockchain or using UXTO) then the data must be signed.
domob wrote: What I'd be more interested in is defining a format for querying the API - i. e., which HTTP POST/GET requests to send to some API URL to retrieve the name_show output as result. (And what the server responds to for errors like "name doesn't exist".) If this was defined, I could easily implement this in PHP also on nameid.org and in client applications (like Bitmessage or other things, even FreeSpeechMe could optionally use this). When that is done, Namecoin will be easily usable for people who don't require full security but just want to play around a little.
I'm enthusiastic about the first part of that statement, I want an API that enables simple client-side applications and fosters experimentation. However, I'm willing to eat crow on everything else I wrote on this post, but I won't contribute to any API which does not include security information by default : /

But that's a pretty sour note to end on! : )

I really do think we have the same goal in mind, just different perspectives. And from my perspective, there should be separate parameters for developers who want information beyond the record value. That's what the blockexplorer is for anyway, and I got 0 traction when I suggested we consolidate the two APIs.
DNS is much more than a key->value datastore.

Post Reply