RESTFul API actions

pty 882 Posting Pro · 2017-01-04T11:19:58+00:00

Perhaps what I am doing, however, is creating a new membership record?

This is correct, but when you're posting to /groups/memberships, to which group are you adding a member?

I'd expect to see something more like this. Of course, you don't need all of these actions (i.e. updating a membership, which is probably just a record in a link table, might not be necessary) but it's obvious at every point what each action does.

GET    /groups/                                     get a list of all groups
POST   /groups/                                     create a new group
GET    /groups/:group_id                            get an individual group :group_id
PUT    /groups/:group_id                            replace group :group_id
PATCH  /groups/:group_id                            modify group :group_id
DELETE /groups/:group_id                            delete group :group_id

GET    /groups/:group_id/memberships                get a list of members for group :group_id
POST   /groups/:group_id/memberships                create a new membership for the group (i.e. add a member)
GET    /groups/:group_id/memberships/:member_id     get an individual membership :member_id for group :group_id
PUT    /groups/:group_id/memberships/:member_id     replace membership :member_id
PATCH  /groups/:group_id/memberships/:member_id     modify membership :member_id
DELETE /groups/:group_id/memberships/:member_id     remove membership :member_id from group :group_id

Also, it would appear that your Markdown implementation doesn't support tables! :)

pty 882 Posting Pro · 2017-01-04T15:42:36+00:00

Incidentally, in my earlier post, I used Rails-esque route notation :group_id and :membership_id. In case it's not obvious, that's where your identifiers would go, so the actual path would be /groups/1/memberships/3 (or if you prefer slugs, /groups/admin/memberships/joey)

Dani · 2017-01-04T20:23:58+00:00

My apologies! I accidentally left out the :id from the URI in my question. I had meant to include it.

Yes, we purposefully don't support tables.

pty 882 Posting Pro · 2017-01-04T23:35:52+00:00

Yes, we purposefully don't support tables.

Ah, what's the reasoning behind that? It doesn't need to be provided by the editor, but in some cases it makes a post much easier to read.

Dani · 2017-01-04T23:50:10+00:00

There was a reason back when we first switched to Markdown from BBCode and I forget specifically what it is. I think it either had something to do with backwards compatibility or it fubar'ing up too many posts. I think it might also be related to the Markdown parser we were using at the time. It was so many years ago, I unfortunately don't recall. And this is the first time in all those years that anyone's ever missed it.

jkon 636 Posting Whiz in Training Featured Poster · 2017-01-11T12:04:27+00:00

Dani what are you really doing ? Where is the WSDL that will describe those services ?

Dani · 2017-01-18T09:58:01+00:00

I'm not sure what wsdl means but there is a little question mark help button in the editor toolbar that leads to a documentation page for our flavor of markdown.

cereal 1,524 Nearly a Senior Poster Featured Poster · 2017-01-18T10:55:04+00:00

Hi!

I think jkon refers to this https://www.w3.org/TR/wsdl20/
i.e. a document that describes the features of the API endpoints. This is widely used in SOAP. See also:

http://www.ibm.com/developerworks/webservices/library/ws-restwsdl/

pty 882 Posting Pro · 2017-01-18T12:44:46+00:00

To clarify, I think @jkon was making a joke. SOAP was a nightmare at the best of times and unusable at the worst, and I'm glad I've not had to deal with WSDL in years.

Keeping an interface simple, RESTful and documenting it with Swagger or Stoplight is a much better idea.

Dani · 2017-01-18T20:13:16+00:00

I'm working on Swagger documentation for Dazah as we speak. Should be done within the next half hour or so.

Dani · 2017-01-18T20:14:26+00:00

Oh, I just want to point out that my post last night was in bed from my iPhone, and I hadn't scrolled up enough in the conversation to realize he was talking about API documentation and not markdown documentation. Oopsies.

Dani · 2017-01-19T11:01:39+00:00

OK, so now that we've launched, I was wondering if you guys could check it out and give it a lookover and let me know if it's RESTful.

https://www.dazah.com/developers/reference

There are links at the bottom to the previous versions so you can see the progress towards restfulness.

Suggestions appreciated!!

pty 882 Posting Pro · 2017-01-23T23:13:13+00:00

For the most part I'd say it looks ok.

The tilde in some of the currently-OAuthed user routes doesn't 'feel' right to me, though. The current user (or you), and the current user's CV (your CV) are both singular resources; you already know their ID if they are authenticated so you needn't treat them as if they are a collection:

GET  profile
POST profile
GET  profile/cv
POST profile/cv

Dani · 2017-01-24T06:48:36+00:00

you already know their ID if they are authenticated so you needn't treat them as if they are a collection

Thank you sooo much for looking it all over for me! I kinda sorta understand what you mean, but can you clarify a little bit?

We are treating the tilde as a wildcard character for whomever we happen to be. Therefore, while GET users/1 would get a collection with only one element, user 1, while, GET users/~ would get us. The difference, I guess, is that users/1 returns the results as a collection while users/~ returns a singular resource, us. Is that what you were referring to?

Dani · 2017-01-24T06:59:50+00:00

OK, so you uncovered something else that I feel needs fixing.

I actually prefer GET me as opposed to GET profile. However, as our scopes are profile_read and profile_write, then using profile over me probably makes more semantic sense.

That being said, the user schema contains an object called profile. That would not make sense ... a profile contained within a profile?! I want to rename this profile component but I'm not quite sure what to.

pty 882 Posting Pro · 2017-01-24T10:05:21+00:00

Incidentally, in my previous post the profile's POST line should have been a PUT or PATCH; you'd assume that every user has one and shouldn't be able to create another.

The difference, I guess, is that users/1 returns the results as a collection while users/~ returns a singular resource, us. Is that what you were referring to?

Not exactly. /users/~, or /users/{my_id} is fine for accessing my profile page, but it's also the way you'd access anyone else's profile page.

This approach means you'd need additional logic in the views (and checks in the controller) to display things that only a user can do to themselves, such as modify personal details, update profile picture, view their own event log, etc.

This makes the functionality more difficult to write, test and maintain, because the action isn't specific.

Splitting it out makes both sides simpler.

Also, I agree, /me sounds better than /profile, but /me/cv sounds odd. What is the profile component? Does a user only have one?

There are only two hard things in Computer Science: cache invalidation and naming things. - Phil Karlton

Dani · 2017-01-24T10:18:30+00:00

Regarding PUT and PATCH, I looked and looked, and was unable to find a working tutorial on how to implement them in Nginx. Everything I read said that Nginx does not have support??

Not exactly. /users/~, or /users/{my_id} is fine for accessing my profile page, but it's also the way you'd access anyone else's profile page.

Sorry, I'm not understanding you there. /users/~ returns a single object, my own User. /users/{comma-delimited-ids} returns an array of one or more user objects.

What is the profile component? Does a user only have one?

{
  "data": {
    "id": 0,
    "location": {
      "city": "string",
      "country": "string",
      "ip_address": "string",
      "latitude": 0,
      "longitude": 0,
      "region": "string"
    },
    "matching": {
      "goals": [
        "string"
      ],
      "location_importance": "string",
      "targeted_industry": "string"
    },
    "personal": {
      "birthday": "string",
      "gender": "string",
      "relationship_status": "string"
    },
    "picture": "string",
    "profile": {
      "first_name": "string",
      "headline": "string",
      "industry": "string",
      "introduction": "string",
      "last_name": "string",
      "pitch": "string",
      "website": {
        "thumbshot": "string",
        "url": "string"
      }
    },
    "settings": {
      "email": "string",
      "email_verified": true,
      "notifications": "string",
      "timezone": 0
    },
    "thumbnail": "string",
    "usage": {
      "available_status": true,
      "joined_timestamp": "2017-01-24T10:16:25.041Z",
      "last_activity_timestamp": "2017-01-24T10:16:25.041Z",
      "online_status": true
    }
  }
}

This approach means you'd need additional logic in the views (and checks in the controller) to display things that only a user can do to themselves, such as modify personal details, update profile picture, view their own event log, etc.
This makes the functionality more difficult to write, test and maintain, because the action isn't specific.

I'm really sorry ... I'm not understanding you? If you use the GET/POST /users/~ endpoints, then you're performing on yourself. I'm sorry, I'm not understanding what you mean?

Dani · 2017-01-24T10:31:20+00:00

Let me try to explain myself better, because I'm thoroughly confused by your concern.

This is what is returned when you call GET /users/~ ... You can see that it is a single object about you, and it includes private information such as your email address and settings.

{
  "data": {
    "id": 0,
    "location": {
      "city": "string",
      "country": "string",
      "ip_address": "string",
      "latitude": 0,
      "longitude": 0,
      "region": "string"
    },
    "matching": {
      "goals": [
        "string"
      ],
      "location_importance": "string",
      "targeted_industry": "string"
    },
    "personal": {
      "birthday": "string",
      "gender": "string",
      "relationship_status": "string"
    },
    "picture": "string",
    "profile": {
      "first_name": "string",
      "headline": "string",
      "industry": "string",
      "introduction": "string",
      "last_name": "string",
      "pitch": "string",
      "website": {
        "thumbshot": "string",
        "url": "string"
      }
    },
    "settings": {
      "email": "string",
      "email_verified": true,
      "notifications": "string",
      "timezone": 0
    },
    "thumbnail": "string",
    "usage": {
      "available_status": true,
      "joined_timestamp": "2017-01-24T10:16:25.041Z",
      "last_activity_timestamp": "2017-01-24T10:16:25.041Z",
      "online_status": true
    }
  }
}

This is what is returned when you call /users/{IDS} ... You can see it is an array of one or more User objects, none of which contain any private information.

{
  "data": [
    {
      "id": 0,
      "location": {
        "city": "string",
        "country": "string",
        "region": "string"
      },
      "picture": "string",
      "profile": {
        "first_name": "string",
        "headline": "string",
        "industry": "string",
        "last_name": "string",
        "pitch": "string",
        "website": {
          "thumbshot": "string",
          "url": "string"
        }
      },
      "thumbnail": "string",
      "usage": {
        "available_status": true,
        "joined_timestamp": "2017-01-24T10:28:16.623Z",
        "last_activity_timestamp": "2017-01-24T10:28:16.623Z",
        "online_status": true
      }
    }
  ]
}

I'm incredibly confused what logic you're reffering to??

pty 882 Posting Pro · 2017-01-24T13:19:18+00:00

Sorry if I didn't make it clear. When I refer to 'additional logic', I mean checking that the API's response is suitable for the user making the request.

There are two scenarios in your example, generally listing and viewing user profiles and viewing and modifying your own profile. It makes sense from an API point of view and a code-organisation point of view to split these between two controllers. Requests to /users/20 and /users/~ will be routed to the same place (by every router I've worked with, at least!), because the paths match; the only difference is the parameter on the end (20 vs ~). This means you'd need additional code to cope with the params and react accordingly.

To attempt to clarify what I mean, here's a quick example of how a single controller that's trying to perform both roles might look. Note how inside #update and #show, there's additional logic that makes the controller more difficult to understand because it no longer has a single responsibility.

If we split that functionality into two, we have two simple controllers; the user controller that only provides allowed actions and the profile controller that only has access to the currently logged-in user.

If you're also building a web interface on top of (or to compliment) this API, it makes even more sense, because you want to serve a different page for someone visiting their own profile vs visiting someone elses. Again, it's simpler and cleaner to do this with separate views than to clutter a single template with this kind of logic:

if user is me
    show 'edit' button
else
   show 'send message' button

Let me know if it's still confusing!

pty 882 Posting Pro · 2017-01-24T14:17:33+00:00

Regarding PUT and PATCH, I looked and looked, and was unable to find a working tutorial on how to implement them in Nginx. Everything I read said that Nginx does not have support??

NGINX definitely supports them, as far as I know you don't need to enable anything. It should just work.

I just wrote (well, copied, pasted and amended) a tiny app in PHP (the first time I've ever tried PHP!) and it handled PUT like a boss.

Dani · 2017-01-24T20:51:02+00:00

Requests to /users/20 and /users/~ will be routed to the same place (by every router I've worked with, at least!), because the paths match; the only difference is the parameter on the end (20 vs ~). This means you'd need additional code to cope with the params and react accordingly.

Do you mean additional code on MY side of things (as the API provider) and not on the consumer's side of things? Our API is live and functional and being used by DaniWeb, and there was no additional logic needed. We aren't routing those two URIs to the same place and there was no conditional logic in the controllers needed. (We use CodeIgniter PHP framework and I'm happy with its URI routing functionality).

I guess my question to you is, from a routing perspective, why would /users/([0-9]+) (essentially users/ followed by a positive integer) be confused with /users/~ (users/ followed by a tilde), but not be confused with /users/searches or /users/nearby??

My concern is from the perspective of an API consumer ... will our existing URI structure add an unnecessary burden from their side of things?

... And regarding Nginx, if you were able to get Nginx to work with PUT and PATCH, please let me know how! Whenever I try, Nginx short circuits and returns back a status of 501 not implemented and with a message body of "This method may not be used."

From my understanding, you can compile Nginx with a module to override this, and enable PUT, PATCH, and DELETE but when doing so, Nginx again short circuits PHP and actually PUTS/DELETEs files in the file system!

cereal 1,524 Nearly a Senior Poster Featured Poster · 2017-01-24T21:51:55+00:00

@Dani

... And regarding Nginx, if you were able to get Nginx to work with PUT and PATCH, please let me know how! Whenever I try, Nginx short circuits and returns back a status of 501 not implemented and with a message body of "This method may not be used."

From my understanding, you can compile Nginx with a module to override this, and enable PUT, PATCH, and DELETE but when doing so, Nginx again short circuits PHP and actually PUTS/DELETEs files in the file system!

I have tried that and, yes, it works like in your description but only if webdav is enabled for that location. Otherwise it works like in pty's example.

For a basic test try:

<?php

$stream = [];
$method = $_SERVER['REQUEST_METHOD'];

parse_str(file_get_contents('php://input'), $stream);

print "Method $method" . PHP_EOL;
print print_r($stream, TRUE) . PHP_EOL;

And then send requests:

http --form PUT http://site/script.php msg="hello" --verbose

And you should see something like:

PUT /a.php HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 9
Content-Type: application/x-www-form-urlencoded; charset=utf-8
User-Agent: HTTPie/0.9.2

msg=hello

HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Type: text/html; charset=UTF-8
Server: nginx/1.6.2
Transfer-Encoding: chunked

Method PUT
Array
(
    [msg] => hello
)

pty 882 Posting Pro · 2017-01-25T10:14:45+00:00

I guess my question to you is, from a routing perspective, why would /users/([0-9]+) (essentially users/ followed by a positive integer) be confused with /users/~ (users/ followed by a tilde), but not be confused with /users/searches or /users/nearby??

I'd say in a 'pure' world, yes. If, as a user, I can only update their own profile (and not anyone else's), and they only have one of them, why is /users in the path at all? Also, Unix folk will know that ~ is 'home', but other people wont. The URL, which is a legitimate part of the UI, isn't as descriptive or friendly as it could be.

It also means that you need to pay extra attention to the order in which your routes are applied (same applies to /users/searches and /users/nearby). This can be a bigger headache if you're using slugs; what happens if a user called 'searches' registers? It's very simple with a few (or a few dozen) endpoints. I've seen some monstrosities in the past and ensuring the ordering is correct becomes very complicated indeed.

There's no right and wrong here. If you're happy with the /users/~ route and there's nothing that will have an impact on its functionality, go with it. From the consumer point of view, so long as it's obvious what's going on and there's some documentation (or better yet, a client library that does it for you), it should be fine.

pty 882 Posting Pro · 2017-01-26T10:09:56+00:00

Regarding PUT and PATCH, I looked and looked, and was unable to find a working tutorial on how to implement them in Nginx. Everything I read said that Nginx does not have support??

Hmm, @cereal's script works for me using NGINX 1.10.2 with a pretty standard config and PHP 7.0.14

http --form PUT http://localhost:8080/test.php msg="hello" --verbose
PUT /test.php HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 9
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:8080
User-Agent: HTTPie/0.9.2

msg=hello

HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/html; charset=UTF-8
Date: Thu, 26 Jan 2017 10:07:03 GMT
Host: localhost:9000
Server: nginx/1.10.2
Transfer-Encoding: chunked
X-Powered-By: PHP/7.0.14

Method PUT
Array
(
    [msg] => hello
)

Dani · 2017-01-27T21:40:07+00:00

I actually imitated the /users/~ endpoint after LinkedIn's API, which does:

https://api.linkedin.com/v1/people/123 to retrieve User #123 and
https://api.linkedin.com/v1/people/~ to retrieve yourself

Since Dazah functions primarily as a lead generation API, and LinkedIn is the most popular other lead generation API out there, I decided that our API consumers would be most likely familiar with LinkedIn's API, and the concept of users/123 and users/~ wouldn't be a foreign concept to them.

Dani · 2017-02-10T23:33:03+00:00

It turned out that the load balancer was blocking non-GET/POST requests. It wasn't an Nginx problem after all!

pty 882 Posting Pro · 2017-02-11T17:17:43+00:00

It turned out that the load balancer was blocking non-GET/POST requests. It wasn't an Nginx problem after all!

This doesn't suprise me at all! :)

I actually imitated the /users/~ endpoint after LinkedIn's API, which does:

Yes, but LinkedIn's API is read only.

If Dazah's API allows you to update your profile then it definitely makes sense to split it. Allowing PUT or PATCH to work if the :user_id param is ~ but not if it's anything else adds inconsistency to the API and will add to the complexity either in your code or in your documentation.

Also, because the returned data probably differs despite having the same endpoint (as you expect to see a bit more data on your own profile compared to someone else's), the API in general is less predictable than if you used two disparate endpoints for interacting with profiles generally and viewing/modifying your own.

diafol · 2017-02-11T20:20:31+00:00

I've been following this with some interest. Bad practice to end with a verb? I noticed that several frameworks suggest this, e.g. (domain)/users/23/edit - so this is not acceptable?

Dani · 2017-02-11T22:13:54+00:00

Yes, but LinkedIn's API is read only.

Not entirely, no. You can share links on LinkedIn, so that's a write activity. I feel like they might have other non-public write activities as well, since they locked down their public API, but I don't recall what they are.

If Dazah's API allows you to update your profile then it definitely makes sense to split it. Allowing PUT or PATCH to work if the :user_id param is ~ but not if it's anything else adds inconsistency to the API and will add to the complexity either in your code or in your documentation.

Essentially that is what I'm dong, but I'm treating users/:id as a separate endpoint to users/~. The first only allows GET while the latter allows GET and PATCH.

While I really do like /api/me, the reason why I think I've settled on /api/users/~ is because it returns a user object, so I feel like it should be under the users path. I'm still a little confused about how to treat /api/positions now though. You can retrieve the positions from any user but you can only add or modify positions for the O'Auth'ed end-user.

Bad practice to end with a verb? I noticed that several frameworks suggest this, e.g. (domain)/users/23/edit - so this is not acceptable?

Yes, to be RESTful, that is not acceptable. If you check out our documentation here:

https://www.dazah.com/developers/reference/2
https://www.dazah.com/developers/reference/3

you will see that version 3 adheres to these practices while version 2 does not. The idea is that every resource is a noun describing the object, and then the type of HTTP request determines the action to be taken on the noun. So, in other words, instead of the endpoint name defining the action, the HTTP request type defines the action. The URI defines the noun (the object being acted upon), and then the HTTP request is either GET to retrieve, POST to create, PUT to replace, PATCH to modify, or DELETE to remove. It's very cruddy.