Category Archives: PHP

Documenting APIs using Apigility

One of the cool feature of Apigility is the ability to generate API documentation using a simple UI. The documentation is generated in HTML format, and optionally in Swagger format. The API documentation is reported in Apigility in the top bar, under the menu “API Docs” (Figure 1, using Apigility 1.0.0beta1).

API documentationFigure 1

In order to generate the API documentation you need to insert some desciptions before. All the information to edit are reported in the Documentation tab on each REST or RPC service (Figure 2).

REST API documentationFigure 2

For each service and for each HTTP method, you can specify a description of the action. In case of RESTful services you can also specify different information for an Entity and a Collection. An interesting feature of the API documentation is the ability to generate the Response Body specification from the configuration, using the “generate from configuration” button (Figure 3).

Generate from documentationFigure 3

This button read the configuration of the API and propose a JSON response based on the fields specified (the fields are documented under the Fields tab of each REST and RPC service). Of course, you can edit the response body changing the output, if you need.

Once you have added some API descriptions, you can go to the “API Docs” menu and show the API documentation (in our case version 1, Figure 4).

API documentation in HTML formatFigure 4

You will see all the API documentation in HTML format, using the Bootstrap 3 template.
You can expand and collapse the information on each HTTP method clicking on the name. All the API documentation are exposed in the /apigility/documentation base URL.

How to install the Swagger adapter

To activate the Swagger adapter for the API documentation, you need to add the following dependency in the composer.json file (in the require field):

"zfcampus/zf-apigility-documentation-swagger": "~1.0-dev"

and execute the composer update commmand.

After the installation of zf-apigility-documentation-swagger you need to enable this module in the config/application.config.php file. You have to edit this configuration file and add the following line after the ‘ZF\Apigility\Documentation’:

'ZF\Apigility\Documentation\Swagger',

Now you can go to the Swagger documentation from the welcome screen, clicking on the Swagger API documentation button, or going directly to the /apigility/swagger URL.
To show the Swagger UI render you have to select the API service version and you will see a web page like the one reported in Figure 5 (using Swagger UI).

API documentation in Swagger formatFigure 5

Customizing the API documentation module

The API documentation feature is offered by Apigility using the zf-apigility-documentation module, written in Zend Framework 2. This module provide an object model of all captured documentation information, including:

  • All APIs available
  • All Services available in each API
  • All Operations available in each API
  • All required/expected Accept and Content-Type request headers, and expected Content-Type response header, for each available API Service Operation.
  • All configured fields for each service

Moreover, it provides a configurable MVC endpoint for returning documentation

  • documentation will be delivered in a serialized JSON structure by default
  • end-users may configure alternate/additional formats via content-negotiation

If you want to customize the format of your API documentation you can have a look at the source code of the zf-apigility-documentation-swagger module. Basically, you need to create a custom route for your format (see the Swagger module.config.php) and use the ZF\Apigility\Documentation\ApiFactory to access the data for the API documentation services. The view model to implement needs to manage a list view and a show view, that’s it.

All the API documentation formats are driven by content negotiation (using the zf-content-negotiation module). For instance, to get the API documentation data in Swagger format you can use the content negotiation “application/vnd.swagger+json”.

For example, if you want to retrieve the API documentation data in JSON format you can use the following request (using HTTPie):

http GET http://localhost:8888/apigility/documentation[/api]/[service] 'Accept:application/json'

where [api] is the name of the API and [service] is the name of the REST or RPC service. To get the same result in Swagger format you can use the following request:

http GET http://localhost:8888/apigility/documentation/[api]/[service] 'Accept:application/vnd.swagger+json'

API authentication in PHP with OAuth2 using Apigility

OAuth2 and Apigility In this post, I show how to build an API authentication system in PHP with OAuth2 using Apigility. In my previous post, I already presented Apigility, the new open source project to create and maintain API for PHP projects.

Apigility offers the following API authentication systems: HTTP Basic, HTTP Digest and OAuth2.

OAuth2 is a well know protocol used worldwide, for instance Facebook, Github, Twitter, etc, use this protocol to authenticate their API. Before start the Apigility functionalities of OAuth2 I would like to introduce briefly the core concepts of this protocol.

In the OAuth2 specification (RFC 6749) we have the following definitions:

  • Resource Owner: the User
  • Resource Server: the API
  • Authorization Server: often the same as the API server
  • Client: the Third-Party Application

In Apigility, the Resource Server and the Authorization Server are delivered from the same API server. The OAuth2 protocol can be considered as a framework for authorization. From the abstract of the RFC 6749 we can read:

The OAuth 2.0 authorization framework enables a third-party application to obtain limited access to an HTTP service, either on behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf.

The uses cases covered by the OAuth2 framework are:

In all these uses cases, the goal of the OAuth2 protocol is to exchange a token string between the Client and the Resource Server. This token is used to authenticate all the API calls using the Authorization HTTP header. Below is reported an example of the Bearer token (RFC 7650), the most used token type of OAuth2:

Authorization: Bearer RsT5OjbzRn430zqMLgV3Ia

Security considerations of OAuth2

The OAuth2 protocol doesn’t guarantee confidentiality and integrity of the communications. That means you must protect the HTTP communications using an additional layer. One possible solution is the usage of TLS/SSL (https) to encrypt the communiation channel from the client to the server.

OAuth1 supported an authentication mechanism based on HMAC algorithm to guarantee confidentiality and integrity, OAuth2 doesn’t (there’s a Internet-Draft proposal to support MAC token). That’s one of the main concern about the security of OAuth2 and most developers complain about that (for instance, you can read the blog post of Eran Hammer, the ex-lead author and editor of the OAuth specifications).

That said, use always HTTPS for OAuth2!

Setup OAuth2 in Apigility

Before we jump into the different use cases for OAuth2 authentication we need to configure Apigility to use OAuth2. If you don’t have Apigility installed on your computer you can go to the apigility.org website and follow the instruction reported here.

To use OAuth2 with Apigility you need to go to the dashboard page admin UI and click on the OAuth2 button, you will see a form like that:

OAuth2 configuration

The OAuth2 implementation in Apigility uses the oauth2-server-php library by Brent Shaffer.
Apigility uses a PDO database to store all the information related to the OAuth2 protocol. You can create your database using the schema reported in the file /vendor/zfcampus/zf-oauth2/data/db_oauth2.sql under your Apigility installation folder (zf-apigility-skeleton).

For testing purposes, you can use the example SQLite database that we shipped in the zf-oauth2 module, /vendor/zfcampus/zf-oauth2/data/dbtest.sqlite. In this case, you need to specify the absolute path of the dbtest.sqlite database in the PDO DSN field, using the syntax sqlite:/ /dbtest.sqlite. Moreover, you need to choose the URI for the authentication API in the OAuth2 route field, for instance /oauth.

In the example database we created a client with client_id testclient and client_secret testpass, and a user with username testuser and password testpass. We will use this example data in the following use cases.

All the sensitive data such as client_secret (in the oauth_clients table) and password (in the oauth_users table), are encrypted by Apigility using the bcrypt algorithm. If you want to generate the bcrypt hash value of a plaintext password in PHP, you can use the Zend\Crypt\Password\Bcrypt component of Zend Framework 2.

In order to facilitate the usage of the PDO database for OAuth2 we included a simple PHP script that can be executed from the command line to generate bcrypt hash values. This command line tool is available in the /vendor/zfcampus/zf-oauth2/bin folder. For instance, you can execute the following command to generate the hash of the string “test” with a cost value of 10 (the cost is a parameter of the bcrypt algorithm that indicates the computational time to consume):

php bcrypt.php test 10

You will see an output like that:

$2y$10$8gHQy/sn0vB8H5wbAbhUi.tbUfpf6aE7PBllKHeKaCYTqEyd7vjo6

The output of the bcrypt algorithm is a string of 60 bytes.

Web-server applications

The Web-server applications scenario is used to authenticate a web application with a third-party service (e.g. imagine you built a web application that needs to consume the API of Facebook). You can authenticate your application using the third-party server with a 3 steps flow as reported in the diagram below:

OAuth2 web server application

The web application send a request (including the client_id and the redirect_uri) to the third-party service asking for an Authorization code (1).
The third-party server show an Allow/Deny page to request the authorization for the access. If the user click on Allow the server send the Authorization Code to the web application using the redirect_uri (2). The web application can now perform a token request passing the client_id, the redirect_uri and the client_secret, to proof that is authorized to perform this request (3). The third-party server send the token as response if the request is valid (4).

Using Apigility we can request an access code using the following 3 steps:

1) Request the authorization code

Using a browser you can request the authorization approval from this page:

http://<apigility URL>/oauth/authorize?response_type=code
&client_id=testclient&redirect_uri=/oauth/receivecode&state=xyz

You will see a web page like the follow:

OAuth2 authorization access

You can customize this web page using the view file /vendor/zfcampus/zf-oauth2/view/zf/auth/authorize.phtml.

2) Approve the authorization access

If you Approve the authorization access clicking the Yes button, Apigility will redirect you to the URI specified in the redirect_uri passing the authorization code in the query string (code). In our example you will be redirect to the page /oauth/receive as reported below:

OAuth2 access code

3) Request the Bearer token

Now that we have the authorization code we can request an access token with a POST to /oauth passing the authorization code, the client_id, the client_secret and the redirect_uri as reported in the following HTTPie command:

http -f POST http://<apigility URL>/oauth grant_type=authorization_code
redirect_uri=/oauth/receivecode client_id=testclient client_secret=testpass
code=e906c671ac845c60c7a6b9abee113f641524fc12

The OAuth2 server will reply with the token using a JSON structure like that:

{
    "access_token": "907c762e069589c2cd2a229cdae7b8778caa9f07", 
    "expires_in": 3600, 
    "refresh_token": "43018382188f462f6b0e5784dd44c36f476ccce6", 
    "scope": null, 
    "token_type": "Bearer"
}

You have 30 seconds to request the access token starting from the time that you get the authorization code.

Finally, we can access the API using the Bearer token in the HTTP header request. For instance, we provided a test resource in zf-oauth2 module, you can access it using the following HTTPie command:

http http://<Apigility URL>/oauth/resource
"Authorization:Bearer 907c762e069589c2cd2a229cdae7b8778caa9f07"

Browser-based applications

This scenario is quite common when you have a Javascript client (e.g. a Single Page Application) that requests access to the API of a third-party server.
In a browser-based application you cannot store the client_secret in a secure way, that means you cannot use the previous workflow. We need to use an implicit grant. This is similar to the authorization code, but rather than an authorization code being returned from the authorization request, a token is returned.

In the following diagram we reported the 2 steps needed for the authentication of browser-based application scenarios:

OAuth2 Browser-based applications

The browser-based application request the authorization page to third-party service (Step 1). This page contains the Allow/Deny buttons to authorize the API access to the application. If the user click on the Allow button the third-party server send the access token using the URI fragment identifier (#access_token in Step 2). The usage of the fragment identifier for the access_token is important here, from a security point of view, because the token is not passed to the server, the scope of the token is only in the client side (the browser).

The browser-based applications scenario is supported by Apigility using the implicit grant type. This grant type is disabled by default and you need to enable it by hand, changing the configuration of allow_implicit to true in the /config/autoload/local.php file:

return array(
    'zf-oauth2' => array(
        // ...
        'allow_implicit' => true,
        // ...
    ),
);

After this change, we can request the access token using the browser-based application 2 steps:

1) Request the authorization token

We need to request the same URL used in step 1 of Web-server application scenario:

http://<Apigility URL>/oauth/authorize?response_type=token
&client_id=testclient&redirect_uri=/oauth/receivecode&state=xyz

We will see the same web page of the Web-server application scenario asking for the authorization approval.

2) Approve the authorization access

If we approve the authorization access, clicking on Yes, Apigility will send the access token to the redirect_uri using a URI fragment identifier (#access_token).

In our example, we redirect the access token to the /oauth/receive page, reported below:

OAuth2 access token

If you click on the “Click here to read…” you will see the access token appear on the page. This action is performed by a simple javascript code that parse the URL to extract the access_token value. An example of this Javascript code is reported below:

// function to parse fragment parameters
var parseQueryString = function( queryString ) {
    var params = {}, queries, temp, i, l;

    // Split into key/value pairs
    queries = queryString.split("&");

    // Convert the array of strings into an object
    for ( i = 0, l = queries.length; i < l; i++ ) {
        temp = queries[i].split('=');
        params[temp[0]] = temp[1];
    }
    return params;
};

// get token params from URL fragment
var tokenParams = parseQueryString(window.location.hash.substr(1));

Mobile apps

This OAuth2 scenario is similar to previous for browser-based applications. The only difference is the redirect_uri that in the mobile world can be a custom URI scheme. This allow native mobile apps to interact with a web browser application, opening a URL from a native app and going back to the app with a custom URI.
For instance, iPhone apps can register a custom URI protocol such as “facebook://”. On Android, apps can register URL matching patterns which will launch the native app if a URL matching the pattern is visited.

Below is reported the diagram for the OAuth2 authentication with Mobile apps:

OAuth2 Mobile app

As you can see the flow is a 2 steps authentication mechanism as for the browser-based applications.

Username and password access

This use case can be used to authenticate an API with a user based grants (password grant). The typical scenario includes a Login web page with username and password that is used to authenticate against a first-party API. Password grant is only appropriate for trusted clients. If you build your own website as a client of your API, thenk this is a great way to handle loggin in.

The authentication mechanism is very simple and it is just 1 step (see diagram below).

OAuth2 username and password

The client application send a POST to the OAuth2 server with the username and password values. The OAuth2 server gives the token access as response in JSON format.

Application access

This use case can be used to authenticate against application access, mosty likely in machine to machine scenarios. The OAuth2 grant type for this use case is the client_credential. The usage is similar to the username and password access reported above, the application send a POST request to the OAuth2 server passing the client_id, the client_secret, that acts like the user’s password. The server reply with the token if the client credentials are valid.

Refresh OAuth2 token

The OAuth2 protocol gives you the possibility to refresh the access token generating a new one, with a new life time. This action can be performed using the refresh_token that the OAuth2 server gives as response during the authentication step.

In Apigility you can refresh the access token with a POST to the OAuth2 server endpoint. In our OAuth2 database example we can perform a refresh token using the following command:

http -f POST http://<Apigility URL>/oauth grant_type=refresh_token 
refresh_token=<here the refresh_token> client_id=testclient
client_secret=testpass

The response will be something like that:

{
    "access_token": "470d9f3c6b0371ff2a88d0c554cbee9cad495e8d", 
    "expires_in": 3600, 
    "scope": null, 
    "token_type": "Bearer"
}

Revoke OAuth2 token

Recently (August 2013) the IETF published the RFC 7009 about the OAuth2 token revocation. The actual version 0.8 of Apigility doesn’t support the token revocation, we will support this feature soon. Anyway, is still possible to revoke specific access token using the PDO database. All the tokens are stored in the oauth_access_tokens table, if you want to revoke a token you can delete it from the table, with a SQL query like that:

DELETE FROM oauth_access_tokens WHERE access_token="<token to remove>";

Conclusion

In Apigility we started to support OAuth2 authentication from version 0.8. We are still working on some of the feature of this protocol, like the token revocation. We are collecting feedbacks from the community and if you want to share your comments you are more than welcome to join our apigility-user or apigility-dev mailing list. More information are available on the official web site of the project, apigility.org.

Create RESTful API for Symfony2 with Apigility

Apigility Apigility is an open source project to simplify the implementation of HTTP API for PHP applications. The project has been implemented in Zend Framework 2 but it can be used to create API for any PHP application. To prove this aspect, I show in this post how to use Apigility to create a RESTful API for an existing Symfony2 project.

This use case is an extension of the proof of concept example that has been presented at the PHP Forum 2013 in Paris, during the talk “Symfony2 and Zend Framework 2: the perfect team” of Stefan Koopmanschap and myself.

I would like to thanks Alessandro Nadalin for the suggestion about the bootstrap part of Symfony2.

Installation

You can install the Symfony2 application and the API implementation at the following github repository https://github.com/ezimuel/apigility-symfony2-use-case. This repository includes the Symfony2 application in the folder symfony2app and the Apigility API in the folder apigilityapi.

You need to install the Symfony2 application before to start using Apigility. To install the Symfony2 application follow the steps reported in this README.md file.

After this installation, please verify that you can execute the web application in a browser, if you are using PHP 5.4+ you can execute it using the following command line, from the symfony2app folder:

php app/console server:run

This command executes the internal PHP web server on localhost with default port 8000. You can check if the application is working correctly pointing your browser to http://localhost:8000/post. Try to add some post using the web interface in order to populate the database with some data.

If everything is working fine you can now install the API implementation provided by Apigility. You need to follow the instructions reported in this README.md file.

After this step you are ready to execute the RESTful API using a web server! If you are using PHP 5.4.8+ you can use the internal PHP web server using the following command, from the apigilityapi folder:

php -S 0:8080 -t public/ public/index.php

Consuming the RESTful API

The CRUD operations offered by the Symfony2 web application are now exposed as RESTful API, thanks to Apigility. You can access it using the following HTTP actions:

GET    /post        get the entire list of posts
GET    /post[/:id]  get the post specified by id
POST   /post        create a new post
PUT    /post[/:id]  update a post
PATCH  /post[/:id]  partial update a post
DELETE /post[/:id]  delete the post specified by id

For instance, you can get the lists of all the posts from the command line using the following HTTPie instruction:

http GET http://localhost:8080/post

You should retrieve a result like that:

HTTP/1.1 200 OK
Connection: close
Content-Type: application/hal+json
Host: localhost:8080

{
    "_embedded": {
        "post": [
            {
                "_links": {
                    "self": {
                        "href": "http://localhost:8080/post/1"
                    }
                }, 
                "content": "test", 
                "id": 1, 
                "publish_date": {
                    "date": "2013-11-29 15:40:00", 
                    "timezone": "Europe/Berlin", 
                    "timezone_type": 3
                }, 
                "title": "test"
            }, 
			...
		]
    }, 
    "_links": {
        "self": {
            "href": "http://localhost:8080/post"
        }
    }
}

As you can see the body response is represented in JSON HAL format, the default format of Apigility.

One of the cool feature of Apigility is the error management, for instance if you try to execute a POST on the /post resource without the JSON data reported in the body, you will get the following error message:

http POST http://localhost:8080/post

HTTP/1.1 400 Bad Request
Connection: close
Content-Type: application/api-problem+json
Host: localhost:8080

{
    "detail": "You need a title and a content at least", 
    "httpStatus": 400, 
    "problemType": "http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html", 
    "title": "Bad Request"
}

This error is managed directly by Apigility and you don’t need to do nothing in the source code of your API.

If you want to add a valid post you need to specify the title and the content as JSON parameter. Using HTTPie you can execute the following command:

http --json POST http://localhost:8080/post title=Foo content=Bar

The HTTP response will be something like that:

HTTP/1.1 201 Created
Connection: close
Content-Type: application/hal+json
Host: localhost:8080
Location: http://localhost:8080/post/4

{
    "_links": {
        "self": {
            "href": "http://localhost:8080/post/4"
        }
    }, 
    "content": "Bar", 
    "id": 4, 
    "publish_date": {
        "date": "2013-12-04 16:00:03", 
        "timezone": "Europe/Berlin", 
        "timezone_type": 3
    }, 
    "title": "Foo"
}

Basically, using Apigility we exposed the model part of the Symfony2 project reusing the original code of the application. This use case can be interesting for existing Symfony2 projects where you want to have a RESTful API with versioning, error handling, authentication, etc ready in minutes using Apigility.

In the next section I explain the procedure that we used to create this API.

How to use Apigility

Apigility can be used to create API using a simple web user interface (built with AngularJS).
In order to execute this UI you need to enable the development mode of Apigility executing the following command, inside the root folder of the apigility skeleton application:

php public/index.php development enable

After that, you can execute Apigility configuring your web server to point to the public folder of the project. If you are using PHP 5.4.8+ you can use the internal web server of PHP using the following command:

php -S 0:8080 -t public/ public/index.php

If you point your browser to http://localhost:8080, you should see the welcome page of Apigility.

Welcome page of Apigility

To create a new API you can click on the Create New API button (top/right of the page). In our case, we created a new ApiBlog API.
Now you can choose the ApiBlog section (top of the page) and create a new RESTful service, choose the REST Service option of the menu on the left and click on the Create New REST Service button. Because we want to build API for an existing Symfony2 project, we need to choose a Code-Connected service, we choosed the Post name for this service.

Create New Service with Apigility

To create a REST service you need to insert some information, like the Route to match, the HTTP methods allowed for ENTITIES and for COLLECTIONS, the Identifier name, etc. In our case we choosed the following set:

  • Route to match: /post[/:id]
  • Identifier name: id
  • Collection name: post

We used the default values for the other parameters.

After this step we need to customize the Apigility configuration to use the Post entity provided by the Symfony2 project. The idea is to expose the Post entity, coming from the Symfony2 application, as RESTful resource.

The first step is to add to the composer.json of Apigility all the dependencies used by the composer.json of the Symfony2 application. In this way, we can reuse all the existing classes of the Symfony2 project in Apigility.

The second step is to edit the config/module.config.php file included in the ApiBlog that we just created (all the API produced by Apigility are shipped as ZF2 module in the /module folder).

You need to change the zf-rest[‘entity-class’] value to the Symfony2 entity ‘Blog\\ExampleBundle\\Entity\\Post’ (line 45 of our module.config.php) and the zf-hal[‘metadata_map’] value of ‘ApiBlog\\V1\\Rest\\PostEntity’ to ‘Blog\\ExampleBundle\\Entity\\Post’. You need also to add the value ‘hydrator’ => ‘ClassMethods’ to this Post entity (line 72 of our module.config.php).

The third and last step is to edit the PostResource class of Apigility to add all the specific RESTful actions for each HTTP methods. If we want to expose all the CRUD operations of the Symfony2 application we need to fill the following methods:

  • create($data), POST /post
  • delete($id), DELETE /post[/:id]
  • fetch($id), GET /post[/:id]
  • fetchAll($params), GET /post
  • patch($id, $data), PATCH /post[/:id]
  • update($id, $data), PUT /post[/:id]

The idea is to bootstrap the Symfony2 application (without the HTTP request management) in the constructor of the PostResource class and store the specific service that we need in a protected variable, in order to be used in the specific methods to retrieve data from the businees logic of the Symfony2 application.

In our case, the code that we used to bootstrap the Symfony2 application is reported below:

public function __construct()
{
    $symfonyApp = '/path/to/your/symfony2app';
    require_once $symfonyApp . '/app/AppKernel.php';

    $kernel = new \AppKernel('prod', true);
    $kernel->loadClassCache();
    $kernel->boot();

    $this->doctrine = $kernel->getContainer()
                             ->get('doctrine')
                             ->getManager();
}

The usage of the AppKernel class of Symfony2 is very similar to the code used in the web/app.php file of the Symfony Standard Edition distribution.

In our case, we used the doctrine manager service because our Symfony2 application used a simple entity of Doctrine as model. According to your Symfony2 application, you can get the specific service/object that you need to consume in the API from the Container of your application ($kernel->getContainer()).

Using the $this->doctrine object we can reuse the Post Entity of the Symfony2 application in all the API methods. You can see how we implemented our PostResource class, here.

Note that we can return the post entity object as result in all the API methods because we specified the usage of ClassMethods hydrator in our configuration file config/module.config.php.

Conclusion

In our use case we showed how to reuse an existing Symfony2 application and consume the Model part to build a RESTful API. We decided to bootstrap the Symfony2 application because in this way we are really reusing existing code, especially if the Model part of your application uses different service and not only Doctrine entities.

Regarding the performance of this approach I can say that they are related to the specific implementations of the Symfony2 application. Generally speaking, the performance of the Symfony2 bootstrap part are good thanks to the lazy loading mechanism of the framework. Of course, you can reduce the bootstrap part for the API removing the services and components that are not necessary for the Model part.

If you are interested in that optimization I suggest to have a look at the app/AppKernel.php and the Symfony\Component\HttpKernel\Kernel component of Symfony2.

In the case that your Symfony2 application uses only Doctrine as Model you can also use directly the ZF2 module of Doctrine2 to create the API, instead of bootstrap the Symfony2 application. Recently, Tom Anderson developed a tool to create an Apigility API from Doctrine entities in scope, you can find this project here.

The use case presented in this post can be considered a first approach to the usage of Apigility to create API for Symfony2, I’m sure we will see more examples in the next future.

To have more info about Apigility and to follow the future implementations of the project you can visit the web page apigility.org.