Creating a production ready API with Python and Django Rest Framework – part 3

In the previous part we implemented authentication, permissions and the possibility to POST new products for admins. In this new episode we will see how to implement details management, relations between models, nested APIs and a different level of permissions.

If you haven’t completed the previous parts or if you want to begin from this one, checkout the right code first:

Handling Product Details

Our current API methods allow us to list all the products we have in our catalog and to create a new one (if we have admin permissions), but what if we wanted to delete or update a single one? What if we wanted to get only a specific product? We need to handle details.

As first thing we need to change the ProductSerializer to return the id of the product. Edit catalog/serializers.py and change the class in this way:

After changing the serializer we need to implement a new view called ProductDetail. Edit catalog/views.py and add the following imports:

and the following class:

let’s connect the new view to the urls, editing catalog/urls.py and changing the code in this way:

If we try to PUT, DELETE or GET a product like /products/1/ we can now update, delete or retrieve an existing item, but there is a little problem: we haven’t set any permission on this class, so anyone can do it. The previous view was also more compact, why don’t we use a generic view to perform these basic operations? Let’s refactor ProductDetail with a RetrieveUpdateDestroyAPIView generic class. Open catalog/views.py and change the class code in this way:

That’s it! With just three lines of code we have now implemented the same feature of the previous class, plus we have set the correct permissions.

To checkout the code at this point:

Reviews – Relations between models

As many online catalogs already have, it would be nice if our API had an endpoint where it is possible to leave a review for a product and get a list of reviews for a specific product. To implement this feature we need to add a new model to our application. Edit catalog/models.py adding this import:

and this Django model:

after creating the model, please remember to create the related DB migration:

When the model is ready, we have to do some changes to the serializers. First of all we need to write a new one, for our new Review model. Then we have to change our ProductSerializer so that it will return its related reviews. Each Product can have multiple Review. And each Review will be always linked to a specific Product. Edit catalog/serializers.py and change it in this way:

Note: in ReviewSerializer when we serialise the user contained in created_by field, return the username instead of the id (to make it more human readable). Another important thing to notice is that the value of the related_name we have set in the Review model must match with the field name we have added in ProductSerializer fields property. In this case we have set it to reviews.

At this point we need to add a new view. Edit catalog/views.py and add the following imports:

then add this class:

As you can notice, I had to customise the perform_create method because the default one doesn’t know anything about the fact we want to set the created_by and product_id fields. Finally we need to bind this new view to a specific url, so we need to edit catalog/urls.py and add this:

At this point any authenticated user should be able to POST a review for a product and anyone should be able to get the list of reviews for each product. If you have any problem with the code and want to move to this point, please checkout this:

Nested APIs details

To complete our API endpoints for Review, we need to add an additional feature that will let users to edit/delete their own review. Before implementing the new view, we need a little bit of refactoring and a new permission class. Edit catalog/permissions.py and add this new class:

Basically this will permit changes to the review only to its author. Now we are going to add new urls and doing some refactoring at the same time. Edit catalog/urls.py and change the urls in this way:

You may have noticed that I substituted pk with product_id. In the latest url I added, we need to be able to identify two primary keys: the one for the product and the one for the review. I renamed the previous ones for consistency. Now it’s time to add the new view for Review details. Edit catalog/view.py and add this class:

What are we doing here? You may have noticed that we set a new property called lookup_url_kwarg. That property is being used to determine the keyword in urls.py to be used for the primary key lookup.

You will also need to do some refactoring to the other views, to adapt them to the changes we just did to the urls. I suggest you to have a look at the diffs here: https://github.com/andreagrandi/drf-tutorial/compare/tutorial-1.13…tutorial-1.14 or you can have a look at the whole file here https://github.com/andreagrandi/drf-tutorial/blob/541bf31c11fd1dbf2bcc1d31312086995e3e5b48/drftutorial/catalog/views.py

In alternative, you can fetch the whole source code at this point:

Wrapping Up

In this third part of the tutorial you learned how to handle model details in the API and how relations between different model work. In the next part of the tutorial we will do something we should have done since the beginning: adding tests to our code and learn how to properly test the API.

Feedback Please

I know, this blog doesn’t have any “comment” feature (I was tired of dealing with spam), but if you want to provide some feedback you can still do it by email. Just visit my About page, you will find my email there.

Creating a production ready API with Python and Django Rest Framework – part 2

In the first part of this tutorial we have seen how to create a basic API using Django Rest Framework. This second part will explain how to implement POST methods and add different levels of permissions and authentication. If you are starting from part 2, you may want to checkout the source code at this exact point:

A step back

Before showing how easy it is to implement a POST method for our existing API, I want to do a step back and show you the “manual way”, using just the APIView class. Edit the file catalog/views.py and change the code in this way:

If we try to use the API again (from the browser of from the http client), it will still work in the same way. The difference here is that we are using the very basic APIView class and we have explicitly defined the GET method for it.

Implementing a POST method with APIView

An API is not being used at its full potential if it’s read only. We are going to implement a POST method for the existing view and testing it with httpie client again. First of all we need to add an import to catalog/views.py

then we add this method to our ProductList class:

Now let’s test our POST method we just implemented:

It works! In case something doesn’t work, try to fetch the source code at this point:

Implementing a POST method with ListCreateAPIView

Do you remember when I mentioned at the beginning that there is an easy way to do the same thing? I wasn’t cheating. Let’s change again our old code in catalog/views.py but this time we will use a different base class:

let’s test this again with httpie:

We just POSTed some data on the API! How can it work? Well, we have changed the base class from ListAPIView to ListCreateAPIView. This particular class implements a generic POST method that will accept and validate all the fields through the specified serializer.

Authentication

Now our API let us add products to the catalog, amazing! But… is it exactly what we want? In a real scenario we don’t want any random user to be able to add products in our database, so we are going to protect the POST method allowing only Admin users.

Before digging into Django Rest Framework permissions, we need to setup an authentication system. For simplicity we will implement TokenAuthentication. As first step we need to edit settings.py and insert rest_framework.authtoken in the INSTALLED_APPS:

after this, we need to add TokenAuthentication as default authentication class (append this in settings.py at the end):

Finally we need to add a particular URL to the project so that clients will be able to call an endpoint passing username and password to get a token back. Edit drftutorial/urls.py and make it’s like this:

Don’t forget to re-run the migrations, because TokenAuthorization needs to change a couple of tables:

In case you had any problem changing the code up to this point, you can always fetch the related git tag:

Testing the Authentication

Before testing the authentication, make sure you created at least the Django superuser with:

now let’s try to obtain the token we will need later for our API calls:

We will need to pass this token in every API call we want to be authenticated. The token is being passed through the “Authentication” header parameter.

API Permissions

Authentication is something that identify the user with a particular system. Permissions instead are the level of things that are allowed or not allowed for a particular user. In our case we said we want to let Admin users to be able to POST new products and we want to let even anonymous users to GET the product list.

Django Rest Framework has some built-in classes that we can apply to our views to define the level of permissions. We could have used the IsAdminUser class, but it would not allow anonymous users to perform the GET request. Or we could have used IsAuthenticatedOrReadOnly class, but this would allow any registered user to add products (and we want to let only admins).

Or…we can define our own permission class and have exactly what we want. Create a new file catalog/permissions.py

Just as a side note, SAFE_METHODS are GET, HEAD and OPTIONS. These method are considered “safe” because they don’t change any existing data. Open catalog/views.py again, import this at the beginning:

and set this as permission_classes to ProductList:

Let’s now try to add a new product using the token we got before (you will have to use your own token of course, mine only works on my local db):

It worked! We have now protected our API so that not admin people can’t create any product. If you have any problem with the code, you can check it out with this tag:

Wrapping Up

We have now implemented the POST method to add new products to our catalog. In the next episode we will see how to implement endpoints to get a single product, to update or delete products and finally we will allow registered users to send a review for a specific product.

Feedback Please

I know, this blog doesn’t have any “comment” feature (I was tired of dealing with spam), but if you want to provide some feedback you can still do it by email. Just visit my About page, you will find my email there.

Creating a production ready API with Python and Django Rest Framework – part 1

The aim if this tutorial is to show how to create a production ready solution for a REST API, using Python and Django Rest Framework. I will show you how to first create a very basic API, how to handle the authentication and permissions and I will cover deployment and hosting of images. The full source code of the tutorial is available at: https://github.com/andreagrandi/drf-tutorial

Summary of the complete tutorial

  1. Create the basic structure for the API
  2. Add Authentication and POST methods
  3. Handling details and changes to existing data
  4. Testing the API
  5. Switching from Sqlite to PostgreSQL
  6. Hosting the API on Heroku
  7. Add an Image field and save images to S3

Create the basic structure for the API

For this tutorial I will assume you have correctly installed at least Python (I will use Python 2.7.x), virtualenv and virtualenvwrapper on your system and I will explain how to create everything else step by step.

Note: at the time of writing, the tutorial has been based on Django 1.10.1 and Django Rest Framework 3.4.7

Creating the main project structure

Data Model

We will create the API for a generic products catalog, using a very simple structure (to keep things simple). Edit the file catalog/models.py adding these lines:

after creating the model, we need to add ‘catalog’ application to INSTALLED_APPS. Edit settings.py and add the app at the end of the list:

at this point the Django application will be recognised by the project and we can create and apply the schema migration:

API Serializer

Serializers are those components used to convert the received data from JSON format to the relative Django model and viceversa. Create the new file catalog/serializers.py and place this code inside:

In this case we are using a ModelSerializer. We need to create a new class from it, and specify the model attribute, that’s it. In this case we also specify the fields we want to return.

API View

The serializer alone is not able to respond to an API request, that’s why we need to implement a view. In this first version of the view (that we will improve soon) we will “manually” transform the data available in the serializer dictionary to a JSON response. In catalog/views.py add this code:

At this point we need to tell our Django app to use this API view when a certain URL is requested. We first need to add this code in catalog/urls.py

and finally we need to add this to drftutorial/urls.py

Testing our work

At this point we should be able to start our Django app:

Let’s install a tool that will help us to test the API:

now we can use it to call our URL:

It works! It’s an empty response of course, because we still don’t have any data to show, but we will see later how to load some example data in our database. If you have been able to follow the tutorial up to this point, that’s good. If not, don’t worry. You can checkout the code at exactly this point of the tutorial doing:

Improving the API View

There is a quicker and more efficient way of implementing the same API view we have seen before. We can use a class based view, in particular the APIView class and also have the JSON response implemented automatically. Change the code inside catalog/views.py with this one:

You will also have to change catalog/urls.py in this way:

You can check the source code for this step of the tutorial with:

There is also another way of writing the same view. Let’s try it with ListAPIView. Edit catalog/views.py again and substitute the code with this one:

With a ListAPIView we are basically creating a read-only API that is supposed to return a list of items. We need to specify a queryset and the serializer_class parameters. Once again, you can get up to this point, checking out the related git tag:

Creating Initial Data

An API that doesn’t return any data is not very useful, right? Also, at the moment we haven’t implemented yet any feature that let us insert any data. That’s why I’ve created a data migration for you that will insert some data for you. You may notice that the example data contains some Italian products… out of the scope of this tutorial, I strongly advise you to google this products and if you ever happen to visit Italy, try them. You won’t regret!

Going back to our data migration, you first have to create an empty one with:

and then open the file that has been created under catalog/migrations/ named 0002_….. (your name will be different from mine, so just edit the one starting with 0002 and you will be fine) and fill it with this code:

to apply the migration we just created, just do:

If you try to test the API again from the command line, you will get these products back:

Again, you can get up to this point with:

One more thing…

No, we are not going to present a new amazing device, I’m sorry, but I want to show you a nice Django Rest Framework feature you can have without much additional work. Edit settings.py and add rest_framework to the list of INSTALLED_APPS:

Now, if you are still running the Django app, try to visit this url from your browser: http://127.0.0.1:8000/products/
That’s very nice, isn’t it? You can have browsable APIs at no cost.

Wrapping Up

I’ve mentioned at the beginning that this is just the first part of my API tutorial. In the next part I will show you how to let the consumer of your API add some products and leave reviews (we will introduce a new model for this). Also, we will see how to set proper permissions to these new API methods so that only admin users will be able to add products while normal users will be able to add reviews. So, if you feel ready, you can begin to follow the second part of this tutorial: https://www.andreagrandi.it/2016/10/01/creating-a-production-ready-api-with-python-and-django-rest-framework-part-2/

References

Some parts of this tutorial and a few examples have been taken directly from the original Django Rest Framework tutorial.

Using Python ipdb from Jupyter

If we try to use the usual ipdb commands from a Jupyter (IPython notebook)

we will get a similar error:

The solution is to use Tracer instead:

Source: http://stackoverflow.com/questions/35613249/using-ipdb-to-debug-python-code-in-one-cell-jupyter-or-ipython

How to publish a Python package to PyPI

PyPI is the Python Package Index, that archive that let you install a package using pip, for example: pip install Flask

In the past days I started writing a Python API client for Toshl expense manager and I decided to publish the library on PyPI. You can have a look at my library here https://github.com/andreagrandi/toshl-python (please note: it’s still in development and Toshl API is not even public yet) in case you are not sure how to structure it.

I found a nice guide but it wasn’t complete (for example it didn’t say how to sign packages) so I decided to rewrite it adding more information.

Create PyPI accounts

To publish packages on PyPI you need to create two accounts: one for the production server and another one for the test server. When you register, please specify (if you have one, but I really hope you do) the PGP id of your public key. Once the accounts are created, you need to create a file named .pypirc in your $HOME folder containing the following configuration:

Please substitute your_username and your_password with the details you sent during the registration.

Preparing the package

I assume you have structured your library in the proper way and have included a setup.py with all the configuration (it’s not something specific to PyPI so you should have done it already). If you haven’t I remember you can give a look at my library here https://github.com/andreagrandi/toshl-python in particular to the setup.py:

Upload the package to PyPI Test server

The first time you upload the package you will need to register it:

and then you will need to build the package and upload it (please note I’m using the –sign to sign the package with PGP):

Upload the package to PyPI production server

Once you have verified that you are able to build and upload the package to the test server (without getting any errors), you should upload it to the production server:

This is everything you need to do if you want to publish a Python package on PyPI. Happy coding!