Python rest api test

Python REST API Unit Testing for External APIs

Posted on 2022-10-19 Edited on 2022-11-18 In Best Practices Symbols count in article: 2.8k Reading time ≈ 10 mins.

So you’re tasked with building a service that talks to external REST API.

You use the Requests or similar library, whip up some GET or POST methods and voila! All done.

Well not so. If you’re doing it for a hobby or quick side project then yes.

But as a professional developer, you know that you’ve to account for all edge cases.

The challenge when dealing with external APIs is that the behaviour is outside your control.

Schema or payload changes, new error codes, and updated speed caps are some problems that may plague you.

A good external API will issue release notes and ensure the new version is backwards compatible (meaning the old version still works).

But it’s not an ideal world, sometimes this may change without notice and break your system.

So how do you safeguard against it? Well, you can’t. You can only control YOUR code. This is where Unit Tests come in handy.

Python REST API unit testing helps engineer your code to gracefully handle most edge cases.

• What Is A REST API?
• Why We Use REST APIs
• Simple REST API — Sample Code
• How To Test External Rest APIs?
  • Functionality Test
  • Authentication Test
  • Error Handling Test
  • Schema Test
  • Speed and RunTime Tests
  • Benefits Of Mocking/Patching
  • Concerns with Mocking

  We’ll look at real examples of how to implement this. You can find the link to the code on GitHub below.

  What Is A REST API?

  If you’re wondering or have no idea what a REST API is, a good starting place would be this article on What is a REST API, from IBM.

  In brief, a REST API is a form of Application Programming Interface (API) that conforms to the REST architectural style and patterns.

  REST stands for Representational State Transfer.

  The above IBM article explains the REST Architectural Style in detail.

  At a fundamental level, REST APIs are stateless.

  There is no state maintained between the client and server, and each client request has to include all the information required by the server to process it.

  Why We Use REST APIs

  Since the dawn of the internet, sharing data between applications has been of importance.

  Without an API, sharing data between tools and applications would be near impossible.

  Each service has a custom data model and the use of unified standards (such as JSON, XML) has made data sharing possible.

  • Share health data from your Smart Watch to a Fitness App so you receive personalised recommendations.
  • Share marketing, sales or CRM data into an Aggregation tool so you can compare performance.
  • Get live weather or maps information from an external provider so you can calculate ETA.
  • Get stock price data so you can tell your clients when to buy or sell.
  • Get the latest currency exchange values if you’re a currency trading house.

  The list is endless and I’m sure you can see the immense value of various applications talking to one another in a standard format.

  Simple REST API — Sample Code

  OK, enough with the talk. Let’s dive into some sample code.

  For this example, we’ll use the Dog API to get some data about dogs.

  Why a Dog API? Because every day is a Dog Day 🙂

  This is a simple API that uses an API Key which is sent via email after you register. And it’s totally free (for up to 10,000 requests/month).

  Pre-Requisite

  This API Key needs to be sent as a header when making Requests to the API.

  Strangely, even though the documentation says it needs an API Key, the GET requests seem to work without one :/

  
  
  
  
  
  
  RequestType(Enum):
  
  
  DogAPI:
  __init__(
  

  call_api(
  
  
  
  
  
  
  
  
  
  
  
  
  
  >)
  
  
  
  
  
  
  
  
  
  list_breeds(
  
  
  
  
  
  
  )
  
  search_breeds(
  
  
  
  
  
  
  )
  )
  
  create_vote(
  
  
  
  
  
  
  
  
  )
  

  In a subsequent update of this article, we’ll cover more API Methods.

  The code also contains a wrapper method around the Requests library call_api() .

  We use the RequestType enum class to handle each REST API Method Type e.g. GET, POST, PUT, PATCH, DELETE.

  How To Test External Rest APIs?

  Now that we’ve defined the Source Code let’s dive into the core idea. Python REST API unit testing for External APIs.

  Functionality Test

  Does your code do what it’s supposed to do? Does it produce the correct data and or result?

  Here’s an example of some functionality unit tests that you can find in the repo under /tests .

  Environment variables for the Unit Test are specified in a pytest.ini file

  2. Test Search Breeds and Create Vote (GET, POST)

  These types of Unit Tests verify the functionality of each method (it does what it says on the tin).

  Note the use of conftest.py to define the class instance object as a pytest fixture.

  Authentication Test

  What happens if our Environment Variable (& API Key) is lost?

  Our API will not work. So we need to be notified and look through the logs.

  Here’s an example of a quick authentication check.

  In this case, we override the initialised class and headers with a temporary one, containing a FAKE Key string. Obviously, this is not going to work and as expected, the API returns a 401 status code.

  Error Handling Test

  Can your code handle errors from the server?

  For example — payload errors, API limit errors or maybe the pagination times out?

  Testing for all these use cases is important.

  Maybe you want to add throttling to your API (reduce the # of requests/second) if the 3rd party API cannot handle it.

  test_list_breeds_wrong_arg_type_value_error(
  
  test_create_vote_wrong_arg_type_value_error(
  
  test_create_vote_wrong_payload_value_error(
  

  Schema Test

  One of the biggest cases of API breakage is schema changes.

  Perhaps the API releases a new version and updates the response schema and you were unaware of the release.

  You can’t always trust 3rd party APIs to honour backwards compatibility.

  Testing for schema changes, backwards compatibility and notifying you ASAP is extremely important.

  Speed and RunTime Tests

  What’s the average response time or latency of the 3rd party API?

  Has it suddenly gone from 100ms to 3000ms due to a new release or issues with server capacity?

  This could have a massive impact on your service and affect your users.

  It’s vital you know and do something about it before your users experience it.

  Should You Call The Real API?

  While including a suite of tests is critical for your application to function correctly, what if you need to run these tests all the time?

  Perhaps you have a monitoring environment that runs tests every hour to make sure the API is working as expected and the service is marked as healthy .

  This would mean you’d hit your API limit pretty easily, costing you more $$$ and wasting precious computing resources.

  When running Unit Tests you want to be in full control of the state and reduce dependence on external services.

  As you can see, calling the External API ALL the time is a bad idea.

  So what’s the solution you ask?

  We can gracefully “mock” the response of the External API thanks to libraries like pytest-mock and requests-mock .

  Running Unit Tests as part of a CI/CD Pipeline, especially when releasing to Production is a good idea.

  You expect your tests to pass and then promote the deployment to the next environment.

  But for local or high-volume repeated environment testing, mocking is a good strategy.

  Let’s look at how to do this now.

  Mock The API Response Instead

  How can we simulate the API response?

  The same way as pilots train in a simulator or medical staff in training first practice on dummy patients?

  Mocking or Patching is the practice of telling the Unit Testing framework to return a specific value for a variable, function or class object instantiation.

  Let’s look at how this is achieved with some code.

  test_mock_list_breeds() ->
  , >,
  ,
  , ,
  ,
  
  
  test_mock_search_breeds() ->
  , >,
  , >,
  , ,
  >]
  
  
  )
  
  test_mock_create_vote() ->
  
  
  
  

  In this example, we’ll use the pytest-mock library, which is a wrapper around the patching API provided by the mock package.

  If you follow the source code, you’ll see that the list_breeds() , search_breeds() and create_vote() API methods use the call_api() class method.

  The call_api() function has some logical conditions to execute a GET, PUT request. But we don’t really care about the working of this method.

  All we’re really interested in is the response of the list_breeds() , search_breeds() and create_vote() methods.

  So we “mock” the response of call_api() i.e. produce a fake/mocked response.

  This helps us test the functionality of the methods we’re interested in without worrying about the behaviour of underlying methods.

  This can be taken further to mock a variable or even an object instantiation of a class.

  This article from Chang Hsin Lee nicely explains how to use pytest-mock in great detail.

  So you may ask what’s the benefit of mocking?

  Well, in case you haven’t deduced it yourself, here are a few.

  Benefits Of Mocking/Patching

  • Avoid calling the external API to test functionality repeatedly.
  • Much faster — (slower methods and API calls) can be mocked to reduce test execution time.
  • Predictable — The responses being predictable (essentially faked) allows us to write exact assert statements and guarantee responses.

  Concerns with Mocking

  Like all things in life, everything comes with pros and cons. Well, probably not dogs 🙂

  While Mocking is great, there are some things you must be aware of when deciding to use mock functionality in your Unit Tests.

  • Tightly coupled — Tests are tightly coupled to implementation details, specifically how you import files and modules and your code structure.
  • Remember to @mock.patch every test — You have to remember to patch every test.
  • Mix testing of business logic with testing technical implementation.
  • Tests are uglier and less readable.

  So what’s the best solution?

  In a later update, you’ll learn more about using Adaptors and libraries like VCR.py` to solve some of these challenges.

  This video from Harry Percival explains some of the above concerns with solutions.

  Conclusion

  Thanks for sticking with me through this article, I hope it’s been useful.

  You came here with the goal to learn Python REST API unit testing and we covered a wide range of topics, including

  • Some basics of REST APIs
  • Example of working with a REST API
  • Types of tests to cover
  • Mocking and Patching (Benefits and Concerns)

  This will help you write production-ready and more robust code to give you confidence in your abilities in delivering a good product.

  Some other topics I’ve planned to expand this article or another include:

  1. Using Adaptors and VCR.py as an alternative to Mocking.
  2. Handling various HTTP Response Status Codes from External APIs.
  3. Run Unit Tests as part of your CI/CD Pipeline.
  4. Coverage of more API Methods and variation of tests from the DogAPI.
  5. Test various types of Authentication

  Please let me know what would be more helpful to your career (via the comments) and I’ll do my best to include them.

  Till the next time… Cheers!

  Источник