Welcome to aiotapioca-wrapper documentation!

Contents:

About

aiotapioca-wrapper provides an easy way to make explorable python API wrappers. APIs wrapped by Tapioca follow a simple interaction pattern that works uniformly so developers don’t need to learn how to use a new coding interface/style for each service API.

aiotapioca-wrapper is an asynchronous fork of the tapioca-wrapper library.

AioTapioca

Quickstart

Using a aiotapioca package

Yes, you are in the right place

There is a good chance you found this page because you clicked a link from some python package called aiotapioca-SOMETHING. Well, welcome! You are in the right place. This page will teach you the basics of how to use the package that sent you here. If you didn’t arrive here from another package, then please keep reading. The concepts learned here apply to any aiotapioca-package available.

What’s aiotapioca?

aiotapioca is an API wrapper maker. It helps Python developers creating packages for APIs (like the Facebook Graph API or the Twitter REST API. You can find a full list of available API packages made with aiotapioca here.

All wrappers made with aiotapioca follow a simple interaction pattern that works uniformly, so once you learn how aiotapioca works, you will be able to work with any aiotapioca package available.

Getting started

We will use aiotapioca-facebook as example to guide us through aiotapioca concepts. Let’s install aiotapioca-facebook:

$ pip install aiotapioca-facebook

To better experience aiotapioca, we will also use IPython:

$ pip install ipython

Let’s explore!

Go to https://developers.facebook.com/tools/explorer/, click “Get Access Token”, select all “User Data Permissions” and “Extended Permissions”, and click “Get Access Token”. This will give you a temporary access token to play with Facebook API. In case it expires, just generate a new one.

TapiocaClient object

This is how you initialize your aiotapioca client:

from aiotapioca_facebook import Facebook

async with Facebook(access_token='{your_genereated_access_token}') as api:
        ...
# or

api = Facebook(access_token='{your_genereated_access_token}')
...

If you are using IPython, you can now list available endpoints by typing api. and pressing tab.

>>> api.
api.user_likes                  api.page_blocked                 api.page_locations
api.page_statuses                api.user_applications_developer  api.user_friends
api.user_invitable_friends       api.user_photos                  api.user_videos
api.object                       api.page_conversations           api.page_milestones
...

Resources

Those are the available endpoints for the Facebook API. As we can see, there is one called user_likes. Let’s take a closer look.

Type api.user_likes? and press enter.

In [3]: api.user_likes?
...
Docstring:
Automatic generated __doc__ from resource_mapping.
Resource: {id}/likes
Docs: https://developers.facebook.com/docs/graph-api/reference/v2.2/user/likes

As we can see, the user_likes resource requires an id to be passed to the URL. Let’s do it:

api.user_likes(id='me')

Fetching data

To request the current user likes, its easy:

likes = await api.user_likes(id='me').get()

To print the returned data:

In [9]: likes.data()
OUT [9]: {
        'data': [...],
        'paging': {...}
}

Exploring data

We can also explore the returned data using the IPython tab auto-complete:

In [9]: likes.
likes.data    likes.paging

Iterating over data

You can iterate over returned data:

likes = await api.user_likes(id='me').get()

for like in likes.data:
        print(like.id())

Items passed to the for loop will be wrapped in aiotapioca so you still have access to all features.

TapiocaClientExecutor object

Whenever you make a “call” on a TapiocaClient, it will return an TapiocaClientExecutor object. You will use the executor every time you want to perform an action over data you possess.

We did this already when we filled the URL parameters for the user_like resource (calling it and passing the argument id='me'). In this new object, you will find many methods to help you play with the data available.

Here is the list of the methods available in a TapiocaClientExecutor:

Making requests

Tapioca uses the aiohttp library to make requests so HTTP methods will work just the same (get()/post()/patch()/put()/delete()/head()/options()). The only difference is that we don’t need to pass a URL since aiotapioca will take care of this.

likes = await api.user_likes(id='me').get()

URL params

To pass query string parameters in the URL, you can use the `params` parameter:

likes = await api.user_likes(id='me').get(
        params={'limit': 5})

This will return only 5 results.

Body data

If you need to pass data in the body of your request, you can use the `data` parameter. For example, let’s post a message to a Facebook wall:

# this will only work if you have a post to wall permission
await api.user_feed(id='me').post(
        data={'message': 'I love tapiocas!! S2'})

Please read aiohttp for more detailed information about how to use HTTP methods.

Multiple requests

To perform multiple requests asynchronously, you can use batch methods (post_batch()/patch_batch()/put_batch()/delete_batch()):

# this will only work if you have a post to wall permission
await api.user_feed(id='me').post_batch(
        data=[
                {'message': 'I love tapiocas!! S2'},
                {'message': 'I love tapiocas too!!'},
                ...
        ])

Accessing raw data

Use data() to return data contained in the ProcessData object.

>>> likes = await api.user_likes(id='me').get()
>>> likes.data()
{
        'data': [...],
        'paging': {...}
}
>>> # this will print only the array contained
>>> # in the 'data' field of the response
>>> likes.data.data()
>>> [...]

Dynamically fetching pages

Many APIs use a paging concept to provide large amounts of data. This way, data is returned in multiple requests to avoid a single long request. Tapioca is built to provide an easy way to access paged data using the pages() method:

likes = await api.user_likes(id='me').get()

async for like in likes().pages():
        print(like.data.name())

This will keep fetching user likes until there are none left. Items passed to the for loop will be wrapped in aiotapioca so you still have access to all features.

This method also accepts max_pages and max_items parameters. If both parameters are used, the for loop will stop after max_pages are fetched or max_items are yielded, whichever comes first:

async for item in resp().pages(max_pages=2, max_items=40):
        print(item)
# in this example, the for loop will stop after two pages are fetched or 40 items are yielded,
# whichever comes first.

Accessing wrapped data attributes

It’s possible to access wrapped data attributes on executor. For example, it’s possible to reverse a wrapped list:

likes = await api.user_likes(id='me').get()

likes_list = likes.data
likes_list().reverse()
# items in the likes_list are now in reverse order
# but still wrapped in a aiotapioca object

Opening documentation in the browser

If you are accessing a resource, you can call open_docs to open the resource documentation in a browser:

api.user_likes.open_docs()

Features

Here are some features aiotapioca supports. The wrapper you are using may support them or not, it will depend on the aiotapioca-wrapper version it is tied to and if the developer implemented the methods needed to support the feature. Either way, if you find yourself in a situation where you need one of these features, clone the wrapper, update the aiotapioca-wrapper version to the latest one, implement the features you need and submit a pull request to the developer. You will be helping a lot of people!

TapiocaClient

The first object you get after you instanciate a aiotapioca wrapper is an instance of the TapiocaClient class. This class is capable of accessing the API endpoints of the wrapper and traversing response objects. No other action besides those can be achieved from a TapiocaClient. To retrieve the raw data returned from the API call you will need to transform it in a TapiocaClientExecutor.

TODO: add examples

Default URL params

Sometimes URLs templates need parameters that will be repeated across all API calls. For example, an user id:

http://www.someapi.com/{user_id}/resources/
http://www.someapi.com/{user_id}/resources/{resource_id}/
http://www.someapi.com/{user_id}/other-resources/{other_id}/

In this cases you can instantiate the wrapper passing a default_url_params parameter, and they will be used automatically to fill URL templates.

cli = MyWrapper(access_token='some_token', default_url_params={'user_id': 123456}):
cli.resources() # http://www.someapi.com/123456/resources/

Using an existing requests.Session

Requests provides access to a number of advanced features by letting users maintain a Session object.

To use these features you can create a TapiocaClient with an existing session by passing it to the new client as the session parameter:

session = aiohttp.ClientSession()
async with MyWrapper(access_token='some_token', session=session) as cli:
            cli.resources() # http://www.someapi.com/123456/resources/

This allows us to perform some interesting operations without having to support them directly in TapiocaClient and instantiate it using the async with construct. For example caching for github requests using cachecontrol:

from cachecontrol import CacheControl
from cachecontrol.caches import FileCache
import requests
import tapioca_github

session = CacheControl(requests.Session(), cache=FileCache('webcache'))
gh = tapioca_github.Github(client_id='some_id', access_token='some_token', session=session)
response  = gh.repo_single(owner="ilindrey", repo="aiotapioca-wrapper").get()
repo_data = response().data

This will cache the E-tags provided by github to the folder webcache.

TapiocaClientExecutor

Every time you call in TapiocaClient you will get a TapiocaClientExecutor. Here are the features available in a TapiocaClientExecutor:

Accessing raw response data

To access the raw data contained in the executor, use the data attribute. To access the raw response, use the response attribute. To access the status code of the response, use the status attribute.

cli = MyWrapper(access_token='some_token')

    response = await cli.some_resource().get()

    data = response.data()
    response = response.response  # return aiohttp.ClientResponse
    status = response.status

HTTP calls

Executors have access to make HTTP calls using the current data it possesses as the URL. The aiohttp library is used as the engine to perform API calls. Every key word parameter you pass to: get(), post(), put(), patch(), delete() methods will be directly passed to the request library call. This means you will be using params={'myparam': 'paramvalue'} to send querystring arguments in the url and data={'datakey': 'keyvalue'} to send data in the body of the request.

cli = MyWrapper()
    response = await cli.some_resource().get(params={'myparam': 'paramvalue'})
    response = await cli.some_resource().post(data={'datakey': 'keyvalue'})
    response = await cli.some_resource().delete(data={'id': 123})

For perform multiple requests asynchronously, you can use batch methods as like a post_batch(), patch_batch(), put_batch(), delete_batch(). The data in the list must be passed to the data parameter in order to execute requests.

cli = MyWrapper()
    response = await cli.some_resource().post_batch(data=[
                    {'datakey': 'keyvalue1'},
                    {'datakey': 'keyvalue2'},
            ])

Auth refreshing (*)

Some clients need to update its token once they have expired. If the client supports this feature, you might specify refresh_token=True in the adapter class, instantiate it passing refresh_token=True or make any HTTP call passing refresh_auth=True (both default to False). Note that if your adapter claass or client instance has refresh_token=True, then you don’t need to explicitly set it on HTTP calls.

class MyAPIAdapter(TapiocaAdapter):
    refresh_token=True
    ...

MyWrapper = generate_wrapper_from_adapter(MyAPIAdapter)

# or

cli = MyWrapper(refresh_token=True)
        ...

# or

cli = MyWrapper()
response = await cli.some_resource().post(refresh_token=True)
...

*the wrapper you are current using may not support this feature

Pagination (*)

Use pages() method to call an endpoint that returns a collection of objects in batches. This will make your client automatically fetch more data untill there is none more left. You may use max_pages and/or max_items to limit the number of items you want to iterate over.

cli = MyWrapper():

response = await cli.some_resource().get(params=...)

async for page in response().pages():
    print(page.data())
    print(page.response)
    ...

    # or

    async for page in response().pages(max_pages=2):
            ...

    # or

    async for page in response().pages(max_items=10):
            ...

    # or

    async for page in response().pages(max_pages=2, max_items=10):
            ...

*the wrapper you are current using may not support this feature

Open docs (*)

When accessing an endpoint, you may want to read it’s documentation in the internet. By calling open_docs() in a python interactive session, the doc page will be openned in a browser.

cli = MyWrapper()
cli.some_resource.open_docs()

*the wrapper you are current using may not support this feature

Open in the browser (*)

Whenever the data contained in the executor is a URL, you can directly open it in the browser from an interactive session by calling open_in_browser()

cli = MyWrapper()
response = await cli.some_resource().get()
response.data.url.open_in_browser()

*the wrapper you are current using may not support this feature

Exceptions

AioTapioca built in exceptions will help you to beautifuly catch and handle whenever there is a client or server error. Make sure the wrapper you are using correctly raises exceptions, the developer might not have treated this. Please refer to the exceptions for more information about exceptions.

Serializers

Serializers will help you processing data before it is sent to the endpoint and transforming data from responses into python objects. Please refer to the serializers for more information about serializers.

Serializers

Serializer classes are capable of performing serialization and deserialization of data.

Serialization is the transformation of data in a native format (in our case Python data types) into a serialized format (e.g. text). For example, this could be transforming a native Python Datetime instance containing a date into a string.

Deserialization is the transformation of data in a serialized format (e.g. text) into a native format. For example, this could be transforming a string containing a date into a native Python Datetime instance.

Usage

Serialization

Data serialization is done in the background when aiotapioca is executing the request. It will traverse any data structure passed to the data param of the request and convert Python data types into serialized types.

>>> response = await cli.the_resource().post(data={'date': datetime.today()})

In this example, datetime.today() will be converted into a string formatted date just before the request is executed.

Deserialization

To deserialize data, you need to transform your client into an executor and then call a deserialization method from it:

>>> response = await cli.the_resource().get()
>>> print(response.data.created_at)
<TapiocaClientExecutor object
2015-10-25T22:34:51+00:00>
>>> print(response.data.created_at.to_datetime())
2015-10-25 22:34:51+00:00
>>> print(type(response.data.created_at.to_datetime()))
datetime.datetime

Swapping the default serializer

Clients might have the default SimpleSerializer, some custom serializer designed by the wrapper creator, or even no serializer. Either way, you can swap it for one of your own even if you were not the developer of the library. For this, you only need to pass the desired serializer class during the client initialization:

from my_serializers import MyCustomSerializer

cli = MyServiceClient(
        access_token='blablabla',
        serializer_class=MyCustomSerializer)

Built-ins

class SimpleSerializer

SimpleSerializer is a very basic and generic serializer. It is included by default in adapters unless explicitly removed. It supports serialization from Decimal and datetime and deserialization methods to those two types as well. Here is its full code:

class SimpleSerializer(BaseSerializer):

def to_decimal(self, value):
        return Decimal(value)

def serialize_decimal(self, data):
        return str(data)

As you can see, datetime values will be formatted to iso format.

Writing a custom serializer

To write a custom serializer, you just need to extend the BaseSerializer class and add the methods you want. But you can also extend from SimpleSerializer to inherit its functionalities.

Serializing

To allow serialization of any desired data type, add a method to your serializer named using the following pattern: serialize_ + name_of_your_data_type_in_lower_case. For example:

class MyCustomDataType(object):
        message = ''

class MyCustomSerializer(SimpleSerializer):

        def serialize_mycustomdatatype(self, data):
                return data.message

Deserializing

Any method starting with to_ in your custom serializer class will be available for data deserialization. It also accepts key word arguments.

from aiotapioca.serializers import BaseSerializer

class MyCustomSerializer(BaseSerializer):

        def to_striped(self, value, **kwargs):
                return value.strip()

Here’s a usage example for it:

from my_serializers import MyCustomSerializer

cli = MyServiceClient(
        access_token='blablabla',
        serializer_class=MyCustomSerializer)

response = await cli.the_resource().get()

striped_data = response.the_data().to_striped()

Exceptions

Catching API errors

AioTapioca supports 2 main types of exceptions: ClientError and ServerError. The default implementation raises ClientError for HTTP response 4xx status codes and ServerError for 5xx status codes. Since each API has its own ways of reporting errors and not all of them follow HTTP recommendations for status codes, this can be overriden by each implemented client to reflect its behaviour. Both of these exceptions extend TapiocaException which can be used to catch errors in a more generic way.

class TapiocaException

Base class for aiotapioca exceptions. Example usage:

from aiotapioca.exceptions import TapiocaException

try:
        await cli.fetch_something().get()
except TapiocaException, e:
        print("API call failed with error %s", e.status_code)

You can also access a aiotapioca client that contains response data from the exception:

from aiotapioca.exceptions import TapiocaException

try:
        await cli.fetch_something().get()
except TapiocaException, e:
        print(e.client().data)
class ClientError

Default exception for client errors. Extends from TapiocaException.

class ServerError

Default exception for server errors. Extends from TapiocaException.

Flavours

Available Flavours

aiotapioca-wrapper library

Facebook

https://github.com/ilindrey/aiotapioca-facebook

Yandex Metrika

https://github.com/ilindrey/aiotapioca-yandex-metrika

tapioca-wrapper library

Facebook

https://github.com/vintasoftware/tapioca-facebook

Twitter

https://github.com/vintasoftware/tapioca-twitter

Mandrill

https://github.com/vintasoftware/tapioca-mandrill

Parse

https://github.com/vintasoftware/tapioca-parse

Bitbucket

https://github.com/vintasoftware/tapioca-bitbucket

Disqus

https://github.com/marctc/tapioca-disqus

Harvest

https://github.com/vintasoftware/tapioca-harvest

CrunchBase

https://github.com/vu3jej/tapioca-crunchbase

Otter

https://github.com/vu3jej/tapioca-otter

GitHub

https://github.com/gileno/tapioca-github

Meetup

https://github.com/lightstrike/tapioca-meetup

Toggl

https://github.com/hackebrot/tapioca-toggl

Braspag

https://github.com/parafernalia/tapioca_braspag

Iugu

https://github.com/solidarium/tapioca-iugu

Instagram

https://github.com/vintasoftware/tapioca-instagram

Youtube

https://github.com/vintasoftware/tapioca-youtube

Asana

https://github.com/henriquebastos/tapioca-asana

Desk

https://github.com/medeirosthiago/tapioca-desk

Mailgun

https://github.com/vintasoftware/tapioca-mailgun

Discourse

https://github.com/humrochagf/tapioca-discourse

StatusPage

https://github.com/olist/tapioca-statuspage

Trello

https://github.com/humrochagf/tapioca-trello

Your flavour

To create a new wrapper, please refer to Building a wrapper. Upload it to pypi and send a pull request here for it to be added to the list.

Building a wrapper

Wrapping an API with AioTapioca

The easiest way to wrap an API using aiotapioca is starting from the cookiecutter template.

To use it, install cookiecutter in your machine:

pip install cookiecutter

and then use it to download the template and run the config steps:

cookiecutter gh:vintasoftware/cookiecutter-tapioca

After this process, it’s possible that you have a ready to go wrapper. But in most cases, you will need to customize stuff. Read through this document to understand what methods are available and how your wrapper can make the most of tapioca. Also, you might want to take a look in the source code of other wrappers to get more ideas.

Adapter

AioTapioca features are mainly implemented in the TapiocaClient and TapiocaClientExecutor classes. Those are generic classes common to all wrappers and cannot be customized to specific services. All the code specific to the API wrapper you are creating goes in your adapter class, which should inherit from TapiocaAdapter and implement specific behaviours to the service you are working with.

Take a look in the generated code from the cookiecutter or in the aiotapioca-facebook adapter to get a little familiar with it before you continue. Note that at the end of the module you will need to perform the transformation of your adapter into a client:

Facebook = generate_wrapper_from_adapter(FacebookClientAdapter)

Plase refer to the TapiocaAdapter class document for more information on the available methods.

Features

Here is some information you should know when building your wrapper. You may choose to or not to support features marked with (optional).

Resource Mapping

The resource mapping is a very simple dictionary which will tell your aiotapioca client about the available endpoints and how to call them. There’s an example in your cookiecutter generated project. You can also take a look at aiotapioca-facebook’s resource mapping.

AioTapioca uses aiohttp to perform HTTP requests. This is important to know because you will be using the method get_request_kwargs to set authentication details and return a dictionary that will be passed directly to the request method.

Formatting data

Use the methods format_data_to_request and response_to_native to correctly treat data leaving and being received in your wrapper.

TODO: add examples

You might want to use one of the following mixins to help you with data format handling in your wrapper:

  • FormAdapterMixin

  • JSONAdapterMixin

  • XMLAdapterMixin

  • PydanticAdapterMixin

Exceptions

Overwrite the process_response method to identify API server and client errors raising the correct exception accordingly. Please refer to the exceptions for more information about exceptions.

TODO: add examples

Pagination (optional)

get_iterator_list and get_iterator_next_request_kwargs are the two methods you will need to implement for the executor pages() method to work.

TODO: add examples

Serializers (optional)

Set a serializer_class attribute or overwrite the get_serializer() method in your wrapper for it to have a default serializer.

from aiotapioca import TapiocaAdapter
from aiotapioca.serializers import SimpleSerializer

class MyAPISerializer(SimpleSerializer):

        def serialize_datetime(self, data):
                return data.isoformat()


class MyAPIAdapter(TapiocaAdapter):
        serializer_class = MyAPISerializer
        ...

In the example, every time a datetime is passed to the parameters of an HTTP method, it will be converted to an ISO formatted string.

It’s important that you let people know you are providing a serializer, so make sure you have it documented in your README.

## Serialization
- datetime
- Decimal

## Deserialization
- datetime
- Decimal

Please refer to the serializers for more information about serializers.

Refreshing Authentication (optional)

You can implement the refresh_authentication and is_authentication_expired methods in your TapiocaClient to refresh your authentication token every time it expires.

is_authentication_expired receives an error object from the request method (it contains the server response and HTTP Status code). You can use it to decide if a request failed because of the token. This method should return True if the authentication is expired or False otherwise (default behavior).

refresh_authentication receives api_params and should perform the token refresh protocol. If it is successfull it should return a truthy value (the original request will then be automatically tried). If the token refresh fails, it should return a falsy value (and the the original request wont be retried).

Once these methods are implemented, the client can be instantiated with refresh_token=True (or pass refresh_token=True in HTTP calls) and refresh_authentication will be called automatically. You can also specify refresh_token=True in the adapter class.

    def is_authentication_expired(self, exception, *args, **kwargs):
            ...

def refresh_authentication(self, api_params, *args, **kwargs):
    ...

XMLAdapterMixin Configuration (only if required)

Additionally, the XMLAdapterMixin accepts configuration keyword arguments to be passed to the xmltodict library during parsing and unparsing by prefixing the xmltodict keyword with xmltodict_parse__ or xmltodict_unparse respectively. These parameters should be configured so that the end-user has a consistent experience across multiple Tapioca wrappers irrespective of various API requirements from wrapper to wrapper.

Note that the end-user should not need to modify these keyword arguments themselves. See xmltodict docs and source for valid parameters.

Users should be able to construct dictionaries as defined by the xmltodict library, and responses should be returned in the canonical format.

Example XMLAdapterMixin configuration keywords:

class MyXMLClientAdapter(XMLAdapterMixin, TapiocaAdapter):
    ...
    def get_request_kwargs(self, api_params, *args, **kwargs):
        ...
        # omits XML declaration when constructing requests from dictionary
        kwargs['xmltodict_unparse__full_document'] = False
        ...

TapiocaAdapter class

class TapiocaAdapter

Attributes

api_root

This should contain the base URL that will be concatenated with the resource mapping itens and generate the final request URL. You can either set this attribute or use the get_api_root method.

resource_mapping

A dictionary containing a mapping of the resources. For example:

You can either set this attribute or use the get_resource_mapping method.

serializer_class

For more information about the serializer_class attribute, read the serializers documentation.

refresh_token

If the HTTP method was called with refresh_token=True, then it will automatically call refresh_authentication method and retry the original request.

semaphore

For more information about the semaphore attribute, read the Python’s standard library. Default value 10.

Methods

get_resource_mapping(self, api_params, **kwargs)

This method can be used instead of the resource_mapping attribute. Returns the resource_mapping attribute by default.

get_api_root(self, api_params, **kwargs)

This method can be used instead of the api_root attribute. You might also use it to decide which base URL to use according to a user input.

def get_api_root(self, api_params, **kwargs):
        if api_params.get('development'):
                return 'http://api.the-dev-url.com/'
        return 'http://api.the-production-url.com/'

You may also need to set different api_root to a specific resource. To do that you can access the resource_name inside kwargs.

def get_api_root(self, api_params, **kwargs):
    if kwargs.get('resource_name') == 'some_resource_name':
        return 'http://api.another.com/'
    else:
        return self.api_root
def get_api_root(self, api_params, **kwargs):
        if api_params.get('development'):
                return 'http://api.the-dev-url.com/'
        return 'http://api.the-production-url.com/'
get_request_kwargs(self, api_params, *args, **kwargs)

This method is called just before any request is made. You should use it to set whatever credentials the request might need. The api_params argument is a dictionary and has the parameters passed during the initialization of the aiotapioca client:

cli = Facebook(access_token='blablabla', client_id='thisistheis')
...

For this example, api_params will be a dictionary with the keys access_token and client_id.

Here is an example of how to implement Basic Auth:

from aiohttp.helpers import BasicAuth

class MyServiceClientAdapter(TapiocaAdapter):
        ...
        def get_request_kwargs(self, api_params, *args, **kwargs):
                params = super(MyServiceClientAdapter, self).get_request_kwargs(
                        api_params, *args, **kwargs)

                params['auth'] = BasicAuth(
                        api_params.get('user'), api_params.get('password'))

                return params
process_response(self, response, **kwargs)

This method is responsible for converting data returned in a response to a dictionary (which should be returned). It should also be used to raise exceptions when an error message or error response status is returned.

format_data_to_request(self, data)

This converts data passed to the body of the request into text. For example, if you need to send JSON, you should use json.dumps(data) and return the response. See the mixins section above.

response_to_native(self, response, **kwargs)

This method receives the response of a request and should return a dictionay with the data contained in the response. see the mixins section above.

get_iterator_next_request_kwargs(self, iterator_request_kwargs, response_data, response, **kwargs)

Override this method if the service you are using supports pagination. It should return a dictionary that will be used to fetch the next batch of data, e.g.:

def get_iterator_next_request_kwargs(self,
                iterator_request_kwargs, response_data, response):
        paging = response_data.get('paging')
        if not paging:
                return
        url = paging.get('next')

        if url:
                iterator_request_kwargs['url'] = url
                return iterator_request_kwargs

In this example, we are updating the URL from the last call made. iterator_request_kwargs contains the paramenters from the last call made, response_data contains the response data after it was parsed by process_response method, and response is the full response object with all its attributes like headers and status code.

get_iterator_list(self, response_data, **kwargs)

Many APIs enclose the returned list of objects in one of the returned attributes. Use this method to extract and return only the list from the response.

def get_iterator_list(self, response_data):
        return response_data['data']

In this example, the object list is enclosed in the data attribute.

is_authentication_expired(self, exception, *args, **kwargs)

Given an exception, checks if the authentication has expired or not. If the HTTP method was called with refresh_token=True, then it will automatically call refresh_authentication method and retry the original request.

If not implemented, is_authentication_expired will assume False, refresh_token also defaults to False in the client initialization.

refresh_authentication(self, api_params, *args, **kwargs):

Should do refresh authentication logic. Make sure you update api_params dictionary with the new token. If it successfully refreshs token it should return a truthy value that will be stored for later access in the executor class in the refresh_data attribute. If the refresh logic fails, return a falsy value. The original request will be retried only if a truthy is returned.

retry_request(self, exception, error_message, repeat_number, **kwargs):

Conditions for repeating a request. If it returns True, the request will be repeated.

error_handling(self, exception, error_message, repeat_number, **kwargs):

Wrapper for throwing custom exceptions. When, for example, the server responds with 200, and errors are passed inside json.

Contributors

Thanks!

Release v4.2.0 (2022-10-19)

  • Added linters, mypy and pre-commit hooks.

  • Added CI for tests, pre-commit and release.

  • Migrate to PDM.

  • Optional support orjson and ujson when available.

  • Optional dependencies xml and pydantic mixins.

  • Replaced setup.py to pyproject.toml.

Release v4.1.1 (2022-06-21)

  • Added a py.typed marker for type hints.

Release v4.0.2 (2022-06-17)

  • Added to_dict_by_alias option in TapiocaAdapterPydanticMixin.

Release v4.0.1 (2022-06-15)

  • Added option to specify a session when generating a wrapper.

  • Added max_retries_requests flag to limit the number of retries requests.

  • Added ability to specify classmethod in parsers.

  • Dump and load data in threads. Added more abstract methods to prepare parameters and data for requests or prepare native data.

  • Drop support Python 3.6 and below.

  • Implemented multiple use of aiohttp.ClientSession, outside the context manager.

  • Rework of the library structure. Splitting Tapioca client structure into classes: TapiocaClient, TapiocaClientResource, TapiocaClientExecutor and TapiocaClientResponse.

  • Reworked exception handling v.2.

  • Minor fixes.

Release v3.8.0 (2022-05-15)

  • Added propagate_exception flag after retry_request call.

  • Added data parsing mechanism. Parsers can be specified in resource_mapping.

  • Reworked exception handling.

Release v3.7.0 (2022-04-27)

  • Removed debug option.

  • Expanded the possibility of error handling. Added catch aiohttp exceptions.

Release v3.6.0 (2022-04-22)

  • Added context transfer to get_request_kwargs method.

  • Peddling kwargs to format_data_to_request and serialize_data methods.

  • Increased the debugging data output limit.

  • Removed api_params argument from get_request_kwargs method.

  • Removed PydanticSerializer.

  • Added PydanticAdapterMixin.

Release v3.5.0 (2022-04-12)

  • migration to orjson

Release v3.4.2 (2022-04-08)

  • Fixed requirements.

Release v3.4.1 (2022-04-08)

  • Fixed requirements.

Release v3.4.0 (2022-04-0)

  • Using aiologger for debugging logs.

  • Fix for recursion due to refresh_token flag.

  • Added attribute semaphore to TapiocaAdapter.

  • Added ability to pass Semaphore as a client or request parameter.

  • Added get_resource_mapping method to TapiocaAdapter.

  • Fixed an unnecessary request.

  • Added serialisation from the pydantic model.

  • Reworked flag debug.

Release v3.3.1 (2022-03-24)

  • Expanding debugging information.

Release v3.3.0 (2022-03-24)

  • The handling of the refresh token parameter was changed.

  • Added refresh_token attribute to the TapiocaAdapter class.

  • Removed refresh_token_by_default parameter in the tapioca classes.

  • Parameters passing was changed in _wrap_in_tapioca and _wrap_in_tapioca_executor.

  • Minor fixes.

Release v3.2.4 (2022-03-23)

  • Fixed “This instance has no response object” error in _wrap_in_tapioca and _wrap_in_tapioca_executor (empty response in property descendants and pages).

Release v3.2.3 (2022-03-22)

  • Returned pass request_method as param in get_request_kwargs.

Release v3.2.2 (2022-03-22)

  • Fixed fill resource template url.

Release v3.2.1 (2022-03-22)

  • Context transmission was extended.

Release v3.2.0 (2022-03-22)

  • Added retry_request and error_handling methods.

  • Added context passed to different adapter methods.

Release v3.1.1 (2022-03-21)

  • Fixed debugging flag.

Release v3.1.0 (2022-03-21)

  • Added PydanticSerializer.

Release v3.0.0 (2022-03-21)

  • Implementing an asynchronous fork.

Release v2.1.0 (2022-03-19)

  • Make TapiocaClient and TapiocaClientExecutor pickle-able.

Release v2.0.2 (2022-02-25)

  • Updated deprecated collections import

  • Adds support for python 3.10

Release v2.0.1 (2020-01-25)

  • Updates the list of supported versions in setup.py

Release v2.0.0 (2020-01-25)

  • Drops support for python 2.7 and 3.4

  • Adds support for python 3.7 and 3.8

Release v1.5.1 (2019-04-19)

  • Adds a resource_name kwarg to the get_api_root method

Release v1.5.0 (2019-04-19)

  • Removes support for Python 3.3

Release v1.4.3 (2017-06-15)

Release v1.4.1 (2017-05-25)

Release v1.4.0 (2017-03-28)

  • Adds support to Session requests

Release v1.3.0 (2017-01-20)

  • refresh_authentication should return data about the refresh token process

  • If a falsy value is returned by refresh_authentication the request wont be retried automatically

  • Data returned by refresh_authentication is stored in the tapioca class and can be accessed in the executor via the attribute refresh_data

Release v1.2.3 (2016-09-28)

  • refresh_token_by_default introduced to prevent passing refresh_token on every request.

Release v1.2.2 (2016-04-23)

Release v1.2.1 (2016-01-02)

Release v1.1.12 (2016-05-31)

Release v1.1.11 (2016-05-31)

Release v1.1.10 (2016-03-27)

  • Fixed bugs regarding request_kwargs passing over calls

  • Fixed bugs regarding external serializer passing over calls

  • Wrapper instatiation now accepts default_url_params

Release v1.1.9 (2016-03-27)

Release v1.1.8 (2016-03-27)

Release v1.1.7 (2016-03-27)

Release v1.1.6 (2016-02-29)

Release v1.1.4 (2016-02-27)

Release v1.1.0 (2016-02-27)

  • Automatic refresh token support

  • Added Python 3.5 support

  • Added support for OrderedDict

  • Documentation cleanup

Release v1.0.0 (2015-11-10)

  • Data serialization and deserialization

  • Access CamelCase attributes using snake_case

  • Dependencies are now tied to specific versions of libraries

  • data and response are now attributes instead of methods in the executor

  • Added status_code attribute to tapioca executor

  • Renamed status exception attribute to status_code

  • Fixed return for dir call on executor, so it’s lot easier to explore it

  • Multiple improvments to documentation

Release v0.6.0 (2015-09-23)

  • Giving access to request_method in get_request_kwargs

  • Verifying response content before trying to convert it to json on JSONAdapterMixin

  • Support for in operator

  • pep8 improvments

Release v0.5.3 (2015-04-10)

  • Adding max_pages and max_items to pages method

Release v0.5.1 (2015-08-06)

  • Verifying if there’s data before json dumping it on JSONAdapterMixin

Release v0.5.0 (2015-08-05)

  • Automatic pagination now requires an explicit pages() call

  • Support for len()

  • Attributes of wrapped data can now be accessed via executor

  • It’s now possible to iterate over wrapped lists

Release v0.4.1 (2015-08-01)

  • changed parameters for Adapter’s get_request_kwargs. Also, subclasses are expected to call super.

  • added mixins to allow adapters to easily choose witch data format they will be dealing with.

  • ServerError and ClientError are now raised on 4xx and 5xx response status. This behaviour can be customized for each service by overwriting adapter’s process_response method.