S3 Tutorial

This functionality requires an experimental s3 plugin. Read More

Frictionless supports reading data from an S3 cloud source. You can read files in any format that is available in your bucket.

CLI

pip install frictionless[s3]

Reading Data

You can read from this source using Package/Resource, for example:

Python

from pprint import pprint

from frictionless import Resource

resource = Resource(path='s3://bucket/table.csv')

pprint(resource.read_rows())

For reading from a private bucket you need to setup AWS creadentials as it's described in the Boto3 documentation.

Writing Data

A similiar approach can be used for writing:

Python

from frictionless import Resource

resource = Resource(path='data/table.csv')

resource.write('s3://bucket/table.csv')

Configuring Data

There is a Control to configure how Frictionless read files in this storage. For example:

Python

from frictionless import Resource

from frictionless.plugins.s3 import S3Control

resource = Resource(data=[['id', 'name'], [1, 'english'], [2, 'german']])

resource.write('table.new.csv', control=controls.S3Control(endpoint_url='<url>'))

References:

• S3 Control