Go4Expert

Go4Expert (http://www.go4expert.com/)
-   Python (http://www.go4expert.com/articles/python-tutorials/)
-   -   How to Access Amazon S3 in Python (http://www.go4expert.com/articles/access-amazon-s3-python-t28964/)

pradeep 29Aug2012 18:47

How to Access Amazon S3 in Python
 
Amazon Simple Storage Service (S3) is a low-cost & high-availability storage service provided by Amazon Web Services. It's really popular, more so due to the recent switching of IT infrastructures to cloud-based solutions. You can know more about Amazon S3, it's pricing structure etc. at http://aws.amazon.com/s3/

In this article we'll be looking at access/uploading files from/to the Amazon S3 service using Python.

Welcome Boto



Boto is a Python package which provides interfaces to various Amazon Web Services, one of them being S3. Let's start with installation of Boto package.

If you are running Debian-based systems, here the easy way:
Code:

sudo apt-get install python-boto


If you need to install from source, that's also not very tough as Boto does not have any dependencies other than what already comes with Python. Run the following commands as root.
Code:

git clone https://github.com/boto/boto.git
cd boto
python setup.py install


Accessing S3 with Boto



Boto provides a very simple and intuitive interface to Amazon S3, even a novice Python programmer and easily get himself acquainted with Boto for using Amazon S3. The following demo code will guide you through the operations in S3, like uploading files, fetching files, setting file ACLs/permissions, etc.

Code: Python

#!/usr/bin/python

## import the module
from boto.s3.connection import S3Connection

## create connection to the service
conn = S3Connection('<your_aws_access_key>', '<your_aws_secret_key>')

## creating a bucket
bucket = conn.create_bucket('go4expert_python')

## uploading data
from boto.s3.key import Key

## set target bucket
key_obj = Key(bucket)
## set filename
key_obj.key = 'my_file_1'
## set the permission/ACL of the file
key_obj.set_acl('public-read')
## set the data for the file
key_obj.set_contents_from_string('Go4expert rocks')
## set data from an existing file
key_obj.set_contents_from_filename('g4e.txt')

## downloading data
from boto.s3.key import Key
key_obj = Key(bucket)
key_obj.key = 'my_file_1'

contents = key_obj.get_contents_as_string()

## Download data into a file
key_obj.get_contents_to_filename('g4e-downloaded.txt')

## deleting a file
key_deleted = bucket.delete_key('my_file_1')

## list files/keys in a bucket
rs_keys = bucket.get_all_keys()

## now, let's look at some miscellaneous operations
## get a list of all the buckets available

## returns ResultSet object
rs_buckets = conn.get_all_buckets()

for bucket_obj in rs_buckets:
    print bucket_obj.name


Well, that was working on Amazon S3 with Python in a nutshell. Hopefully this will help you.


All times are GMT +5.5. The time now is 22:02.