How to Access Amazon S3 in Python

Discussion in 'Python' started by pradeep, Aug 29, 2012.

  1. pradeep

    pradeep Team Leader

    Joined:
    Apr 4, 2005
    Messages:
    1,645
    Likes Received:
    87
    Trophy Points:
    0
    Occupation:
    Programmer
    Location:
    Kolkata, India
    Home Page:
    http://blog.pradeep.net.in
    Amazon Simple Storage Service (S3) is a low-cost & high-availability storage service provided by Amazon Web Services. It's really popular, more so due to the recent switching of IT infrastructures to cloud-based solutions. You can know more about Amazon S3, it's pricing structure etc. at http://aws.amazon.com/s3/

    In this article we'll be looking at access/uploading files from/to the Amazon S3 service using Python.

    Welcome Boto



    Boto is a Python package which provides interfaces to various Amazon Web Services, one of them being S3. Let's start with installation of Boto package.

    If you are running Debian-based systems, here the easy way:
    Code:
    sudo apt-get install python-boto
    If you need to install from source, that's also not very tough as Boto does not have any dependencies other than what already comes with Python. Run the following commands as root.
    Code:
    git clone https://github.com/boto/boto.git
    cd boto 
    python setup.py install
    

    Accessing S3 with Boto



    Boto provides a very simple and intuitive interface to Amazon S3, even a novice Python programmer and easily get himself acquainted with Boto for using Amazon S3. The following demo code will guide you through the operations in S3, like uploading files, fetching files, setting file ACLs/permissions, etc.

    Code:
    #!/usr/bin/python
    
    ## import the module
    from boto.s3.connection import S3Connection
    
    ## create connection to the service
    conn = S3Connection('<your_aws_access_key>', '<your_aws_secret_key>')
    
    ## creating a bucket
    bucket = conn.create_bucket('go4expert_python')
    
    ## uploading data
    from boto.s3.key import Key
    
    ## set target bucket
    key_obj = Key(bucket)
    ## set filename
    key_obj.key = 'my_file_1'
    ## set the permission/ACL of the file
    key_obj.set_acl('public-read')
    ## set the data for the file
    key_obj.set_contents_from_string('Go4expert rocks')
    ## set data from an existing file
    key_obj.set_contents_from_filename('g4e.txt')
    
    ## downloading data
    from boto.s3.key import Key
    key_obj = Key(bucket)
    key_obj.key = 'my_file_1'
    
    contents = key_obj.get_contents_as_string()
    
    ## Download data into a file
    key_obj.get_contents_to_filename('g4e-downloaded.txt')
    
    ## deleting a file
    key_deleted = bucket.delete_key('my_file_1')
    
    ## list files/keys in a bucket
    rs_keys = bucket.get_all_keys()
    
    ## now, let's look at some miscellaneous operations
    ## get a list of all the buckets available
    
    ## returns ResultSet object
    rs_buckets = conn.get_all_buckets()
    
    for bucket_obj in rs_buckets:
        print bucket_obj.name
    
    
    Well, that was working on Amazon S3 with Python in a nutshell. Hopefully this will help you.
     

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice