Python S3 Examples

Creating a Connection

This creates a connection so that you can interact with the server.

  1. import boto
  2. import boto.s3.connection
  3. access_key = 'put your access key here!'
  4. secret_key = 'put your secret key here!'
  5. conn = boto.connect_s3(
  6. aws_access_key_id = access_key,
  7. aws_secret_access_key = secret_key,
  8. host = 'objects.dreamhost.com',
  9. #is_secure=False, # uncomment if you are not using ssl
  10. calling_format = boto.s3.connection.OrdinaryCallingFormat(),
  11. )

Listing Owned Buckets

This gets a list of Buckets that you own. This also prints out the bucket name and creation date of each bucket.

  1. for bucket in conn.get_all_buckets():
  2. print "{name}\t{created}".format(
  3. name = bucket.name,
  4. created = bucket.creation_date,
  5. )

The output will look something like this:

  1. mahbuckat1 2011-04-21T18:05:39.000Z
  2. mahbuckat2 2011-04-21T18:05:48.000Z
  3. mahbuckat3 2011-04-21T18:07:18.000Z

Creating a Bucket

This creates a new bucket called my-new-bucket

  1. bucket = conn.create_bucket('my-new-bucket')

Listing a Bucket’s Content

This gets a list of objects in the bucket. This also prints out each object’s name, the file size, and last modified date.

  1. for key in bucket.list():
  2. print "{name}\t{size}\t{modified}".format(
  3. name = key.name,
  4. size = key.size,
  5. modified = key.last_modified,
  6. )

The output will look something like this:

  1. myphoto1.jpg 251262 2011-08-08T21:35:48.000Z
  2. myphoto2.jpg 262518 2011-08-08T21:38:01.000Z

Deleting a Bucket

Note

The Bucket must be empty! Otherwise it won’t work!

  1. conn.delete_bucket(bucket.name)

Forced Delete for Non-empty Buckets

Attention

not available in python

Creating an Object

This creates a file hello.txt with the string "Hello World!"

  1. key = bucket.new_key('hello.txt')
  2. key.set_contents_from_string('Hello World!')

Change an Object’s ACL

This makes the object hello.txt to be publicly readable, and secret_plans.txt to be private.

  1. hello_key = bucket.get_key('hello.txt')
  2. hello_key.set_canned_acl('public-read')
  3. plans_key = bucket.get_key('secret_plans.txt')
  4. plans_key.set_canned_acl('private')

Download an Object (to a file)

This downloads the object perl_poetry.pdf and saves it in /home/larry/documents/

  1. key = bucket.get_key('perl_poetry.pdf')
  2. key.get_contents_to_filename('/home/larry/documents/perl_poetry.pdf')

Delete an Object

This deletes the object goodbye.txt

  1. bucket.delete_key('goodbye.txt')

Generate Object Download URLs (signed and unsigned)

This generates an unsigned download URL for hello.txt. This works because we made hello.txt public by setting the ACL above. This then generates a signed download URL for secret_plans.txt that will work for 1 hour. Signed download URLs will work for the time period even if the object is private (when the time period is up, the URL will stop working).

  1. hello_key = bucket.get_key('hello.txt')
  2. hello_url = hello_key.generate_url(0, query_auth=False, force_http=True)
  3. print hello_url
  4. plans_key = bucket.get_key('secret_plans.txt')
  5. plans_url = plans_key.generate_url(3600, query_auth=True, force_http=True)
  6. print plans_url

The output of this will look something like:

  1. http://objects.dreamhost.com/my-bucket-name/hello.txt
  2. http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX

Using S3 API Extensions

To use the boto3 client to tests the RadosGW extensions to the S3 API, the extensions file should be placed under: ~/.aws/models/s3/2006-03-01/ directory. For example, unordered list of objects could be fetched using:

  1. print conn.list_objects(Bucket='my-new-bucket', AllowUnordered=True)

Without the extensions file, in the above example, boto3 would complain that the AllowUnordered argument is invalid.