AWS S3 is one of the most popular services provided by AWS. The S3 stands for Simple Storage Service and lets you fetch or upload any amount of data from any place. It provides access to highly scalable, reliable, fast, and cost-efficient data storage. There are many applications of AWS S3 for example you would want to store CSVs, images, XML, or any other file on AWS S3 rather than any other web host because it’s a very cost-effective and reliable method. To upload any file on AWS S3, you can use AWS Boto in Python.

Boto3 or Boto is the official SDK from AWS to integrate and use AWS S3 into your Python application. It allows you to perform CRUD or you can say Create, Read, Update and Delete objects in an S3 bucket.

By the end of this tutorial, you should be able to perform basic operations on AWS S3 like:

  • Upload an object on AWS S3 bucket
  • Fetch an object from AWS S3 bucket
  • Delete an object on the AWS S3 bucket

Prerequisites:

Step -1: Dependency Installation

Before we move ahead and dive into the coding part, let’s install our main player i.e. Boto3. So open your terminal and paste the below code: 

pip3 install boto3

Once installed successfully, you should now be able to import the SDK into your code.

Step -2: Uploading an object to AWS S3 Bucket

Now open your favourite code editor and let’s write some code to upload an object to an AWS S3 Bucket: 

import boto3
from botocore.exceptions import NoCredentialsError

ACCESS_KEY = 'ENTER YOU ACCESS KEY HERE'
SECRET_KEY = 'ENTER YOUR SECRET KEY HERE'

def upload_to_s3(local_file, bucket, s3_file):
     s3 = boto3.client('s3', aws_access_key_id=ACCESS_KEY,
                       aws_secret_access_key=SECRET_KEY)
     try:
         s3.upload_file(local_file, bucket, s3_file)
         print("Upload Successful")
         return True
     except FileNotFoundError:
         print("The file was not found")
         return False
     except NoCredentialsError:
         print("Credentials not available")
         return False

Code Explanation:

In the above code, we’re importing the AWS SDK i.e. boto3 and we also have defined two variables ACCESS_KEY and SECRET_KEY. You can get these credentials from the AWS console and paste it in the code. After that we have a function defined upload_to_s3() which takes three parameters: path of the local file, bucket name and the key of the file to AWS S3.

To use it, simply pass the required params and it’ll upload the object/file to the required destination. For example:

upload_to_s3('PATH TO MY LOCAL FILE', 'BUCKET NAME', ' S3 FOLDER/File_name.xyz')

Note: To replace the file, you can use the same code as it’ll over write the existing file.

Step -3: Fetching all the objects from S3 Bucket

To fetch all the objects from an S3 Bucket, you can use the below code: 

import boto3
from botocore.exceptions import NoCredentialsError

ACCESS_KEY = 'ENTER YOU ACCESS KEY HERE'
SECRET_KEY = 'ENTER YOUR SECRET KEY HERE'

def fetchFromS3(bucket_name):
 s3 = boto3.client('s3', aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY)
 bucket = s3.Bucket(bucket_name)
 for obj in bucket.objects.all():
     key = obj.key
     body = obj.get()['Body'].read()
     print(body)

Similar to the uploading, to fetch all the objects you have to pass the API credentials in place of ACCESS_KEY and SECRET_KEY. To fetch the objects within a bucket use the below code: 

fetchFromS3('MY BUCKET NAME')

The above code will print the objects within the given bucket.

Step -4: Deleting an object in AWS S3

This is one of the most easiest step in this tutorial. To delete a file/object, you just need to pass the key of the file and that’s it!. 

import boto3
from botocore.exceptions import NoCredentialsError

ACCESS_KEY = 'ENTER YOU ACCESS KEY HERE'
SECRET_KEY = 'ENTER YOUR SECRET KEY HERE'

def deleteObj(bucket_name, s3_path):
  s3 = boto3.client('s3', aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY)
  s3.Object(bucket_name, s3_path).delete()

Code Explanation:

We have defined a function deleteObj() which takes two parameters: bucket name and S3 path of the file/object. After that we created an instance of S3 using our AWS credentials. The bucket name and object name is then passed to Object() which calls the delete().

To delete a file/object use: 

deleteObj('test-bucket', 'folder_name/file_name')

Final Words

See? How easy it was to perform CRUD operations on the AWS S3 using AWS Boto in Python. There are a lot of things you can perform using the AWS Boto in Python. This was just a gist on how you can perform the basic operations using AWS Boto in Python. I hope you understood the tutorial and if you’re stuck anywhere let us know in the comments section.