AWS Serverless Image Processing Lab

An end-to-end serverless image processing solution built with Amazon S3, Amazon SNS, Amazon SQS, AWS Lambda, and Amazon CloudWatch.

The project demonstrates an event-driven architecture that automatically processes uploaded images using fully managed AWS serverless services.

---

Project Objective

This lab demonstrates how an image uploaded to Amazon S3 automatically triggers a serverless processing pipeline. The solution uses Amazon SNS to distribute notifications, Amazon SQS to decouple processing, and AWS Lambda functions to generate multiple image sizes.

---

Architecture Diagram

The following diagram illustrates the serverless event-driven architecture implemented in this lab.

---

Architecture Overview

The solution follows an event-driven architecture in which image uploads automatically trigger a serverless processing pipeline.

1. An image is uploaded to the ingest/ folder in Amazon S3.
2. Amazon S3 publishes an event notification to an Amazon SNS topic.
3. Amazon SNS distributes the event to three Amazon SQS queues using the fan-out pattern.
4. Each Amazon SQS queue triggers a dedicated AWS Lambda function.
5. The Lambda functions generate thumbnail, web, and mobile versions of the image.
6. The processed images are stored in Amazon S3, and execution logs are recorded in Amazon CloudWatch.

---

Implementation Screenshots

---

Amazon S3 Event Notification

The Amazon S3 bucket is configured to publish an event notification whenever a .jpg image is uploaded to the ingest/ folder. The notification is sent to an Amazon SNS topic, initiating the serverless image processing workflow.

---

Amazon SNS → Amazon SQS Integration

Amazon SNS distributes image upload events to multiple Amazon SQS queues using the publish/subscribe pattern. This fan-out architecture enables independent and scalable processing pipelines.

---

Amazon SQS Queues

Three Amazon SQS queues decouple the image processing workflow. Each queue triggers a dedicated AWS Lambda function responsible for generating a specific image size.

• thumbnail-queue
• web-queue
• mobile-queue

---

AWS Lambda Functions

The solution uses three AWS Lambda functions to process images asynchronously. Each function is responsible for generating a different output size after receiving messages from its dedicated Amazon SQS queue.

• CreateThumbnail – Generates thumbnail-sized images.
• CreateWebImage – Generates web-optimized images.
• CreateMobileImage – Generates mobile-sized images.

---

CreateThumbnail Lambda

The CreateThumbnail Lambda function is triggered by Amazon SQS and resizes uploaded images using the Pillow library. The processed thumbnail is then stored back in Amazon S3.

---

Amazon S3 Bucket Structure

The Amazon S3 bucket is organized into separate folders for the original uploads and the processed images generated by the Lambda functions.

• ingest/ – Original uploaded images
• thumbnail/ – Thumbnail-sized images
• web/ – Web-optimized images
• mobile/ – Mobile-sized images

---

Amazon CloudWatch Logs

CloudWatch Logs confirm the successful execution of the CreateThumbnail Lambda function after an image upload.

The logs show the complete invocation lifecycle (INIT_START → START → END → REPORT) and confirm that the image ingest/InputFile.jpg was processed successfully.

---

AWS Services Used

Service	Purpose
Amazon S3	Store original and processed images
Amazon SNS	Publish image upload events
Amazon SQS	Queue messages for asynchronous processing
AWS Lambda	Resize uploaded images
Amazon CloudWatch	Logging and monitoring
IAM	Grant permissions between AWS services

---

Features

• Event-driven architecture
• Amazon S3 Event Notifications
• Amazon SNS fan-out messaging
• Amazon SQS message decoupling
• AWS Lambda image processing
• Automatic image resizing
• Amazon CloudWatch monitoring
• Serverless architecture
• Python-based image manipulation

---

Workflow

1. Upload image (.jpg)

        │

        ▼

2. Amazon S3 Event Notification

        │

        ▼

3. Amazon SNS Topic

        │

        ├──────────────┐
        │              │
        ▼              ▼
 Thumbnail Queue   Web Queue   Mobile Queue
        │              │             │
        ▼              ▼             ▼
   Lambda #1      Lambda #2     Lambda #3

        │
        ▼

4. Resized images stored in S3

        │
        ▼

5. Logs available in Amazon CloudWatch

---

Repository Structure

.
├── README.md
├── LICENSE
├── architecture/
│   └── architecture.png
└── screenshots/
    ├── s3-event-notification.png
    ├── sns-sqs-integration.png
    ├── sqs-queues.png
    ├── lambda-functions.png
    ├── lambda-thumbnail.png
    ├── s3-bucket-structure.png
    └── cloudwatch-logs.png

---

Skills Demonstrated

Cloud Computing

• AWS Serverless
• Event-Driven Architecture
• Distributed Systems

Messaging

• Amazon SNS
• Amazon SQS
• Publish/Subscribe Pattern
• Fan-out Pattern
• Asynchronous Messaging

Compute

• AWS Lambda
• Event-driven Functions
• Serverless Processing

Storage

• Amazon S3
• Event Notifications

Monitoring

• Amazon CloudWatch
• Log Analysis

Programming

• Python 3.11
• Image Processing
• AWS SDK (Boto3)

---

Technologies Used

• Amazon S3
• Amazon SNS
• Amazon SQS
• AWS Lambda
• Amazon CloudWatch
• IAM
• Python 3.11
• Boto3
• Pillow (PIL)

---

Learning Outcomes

By completing this lab, I gained hands-on experience with:

• Designing event-driven architectures
• Building serverless applications
• Decoupling distributed systems
• Processing asynchronous workloads
• Monitoring Lambda executions
• Integrating multiple AWS managed services

---

Author

Ze Mendes

Information Security Analyst with experience in cloud infrastructure, cybersecurity, automation, and governance.

• GitHub: https://github.com/zcmendes
• LinkedIn: https://www.linkedin.com/in/zcmendes