diff --git a/.github/workflows/python-ci.yml b/.github/workflows/python-ci.yml new file mode 100644 index 0000000000..6be8601dfb --- /dev/null +++ b/.github/workflows/python-ci.yml @@ -0,0 +1,80 @@ +name: Python CI + +on: + push: + paths: + - "app_python/**" + - ".github/workflows/python-ci.yml" + pull_request: + paths: + - "app_python/**" + - ".github/workflows/python-ci.yml" + +permissions: + contents: read + +jobs: + test: + name: Dependencies, Linter and Tests + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + cache: "pip" + cache-dependency-path: | + app_python/requirements.txt + app_python/requirements-dev.txt + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install -r app_python/requirements.txt -r app_python/requirements-dev.txt + + - name: Run linter + run: ruff check app_python + + - name: Run tests + run: python -m pytest app_python/tests -q + + - name: Set up Snyk + uses: snyk/actions/setup@master + + - name: Run Snyk vulnerability check + env: + SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }} + run: snyk test --file=app_python/requirements.txt --package-manager=pip --severity-threshold=high + + docker: + name: Docker Build and Push + runs-on: ubuntu-latest + needs: test + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Login to Docker Hub + if: github.event_name == 'push' + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + - name: Build and push Docker image + uses: docker/build-push-action@v6 + with: + context: ./app_python + file: ./app_python/Dockerfile + push: ${{ github.event_name == 'push' }} + tags: ${{ secrets.DOCKERHUB_USERNAME }}/moscow-time-app:latest + cache-from: type=gha + cache-to: type=gha,mode=max diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000000..88fb42b0e5 --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +venv/ +__pycache__/ +*.pyc +gunicorn.ctl +.pytest_cache/ +.ruff_cache/ diff --git a/app_python/.dockerignore b/app_python/.dockerignore new file mode 100644 index 0000000000..6a9fc7af05 --- /dev/null +++ b/app_python/.dockerignore @@ -0,0 +1,11 @@ +venv/ +__pycache__/ +*.pyc +gunicorn.ctl +.pytest_cache/ +.ruff_cache/ +tests/ +requirements-dev.txt +CI.md +PYTHON.md +README.md diff --git a/app_python/CI.md b/app_python/CI.md new file mode 100644 index 0000000000..dcccf75bea --- /dev/null +++ b/app_python/CI.md @@ -0,0 +1,67 @@ +# Continuous Integration + +## Overview + +This project uses GitHub Actions to run automated checks for the Python web application. + +The CI workflow runs on push and pull request events when files inside the `app_python` directory or the workflow file are changed. + +## Workflow Steps + +The workflow includes the following stages: + +1. Checkout repository. +2. Set up Python. +3. Install dependencies. +4. Run linter. +5. Run unit tests. +6. Run Snyk vulnerability checks. +7. Set up Docker Buildx. +8. Login to Docker Hub. +9. Build and push Docker image. + +## CI Best Practices Used + +### Explicit Python Version + +The workflow uses a fixed Python version to make builds reproducible. + +### Dependency Caching + +The workflow uses pip cache to speed up dependency installation. + +### Docker Build Cache + +Docker Buildx cache is used to speed up repeated image builds. + +### Separate Test and Docker Jobs + +The Docker image is built only after the test job has completed successfully. + +### Path Filters + +The workflow runs only when files in the `app_python` directory or the workflow file itself are changed. + +### Secrets + +Docker Hub credentials and the Snyk token are stored in GitHub Actions secrets. + +Used secrets: + +- `DOCKERHUB_USERNAME` +- `DOCKERHUB_TOKEN` +- `SNYK_TOKEN` + +### Pull Request Validation + +The workflow runs on pull requests to validate code before merging. + +### Linting + +Ruff is used to check code quality before tests and Docker publishing. + +## Snyk + +Snyk is used to check Python dependencies for known vulnerabilities. + +The workflow fails if Snyk finds vulnerabilities with high severity or higher. diff --git a/app_python/DOCKER.md b/app_python/DOCKER.md new file mode 100644 index 0000000000..11a0f0f644 --- /dev/null +++ b/app_python/DOCKER.md @@ -0,0 +1,101 @@ +# Docker + +## Dockerfile + +The application is containerized with Docker using the following image: + +```dockerfile +FROM python:3.12-alpine3.20 +``` + +A fixed Python and Alpine version is used instead of a floating tag. + +## Best practices used + +### Non-root user + +The container runs the application as `appuser`, not as `root`: + +```dockerfile +RUN addgroup -S appgroup \ + && adduser -D -h /home/appuser -G appgroup appuser + +USER appuser +``` + +The user's home directory is used as the working directory: + +```dockerfile +WORKDIR /home/appuser +``` + +### Specific files are copied + +The image does not copy the whole project directory. Only files required to run the application are copied: + +```dockerfile +COPY --chown=appuser:appgroup requirements.txt . +COPY --chown=appuser:appgroup app.py . +``` + +### Layer order + +Dependencies are copied and installed before the application code: + +```dockerfile +COPY --chown=appuser:appgroup requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY --chown=appuser:appgroup app.py . +``` + +This keeps dependency installation in a separate layer. + +### No pip cache + +Python packages are installed without storing the pip cache: + +```dockerfile +RUN pip install --no-cache-dir -r requirements.txt +``` + +### .dockerignore + +The `.dockerignore` file excludes files that are not required for building the image: + +```dockerignore +venv/ +__pycache__/ +*.pyc +gunicorn.ctl +``` + +## Build + +```bash +docker build -t moscow-time-app:1.0.0 . +``` + +## Run + +```bash +docker run --rm -d --name moscow-time-app -p 8080:8080 moscow-time-app:1.0.0 +``` + +## Check + +```bash +curl http://localhost:8080 +``` + +```bash +docker exec moscow-time-app id +``` + +The `id` command must show that the user id is not `0`. + +## Stop + +```bash +docker stop moscow-time-app +``` diff --git a/app_python/Dockerfile b/app_python/Dockerfile new file mode 100644 index 0000000000..a9a9f4e135 --- /dev/null +++ b/app_python/Dockerfile @@ -0,0 +1,17 @@ +FROM python:3.12-alpine3.20 + +RUN addgroup -S appgroup \ + && adduser -D -h /home/appuser -G appgroup appuser + +WORKDIR /home/appuser + +COPY --chown=appuser:appgroup requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY --chown=appuser:appgroup app.py . + +EXPOSE 8080 + +USER appuser + +CMD ["gunicorn", "-b", "0.0.0.0:8080", "app:app"] diff --git a/app_python/PYTHON.md b/app_python/PYTHON.md new file mode 100644 index 0000000000..81f9caa107 --- /dev/null +++ b/app_python/PYTHON.md @@ -0,0 +1,54 @@ +# Python Web Application + +## Framework Choice + +For this lab I used Flask via gunicorn because it is a simple and lightweight Python web framework. +It is a good choice for a small application that only needs to display the current time. + +## Best Practices Used + +- Dependencies are kept minimal. +- Runtime dependencies are separated from development dependencies. +- The application logic for Moscow time generation is separated into a dedicated function. +- The project structure is simple and clear. +- Tests are automated and can be executed locally and in CI. + +## Coding Standards + +- Code is written in PEP 8 format. +- Ruff is used as a linter. + +## Testing + +The application includes automated unit tests written with pytest. + +Implemented tests: + +- `test_get_moscow_time_format` checks that the Moscow time function returns time in the expected format. +- `test_index_returns_success_status_code` checks that the main route returns HTTP 200. +- `test_index_contains_expected_content` checks that the response contains the expected page text. +- `test_index_contains_time_value` checks that the page contains a generated time value. + +Testing best practices used: + +- Tests are isolated and do not require a running external server. +- Flask test client is used instead of manual browser checks. +- The time generation logic is separated into a function to make it easier to test. +- Tests validate both application logic and HTTP response behavior. +- Tests import the application as a normal Python package: `app_python.app`. + +Run tests from the repository root: + +```bash +python -m pytest app_python/tests -q +``` + +## Code Quality + +Code quality is checked automatically in CI with Ruff. + +Run linting from the repository root: + +```bash +ruff check app_python +``` diff --git a/app_python/README.md b/app_python/README.md new file mode 100644 index 0000000000..08ef25270a --- /dev/null +++ b/app_python/README.md @@ -0,0 +1,153 @@ +# Moscow Time App + +[![Python CI](https://github.com/SadPixels/DevOps-Core-Course/actions/workflows/python-ci.yml/badge.svg?branch=lab3)](https://github.com/SadPixels/DevOps-Core-Course/actions/workflows/python-ci.yml) + +## Overview + +This is a simple Python web application built with Flask. +It displays the current time in Moscow and updates it every time the page is refreshed. + +## Requirements + +- Python 3.11+ +- Flask 3.1.3+ +- gunicorn 25.3.0+ + +## Installation + +1. Create a virtual environment: + + ```bash + python3 -m venv venv + source venv/bin/activate + ``` + +2. Install dependencies: + + ```bash + pip install -r requirements.txt + ``` + +3. Launch application via Gunicorn: + + ```bash + gunicorn -b 127.0.0.1:8080 app:app + ``` + +## Usage + +```bash +gunicorn -b 127.0.0.1:8080 app:app +``` + +## Unit Tests + +The application includes automated unit tests written with pytest. + +The tests check: + +- Moscow time formatting. +- Successful response from the main route. +- Expected HTML content on the page. +- Presence of the generated time value in the response. + +Run tests from the repository root: + +```bash +pip install -r app_python/requirements.txt -r app_python/requirements-dev.txt +python -m pytest app_python/tests -q +``` + +Run the linter from the repository root: + +```bash +ruff check app_python +``` + +## CI Workflow + +The project uses GitHub Actions for continuous integration. + +The workflow includes: + +- Dependency installation. +- Python linting with Ruff. +- Unit tests with pytest. +- Snyk vulnerability checks. +- Docker image build. +- Docker Hub login. +- Docker image push. +- Python dependency cache. +- Docker build cache. + +The workflow runs only when files inside `app_python` or the workflow file itself are changed. + +## Docker + +Build the Docker image: + +```bash +docker build -t moscow-time-app:1.0.0 . +``` + +Run the container in the background: + +```bash +docker run --rm -d --name moscow-time-app -p 8080:8080 moscow-time-app:1.0.0 +``` + +Check that the application works: + +```bash +curl http://localhost:8080 +``` + +Check that the container is not running as root: + +```bash +docker exec moscow-time-app id +``` + +Stop the container: + +```bash +docker stop moscow-time-app +``` + +### Docker Hub + +Log in to Docker Hub: + +```bash +docker login +``` + +Tag the image: + +```bash +docker tag moscow-time-app:1.0.0 /moscow-time-app:1.0.0 +``` + +Push the image: + +```bash +docker push /moscow-time-app:1.0.0 +``` + +Pull the image: + +```bash +docker pull /moscow-time-app:1.0.0 +``` + +Run the pulled image: + +```bash +docker run --rm -d --name moscow-time-app -p 8080:8080 /moscow-time-app:1.0.0 +``` + +Check the pulled image: + +```bash +curl http://localhost:8080 +``` diff --git a/app_python/__init__.py b/app_python/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/app_python/app.py b/app_python/app.py new file mode 100644 index 0000000000..03771554ae --- /dev/null +++ b/app_python/app.py @@ -0,0 +1,32 @@ +from datetime import datetime +from zoneinfo import ZoneInfo + +from flask import Flask + +app = Flask(__name__) + + +def get_moscow_time() -> str: + """Return the current time in Moscow in a stable display format.""" + return datetime.now(ZoneInfo("Europe/Moscow")).strftime("%Y-%m-%d %H:%M:%S") + + +@app.route("/") +def index(): + moscow_time = get_moscow_time() + return f""" + + + Moscow Time + + +

Current time in Moscow

+

{moscow_time}

+

Refresh the page to update the time.

+ + + """ + + +if __name__ == "__main__": + app.run(host="127.0.0.1", port=8080) diff --git a/app_python/requirements-dev.txt b/app_python/requirements-dev.txt new file mode 100644 index 0000000000..71cc5395b4 --- /dev/null +++ b/app_python/requirements-dev.txt @@ -0,0 +1,2 @@ +pytest +ruff diff --git a/app_python/requirements.txt b/app_python/requirements.txt new file mode 100644 index 0000000000..fc8deec873 --- /dev/null +++ b/app_python/requirements.txt @@ -0,0 +1,2 @@ +Flask==3.1.3 +gunicorn==25.3.0 diff --git a/app_python/tests/test_app.py b/app_python/tests/test_app.py new file mode 100644 index 0000000000..72936c905d --- /dev/null +++ b/app_python/tests/test_app.py @@ -0,0 +1,36 @@ +import re + +from app_python.app import app, get_moscow_time + + +def test_get_moscow_time_format(): + result = get_moscow_time() + + assert re.fullmatch(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}", result) + + +def test_index_returns_success_status_code(): + client = app.test_client() + + response = client.get("/") + + assert response.status_code == 200 + + +def test_index_contains_expected_content(): + client = app.test_client() + + response = client.get("/") + body = response.get_data(as_text=True) + + assert "Current time in Moscow" in body + assert "Refresh the page to update the time." in body + + +def test_index_contains_time_value(): + client = app.test_client() + + response = client.get("/") + body = response.get_data(as_text=True) + + assert re.search(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}", body) diff --git a/lab1.md b/lab1.md deleted file mode 100644 index 30b74c95f5..0000000000 --- a/lab1.md +++ /dev/null @@ -1,65 +0,0 @@ -# Lab 1: Web Application Development - -## Overview - -In this lab assignment, you will develop a simple web application using Python and best practices. You will also have the opportunity to create a bonus web application using a different programming language. Follow the tasks below to complete the lab assignment. - -## Task 1: Python Web Application - -**6 Points:** - -1. Create `app_python` Folder: - - Create a folder named `app_python` to contain your Python web application files. - - Inside the `app_python` folder, create a file named `PYTHON.md`. - -2. Develop and Test Python Web Application: - - Develop a Python web application that displays the current time in Moscow. - - Choose a suitable framework for your web application and justify your choice in the `PYTHON.md` file. - - Implement best practices in your code and follow coding standards. - - Test your application to ensure the displayed time updates upon page refreshing. - -## Task 2: Well Decorated Description - -**4 Points:** - -1. Update `PYTHON.md`: - - Describe best practices applied in the web application. - - Explain how you followed coding standards, implemented testing, and ensured code quality. - -2. Create `README.md` in `app_python` folder: - - Use a Markdown template to document the Python web application. - -3. Ensure: - - Maintain a clean `.gitignore` file. - - Use a concise `requirements.txt` file for required dependencies. - -### List of Requirements - -- MSK Time timezone set up -- 2 PRs created -- README includes Overview -- Nice Markdown decoration -- Local installation details in README - -## Bonus Task: Additional Web Application - -**2.5 Points:** - -1. Create `app_*` Folder: - - Create a folder named `app_*` in the main project directory, replacing `*` with a programming language of your choice (other than Python). - - Inside the `app_*` folder, create a file named `*`.md. - -2. Develop Your Own Web App: - - Create a web application using the programming language you chose. - - Decide what your web application will display or do, and use your creativity. - -3. Follow Main Task Steps: - - Implement your bonus web application following the same suggestions and steps as the main Python web application task. - -### Guidelines - -- Use proper Markdown formatting and structure for the documentation files. We will use [online one](https://dlaa.me/markdownlint/) to check your `.md` files. -- Organize the files within the lab folder using appropriate naming conventions. -- Create a PR from your fork to the master branch of this repository and from your fork's branch to your fork's master branch with your completed lab assignment. - -> Note: Apply best practices, coding standards, and testing to your Python web application. Explore creativity in your bonus web application, and document your process using Markdown. diff --git a/lab10.md b/lab10.md deleted file mode 100644 index c472086168..0000000000 --- a/lab10.md +++ /dev/null @@ -1,91 +0,0 @@ -# Lab 10: Introduction to Helm - -## Overview - -In this lab, you will become familiar with Helm, set up a local development environment, and generate manifests for your application. - -## Task 1: Helm Setup and Chart Creation - -**6 Points:** - -1. Learn About Helm: - - Begin by exploring the architecture and concepts of Helm: - - [Helm Architecture](https://helm.sh/docs/topics/architecture/) - - [Understanding Helm Charts](https://helm.sh/docs/topics/charts/) - -2. Install Helm: - - Install Helm using the instructions provided: - - [Helm Installation](https://helm.sh/docs/intro/install/) - - [Chart Repository Initialization](https://helm.sh/docs/intro/quickstart/#initialize-a-helm-chart-repository) - -3. Create Your Own Helm Chart: - - Generate a Helm chart for your application. - - Inside the `k8s` folder, create a Helm chart template by using the command `helm create your-app`. - - Replace the default repository and tag inside the `values.yaml` file with your repository name. - - Modify the `containerPort` setting in the `deployment.yml` file. - - If you encounter issues with `livenessProbe` and `readinessProbe`, you can comment them out. - - > For troubleshooting, you can use the `minikube dashboard` command. - -4. Install Your Helm Chart: - - Install your custom Helm chart and ensure that all services are healthy. Verify this by checking the `Workloads` page in the Minikube dashboard. - -5. Access Your Application: - - Confirm that your application is accessible by running the `minikube service your_service_name` command. - -6. Create a HELM.md File: - - Construct a `HELM.md` file and provide the output of the `kubectl get pods,svc` command within it. - -## Task 2: Helm Chart Hooks - -**4 Points:** - -1. Learn About Chart Hooks: - - Familiarize yourself with [Helm Chart Hooks](https://helm.sh/docs/topics/charts_hooks/). - -2. Implement Helm Chart Hooks: - - Develop pre-install and post-install pods within your Helm chart, without adding any complex logic (e.g., use "sleep 20"). You can refer to [Example 1 in the guide](https://www.golinuxcloud.com/kubernetes-helm-hooks-examples/). - -3. Troubleshoot Hooks: - - Execute the following commands to troubleshoot your hooks: - 1. `helm lint ` - 2. `helm install --dry-run helm-hooks ` - 3. `kubectl get po` - -4. Provide Output: - - Execute the following commands and include their output in your report: - 1. `kubectl get po` - 2. `kubectl describe po ` - 3. `kubectl describe po ` - -5. Hook Delete Policy: - - Implement a hook delete policy to remove the hook once it has executed successfully. - -**List of Requirements:** - -- Helm Chart with Hooks implemented, including the hook delete policy. -- Output of the `kubectl get pods,svc` command in `HELM.md`. -- Output of all commands from the step 4 of Task 2 in `HELM.md`. - -## Bonus Task: Helm Library Chart - -**To Earn 2.5 Additional Points:** - -1. Helm Chart for Extra App: - - Prepare a Helm chart for an additional application. - -2. Helm Library Charts: - - Get acquainted with [Helm Library Charts](https://helm.sh/docs/topics/library_charts/). - -3. Create a Library Chart: - - Develop a simple library chart that includes a "labels" template. You can follow the steps outlined in [the Using Library Charts guide](https://austindewey.com/2020/08/17/how-to-reduce-helm-chart-boilerplate-with-library-charts/). Use this library chart for both of your applications. - -### Guidelines - -- Ensure your documentation is clear and well-structured. -- Include all the necessary components. -- Follow appropriate file and folder naming conventions. -- Create and participate in PRs for the peer review process. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Detailed documentation is crucial to ensure that your Helm deployment and hooks function as expected. Engage with the bonus tasks to further enhance your understanding and application deployment skills. diff --git a/lab11.md b/lab11.md deleted file mode 100644 index 4994bb1a80..0000000000 --- a/lab11.md +++ /dev/null @@ -1,85 +0,0 @@ -# Lab 11: Kubernetes Secrets and Hashicorp Vault - -## Overview - -In this lab, you will learn how to manage sensitive data, such as passwords, tokens, or keys, within Kubernetes. Additionally, you will configure CPU and memory limits for your application. - -## Task 1: Kubernetes Secrets and Resource Management - -**6 Points:** - -1. Create a Secret Using `kubectl`: - - Learn about Kubernetes Secrets and create a secret using the `kubectl` command: - - [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) - - [Managing Secrets with kubectl](https://kubernetes.io/docs/tasks/configmap-secret/managing-secret-using-kubectl/#decoding-secret) - -2. Verify and Decode Your Secret: - - Confirm and decode the secret, then create an `11.md` file within the `k8s` folder. Provide the output of the necessary commands inside this file. - -3. Manage Secrets with Helm: - - Use Helm to manage your secrets. - - Create a `secrets.yaml` file in the `templates` folder. - - Define a `secret` object within this YAML file. - - Add an `env` field to your `Deployment`. The path to update is: `spec.template.spec.containers.env`. - - > Refer to this [Helm Secrets Video](https://www.youtube.com/watch?v=hRSlKRvYe1A) for guidance. - - - Update your Helm deployment as instructed in the video. - - Retrieve the list of pods using the command `kubectl get po`. Use the name of the pod as proof of your success within the report. - - Verify your secret inside the pod, for example: `kubectl exec demo-5f898f5f4c-2gpnd -- printenv | grep MY_PASS`. Share this output in `11.md`. - -## Task 2: Vault Secret Management System - -**4 Points:** - -1. Install Vault Using Helm Chart: - - Install Vault using a Helm chart. Follow the steps provided in this guide: - - [Vault Installation Guide](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar#install-the-vault-helm-chart) - -2. Follow the Tutorial with Your Helm Chart: - - Adapt the tutorial to work with your Helm chart, including the following steps: - - [Set a Secret in Vault](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar#set-a-secret-in-vault) - - [Configure Kubernetes Authentication](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar#configure-kubernetes-authentication) - - Be cautious with the service account. If you used `helm create ...`, it will be created automatically. In the guide, they create it manually. - - [Manually Define a Kubernetes Service Account](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar#define-a-kubernetes-service-account) - -3. Implement Vault Secrets in Your Helm Chart: - - Use the steps from the guide as an example for your Helm chart: - - [Update values.yaml](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar#launch-an-application) - - [Add Labels](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-sidecar#inject-secrets-into-the-pod) - - Test to ensure your credentials are injected successfully. Use the `kubectl exec -it -- bash` command to access the container. Verify the injected secrets using `cat /path/to/your/secret` and `df -h`. Share the output in the `11.md` report. - - Apply a template as described in the guide. Test the updates as you did in the previous step and provide the outputs in `11.md`. - -**List of Requirements:** - -- Proof of work with a secret in `11.md` for the Task 1 - steps 2 and 3. -- `secrets.yaml` file. -- Resource requests and limits for CPU and memory. -- Vault configuration implemented, with proofs in `11.md`. - -## Bonus Task: Resource Management and Environment Variables - -**2.5 Points:** - -1. Read About Resource Management: - - Familiarize yourself with resource management in Kubernetes: - - [Resource Management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) - -2. Set Up Requests and Limits for CPU and Memory for Both Helm Charts: - - Configure resource requests and limits for CPU and memory for your application. - - Test to ensure these configurations work correctly. - -3. Add Environment Variables for Your Containers for Both Helm Charts: - - Read about Kubernetes environment variables: - - [Kubernetes Environment Variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) - - Update your Helm chart with several environment variables using named templates. Move these variables to the `_helpers.tpl` file: - - [Helm Named Templates](https://helm.sh/docs/chart_template_guide/named_templates/) - -### Guidelines - -- Ensure that your documentation is clear and organized. -- Include all the necessary components. -- Follow appropriate file and folder naming conventions. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Thorough documentation is essential to demonstrate your success in managing secrets and resource allocation in Kubernetes. Explore the bonus tasks to enhance your skills further. diff --git a/lab12.md b/lab12.md deleted file mode 100644 index efb72a29ec..0000000000 --- a/lab12.md +++ /dev/null @@ -1,68 +0,0 @@ -# Lab 12: Kubernetes ConfigMaps - -## Overview - -In this lab, you'll delve into Kubernetes ConfigMaps, focusing on managing non-confidential data and upgrading your application for persistence. ConfigMaps provide a way to decouple configuration artifacts from image content, allowing you to manage configuration data separately from the application. - -## Task 1: Upgrade Application for Persistence - -**6 Points:** - -1. Upgrade Your Application: - - Modify your application to: - - Implement a counter logic in your application to keep track of the number of times it's accessed. - - Save the counter number in the `visits` file. - - Introduce a new endpoint `/visits` to display the recorded visits. - - Test the changes: - - Update your `docker-compose.yml` to include a new volume with your `visits` file. - - Verify that the enhancements work as expected, you must see the updated number in the `visits` file on the host machine. - - Update the `README.md` for your application. - -## Task 2: ConfigMap Implementation - -**4 Points:** - -1. Understand ConfigMaps: - - Read about ConfigMaps in Kubernetes: - - [ConfigMaps](https://kubernetes.io/docs/concepts/configuration/configmap/) - -2. Mount a Config File: - - Create a `files` folder with a `config.json` file. - - Populate `config.json` with data in JSON format. - - Use Helm to mount `config.json`: - - Create a `configMap` manifest, extracting data from `config.json` using `.Files.Get`. - - Update `deployment.yaml` with `Volumes` and `VolumeMounts`. - - [Example](https://carlos.mendible.com/2019/02/10/kubernetes-mount-file-pod-with-configmap/) - - Install the updated Helm chart and verify success: - - Retrieve the list of pods: `kubectl get po`. - - Use the pod name as proof of successful deployment. - - Check the ConfigMap inside the pod, e.g., `kubectl exec demo-758cc4d7c4-cxnrn -- cat /config.json`. - -3. Documentation: - - Create `12.md` in the `k8s` folder and include the output of relevant commands. - -**List of Requirements:** - -- `config.json` in the `files` folder. -- `configMap` retrieving data from `config.json` using `.Files.Get`. -- `Volume`s and `VolumeMount`s in `deployments.yml`. -- `12.md` documenting the results of commands. - -## Bonus Task: ConfigMap via Environment Variables - -**2.5 Points:** - -1. Upgrade Bonus App: - - Implement persistence logic in your bonus app. - -2. ConfigMap via Environment Variables: - - Utilize ConfigMap via environment variables in a running container using the `envFrom` property. - - Provide proof with the output of the `env` command inside your container. - -### Guidelines - -- Maintain clear and organized documentation. -- Use appropriate naming conventions for files and folders. -- For your repository PR, ensure it's from the `lab12` branch to the main branch. - -> Note: Clear documentation is crucial to demonstrate successful data persistence and ConfigMap utilization in Kubernetes. Explore the bonus tasks to further enhance your skills. diff --git a/lab13.md b/lab13.md deleted file mode 100644 index e6f6c919f8..0000000000 --- a/lab13.md +++ /dev/null @@ -1,212 +0,0 @@ -# Lab 13: ArgoCD for GitOps Deployment - -## Overview - -In this lab, you will implement ArgoCD to automate Kubernetes application deployments using GitOps principles. You’ll install ArgoCD via Helm, configure it to manage your Python app, and simulate production-like workflows. - -## Task 1: Deploy and Configure ArgoCD - -**6 Points:** - -1. Install ArgoCD via Helm - - Add the ArgoCD Helm repository: - - ```bash - helm repo add argo https://argoproj.github.io/argo-helm - ``` - - [ArgoCD Helm Chart Docs](https://github.com/argoproj/argo-helm) - - - Install ArgoCD: - - ```bash - helm install argo argo/argo-cd --namespace argocd --create-namespace - ``` - - [ArgoCD Installation Guide](https://argo-cd.readthedocs.io/en/stable/getting_started/) - - - Verify installation: - - ```bash - kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=argocd-server -n argocd --timeout=90s - ``` - -2. Install ArgoCD CLI - - Install the ArgoCD CLI tool (required for command-line interactions): - - ```bash - # For macOS (Homebrew): - brew install argocd - - # For Debian/Ubuntu: - sudo apt-get install -y argocd - - # For other OS/architectures: - curl -sSL -o argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64 - chmod +x argocd - sudo mv argocd /usr/local/bin/ - ``` - - [ArgoCD CLI Docs](https://argo-cd.readthedocs.io/en/stable/cli_installation/) - - - Verify CLI installation: - - ```bash - argocd version - ``` - -3. Access the ArgoCD UI - - Forward the ArgoCD server port: - - ```bash - kubectl port-forward svc/argocd-server -n argocd 8080:443 & - ``` - - - Log in using the initial admin password: - - ```bash - # Retrieve the password: - kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 --decode - - # Log in via CLI: - argocd login localhost:8080 --insecure - argocd account login - ``` - - [ArgoCD Authentication Docs](https://argo-cd.readthedocs.io/en/stable/user-guide/accessing/) - -4. Configure Python App Sync - - Create an ArgoCD folder: - Add an `ArgoCD` folder in your `k8s` directory for ArgoCD manifests. - - - Define the ArgoCD Application: - Create `argocd-python-app.yaml` in the `ArgoCD` folder: - - ```yaml - apiVersion: argoproj.io/v1alpha1 - kind: Application - metadata: - name: python-app - namespace: argocd - spec: - project: default - source: - repoURL: https://github.com//S25-core-course-labs.git - targetRevision: lab13 - path: - helm: - valueFiles: - - values.yaml - destination: - server: https://kubernetes.default.svc - namespace: default - syncPolicy: - automated: {} - ``` - - [ArgoCD Application Manifest Docs](https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative_setup/) - - - Apply the configuration: - - ```bash - kubectl apply -f ArgoCD/argocd-python-app.yaml - ``` - - - Verify sync: - - ```bash - argocd app sync python-app - argocd app status python-app - ``` - -5. Test Sync Workflow - - Modify `values.yaml` (e.g., update `replicaCount`). - - Commit and push changes to the target branch from the config. - - Observe ArgoCD auto-sync the update: - - ```bash - argocd app status python-app - ``` - -### Task 2: Multi-Environment Deployment & Auto-Sync - -**4 Points:** - -1. Set Up Multi-Environment Configurations - - Extend your Python app’s Helm chart to support `dev` and `prod` environments. - - Create environment-specific values files (`values-dev.yaml`, `values-prod.yaml`). - -2. Create Namespaces - - ```bash - kubectl create namespace dev - kubectl create namespace prod - ``` - -3. Deploy Multi-Environment via ArgoCD - - Define two ArgoCD applications with auto-sync: - `argocd-python-dev.yaml` and `argocd-python-prod.yaml` (as before). - -4. Enable Auto-Sync - - Test auto-sync by updating `values-prod.yaml` and pushing to Git. - -5. Self-Heal Testing - - Test 1: Manual Override of Replica Count - 1. Modify the deployment’s replica count manually: - - ```bash - kubectl patch deployment python-app-prod -n prod --patch '{"spec":{"replicas": 3}}' - ``` - - 2. Observe ArgoCD auto-revert the change (due to `syncPolicy.automated`): - - ```bash - argocd app sync python-app-prod - argocd app status python-app-prod - ``` - - - Test 2: Delete a Pod (Replica) - 1. Delete a pod in the `prod` namespace: - - ```bash - kubectl delete pod -n prod -l - ``` - - 2. Verify Kubernetes recreates the pod to match the deployment’s `replicaCount`: - - ```bash - kubectl get pods -n prod -w - ``` - - 3. Confirm ArgoCD shows no drift (since pod deletions don’t affect the desired state): - - ```bash - argocd app diff python-app-prod - ``` - -6. Documentation - - In `13.md`, include: - - Output of `kubectl get pods -n prod` before and after pod deletion. - - Screenshots of ArgoCD UI showing sync status and the dashboard after both tests. - - Explanation of how ArgoCD handles configuration drift vs. runtime events. - -## Bonus Task: Sync Your Bonus App with ArgoCD - -**2.5 Points:** - -1. Configure ArgoCD for Bonus App - - Create an `argocd--app.yaml` similar to Task 1, pointing to your bonus app’s helm chart folder. - - Sync and validate deployment with: - - ```bash - kubectl get pods -n - ``` - -### Guidelines - -- Follow the [ArgoCD docs](https://argo-cd.readthedocs.io/) for advanced configurations. -- Use consistent naming conventions (e.g., `lab13` branch for Git commits). -- Document all steps in `13.md` (include diffs, outputs, and UI screenshots). -- For your repository PR, ensure it's from the `lab14` branch to the main branch. - -> **Note**: This lab emphasizes GitOps workflows, environment isolation, and automation. Mastery of ArgoCD will streamline your CI/CD pipelines in real-world scenarios. diff --git a/lab14.md b/lab14.md deleted file mode 100644 index d1d6ba51cd..0000000000 --- a/lab14.md +++ /dev/null @@ -1,106 +0,0 @@ -# Lab 14: Kubernetes StatefulSet - -## Overview - -In this lab, you'll explore Kubernetes StatefulSets, focusing on managing stateful applications with guarantees about the ordering and uniqueness of a set of Pods. - -## Task 1: Implement StatefulSet in Helm Chart - -**6 Points:** - -1. Understand StatefulSets: - - Read about StatefulSet objects: - - [Concept](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) - - [Tutorial](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/) - -2. Update Helm Chart: - - Rename `deployment.yml` to `statefulset.yml`. - - Create a manifest for StatefulSet following the tutorial. - - Test with command: `helm install --dry-run --debug name_of_your_chart path_to_your_chart`. - - Fix any issues and deploy it. - - Apply best practices by moving values to variables in `values.yml` meaningfully. - -## Task 2: StatefulSet Exploration and Optimization - -**4 Points:** - -1. Research and Documentation: - - Create `14.md` report. - - Include the output of `kubectl get po,sts,svc,pvc` commands. - - Use `minikube service name_of_your_statefulset` command to access your app. - - Access the root path of your app from different tabs and modes in your browser. - - Check the content of your file in each pod, e.g., `kubectl exec pod/demo-0 -- cat visits`, and provide the output for all replicas. - - Describe and explain differences in the report. - -2. Persistent Storage Validation - - Delete a pod: - - ```bash - kubectl delete pod app-stateful-0 - ``` - - - Verify that the PVC and data persist: - - ```bash - kubectl get pvc - kubectl exec app-stateful-0 -- cat /data/visits - ``` - -3. Headless Service Access - - Access pods via DNS: - - ```bash - kubectl exec app-stateful-0 -- nslookup app-stateful-1.app-stateful - ``` - - - Document DNS resolution in `14.md`. - -4. Monitoring & Alerts - - Add liveness/readiness probes to your StatefulSet. - - Describe in `14.md`: - - How probes ensure pod health. - - Why they’re critical for stateful apps. - -5. Ordering Guarantee and Parallel Operations: - - Explain why ordering guarantees are unnecessary for your app. - - Implement a way to instruct the StatefulSet controller to launch or terminate all Pods in parallel. - -**List of Requirements:** - -- Outputs of commands in `14.md`. -- Results of the "number of visits" command for each pod, with an explanation in `14.md`. -- Answers to questions in point 2 of `14.md`. -- Implementation of parallel launch and terminate. - -## Bonus Task: Update Strategies - -**2.5 Points:** - -1. Apply StatefulSet to Bonus App - - Convert your bonus app’s Helm chart to use a StatefulSet. - -2. Explore Update Strategies - - Implement Rolling Updates: - - ```yaml - spec: - updateStrategy: - type: RollingUpdate - rollingUpdate: - partition: 1 - ``` - - - Test Canaries: - Update a subset of pods first. - - - Document in `14.md`: - - Explain `OnDelete`, `RollingUpdate`, and their use cases. - - Compare with Deployment update strategies. - -### Guidelines - -- Maintain clear and organized documentation. -- Use appropriate naming conventions for files and folders. -- For your repository PR, ensure it's from the `lab14` branch to the main branch. - -> Note: Understanding StatefulSets and their optimization is crucial for managing stateful applications in Kubernetes. Explore the bonus tasks to further enhance your skills. diff --git a/lab15.md b/lab15.md deleted file mode 100644 index 887587145d..0000000000 --- a/lab15.md +++ /dev/null @@ -1,78 +0,0 @@ -# Lab 15: Kubernetes Monitoring and Init Containers - -## Overview - -In this lab, you will explore Kubernetes cluster monitoring using Prometheus with the Kube Prometheus Stack. Additionally, you'll delve into the concept of Init Containers in Kubernetes. - -## Task 1: Kubernetes Cluster Monitoring with Prometheus - -**6 Points:** - -1. This lab was tested on a specific version of components: - - Minikube v1.33.0 - - Minikube kubectl v1.28.3 - - kube-prometheus-stack-57.2.0 v0.72.0 - - the minikube start command - `minikube start --driver=docker --container-runtime=containerd` - -2. Read about `Kube Prometheus Stack`: - - [Helm chart with installation guide](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack) - - [Explanation of components](https://github.com/prometheus-operator/kube-prometheus#kubeprometheus) - -3. Describe Components: - - Create `15.md` and detail the components of the Kube Prometheus Stack, explaining their roles and functions. Avoid direct copy-pasting; provide a personal understanding. - -4. Install Helm Charts: - - Install the Kube Prometheus Stack to your Kubernetes cluster. - - Install your app's Helm chart. - - Provide the output of the `kubectl get po,sts,svc,pvc,cm` command in the report and explain each part. - -5. Utilize Grafana Dashboards: - - Access Grafana using `minikube service monitoring-grafana`. - - Explore existing dashboards to find information about your cluster: - 1. Check CPU and Memory consumption of your StatefulSet. - 2. Identify Pods with higher and lower CPU usage in the default namespace. - 3. Monitor node memory usage in percentage and megabytes. - 4. Count the number of pods and containers managed by the Kubelet service. - 5. Evaluate network usage of Pods in the default namespace. - 6. Determine the number of active alerts; also check the Web UI with `minikube service monitoring-kube-prometheus-alertmanager`. - - Provide answers to all these points in the report. - -## Task 2: Init Containers - -**4 Points:** - -1. Read about `Init Containers`: - - [Concept](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) - - [Tutorial](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container) - -2. Implement Init Container: - - Create a new Volume. - - Implement an Init container to download any file using `wget` (you can use a site from the example). - - Provide proof of success, e.g., `kubectl exec pod/demo-0 -- cat /test.html`. - -**List of Requirements:** - -- Detailed explanation of monitoring stack components in `15.md`. -- Output and explanation of `kubectl get po,sts,svc,pvc,cm`. -- Answers to all 6 questions from point 4 in `15.md`. -- Implementation of Init Container. -- Proof of Init Container downloading a file. - -## Bonus Task: App Metrics & Multiple Init Containers - -**2.5 Points:** - -1. App Metrics: - - Fetch metrics from your app and provide proof. - -2. Init Container Queue: - - Create a queue of three Init containers, with any logic like adding new lines to the same file. - - Provide proof using the `cat` tool. - -### Guidelines - -- Ensure clear and organized documentation. -- Use appropriate naming conventions for files and folders. -- For your repository PR, ensure it's from the `lab15` branch to the main branch. - -> Note: Demonstrate successful implementation and understanding of Kubernetes monitoring and Init Containers. Take your time to explore the bonus tasks for additional learning opportunities. diff --git a/lab16.md b/lab16.md deleted file mode 100644 index 37912fc50b..0000000000 --- a/lab16.md +++ /dev/null @@ -1,75 +0,0 @@ -# Lab 16: IPFS and Fleek - -In this lab, you will explore essential DevOps tools and set up a project on the Fleek service. Follow the tasks below to complete the lab assignment. - -## Task 1: Set Up an IPFS Gateway Using Docker - -Objective: Understand and implement an IPFS gateway using Docker, upload a file, and verify it via an IPFS cluster. - -1. Set Up IPFS Gateway: - - Install Docker on your machine if it's not already installed. - - [Docker Installation Guide](https://docs.docker.com/get-docker/) - - - Pull the IPFS Docker image and run an IPFS container: - - ```sh - docker pull ipfs/go-ipfs - docker run -d --name ipfs_host -v /path/to/folder/with/file:/export -v ipfs_data:/data/ipfs -p 8080:8080 -p 4001:4001 -p 5001:5001 ipfs/go-ipfs - ``` - - - Verify the IPFS container is running: - - ```sh - docker ps - ``` - -2. Upload a File to IPFS: - - Open a browser and access the IPFS web UI: - - ```sh - http://127.0.0.1:5001/webui/ - ``` - - - Explore the web UI and wait for 5 minutes to sync up with the network. - - Upload any file via the web UI. - - Use the obtained hash to access the file via any public IPFS gateway. Here are a few options: - - [IPFS.io Gateway](https://ipfs.io/ipfs/) - - [Cloudflare IPFS Gateway](https://cloudflare-ipfs.com/ipfs/) - - [Infura IPFS Gateway](https://ipfs.infura.io/ipfs/) - - - Append your file hash to any of the gateway URLs to verify your file is accessible. Note that it may fail due to network overload, so don't worry if you can't reach it. - -3. Documentation: - - Create a `submission2.md` file. - - Share information about connected peers and bandwidth in your report. - - Provide the hash and the URLs used to verify the file on the IPFS gateways. - -## Task 2: Set Up Project on Fleek.xyz - -Objective: Set up a project on the Fleek service and share the IPFS link. - -1. Research: - - Understand what IPFS is and its purpose. - - Explore Fleek's features. - -2. Set Up: - - Sign up for a Fleek account if you haven't already. - - Use your fork of the Labs repository as your project source. Optionally, set up your own website (notify us in advance). - - Configure the project settings on Fleek. - - Deploy the Labs repository to Fleek, ensuring it is uploaded to IPFS. - -3. Documentation: - - Share the IPFS link and domain of the deployed project in the `submission2.md` file. - -## Additional Resources - -- [IPFS Documentation](https://docs.ipfs.io/) -- [Fleek Documentation](https://docs.fleek.xyz/) - -### Guidelines - -- Use proper Markdown formatting for documentation files. -- Organize files with appropriate naming conventions. -- Create a Pull Request to the main branch of the repository with your completed lab assignment. - -> Note: Actively explore and document your findings to gain hands-on experience with IPFS and Fleek. diff --git a/lab16/index.html b/lab16/index.html deleted file mode 100644 index acce39eee3..0000000000 --- a/lab16/index.html +++ /dev/null @@ -1,303 +0,0 @@ - - - - - - DevOps Engineering Expert Track - - - - -
-
- -
-
- -
-
-

Master Modern DevOps Practices

-

16 hands-on labs covering Kubernetes, Terraform, CI/CD, and more

- Start Free Trial → -
-
- -
-

Why This Course?

-
-
- -

16 Advanced Labs

-

Build production-ready systems from scratch

-
-
- -

Industry-Standard Tools

-

Terraform, ArgoCD, Prometheus, Vault, and more

-
-
- -

Job-Ready Skills

-

Learn tools used by top tech companies

-
-
-
- -
-
-

Lab Syllabus (2025 Edition)

-
    -
  1. Lab 1: Web Application Development
    2. -
  2. Lab 2: Containerization
    3. -
  3. Lab 3: Continuous Integration
    4. -
  4. Lab 4: Infrastructure as Code & Terraform
    5. -
  5. Lab 5: Configuration Management
    6. -
  6. Lab 6: Ansible Automation
    7. -
  7. Lab 7: Observability, Logging, Loki Stack
    8. -
  8. Lab 8: Monitoring & Prometheus
    9. -
  9. Lab 9: Kubernetes & Declarative Manifests
    10. -
  10. Lab 10: Helm Charts & Library Charts
    11. -
  11. Lab 11: Kubernetes Secrets Management (Vault, ConfigMaps)
    12. -
  12. Lab 12: Kubernetes ConfigMaps & Environment Variables
    13. -
  13. Lab 13: GitOps with ArgoCD
    14. -
  14. Lab 14: StatefulSet Optimization
    15. -
  15. Lab 15: Kubernetes Monitoring & Init Containers
    16. -
  16. Lab 16: IPFS & Fleek Decentralization
    17. -
-
-
- -
-

Learning Progression

-
-

Phase 1: Foundations (Labs 1-6)

-

Web Dev → Containers → CI/CD → IaC → Ansible

-
-
-

Phase 2: Observability (Labs 7-8)

-

Logging → Monitoring → Loki/Prometheus

-
-
-

Phase 3: Kubernetes Mastery (Labs 9-12)

-

Deployments → Helm → Secrets → ConfigMaps

-
-
-

Phase 4: Expert Track (Labs 13-16)

-

GitOps → StatefulSets → IPFS → Final Project

-
-
- - - - - - \ No newline at end of file diff --git a/lab2.md b/lab2.md deleted file mode 100644 index ff71bc227d..0000000000 --- a/lab2.md +++ /dev/null @@ -1,85 +0,0 @@ -# Lab 2: Containerization - Docker - -## Overview - -In this lab assignment, you will learn to containerize applications using Docker, while focusing on best practices. Additionally, you will explore Docker multi-stage builds. Follow the tasks below to complete the lab assignment. - -## Task 1: Dockerize Your Application - -**6 Points:** - -1. Create a `Dockerfile`: - - Inside the `app_python` folder, craft a `Dockerfile` for your application. - - Research and implement Docker best practices. Utilize a Dockerfile linter for quality assurance. - -2. Build and Test Docker Image: - - Build a Docker image using your Dockerfile. - - Thoroughly test the image to ensure it functions correctly. - -3. Push Image to Docker Hub: - - If you lack a public Docker Hub account, create one. - - Push your Docker image to your public Docker Hub account. - -4. Run and Verify Docker Image: - - Retrieve the Docker image from your Docker Hub account. - - Execute the image and validate its functionality. - -## Task 2: Docker Best Practices - -**4 Points:** - -1. Enhance your docker image by implementing [Docker Best Practices](https://docs.docker.com/build/building/best-practices/). - - No root user inside, or you will get no points at all. - -2. Write `DOCKER.md`: - - Inside the `app_python` folder, create a `DOCKER.md` file. - - Elaborate on the best practices you employed within your Dockerfile. - - Implementing and listing numerous Docker best practices will earn you more points. - -3. Enhance the README.md: - - Update the `README.md` file in the `app_python` folder. - - Include a dedicated `Docker` section, explaining your containerized application and providing clear instructions for execution. - - How to build? - - How to pull? - - How to run? - -### List of Requirements - -- Rootless container. -- Use COPY, but only specific files. -- Layer sanity. -- Use `.dockerignore`. -- Use a precise version of your base image and language, example `python:3-alpine3.15`. - -## Bonus Task: Multi-Stage Builds Exploration - -**2.5 Points:** - -1. Dockerize Previous App: - - Craft a `Dockerfile` for the application from the prior lab. - - Place this Dockerfile within the corresponding `app_*` folder. - -2. Follow Main Task Guidelines: - - Apply the same steps and suggestions as in the primary Dockerization task. - -3. Study Docker Multi-Stage Builds: - - Familiarize yourself with Docker multi-stage builds. - - Consider implementing multi-stage builds, only if they enhance your project's structure and efficiency. - -4. Study Distroless Images: - - Explore how to use Distroless images by reviewing the official documentation: [GoogleContainerTools/distroless](https://github.com/GoogleContainerTools/distroless). - - Create new `distroless.Dockerfile` files for your Python app and your second app. - - Use the `nonroot` tag for both images to ensure they run with non-root privileges. - - Verify that the applications work correctly with the Distroless images. - - Compare the sizes of your previous Docker images with the new Distroless-based images. - - In the `DOCKER.md` file, describe the differences between the Distroless images and your previous images. Explain why these differences exist (e.g., smaller size, reduced attack surface, etc.). - - Include a screenshot of your final results (e.g., image sizes). - - Add a new section to the `README.md` file titled "Distroless Image Version". - -### Guidelines - -- Utilize appropriate Markdown formatting and structure for all documentation. -- Organize files within the lab folder with suitable naming conventions. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Utilize Docker to containerize your application, adhering to best practices. Explore Docker multi-stage builds for a deeper understanding, and document your process using Markdown. diff --git a/lab4.md b/lab4.md deleted file mode 100644 index e88b5e63e5..0000000000 --- a/lab4.md +++ /dev/null @@ -1,84 +0,0 @@ -# Lab 4: Infrastructure as Code Lab - -## Overview - -In this lab assignment, you will explore Infrastructure as Code (IAC) using Terraform. You'll build Docker and AWS infrastructures and dive into managing GitHub repositories through Terraform. Additionally, there are bonus tasks to enhance your Terraform skills. Follow the tasks below to complete the lab assignment. - -## Task 1: Introduction to Terraform - -**6 Points:** - -0. You will need a VPN tool for this lab - -1. Get Familiar with Terraform: - - Begin by familiarizing yourself with Terraform by reading the [introduction](https://www.terraform.io/intro/index.html) and exploring [best practices](https://www.terraform.io/docs/cloud/guides/recommended-practices/index.html). - -2. Set Up Terraform Workspace: - - Create a `terraform` folder to organize your Terraform workspaces. - - Inside the `terraform` folder, create a file named `TF.md`. - -3. Docker Infrastructure Using Terraform: - - Follow the [Docker tutorial](https://learn.hashicorp.com/collections/terraform/docker-get-started) for building a Docker infrastructure with Terraform. - - Perform the following tasks as instructed in the tutorial: - - Install Terraform. - - Build the Infrastructure. - - Provide the output of the following commands in the `TF.md` file: - - ```sh - terraform state show - terraform state list - ``` - - - Document a part of the log with the applied changes. - - Utilize input variables to rename your Docker container. - - Finish the tutorial and provide the output of the `terraform output` command in the `TF.md` file. - -4. Yandex Cloud Infrastracture Using Terraform: - - Create an account on [Yandex Cloud](https://cloud.yandex.com/). - - Check for available free-tier options and select a free VM instance suitable for this lab. - - Follow the [Yandex Quickstart Guide](https://yandex.cloud/en-ru/docs/tutorials/infrastructure-management/terraform-quickstart#linux_1) to set up and configure Terraform for managing Yandex Cloud resources. - - Document the entire process, including setup steps, configurations, and any challenges encountered, in the `TF.md` file. - -5. [Optioinal] AWS Infrastructure Using Terraform: - - Follow the [AWS tutorial](https://learn.hashicorp.com/tutorials/terraform/aws-build?in=terraform/aws-get-started) alongside the instructions from the previous step. - -## Task 2: Terraform for GitHub - -**4 Points:** - -1. GitHub Infrastructure Using Terraform: - - Utilize the [GitHub provider for Terraform](https://registry.terraform.io/providers/integrations/github/latest/docs). - - Create a directory inside the `terraform` folder specifically for managing your GitHub project infrastructure. - - Build GitHub infrastructure following a reference like [this example](https://dev.to/pwd9000/manage-and-maintain-github-with-terraform-2k86). Prepare `.tf` files that include: - - Repository name - - Repository description - - Visibility settings - - Default branch - - Branch protection rule for the default branch - - Avoid placing your token as a variable in the code; instead, use an environment variable. - -2. Import Existing Repository: - - Use the `terraform import` command to import your current GitHub repository into your Terraform configuration. No need to create a new one. Example: `terraform import "github_repository.core-course-labs" "core-course-labs"`. - -3. Apply Terraform Changes: - - Apply changes from your Terraform configuration to your GitHub repository. - -4. Document Best Practices: - - Provide Terraform-related best practices that you applied in the `TF.md` file. - -## Bonus Task: Adding Teams - -**2.5 Points:** - -1. GitHub Teams Using Terraform: - - You need to create a new organization. - - Extend your Terraform configuration to add several teams to your GitHub repository, each with different levels of access. - - Apply the changes and ensure they take effect in your GitHub repository. - -### Guidelines - -- Use proper Markdown formatting and structure for documentation files. -- Organize files within the lab folder with suitable naming conventions. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Dive into Terraform to manage infrastructures efficiently. Explore the AWS and Docker tutorials, and don't forget to document your process and best practices in the `TF.md` file. diff --git a/lab5.md b/lab5.md deleted file mode 100644 index bead18ceea..0000000000 --- a/lab5.md +++ /dev/null @@ -1,141 +0,0 @@ -# Lab 5: Ansible and Docker Deployment - -## Overview - -In this lab, you will get acquainted with Ansible, a powerful configuration management and automation tool. Your objective is to use Ansible to deploy Docker on a newly created cloud VM. This knowledge will be essential for your application deployment in the next lab. - -## Task 1: Initial Setup - -**6 Points:** - -1. Repository Structure: - - Organize your repository following the recommended structure below: - - ```sh - . - |-- README.md - |-- ansible - | |-- inventory - | | `-- default_aws_ec2.yml - | |-- playbooks - | | `-- dev - | | `-- main.yaml - | |-- roles - | | |-- docker - | | | |-- defaults - | | | | `-- main.yml - | | | |-- handlers - | | | | `-- main.yml - | | | |-- tasks - | | | | |-- install_compose.yml - | | | | |-- install_docker.yml - | | | | `-- main.yml - | | | `-- README.md - | | `-- web_app - | | |-- defaults - | | | `-- main.yml - | | |-- handlers - | | | `-- main.yml - | | |-- meta - | | | `-- main.yml - | | |-- tasks - | | | `-- main.yml - | | `-- templates - | | `-- docker-compose.yml.j2 - | `-- ansible.cfg - |-- app_go - |-- app_python - `-- terraform - ``` - -2. Installation and Introduction: - - Install Ansible and familiarize yourself with its basics. You can follow the [Ansible installation guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html). - -3. Use an Existing Ansible Role for Docker: - - Utilize an existing Ansible role for Docker from `ansible-galaxy` as a template. You can explore [this Docker role](https://github.com/geerlingguy/ansible-role-docker) as an example. - -4. Create a Playbook and Testing: - - Develop an Ansible playbook for deploying Docker. - - Test your playbook to ensure it works as expected. - -## Task 2: Custom Docker Role - -**4 Points:** - -1. Create Your Custom Docker Role: - - Develop a custom Ansible role for Docker with the following tasks: - 1. Install Docker and Docker Compose. - 2. Update your playbook to utilize this custom role. [Tricks and Tips](https://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html). - 3. Test your playbook with the custom role to ensure successful deployment. - 4. Make sure the role has a task to configure Docker to start on boot (`systemctl enable docker`). - 5. Include a task to add the current user to the `docker` group to avoid using `sudo` for Docker commands. - -2. Documentation: - - Develop an `ANSIBLE.md` file in the `ansible` folder to document your Ansible-related work. - - Create a `README.md` file in the `ansible/roles/docker` folder. - - Use a Markdown template to describe your Docker role, its requirements and usage. - - Example `README.md` template for the Docker role: - - ```markdown - # Docker Role - - This role installs and configures Docker and Docker Compose. - - ## Requirements - - - Ansible 2.9+ - - Ubuntu 22.04 - - ## Role Variables - - - `docker_version`: The version of Docker to install (default: `latest`). - - `docker_compose_version`: The version of Docker Compose to install (default: `1.29.2`). - - ## Example Playbook - - ```yaml - - hosts: all - roles: - - role: docker - ``` - -3. Deployment Output: - - Execute your playbook to deploy the Docker role. - - Provide the last 50 lines of the output from your deployment command in the `ANSIBLE.md` file. - - Use the `--check` flag with `ansible-playbook` to perform a dry run and verify changes before applying them. - - Example command: - - ```sh - ansible-playbook --diff - ``` - -4. **Inventory Details:** - - Execute the following command `ansible-inventory -i .yaml --list` and provide its output in the `ANSIBLE.md` file. - - Validate the inventory file using `ansible-inventory -i .yaml --graph` to visualize the inventory structure. - - Ensure you have documented the inventory information. - -## Bonus Task: Dynamic Inventory - -**2.5 Points:** - -1. Set up Dynamic Inventory: - - Implement dynamic inventory for your cloud environment, if available. - - You may explore ready-made solutions for dynamic inventories: - - - [AWS Example](https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_ec2_inventory.html) - - [Yandex Cloud (Note: Not tested)](https://github.com/rodion-goritskov/yacloud_compute) - - Implementing dynamic inventory can enhance your automation capabilities. - -2. Secure Docker Configuration: - - Add a task to configure Docker security settings, disable root access. - - Use the `copy` module and modify the `daemon.json` file. - -### Guidelines - -- Use proper Markdown formatting and structure for documentation files. -- Organize files within the lab folder with suitable naming conventions. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Ensure that your repository is well-structured, follow Ansible best practices, and provide clear documentation for a successful submission. diff --git a/lab6.md b/lab6.md deleted file mode 100644 index cc8249390d..0000000000 --- a/lab6.md +++ /dev/null @@ -1,139 +0,0 @@ -# Lab 6: Ansible and Application Deployment - -## Overview - -In this lab, you will utilize Ansible to set up a Continuous Deployment (CD) process for your application. - -## Task 1: Application Deployment - -**6 Points:** - -1. Create an Ansible Role: - - Develop an Ansible role specifically for deploying your application's Docker image, it can be done manually or via `ansible-galaxy init roles/web_app`. Call it `web_app`. - - Define variables in `roles/web_app/defaults/main.yml`. - - Add tasks to `roles/web_app/tasks/main.yml` to pull the Docker image and start the container. - - > Managing just a container is bad practice, you can omit it and move to the Task 2 directly. - -2. Update the Playbook: - - Modify your Ansible playbook to integrate the new role you've created for Docker image deployment. - -3. Deployment Output: - - Execute your playbook to deploy the role. - - Provide the last 50 lines of the output from your deployment command in the `ANSIBLE.md` file. - -## Task 2: Ansible Best Practices - -**4 Points:** - -1. Group Tasks with Blocks: - - Organize related tasks within your playbooks using Ansible blocks. - - Implement logical blocks. For example: - - ```yaml - - name: Setup Docker Environment - block: - - name: Install Docker - apt: - name: docker.io - state: present - - - name: Start Docker Service - service: - name: docker - state: started - enabled: yes - tags: - - setup - ``` - -2. Role Dependency: - - Set the role dependency for your `web_app` role to include the `docker` role. - - Specify dependencies in `roles/web_app/meta/main.yml`. - -3. Apply Tags: - - Implement Ansible tags to group tasks logically and enable selective execution. For example: - - ```yaml - - name: Pull Docker image - docker_image: - name: "{{ docker_image }}" - source: pull - tags: - - docker - ``` - - - Run specific tags. For example: - - ```bash - ansible-playbook site.yml --tags docker - ``` - -4. Wipe Logic: - - Create a wipe logic in `roles/web_app/tasks/0-wipe.yml`. This should include removing your Docker container and all related files. - - Ensure that this wipe process can be enabled or disabled by using a variable, for example, `web_app_full_wipe=true`. - -5. Separate Tag for Wipe: - - Utilize a distinct tag for the **Wipe** section of your Ansible playbook. This allows you to run the wipe tasks independently from the main tasks. - -6. Docker Compose File: - - Write a Jinja2 template (`roles/web_app/templates/docker-compose.yml.j2`). For example: - - ```yaml - version: '3' - services: - app: - image: "{{ docker_image }}" - ports: - - "{{ app_port }}:80" - ``` - - - Deliver the template using the `template` module in `roles/web_app/tasks/main.yml`. - - Suggested structure: - - ```sh - . - |-- defaults - | `-- main.yml - |-- meta - | `-- main.yml - |-- tasks - | |-- 0-wipe.yml - | `-- main.yml - `-- templates - `-- docker-compose.yml.j2 - ``` - -7. Create `README.md`: - - Create a `README.md` file in the `ansible/roles/web_app` folder. - - Use a suggested Docker Markdown template from the previous lab to describe your role, its requirements and usage. - -## Bonus Task: CD Improvement - -**2.5 Points:** - -1. Create an Extra Playbook: - - Develop an additional Ansible playbook specifically for your bonus application. - - You can reuse the existing Ansible role you created for your primary application or create a new one. - - Suggested structure: - - ```sh - . - `--ansible - `-- playbooks - `-- dev - |-- app_python - | `-- main.yaml - `-- app_go - `-- main.yaml - ``` - -### Guidelines - -- Use proper Markdown formatting and structure for documentation files. -- Organize files within the lab folder with suitable naming conventions. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. -- Follow the suggested structure for your Ansible roles, tasks, and templates. -- Utilize Ansible best practices such as grouping tasks with blocks, applying tags, and separating roles logically. - -> Note: Apply diligence to your Ansible implementation, follow best practices, and clearly document your work to achieve the best results in this lab assignment. diff --git a/lab7.md b/lab7.md deleted file mode 100644 index 48e65eb202..0000000000 --- a/lab7.md +++ /dev/null @@ -1,59 +0,0 @@ -# Lab 7: Monitoring and Logging - -## Overview - -In this lab, you will become familiar with a logging stack that includes Promtail, Loki, and Grafana. Your goal is to create a Docker Compose configuration and configuration files to set up this logging stack. - -## Task 1: Logging Stack Setup - -**6 Points:** - -1. Study the Logging Stack: - - Begin by researching the components of the logging stack: - - [Grafana Webinar: Loki Getting Started](https://grafana.com/go/webinar/loki-getting-started/) - - [Loki Overview](https://grafana.com/docs/loki/latest/overview/) - - [Loki GitHub Repository](https://github.com/grafana/loki) - -2. Create a Monitoring Folder: - - Start by creating a new folder named `monitoring` in your project directory. - -3. Docker Compose Configuration: - - Inside the `monitoring` folder, prepare a `docker-compose.yml` file that defines the entire logging stack along with your application. - - To assist you in this task, refer to these resources for sample Docker Compose configurations: - - [Example Docker Compose Configuration from Loki Repository](https://github.com/grafana/loki/blob/main/production/docker-compose.yaml) - - [Promtail Configuration Example](https://github.com/black-rosary/loki-nginx/blob/master/promtail/promtail.yml) (Adapt it as needed) - -4. Testing: - - Verify that the configured logging stack and your application work as expected. - -## Task 2: Documentation and Reporting - -**4 Points:** - -1. Logging Stack Report: - - Create a new file named `LOGGING.md` to document how the logging stack you've set up functions. - - Provide detailed explanations of each component's role within the stack. - -2. Screenshots: - - Capture screenshots that demonstrate the successful operation of your logging stack. - - Include these screenshots in your `LOGGING.md` report for reference. - -## Bonus Task: Additional Configuration - -**2.5 Points:** - -1. Integrating Your Extra App: - - Extend the `docker-compose.yml` configuration to include your additional application. - -2. Configure Stack for Comprehensive Logging: - - Modify the logging stack's configuration to collect logs from all containers defined in the `docker-compose.yml`. - - Include screenshots in your `LOGGING.md` report to demonstrate your success. - -### Guidelines - -- Ensure that your documentation in `LOGGING.md` is well-structured and comprehensible. -- Follow proper naming conventions for files and folders. -- Use code blocks and Markdown formatting where appropriate. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Thoroughly document your work, and ensure the logging stack functions correctly. Utilize the bonus points opportunity to enhance your understanding and the completeness of your setup. diff --git a/lab8.md b/lab8.md deleted file mode 100644 index 8eb0752ec7..0000000000 --- a/lab8.md +++ /dev/null @@ -1,71 +0,0 @@ -# Lab 8: Monitoring with Prometheus - -## Overview - -In this lab, you will become acquainted with Prometheus, set it up, and configure applications to collect metrics. - -## Task 1: Prometheus Setup - -**6 Points:** - -1. Learn About Prometheus: - - Begin by reading about Prometheus and its fundamental concepts: - - [Prometheus Overview](https://prometheus.io/docs/introduction/overview/) - - [Prometheus Naming Best Practices](https://prometheus.io/docs/practices/naming/) - -2. Integration with Docker Compose: - - Expand your existing `docker-compose.yml` file from the previous lab to include Prometheus. - -3. Prometheus Configuration: - - Configure Prometheus to collect metrics from both Loki and Prometheus containers. - -4. Verify Prometheus Targets: - - Access `http://localhost:9090/targets` to ensure that Prometheus is correctly scraping metrics. - - Capture screenshots that confirm the successful setup and place them in a file named `METRICS.md` within the monitoring folder. - -## Task 2: Dashboard and Configuration Enhancements - -**4 Points:** - -1. Grafana Dashboards: - - Set up dashboards in Grafana for both Loki and Prometheus. - - You can use examples as references: - - [Example Dashboard for Loki](https://grafana.com/grafana/dashboards/13407) - - [Example Dashboard for Prometheus](https://grafana.com/grafana/dashboards/3662) - - Capture screenshots displaying your successful dashboard configurations and include them in `METRICS.md`. - -2. Service Configuration Updates: - - Enhance the configuration of all services in the `docker-compose.yml` file: - - Add log rotation mechanisms. - - Specify memory limits for containers. - - Ensure these changes are documented within your `METRICS.md` file. - -3. Metrics Gathering: - - Extend Prometheus to gather metrics from all services defined in the `docker-compose.yml` file. - -## Bonus Task: Metrics and Health Checks - -**To Earn 2.5 Additional Points:** - -1. Application Metrics: - - Integrate metrics into your applications. You can refer to Python examples like: - - [Monitoring a Synchronous Python Web Application](https://dzone.com/articles/monitoring-your-synchronous-python-web-application) - - [Metrics Monitoring in Python](https://opensource.com/article/18/4/metrics-monitoring-and-python) - -2. Obtain Application Metrics: - - Configure your applications to export metrics. - -3. METRICS.md Update: - - Document your progress with the bonus tasks, including screenshots, in the `METRICS.md` file. - -4. Health Checks: - - Further enhance the `docker-compose.yml` file's service configurations by adding health checks for the containers. - -### Guidelines - -- Maintain a well-structured and comprehensible `METRICS.md` document. -- Adhere to file and folder naming conventions. -- Utilize code blocks and Markdown formatting where appropriate. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Ensure thorough documentation of your work, and guarantee that Prometheus correctly collects metrics. Take advantage of the bonus tasks to deepen your understanding and enhance the completeness of your setup. diff --git a/lab9.md b/lab9.md deleted file mode 100644 index 5493f042a6..0000000000 --- a/lab9.md +++ /dev/null @@ -1,76 +0,0 @@ -# Lab 9: Introduction to Kubernetes - -## Overview - -In this lab, you will explore Kubernetes, set up a local development environment, and create manifests for your application. - -## Task 1: Kubernetes Setup and Basic Deployment - -**6 Points:** - -1. Learn About Kubernetes: - - Begin by studying the fundamentals of Kubernetes: - - [What is Kubernetes](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) - - [Kubernetes Components](https://kubernetes.io/docs/concepts/overview/components/) - -2. Install Kubernetes Tools: - - Install `kubectl` and `minikube`, essential tools for managing Kubernetes. - - [Kubernetes Tools](https://kubernetes.io/docs/tasks/tools/) - -3. Deploy Your Application: - - Deploy your application within the Minikube cluster using the `kubectl create` command. Create a `Deployment` resource for your app. - - [Example of Creating a Deployment](https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-deployment) - - [Deployment Overview](https://kubernetes.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro/) - -4. Access Your Application: - - Make your application accessible from outside the Kubernetes virtual network. Achieve this by creating a `Service` resource. - - [Example of Creating a Service](https://kubernetes.io/docs/tutorials/hello-minikube/#create-a-service) - - [Service Overview](https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/) - -5. Create a Kubernetes Folder: - - Establish a `k8s` folder within your repository. - - Create a `README.md` report within this folder and include the output of the `kubectl get pods,svc` command. - -6. Cleanup: - - Remove the `Deployment` and `Service` resources that you created, maintaining a tidy Kubernetes environment. - -## Task 2: Declarative Kubernetes Manifests - -**4 Points:** - -1. Manifest Files for Your Application: - - As a more efficient and structured approach, employ configuration files to deploy your application. - - Create a `deployment.yml` manifest file that describes your app's deployment, specifying at least 3 replicas. - - [Kubernetes Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) - - [Declarative Management of Kubernetes Objects Using Configuration Files](https://kubernetes.io/docs/tasks/manage-kubernetes-objects/declarative-config/) - -2. Service Manifest: - - Develop a `service.yml` manifest file for your application. - -3. Manifest Files in `k8s` Folder: - - Store these manifest files in the `k8s` folder of your repository. - - Additionally, provide the output of the `kubectl get pods,svc` command in the `README.md` report. - - Include the output of the `minikube service --all` command and the result from your browser, with a screenshot demonstrating that the IP matches the output of `minikube service --all`. - -## Bonus Task: Additional Configuration and Ingress - -**To Earn 2.5 Additional Points:** - -1. Manifests for Extra App: - - Create `deployment` and `service` manifests for an additional application. - -2. Ingress Manifests: - - Construct [Ingress manifests](https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/) for your applications. - -3. Application Availability Check: - - Utilize `curl` or a similar tool to verify the availability of your applications. Include the output in the report. - -**Guidelines:** - -- Maintain a clear and well-structured `README.md` document. -- Ensure that all required components are included. -- Adhere to file and folder naming conventions. -- Create and participate in PRs to facilitate the peer review process. -- Create pull requests (PRs) as needed: from your fork to the main branch of this repository, and from your fork's branch to your fork's master branch. - -> Note: Detailed documentation is crucial to ensure that your Kubernetes deployment is fully functional and accessible. Engage with the bonus tasks to further enhance your understanding and application deployment skills.