Skip to content

Feature/sw architecture#2

Merged
NejcKle merged 28 commits into
mainfrom
feature/sw-architecture
Oct 3, 2025
Merged

Feature/sw architecture#2
NejcKle merged 28 commits into
mainfrom
feature/sw-architecture

Conversation

@NejcKle

@NejcKle NejcKle commented Jul 18, 2025

Copy link
Copy Markdown
Collaborator

Description

This PR introduces the key system components for the backend architecture. These include:

  • Ingest Service to consume data from various sources (TTN and general HTTP currently supported, mock devices/applications are found in mock directory)
  • Processor Service to consume the Kafka message queue and store data dynamically into the database
  • REST API to expose sensor, application info as well as sensor measurements

Closes #1

Areas of interest for the reviewer

Please review the code in the following order:

  1. docker-compose file and the system architecture from the docs folder
  2. Ingest Service
  3. Mock device
  4. Processor Service
  5. REST API

After reviewing you can test the application if you wish to do so, this is not necessary.

Checklist

  • My code follows the style guidelines as defined by IRNAS.
  • I have performed a self-review of my code.
  • My changes generate no new warnings.
  • I added/updated source code documentation for all newly added or changed functions.
  • I updated all customer-facing technical documentation.

After-review steps

  • I will merge PR by myself.

@github-actions github-actions Bot added the pull request Pull request, added automatically by CI. label Jul 18, 2025
@NejcKle NejcKle requested a review from vid553 July 18, 2025 11:50

@vid553 vid553 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general looks good, but some improvements are needed.
I've mostly pointed out missing docstrings and other comments, but there are also some code change suggestions and questions.
You should also add documentation (installation and usage).

As I have made a review per commits, there may be some outdated commend left. I went through all the files from all commits again and cleaned them, but it's possible I've missed some.

I have also tested the code locally by running the containers. I have not tested services that are currently not containers.

Comment thread init/postgres.sql
Comment thread ingest_service/Makefile
Comment thread processor_service/main.py
Comment thread ingest_service/util/util.py
Comment thread ingest_service/README.md

To start the Ingest Service run `make run-local-dev` from this directory. This will create a Python
virtual environment, install the required dependencies, and start the service on
`http://localhost:50005`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention that this applies for developing, but for prod, the docker compose should be used.

Comment thread rest_api/main.py Outdated
Comment thread rest_api/main.py

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • consider adding some exception handling, at least for database operations
  • consider adding pagination or at least limit the number of results returned

Comment thread processor_service/src/sql.py
Comment thread processor_service/src/handlers.py
Comment thread processor_service/util/util.py
@NejcKle NejcKle merged commit 4527d4f into main Oct 3, 2025
1 check failed
@NejcKle NejcKle deleted the feature/sw-architecture branch April 28, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pull request Pull request, added automatically by CI.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design and implement a modular system architecture

2 participants