Discussion: StackHead v2

[saitho]

Vision

The vision of StackHead is to provide a simple way to set up everything needed to serve a containerbased web application.
Such may be:

• create the webserver configuration (reverse proxy)
• start up the multiple containers and connect them to multiple domains or ports
• generate and renew SSL certificates for the domains

StackHead should NOT reinvent the wheel.

Current state

StackHead in it's current form "just works", however in a long term there are several challenges when thinking about ongoing maintainance and development.

• Ansible is used to install the software on the target webserver and generate Terraform files via Jinja2 templates
• StackHead project definitions are processed via Ansible
• Terraform is used to place configuration files (webserver configuration, certificates, etc)
• StackHead is composed of individual StackHead modules (webserver, container, DNS) to enable using different technologies
• StackHead CLI enables easier usage of the Ansible playbooks by installing the required Ansible collections and roles and playbook execution via a temporary inventory

Reasoning

• Ansible has many roles for installing the required software on a server (e.g. Docker, Terraform, Nginx), so we do not have to reimpement those ourselves. That is why StackHead is executed via Ansible, as an Ansible collection.
• Terraform is used because we do not need to tear down resources before we delete the definition (in contrast to Ansible's/Chef's/Puppet's "ensure: present/absent")

Current issues

• maintainence of logic "code" (YAML files) with new Ansible versions (to be expected as StackHead essentially is an Ansible collection)
• maintainance for Terraform files and the Terraform version installed on the server
• dependency management of external (i.a. third-party or StackHead modules) Ansible roles and Terraform providers
• The config file format may be hard to read, even though it is YAML (Alternative config schema #131)
• makeshift, immature dependency management for StackHead modules
• Python code needs to be defined inside the StackHead collection and can not be used from the roles

Improvement ideas

Drop Ansible

• maintainence of logic "code" (YAML files) with new Ansible versions (to be expected as StackHead essentially is an Ansible collection)
• dependency management of external (i.a. third-party or StackHead modules) Ansible roles
• makeshift, immature dependency management for StackHead modules
• Python code needs to be defined inside the StackHead collection and can not be used from the roles

Ansible is only used as SSH client to run code on the target system (of which we currently only support Ubuntu). The required logic for that is pretty simple. A module system based on Ansible roles was developed in order to "easily" install only the required modules and not bundle a giant collection.

StackHead was constructed as Ansible collection, containing all the logics for processing the YAML StackHead configuration files (project definitions and plugin configuratiins). Furthermore the StackHead CLI was developed to simplify calling the Ansible commands, also utilizing a temporary inventory for deployment onto a single server.

Conclusion:
The main work of Ansible, i.e. acting as a SSH client, can easily be done via Golang. Since we only support Ubuntu safely, we would't lose much logic since we'd mainly use APT. Needing a wrapper CLI for "easier access" proves that Ansible's usage, requiring inventories etc. may be too much to handle.

Suggestion:
Based on the discussion above, keeping the code base compatible with future Ansible versions is not worth it. Therefore Ansible is removed from the project and the logic is being moved to Golang with the StackHead CLI. StackHead modules are being reworked, possibly as Go plugin.

Extract logic from YAML to code

Move logic from YAML files to Python code for being able to unit test, and to separate the logic from Ansible as executor.

Drop Terraform

• maintainance for Terraform files and the Terraform version installed on the server
• dependency management of external (i.a. third-party or StackHead modules) Ansible roles and Terraform providers

TBD...
Does this goes against the idea to not reimplement everything ourselves?

New project definition schema (Configuration blocks)

• The config file format may be hard to read, even though it is YAML (Alternative config schema #131)

See #131

[saitho]

It must be possible to update all deployed projects, e.g. when Terraform or its providers are updated during a new StackHead release. In this case, it should redeploy all currently deployed applications after one another.