GSoC 2022 Idea: Mapping existing configuration parameters to the APIs

[harikrishna-patnala]

Background:

Cloudstack has global settings in which there are around 672 configuration parameters. Using these parameters one can adjust the values according to the need of the environment. For example "allow.duplicate.networkname" default value of this parameter is true, which means networks can be created with same name in an account. If the value is set to false then duplicate network names are not allowed.

Problem statement:
When admin wants to change the API behaviour or during debugging an issue, admin has to search for the corresponding configuration parameter. The only way currently everyone is using is by using string search or looking into corresponding documentation. Searching over 672 configuration parameters which is a huge list is not straight forward or may lead to missing few parameters.

Solution:
To address this problem I would like to propose the solution which maps the configurations parameters to the corresponding APIs. One can know what are the configuration parameters involved for a specific API. For example "createNetwork" API will be mapped to "allow.duplicate.networkname". When admin wants to see what are the configuration parameters used for "createNetwork" API , this mapping will help. In the final result we will have a table with APIs in one column and configuration parameters in another column.

Proposed Tasks

• Get started on basic CloudStack codebase and development (building and running CloudStack)
• Understand the marvin test framework on how to setup and run tests
• Setup any hypervisor (VMware/KVM/XenServer) based CloudStack dev/test environment
• Understand how global configurations are defined and used in CloudStack
• Use marvin tests to call all the APIs and record the configuration parameters used for each API call by using wrapper around configuration framework
• Deliverables: documentation and community pull request, end-to-end demo of feature

Relevant Skills

• Java and Python
• Virtualization/Cloud domain knowledge and usage

Proposed Mentor

Harikrishna Patnala - Committer Apache CloudStack

Difficulty

Major

Potential Mentors

Harikrishna Patnala - Committer Apache CloudStack
Any developer from the community

Example and references

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuration
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python#MarvinTestingwithPython-Installation

ISSUE TYPE

• Feature Idea

[damanBirSingh]

Hello,
I am an ME student from Bits Pilani, Goa, India. I would love to contribute to an open source project like Cloudstack. I am currently going through the Cloudstack/Hackerbook and would start to draft a proposal asap. Furthermore, I also got a reply from @harikrishna-patnala with some resources to follow and will definitely go through those. Please let me know if there is anything else I should know.

Thanks & Regards,
Daman

[harikrishna-patnala]

Hi @damanBirSingh,
It's great to hear from you and your interest on this project. I'm happy to mentor and work with you.
Like we already discussed please go through the links provided and start drafting the application as the end date is reaching soon.

You might have already done the following, if not please do it.
I would advise that you read the community GSoC guidelines (https://community.apache.org/gsoc.html), join the dev mailing list (http://cloudstack.apache.org/mailing-lists.html) to discuss your GSoC idea/proposal/application, and start drafting your application (for example https://google.github.io/gsocguides/student/proposal-example-1). With your proposed mentor, start an early discussion with them and you can ask them to review your application draft (for example, compose your draft and share with a Google doc).

I can help you with drafting application. Please let me know.
Thanks

[khamarshaikh]

@harikrishna-patnala , is this feature still open ? If yes , i would like to contribute to it.

[yadvr]

cc @sureshanaparti you have a global settings re-mapping/categorisation PR, is this related to your PR, or this issue isn't needed anymore as it may be complex for a student project @harikrishna-patnala ?

[sureshanaparti]

cc @sureshanaparti you have a global settings re-mapping/categorisation PR, is this related to your PR, or this issue isn't needed anymore as it may be complex for a student project @harikrishna-patnala ?

@rohityadavcloud this is not related to global settings categorisation.

[harikrishna-patnala]

@rohityadavcloud, this is not related to @sureshanaparti 's work and I think this is not a complex one as it requires understanding only how to run marvin tests and a small wrapper around configuration framework.

[Damans227]

@harikrishna-patnala Ok, I read through the ConfigKey<T> Class from cloud-framework-config package and understood how we can use it to declare a configuration parameter in an interface and grab its value using value() method. I have some questions to understand the workflow of the key management further:

1_ Is cloudstack/api the package which comprises all the apis where we want to expose the global configuration parameters?
2_ You have mentioned in the solution above that "In the final result we will have a table with APIs in one column and configuration parameters in another column" So is this going to be a new table we will create in the database or is it the existing configuration table we want to store this information in?

Thanks!

[harikrishna-patnala]

Hi @Damans227, great to see your interest here and the learning you have gone through.
To answer your questions

1. Yes, most of the APIs are here, but there are other places as well like in plugins/
2. Either ways we can do, with the new table or with new columns in existing "configurations" table.