Adding possibility to generate a json schema file when calling jsonargparse.ArgumentParser.save

[martinResearch]

🚀 Feature request

Add option to generate a json schema alongside the json file when calling jsonargparse.ArgumentParser.save and link to that schema file in the default config json file using the "$schema" key.

Motivation

When I edit json configuration files, it is error prone. I can do spelling mistakes, I do not know what filed are optional, what the possible values for enums are etc.

Pitch

It would be nice if we could generate a json schema alongside the json file when calling jsonargparse.ArgumentParser.save and link to that schema file in the default config json file using the "$schema" key. This would enable the use of a plugin like Json Schema validator in vs code to get auto-completion and validation directly when editing the configuration file.

[mauvilsa]

Thank you for the proposal. This had been requested in the past in Lightning-AI/pytorch-lightning#17564, though having it in this repo is better.

I agree that it would be nice to have a json schema. But note that it is a rather complex feature, which is unlikely to happen soon. The hardest part is the support of subclasses. There can be a large amount of possibilities which would generate quite a large and complex json schema, with plenty of nested oneOf. It could even be that auto-completion plug ins don't even work with such complex schemas.

Another note is, having a way to generate a json schema sounds good. But this being done by save including a $schema key does not sound like the direction to go.

[martinResearch]

Would it make sense to have a first implementation with limited support for subclasses that would be well documented? It would throw an error for unsupported cases. It might be sufficient for a large majority of users?
"But this being done by save including a $schema key does not sound like the direction to go." would be interested to know what you think would be a better alternative.

[mauvilsa]

Currently jsonargparse supports shtab, and to get the tab completions script you do like cli.py --print_shtab bash. A jsonschema feature would be just one more type of completion. I would change this, so it could be for example like cli.py --print_completions shtab-bash and also cli.py --print_completions jsonschema. Note that this allows to get the schema for any CLI. There is no need to implement something specific that calls save. Furthermore, a schema is needed/useful before someone ever wants to save.

[mauvilsa]

Experimental support has been added with #961. Additional to that I am planning to add a way to search for subclasses, so that not just the ones that happen to be imported are included. Also note that since it includes the details of the subclasses, the size of the schemas can be large.