[PROPOSAL]: refactor zero/gemini folder structure

[ver217]

Proposal

Current folder structure of zero/gemini is very confusing. The main components are implemented in different folders. E.g.

Component	Folder
Legacy Zero	zero/
Low level Zero	zero/
Gemini Hook	zero/utils
ColoInitContext	utils/model/colo_init_context
Chunk/Gemini Manager	gemini/
Zero Optimizer	nn/optimizer/zero_optimizer
ZeroDDP/GeminiDDP	nn/parallel
zero_wrapper	nn/parallel

We can reorganize the folder structure. E.g.

zero
| --- legacy # all legacy code related to legacy zero
| --- low_level # code related to low level zero optim
       | --- LowLevelZeroOptimizer
| --- gemini 
       | --- GeminiDDP
       | --- GeminiOptimizer

init_ctx
| --- ColoInitContext
| --- LazyInitContext

Self-service

• I'd be willing to do some initial work on this proposal myself.

[FrankLeeeee]

Can you convert this to discussion?