This is a research prototype demonstrating how LLMs can assist in educational tooling and QA for e-learning platforms. The system uses browser automation to extract interactive problems, generate solutions using AI language models, and validate automated workflows.
This repository contains research/prototype code intended only for authorized testing, instructor-authorized QA, accessibility aids, or developer experimentation. It is not intended to be used to submit coursework or to violate the terms of service of any third-party platform. By using this code you agree to use it responsibly and to comply with all applicable platform terms.
The system basically:
- Authenticates securely to e-learning platforms (without storing credentials)
- Navigates through interactive content and exercises
- Extracts problem descriptions and starter code
- Sends everything to your chosen AI model (like LM Studio, Google Gemini, or others)
- Gets back solution suggestions for validation and testing
- Supports automated QA workflows with configurable safety modes
First, you need to set up authentication. Run this once to save your login session:
python playwright/auth.pyThis opens a browser, lets you log in manually, and saves your session so you don't have to log in again.
Create a .env file with your settings. All app settings use the AUTOMATION_ prefix (see .env.example):
# Platform URL (generic e-learning platform)
AUTOMATION_COURSE_URL=https://example-platform.com/your-course-url
# Which LLM provider to use: lm_studio, gemini, or openrouter
AUTOMATION_LLM_PROVIDER=lm_studio
# LM Studio (local)
AUTOMATION_LLM_BASE_URL=http://127.0.0.1:1234
AUTOMATION_LLM_MODEL=your-local-model-name
# Or Google Gemini (set provider to gemini and provide key)
# AUTOMATION_LLM_API_KEY=your-gemini-api-key
# AUTOMATION_LLM_MODEL=your-gemini-model-name
# Other settings
AUTOMATION_BUTTON_CLICK_WAIT_TIME=1.0
AUTOMATION_RANDOM_ROUTING=0 # 0 = do tasks in order, 1 = pick random tasks
AUTOMATION_SESSION_STATE_FILE=state.jsonOnce everything's set up:
python main.pyIt will start navigating through content, finding interactive elements, extracting problems, and processing them automatically.
- Smart Navigation: Automatically finds interactive content and works through them systematically
- Multiple AI Options: Works with local models (LM Studio), Google Gemini, or other compatible APIs
- Error Recovery: If something goes wrong, it can retry or switch to a different approach
- Flexible Starting: Can start from any specific content URL or explore automatically
- Safe Operation: Includes lots of checks to avoid breaking things or getting stuck
- QA Mode: Configurable safety features for controlled testing environments
├── main.py # Refactored main entry point
├── core/ # Core orchestration components
│ ├── automation_engine.py # Main automation orchestrator
│ ├── rule_engine.py # Rule management and execution
│ └── session_manager.py # Session persistence and auth
├── rules/ # Modular rule implementations
│ ├── base_rule.py # Abstract rule base class
│ ├── rule_1_navigation.py # Uncompleted task navigation
│ ├── rule_2_navigation.py # Content navigation in modals
│ ├── rule_3_theory_viewer.py # Theory viewer handling
│ ├── rule_4_trainer.py # Trainer content & LLM integration
│ ├── rule4_components/ # Sub-components for rule 4
│ │ ├── code_manager.py # Code extraction and injection
│ │ ├── debug_monitor.py # Debugging and error monitoring
│ │ ├── element_processor.py # HTML element processing
│ │ ├── error_recovery_manager.py # Error recovery and retries
│ │ ├── hint_processor.py # Hint extraction and usage
│ │ └── task_description_parser.py # Task description parsing
│ ├── rule_5_next_button.py # Next button navigation
│ ├── rule_6_notification_handler.py # Notification handling
│ ├── rule_7_theory_action_button.py # Theory action buttons
│ ├── rule_8_quiz_handler.py # Quiz form handling
│ ├── rule_9_solution_modal.py # Solution extraction (clipboard)
│ ├── rule_10_last_coding_task.py # Solution extraction (HTML)
│ ├── rule_11_url_blacklist.py # URL blacklist detection
│ ├── rule_12_likert_scale.py # Survey handling
│ ├── rule_13_quiz_feedback.py # Feedback questions
│ ├── rule_14_quiz_matching.py # Matching elements
│ └── rule_15_macaroni_pin.py # Pin-based interactions
├── services/ # Service layer abstractions
│ ├── browser_service.py # Browser operations interface
│ └── llm_service.py # LLM operations interface
├── config/ # Modular configuration system
│ ├── settings.py # Main settings management
│ ├── logging_config.py # Logging configuration
│ └── validation.py # System readiness validation
├── llm/ # AI model integrations
│ ├── llm_client.py # Abstract LLM interface
│ ├── llm_selector.py # Provider selection strategy
│ ├── local_llm.py # Local LLM APIs (LM Studio)
│ ├── gemini.py # Google Gemini integration
│ └── openrouter.py # OpenRouter API integration
├── playwright/ # Authentication components
│ ├── auth.py # Login handling
│ └── test_auth.py # Authentication testing
├── docs/ # Documentation
│ └── automation-rules/ # Rule documentation
│ ├── rule1.md
│ ├── rule2.md
│ └── ... (one for each rule)
├── utils/ # Helper utilities
│ ├── response_cleaner.py # Response cleaning utilities
│ ├── element_helpers.py # Element interaction helpers
│ └── text_processing.py # Text processing utilities
├── legacy/ # Legacy code preservation
│ └── browser_automation.py # Original monolithic implementation
├── contracts.py # Data structure definitions
├── tests/ # Unit and integration tests
├── blacklist.txt # URL blacklist for unsolvable tasks
├── state.json # Authentication session state
├── requirements.txt # Python dependencies
└── .env.example # Environment configuration example
Run the tests to make sure everything works:
python -m pytest tests/