title	Mevitsa Demo
emoji	🚀
colorFrom	blue
colorTo	red
sdk	gradio
app_file	src/app_gradio.py
pinned	false

MeViTSA: Multimodal Ensemble Approach for Sentiment Analysis of Visuals Integrated Text Data

Authors: Soham Bhattacharya^1,4, Ali Reza Alaei², Sukalpa Chanda ³ & Umapada Pal⁴

Affiliated institutions: ¹Ramakrishna Mission Vivekananda Educational and Research Institute Belur, India; ²Southern Cross University, Australia; ³Østfold University of Applied Sciences, Norway; ⁴Indian Statistical Institute Kolkata, India;

How to get the models: Please contact the author directly to request access to the trained model files. Once received, place them in the src/trained/ folder before running the application.

Demo

You may try the application yourself with any image. Please feel free to send us your feedback about the model responses. Here is the link for the application: mevitsa-demo

Installation

1. Clone the repository

   git clone [https://github.com/soham-b-github/mevitsa.git](https://github.com/soham-b-github/mevitsa.git)
   cd mevitsa

2. Install dependencies

   pip install -r requirements.txt

   Key requirements: torch, transformers, google-cloud-vision, clip (OpenAI), Pillow, flask / streamlit (for app.py).

3. Setup Google Cloud credentials

   • Export your service account key:

     export GOOGLE_APPLICATION_CREDENTIALS="path/to/your/service-account-file.json"

Running the App

Once you have obtained the trained model files from the author:

1. Ensure the model file is in the correct directory.
2. Run the frontend application:

   streamlit run app.py

3. Upload an image (e.g., a meme or poster) to receive the predicted sentiment (Positive, Negative, or Neutral).

Evaluation setup

To evaluate the framework, you can either use the samples provided in this repository or integrate your own dataset. The system expects a specific directory hierarchy within the data/ folder:

• Using provided samples: The repository includes sample images located in data/docimsentv1-samples/ and data/external-samples/ . These are ready for immediate testing.
• Using custom datasets: If you wish to evaluate the framework on a full dataset, create a new subdirectory within the data/ folder and place your images there.
• Run the following code in the terminal:

python3 evaluation.py --DATASET_PATH=./../data/my_new_dataset

Recommended directory structure for data/ folder:

mevitsa/
├── data/
│   ├── docimsentv1-samples/  # Will be publicly available post publication
│   ├── external-samples/     # Existing sample data from the web
│   └── [your-dataset-name]/  # <--- PLACE CUSTOM DATASETS HERE
│       ├── image_001.jpg
│       └── image_002.png

Citation

This repository is associated with a research paper currently under communication with ICPR 2026 workshop. Citation information will be made available after the paper is accepted and published.