Skip to content

Release PromptSecurity-Eval on Hugging Face #1

Description

@NielsRogge

Hi @youbin2014 🤗

I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work through Papers with Code as your paper, SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defenses, recently got featured: https://paperswithcode.co/paper/sok-systematizing-llm-prompt-security.

First of all, congratulations on this amazing systematization work! It's super exciting to see that you have already hosted JailbreakDB on Hugging Face (https://huggingface.co/datasets/youbin2014/JailbreakDB).

I noticed in your GitHub repository (datasec-lab/PromptSecurity) that the companion evaluation-result dataset PromptSecurity-Eval (with the 166,200 harmful-query output records and 9,200 benign-utility records) is currently hosted inside the repository under the dataset/promptsecurity_eval/ folder.

Would you like to also host PromptSecurity-Eval as a dataset on the Hugging Face Hub (https://huggingface.co/datasets)?

Hosting it on Hugging Face would give the evaluation records much more visibility, better discoverability, and would allow researchers to easily explore and load the evaluation results with a single line of code:

from datasets import load_dataset

dataset = load_dataset("datasec-lab/promptsecurity-eval")

If you're interested, you can find a quick guide on how to upload datasets here: https://huggingface.co/docs/datasets/loading. Additionally, Hugging Face offers a built-in dataset viewer that lets anyone preview and filter the rows directly from their web browser.

Once uploaded, we can also link both JailbreakDB and PromptSecurity-Eval directly to your Hugging Face paper page (https://huggingface.co/papers/2510.15476) so readers can find all related artifacts in one place.

Let me know if you're interested or if you need any guidance!

Kind regards,

Niels

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions