Skip to content

chore: simplify save_raw_data#61

Draft
dkrako wants to merge 4 commits intomainfrom
chore/simplify-save-raw-data
Draft

chore: simplify save_raw_data#61
dkrako wants to merge 4 commits intomainfrom
chore/simplify-save-raw-data

Conversation

@dkrako
Copy link
Copy Markdown
Collaborator

@dkrako dkrako commented Feb 19, 2026

No description provided.

directory = Path(directory) / session / constants.RAW_DATA_FOLDER
directory.mkdir(parents=True, exist_ok=True)

new_data = data.clone()
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the clone shouldn't be necessary

directory.mkdir(parents=True, exist_ok=True)

new_data = data.clone()
trials = data.split(by="trial", as_dict=True)
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe it would be nice to store the keys that the data is split on into the new Gaze.metadata field.

I created an issue for this: pymovements/pymovements#1502

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Collaborator Author

@dkrako dkrako Feb 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as pymovements/pymovements#1504 is merged I updated this PR accordingly in 7330f72

trial.unnest()
trial.frame = trial.frame["time", "pixel_x", "pixel_y", "pupil", "page"]

trial.save_samples(directory / filename)
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the save samples method could include two additional arguments:

  • unnest: bool | None = None: if True, always unnest, if None only unnest for csv files.
  • columns: list[str] | None: if list of strings, only save these columns

name = f"{session}_{trial}_{stimulus}_raw_data.csv"
df = df["time", "pixel_x", "pixel_y", "pupil", "page"]
df.write_csv(directory / name)
trial.unnest()
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
trial.unnest()
trial.unnest('pixel')

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this way no warning should be raised

@theDebbister
Copy link
Copy Markdown
Member

theDebbister commented Feb 24, 2026

When I run the code locally:
Warning: No columns to unnest. Please specify columns to unnest via the "input_columns" argument.

It gets raised when unnesting the gaze per default in the save methods.

So, we'd need the unnest parameter here to make it work. and set it to False as it is already unnested. I could not find an issue in pm for this, is there one already?

@dkrako
Copy link
Copy Markdown
Collaborator Author

dkrako commented Mar 4, 2026

Thanks for the feedback! pymovements/pymovements#1522 should fix that issue.

@dkrako dkrako marked this pull request as draft March 4, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants