Skip to content

Speed up emap-star waveform table writes by ~50%#160

Merged
jeremyestein merged 2 commits into
developfrom
jeremy/waveform-equals
Mar 9, 2026
Merged

Speed up emap-star waveform table writes by ~50%#160
jeremyestein merged 2 commits into
developfrom
jeremy/waveform-equals

Conversation

@jeremyestein

@jeremyestein jeremyestein commented Mar 5, 2026

Copy link
Copy Markdown
Collaborator

Hibernate needs to be able to tell if the in-memory object has changed vs what it thinks is in the database. It uses the equals method of the Entity object to do so. If it looks like it has changed, it will issue an UPDATE to the database.

Our equals method always returned false, thus causing an unecessary UPDATE shortly after we do the initial INSERT.

Objects.equals(a1, a2) delegates to a1.equals(a2) in the case of Java arrays, which only tests for referential equality! The fix is to use Arrays.equals(a1, a2), which tests for equality of the values inside.

(Note that this is not part of the Waveform export pipeline that we've been actively working on the last few months. This is the waveform table inside the Postgres DB.)

references, thus causing Hibernate to issue unnecessary UPDATEs.
@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

PR checklist

Default guide for a PR (if multiple PRs for the work, only keep one version of it and link to it on the other PRs)

  • From the UCLH data science desktop, a validation run has been set off
  • Check that content that reveals Epic IP (eg. Clarity/Caboodle queries) has not been checked into a public repo.
  • load times
    in UCL teams has been populated with the run information
  • During the run, glowroot has been checked for any queries which are taking a substantial proportion of the
    total processing time. This can be useful to identify indexes that are required.
  • After the run, look for any unexpected errors in the etl_per_message_logging table, the error_search.sql file
    on the shared drive can be used for this \\sharefs6\UCLH6\EMAP\Shared\EmapSqlScripts\devops\error_search.sql.
    Create an issue if you find an unexpected exception and is not related to the changes you've made, otherwise
    fix them!
  • After the run, populate the end time in
    load times
  • Let Aasiyah know about the completed validation and give her information on the changes and where to start
    with the validation
  • Check validation report and give any feedback to Aasiyah if there are any changes needed on her side,
    iterate on getting the validation to match at least 99% (validation and emap code).

@skeating skeating left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really good fix

Base automatically changed from jeremy/fakeuds to develop March 9, 2026 11:49
@jeremyestein jeremyestein merged commit f2bfff6 into develop Mar 9, 2026
9 checks passed
@jeremyestein jeremyestein deleted the jeremy/waveform-equals branch March 9, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants