Skip to content

Speed up Emap by increasing DB sequence increment#161

Merged
jeremyestein merged 3 commits into
developfrom
jeremy/sequence-fix
Mar 9, 2026
Merged

Speed up Emap by increasing DB sequence increment#161
jeremyestein merged 3 commits into
developfrom
jeremy/sequence-fix

Conversation

@jeremyestein

Copy link
Copy Markdown
Collaborator

The single biggest SQL query in Emap was select nextval('your_schema.hibernate_sequence'), which had to be called once for every single row inserted into the database.

Details should be explained by docs/technical_overview/ADR/hibernate-sequence.md

@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

PR checklist

Default guide for a PR (if multiple PRs for the work, only keep one version of it and link to it on the other PRs)

  • From the UCLH data science desktop, a validation run has been set off
  • Check that content that reveals Epic IP (eg. Clarity/Caboodle queries) has not been checked into a public repo.
  • load times
    in UCL teams has been populated with the run information
  • During the run, glowroot has been checked for any queries which are taking a substantial proportion of the
    total processing time. This can be useful to identify indexes that are required.
  • After the run, look for any unexpected errors in the etl_per_message_logging table, the error_search.sql file
    on the shared drive can be used for this \\sharefs6\UCLH6\EMAP\Shared\EmapSqlScripts\devops\error_search.sql.
    Create an issue if you find an unexpected exception and is not related to the changes you've made, otherwise
    fix them!
  • After the run, populate the end time in
    load times
  • Let Aasiyah know about the completed validation and give her information on the changes and where to start
    with the validation
  • Check validation report and give any feedback to Aasiyah if there are any changes needed on her side,
    iterate on getting the validation to match at least 99% (validation and emap code).

@skeating skeating left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too bad it didn't work out fully, but even some improvement is better than none

Base automatically changed from jeremy/fakeuds to develop March 9, 2026 11:49
@jeremyestein jeremyestein merged commit cf32a54 into develop Mar 9, 2026
9 checks passed
@jeremyestein jeremyestein deleted the jeremy/sequence-fix branch March 9, 2026 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants