-
Notifications
You must be signed in to change notification settings - Fork 2
Add UDP packet format docs #205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| # Datastreaming: UDP packet formats | ||
|
|
||
| This page describes the UDP packet format for instruments which stream UDP from detectors (e.g. HRPD-X). | ||
|
|
||
| Translation between this UDP format and the {doc}`flatbuffers format <ADRs/000_kafka>` is done by [`event_udp_to_kafka`](https://github.com/ISISComputingGroup/event_udp_to_kafka), which initially get written to the `_rawEvents` Kafka topic. | ||
|
|
||
| The data comes in over UDP packets, each of which is made up of 32-bit words. All data is transmitted in big-endian | ||
| format. | ||
|
Comment on lines
+7
to
+8
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can/will the UDP packets have trailing padding bytes? |
||
|
|
||
| ## Header format | ||
|
|
||
| The header is 16 words (64 bytes). | ||
|
|
||
| ### Word 0: marker byte | ||
|
|
||
| Always `0xFFFFFFFF`. | ||
|
|
||
| ### Word 1: header information | ||
|
|
||
| - **Bit 0**: End of run header marker (active low) | ||
| - **Bit 1**: Veto frame packet header marker (active low) | ||
| - **Bit 2**: Sample environment packet header marker (active low) | ||
| - **Bit 3**: Not used | ||
| - **Bits 4..=31**: Always `0xFFFFFFF` | ||
|
Comment on lines
+20
to
+24
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If all bits are high is that a neutron data packet? |
||
|
|
||
| ### Word 2: Information | ||
|
|
||
| - **Bits 0..=3**: Reserved for header type | ||
| - **Bits 4..=8**: Length of header | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Bytes or words? |
||
| - **Bits 9..=10**: Not used | ||
| - **Bit 11**: Bad frame (not currently used by streaming control board) | ||
| - **Bit 12**: Run will continue (not currently used by streaming control board) | ||
| - **Bit 13**: No frame sync | ||
| - **Bit 14**: Frame Memory Full veto | ||
| - **Bit 15**: Frame data overrun | ||
| - **Bits 16..=31**: Not used | ||
|
|
||
| ### Word 3: Frame number | ||
|
|
||
| - **Bits 0..=31**: frame number as u32 | ||
|
|
||
| ### Word 4: GPS timestamp | ||
|
|
||
| - **Bits 0..=3**: seconds (most significant bits; combine with least significant bits from word 5) | ||
| - **Bits 4..=9**: minutes | ||
| - **Bits 10..=14**: hours | ||
| - **Bits 15..=23**: days | ||
| - **Bits 24..=31**: years (as offset from year 2000) | ||
|
|
||
| ### Word 5: GPS timestamp | ||
|
|
||
| - **Bits 0..=9**: nanoseconds | ||
| - **Bits 10..=19**: microseconds | ||
| - **Bits 20..=29**: milliseconds | ||
| - **Bits 30..=31**: seconds (least significant bits; combine with most significant bits from word 4) | ||
|
|
||
| ### Word 6: period number | ||
|
|
||
| - **Bits 0..=15**: Period number | ||
| - **Bits 16..=31**: Not used | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. in event mode period number is just an extra metadata tag - is it worth just letting it be 32 bits? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea, just an extra metadata tag in event mode so I don't really mind either was on this. |
||
|
|
||
| ### Word 7: events in frame | ||
|
|
||
| - **Bits 0..=31**: number of neutron events in this frame. | ||
|
|
||
| :::{note} | ||
| This is not necessarily the same as the number of events in this UDP message, as the events may be split between | ||
| multiple UDP messages. This is the *total* number of events in the ISIS frame. | ||
|
|
||
| See header word 13 if looking for length of *this* message. | ||
| ::: | ||
|
|
||
| ### Word 8: protons-per-pulse | ||
|
|
||
| - **Bits 0..=7**: protons-per-pulse in this ISIS frame. | ||
| - **Bits 8..=31**: unused | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. DAE3 seems to send a 10 bit PPP looking at the packet definition, but i guess we don't expect to need that amount? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The DAE3 maybe sends a 10 bit PPP but the two lowest bits are padded with 0s. If required I can pad it with 0s to make it 10 bit? |
||
|
|
||
| To convert to {math}`\mu Ah` delivered during this ISIS frame, multiply by {math}`1.738{\times}10^{-6}`. | ||
|
|
||
| ### Word 9: vetoes | ||
|
|
||
| - **Bit 0**: FIFO veto | ||
| - **Bit 1**: SMP veto | ||
| - **Bit 2**: TS2 pulse veto | ||
| - **Bit 3**: Wrong pulse veto | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What exactly is a "wrong pulse"? Is this e.g. not the first TS1 pulse when someone has selected "First TS1" timing source? |
||
| - **Bit 4**: Unused | ||
| - **Bit 5**: ISIS slow | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What exactly does this mean? |
||
| - **Bits 6..=9**: External vetoes | ||
| - **Bits 10..=13**: Fast chopper vetoes | ||
| - **Bits 14..=15**: Reserved vetoes | ||
| - **Bits 16..=23**: Unused | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there also a "isis not at 50hz" veto? (or is that "isis slow") |
||
| - **Bits 24..=31**: Frame repeat number | ||
|
|
||
| ### Word 10: next frame address | ||
|
|
||
| - **Bits 0..=31**: Address of the next frame, in bytes. | ||
|
|
||
| ### Word 11: next frame address | ||
|
|
||
| - **Bits 0..=31**: Address of the next frame, in 64-bit words. | ||
|
Tom-Willemsen marked this conversation as resolved.
|
||
|
|
||
| ### Word 12: streamed frame number | ||
|
|
||
| - **Bits 0..=31**: streamed frame number. | ||
|
|
||
| ### Word 13: packet length | ||
|
|
||
| - **Bits 0..=11**: number of 32-bit words from the beginning of this header to the start of the next header. | ||
| - **Bits 12..=31**: Unused | ||
|
|
||
| ### Word 14: unused | ||
|
|
||
| - **Bits 0..=31**: Unused | ||
|
|
||
| ### Word 15: checksum | ||
|
|
||
| - **Bits 0..=31**: Pre-DDR checksum | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Think we decided to drop this checksum at this level... |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -192,6 +192,7 @@ dbs | |
| dcalc | ||
| dcom | ||
| dcomcnfg | ||
| DDR | ||
| deadband | ||
| deadbands | ||
| deadman | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can have multiple "messages" (headers + their data) within a single UDP packet, right?