CDA-74 Created ADR for timeseries csv formatting#1634
CDA-74 Created ADR for timeseries csv formatting#1634rma-bryson wants to merge 23 commits intodevelopfrom
Conversation
There was a problem hiding this comment.
See if you can improve the formatting of the numbering and add more sub section headers to break things up
i.e.
1) vs 1.
Might help in reading this visually when it is rendered!
Optional fix, not a blocker for me.
|
Updated formatting to be more readable. |
Co-authored-by: Mike Neilson <michael.a.neilson@usace.army.mil>
…, it breaks the standard of column to field naming used by jackson.
rma-bryson
force-pushed
the
feature/CDA-74-ADR-for-TimeSeries-CSV
branch
from
590a37c to
4581866
Compare
MikeNeilson
left a comment
MikeNeilson
left a comment
There was a problem hiding this comment.
Looks reasonable to me.
Will let @krowvin give the final approval.
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
…s-CSV' into feature/CDA-74-ADR-for-TimeSeries-CSV
| - Always include ``date-time`` and ``value``; include units in the value column header as parentheses (e.g., ``value (ft)``) | ||
| - Units must exist in exactly one canonical location in all modes. | ||
| * - Optional columns | ||
| - Optional (off by default): ``time-series-id``, ``office-id``, ``version-date``, ``data-entry-date``, ``quality`` |
There was a problem hiding this comment.
I don't think time-series-id or office-id should ever be columns. If this information is needed, metadata comments should be turned on.
We had decided for JSON/XML timeseries, that there has not been a use-case for version-date at the row level. So far we've only needed base, a specific version, or aggregate.
Should be added somewhere that a future update could include an optional text-annotation column.
| - Optional (off by default): ``time-series-id``, ``office-id``, ``version-date``, ``data-entry-date``, ``quality`` | |
| - Optional (off by default): ``data-entry-date``, ``quality``. |
There was a problem hiding this comment.
This makes sense to me so as to avoid repeated data. The one benefit of using columns is that follows standards (using '#' comments isn't standard). The idea here was to allow for either comments or columns. Will let @MikeNeilson weigh in
| - Optional (off by default): ``time-series-id``, ``office-id``, ``version-date``, ``data-entry-date``, ``quality`` | ||
| - Everything except ``date-time`` and ``value`` (with units in the header) is optional. Because headers are always included, optional columns can be toggled without breaking parsing. Clients should rely on column names, not indices. | ||
| * - Metadata fields | ||
| - May be emitted as top-of-payload comments (``metadata-format=comment``) or as actual columns (``metadata-format=column``) |
There was a problem hiding this comment.
| - May be emitted as top-of-payload comments (``metadata-format=comment``) or as actual columns (``metadata-format=column``) | |
| - The payload may include prefix comments using the mime-type property: (``metadata-format=comment``) |
If there is a case where meta-data columns would be preferred, please reference here as that redundant data is unnecessary given any parsing API (python, javascript, hand-edited excel, java, etc) can remove comments.
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
…record definition
| .. code-block:: text | ||
|
|
||
| time-series-id, office-id, date-time, value (cfs), version-date, data-entry-date, quality-code | ||
| ALAT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI, SWT, 2021-06-21T00:00:00Z, 0.0, aggregate, 2021-06-21T00:05:00Z, 5 |
There was a problem hiding this comment.
If you're proposing to add version-date to a column, the representation is then no longer aggregate as the term aggregate refers to using the latest version-date across all time steps. I don't think we should add version-date here unless that feature is requested.
There was a problem hiding this comment.
good point - this makes me more inclined to not have metadata columns at all. comments only.
…made at this level
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
Co-authored-by: Adam Korynta <47677856+adamkorynta@users.noreply.github.com>
4 participants
No description provided.