diff --git a/ADQL.tex b/ADQL.tex index 1eb9e5d..c8ba8eb 100644 --- a/ADQL.tex +++ b/ADQL.tex @@ -559,6 +559,49 @@ \subsubsection{Subqueries} WHERE alpha_source.id >= 5 \end{verbatim} +\subsubsection{Common table expressions} +\label{sec:common-table} + +Common Table Expressions (CTE) are introduced with the \texttt{WITH} clause. +They create a temporary named result set that can be referred to elsewhere in +the main query. + +Using a CTE can simplify complex queries by factoring sub-queries out of the +main ADQL statement. Additionally, some implementations may optimise the use of +sets defined with a CTE, resulting in faster query execution. + +For example, the following query with a nested sub-query: +\begin{verbatim} + SELECT ra, dec + FROM ( + SELECT * + FROM alpha_source + WHERE id % 10 = 0 + ) AS alpha_subset + WHERE ra > 10 + AND ra < 20 +\end{verbatim} + +can be refactored as a named \texttt{WITH} query and a simpler main query: + +\begin{verbatim} + WITH alpha_subset AS ( + SELECT * + FROM alpha_source + WHERE id % 10 = 0 + ) + SELECT ra, dec + FROM alpha_subset + WHERE ra > 10 + AND ra < 20 +\end{verbatim} + +The current version of ADQL does not support recursive common table expressions. + +% Recursive CTE are not yet supported by all DBMS (e.g. MySQL). + +CTE can be defined only in the main query. They are not allowed in sub-queries. + \subsubsection{Joins} \label{sec:joins} %TBD - cosmopterix tests for this @@ -570,6 +613,128 @@ \subsubsection{Joins} %REMOVED: The join condition does not support embedded sub joins. %REASON: The BNF allows nested JOINs. +\subsubsection{Set operations} +\label{sec:set.operators} + +An ADQL service implementation MUST include support for the following set +operators: + +\begin{itemize} + \item \verb:UNION: + \item \verb:EXCEPT: + \item \verb:INTERSECT: +\end{itemize} + +For a set operation to be valid in ADQL, the following criteria must be met: +\begin{itemize} + \item the two queries MUST result in the same number of columns + \item the columns in the operands MUST have the same datatypes. +\end{itemize} + +In addition, the columns returned by a set operation SHOULD have the same +metadata, e.g. units, UCD, etc. These metadata SHOULD be generated from the +left-hand operand of the set operation. + +\paragraph{UNION} + +This operator combines the results of two queries, accepting rows from +both the first and second set of results. + +\paragraph{EXCEPT} + +This operator combines the results of two queries, accepting rows that are +in the first set of results but are not in the second one. + +\paragraph{INTERSECT} + +This operator combines the results of two queries, accepting rows +that are strictly in both the first and second set of results. + +\paragraph{Duplicated rows} + +\verb:UNION:, \verb:EXCEPT: and \verb:INTERSECT: remove duplicated rows, +while \verb:UNION ALL:, \verb:EXCEPT ALL: and \verb:INTERSECT ALL: keep all of +them. + +Note that the comparison used for removing duplicated rows is based purely on +the column value and does not take into account the units. This means that a row +with a numeric value of \verb:2: and unit of \verb:m: and a row with a numeric +value of \verb:2: and unit of \verb:km: will be considered equal, despite the +difference in units. + +\paragraph{Operands} + +Operands of any of the set operators can only be \verb:SELECT: queries. +Unless within parentheses, such queries can not use any \verb:ORDER BY: or +\verb:OFFSET: clause. + +Example: sorting result of a \verb:UNION: operation: + +\begin{verbatim} + SELECT id, ra, dec FROM table1 + UNION + SELECT id, ra, dec FROM table2 + ORDER BY id -- sort the UNION result +\end{verbatim} + +Example: sorting result of the \verb:UNION: operands: + +\begin{verbatim} + -- take the 10 first + (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id ASC) + UNION + -- take the 10 last + (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id DESC) +\end{verbatim} + +Common Table Expressions are not allowed in any set operator operand. They must +always be declared at the main level. + +Example: sorting result of the \verb:UNION: operands: with common table +expressions + +\begin{verbatim} +WITH tenFirst AS (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id ASC), + tenLast AS (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id DESC) + SELECT * FROM tenFirst +UNION + SELECT * FROM tenLast +\end{verbatim} + +\paragraph{Precedence} + +When set operators are used together, the resulting expression is +evaluated in the context of the following precedence: + +\begin{enumerate} + \item Expressions within parentheses + \item The \verb:INTERSECT: operator + \item The \verb:UNION: and \verb:EXCEPT: operators evaluated from left to + right +\end{enumerate} + +Example: + +\begin{verbatim} + SELECT id, ra, dec FROM table1 + UNION + SELECT id, ra, dec FROM table2 + INTERSECT + SELECT id, ra, dec FROM table3 +\end{verbatim} + +is equivalent to: + +\begin{verbatim} + SELECT id, ra, dec FROM table1 + UNION + ( + SELECT id, ra, dec FROM table2 + INTERSECT + SELECT id, ra, dec FROM table3 + ) +\end{verbatim} + \subsubsection{Search condition} \label{sec:search} @@ -588,14 +753,39 @@ \subsubsection{Search condition} \item Non-empty subquery check: \verb:EXISTS: \end{itemize} -In addition, some service implementations may also support the optional \verb:ILIKE: -case-insensitive string comparison operator, defined in \SectionRef{sec:string.functions.ilike}. +In addition, some service implementations may also support the optional +\verb:ILIKE: case-insensitive string comparison operator, defined in +\SectionRef{sec:optional.string.functions.ilike}. \begin{itemize} \item \verb:ILIKE: \end{itemize} -\subsection{Mathematical and Trigonometrical Functions} +\subsubsection{Offset} +\label{sec:offset} + +An ADQL service implementation MUST include support for the \texttt{OFFSET} +clause which limits the number of rows returned by removing a specified number +of rows from the beginning of the result set. + +In order to guarantee the consistency in the returned rows, an \texttt{ORDER BY} +clause MUST always be used when the \texttt{OFFSET} clause is present. The +\texttt{ORDER BY} is applied before the specified number of rows are dropped by +the \texttt{OFFSET} clause. +% +% ORDER BY is mandatory with OFFSET, in MS-SQLServer databases but not in +% PostgreSQL and MySQL databases. Making this mandatory in ADQL helps producing +% consistent results and allows a better support on the most used DBMS. + +If the total number of rows is less than the value +specified by the \texttt{OFFSET} clause, then the result set is empty. + +If a query contains both an \texttt{OFFSET} clause and a \texttt{TOP} clause, +then the \texttt{OFFSET} clause is applied first, dropping the specified +number of rows from the beginning of the result set before the +\texttt{TOP} clause is applied to limit the number of rows returned. + +\subsection{Mathematical and trigonometrical functions} \label{sec:math.functions} ADQL declares a list of reserved keywords \SectionSee{sec:keywords} which @@ -696,275 +886,275 @@ \subsubsection{Trigonometrical Functions} Returns the tangent of the angle \textit{x} in radians. \end{description} -\section{Type system} -\label{sec:types} +\subsection{String functions} +\label{sec:string.functions} -ADQL defines no data definition language (DDL). -It is assumed that table definition and data ingestion are performed in -the underlying database's native language and type system. +An ADQL service implementation MUST include support for the following string +manipulation functions: -However, service metadata needs to give column types in order to allow the -construction of queries that are both syntactically and semantically correct. -Examples of such metadata includes the \verb:TAP_SCHEMA: tables defined in the -\TAPSpec{} and the \verb:/tables: webservice response defined in the -\VOSISpec{}. +\begin{itemize} + \item \verb:LOWER(): Lower case conversion + \item \verb:UPPER(): Upper case conversion +\end{itemize} -Services SHOULD, if at all possible, try to express their column metadata in -these terms even if the underlying database employs different types. -Services SHOULD also use the following mappings when interfacing to user data, -either by serializing result sets into VOTables or by ingesting user-provided -VOTables into ADQL-visible tables. +\subsubsection{Case folding} -\subsection{Numeric types} -\label{sec:types.numeric} +Since case folding is a nontrivial operation in a multi-encoding world, ADQL +requires standard behaviour for the ASCII characters, and recommends +following algorithms described in Section 3.13, ``Default Case Algorithms'' +of \citet{std:UNICODE} for characters outside the ASCII set: -\subsubsection{Numeric primitives} -\label{sec:types.numeric.primitive} +\begin{itemize} + \item algorithm R1 for \verb:UPPER(): + \item algorithm R2 for \verb:LOWER(): and \verb:ILIKE: \SectionRef{sec:optional.string.functions.ilike} +\end{itemize} -The numeric datatypes, \verb:BIT:, \verb:SMALLINT:, \verb:INTEGER:, -\verb:BIGINT:, \verb:REAL: \linebreak and \verb:DOUBLE PRECISION: map to the -corresponding datatypes defined in the \VOTableSpec{}. +\subsubsection{LOWER} +\label{sec:string.functions.lower} -\begin{table}[h]\footnotesize - \begin{tabular} - {|p{0.30\textwidth}|p{0.26\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} +The \texttt{LOWER} function converts its string parameter to lower case in +accordance with the rules of the database's locale. - \hline - \multicolumn{1}{|c|}{\textbf{ADQL}} & - \multicolumn{3}{c|}{\textbf{VOTable}} - \tabularnewline +\begin{verbatim} + LOWER('Francis Albert Augustus Charles Emmanuel') + => + francis albert augustus charles emmanuel +\end{verbatim} - \hline - \textbf{type} & - \textbf{datatype} & - \textbf{arraysize} & - \textbf{xtype} - \tabularnewline +\subsubsection{UPPER} +\label{sec:string.functions.upper} - \hline - BIT & - bit & - - & - - - \tabularnewline +The \texttt{UPPER} function converts its string parameter to upper case in +accordance with the rules of the database's locale. - \hline - SMALLINT & - short & - - & - - - \tabularnewline +\begin{verbatim} + UPPER('Francis Albert Augustus Charles Emmanuel') + => + FRANCIS ALBERT AUGUSTUS CHARLES EMMANUEL +\end{verbatim} - \hline - INTEGER & - int & - - & - - - \tabularnewline +\subsection{Type operations} +\label{sec:type} - \hline - BIGINT & - long & - - & - - - \tabularnewline +An ADQL service implementation MUST include support for the following +type conversion functions: - \hline - REAL & - float & - - & - - - \tabularnewline +\begin{itemize} + \item \verb:CAST(): +\end{itemize} - \hline - DOUBLE PRECISION & - double & - - & - - - \tabularnewline - \hline - \end{tabular} - \caption{ADQL type mapping for numeric values} - \label{table:types.numeric.primitive} -\end{table} - -Where possible ADQL numeric values SHOULD be implemented using database types -that correspond to the VOTable serialization types, e.g. \verb:SMALLINT: should -map to a 16 bit integer, \verb:INTEGER: should map to a 32 bit integer, etc. +\subsubsection{CAST} +\label{sec:type.cast} -\subsubsection{INTERVAL} -\label{sec:types.numeric.interval} +The \verb:CAST(): function returns the value of the first argument converted +into the datatype specified by the second argument. -The \DALISpec{} defines \verb:INTERVAL: as a pair of integer or floating-point -numeric values which are serialized as an array of numbers. +\paragraph{Syntax} \verb:: +\begin{verbatim} +CAST + AS + +\end{verbatim} -None of the ADQL operators apply to \verb:INTERVAL: values. -However, specific implementations MAY provide user defined functions that -operate on some \verb:INTERVAL: values. +\paragraph{Target types} -\begin{table}[h]\footnotesize - \begin{tabular} - {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} +This function does not replicate the full functionality and range of types +supported by common RDBMS implementations of \verb:CAST():. Here is the minimum +range of types that MUST be supported: - \hline - \multicolumn{1}{|c|}{\textbf{ADQL}} & - \multicolumn{3}{|c|}{\textbf{VOTable}} - \tabularnewline +\begin{itemize} + \item Exact numeric: + \begin{itemize} + \item \verb:INTEGER: + \item \verb:SMALLINT: + \item \verb:BIGINT: + \end{itemize} + \item Approximate numeric: + \begin{itemize} + \item \verb:REAL: + \item \verb:DOUBLE PRECISION: + \end{itemize} + \item Character: + \begin{itemize} + \item \verb:CHAR: or \verb:CHAR(n): (where n is the fixed string length) + \item \verb:VARCHAR: or \verb:VARCHAR(n): (where n is the maximum string length) + \end{itemize} + \item Date, Time: + \begin{itemize} + \item \verb:TIMESTAMP: + \end{itemize} +\end{itemize} - \hline - \textbf{type} & - \textbf{datatype} & - \textbf{arraysize} & - \textbf{xtype} - \tabularnewline +Examples: - \hline - INTERVAL & - short, int, float, double & - 2 & - interval - \tabularnewline - \hline - \end{tabular} - \caption{ADQL type mapping for INTERVAL} - \label{table:types.numeric.interval} -\end{table} +\begin{verbatim} + CAST(3 AS REAL) + CAST('3.14159265358979323846' AS DOUBLE PRECISION) +\end{verbatim} -The details of how \verb:INTERVAL: values behave in ADQL are not yet -defined. +\paragraph{Input types} -\subsection{Date and time} -\label{sec:types.datetime} +The range of types allowed for the value to cast entirely depends on the target +type. Although cast operations may vary from one implementation to another, ADQL +SHOULD support the ones listed in Table \ref{table:cast.inputtypes}. -Where possible, date and time values SHOULD be implemented as described in the -\DALISpec{}. +\begin{table}[!h] + \center{ + \resizebox{\linewidth}{!}{ + \begin{tabular}{| c | c | c | c | c | c |} + \hline + \multirow{3}{*}{\diaghead{\theadfont Output TyInput Ty}% + {\textbf{Input}}{\textbf{Output}}} + & \textbf{Exact} & \textbf{Approximate} & \textbf{Variable} & \textbf{Fixed} & \\ + & \textbf{numeric} & \textbf{numeric} & \textbf{length} & \textbf{length} & \textbf{Timestamp} \\ + & & & \textbf{character} & \textbf{character} & \\ + \hline + \textbf{Exact} & \multirow{2}{*}{X} & \multirow{2}{*}{X} & \multirow{2}{*}{X} & \multirow{2}{*}{X*} & \\ + \textbf{numeric} & & & & & \\ + \hline + \textbf{Approximate} & \multirow{2}{*}{X*} & \multirow{2}{*}{X} & \multirow{2}{*}{X} & \multirow{2}{*}{X*} & \\ + \textbf{numeric} & & & & & \\ + \hline + \textbf{Character} & X & X & X & X* & X \\ + \hline + \textbf{Timestamp} & & & X & X* & X \\ + \hline + \end{tabular} + } + \textit{\footnotesize{X: supported ; X*: supported but possible implementation differences}} + \caption{CAST allowed types} + \label{table:cast.inputtypes} + } +\end{table} -\subsubsection{TIMESTAMP} -\label{sec:types.datetime.timestamp} +\paragraph{Cast into a smaller datatype} -The \verb:TIMESTAMP: datatype maps to the corresponding type defined in the -\DALISpec{}. +Converting a value to a datatype that is too small to represent it SHOULD be +treated as an error. Details of the mechanism for reporting the error condition +are implementation dependent. -\begin{table}[h]\footnotesize - \begin{tabular} - {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} +This rule especially applies when casting a value into a character string too +small to contain its entire serialization. The output string may be truncated, +adjusted to the needed length, or an error may be thrown. - \hline - \multicolumn{1}{|c|}{\textbf{ADQL}} & - \multicolumn{3}{|c|}{\textbf{VOTable}} - \tabularnewline +\paragraph{Fixed-length character} - \hline - \textbf{type} & - \textbf{datatype} & - \textbf{arraysize} & - \textbf{xtype} - \tabularnewline +The creation of a fixed-length character string is implementation dependent. +In function of the implementation, \verb:CHAR: may be equivalent to +\verb:CHAR(1): or to a \verb:CHAR: just big enough to contain the entire string +to create. - \hline - TIMESTAMP & - char & - n, n*, * & - timestamp - \tabularnewline - \hline - \end{tabular} - \caption{ADQL type mapping for TIMESTAMP} - \label{table:types.datetime.timestamp} -\end{table} +\paragraph{Approximate numeric} -\verb:TIMESTAMP:-s can be created from string literals using the \verb:CAST(): function -(if supported) described in \SectionRef{sec:type.cast}. +The rounding mechanism used when converting from approximate numerics +(\verb:REAL: or \verb:DOUBLE PRECISION:) to precise numerics (\verb:SMALLINT:, +\verb:INTEGER: or \verb:BIGINT:) is implementation dependent. -The basic comparison operators \verb:=:, \verb:<:, \verb:>:, \verb:<=:, \verb:>=:, -\verb:<>: and \verb:BETWEEN: can all be applied to \verb:TIMESTAMP: values. +\paragraph{Timestamp} -For instance: -\begin{itemize} - \item \begin{verbatim} - obstime > CAST('2015-01-01' AS TIMESTAMP) +Only a character string can be casted into a timestamp. This string MUST follow +the syntax defined in the \DALISpec{}: +\begin{verbatim} + YYYY-MM-DD[’T’hh:mm:ss[.SSS][’Z’]] \end{verbatim} - \item \begin{verbatim} - obstime BETWEEN - CAST('2014-01-01' AS TIMESTAMP) - AND - CAST('2014-01-02' AS TIMESTAMP) - \end{verbatim} -\end{itemize} -Within the database, the details of how \verb:TIMESTAMP: values are implemented -are platform dependent. The primary requirement is that the results of the -comparison operators on \verb:TIMESTAMP: values are consistent with respect to -chronological time. +Example: -\subsection{Character types} -\label{sec:types.character} +\begin{verbatim} + CAST('2021-01-14T11:25:00' AS TIMESTAMP) +\end{verbatim} -\subsubsection{Character primitives} -\label{sec:types.character.primitive} +Note that other serializations or any other kind of value MAY also be supported. -The \verb:CHAR: and \verb:VARCHAR: datatypes map to the \verb:char: or -\verb:unicodeChar: type defined in the \VOTableSpec{}. +\paragraph{Geometry} -The choice of whether \verb:CHAR: and \verb:VARCHAR: map to \verb:char: or -\verb:unicodeChar: is implementation dependent and may depend on the data -content. +\verb:CAST(): MAY also produce geometries. If an implementation wants to support +this particular cast operation, it MUST accept a character string following the +DALI serialization matching the precise geometry type to produce. -\begin{table}[h]\footnotesize - \begin{tabular} - {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} +Then, the supported geometry types SHOULD be: - \hline - \multicolumn{1}{|c|}{\textbf{ADQL}} & - \multicolumn{3}{|c|}{\textbf{VOTable}} - \tabularnewline +\begin{itemize} + \item \verb:POINT: + \item \verb:CIRCLE: + \item \verb:POLYGON: +\end{itemize} - \hline - \textbf{type} & - \textbf{datatype} & - \textbf{arraysize} & - \textbf{xtype} - \tabularnewline +Examples: - \hline - CHAR(n) & - char, unicodeChar & - n & - - - \tabularnewline +\begin{verbatim} + CAST('12.3 45.6' AS POINT) + CAST('12.3 45.6 1.0' AS CIRCLE) + CAST('1.0 0.1 2.0 0.2 3.0 0.3' AS POLYGON) +\end{verbatim} - \hline - VARCHAR(n) & - char, unicodeChar & - n* & - - - \tabularnewline - \hline - \end{tabular} - \caption{ADQL type mapping for character strings} - \label{table:types.character.primitive} -\end{table} +Note that other serializations (e.g. STC-S) or any other kind of value MAY also +be supported. -\subsubsection{CLOB} -\label{sec:types.character.clob} +\subsection{Conditional Functions} +\label{sec:condfunc} -To provide support for string values which are generated by the server, -ADQL includes the Character Large OBject (\verb:CLOB:) datatype, -which behaves as an opaque immutable string of characters. +An ADQL service implementation MUST include support for the following +conditional functions: -None of the ADQL operators apply to \verb:CLOB: values. -However, specific database implementations MAY provide user -defined functions that operate on some \verb:CLOB: values. +\begin{itemize} + \item \verb:COALESCE(): +\end{itemize} -\verb:CLOB: values are serialized as arrays of characters. +\subsubsection{COALESCE} +\label{sec:coalesce} + +The \texttt{COALESCE} function returns the first of its arguments that is not +\verb|NULL|. \verb|NULL| is returned only if all arguments are \verb|NULL|. + +All arguments must be of the same datatype. An error should be returned +if this rule is not respected. The way to report this error is implementation +dependent. + +This is typically used to provide fallback values. For instance, + +\begin{verbatim} + COALESCE(access_url, '') +\end{verbatim} + +\noindent will return an empty string when \verb|access_url| is \verb|NULL|. + +\section{Type system} +\label{sec:types} + +ADQL defines no data definition language (DDL). +It is assumed that table definition and data ingestion are performed in +the underlying database's native language and type system. + +However, service metadata needs to give column types in order to allow the +construction of queries that are both syntactically and semantically correct. +Examples of such metadata includes the \verb:TAP_SCHEMA: tables defined in the +\TAPSpec{} and the \verb:/tables: webservice response defined in the +\VOSISpec{}. + +Services SHOULD, if at all possible, try to express their column metadata in +these terms even if the underlying database employs different types. +Services SHOULD also use the following mappings when interfacing to user data, +either by serializing result sets into VOTables or by ingesting user-provided +VOTables into ADQL-visible tables. + +\subsection{Numeric types} +\label{sec:types.numeric} + +\subsubsection{Numeric primitives} +\label{sec:types.numeric.primitive} + +The numeric datatypes, \verb:BIT:, \verb:SMALLINT:, \verb:INTEGER:, +\verb:BIGINT:, \verb:REAL: \linebreak and \verb:DOUBLE PRECISION: map to the +corresponding datatypes defined in the \VOTableSpec{}. \begin{table}[h]\footnotesize \begin{tabular} - {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + {|p{0.30\textwidth}|p{0.26\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} \hline \multicolumn{1}{|c|}{\textbf{ADQL}} & - \multicolumn{3}{|c|}{\textbf{VOTable}} + \multicolumn{3}{c|}{\textbf{VOTable}} \tabularnewline \hline @@ -975,87 +1165,65 @@ \subsubsection{CLOB} \tabularnewline \hline - CLOB & - char, unicodeChar & - n, n*, * & - adql:clob + BIT & + bit & + - & + - \tabularnewline - \hline - \end{tabular} - \caption{ADQL type mapping for CLOB} - \label{table:types.character.clob} -\end{table} - -The details of how \verb:CLOB: values are handled within a -database is implementation dependent. - -An example use case for \verb:CLOB: is a URL field that is generated on the fly -using one or more fields stored in the database. -Although some of the components are stored in the database, the final URL -that appears in the results is not stored in the database. -Hence it would not be possible to apply ADQL functions or operators to the -URL field without special knowledge of the internal database structure. -However, a service implementation could provide user defined functions -that used knowledge of the internal database structure to perform -specific operations on the generated URL field. - -\subsection{Binary types} -\label{sec:types.binary} - -\subsubsection{Binary primitives} -\label{sec:types.binary.primitive} - -The \verb:BINARY: and \verb:VARBINARY: datatypes map to the \verb:unsignedByte: -type defined in the \VOTableSpec{}. -\begin{table}[h]\footnotesize - \begin{tabular} - {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + \hline + SMALLINT & + short & + - & + - + \tabularnewline \hline - \multicolumn{1}{|c|}{\textbf{ADQL}} & - \multicolumn{3}{|c|}{\textbf{VOTable}} + INTEGER & + int & + - & + - \tabularnewline \hline - \textbf{type} & - \textbf{datatype} & - \textbf{arraysize} & - \textbf{xtype} + BIGINT & + long & + - & + - \tabularnewline \hline - BINARY(n) & - unsignedByte & - n & + REAL & + float & + - & - \tabularnewline \hline - VARBINARY(n) & - unsignedByte & - n* & + DOUBLE PRECISION & + double & + - & - \tabularnewline \hline \end{tabular} - \caption{ADQL type mapping for binary arrays} - \label{table:types.binary.primitive} + \caption{ADQL type mapping for numeric values} + \label{table:types.numeric.primitive} \end{table} -\subsubsection{BLOB} -\label{sec:types.binary.blob} +Where possible ADQL numeric values SHOULD be implemented using database types +that correspond to the VOTable serialization types, e.g. \verb:SMALLINT: should +map to a 16 bit integer, \verb:INTEGER: should map to a 32 bit integer, etc. -To support large blocks of binary data such as images, -ADQL includes the Binary Large OBject (\verb:BLOB:) datatype, -which behaves as an opaque immutable array of bytes. +\subsubsection{INTERVAL} +\label{sec:types.numeric.interval} -None of the ADQL operators apply to \verb:BLOB: values. -However, specific database implementations MAY provide user -defined functions that operate on some \verb:BLOB: values. +The \DALISpec{} defines \verb:INTERVAL: as a pair of integer or floating-point +numeric values which are serialized as an array of numbers. -\verb:BLOB: values are serialized as arrays of \verb:unsignedByte: defined -in the \VOTableSpec{}. +None of the ADQL operators apply to \verb:INTERVAL: values. +However, specific implementations MAY provide user defined functions that +operate on some \verb:INTERVAL: values. \begin{table}[h]\footnotesize \begin{tabular} @@ -1074,47 +1242,32 @@ \subsubsection{BLOB} \tabularnewline \hline - BLOB & - unsignedByte & - n, n*, * & - adql:blob + INTERVAL & + short, int, float, double & + 2 & + interval \tabularnewline \hline \end{tabular} - \caption{ADQL type mapping for BLOB} - \label{table:types.binary.blob} + \caption{ADQL type mapping for INTERVAL} + \label{table:types.numeric.interval} \end{table} -The details of how \verb:BLOB: values are handled within a -database is implementation dependent. - -An example use case for \verb:BLOB: is for storing thumbnail images -in the database alongside the tabular data. -ADQL does not provide functions or operations that operate on -images. -However, a service implementation could provide user defined -functions that use implementation specific features to perform -operations on the image data. - -\subsection{Geometric types} -\label{sec:types.geom} +The details of how \verb:INTERVAL: values behave in ADQL are not yet +defined. -ADQL provides support for the \verb:POINT:, \verb:CIRCLE: and \verb:POLYGON: -spherical geometry types defined in the \DALISpec{}. +\subsection{Date and time} +\label{sec:types.datetime} -ADQL also provides support for STC-S based geometric regions, as defined in the -\STCSAppendix{}, using the \verb:REGION: datatype. +Where possible, date and time values SHOULD be implemented as described in the +\DALISpec{}. -\subsubsection{POINT} -\label{sec:types.geom.point} +\subsubsection{TIMESTAMP} +\label{sec:types.datetime.timestamp} -The \verb:POINT: datatype maps to the corresponding type -for spherical coordinates defined in the +The \verb:TIMESTAMP: datatype maps to the corresponding type defined in the \DALISpec{}. -\verb:POINT: values are serialized as arrays of floating point numbers -using the \verb:point: xtype defined in the \DALISpec{}. - \begin{table}[h]\footnotesize \begin{tabular} {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} @@ -1132,34 +1285,53 @@ \subsubsection{POINT} \tabularnewline \hline - POINT & - float, double & - 2 & - point + TIMESTAMP & + char & + n, n*, * & + timestamp \tabularnewline \hline \end{tabular} - \caption{ADQL type mapping for POINT} - \label{table:types.geom.point} + \caption{ADQL type mapping for TIMESTAMP} + \label{table:types.datetime.timestamp} \end{table} -\verb:POINT: literals can be expressed using the \verb:POINT(): -constructor defined in \SectionRef{sec:functions.geom.point}. +\verb:TIMESTAMP:-s can be created from string literals using the \verb:CAST(): +function described in \SectionRef{sec:type.cast}. -For example: -\begin{verbatim} - POINT(12.3, 45.6) +The basic comparison operators \verb:=:, \verb:<:, \verb:>:, \verb:<=:, \verb:>=:, +\verb:<>: and \verb:BETWEEN: can all be applied to \verb:TIMESTAMP: values. + +For instance: +\begin{itemize} + \item \begin{verbatim} + obstime > CAST('2015-01-01' AS TIMESTAMP) \end{verbatim} + \item \begin{verbatim} + obstime BETWEEN + CAST('2014-01-01' AS TIMESTAMP) + AND + CAST('2014-01-02' AS TIMESTAMP) + \end{verbatim} +\end{itemize} -\subsubsection{CIRCLE} -\label{sec:types.geom.circle} +Within the database, the details of how \verb:TIMESTAMP: values are implemented +are platform dependent. The primary requirement is that the results of the +comparison operators on \verb:TIMESTAMP: values are consistent with respect to +chronological time. -The \verb:CIRCLE: datatype maps to the corresponding type -for spherical coordinates defined in the -\DALISpec{}. +\subsection{Character types} +\label{sec:types.character} -\verb:CIRCLE: values are serialized as arrays of floating point numbers -using the \verb:circle: xtype defined in the \DALISpec{}. +\subsubsection{Character primitives} +\label{sec:types.character.primitive} + +The \verb:CHAR: and \verb:VARCHAR: datatypes map to the \verb:char: or +\verb:unicodeChar: type defined in the \VOTableSpec{}. + +The choice of whether \verb:CHAR: and \verb:VARCHAR: map to \verb:char: or +\verb:unicodeChar: is implementation dependent and may depend on the data +content. \begin{table}[h]\footnotesize \begin{tabular} @@ -1178,16 +1350,267 @@ \subsubsection{CIRCLE} \tabularnewline \hline - CIRCLE & - float, double & - 3 & - circle + CHAR(n) & + char, unicodeChar & + n & + - + \tabularnewline + + \hline + VARCHAR(n) & + char, unicodeChar & + n* & + - \tabularnewline \hline \end{tabular} - \caption{ADQL type mapping for CIRCLE} - \label{table:types.geom.circle} -\end{table} + \caption{ADQL type mapping for character strings} + \label{table:types.character.primitive} +\end{table} + +\subsubsection{CLOB} +\label{sec:types.character.clob} + +To provide support for string values which are generated by the server, +ADQL includes the Character Large OBject (\verb:CLOB:) datatype, +which behaves as an opaque immutable string of characters. + +None of the ADQL operators apply to \verb:CLOB: values. +However, specific database implementations MAY provide user +defined functions that operate on some \verb:CLOB: values. + +\verb:CLOB: values are serialized as arrays of characters. + +\begin{table}[h]\footnotesize + \begin{tabular} + {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + + \hline + \multicolumn{1}{|c|}{\textbf{ADQL}} & + \multicolumn{3}{|c|}{\textbf{VOTable}} + \tabularnewline + + \hline + \textbf{type} & + \textbf{datatype} & + \textbf{arraysize} & + \textbf{xtype} + \tabularnewline + + \hline + CLOB & + char, unicodeChar & + n, n*, * & + adql:clob + \tabularnewline + \hline + \end{tabular} + \caption{ADQL type mapping for CLOB} + \label{table:types.character.clob} +\end{table} + +The details of how \verb:CLOB: values are handled within a +database is implementation dependent. + +An example use case for \verb:CLOB: is a URL field that is generated on the fly +using one or more fields stored in the database. +Although some of the components are stored in the database, the final URL +that appears in the results is not stored in the database. +Hence it would not be possible to apply ADQL functions or operators to the +URL field without special knowledge of the internal database structure. +However, a service implementation could provide user defined functions +that used knowledge of the internal database structure to perform +specific operations on the generated URL field. + +\subsection{Binary types} +\label{sec:types.binary} + +\subsubsection{Binary primitives} +\label{sec:types.binary.primitive} + +The \verb:BINARY: and \verb:VARBINARY: datatypes map to the \verb:unsignedByte: +type defined in the \VOTableSpec{}. + +\begin{table}[h]\footnotesize + \begin{tabular} + {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + + \hline + \multicolumn{1}{|c|}{\textbf{ADQL}} & + \multicolumn{3}{|c|}{\textbf{VOTable}} + \tabularnewline + + \hline + \textbf{type} & + \textbf{datatype} & + \textbf{arraysize} & + \textbf{xtype} + \tabularnewline + + \hline + BINARY(n) & + unsignedByte & + n & + - + \tabularnewline + + \hline + VARBINARY(n) & + unsignedByte & + n* & + - + \tabularnewline + \hline + \end{tabular} + \caption{ADQL type mapping for binary arrays} + \label{table:types.binary.primitive} +\end{table} + +\subsubsection{BLOB} +\label{sec:types.binary.blob} + +To support large blocks of binary data such as images, +ADQL includes the Binary Large OBject (\verb:BLOB:) datatype, +which behaves as an opaque immutable array of bytes. + +None of the ADQL operators apply to \verb:BLOB: values. +However, specific database implementations MAY provide user +defined functions that operate on some \verb:BLOB: values. + +\verb:BLOB: values are serialized as arrays of \verb:unsignedByte: defined +in the \VOTableSpec{}. + +\begin{table}[h]\footnotesize + \begin{tabular} + {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + + \hline + \multicolumn{1}{|c|}{\textbf{ADQL}} & + \multicolumn{3}{|c|}{\textbf{VOTable}} + \tabularnewline + + \hline + \textbf{type} & + \textbf{datatype} & + \textbf{arraysize} & + \textbf{xtype} + \tabularnewline + + \hline + BLOB & + unsignedByte & + n, n*, * & + adql:blob + \tabularnewline + \hline + \end{tabular} + \caption{ADQL type mapping for BLOB} + \label{table:types.binary.blob} +\end{table} + +The details of how \verb:BLOB: values are handled within a +database is implementation dependent. + +An example use case for \verb:BLOB: is for storing thumbnail images +in the database alongside the tabular data. +ADQL does not provide functions or operations that operate on +images. +However, a service implementation could provide user defined +functions that use implementation specific features to perform +operations on the image data. + +\subsection{Geometric types} +\label{sec:types.geom} + +ADQL provides support for the \verb:POINT:, \verb:CIRCLE: and \verb:POLYGON: +spherical geometry types defined in the \DALISpec{}. + +ADQL also provides support for STC-S based geometric regions, as defined in the +\STCSAppendix{}, using the \verb:REGION: datatype. + +\subsubsection{POINT} +\label{sec:types.geom.point} + +The \verb:POINT: datatype maps to the corresponding type +for spherical coordinates defined in the +\DALISpec{}. + +\verb:POINT: values are serialized as arrays of floating point numbers +using the \verb:point: xtype defined in the \DALISpec{}. + +\begin{table}[h]\footnotesize + \begin{tabular} + {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + + \hline + \multicolumn{1}{|c|}{\textbf{ADQL}} & + \multicolumn{3}{|c|}{\textbf{VOTable}} + \tabularnewline + + \hline + \textbf{type} & + \textbf{datatype} & + \textbf{arraysize} & + \textbf{xtype} + \tabularnewline + + \hline + POINT & + float, double & + 2 & + point + \tabularnewline + \hline + \end{tabular} + \caption{ADQL type mapping for POINT} + \label{table:types.geom.point} +\end{table} + +\verb:POINT: literals can be expressed using the \verb:POINT(): +constructor defined in \SectionRef{sec:functions.geom.point}. + +For example: +\begin{verbatim} + POINT(12.3, 45.6) +\end{verbatim} + +\subsubsection{CIRCLE} +\label{sec:types.geom.circle} + +The \verb:CIRCLE: datatype maps to the corresponding type +for spherical coordinates defined in the +\DALISpec{}. + +\verb:CIRCLE: values are serialized as arrays of floating point numbers +using the \verb:circle: xtype defined in the \DALISpec{}. + +\begin{table}[h]\footnotesize + \begin{tabular} + {|p{0.26\textwidth}|p{0.30\textwidth}|p{0.15\textwidth}|p{0.15\textwidth}|} + + \hline + \multicolumn{1}{|c|}{\textbf{ADQL}} & + \multicolumn{3}{|c|}{\textbf{VOTable}} + \tabularnewline + + \hline + \textbf{type} & + \textbf{datatype} & + \textbf{arraysize} & + \textbf{xtype} + \tabularnewline + + \hline + CIRCLE & + float, double & + 3 & + circle + \tabularnewline + \hline + \end{tabular} + \caption{ADQL type mapping for CIRCLE} + \label{table:types.geom.circle} +\end{table} \verb:CIRCLE: literals can be expressed using the \verb:CIRCLE(): constructor defined in \SectionRef{sec:functions.geom.circle}. @@ -1325,1309 +1748,898 @@ \subsubsection{Overview} \item INTERSECTS \item POINT \item POLYGON - \item REGION -\end{itemize} - -\subsubsection{Language feature} -\label{sec:functions.geom.feature} - -All functions described in this section use the following IVOID: - -\begin{verbatim} - ivo://ivoa.net/std/tapregext#features-adqlgeo -\end{verbatim} - -Each geometrical function is declared using this IVOID and its name. -See \SectionRef{sec:capabilities} for more details. - -All other optional features have an IVOID starting with: - -\begin{verbatim} - ivo://ivoa.net/std/tapregext#features-adql- -\end{verbatim} - -Note the ending hyphen. The IVOID for the geometry features does not have this -hyphen. This is not a typo. It actually comes from the \TAPRegSpec{} which -originally defined this IVOID. At that time, it was not clear yet that other -language features would be created for ADQL. So, for historical and -interoperability reasons the IVOID for the geometry features must not have this -hyphen. - -\subsubsection{Datatype functions} -\label{sec:functions.geom.type} - -Some of the functions described in this section (e.g. \verb:POINT:, -\verb:CIRCLE:) are constructors for each of the geometry datatypes. The -semantics of these datatypes are based on the corresponding concepts from the -\STCSpec{} data model. - -The geometry datatypes and expressions are part of the core -\verb:: in the ADQL grammar. - -\begin{verbatim} - ::= - NULL - | - | - | -\end{verbatim} - -A \verb:: does not simply cover the geometry datatype -constructors (POINT, CIRCLE, etc.) but also includes user defined functions and -column values where a geometry datatype is stored in a column. - -Therefore, \verb:: is expanded as: -\begin{verbatim} - ::= - - | -\end{verbatim} -\noindent -where -\begin{verbatim} - ::= - - | - | - | - | - | - | -\end{verbatim} -and \verb:: enables the use of geometric functions -and column references. - -\subsubsection{Coordinate limits} -\label{sec:functions.geom.limits} - -The arguments for a geometric function represent spherical coordinates -in units of degrees (square degrees for area). - -ADQL implementors and users MUST follow coordinate ranges defined in DALI 1.1 -and later. - -Note that at the time of the ADQL 2.1 recommendation, no agreed-upon, reliable, -IVOA-approved convention for what ranges apply to which reference system exists. -Such convention is foreseen to be defined in DALI. Presently, DALI 1.1 only -defines ranges for equatorial coordinates. However, this is expected to be -updated in future DALI versions. - -Details of the mechanism for reporting the out of range arguments are -implementation dependent. - -\subsubsection{Coordinate system} -\label{sec:geom.coordsys.param} - -For historical reasons, the geometry constructors (\verb:BOX:, \verb:CIRCLE:, -\verb:POINT: and \verb:POLYGON:) all accept a string literal as the first -argument, hereafter called the COOSYS argument. - -The COOSYS argument was originally intended to carry information on -a reference system or other coordinate system metadata. This was helpful in -order to deal with data specified in different coordinate systems while -performing geometric operations. It was up to the ADQL service to perform -the appropriate conversion to make these operations possible. - -Since version 2.1, this argument is deprecated and has been made optional. -Future versions of this specification will remove this parameter from the listed -functions. - -Coordinate conversions SHOULD now be explictly requested. The ADQL implementers -have to allow it through User Defined Functions. An interoperable facility for -frame transformations is in preparation as of this writing and is expected to be -part of the \CatalogueUDF{}. -% Catalogue of {ADQL} User Defined Functions - Endorsed Note 1.0+ -% http://www.ivoa.net/documents/udf-catalogue/index.html - -\verb:DISTANCE:, \verb:CONTAINS: and \verb:INTERSECTS: MAY still convert -coordinates of its geometric operands if they are expressed in different -coordinate systems. However, be aware that in a future version of ADQL, these -functions will no longer be expected to perform any coordinate conversion. -Consequently, it is recommanded to avoid relying on this deprecated feature. -For interoperability reasons, queries against 2.0 or later services SHOULD NOT -pass arguments with differing COOSYS arguments to \verb:DISTANCE:, -\verb:CONTAINS: or \verb:INTERSECTS:, as behaviour is undefined in that case. - -\subsubsection{Predicate functions} -\label{sec:functions.geom.predicate} - -Functions CONTAINS and INTERSECTS each accept two geometry datatypes -and return a numeric value of 1 or 0 according to whether the relevant -verb (e.g. contains) is satisfied against the two input geometries; -1 if the condition is met and 0 if it is not. - -Each of these functions can be used as a WHERE clause predicate by -comparing the numeric result with zero or one. -For example: -\begin{verbatim} - SELECT * - FROM table - WHERE 1 = CONTAINS(POINT(...), CIRCLE(...)) -\end{verbatim} - -%REMOVED - speculative, not definitive. -%\noindent -%One would expect later additions to ADQL to add to this range of functions. For -%example, equals, disjoint, touches, crosses, within, overlaps and relate -%are possibilities. - -\subsubsection{Preferred sky crossmatch syntax} -\label{sec:functions.geom.crossmatch} - -An especially common operation that astronomers require when working -with source catalogues is the positional sky crossmatch. -In its simplest form this is a join between two tables with the -requirement that the distance along a great circle between the -sky positions of the two associated rows is less than or equal to -a given threshold. - -The geometrical functions provided by ADQL offer a number of -semantically equivalent ways to specify such a condition -in either the JOIN or the WHERE clause, using various -combinations of POINT, CIRCLE and DISTANCE. -While a correct implementation MUST generate the same result for -any of these alternatives, the performance characteristics may -differ dramatically depending on implementation. -Given this, it is difficult for (human or machine) ADQL authors -to know how to phrase a crossmatch with the expectation that it -will be executed efficiently, and difficult for services to know -which forms of query to optimise. The result can be the -unnecessarily slow operation of the common sky crossmatch operation. - -This section therefore recommends a preferred form of ADQL -to use for sky crossmatching and the related cone search operation, -namely to impose an upper limit on one of the two forms of the -DISTANCE function. -Clients submitting crossmatch-like and cone-like -queries are advised to phrase them in this way rather than using semantically -equivalent alternatives, and services are encouraged to ensure that -these forms of query are executed efficiently; this might involve -identifying such ADQL input clauses and rewriting them appropriately -for efficient processing on the database backend. -Alternative semantically equivalent forms however MAY still be -used by clients, and MUST still be handled correctly by services. - -An example sky position-only crossmatch joining rows of tables -\textit{t1} and \textit{t2} within one arcsecond might therefore look like: -\begin{verbatim} - JOIN ... - ON DISTANCE(t1.ra, t1.dec, t2.ra, t2.dec) < 0.00027 -\end{verbatim} -and a cone search for rows within 2.5 degrees of -the center of M31 might look like: -\begin{verbatim} - SELECT ... - WHERE DISTANCE(t.pos, POINT(10.68, 41.27)) <= 2.5 -\end{verbatim} - -\subsubsection{AREA} -\label{sec:functions.geom.area} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: AREA|}\\ - -The AREA function computes the area, in square degrees, of a given geometry. - -For example, an expression to calculate the area of a POLYGON could be -written as follows: -\begin{verbatim} - AREA(POLYGON(10.0, -10.5, 20.0, 20.5, 30.0, 30.5)) -\end{verbatim} - -The AREA of a single POINT is zero. - -The geometry argument may be a literal value, as above, or it may be a -column reference, function or expression that returns a geometric type. -For example: -\begin{verbatim} - AREA(t1.footprint) -\end{verbatim} -where \textit{t1.footprint} is a reference to a database column that -contains geometric (POINT, BOX, CIRCLE, POLYGON or REGION) values. - -\subsubsection{BOX} -\label{sec:functions.geom.box} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: BOX|}\\ - -Note - the \verb|BOX| function has been deprecated in version 2.1 of the standard, -and will be removed from future versions of the specification. - -The BOX function expresses a box on the sky. A BOX is a special case of POLYGON, -defined purely for convenience, -and it corresponds semantically to the equivalent term, Box, defined in -the \STCSpec{}. -%(STC Box, Section 4.5.1.5) - -It is specified by a center position and size -(in both axes) defining a cross centered on the center position and -with arms extending, parallel to the coordinate axes at the center position, -for half the respective sizes on either side. The box’s sides are line -segments or great circles intersecting the arms of the cross in its end -points at right angles with the arms. - -% Original text in ADQL. -% It is specified by a center position and size -% (in both axes) defining a cross centered on the center position and -% with arms extending, parallel to the coordinate axes at the center position, -% for half the respective sizes on either side. The box’s sides are line -% segments or great circles intersecting the arms of the cross in its end -% points at right angles with the arms. - -% Text from STC-20071030 -% A Box is a special case of a Polygon, defined purely for convenience. -% It is specified by a center position and size (in both coordinates) -% defining a cross centered on the center position and with arms -% extending, parallel to the coordinate axes at the center position, -% for half the respective sizes on either side. -% The box’s sides are line segments or great circles intersecting the -% arms of the cross in its end points at right angles with the arms. - -The function arguments specify the center position and the width and height, -where: -\begin{itemize} - \item the center position is given by a pair of numeric coordinates - in degrees, or a single geometric POINT - \item the values of coordinates of the center position are subject to the - constraints laid down in \SectionRef{sec:functions.geom.limits} - \item the width and height are given by numeric values in degrees -\end{itemize} - -For example, a BOX of ten degrees centered on a position -(25.4, -20.0) in degrees could be written as follows: -\begin{verbatim} - BOX(25.4, -20.0, 10.0, 10.0) -\end{verbatim} - -Alternatively, the center position could be expressed as a POINT: -\begin{verbatim} - BOX(POINT(25.4, -20.0), 10.0, 10.0) -\end{verbatim} - -The function arguments may be literal values, as above, or they may be -column references, functions or expressions that return the appropriate -datatypes. -For example: -\begin{verbatim} - BOX(t1.center, t1.width, t1.height) -\end{verbatim} -where \textit{t1.center}, \textit{t1.width} and \textit{t1.height} -are references to database columns that contain POINT, DOUBLE -and DOUBLE values respectively. -%TODO - ObsCore example - -%coordsys param -For historical reasons, the BOX function accepts an optional string literal as -the first argument. -As of version 2.1 of the specification this parameter has been -marked as deprecated. -Future versions of this specification may remove this parameter -\SectionSee{sec:geom.coordsys.param}. - -\subsubsection{CENTROID} -\label{sec:functions.geom.centroid} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: CENTROID|}\\ - -The CENTROID function computes the centroid of a given geometry and returns a POINT. - -For example, an expression to calculate the centroid of a POLYGON could -be written as follows : -\begin{verbatim} - CENTROID(POLYGON(10.0, -10.5, 20.0, 20.5, 30.0, 30.5)) -\end{verbatim} - -The CENTROID of a single POINT is that POINT. - -The geometry argument may be a literal value, as above, or it may be a -column reference, function or expression that returns a geometric type. -For example: -\begin{verbatim} - CENTROID(t1.footprint) -\end{verbatim} -where \textit{t1.footprint} is a reference to a database column that -contains geometric (POINT, BOX, CIRCLE, POLYGON or REGION) values. -%TODO - ObsCore example - -\subsubsection{CIRCLE} -\label{sec:functions.geom.circle} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: CIRCLE|}\\ - -The CIRCLE function expresses a circular region on the sky (a cone in space), -and it corresponds semantically to the equivalent term, Circle, defined in -the \STCSpec{}. -%(STC Circle, Section 4.5.1.2) - -The function arguments specify the center position and the radius, where: -\begin{itemize} - \item the center position is given by a pair of numeric coordinates - in degrees, or a single geometric POINT - \item the values of coordinates of the center position are subject to the - constraints laid down in \SectionRef{sec:functions.geom.limits} - \item the radius is a numeric value in degrees. -\end{itemize} - -For example, a CIRCLE of ten degrees radius centered on position -(25.4, -20.0) in degrees could be written as follows: -\begin{verbatim} - CIRCLE(25.4, -20.0, 10.0) -\end{verbatim} - -Alternatively, the center position may be expressed as a POINT: -\begin{verbatim} - CIRCLE(POINT(25.4, -20.0), 10.0) -\end{verbatim} - -The position argument may be a literal value, as above, or it may be a -column reference, function or expression that returns a geometric type. -For example: -\begin{verbatim} - CIRCLE(t1.center, t1.radius) -\end{verbatim} -where \textit{t1.center} and \textit{t1.radius} are references to -database columns that contain POINT and DOUBLE values respectively. -%TODO - ObsCore example - -%coordsys param -For historical reasons, the CIRCLE function accepts an optional string literal -as the first argument. -As of version 2.1 of the specification this parameter has been -marked as deprecated. -Future versions of this specification may remove this parameter -\SectionSee{sec:geom.coordsys.param}. - -\subsubsection{CONTAINS} -\label{sec:functions.geom.contains} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: CONTAINS|}\\ - -The CONTAINS function determines if a geometry is wholly contained within -another. This is most commonly used to express a ``point-in-shape'' condition. - -For example, an expression to determine whether the point (25.0, -19.5) degrees -is within a circle of ten degrees radius centered on position (25.4, -20.0) -degrees, could be written as follows: -\begin{verbatim} - CONTAINS(POINT(25.0, -19.5), CIRCLE(25.4, -20.0, 10.0)) -\end{verbatim} - -The CONTAINS function is not symmetric in the meaning of the arguments. - -The CONTAINS function returns the integer value 1 if the first argument -is in, or on, the boundary of the second argument and the integer value 0 -if it is not. - -When used as a predicate in the WHERE clause of a query, the returned integer -value must be compared to the integer values 0 or 1 to form a SQL predicate: -\begin{verbatim} - WHERE 1 = CONTAINS(POINT(25.0, -19.5), - CIRCLE(25.4, -20.0, 10.0)) -\end{verbatim} -\noindent -for ``does contain'' and -\begin{verbatim} - WHERE 0 = CONTAINS(POINT(25.0, -19.5), - CIRCLE(25.4, -20.0, 10.0)) -\end{verbatim} -\noindent -for ``does not contain''. - -%TODO - CONTAINS(thing, POINT) ? - -The geometric arguments for CONTAINS may be literal values, as above, -or they may be column references, functions or expressions that return -geometric values. -For example: -\begin{verbatim} - WHERE 0 = CONTAINS(t1.center, t2.footprint) -\end{verbatim} -where \textit{t1.center} and \textit{t2.footprint} are references to -database columns that contain POINT and geometric (BOX, CIRCLE, POLYGON or REGION) -values respectively. -%TODO - ObsCore example - -%coordsys trans -Geometric arguments SHOULD be expressed in the same coordinate system. -See \SectionRef{sec:geom.coordsys.param} for more details. - -\subsubsection{COORD1} -\label{sec:functions.geom.coord1} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: COORD1|}\\ - -The COORD1 function extracts the first coordinate value, in degrees, of a given -POINT \SectionSee{sec:functions.geom.point} or column reference. - -For example, the right ascension of a point with position (25, -19.5) in -degrees would be obtained using the following expression: -\begin{verbatim} - COORD1(POINT(25.0, -19.5)) -\end{verbatim} -\noindent -which would return a numeric value of 25.0 degrees. - -For example: -\begin{verbatim} - COORD1(t.center) -\end{verbatim} -\noindent -where \textit{t.center} is a reference to a column that contains POINT values. - -\subsubsection{COORD2} -\label{sec:functions.geom.coord2} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: COORD2|}\\ + \item REGION +\end{itemize} -The COORD2 function extracts the second coordinate value, in degrees, of a given -POINT \SectionSee{sec:functions.geom.point} or column reference. +\subsubsection{Language feature} +\label{sec:functions.geom.feature} + +All functions described in this section use the following IVOID: -For example, the declination of a point with position (25, -19.5) in degrees, -could be obtained using the following expression: \begin{verbatim} - COORD2(POINT(25.0, -19.5)) + ivo://ivoa.net/std/tapregext#features-adqlgeo \end{verbatim} -\noindent -which would return a numeric value of -19.5 degrees. -The COORD2 function may be applied to any expression that returns a -geometric POINT value. -For example: +Each geometrical function is declared using this IVOID and its name. +See \SectionRef{sec:capabilities} for more details. + +All other optional features have an IVOID starting with: + \begin{verbatim} - COORD2(t.center) + ivo://ivoa.net/std/tapregext#features-adql- \end{verbatim} -\noindent -where \textit{t.center} is a reference to a column that contains POINT values. -\subsubsection{COORDSYS} -\label{sec:functions.geom.coordsys} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: COORDSYS|}\\ +Note the ending hyphen. The IVOID for the geometry features does not have this +hyphen. This is not a typo. It actually comes from the \TAPRegSpec{} which +originally defined this IVOID. At that time, it was not clear yet that other +language features would be created for ADQL. So, for historical and +interoperability reasons the IVOID for the geometry features must not have this +hyphen. -As of version 2.1 of the specification the COORDSYS function has -been marked as deprecated. This function may be removed in future versions -of this specification. -Details of the coordinate system for a database column are available as part of -the service metadata, available via the \verb:TAP_SCHEMA: tables defined in the -\TAPSpec{} and the \verb:/tables: webservice response defined in the \VOSISpec{}. +\subsubsection{Datatype functions} +\label{sec:functions.geom.type} -%As described in \SectionRef{sec:functions.geom.overview}, the allowed return values must be defined -%by any service making use of ADQL, and a list of standard coordinate system -%literals can be found in the STC specification. -% STC-reference 'STC specification [3]' +Some of the functions described in this section (e.g. \verb:POINT:, +\verb:CIRCLE:) are constructors for each of the geometry datatypes. The +semantics of these datatypes are based on the corresponding concepts from the +\STCSpec{} data model. -The COORDSYS function returns the formal name of the coordinate system for -a given geometry as a string. +The geometry datatypes and expressions are part of the core +\verb:: in the ADQL grammar. -The following example would return the coordinate system of a POINT literal: \begin{verbatim} - COORDSYS(POINT(25.0, -19.5)) + ::= + NULL + | + | + | \end{verbatim} -\noindent -which would return a string value representing the coordinate system used -to create the POINT. -The COORDSYS function may be applied to any expression that returns a -geometric datatype. For example: +A \verb:: does not simply cover the geometry datatype +constructors (POINT, CIRCLE, etc.) but also includes user defined functions and +column values where a geometry datatype is stored in a column. + +Therefore, \verb:: is expanded as: \begin{verbatim} - COORDSYS(t.footprint) + ::= + + | \end{verbatim} \noindent -where \textit{t.footprint} is a reference to a database column that -contains geometric (POINT, BOX, CIRCLE, POLYGON or REGION) values. +where +\begin{verbatim} + ::= + + | + | + | + | + | + | +\end{verbatim} +and \verb:: enables the use of geometric functions +and column references. -\subsubsection{DISTANCE} -\label{sec:functions.geom.distance} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: DISTANCE|}\\ +\subsubsection{Coordinate limits} +\label{sec:functions.geom.limits} -The DISTANCE function computes the arc length along a great circle between two -points and returns a numeric value expression in degrees. +The arguments for a geometric function represent spherical coordinates +in units of degrees (square degrees for area). -The specification defines two versions of the DISTANCE function, one that -accepts two POINT values, and a second that accepts four separate numeric -values. +ADQL implementors and users MUST follow coordinate ranges defined in DALI 1.1 +and later. -If an ADQL service implementation declares support for DISTANCE, -then it must implement both the two parameter and four parameter -forms of the function. +Note that at the time of the ADQL 2.1 recommendation, no agreed-upon, reliable, +IVOA-approved convention for what ranges apply to which reference system exists. +Such convention is foreseen to be defined in DALI. Presently, DALI 1.1 only +defines ranges for equatorial coordinates. However, this is expected to be +updated in future DALI versions. -For example, an expression calculating the distance between two points of -coordinates (25,-19.5) and (25.4,-20) could be written as follows: -\begin{verbatim} - DISTANCE(POINT(25.0, -19.5), POINT(25.4, -20.0)) -\end{verbatim} -\noindent -where all numeric values and the returned arc length are in degrees. +Details of the mechanism for reporting the out of range arguments are +implementation dependent. -The equivalent call to the four parameter form of the function would be: -\begin{verbatim} - DISTANCE(25.0, -19.5, 25.4, -20.0) -\end{verbatim} +\subsubsection{Coordinate system} +\label{sec:geom.coordsys.param} -The DISTANCE function may be applied to any expression that returns a -geometric POINT value. Behaviour for expressions returning a geometry different -from a POINT is undefined at this point (but may be defined later). +For historical reasons, the geometry constructors (\verb:BOX:, \verb:CIRCLE:, +\verb:POINT: and \verb:POLYGON:) all accept a string literal as the first +argument, hereafter called the COOSYS argument. -For example, the distance between two points stored in the database could -be calculated as follows: -\begin{verbatim} - DISTANCE(t1.base, t2.target) -\end{verbatim} -\noindent -where \textit{t1.base} and \textit{t2.target} are references to -database columns that contain POINT values. +The COOSYS argument was originally intended to carry information on +a reference system or other coordinate system metadata. This was helpful in +order to deal with data specified in different coordinate systems while +performing geometric operations. It was up to the ADQL service to perform +the appropriate conversion to make these operations possible. -%coordsys trans -Geometric arguments SHOULD be expressed in the same coordinate system, even in -the four numeric parameter form. See \SectionRef{sec:geom.coordsys.param} for -more details. +Since version 2.1, this argument is deprecated and has been made optional. +Future versions of this specification will remove this parameter from the listed +functions. -\subsubsection{INTERSECTS} -\label{sec:functions.geom.intersects} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: INTERSECTS|}\\ +Coordinate conversions SHOULD now be explictly requested. The ADQL implementers +have to allow it through User Defined Functions. An interoperable facility for +frame transformations is in preparation as of this writing and is expected to be +part of the \CatalogueUDF{}. +% Catalogue of {ADQL} User Defined Functions - Endorsed Note 1.0+ +% http://www.ivoa.net/documents/udf-catalogue/index.html -The INTERSECTS function determines if two geometry values overlap. This is -most commonly used to express a ``shape-vs-shape'' intersection test. +\verb:DISTANCE:, \verb:CONTAINS: and \verb:INTERSECTS: MAY still convert +coordinates of its geometric operands if they are expressed in different +coordinate systems. However, be aware that in a future version of ADQL, these +functions will no longer be expected to perform any coordinate conversion. +Consequently, it is recommanded to avoid relying on this deprecated feature. +For interoperability reasons, queries against 2.0 or later services SHOULD NOT +pass arguments with differing COOSYS arguments to \verb:DISTANCE:, +\verb:CONTAINS: or \verb:INTERSECTS:, as behaviour is undefined in that case. -For example, an expression to determine whether a circle of one degree radius -centered on position (25.4, -20.0) degrees overlaps with a POLYGON, could be -written as follows: -\begin{verbatim} - INTERSECTS(CIRCLE(25.4, -20.0, 1), - POLYGON(20.0, -15.0, - 20.0, -5.0, - 10.0, -5.0, - 10.0, -15.0)) -\end{verbatim} -\noindent -where the INTERSECTS function returns the integer value 1 if the two arguments -overlap and 0 if they do not. +\subsubsection{Predicate functions} +\label{sec:functions.geom.predicate} -When used as a predicate in the WHERE clause of a query, the returned integer -value should be compared to the integer values 0 or 1 to form a SQL predicate: -\begin{verbatim} - WHERE 1 = INTERSECTS(CIRCLE(25.4, -20.0, 1), - POLYGON(20.0, -15.0, - 20.0, -5.0, - 10.0, -5.0, - 10.0, -15.0)) -\end{verbatim} -\noindent -for ``does intersect'' and -\begin{verbatim} - WHERE 0 = INTERSECTS(CIRCLE(25.4, -20.0, 1), - POLYGON(20.0, -15.0, - 20.0, -5.0, - 10.0, -5.0, - 10.0, -15.0)) -\end{verbatim} -\noindent -for ``does not intersect''. +Functions CONTAINS and INTERSECTS each accept two geometry datatypes +and return a numeric value of 1 or 0 according to whether the relevant +verb (e.g. contains) is satisfied against the two input geometries; +1 if the condition is met and 0 if it is not. -The geometric arguments for INTERSECTS may be literal values, as above, -or they may be column references, functions or expressions that return -geometric values. +Each of these functions can be used as a WHERE clause predicate by +comparing the numeric result with zero or one. For example: \begin{verbatim} - WHERE 0 = INTERSECTS(t1.target, t2.footprint) + SELECT * + FROM table + WHERE 1 = CONTAINS(POINT(...), CIRCLE(...)) \end{verbatim} -where \textit{t1.target} and \textit{t2.footprint} are references to -database columns that contain geometric (BOX, CIRCLE, POLYGON or REGION) values. - -The arguments to INTERSECTS SHOULD be geometric expressions evaluating to -either BOX, CIRCLE, POLYGON or REGION. -Previous versions of this specification also allowed POINT values and required -server implementations to interpret the expression as a CONTAINS with the POINT -moved into the first position. Server implementations SHOULD still implement -that behaviour, but clients SHOULD NOT expect it. This behaviour MAY be dropped -in the next major version of this specification. +%REMOVED - speculative, not definitive. +%\noindent +%One would expect later additions to ADQL to add to this range of functions. For +%example, equals, disjoint, touches, crosses, within, overlaps and relate +%are possibilities. -%coordsys trans -Geometric arguments SHOULD be expressed in the same coordinate system. -See \SectionRef{sec:geom.coordsys.param} for more details. +\subsubsection{Preferred sky crossmatch syntax} +\label{sec:functions.geom.crossmatch} -\subsubsection{POINT} -\label{sec:functions.geom.point} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: POINT|}\\ +An especially common operation that astronomers require when working +with source catalogues is the positional sky crossmatch. +In its simplest form this is a join between two tables with the +requirement that the distance along a great circle between the +sky positions of the two associated rows is less than or equal to +a given threshold. -The POINT function expresses a single location on the sky, -and it corresponds semantically to the equivalent term, SpatialCoord, defined in -the \STCSpec{}. -%(STC SpatialCoord, Section 4.4.2.2) +The geometrical functions provided by ADQL offer a number of +semantically equivalent ways to specify such a condition +in either the JOIN or the WHERE clause, using various +combinations of POINT, CIRCLE and DISTANCE. +While a correct implementation MUST generate the same result for +any of these alternatives, the performance characteristics may +differ dramatically depending on implementation. +Given this, it is difficult for (human or machine) ADQL authors +to know how to phrase a crossmatch with the expectation that it +will be executed efficiently, and difficult for services to know +which forms of query to optimise. The result can be the +unnecessarily slow operation of the common sky crossmatch operation. -The function arguments specify the position, where: -\begin{itemize} - \item the position is given by a pair of numeric coordinates in degrees - \item the values of coordinates are subject to the constraints laid down in - \SectionRef{sec:functions.geom.limits} -\end{itemize} +This section therefore recommends a preferred form of ADQL +to use for sky crossmatching and the related cone search operation, +namely to impose an upper limit on one of the two forms of the +DISTANCE function. +Clients submitting crossmatch-like and cone-like +queries are advised to phrase them in this way rather than using semantically +equivalent alternatives, and services are encouraged to ensure that +these forms of query are executed efficiently; this might involve +identifying such ADQL input clauses and rewriting them appropriately +for efficient processing on the database backend. +Alternative semantically equivalent forms however MAY still be +used by clients, and MUST still be handled correctly by services. -For example, a function expressing a point with right ascension of 25 degrees -and declination of -19.5 degrees would be written as follows: +An example sky position-only crossmatch joining rows of tables +\textit{t1} and \textit{t2} within one arcsecond might therefore look like: \begin{verbatim} - POINT(25.0, -19.5) + JOIN ... + ON DISTANCE(t1.ra, t1.dec, t2.ra, t2.dec) < 0.00027 \end{verbatim} -\noindent -where numeric values are in degrees. - -The coordinates for POINT may be literal values, as above, -or they may be column references, functions or expressions that return -numeric values. -For example: +and a cone search for rows within 2.5 degrees of +the center of M31 might look like: \begin{verbatim} - POINT(t.ra, t.dec) + SELECT ... + WHERE DISTANCE(t.pos, POINT(10.68, 41.27)) <= 2.5 \end{verbatim} -\noindent -where \textit{t.ra} and \textit{t.dec} are references to database -columns that contain numeric values. -%TODO - ObsCore example - -%coordsys param -For historical reasons, the POINT function accepts an optional string literal -as the first argument. -As of version 2.1 of the specification this parameter has been -marked as deprecated. -Future versions of this specification may remove this parameter -\SectionSee{sec:geom.coordsys.param}. -\subsubsection{POLYGON} -\label{sec:functions.geom.polygon} +\subsubsection{AREA} +\label{sec:functions.geom.area} {\footnotesize Language feature :}\\ {\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: POLYGON|}\\ - -The POLYGON function expresses a region on the sky with boundaries denoted by great -circles passing through specified coordinates. It corresponds semantically -to the STC Polygon. -%(STC Polygon, Section 4.5.1.4) - -A polygon is described by a list of vertices in a single coordinate system, with -each vertex connected to the next along a great circle and the last vertex -implicitly connected to the first vertex. +{\footnotesize \verb|name: AREA|}\\ -The function arguments specify three or more vertices, where: -\begin{itemize} - \item the position of the vertices are given as a sequence of numeric - coordinates in degrees, or as a sequence of geometric POINTs - \item the values of coordinates are subject to the constraints laid down in - \SectionRef{sec:functions.geom.limits} -\end{itemize} +The AREA function computes the area, in square degrees, of a given geometry. -For example, a function expressing a triangle with vertices at (10.0, --10.5), (20.0, 20.5) and (30.0,30.5) in degrees would be written -as follows: +For example, an expression to calculate the area of a POLYGON could be +written as follows: \begin{verbatim} - POLYGON(10.0, -10.5, 20.0, 20.5, 30.0, 30.5) + AREA(POLYGON(10.0, -10.5, 20.0, 20.5, 30.0, 30.5)) \end{verbatim} -\noindent -where all numeric values are in degrees. -The coordinates for the POLYGON vertices may be literal values, as above, -or they may be column references, functions or expressions that return -numeric values. -For example: -\begin{verbatim} - POLYGON(t1.ra , t1.dec + 5, - t1.ra - 5, t1.dec - 5, - t1.ra - 5, t1.dec + 5) -\end{verbatim} -\noindent -where \textit{t1.ra} and \textit{t1.dec} are references to database columns -that contain numeric values. -%TODO - ObsCore example +The AREA of a single POINT is zero. -Alternatively, the coordinates for the POLYGON vertices may be column references, -functions or expressions that return POINT values. +The geometry argument may be a literal value, as above, or it may be a +column reference, function or expression that returns a geometric type. For example: \begin{verbatim} - POLYGON(t2.toppoint, t2.bottomleft, t2.bottomright) + AREA(t1.footprint) \end{verbatim} -\noindent -where \textit{t2.toppoint}, \textit{t2.bottomleft} and \textit{t2.bottomright} -are references to database columns that contain POINT values. -%TODO - ObsCore example - -The coordinates for the vertices MUST all be expressed in the same datatype. -The POLYGON function does not support a mixture of numeric and POINT -arguments. - -%coordsys param -For historical reasons, the POLYGON function accepts an optional string literal -as the first argument. -As of version 2.1 of the specification this parameter has been -marked as deprecated. -Future versions of this specification may remove this parameter -\SectionSee{sec:geom.coordsys.param}. - -\subsubsection{REGION} -\label{sec:functions.geom.region} +where \textit{t1.footprint} is a reference to a database column that +contains geometric (POINT, BOX, CIRCLE, POLYGON or REGION) values. +\subsubsection{BOX} +\label{sec:functions.geom.box} {\footnotesize Language feature :}\\ {\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ -{\footnotesize \verb|name: REGION|}\\ - +{\footnotesize \verb|name: BOX|}\\ -The REGION function provides a way of expressing a complex region -represented by a single string literal. The standard expressly only -requires literals as arguments rather than string expressions or column -references. The latter would require parsing these representations -within the database, which is not intended. +Note - the \verb|BOX| function has been deprecated in version 2.1 of the standard, +and will be removed from future versions of the specification. -This document does not specify possible syntaxes for REGION literals. A de-facto -standard that many services understanding ADQL 2.0 implemented at least -partially is given by the \STCSAppendix{}, and implementations of ADQL 2.1 are -encouraged to support as much of that as reasonable for them. +The BOX function expresses a box on the sky. A BOX is a special case of POLYGON, +defined purely for convenience, +and it corresponds semantically to the equivalent term, Box, defined in +the \STCSpec{}. +%(STC Box, Section 4.5.1.5) -\subsection{User defined functions} -\label{sec:user.functions} -\subsubsection{Overview} +It is specified by a center position and size +(in both axes) defining a cross centered on the center position and +with arms extending, parallel to the coordinate axes at the center position, +for half the respective sizes on either side. The box’s sides are line +segments or great circles intersecting the arms of the cross in its end +points at right angles with the arms. -ADQL also provides a place holder to define user specific functions. The grammar -definition for user defined functions includes a variable list of parameters. +% Original text in ADQL. +% It is specified by a center position and size +% (in both axes) defining a cross centered on the center position and +% with arms extending, parallel to the coordinate axes at the center position, +% for half the respective sizes on either side. The box’s sides are line +% segments or great circles intersecting the arms of the cross in its end +% points at right angles with the arms. -\begin{verbatim} - ::= - - [ - - [ - { - - }... - ] - ] - -\end{verbatim} +% Text from STC-20071030 +% A Box is a special case of a Polygon, defined purely for convenience. +% It is specified by a center position and size (in both coordinates) +% defining a cross centered on the center position and with arms +% extending, parallel to the coordinate axes at the center position, +% for half the respective sizes on either side. +% The box’s sides are line segments or great circles intersecting the +% arms of the cross in its end points at right angles with the arms. -In order to avoid name conflicts, user defined function names SHOULD include -a prefix which indicates the name of the institute or project which created -the function. +The function arguments specify the center position and the width and height, +where: +\begin{itemize} + \item the center position is given by a pair of numeric coordinates + in degrees, or a single geometric POINT + \item the values of coordinates of the center position are subject to the + constraints laid down in \SectionRef{sec:functions.geom.limits} + \item the width and height are given by numeric values in degrees +\end{itemize} -For example, the names of \verb:align: and \verb:convert: functions developed -by the Wide Field Astronomy Unit (WFAU) could be prefixed as follows: +For example, a BOX of ten degrees centered on a position +(25.4, -20.0) in degrees could be written as follows: \begin{verbatim} - wfau_align() - wfau_convert() + BOX(25.4, -20.0, 10.0, 10.0) \end{verbatim} -This enables users to distinguish between functions with similar names developed -by a different service provider, e.g. the German Astrophysical Virtual -Observatory (GAVO): +Alternatively, the center position could be expressed as a POINT: \begin{verbatim} - gavo_align() - gavo_convert() + BOX(POINT(25.4, -20.0), 10.0, 10.0) \end{verbatim} -The \verb:ivo: prefix is reserved for functions that have been defined in an -IVOA specification or Endorsed Note. For example the \CatalogueUDF{} defines the -following functions: +The function arguments may be literal values, as above, or they may be +column references, functions or expressions that return the appropriate +datatypes. +For example: \begin{verbatim} - ivo_nocasematch() - ivo_hasword() - ivo_hashlist_has() - ivo_string_agg() + BOX(t1.center, t1.width, t1.height) \end{verbatim} +where \textit{t1.center}, \textit{t1.width} and \textit{t1.height} +are references to database columns that contain POINT, DOUBLE +and DOUBLE values respectively. +%TODO - ObsCore example -The \CatalogueUDF{} collects many \verb:ivo: prefixed functions defined in other -IVOA specifications, Endorsed Notes and functions defined in at least two -implementations. At the time of this writing it is the only endorsed IVOA -document providing such a collection of \verb:ivo: prefixed functions. It is -then strongly recommended to follow function names and definitions from this -catalogue when providing functions offering identical or similar functionnality. - -\subsubsection{Metadata} -\label{sec:user.metadata} +%coordsys param +For historical reasons, the BOX function accepts an optional string literal as +the first argument. +As of version 2.1 of the specification this parameter has been +marked as deprecated. +Future versions of this specification may remove this parameter +\SectionSee{sec:geom.coordsys.param}. -The URI for identifying the language feature for a user defined function -is defined as part of the \TAPRegSpec{}. +\subsubsection{CENTROID} +\label{sec:functions.geom.centroid} +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: CENTROID|}\\ -\begin{verbatim} - ivo://ivoa.net/std/tapregext#features-udf -\end{verbatim} +The CENTROID function computes the centroid of a given geometry and returns a POINT. -For user defined functions, the \verb:form: element of the language feature -declaration must contain the signature of the function, written to match -the signature nonterminal in the following grammar: +For example, an expression to calculate the centroid of a POLYGON could +be written as follows : \begin{verbatim} - signature ::= "->" - funcname ::= - arglist ::= "(" { "," } ")" - arg ::= + CENTROID(POLYGON(10.0, -10.5, 20.0, 20.5, 30.0, 30.5)) \end{verbatim} -\verb:: should be one of the terms defined in \SectionRef{sec:types}. +The CENTROID of a single POINT is that POINT. -For example, the following fragment declares a user defined function that -takes two string parameters and returns an integer, zero or one, -depending on the regular expression pattern matching: +The geometry argument may be a literal value, as above, or it may be a +column reference, function or expression that returns a geometric type. +For example: \begin{verbatim} - - -
match(pattern VARCHAR, string VARCHAR) -> INTEGER
- - match returns 1 if the POSIX regular expression pattern - matches anything in string, 0 otherwise. - - - + CENTROID(t1.footprint) \end{verbatim} +where \textit{t1.footprint} is a reference to a database column that +contains geometric (POINT, BOX, CIRCLE, POLYGON or REGION) values. +%TODO - ObsCore example -See the \TAPRegSpec{} for full details on how to use the -XML schema to declare user defined functions. - -\subsection{String functions and operators} -\label{sec:string.functions} - -An ADQL service implementation MAY include support for the following optional -string manipulation and comparison operators: - -\begin{itemize} - \item \verb:LOWER(): Lower case conversion - \item \verb:UPPER(): Upper case conversion - \item \verb:ILIKE: Case-insensitive comparison. -\end{itemize} - -\subsubsection{Case folding} +\subsubsection{CIRCLE} +\label{sec:functions.geom.circle} +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: CIRCLE|}\\ -Since case folding is a nontrivial operation in a multi-encoding world, ADQL -requires standard behaviour for the ASCII characters, and recommends -following algorithms described in Section 3.13, ``Default Case Algorithms'' -of \citet{std:UNICODE} for characters outside the ASCII set: +The CIRCLE function expresses a circular region on the sky (a cone in space), +and it corresponds semantically to the equivalent term, Circle, defined in +the \STCSpec{}. +%(STC Circle, Section 4.5.1.2) +The function arguments specify the center position and the radius, where: \begin{itemize} - \item algorithm R1 for \verb:UPPER(): - \item algorithm R2 for \verb:LOWER(): and \verb:ILIKE: + \item the center position is given by a pair of numeric coordinates + in degrees, or a single geometric POINT + \item the values of coordinates of the center position are subject to the + constraints laid down in \SectionRef{sec:functions.geom.limits} + \item the radius is a numeric value in degrees. \end{itemize} -\subsubsection{LOWER} -\label{sec:string.functions.lower} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-string|}\\ -{\footnotesize \verb|name: LOWER|}\\ +For example, a CIRCLE of ten degrees radius centered on position +(25.4, -20.0) in degrees could be written as follows: +\begin{verbatim} + CIRCLE(25.4, -20.0, 10.0) +\end{verbatim} -The LOWER function converts its string parameter to lower case in accordance with the rules of the database's locale. +Alternatively, the center position may be expressed as a POINT: +\begin{verbatim} + CIRCLE(POINT(25.4, -20.0), 10.0) +\end{verbatim} +The position argument may be a literal value, as above, or it may be a +column reference, function or expression that returns a geometric type. +For example: \begin{verbatim} - LOWER('Francis Albert Augustus Charles Emmanuel') - => - francis albert augustus charles emmanuel + CIRCLE(t1.center, t1.radius) \end{verbatim} +where \textit{t1.center} and \textit{t1.radius} are references to +database columns that contain POINT and DOUBLE values respectively. +%TODO - ObsCore example -\subsubsection{UPPER} -\label{sec:string.functions.upper} +%coordsys param +For historical reasons, the CIRCLE function accepts an optional string literal +as the first argument. +As of version 2.1 of the specification this parameter has been +marked as deprecated. +Future versions of this specification may remove this parameter +\SectionSee{sec:geom.coordsys.param}. + +\subsubsection{CONTAINS} +\label{sec:functions.geom.contains} {\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-string|}\\ -{\footnotesize \verb|name: UPPER|}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: CONTAINS|}\\ -The UPPER function converts its string parameter to upper case in accordance with the rules of the database's locale. +The CONTAINS function determines if a geometry is wholly contained within +another. This is most commonly used to express a ``point-in-shape'' condition. +For example, an expression to determine whether the point (25.0, -19.5) degrees +is within a circle of ten degrees radius centered on position (25.4, -20.0) +degrees, could be written as follows: \begin{verbatim} - UPPER('Francis Albert Augustus Charles Emmanuel') - => - FRANCIS ALBERT AUGUSTUS CHARLES EMMANUEL + CONTAINS(POINT(25.0, -19.5), CIRCLE(25.4, -20.0, 10.0)) \end{verbatim} -\subsubsection{ILIKE} -\label{sec:string.functions.ilike} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-string|}\\ -{\footnotesize \verb|name: ILIKE|}\\ +The CONTAINS function is not symmetric in the meaning of the arguments. -The ILIKE string comparison operator performs a case-insensitive comparison -of its string operands. +The CONTAINS function returns the integer value 1 if the first argument +is in, or on, the boundary of the second argument and the integer value 0 +if it is not. +When used as a predicate in the WHERE clause of a query, the returned integer +value must be compared to the integer values 0 or 1 to form a SQL predicate: \begin{verbatim} - 'Francis' LIKE 'francis' => False - - 'Francis' ILIKE 'francis' => True + WHERE 1 = CONTAINS(POINT(25.0, -19.5), + CIRCLE(25.4, -20.0, 10.0)) +\end{verbatim} +\noindent +for ``does contain'' and +\begin{verbatim} + WHERE 0 = CONTAINS(POINT(25.0, -19.5), + CIRCLE(25.4, -20.0, 10.0)) \end{verbatim} +\noindent +for ``does not contain''. -\subsection{Common table expressions} -\label{sec:common-table} +%TODO - CONTAINS(thing, POINT) ? -An ADQL service implementation MAY include support for the following optional -common table expressions: +The geometric arguments for CONTAINS may be literal values, as above, +or they may be column references, functions or expressions that return +geometric values. +For example: +\begin{verbatim} + WHERE 0 = CONTAINS(t1.center, t2.footprint) +\end{verbatim} +where \textit{t1.center} and \textit{t2.footprint} are references to +database columns that contain POINT and geometric (BOX, CIRCLE, POLYGON or REGION) +values respectively. +%TODO - ObsCore example -\begin{itemize} - \item \verb:WITH: -\end{itemize} +%coordsys trans +Geometric arguments SHOULD be expressed in the same coordinate system. +See \SectionRef{sec:geom.coordsys.param} for more details. -\subsubsection{WITH} +\subsubsection{COORD1} +\label{sec:functions.geom.coord1} {\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-common-table|}\\ -{\footnotesize \verb|name: WITH|}\\ - -The WITH operator creates a temporary named result set that can be referred -to elsewhere in the main query. +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: COORD1|}\\ -Using a common table expression can make complex queries easier to understand -by factoring subqueries out of the main SQL statement. +The COORD1 function extracts the first coordinate value, in degrees, of a given +POINT \SectionSee{sec:functions.geom.point} or column reference. -For example, the following query with a nested subquery: +For example, the right ascension of a point with position (25, -19.5) in +degrees would be obtained using the following expression: \begin{verbatim} - SELECT ra, dec - FROM ( - SELECT * - FROM alpha_source - WHERE id % 10 = 0 - ) AS alpha_subset - WHERE ra > 10 - AND ra < 20 + COORD1(POINT(25.0, -19.5)) \end{verbatim} \noindent -can be refactored as a named WITH query and a simpler main query: +which would return a numeric value of 25.0 degrees. + +For example: \begin{verbatim} - WITH alpha_subset AS ( - SELECT * - FROM alpha_source - WHERE id % 10 = 0 - ) - SELECT ra, dec - FROM alpha_subset - WHERE ra > 10 - AND ra < 20 + COORD1(t.center) \end{verbatim} +\noindent +where \textit{t.center} is a reference to a column that contains POINT values. -The current version of ADQL does not support recursive common table expressions. - -Common table expressions can be defined only in the main -query. They are not allowed in sub-queries. - -\subsection{Set operators} -\label{sec:set.operators} - -An ADQL service implementation MAY include support for the following optional -set operators: - -\begin{itemize} - \item \verb:UNION: - \item \verb:EXCEPT: - \item \verb:INTERSECT: -\end{itemize} - -For a set operation to be valid in ADQL, the following criteria must be met: -\begin{itemize} - \item the two queries MUST result in the same number of columns - \item the columns in the operands MUST have the same datatypes. -\end{itemize} +\subsubsection{COORD2} +\label{sec:functions.geom.coord2} +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: COORD2|}\\ -In addition, the columns returned by a set operation SHOULD have the same -metadata, e.g. units, UCD, etc. These metadata SHOULD be generated from the -left-hand operand of the set operation. +The COORD2 function extracts the second coordinate value, in degrees, of a given +POINT \SectionSee{sec:functions.geom.point} or column reference. -\subsubsection{UNION} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-sets|}\\ -{\footnotesize \verb|name: UNION|}\\ +For example, the declination of a point with position (25, -19.5) in degrees, +could be obtained using the following expression: +\begin{verbatim} + COORD2(POINT(25.0, -19.5)) +\end{verbatim} +\noindent +which would return a numeric value of -19.5 degrees. -The UNION operator combines the results of two queries, accepting rows from -both the first and second set of results. +The COORD2 function may be applied to any expression that returns a +geometric POINT value. +For example: +\begin{verbatim} + COORD2(t.center) +\end{verbatim} +\noindent +where \textit{t.center} is a reference to a column that contains POINT values. -\subsubsection{EXCEPT} +\subsubsection{COORDSYS} +\label{sec:functions.geom.coordsys} {\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-sets|}\\ -{\footnotesize \verb|name: EXCEPT|}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: COORDSYS|}\\ -The EXCEPT operator combines the results of two queries, accepting rows that are -in the first set of results but are not in the second one. +As of version 2.1 of the specification the COORDSYS function has +been marked as deprecated. This function may be removed in future versions +of this specification. +Details of the coordinate system for a database column are available as part of +the service metadata, available via the \verb:TAP_SCHEMA: tables defined in the +\TAPSpec{} and the \verb:/tables: webservice response defined in the \VOSISpec{}. -\subsubsection{INTERSECT} -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-sets|}\\ -{\footnotesize \verb|name: INTERSECT|}\\ +%As described in \SectionRef{sec:functions.geom.overview}, the allowed return values must be defined +%by any service making use of ADQL, and a list of standard coordinate system +%literals can be found in the STC specification. +% STC-reference 'STC specification [3]' -The INTERSECT operator combines the results of two queries, accepting rows -that are strictly in both the first and second set of results. +The COORDSYS function returns the formal name of the coordinate system for +a given geometry as a string. -\subsubsection{Duplicated rows} +The following example would return the coordinate system of a POINT literal: +\begin{verbatim} + COORDSYS(POINT(25.0, -19.5)) +\end{verbatim} +\noindent +which would return a string value representing the coordinate system used +to create the POINT. -\verb:UNION:, \verb:EXCEPT: and \verb:INTERSECT: remove duplicated rows, -while \verb:UNION ALL:, \verb:EXCEPT ALL: and \verb:INTERSECT ALL: keep all of -them. +The COORDSYS function may be applied to any expression that returns a +geometric datatype. For example: +\begin{verbatim} + COORDSYS(t.footprint) +\end{verbatim} +\noindent +where \textit{t.footprint} is a reference to a database column that +contains geometric (POINT, BOX, CIRCLE, POLYGON or REGION) values. -Note that the comparison used for removing duplicated rows is based purely on -the column value and does not take into account the units. This means that a row -with a numeric value of \verb:2: and unit of \verb:m: and a row with a numeric -value of \verb:2: and unit of \verb:km: will be considered equal, despite the -difference in units. +\subsubsection{DISTANCE} +\label{sec:functions.geom.distance} +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: DISTANCE|}\\ -\subsubsection{Operands} +The DISTANCE function computes the arc length along a great circle between two +points and returns a numeric value expression in degrees. -Operands of any of the set operators can only be \verb:SELECT: queries. -Unless within parentheses, such queries can not use any \verb:ORDER BY: or -\verb:OFFSET: clause. +The specification defines two versions of the DISTANCE function, one that +accepts two POINT values, and a second that accepts four separate numeric +values. -Example: sorting result of a \verb:UNION: operation: +If an ADQL service implementation declares support for DISTANCE, +then it must implement both the two parameter and four parameter +forms of the function. +For example, an expression calculating the distance between two points of +coordinates (25,-19.5) and (25.4,-20) could be written as follows: \begin{verbatim} - SELECT id, ra, dec FROM table1 - UNION - SELECT id, ra, dec FROM table2 - ORDER BY id -- sort the UNION result + DISTANCE(POINT(25.0, -19.5), POINT(25.4, -20.0)) \end{verbatim} +\noindent +where all numeric values and the returned arc length are in degrees. -Example: sorting result of the \verb:UNION: operands: - +The equivalent call to the four parameter form of the function would be: \begin{verbatim} - -- take the 10 first - (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id ASC) - UNION - -- take the 10 last - (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id DESC) + DISTANCE(25.0, -19.5, 25.4, -20.0) \end{verbatim} -Common table expressions are not allowed in any set operator operand. They must -always be declared at the main level. - -Example: sorting result of the \verb:UNION: operands: with common table -expressions +The DISTANCE function may be applied to any expression that returns a +geometric POINT value. Behaviour for expressions returning a geometry different +from a POINT is undefined at this point (but may be defined later). +For example, the distance between two points stored in the database could +be calculated as follows: \begin{verbatim} -WITH tenFirst AS (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id ASC), - tenLast AS (SELECT TOP 10 id, ra, dec FROM atable ORDER BY id DESC) - SELECT * FROM tenFirst -UNION - SELECT * FROM tenLast + DISTANCE(t1.base, t2.target) \end{verbatim} +\noindent +where \textit{t1.base} and \textit{t2.target} are references to +database columns that contain POINT values. -\subsubsection{Precedence} - -When set operators are used together, the resulting expression is -evaluated in the context of the following precedence: +%coordsys trans +Geometric arguments SHOULD be expressed in the same coordinate system, even in +the four numeric parameter form. See \SectionRef{sec:geom.coordsys.param} for +more details. -\begin{enumerate} - \item Expressions within parentheses - \item The \verb:INTERSECT: operator - \item The \verb:UNION: and \verb:EXCEPT: operators evaluated from left to right -\end{enumerate} +\subsubsection{INTERSECTS} +\label{sec:functions.geom.intersects} +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: INTERSECTS|}\\ -Example: +The INTERSECTS function determines if two geometry values overlap. This is +most commonly used to express a ``shape-vs-shape'' intersection test. +For example, an expression to determine whether a circle of one degree radius +centered on position (25.4, -20.0) degrees overlaps with a POLYGON, could be +written as follows: \begin{verbatim} - SELECT id, ra, dec FROM table1 - UNION - SELECT id, ra, dec FROM table2 - INTERSECT - SELECT id, ra, dec FROM table3 + INTERSECTS(CIRCLE(25.4, -20.0, 1), + POLYGON(20.0, -15.0, + 20.0, -5.0, + 10.0, -5.0, + 10.0, -15.0)) \end{verbatim} +\noindent +where the INTERSECTS function returns the integer value 1 if the two arguments +overlap and 0 if they do not. -is equivalent to: +When used as a predicate in the WHERE clause of a query, the returned integer +value should be compared to the integer values 0 or 1 to form a SQL predicate: +\begin{verbatim} + WHERE 1 = INTERSECTS(CIRCLE(25.4, -20.0, 1), + POLYGON(20.0, -15.0, + 20.0, -5.0, + 10.0, -5.0, + 10.0, -15.0)) +\end{verbatim} +\noindent +for ``does intersect'' and +\begin{verbatim} + WHERE 0 = INTERSECTS(CIRCLE(25.4, -20.0, 1), + POLYGON(20.0, -15.0, + 20.0, -5.0, + 10.0, -5.0, + 10.0, -15.0)) +\end{verbatim} +\noindent +for ``does not intersect''. +The geometric arguments for INTERSECTS may be literal values, as above, +or they may be column references, functions or expressions that return +geometric values. +For example: \begin{verbatim} - SELECT id, ra, dec FROM table1 - UNION - ( - SELECT id, ra, dec FROM table2 - INTERSECT - SELECT id, ra, dec FROM table3 - ) + WHERE 0 = INTERSECTS(t1.target, t2.footprint) \end{verbatim} +where \textit{t1.target} and \textit{t2.footprint} are references to +database columns that contain geometric (BOX, CIRCLE, POLYGON or REGION) values. -\subsection{Type operations} -\label{sec:type} +The arguments to INTERSECTS SHOULD be geometric expressions evaluating to +either BOX, CIRCLE, POLYGON or REGION. -An ADQL service implementation MAY include support for the following optional -type conversion functions: +Previous versions of this specification also allowed POINT values and required +server implementations to interpret the expression as a CONTAINS with the POINT +moved into the first position. Server implementations SHOULD still implement +that behaviour, but clients SHOULD NOT expect it. This behaviour MAY be dropped +in the next major version of this specification. -\begin{itemize} - \item \verb:CAST(): -\end{itemize} +%coordsys trans +Geometric arguments SHOULD be expressed in the same coordinate system. +See \SectionRef{sec:geom.coordsys.param} for more details. -\subsubsection{CAST} -\label{sec:type.cast} +\subsubsection{POINT} +\label{sec:functions.geom.point} {\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-type|}\\ -{\footnotesize \verb|name: CAST|}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: POINT|}\\ -The \verb:CAST(): function returns the value of the first argument converted -into the datatype specified by the second argument. +The POINT function expresses a single location on the sky, +and it corresponds semantically to the equivalent term, SpatialCoord, defined in +the \STCSpec{}. +%(STC SpatialCoord, Section 4.4.2.2) -\paragraph{Syntax} \verb:: +The function arguments specify the position, where: +\begin{itemize} + \item the position is given by a pair of numeric coordinates in degrees + \item the values of coordinates are subject to the constraints laid down in + \SectionRef{sec:functions.geom.limits} +\end{itemize} + +For example, a function expressing a point with right ascension of 25 degrees +and declination of -19.5 degrees would be written as follows: \begin{verbatim} -CAST - AS - + POINT(25.0, -19.5) \end{verbatim} +\noindent +where numeric values are in degrees. -\paragraph{Target types} +The coordinates for POINT may be literal values, as above, +or they may be column references, functions or expressions that return +numeric values. +For example: +\begin{verbatim} + POINT(t.ra, t.dec) +\end{verbatim} +\noindent +where \textit{t.ra} and \textit{t.dec} are references to database +columns that contain numeric values. +%TODO - ObsCore example -This function does not replicate the full functionality and range of types -supported by common RDBMS implementations of \verb:CAST():. Here is the minimum -range of types that MUST be supported if \verb:CAST(): is implemented: +%coordsys param +For historical reasons, the POINT function accepts an optional string literal +as the first argument. +As of version 2.1 of the specification this parameter has been +marked as deprecated. +Future versions of this specification may remove this parameter +\SectionSee{sec:geom.coordsys.param}. -\begin{itemize} - \item Exact numeric: - \begin{itemize} - \item \verb:INTEGER: - \item \verb:SMALLINT: - \item \verb:BIGINT: - \end{itemize} - \item Approximate numeric: - \begin{itemize} - \item \verb:REAL: - \item \verb:DOUBLE PRECISION: - \end{itemize} - \item Character: - \begin{itemize} - \item \verb:CHAR: or \verb:CHAR(n): (where n is the fixed string length) - \item \verb:VARCHAR: or \verb:VARCHAR(n): (where n is the maximum string length) - \end{itemize} - \item Date, Time: - \begin{itemize} - \item \verb:TIMESTAMP: - \end{itemize} +\subsubsection{POLYGON} +\label{sec:functions.geom.polygon} +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: POLYGON|}\\ + +The POLYGON function expresses a region on the sky with boundaries denoted by great +circles passing through specified coordinates. It corresponds semantically +to the STC Polygon. +%(STC Polygon, Section 4.5.1.4) + +A polygon is described by a list of vertices in a single coordinate system, with +each vertex connected to the next along a great circle and the last vertex +implicitly connected to the first vertex. + +The function arguments specify three or more vertices, where: +\begin{itemize} + \item the position of the vertices are given as a sequence of numeric + coordinates in degrees, or as a sequence of geometric POINTs + \item the values of coordinates are subject to the constraints laid down in + \SectionRef{sec:functions.geom.limits} \end{itemize} -Examples: +For example, a function expressing a triangle with vertices at (10.0, +-10.5), (20.0, 20.5) and (30.0,30.5) in degrees would be written +as follows: +\begin{verbatim} + POLYGON(10.0, -10.5, 20.0, 20.5, 30.0, 30.5) +\end{verbatim} +\noindent +where all numeric values are in degrees. +The coordinates for the POLYGON vertices may be literal values, as above, +or they may be column references, functions or expressions that return +numeric values. +For example: \begin{verbatim} - CAST(3 AS REAL) - CAST('3.14159265358979323846' AS DOUBLE PRECISION) + POLYGON(t1.ra , t1.dec + 5, + t1.ra - 5, t1.dec - 5, + t1.ra - 5, t1.dec + 5) \end{verbatim} +\noindent +where \textit{t1.ra} and \textit{t1.dec} are references to database columns +that contain numeric values. +%TODO - ObsCore example -\paragraph{Input types} +Alternatively, the coordinates for the POLYGON vertices may be column references, +functions or expressions that return POINT values. +For example: +\begin{verbatim} + POLYGON(t2.toppoint, t2.bottomleft, t2.bottomright) +\end{verbatim} +\noindent +where \textit{t2.toppoint}, \textit{t2.bottomleft} and \textit{t2.bottomright} +are references to database columns that contain POINT values. +%TODO - ObsCore example -The range of types allowed for the value to cast entirely depends on the target -type. Although cast operations may vary from one implementation to another, ADQL -SHOULD support the ones listed in Table \ref{table:cast.inputtypes}. +The coordinates for the vertices MUST all be expressed in the same datatype. +The POLYGON function does not support a mixture of numeric and POINT +arguments. -\begin{table}[!h] - \center{ - \resizebox{\linewidth}{!}{ - \begin{tabular}{| c | c | c | c | c | c |} - \hline - \multirow{3}{*}{\diaghead{\theadfont Output TyInput Ty}% - {\textbf{Input}}{\textbf{Output}}} - & \textbf{Exact} & \textbf{Approximate} & \textbf{Variable} & \textbf{Fixed} & \\ - & \textbf{numeric} & \textbf{numeric} & \textbf{length} & \textbf{length} & \textbf{Timestamp} \\ - & & & \textbf{character} & \textbf{character} & \\ - \hline - \textbf{Exact} & \multirow{2}{*}{X} & \multirow{2}{*}{X} & \multirow{2}{*}{X} & \multirow{2}{*}{X*} & \\ - \textbf{numeric} & & & & & \\ - \hline - \textbf{Approximate} & \multirow{2}{*}{X*} & \multirow{2}{*}{X} & \multirow{2}{*}{X} & \multirow{2}{*}{X*} & \\ - \textbf{numeric} & & & & & \\ - \hline - \textbf{Character} & X & X & X & X* & X \\ - \hline - \textbf{Timestamp} & & & X & X* & X \\ - \hline - \end{tabular} - } - \textit{\footnotesize{X: supported ; X*: supported but possible implementation differences}} - \caption{CAST allowed types} - \label{table:cast.inputtypes} - } -\end{table} +%coordsys param +For historical reasons, the POLYGON function accepts an optional string literal +as the first argument. +As of version 2.1 of the specification this parameter has been +marked as deprecated. +Future versions of this specification may remove this parameter +\SectionSee{sec:geom.coordsys.param}. -\paragraph{Cast into a smaller datatype} +\subsubsection{REGION} +\label{sec:functions.geom.region} -Converting a value to a datatype that is too small to represent it SHOULD be -treated as an error. Details of the mechanism for reporting the error condition -are implementation dependent. +{\footnotesize Language feature :}\\ +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adqlgeo|}\\ +{\footnotesize \verb|name: REGION|}\\ -This rule especially applies when casting a value into a character string too -small to contain its entire serialization. The output string may be truncated, -adjusted to the needed length, or an error may be thrown. -\paragraph{Fixed-length character} +The REGION function provides a way of expressing a complex region +represented by a single string literal. The standard expressly only +requires literals as arguments rather than string expressions or column +references. The latter would require parsing these representations +within the database, which is not intended. -The creation of a fixed-length character string is implementation dependent. -In function of the implementation, \verb:CHAR: may be equivalent to -\verb:CHAR(1): or to a \verb:CHAR: just big enough to contain the entire string -to create. +This document does not specify possible syntaxes for REGION literals. A de-facto +standard that many services understanding ADQL 2.0 implemented at least +partially is given by the \STCSAppendix{}, and implementations of ADQL 2.1 are +encouraged to support as much of that as reasonable for them. -\paragraph{Approximate numeric} +\subsection{User defined functions} +\label{sec:user.functions} +\subsubsection{Overview} -The rounding mechanism used when converting from approximate numerics -(\verb:REAL: or \verb:DOUBLE PRECISION:) to precise numerics (\verb:SMALLINT:, -\verb:INTEGER: or \verb:BIGINT:) is implementation dependent. +ADQL also provides a place holder to define user specific functions. The grammar +definition for user defined functions includes a variable list of parameters. -\paragraph{Timestamp} +\begin{verbatim} + ::= + + [ + + [ + { + + }... + ] + ] + +\end{verbatim} -Only a character string can be casted into a timestamp. This string MUST follow -the syntax defined in the \DALISpec{}: +In order to avoid name conflicts, user defined function names SHOULD include +a prefix which indicates the name of the institute or project which created +the function. + +For example, the names of \verb:align: and \verb:convert: functions developed +by the Wide Field Astronomy Unit (WFAU) could be prefixed as follows: \begin{verbatim} - YYYY-MM-DD[’T’hh:mm:ss[.SSS][’Z’]] + wfau_align() + wfau_convert() \end{verbatim} -Example: +This enables users to distinguish between functions with similar names developed +by a different service provider, e.g. the German Astrophysical Virtual +Observatory (GAVO): +\begin{verbatim} + gavo_align() + gavo_convert() +\end{verbatim} +The \verb:ivo: prefix is reserved for functions that have been defined in an +IVOA specification or Endorsed Note. For example the \CatalogueUDF{} defines the +following functions: \begin{verbatim} - CAST('2021-01-14T11:25:00' AS TIMESTAMP) + ivo_nocasematch() + ivo_hasword() + ivo_hashlist_has() + ivo_string_agg() \end{verbatim} -Note that other serializations or any other kind of value MAY also be supported. +The \CatalogueUDF{} collects many \verb:ivo: prefixed functions defined in other +IVOA specifications, Endorsed Notes and functions defined in at least two +implementations. At the time of this writing it is the only endorsed IVOA +document providing such a collection of \verb:ivo: prefixed functions. It is +then strongly recommended to follow function names and definitions from this +catalogue when providing functions offering identical or similar functionnality. -\paragraph{Geometry} +\subsubsection{Metadata} +\label{sec:user.metadata} -\verb:CAST(): MAY also produce geometries. If an implementation wants to support -this particular cast operation, it MUST accept a character string following the -DALI serialization matching the precise geometry type to produce. +The URI for identifying the language feature for a user defined function +is defined as part of the \TAPRegSpec{}. -Then, the supported geometry types SHOULD be: +\begin{verbatim} + ivo://ivoa.net/std/tapregext#features-udf +\end{verbatim} -\begin{itemize} - \item \verb:POINT: - \item \verb:CIRCLE: - \item \verb:POLYGON: -\end{itemize} +For user defined functions, the \verb:form: element of the language feature +declaration must contain the signature of the function, written to match +the signature nonterminal in the following grammar: +\begin{verbatim} + signature ::= "->" + funcname ::= + arglist ::= "(" { "," } ")" + arg ::= +\end{verbatim} -Examples: +\verb:: should be one of the terms defined in \SectionRef{sec:types}. +For example, the following fragment declares a user defined function that +takes two string parameters and returns an integer, zero or one, +depending on the regular expression pattern matching: \begin{verbatim} - CAST('12.3 45.6' AS POINT) - CAST('12.3 45.6 1.0' AS CIRCLE) - CAST('1.0 0.1 2.0 0.2 3.0 0.3' AS POLYGON) + + +
match(pattern VARCHAR, string VARCHAR) -> INTEGER
+ + match returns 1 if the POSIX regular expression pattern + matches anything in string, 0 otherwise. + + + \end{verbatim} -Note that other serializations (e.g. STC-S) or any other kind of value MAY also -be supported. +See the \TAPRegSpec{} for full details on how to use the +XML schema to declare user defined functions. -\subsection{Conditional Functions} -\label{sec:condfunc} +\subsection{String operator} +\label{sec:optional.string.functions} An ADQL service implementation MAY include support for the following optional -conditional functions: +string operators: \begin{itemize} - \item \verb:COALESCE(): + \item \verb:ILIKE: Case-insensitive comparison. \end{itemize} -\subsubsection{COALESCE} +\subsubsection{ILIKE} +\label{sec:optional.string.functions.ilike} {\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-conditional|}\\ -{\footnotesize \verb|name: COALESCE|}\\ - -The COALESCE function returns the first of its arguments that is not -NULL. NULL is returned only if all arguments are NULL. - -All arguments must be of the same datatype. An error should be returned -if this rule is not respected. The way to report this error is -implementation dependent. +{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-string|}\\ +{\footnotesize \verb|name: ILIKE|}\\ -This is typically used to provide fallback values. For instance, +The ILIKE string comparison operator performs a case-insensitive comparison +of its string operands. \begin{verbatim} - COALESCE(access_url, '') -\end{verbatim} + 'Francis' LIKE 'francis' => False -\noindent will return an empty string when \verb|access_url| is NULL. + 'Francis' ILIKE 'francis' => True +\end{verbatim} +% Supported by PostgreSQL but not by SQLServer (only LIKE is available but the +% ADQL's ILIKE could be translated into LOWER(a) LIKE LOWER(b)...are +% performances really worst?...to be tested and discussed) \subsection{Unit operations} \label{sec:unit} @@ -2737,39 +2749,6 @@ \subsubsection{IN\_UNIT} implementation dependent. This mechanism is OPTIONAL and is described here as it could significantly improve the behavior of \verb:IN_UNIT():. -\subsection{Cardinality} -\label{sec:cardinality} - -An ADQL service implementation MAY include support for the following optional -clauses to modify the cardinality of query results: - -\begin{itemize} - \item \verb:OFFSET: -\end{itemize} - -\subsubsection{OFFSET} -\label{sec:offset} - -{\footnotesize Language feature :}\\ -{\footnotesize \verb|type: ivo://ivoa.net/std/tapregext#features-adql-offset|}\\ -{\footnotesize \verb|name: OFFSET|}\\ - -An ADQL service implementation MAY include support for the OFFSET clause -which limits the number of rows returned by removing a specified number -of rows from the beginning of the result set. - -If a query contains both an ORDER BY clause and an OFFSET clause, -then the ORDER BY is applied before the specified number of -rows are dropped by the OFFSET clause. - -If the total number of rows is less than the value -specified by the OFFSET clause, then the result set is empty. - -If a query contains both an OFFSET clause and a TOP clause, -then the OFFSET clause is applied first, dropping the specified -number of rows from the beginning of the result set before the -TOP clause is applied to limit the number of rows returned. - \clearpage % section cut \appendix \section[BNF grammar]{BNF grammar \footnote{ @@ -2818,18 +2797,23 @@ \subsection{Between 2.1 and 2.2} \label{sec:changes-2.2} \begin{itemize} - \item \textbf{Applied ADQL-2.1's errata}: - \begin{itemize} - \item Erratum 1 - Addition of auxiliary files for the BNF grammar - \end{itemize} - \item \textbf{General} + \item Apply Erratum 1 - Addition of auxiliary files for the BNF grammar + \item Convert the tables for mathematical and trigonometrical + functions into lists (see \SectionRef{sec:math.functions}) + \item Make some features mandatory \begin{itemize} - \item \textbf{Updated} - \begin{itemize} - \item Convert the tables for mathematical and trigonometrical - functions into lists (see \SectionRef{sec:math.functions}) - \end{itemize} + \item \texttt{LOWER} (see \SectionRef{sec:string.functions.lower}) + \item \texttt{UPPER} (see \SectionRef{sec:string.functions.upper}) + \item \texttt{OFFSET} (see \SectionRef{sec:offset}) + \item \texttt{COALESCE} (see \SectionRef{sec:coalesce}) + \item \texttt{CAST} (see \SectionRef{sec:type.cast}) + \item Common Table Expressions (\texttt{WITH} clause) + (see \SectionRef{sec:common-table}) + \item Set operations (\texttt{UNION}, \texttt{INTERSECT} and + \texttt{EXCEPT}) (see \SectionRef{sec:set.operators}) \end{itemize} + \item \texttt{OFFSET} require the usage of \texttt{ORDER BY} when it is used + (see \SectionRef{sec:offset}) \end{itemize} \subsection{Between 2.0 and 2.1} @@ -2885,8 +2869,10 @@ \subsection{Between 2.0 and 2.1} \item \textbf{Added} \begin{itemize} \item Case sensitive functions and operators: - \verb:LOWER():, \verb:UPPER(): and \verb:ILIKE: + \verb:LOWER():, \verb:UPPER(): \SectionSee{sec:string.functions} + and \verb:ILIKE: + \SectionSee{sec:optional.string.functions.ilike} \item Common table expressions (i.e. \verb:WITH: keyword) \SectionSee{sec:common-table} \item Set operators: \verb:UNION:, \verb:INTERSECT: and @@ -2898,7 +2884,7 @@ \subsection{Between 2.0 and 2.1} \item Unit conversion function: \verb:IN_UNIT(): \SectionSee{sec:unit} \item \verb:OFFSET: - \SectionSee{sec:cardinality} + \SectionSee{sec:offset} \end{itemize} \item \textbf{Updated} \begin{itemize}