From bdb27e811762b200955ab8a85addfba146345466 Mon Sep 17 00:00:00 2001 From: Luke Sneeringer Date: Thu, 3 Sep 2020 15:53:06 -0700 Subject: [PATCH 1/2] =?UTF-8?q?feat:=20AIP-165=20=E2=80=93=20Criteria-base?= =?UTF-8?q?d=20delete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This adds a refactored AIP-165. I really like some of the precedents set in #2; for example, mirroring the structure there required me to add a full section on errors that I consider to be quite useful. Similarly, separating the behavior from the IDL still works well. There is an open question here about long-running operations, which I do not know how to properly handle in OAS and need to discuss with @mkistler. --- aip/general/0165/aip.md.j2 | 131 +++++++++++++++++++++++++++++++++++ aip/general/0165/aip.yaml | 7 ++ aip/general/0165/purge.proto | 62 +++++++++++++++++ 3 files changed, 200 insertions(+) create mode 100644 aip/general/0165/aip.md.j2 create mode 100644 aip/general/0165/aip.yaml create mode 100644 aip/general/0165/purge.proto diff --git a/aip/general/0165/aip.md.j2 b/aip/general/0165/aip.md.j2 new file mode 100644 index 00000000..77b66f0e --- /dev/null +++ b/aip/general/0165/aip.md.j2 @@ -0,0 +1,131 @@ +# Criteria-based delete + +Occasionally, an API may need to provide a mechanism to delete a large number +of resources based on some set of filter parameters, rather than requiring the +individual resource name of the resources to be deleted. + +This is a rare case, reserved for situations where users need to delete +thousands or more resources at once, in which case the normal Batch Delete +pattern (AIP-235) becomes unwieldy and inconvenient. + +## Guidance + +**Important:** Most APIs **should** use only Delete (AIP-135) or Batch Delete +(AIP-235) for deleting resources, and **should not** implement deleting based +on criteria. This is because deleting is generally irreversible and this type +of operation makes it easy for a user to accidentally lose significant amounts +of data. + +An API **may** implement a Purge operation to permit deleting a large number of +resources based on a filter string; however, this **should** only be done if +the Batch Delete (AIP-235) pattern is insufficient to accomplish the desired +goal: + +### Requests + +Purge operations **must** be made by sending a `POST` request to the collection +with a `:purge` suffix: + +```http +POST /v1/publishers/{publisher}/books:purge HTTP/2 +Host: library.googleapis.com +Accept: application/json + +{ + "filter": "author=vhugo", + "force": false +} +``` + +- The HTTP verb **must** be `POST`. + - The POST body **should** include a `filter` field, for specifying arbitrary + purge criteria. The `filter` field **should** be required, and **must** + follow the same semantics as in List operations (see AIP-160). + - The POST body **must** include a `force` field. + - If the `force` key is not provided or is set to `false`, the service + **must** return a count of the resources that would be deleted as well as + a sample of those resources, without actually performing the deletion. + - If the `force` key is set to `true`, the purge is performed. +- If the service wishes to support deletion across multiple parents, it + **should** accept the `-` character, as described in AIP-159. + +### Responses + +Purge method responses **should** conform to the following interface: + +```typescript +interface PurgeBooksResponse { + // The number of books that this request deleted (or, if `force` is false, + // the number of books that will be deleted). + purge_count: number; + + // A sample of the resource names of books that will be deleted. + // Only populated if `force` is set to false. + purge_sample: ?string[]; +} +``` + +- The `purge_count` field **should** be included, and provide the number of + resources that were deleted (or would be deleted). This count **may** be an + estimate similar to `total_size` in AIP-158 (but the service **should** + document this if so). +- The `purge_sample` field **should** be included: If `force` is `false`, it + **should** provide a sample of resource names that will be deleted. If + `force` is true, this field **should not** be populated. + - The sample **should** be a sufficient size to catch clearly obvious + mistakes: A good rule of thumb is 100. The API **should** document the + size, and **should** document that it is a maximum (it is possible to send + fewer). + - The sample **may** be random or **may** be deterministic (such as the first + matched resource names). The API **should** document which approach is + used. + +**Note:** Even if `purge_count` and `purge_sample` are not included, the +`force` field **must** still be included in the request. + +### Errors + +If the user does not have permission to purge resources on the collection, +regardless of whether or not the collection exists, the service **must** reply +with an HTTP 403 error. A single permission **should** be checked for the whole +collection, and permission **must** be checked prior to checking if the +collection exists. + +If a user with proper permissions requests to purge from a collection that does +not exist, the service **must** reply with an HTTP 404 error. + +If a user with proper permissions provides an invalid filter, the service +**must** reply with an HTTP 400 error. If the user does not provide a filter at +all, the service **should** reply with an HTTP 400 error. + +If a user with proper permissions provides a filter that matches zero +resources, the request **must** succeed and indicate this. + +## Interface Definitions + +{% tab proto -%} + +Purge operations are specified using the following pattern: + +{% sample 'purge.proto', 'rpc PurgeBooks' %} + +- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`. +- The `parent` field **should** be included in the URI. +- The response type **must** be a `google.longrunning.Operation` (see AIP-151). + +Purge operation requests are specified using the following pattern: + +{% sample 'purge.proto', 'message PurgeBooksRequest' %} + +- The `parent` **must** be marked as required. +- The `parent` field corresponds to the `parent` value in the URI, and + **should** include a reference annotation to the parent resource. +- The `filter` field **should** be marked as required. +- The `force` field **must not** be marked as required (which would imply that + the boolean needs to be `true` for the request to succeed). + +Purge operation responses are specified using the following pattern: + +{% sample 'purge.proto', 'message PurgeBooksResponse' %} + +{% endtabs %} diff --git a/aip/general/0165/aip.yaml b/aip/general/0165/aip.yaml new file mode 100644 index 00000000..4aa1b82b --- /dev/null +++ b/aip/general/0165/aip.yaml @@ -0,0 +1,7 @@ +--- +id: 165 +state: approved +created: 2019-12-18 +placement: + category: design-patterns + order: 100 diff --git a/aip/general/0165/purge.proto b/aip/general/0165/purge.proto new file mode 100644 index 00000000..3a7f12a7 --- /dev/null +++ b/aip/general/0165/purge.proto @@ -0,0 +1,62 @@ +// Copyright 2020 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +syntax = "proto3"; + +import "google/api/http.proto"; +import "google/api/field_behavior.proto"; +import "google/api/resource.proto"; +import "google/longrunning/operations.proto"; + +service Library { + // Remove a large number of books, based on the provided criteria. + rpc PurgeBooks(PurgeBooksRequest) returns (google.longrunning.Operation) { + option (google.api.http) = { + post: "/v1/{parent=publishers/*}/books:purge" + body: "*" + }; + option (google.longrunning.operation_info) = { + response_type: "PurgeBooksResponse" + metadata_type: "PurgeBooksMetadata" + }; + } +} + +message PurgeBooksRequest { + // The publisher to purge books from. + // To purge books across publishers, send "publishers/-". + string parent = 1 [ + (google.api.field_behavior) = REQUIRED, + (google.api.resource_reference) = { + child_type: "library-example.googleapis.com/Book" + }]; + + // A filter matching the books to be purged. + string filter = 2 [(google.api.field_behavior) = REQUIRED]; + + // Actually perform the purge. + // If `force` is set to false, the operation will return a sample of + // resource names that would be deleted. + bool force = 3; +} + +message PurgeBooksResponse { + // The number of books that this request deleted (or, if `force` is false, + // the number of books that will be deleted). + int32 purge_count = 1; + + // A sample of the resource names of books that will be deleted. + // Only populated if `force` is set to false. + repeated string purge_sample = 2; +} From e80063e3aa045864343ac80ce8186ad402980a96 Mon Sep 17 00:00:00 2001 From: Luke Sneeringer Date: Tue, 22 Sep 2020 09:30:12 -0700 Subject: [PATCH 2/2] Remove specious question mark. --- aip/general/0165/aip.md.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aip/general/0165/aip.md.j2 b/aip/general/0165/aip.md.j2 index 77b66f0e..462f0c45 100644 --- a/aip/general/0165/aip.md.j2 +++ b/aip/general/0165/aip.md.j2 @@ -61,7 +61,7 @@ interface PurgeBooksResponse { // A sample of the resource names of books that will be deleted. // Only populated if `force` is set to false. - purge_sample: ?string[]; + purge_sample: string[]; } ```