WebDAV subsequent requests fail for protected resources

[paulmillar]

The issue comes from HIFIS ticket 11614.

The problem comes from embedding a WebDAV URL in a URL, similar to <video src="URL">. The URL contains a macaroon that authorises the request, embedded as an ?authz= query parameter in the URL.

The WebDAV door processes the request and redirects the client to the pool node with a URL like:

https://dcache-doma16.desy.de:21840/Helmholtz/HIFIS/BIIGLE/SO242_2_163-1_LowerHD.mp4
    ?dcache-http-uuid=f6e301ef-aaa7-4624-bde5-00a6da7fabfc
    &dcache-http-ref=https%3A%2F%2Fhifis-storage-ht.desy.de%3A2880

(This URL is a single line, but is shown split over three lines here to make it more readable.)

When the user skips to a different part of the video, the browser makes an additional GET request; however, this time targeting the pool node using the redirection URL from the door. In other words, the client is caching the redirection response from the WebDAV door.

The initial GET request targeting the pool "used up" the dcache-http-uuid token. Therefore, the pool will not accept the GET request with this token. Instead, the pool will redirect the client back to the WebDAV door. The constructed URL does not include the macaroon. As the resource is protected, the WebDAV door fails this request and the video will fail.

Possible solutions:

• Redirection from WebDAV door to pool is updated to include the macaroon (or possibly any bearer token). The pool includes this bearer token in any subsequent redirection back to the WebDAV door.
  • Note, this only works if initial webdav door request was authorised used ?authz= compatible token.
• The pool acquires a macaroon that authorises reading the file, perhaps targeting specifically that file. The door includes this macaroon in the redirection back to the WebDAV door.
  • Question: does the pool only do this for "a while" -- avoid clients bypassing AuthZ by sharing the initial redirection URL.
  • Should we bind this behaviour to a specific client IP address?
• When faced with a subsequent GET request, the pool acts like a WebDAV door (either by itself of in collaboration with a WebDAV door) and establishes which pool should handle the request. It then responds to the GET request by redirecting the client to that door.
  • As an optimisation, the pool could skip the redirection and serve the request directly if client-targeted pool and data-serving pool are the same.
  • Question: how does the pool/door authorise such requests?
• The client (the web-browser) and/or the webdav door is updated so the client does not cache the initial redirection response (from door to pool). Subsequent GET requests (e.g., triggered by seeking a specific timestamp in the video) would target the WebDAV door and include the macaroon.
  • Would require work to verify/update the redirection from webdav door to pool, so it must not be cached, including any non-standard behaviour of browsers.
  • May require opening bugs in browser software, which would require the cooperation of web browser maintenance teams.

[DmitryLitvintsev]

What if instead of "using up" dcache-http-uuid pool caches those w/ some lifetime? I doubt it makes this so much insecure compared to what we have now.

[paulmillar]

Yup, @DmitryLitvintsev, that's another possible solution. That said, I'm not sure what all the implications might be.

A pool might want to "evict" a client; for example, if the pool has become overloaded or if the file's data has been deleted (migrated away). However, these are circumstances where the pool could decide this for itself.

In addition, it might be possible to "locked down" subsequent requests; for example, by limiting the reuse to requests comes from the same IP address, to where the previous transfer has finished (either completed or was interrupted), and to when the new request came soon after the previous request finished.