[GRDM-55309] all-projectsモードの追加

.github/workflows/build.yml

@@ -43,5 +43,6 @@ jobs:
           RDM_NODE_ID: ${{ secrets.RDM_NODE_ID }}
           RDM_TOKEN: ${{ secrets.RDM_TOKEN }}
           RDM_STORAGE: ${{ secrets.RDM_STORAGE || 'osfstorage' }}
+          SKIP_TEST_DOCKER: ${{ github.event_name == 'pull_request' && '1' || '' }}
         run: |
           pytest --cov

README.md

@@ -6,10 +6,9 @@ RDMFS is a FUSE filesystem that allows you to mount your GakuNin RDM project as
 
 RDMFS requires libfuse-dev to be installed on your system.
 
-## Run RDMFS on Docker
+## Mount a single project
 
-You can easily try out RDMFS by using a Docker container with libfuse-dev installed.
-You can try RDMFS by executing the following commands.
+You can easily try out RDMFS by using a Docker container with libfuse-dev installed. The following example mounts a single project by supplying its node ID via environment variables.
 
 ```
 $ docker build -t rcosdp/cs-rdmfs .
@@ -28,6 +27,25 @@ googledrive osfstorage
 file1.txt file2.txt
 ```
 
+## Mount all accessible projects
+
+Omit `RDM_NODE_ID` when you launch the Docker container (or pass `--all-projects` to the CLI) to expose every project you can access under the mount root.
+This layout adds `.children` / `.linked` directories that contain symbolic links to related projects, while single-project mounts keep the previous structure and hide them.
+
+```
+$ docker run -it -v $(pwd)/mnt:/mnt -e RDM_TOKEN=YOUR_PERSONAL_TOKEN -e RDM_API_URL=http://192.168.168.167:8000/v2/ -e MOUNT_PATH=/mnt/all --name rdmfs --privileged rcosdp/cs-rdmfs
+$ docker exec -it rdmfs bash
+# cd /mnt/all
+# ls
+abcde fghij klmno
+# ls abcde
+googledrive osfstorage
+# ls abcde/.linked
+klmno -> ../../klmno/
+```
+
+> Links to projects where you are not a contributor (public projects) are not supported; GakuNin RDM deployments do not expose publicly accessible nodes.
+
 # Run Tests on Docker
 
 You can run the tests on a Docker container by executing the following commands.

bin/start.sh

@@ -2,16 +2,47 @@
 
 set -ue
 
-export RDM_MOUNT_PATH=${MOUNT_PATH:-/mnt}
-mkdir -p ${RDM_MOUNT_PATH}
+RDM_MOUNT_PATH=${MOUNT_PATH:-/mnt}
+mkdir -p "${RDM_MOUNT_PATH}"
 
-export RDM_MOUNT_FILE_MODE=${MOUNT_FILE_MODE:-0666}
-export RDM_MOUNT_DIR_MODE=${MOUNT_DIR_MODE:-0777}
+RDM_MOUNT_FILE_MODE=${MOUNT_FILE_MODE:-0666}
+RDM_MOUNT_DIR_MODE=${MOUNT_DIR_MODE:-0777}
+RDM_API_URL=${RDM_API_URL:-https://api.rdm.nii.ac.jp/v2/}
+RDM_NODE_ID=${RDM_NODE_ID:-}
 
-export DEBUG=--debug
+DEBUG=--debug
 export OSF_TOKEN=${RDM_TOKEN}
-python3 -m rdmfs.__main__ \
-    --file-mode ${RDM_MOUNT_FILE_MODE} \
-    --dir-mode ${RDM_MOUNT_DIR_MODE} \
-    --allow-other -p ${RDM_NODE_ID} \
-    --base-url ${RDM_API_URL} ${DEBUG} ${RDM_MOUNT_PATH} $@
+
+
+ALL_PROJECTS=false
+EXTRA_ARGS=()
+for arg in "$@"; do
+    if [ "$arg" = "--all-projects" ]; then
+        ALL_PROJECTS=true
+    else
+        EXTRA_ARGS+=("$arg")
+    fi
+done
+
+if [ -z "${RDM_NODE_ID}" ]; then
+    ALL_PROJECTS=true
+fi
+
+CMD=(python3 -m rdmfs
+    --file-mode "${RDM_MOUNT_FILE_MODE}"
+    --dir-mode "${RDM_MOUNT_DIR_MODE}"
+    --allow-other
+    --base-url "${RDM_API_URL}"
+    ${DEBUG}
+)
+
+if [ "${ALL_PROJECTS}" = true ]; then
+    CMD+=(--all-projects)
+else
+    CMD+=(--project "${RDM_NODE_ID}")
+fi
+
+CMD+=("${RDM_MOUNT_PATH}")
+CMD+=("${EXTRA_ARGS[@]}")
+
+exec "${CMD[@]}"

docs/usage.md

@@ -0,0 +1,113 @@
+# CS-rdmfs Usage Guide
+
+## Overview
+
+CS-rdmfs mounts projects from the Open Science Framework (OSF) as a FUSE filesystem. You can mount either a single project by ID or expose every accessible project beneath the mount root. Each project directory provides additional virtual entries that surface API metadata and related projects.
+
+## Running via `bin/start.sh`
+
+The recommended entrypoint (used by the Docker image) is `bin/start.sh`. It
+reads environment variables, prepares the mount directory, and invokes
+`python -m rdmfs` with the appropriate options.
+
+Required environment variables:
+
+- `RDM_NODE_ID` – project GUID to mount. (Omit or leave empty to mount all projects.)
+- `RDM_TOKEN` – personal access token (forwarded to `OSF_TOKEN`).
+- `RDM_API_URL` – API base URL (defaults to `https://api.rdm.nii.ac.jp/v2/`).
+- `MOUNT_PATH` – mountpoint inside the container (default `/mnt`).
+
+Optional overrides:
+
+- `MOUNT_FILE_MODE` / `MOUNT_DIR_MODE` – forwarded to `--file-mode` and
+  `--dir-mode`.
+- Any extra arguments appended to `start.sh` are passed through to the Python
+  module, enabling `--debug` or additional FUSE options.
+
+If `RDM_NODE_ID` is unset (and you do not explicitly pass `--all-projects`),
+`start.sh` automatically enables the all-projects mode.
+
+### Docker Examples
+
+Mount a single project inside `/mnt/osf`:
+
+```bash
+docker run --rm -it --privileged \
+  -v "$(pwd)/mnt":/mnt \
+  -e RDM_NODE_ID=abc123 \
+  -e RDM_TOKEN=$RDM_TOKEN \
+  -e RDM_API_URL=https://api.rdm.nii.ac.jp/v2/ \
+  -e MOUNT_PATH=/mnt/osf \
+  rcosdp/cs-rdmfs
+```
+
+Mount every accessible project by omitting `RDM_NODE_ID`:
+
+```bash
+docker run --rm -it --privileged \
+  -v "$(pwd)/mnt":/mnt \
+  -e RDM_TOKEN=$RDM_TOKEN \
+  -e RDM_API_URL=https://api.rdm.nii.ac.jp/v2/ \
+  rcosdp/cs-rdmfs
+```
+
+## Direct CLI Usage
+
+When running outside Docker (or bypassing `start.sh`), set `OSF_TOKEN` manually:
+
+```bash
+export OSF_TOKEN=your_token
+python -m rdmfs [mountpoint] \
+  (--project <project-id> | --all-projects) \
+  [--base-url https://api.rdm.nii.ac.jp/v2/] \
+  [--file-mode 0644] [--dir-mode 0755] \
+  [--allow-other] [--debug] [--debug-fuse]
+```
+
+`--project` and `--all-projects` are mutually exclusive. Remaining options match
+those used by `start.sh`.
+
+## Virtual Directory Layout
+
+Each mounted project contains virtual entries ahead of storage providers:
+
+```
+/project-id/
+  .attributes.json   # live view of OSF node attributes (nodes_read)
+  .children/         # child projects returned by nodes_children_list
+  .linked/           # linked projects from collections_linked_nodes_list
+  osfstorage/        # standard storage providers follow the virtual entries
+  ...
+```
+
+- `.attributes.json` is read-only. Every read triggers `GET /v2/nodes/{id}/` and
+  returns `data.attributes` formatted as indented JSON.
+- `.children/` lists child node IDs; each entry behaves like a project directory
+  with the same virtual structure.
+- `.linked/` lists linked nodes; likewise they expose `.attributes.json`,
+  `.children`, `.linked`, and storages.
+
+## API Endpoints
+
+The filesystem relies on:
+
+- `GET /v2/nodes/{id}/` (nodes_read) for node attributes.
+- `GET /v2/nodes/{id}/children/` (nodes_children_list) for child projects.
+- `GET /v2/nodes/{id}/linked_nodes/` (collections_linked_nodes_list) for linked
+  projects.
+
+All collection requests apply `page[size]=100` to reduce page churn and follow
+`links.next` until completion.
+
+## Testing
+
+Run the repository tests inside Docker. Supplying `RDM_NODE_ID` and `RDM_TOKEN`
+allows the Docker-specific test to execute; omitting them results in a single
+expected failure while the rest succeed.
+
+```bash
+docker run --rm -v "$(pwd)":/code -w /code \
+  -e RDM_NODE_ID=your_project_id \
+  -e RDM_TOKEN=$RDM_TOKEN \
+  rcosdp/cs-rdmfs py.test --cov
+```

rdmfs/__main__.py

@@ -1,6 +1,7 @@
 from argparse import ArgumentParser
 import asyncio
 import logging
+import os
 import grp
 import pwd
 import re
@@ -45,8 +46,11 @@ def parse_args():
                               'OSF_PASSWORD environment variable'))
     parser.add_argument('--base-url', default=None,
                         help='OSF API URL (Default is https://api.osf.io/v2/)')
-    parser.add_argument('-p', '--project', default=None,
-                        help='OSF project ID')
+    project_group = parser.add_mutually_exclusive_group()
+    project_group.add_argument('-p', '--project', default=None,
+                               help='OSF project ID')
+    project_group.add_argument('--all-projects', action='store_true', default=False,
+                               help='Mount all accessible projects under the root directory')
     parser.add_argument('--file-mode', default='0644',
                         help='Mode of files. default: 0644')
     parser.add_argument('--dir-mode', default='0755',
@@ -85,7 +89,17 @@ def main():
     options = parse_args()
     init_logging(options.debug)
 
+    placeholder_project = None
+    if options.all_projects and options.project is None:
+        placeholder_project = '__all_projects__'
+        options.project = placeholder_project
+
     osf = cli._setup_osf(options)
+    resolved_project = None if options.all_projects else options.project
+
+    if placeholder_project is not None:
+        options.project = None
+
     file_mode = parse_mode(options.file_mode)
     dir_mode = parse_mode(options.dir_mode)
     uid = parse_uid(options.owner)
@@ -94,7 +108,10 @@ def main():
     if options.writable_whitelist is not None:
         with open(options.writable_whitelist, 'r') as f:
             writable_whitelist = whitelist.Whitelist(f)
-    rdmfs = fs.RDMFileSystem(osf, options.project,
+    if not options.all_projects and resolved_project is None:
+        raise SystemExit('either --project or --all-projects must be specified')
+    rdmfs = fs.RDMFileSystem(osf, resolved_project,
+                             list_all_projects=options.all_projects,
                              file_mode=file_mode, dir_mode=dir_mode,
                              uid=uid, gid=gid,
                              writable_whitelist=writable_whitelist)

rdmfs/fs.py

@@ -18,9 +18,10 @@
 class RDMFileSystem(pyfuse3.Operations):
     def __init__(self, osf, project, dir_mode=0o755, file_mode=0o644,
                  uid=None, gid=None,
-                 writable_whitelist: Optional[Whitelist]=None):
+                 writable_whitelist: Optional[Whitelist]=None,
+                 list_all_projects: bool=False):
         super(RDMFileSystem, self).__init__()
-        self.inodes = Inodes(osf, project)
+        self.inodes = Inodes(osf, project, list_all_projects=list_all_projects)
         self.file_handlers = FileHandlers()
         self.dir_mode = dir_mode
         self.file_mode = file_mode
@@ -36,7 +37,10 @@ async def getattr(self, inode_num, ctx=None):
             if inode is None:
                 raise pyfuse3.FUSEError(errno.ENOENT)
             await inode.refresh(self.inodes)
-            if inode.has_children():
+            if inode.is_symlink:
+                entry.st_mode = (stat.S_IFLNK | 0o777)
+                entry.st_size = len(inode.target)
+            elif inode.has_children():
                 entry.st_mode = (stat.S_IFDIR | self.dir_mode)
                 entry.st_size = 0
             else:
@@ -49,15 +53,15 @@ async def getattr(self, inode_num, ctx=None):
             if self.writable_whitelist is not None and \
                 not self.writable_whitelist.includes(inode):
                 entry.st_mode = entry.st_mode & (~0o200)
-            stamp = 0
-            mstamp = stamp
+            ctime_stamp = 0
+            mtime_stamp = 0
             if inode.date_created is not None:
-                stamp = fromisoformat(inode.date_created)
+                ctime_stamp = fromisoformat(inode.date_created)
             if inode.date_modified is not None:
-                mstamp = fromisoformat(inode.date_modified)
-            entry.st_atime_ns = stamp
-            entry.st_ctime_ns = stamp
-            entry.st_mtime_ns = mstamp
+                mtime_stamp = fromisoformat(inode.date_modified)
+            entry.st_atime_ns = mtime_stamp
+            entry.st_ctime_ns = ctime_stamp
+            entry.st_mtime_ns = mtime_stamp
             entry.st_gid = self.gid
             entry.st_uid = self.uid
             entry.st_ino = inode.id
@@ -101,6 +105,21 @@ async def lookup(self, parent_inode_num, bname, ctx=None):
         except BaseException as e:
             reraise_fuse_error(e)
 
+    async def readlink(self, inode_num, ctx):
+        log.info('readlink: inode={}'.format(inode_num))
+        try:
+            inode = await self.inodes.get(inode_num)
+            if inode is None:
+                raise pyfuse3.FUSEError(errno.ENOENT)
+            if not inode.is_symlink:
+                raise pyfuse3.FUSEError(errno.EINVAL)
+            target = inode.target
+            return os.fsencode(target)
+        except pyfuse3.FUSEError as e:
+            raise e
+        except BaseException as e:
+            reraise_fuse_error(e)
+
     async def opendir(self, inode_num, ctx):
         log.info('opendir: inode={inode}'.format(inode=inode_num))
         try:
@@ -147,6 +166,8 @@ async def open(self, inode_num, flags, ctx):
             inode = await self.inodes.get(inode_num)
             if inode is None:
                 raise pyfuse3.FUSEError(errno.ENOENT)
+            if flags_can_write(flags) and getattr(inode, 'readonly', False):
+                raise pyfuse3.FUSEError(errno.EACCES)
             if flags_can_write(flags) and \
                 self.writable_whitelist is not None and \
                 not self.writable_whitelist.includes(inode):