Update Docs

2020-10-06 13:41:35 +02:00
parent 93138c81ed
commit 24e84f9bc2
12 changed files with 284 additions and 356 deletions
@@ -0,0 +1,45 @@
+---
+title: "Reva"
+date: 2018-05-02T00:00:00+00:00
+weight: 10
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: _index.md
+---
+
+This service provides an ocis extension that wraps [reva](https://github.com/cs3org/reva/) and adds an opinionated configuration to it.
+
+It uses the port range 9140-9179 to preconfigure several services.
+
+| port | service |
+------+---------+
+| 9109 | health? |
+| 9140 | frontend        |
+| 9141 | frontend debug        |
+| 9142 | gateway        |
+| 9143 | gateway debug        |
+| 9144 | users        |
+| 9145 | users debug        |
+| 9146 | authbasic        |
+| 9147 | authbasic debug        |
+| 9148 | authbearer        |
+| 9149 | authbearer debug        |
+| 9150 | sharing        |
+| 9151 | sharing debug        |
+| 9152 | storage root        |
+| 9153 | storage root debug        |
+| 9154 | storage home        |
+| 9155 | storage home debug        |
+| 9156 | storage home data        |
+| 9157 | storage home data debug        |
+| 9158 | storage eos        |
+| 9159 | storage eos debug        |
+| 9160 | storage eos data        |
+| 9161 | storage eos data debug        |
+| 9162 | storage oc        |
+| 9163 | storage oc debug        |
+| 9164 | storage oc data        |
+| 9165 | storage oc data debug        |
+| 9166-9177 | reserved for s3, wnd, custom + data providers |
+| 9178 | storage public link        |
+| 9179 | storage public link data        |
@@ -0,0 +1,26 @@
+---
+title: "Building"
+date: 2018-05-02T00:00:00+00:00
+weight: 30
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: building.md
+---
+
+As this project is built with Go, so you need to install that first. The installation of Go is out of the scope of this document, please follow the official documentation for [Go](https://golang.org/doc/install).To build this project you have to install Go >= v1.13. After the installation of the required tools you need to get the sources:
+
+{{< highlight txt >}}
+git clone https://github.com/owncloud/ocis.git
+cd ocis/storage
+{{< / highlight >}}
+
+All required tool besides Go itself and make are bundled or getting automatically installed within the `GOPATH`. All commands to build this project are part of our `Makefile`.
+
+
+### Backend
+
+{{< highlight txt >}}
+make build
+{{< / highlight >}}
+
+The above command should produce the binary within the `bin/` folder now, give it a try with `./bin/ocis-reva -h` to see all available options.
@@ -0,0 +1,656 @@
+---
+title: "Configuration"
+date: "2020-09-21T13:14:49+0200"
+weight: 20
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: configuration.md
+---
+
+{{< toc >}}
+
+## Configuration
+
+oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
+
+Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
+
+### Configuration using config files
+
+Out of the box extensions will attempt to read configuration details from:
+
+```console
+/etc/ocis
+$HOME/.ocis
+./config
+```
+
+For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. *i.e: ocis-proxy reads `proxy.json | yaml | toml ...`*.
+
+So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/ocis.yml`, `${HOME}/.ocis/ocis.yml` or `$(pwd)/config/ocis.yml`.
+
+### Envrionment variables
+
+If you prefer to configure the service with environment variables you can see the available variables below.
+
+### Commandline flags
+
+If you prefer to configure the service with commandline flags you can see the available variables below. Command line flags are only working when calling the subcommand directly.
+
+## Root Command
+
+Example service for Reva/oCIS
+
+Usage: `ocis-reva [global options] command [command options] [arguments...]`
+
+--config-file | $REVA_CONFIG_FILE
+: Path to config file.
+
+--log-level | $REVA_LOG_LEVEL
+: Set logging level. Default: `info`.
+
+--log-pretty | $REVA_LOG_PRETTY
+: Enable pretty logging.
+
+--log-color | $REVA_LOG_COLOR
+: Enable colored logging.
+
+## Sub Commands
+
+### ocis-reva auth-basic
+
+Start reva authprovider for basic auth
+
+Usage: `ocis-reva auth-basic [command options] [arguments...]`
+
+--debug-addr | $REVA_AUTH_BASIC_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9147`.
+
+--auth-driver | $REVA_AUTH_DRIVER
+: auth driver: 'demo', 'json' or 'ldap'. Default: `ldap`.
+
+--auth-json | $REVA_AUTH_JSON
+: Path to users.json file.
+
+--network | $REVA_AUTH_BASIC_NETWORK
+: Network to use for the reva auth-basic service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_AUTH_BASIC_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_AUTH_BASIC_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9146`.
+
+--url | $REVA_AUTH_BASIC_URL
+: URL to use for the reva service. Default: `localhost:9146`.
+
+### ocis-reva frontend
+
+Start reva frontend service
+
+Usage: `ocis-reva frontend [command options] [arguments...]`
+
+--debug-addr | $REVA_FRONTEND_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9141`.
+
+--transfer-secret | $REVA_TRANSFER_SECRET
+: Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.
+
+--webdav-namespace | $WEBDAV_NAMESPACE
+: Namespace prefix for the /webdav endpoint. Default: `/home/`.
+
+--dav-files-namespace | $DAV_FILES_NAMESPACE
+: Namespace prefix for the webdav /dav/files endpoint. Default: `/oc/`.
+
+--network | $REVA_FRONTEND_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_FRONTEND_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
+
+--addr | $REVA_FRONTEND_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9140`.
+
+--url | $REVA_FRONTEND_URL
+: URL to use for the reva service. Default: `https://localhost:9200`.
+
+--datagateway-prefix | $REVA_FRONTEND_DATAGATEWAY_PREFIX
+: datagateway prefix. Default: `data`.
+
+--ocdav-prefix | $REVA_FRONTEND_OCDAV_PREFIX
+: owncloud webdav endpoint prefix.
+
+--ocs-prefix | $REVA_FRONTEND_OCS_PREFIX
+: open collaboration services endpoint prefix. Default: `ocs`.
+
+--gateway-url | $REVA_GATEWAY_URL
+: URL to use for the reva gateway service. Default: `localhost:9142`.
+
+--upload-disable-tus | $REVA_FRONTEND_UPLOAD_DISABLE_TUS
+: Disables TUS upload mechanism. Default: `false`.
+
+--upload-http-method-override | $REVA_FRONTEND_UPLOAD_HTTP_METHOD_OVERRIDE
+: Specify an HTTP method (ex: POST) that clients should to use when uploading instead of PATCH.
+
+### ocis-reva gateway
+
+Start reva gateway
+
+Usage: `ocis-reva gateway [command options] [arguments...]`
+
+--debug-addr | $REVA_GATEWAY_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9143`.
+
+--transfer-secret | $REVA_TRANSFER_SECRET
+: Transfer secret for datagateway. Default: `replace-me-with-a-transfer-secret`.
+
+--network | $REVA_GATEWAY_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_GATEWAY_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_GATEWAY_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9142`.
+
+--url | $REVA_GATEWAY_URL
+: URL to use for the reva service. Default: `localhost:9142`.
+
+--commit-share-to-storage-grant | $REVA_GATEWAY_COMMIT_SHARE_TO_STORAGE_GRANT
+: Commit shares to the share manager. Default: `true`.
+
+--commit-share-to-storage-ref | $REVA_GATEWAY_COMMIT_SHARE_TO_STORAGE_REF
+: Commit shares to the storage. Default: `true`.
+
+--share-folder | $REVA_GATEWAY_SHARE_FOLDER
+: mount shares in this folder of the home storage provider. Default: `Shares`.
+
+--disable-home-creation-on-login | $REVA_GATEWAY_DISABLE_HOME_CREATION_ON_LOGIN
+: Disable creation of home folder on login.
+
+--storage-registry-driver | $REVA_STORAGE_REGISTRY_DRIVER
+: driver of the storage registry. Default: `static`.
+
+--storage-home-provider | $REVA_STORAGE_HOME_PROVIDER
+: mount point of the storage provider for user homes in the global namespace. Default: `/home`.
+
+--frontend-url | $REVA_FRONTEND_URL
+: URL to use for the reva service. Default: `https://localhost:9200`.
+
+--datagateway-url | $REVA_DATAGATEWAY_URL
+: URL to use for the reva datagateway. Default: `https://localhost:9200/data`.
+
+--users-url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+--auth-basic-url | $REVA_AUTH_BASIC_URL
+: URL to use for the reva service. Default: `localhost:9146`.
+
+--auth-bearer-url | $REVA_AUTH_BEARER_URL
+: URL to use for the reva service. Default: `localhost:9148`.
+
+--sharing-url | $REVA_SHARING_URL
+: URL to use for the reva service. Default: `localhost:9150`.
+
+--storage-root-url | $REVA_STORAGE_ROOT_URL
+: URL to use for the reva service. Default: `localhost:9152`.
+
+--storage-root-mount-path | $REVA_STORAGE_ROOT_MOUNT_PATH
+: mount path. Default: `/`.
+
+--storage-root-mount-id | $REVA_STORAGE_ROOT_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009152`.
+
+--storage-home-url | $REVA_STORAGE_HOME_URL
+: URL to use for the reva service. Default: `localhost:9154`.
+
+--storage-home-mount-path | $REVA_STORAGE_HOME_MOUNT_PATH
+: mount path. Default: `/home`.
+
+--storage-home-mount-id | $REVA_STORAGE_HOME_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009154`.
+
+--storage-eos-url | $REVA_STORAGE_EOS_URL
+: URL to use for the reva service. Default: `localhost:9158`.
+
+--storage-eos-mount-path | $REVA_STORAGE_EOS_MOUNT_PATH
+: mount path. Default: `/eos`.
+
+--storage-eos-mount-id | $REVA_STORAGE_EOS_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009158`.
+
+--storage-oc-url | $REVA_STORAGE_OC_URL
+: URL to use for the reva service. Default: `localhost:9162`.
+
+--storage-oc-mount-path | $REVA_STORAGE_OC_MOUNT_PATH
+: mount path. Default: `/oc`.
+
+--storage-oc-mount-id | $REVA_STORAGE_OC_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
+
+--public-link-url | $REVA_STORAGE_PUBLIC_LINK_URL
+: URL to use for the public links service. Default: `localhost:9178`.
+
+--storage-public-link-mount-path | $REVA_STORAGE_PUBLIC_LINK_MOUNT_PATH
+: mount path. Default: `/public/`.
+
+### ocis-reva sharing
+
+Start reva sharing service
+
+Usage: `ocis-reva sharing [command options] [arguments...]`
+
+--debug-addr | $REVA_SHARING_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9151`.
+
+--network | $REVA_SHARING_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_SHARING_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_SHARING_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9150`.
+
+--url | $REVA_SHARING_URL
+: URL to use for the reva service. Default: `localhost:9150`.
+
+--user-driver | $REVA_SHARING_USER_DRIVER
+: driver to use for the UserShareProvider. Default: `json`.
+
+--user-json-file | $REVA_SHARING_USER_JSON_FILE
+: file used to persist shares for the UserShareProvider. Default: `/var/tmp/reva/shares.json`.
+
+--public-driver | $REVA_SHARING_PUBLIC_DRIVER
+: driver to use for the PublicShareProvider. Default: `json`.
+
+### ocis-reva storage-home-data
+
+Start reva storage-home-data service
+
+Usage: `ocis-reva storage-home-data [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_HOME_DATA_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9157`.
+
+--network | $REVA_STORAGE_HOME_DATA_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_HOME_DATA_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
+
+--addr | $REVA_STORAGE_HOME_DATA_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9156`.
+
+--url | $REVA_STORAGE_HOME_DATA_URL
+: URL to use for the reva service. Default: `localhost:9156`.
+
+--driver | $REVA_STORAGE_HOME_DATA_DRIVER
+: storage driver for home data mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
+
+--prefix | $REVA_STORAGE_HOME_DATA_PREFIX
+: prefix for the http endpoint, without leading slash. Default: `data`.
+
+--temp-folder | $REVA_STORAGE_HOME_DATA_TEMP_FOLDER
+: temp folder. Default: `/var/tmp/`.
+
+--enable-home | $REVA_STORAGE_HOME_ENABLE_HOME
+: enable the creation of home directories. Default: `true`.
+
+--gateway-url | $REVA_GATEWAY_URL
+: URL to use for the reva gateway service. Default: `localhost:9142`.
+
+--users-url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+### ocis-reva health
+
+Check health status
+
+Usage: `ocis-reva health [command options] [arguments...]`
+
+--debug-addr | $REVA_DEBUG_ADDR
+: Address to debug endpoint. Default: `0.0.0.0:9109`.
+
+### ocis-reva storage-home
+
+Start reva storage-home service
+
+Usage: `ocis-reva storage-home [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_HOME_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9155`.
+
+--network | $REVA_STORAGE_HOME_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_HOME_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_STORAGE_HOME_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9154`.
+
+--url | $REVA_STORAGE_HOME_URL
+: URL to use for the reva service. Default: `localhost:9154`.
+
+--driver | $REVA_STORAGE_HOME_DRIVER
+: storage driver for home mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
+
+--mount-path | $REVA_STORAGE_HOME_MOUNT_PATH
+: mount path. Default: `/home`.
+
+--mount-id | $REVA_STORAGE_HOME_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
+
+--expose-data-server | $REVA_STORAGE_HOME_EXPOSE_DATA_SERVER
+: exposes a dedicated data server. Default: `false`.
+
+--data-server-url | $REVA_STORAGE_HOME_DATA_SERVER_URL
+: data server url. Default: `http://localhost:9156/data`.
+
+--enable-home | $REVA_STORAGE_HOME_ENABLE_HOME
+: enable the creation of home directories. Default: `true`.
+
+--users-url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+### ocis-reva storage-eos-data
+
+Start reva storage-eos-data service
+
+Usage: `ocis-reva storage-eos-data [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_OC_DATA_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9161`.
+
+--network | $REVA_STORAGE_EOS_DATA_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_EOS_DATA_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
+
+--addr | $REVA_STORAGE_EOS_DATA_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9160`.
+
+--url | $REVA_STORAGE_EOS_DATA_URL
+: URL to use for the reva service. Default: `localhost:9160`.
+
+--driver | $REVA_STORAGE_EOS_DATA_DRIVER
+: storage driver for eos data mount: eg. local, eos, owncloud, ocis or s3. Default: `eos`.
+
+--prefix | $REVA_STORAGE_EOS_DATA_PREFIX
+: prefix for the http endpoint, without leading slash. Default: `data`.
+
+--temp-folder | $REVA_STORAGE_EOS_DATA_TEMP_FOLDER
+: temp folder. Default: `/var/tmp/`.
+
+--gateway-url | $REVA_GATEWAY_URL
+: URL to use for the reva gateway service. Default: `localhost:9142`.
+
+--users-url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+### ocis-reva reva-storage-public-link
+
+Start reva storage-public-link service
+
+Usage: `ocis-reva reva-storage-public-link [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_PUBLIC_LINK_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9179`.
+
+--network | $REVA_STORAGE_PUBLIC_LINK_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_PUBLIC_LINK_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_STORAGE_PUBLIC_LINK_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9178`.
+
+--url | $REVA_STORAGE_PUBLIC_LINK_URL
+: Address to bind reva service. Default: `localhost:9178`.
+
+--mount-path | $REVA_STORAGE_PUBLIC_LINK_MOUNT_PATH
+: mount path. Default: `/public/`.
+
+--gateway-url | $REVA_GATEWAY_URL
+: URL to use for the reva gateway service. Default: `localhost:9142`.
+
+### ocis-reva users
+
+Start reva users service
+
+Usage: `ocis-reva users [command options] [arguments...]`
+
+--debug-addr | $REVA_SHARING_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9145`.
+
+--network | $REVA_USERS_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_USERS_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_USERS_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9144`.
+
+--url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+--driver | $REVA_USERS_DRIVER
+: user driver: 'demo', 'json', 'ldap', or 'rest'. Default: `ldap`.
+
+--json-config | $REVA_USERS_JSON
+: Path to users.json file.
+
+--rest-client-id | $REVA_REST_CLIENT_ID
+: User rest driver Client ID.
+
+--rest-client-secret | $REVA_REST_CLIENT_SECRET
+: User rest driver Client Secret.
+
+--rest-redis-address | $REVA_REST_REDIS_ADDRESS
+: Address for redis server. Default: `localhost:6379`.
+
+--rest-redis-username | $REVA_REST_REDIS_USERNAME
+: Username for redis server.
+
+--rest-redis-password | $REVA_REST_REDIS_PASSWORD
+: Password for redis server.
+
+--rest-id-provider | $REVA_REST_ID_PROVIDER
+: The OIDC Provider.
+
+--rest-api-base-url | $REVA_REST_API_BASE_URL
+: Base API Endpoint.
+
+--rest-oidc-token-endpoint | $REVA_REST_OIDC_TOKEN_ENDPOINT
+: Endpoint to generate token to access the API.
+
+--rest-target-api | $REVA_REST_TARGET_API
+: The target application.
+
+### ocis-reva auth-bearer
+
+Start reva authprovider for bearer auth
+
+Usage: `ocis-reva auth-bearer [command options] [arguments...]`
+
+--debug-addr | $REVA_AUTH_BEARER_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9149`.
+
+--oidc-issuer | $REVA_OIDC_ISSUER
+: OIDC issuer. Default: `https://localhost:9200`.
+
+--oidc-insecure | $REVA_OIDC_INSECURE
+: OIDC allow insecure communication. Default: `true`.
+
+--oidc-id-claim | $REVA_OIDC_ID_CLAIM
+: OIDC id claim. Default: `preferred_username`.
+
+--oidc-uid-claim | $REVA_OIDC_UID_CLAIM
+: OIDC uid claim.
+
+--oidc-gid-claim | $REVA_OIDC_GID_CLAIM
+: OIDC gid claim.
+
+--network | $REVA_AUTH_BEARER_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_AUTH_BEARER_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_AUTH_BEARER_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9148`.
+
+--url | $REVA_AUTH_BEARER_URL
+: URL to use for the reva service. Default: `localhost:9148`.
+
+### ocis-reva storage-eos
+
+Start reva storage-eos service
+
+Usage: `ocis-reva storage-eos [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_EOS_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9159`.
+
+--network | $REVA_STORAGE_EOS_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_EOS_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_STORAGE_EOS_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9158`.
+
+--url | $REVA_STORAGE_EOS_URL
+: URL to use for the reva service. Default: `localhost:9158`.
+
+--driver | $REVA_STORAGE_EOS_DRIVER
+: storage driver for eos mount: eg. local, eos, owncloud, ocis or s3. Default: `eos`.
+
+--mount-path | $REVA_STORAGE_EOS_MOUNT_PATH
+: mount path. Default: `/eos`.
+
+--mount-id | $REVA_STORAGE_EOS_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009158`.
+
+--expose-data-server | $REVA_STORAGE_EOS_EXPOSE_DATA_SERVER
+: exposes a dedicated data server. Default: `false`.
+
+--data-server-url | $REVA_STORAGE_EOS_DATA_SERVER_URL
+: data server url. Default: `http://localhost:9160/data`.
+
+### ocis-reva storage-oc
+
+Start reva storage-oc service
+
+Usage: `ocis-reva storage-oc [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_OC_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9163`.
+
+--network | $REVA_STORAGE_OC_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_OC_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_STORAGE_OC_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9162`.
+
+--url | $REVA_STORAGE_OC_URL
+: URL to use for the reva service. Default: `localhost:9162`.
+
+--driver | $REVA_STORAGE_OC_DRIVER
+: storage driver for oc mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
+
+--mount-path | $REVA_STORAGE_OC_MOUNT_PATH
+: mount path. Default: `/oc`.
+
+--mount-id | $REVA_STORAGE_OC_MOUNT_ID
+: mount id. Default: `1284d238-aa92-42ce-bdc4-0b0000009162`.
+
+--expose-data-server | $REVA_STORAGE_OC_EXPOSE_DATA_SERVER
+: exposes a dedicated data server. Default: `false`.
+
+--data-server-url | $REVA_STORAGE_OC_DATA_SERVER_URL
+: data server url. Default: `http://localhost:9164/data`.
+
+--users-url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+### ocis-reva storage-oc-data
+
+Start reva storage-oc-data service
+
+Usage: `ocis-reva storage-oc-data [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_OC_DATA_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9165`.
+
+--network | $REVA_STORAGE_OC_DATA_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_OC_DATA_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `http`.
+
+--addr | $REVA_STORAGE_OC_DATA_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9164`.
+
+--url | $REVA_STORAGE_OC_DATA_URL
+: URL to use for the reva service. Default: `localhost:9164`.
+
+--driver | $REVA_STORAGE_OC_DATA_DRIVER
+: storage driver for oc data mount: eg. local, eos, owncloud, ocis or s3. Default: `owncloud`.
+
+--prefix | $REVA_STORAGE_OC_DATA_PREFIX
+: prefix for the http endpoint, without leading slash. Default: `data`.
+
+--temp-folder | $REVA_STORAGE_OC_DATA_TEMP_FOLDER
+: temp folder. Default: `/var/tmp/`.
+
+--gateway-url | $REVA_GATEWAY_URL
+: URL to use for the reva gateway service. Default: `localhost:9142`.
+
+--users-url | $REVA_USERS_URL
+: URL to use for the reva service. Default: `localhost:9144`.
+
+### ocis-reva storage-root
+
+Start reva storage-root service
+
+Usage: `ocis-reva storage-root [command options] [arguments...]`
+
+--debug-addr | $REVA_STORAGE_ROOT_DEBUG_ADDR
+: Address to bind debug server. Default: `0.0.0.0:9153`.
+
+--network | $REVA_STORAGE_ROOT_NETWORK
+: Network to use for the reva service, can be 'tcp', 'udp' or 'unix'. Default: `tcp`.
+
+--protocol | $REVA_STORAGE_ROOT_PROTOCOL
+: protocol for reva service, can be 'http' or 'grpc'. Default: `grpc`.
+
+--addr | $REVA_STORAGE_ROOT_ADDR
+: Address to bind reva service. Default: `0.0.0.0:9152`.
+
+--url | $REVA_STORAGE_ROOT_URL
+: URL to use for the reva service. Default: `localhost:9152`.
+
+--driver | $REVA_STORAGE_ROOT_DRIVER
+: storage driver for root mount: eg. local, eos, owncloud, ocis or s3. Default: `local`.
+
+--mount-path | $REVA_STORAGE_ROOT_MOUNT_PATH
+: mount path. Default: `/`.
+
+--mount-id | $REVA_STORAGE_ROOT_MOUNT_ID
+: mount id. Default: `123e4567-e89b-12d3-a456-426655440001`.
+
+--expose-data-server | $REVA_STORAGE_ROOT_EXPOSE_DATA_SERVER
+: exposes a dedicated data server.
+
+--data-server-url | $REVA_STORAGE_ROOT_DATA_SERVER_URL
+: data server url.
+
@@ -0,0 +1,10 @@
+---
+title: "License"
+date: 2018-05-02T00:00:00+00:00
+weight: 40
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: license.md
+---
+
+This project is licensed under the [Apache 2.0](https://github.com/owncloud/ocis/ocis-revablob/master/LICENSE) license. For the license of the used libraries you have to check the respective sources.
@@ -0,0 +1,31 @@
+---
+title: "Releasing"
+date: 2020-05-22T00:00:00+00:00
+weight: 60
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: releasing.md
+---
+
+{{< toc >}}
+
+To release a new version of the storage submodule, you have to follow a few simple steps.
+
+## Preparation
+
+1. Before releasing, make sure that reva has been [updated to the desired version]({{< ref "updating.md" >}})
+
+## Release
+1. Check out master
+{{< highlight txt >}}
+git checkout master
+git pull origin master
+{{< / highlight >}}
+2. Create a new tag (preferably signed) and replace the version number accordingly. Prefix the tag with the submodule `storage/v`.
+{{< highlight txt >}}
+git tag -s storage/vx.x.x -m "release vx.x.x"
+git push origin storage/vx.x.x
+{{< / highlight >}}
+5. Wait for CI and check that the GitHub release was published.
+
+Congratulations, you just released the storage submodule!
@@ -0,0 +1,194 @@
+---
+title: "Storages"
+date: 2020-04-27T18:46:00+01:00
+weight: 37
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: storages.md
+---
+
+## Storage commands
+
+`storage` has multiple storage provider commands to preconfigure different default configurations for the reva *storage provider* service. While you could rerun `storage storage-oc` multiple times with different flags to get multiple instances we are giving the different commands the necessary default configuration to allow the `ocis` binary to simply start them and not deal with configuration.
+
+## Storage providers
+
+To manage the file tree ocis uses storage *storage providers* that are accessing the underlying storage using a *storage driver*. The driver can be used to change the implementation of a storage aspect to better reflect the actual underlying storage capabilities. As an example a move operation on a POSIX filesystem ([theoretically](https://danluu.com/deconstruct-files/)) is an atomic operation. When trying to implement a file tree on top of S3 there is no native move operation that can be used. A naive implementation might fall back on a COPY and DELETE. Some S3 implementations provide a COPY operation that uses an existing key as the source, so the file at least does not need to be reuploaded. In the worst case scenario, which is renaming a folder with hundreds of thousands of objects, a reupload for every file has to be made. Instead of hiding this complexity a better choice might be to disable renaming of files or at least folders on S3. There are however implementations of filesystems on top of S3 that store the tree metadata in dedicated objects or use a completely different persistence mechanism like a distributed key value store to implement the file tree aspect of a storage.
+
+
+{{< hint info >}}
+While the *storage provider* is responsible for managing the tree, file up and download is delegated to a dedicated *data provider*. See below.
+{{< /hint >}}
+
+## Storage aspects
+A lot of different storage technologies exist, ranging from general purpose file systems with POSIX semantics to software defined storage with multiple APIs. Choosing any of them is making a tradeoff decision. Or, if a storage technology is already in place it automatically predetermines the capabilities that can be made available. *Not all storage systems are created equal.*
+
+Unfortunately, no POSIX filesystem natively supports all storage aspects that ownCloud 10 requires:
+
+
+### A hierarchical file tree
+An important aspect of a filesystem is organizing files and directories in a file hierarchy, or tree. It allows you to create, move and delete nodes. Beside the name a node also has well known metadata like size and mtime that are persisted in the tree as well.
+
+{{< hint info >}}
+**Folders are not directories**
+There is a difference between *folder* and *directory*: a *directory* is a file system concept. A *folder* is a metaphor for the concept of a physical file folder. There are also *virtual folders* or *smart folders* like the recent files folder which are no file system *directories*. So, every *directory* and every *virtual folder* is a *folder*, but not every *folder* is a *directory*. See [the folder metaphor in wikipedia](https://en.wikipedia.org/wiki/Directory_(computing)#Folder_metaphor). Also see the activity history below.
+{{< /hint >}}
+
+#### Id based lookup
+While traditionally nodes in the tree are reached by traversing the path the tree persistence should be prepared to look up a node by an id. Think of an inode in a POSIX filesystem. If this operation needs to be cached for performance reasons keep in mind that cache invalidation is hard and crawling all files to update the inode to path mapping takes O(n), not O(1).
+
+#### ETag propagation
+For the state based sync a client can discover changes by recursively descending the tree and comparing the ETag for every node. If the storage technology supports propagating ETag changes up the tree, only the root node of a tree needs to be checked to determine if a discovery needs to be started and which nodes need to be traversed. This allows using the storage technology itself to persist all metadata that is necessary for sync, without additional services or caches.
+
+#### Subtree size accounting
+The tree can keep track of how many bytes are stored in a folder. Similar to ETag propagation a change in file size is propagated up the hierarchy.
+
+{{< hint info >}}
+**ETag and Size propagation**
+When propagating the ETag (mtime) and size changes up the tree the question is where to stop. If all changes need to be propagated to the root of a storage then the root or busy folders will become a hotspot. There are two things to keep in mind: 1. propagation only happens up to the root of a single space (a user private drive or a single group drive), 2. no cross storage propagation. The latter was used in oc10 to let clients detect when a file in a received shared folder changed. This functionality is moving to the storage registry which caches the ETag for every root so clients can discover if and which storage changed.
+{{< /hint >}}
+
+#### Rename
+Depending on the underlying storage technology some operations may either be slow, up to a point where it makes more sense to disable them entirely. One example is a folder rename: on S3 a *simple* folder rename translates to a copy and delete operation for every child of the renamed folder. There is an exception though: this restriction only applies if the S3 storage is treated like a filesystem, where the keys are the path and the value is the file content. There are smarter ways to implement file systems on top of S3, but again: there is always a tradeoff.
+
+{{< hint info >}}
+**S3 has no rename**
+Technically, [S3 has no rename operation at all](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-s3-objects.html#copy-object). By design, the location of the value is determined by the key, so it always has to do a copy and delete. Another example is the [redis RENAME operation](https://redis.io/commands/rename): while being specified as O(1) it *executes an implicit DEL operation, so if the deleted key contains a very big value it may cause high latency...*
+{{< /hint >}}
+
+#### Arbitrary metadata persistence
+In addition to well known metadata like name size and mtime, users might be able to add arbitrary metadata like tags, comments or [dublin core](https://en.wikipedia.org/wiki/Dublin_Core). In POSIX filesystems this maps to extended attributes.
+
+### Grant persistence
+The CS3 API uses grants to describe access permissions. Storage systems have a wide range of permissions granularity and not all grants may be supported by every storage driver. POSIX ACLs for example have no expiry. If the storage system does not support certain grant properties, e.g. expiry, then the storage driver may choose to implement them in a different way. Expiries could be persisted in a different way and checked periodically to remove the grants. Again: every decision is a tradeoff.
+
+### Trash persistence
+After deleting a node the storage allows listing the deleted nodes and has an undo mechanism for them.
+
+### Versions persistence
+A user can restore a previous version of a file.
+
+{{< hint info >}}
+**Snapshots are not versions**
+Modern POSIX filesystems support snapshotting of volumes. This is different from keeping track of versions to a file or folder, but might be another implementation strategy for a storage driver to allow users to restore content.
+{{< /hint >}}
+
+### Activity History
+The storage keeps an activity history, tracking the different actions that have been performed. This does not only include file changes but also metadata changes like renames and permission changes.
+
+## Storage drivers
+
+Reva currently has four storage driver implementations that can be used for *storage providers* an well as *data providers*.
+
+### Local Storage Driver
+
+The *minimal* storage driver for a POSIX based filesystem. It literally supports none of the storage aspect other than basic file tree management. Sharing can - to a degree - be implemented using POSIX ACLs.
+
+- tree provided by a POSIX filesystem
+  - inefficient path by id lookup, currently uses the file path as id, so ids are not stable
+    - can store a uuid in extended attributes and use a cache to look them up, similar to the ownCloud driver
+  - no native ETag propagation, five options are available:
+    - built in propagation (changes bypassing ocis are not picked up until a rescan)
+    - built in inotify (requires 48 bytes of RAM per file, needs to keep track of every file and folder)
+    - external inotify (same RAM requirement, but could be triggered by external tools, e.g. a workflow engine)
+    - kernel audit log (use the linux kernel audit to capture file events on the storage and offload them to a queue)
+    - fuse filesystem overlay
+  - no subtree accounting, same options as for ETag propagation
+  - efficient rename
+  - arbitrary metadata using extended attributes
+- grant persistence
+  - using POSIX ACLs
+    - requires an LDAP server to make guest accounts available in the OS
+      - OCIS has glauth which contains all users
+      - an existing LDAP could be used if guests ar provisioned in another way
+  - using extended attributes to implement expiry or sharing that does not require OS level integration
+  - fuse filesystem overlay
+- no native trash
+  - could use the [The FreeDesktop.org Trash specification](https://specifications.freedesktop.org/trash-spec/trashspec-latest.html)
+  - fuse filesystem overlay
+- no native versions, multiple options possible
+  - git for folders
+  - rcs for single files
+  - rsnapshot for hourly / daily / weekly / monthly backups ... but this is not versioning as known from oc10
+  - design new freedesktop spec, basically what is done in oc10 without the limitations or borrow ideas from the freedesktop trash spec
+  - fuse filesystem overlay
+
+To provide the other storage aspects we plan to implement a FUSE overlay filesystem which will add the different aspects on top of local filesystems like ext4, btrfs or xfs. It should work on NFSv45 as well, although NFSv4 supports RichACLs and we will explore how to leverage them to implement sharing at a future date. The idea is to use the storages native capabilities to deliver the best user experience. But again: that means making the right tradeoffs.
+
+### OwnCloud Storage Driver
+
+This is the current default storage driver. While it implements the file tree (using redis, including id based lookup), ETag propagation, trash, versions and sharing (including expiry) using the data directory layout of ownCloud 10 it has [known limitations](https://github.com/owncloud/core/issues/28095) that cannot be fixed without changing the actual layout on disk.
+
+To setup it up properly in a distributed fashion, the storage-home and the storage-oc need to share the same underlying FS. Their "data" counterparts also need access to the same shared FS.
+For a simple docker-compose setup, you can create a volume which will be used by the "storage-storage-home", "storage-storage-home-data", "storage-storage-oc" and "storage-storage-oc-data" containers. Using the `owncloud/ocis` docker image, the volume would need to be hooked in the `/var/tmp/ocis` folder insde the containers.
+
+- tree provided by a POSIX filesystem
+  - file layout is mapped to the old ownCloud 10 layout
+    - the root of tree for a user on disk is prefixed with `/path/to/data/<username>/files/`
+  - efficient path by id lookup
+    - all files and folders get assigned a uuid in the extended attributes
+    - when starting the storage provider it will walk all files to populate a redis kv store for uuid to path lookup
+    - slow to boot trees with lots of nodes
+  - build in ETag propagation
+    - ETags are calculated based on mtime
+    - mtime is propagated by the storage driver
+    - changes bypassing ocis are not picked up until a restart of the storage provider
+  - no subtree accounting, same options as for local storage
+  - efficient rename
+    - TODO [update the kv store for path lookup](https://github.com/cs3org/reva/issues/985), this is an O(n) operation
+  - arbitrary metadata using extended attributes
+- grant persistence
+  - using custom ACLs that are stored as extended attributes
+    - a grant corresponds to one extended attribute of 40-100 bytes, effectively limiting the number of shares to ~100-40
+    - extended attributes have varying limitations, based on the underlying filesystem
+      - the linux kernel imposes a limit of 255bytes per name and 64KiB per value
+      - ext2/3/4: total bytes for all attributes of a file is limited to 4KiB (a filesystem block)
+      - xfs: limit of 64KiB per value
+      - btrfs: total bytes used for the name, value, and implementation overhead bytes 16KiB (the default filesystem nodesize value)
+  -  does not require OS level integration
+- built in trash
+  - trashed files are moved to `/path/to/data/<username>/files_trashbin/`
+  - trashed files are appended a timestamp `.d<unixtime>`, which [breaks trashing of files that reach the filesystems specific name limit](https://github.com/owncloud/core/issues/28095)
+- built in versions
+  - file versions are stored in `/path/to/data/<username>/files_versions/`
+  - file versions are appended a timestamp `.d<unixtime>`, which [breaks versioning of files that reach the filesystems specific name limit](https://github.com/owncloud/core/issues/28095)
+
+### EOS Storage Driver
+
+The CERN eos storage has evolved with ownCloud and natively supports id based lookup, ETag propagation, subtree size accounting, sharing, trash and versions. To use it you need to change the default configuration of the `storage storage-home` command (or have a look at the Makefile ̀ eos-start` target):
+
+```
+export STORAGE_STORAGE_HOME_DRIVER=eos
+export STORAGE_STORAGE_EOS_NAMESPACE=/eos
+export STORAGE_STORAGE_EOS_MASTER_URL="root://eos-mgm1.eoscluster.cern.ch:1094"
+export STORAGE_STORAGE_EOS_ENABLE_HOME=true
+export STORAGE_STORAGE_EOS_LAYOUT="dockertest/{{.Username}}"
+```
+
+Running it locally also requires the `eos` and `xrootd` binaries. Running it using `make eos-start` will use CentOS based containers that already have the necessary packages installed.
+
+{{< hint info >}}
+Pull requests to add explicit `storage storage-(s3|custom|...)` commands with working defaults are welcome.
+{{< /hint >}}
+
+### S3 Storage Driver
+
+A naive driver that treats the keys in an S3 capable storage as `/` delimited path names. While it does not support MOVE or ETag propagation it can be used to read and write files. Better integration with native capabilities like versioning is possible but depends on the Use Case. Several storage solutions that provide an S3 interface also support some form of notifications that can be used to implement ETag propagation.
+
+## Data Providers
+
+Clients using the CS3 API use an [InitiateFileDownload](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.InitiateFileDownloadRequest) and ]InitiateUpload](https://cs3org.github.io/cs3apis/#cs3.storage.provider.v1beta1.InitiateFileUploadRequest) request at the [storage gateway](https://cs3org.github.io/cs3apis/#cs3.gateway.v1beta1.GatewayAPI) to obtain a URL endpoint that can be used to either GET the file content or upload content using the resumable [tus.io](https://tus.io) protocol.
+
+The *data provider* uses the same *storage driver* as the *storage provider* but can be scaled independently.
+
+The dataprovider allows uploading the file to a quarantine area where further data analysis may happen before making the file accessible again. One use case for this is anti virus scanning for files coming from untrusted sources.
+
+## Future work
+
+### FUSE overlay filesystem
+We are planning to further separate the concerns and use a local storage provider with a FUSE filesystem overlaying the actual POSIX storage that can be used to capture deletes and writes that might happen outside of ocis/reva.
+
+It would allow us to extend the local storage driver with missing storage aspects while keeping a tree like filesystem that end users are used to see when sshing into the machine.
+
+### Upload to Quarantine area
+Antivirus scanning of random files uploaded from untrusted sources and executing metadata extraction or thumbnail generation should happen in a sandboxed system to prevent malicious users from gaining any information about the system. By spawning a new container with access to only the uploaded data we can further limit the attack surface.
@@ -0,0 +1,148 @@
+---
+title: "Testing"
+date: 2018-05-02T00:00:00+00:00
+weight: 37
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: testing.md
+---
+
+
+## API Acceptance tests
+
+We are using the ownCloud 10 API acceptance testsuite against ocis. To set this up you need the owncloud 10 core repo, a ldap server that the acceptance tests can use to manage users, a redis server for file-versions and the ocis-reva code.
+
+### Getting the tests
+
+All you need to do to get the acceptance tests is check out the core repo:
+```
+git clone https://github.com/owncloud/core.git
+```
+
+### Run a ldap server in a docker container
+
+The ownCloud 10 acceptance tests will need write permission. You can start a suitable ldap server in a docker container with:
+
+```
+docker run --hostname ldap.my-company.com \
+    -e LDAP_TLS_VERIFY_CLIENT=never \
+    -e LDAP_DOMAIN=owncloud.com \
+    -e LDAP_ORGANISATION=ownCloud \
+    -e LDAP_ADMIN_PASSWORD=admin \
+    --name docker-slapd \
+    -p 127.0.0.1:389:389 \
+    -p 636:636 -d osixia/openldap
+```
+### Run a redis server in a docker container
+
+File versions need a redis server. Start one with docker by using:
+
+`docker run -e REDIS_DATABASES=1 -p 6379:6379 -d webhippie/redis:latest`
+
+### Run ocis-reva with that ldap server
+
+`ocis-reva` provides multiple subcommands. To configure them all via env vars you can export these environment variables.
+
+```
+export REVA_USERS_DRIVER=ldap
+export REVA_LDAP_HOSTNAME=localhost
+export REVA_LDAP_PORT=636
+export REVA_LDAP_BASE_DN='dc=owncloud,dc=com'
+export REVA_LDAP_USERFILTER='(&(objectclass=posixAccount)(cn=%s))'
+export REVA_LDAP_GROUPFILTER='(&(objectclass=posixGroup)(cn=%s))'
+export REVA_LDAP_BIND_DN='cn=admin,dc=owncloud,dc=com'
+export REVA_LDAP_BIND_PASSWORD=admin
+export REVA_LDAP_SCHEMA_UID=uid
+export REVA_LDAP_SCHEMA_MAIL=mail
+export REVA_LDAP_SCHEMA_DISPLAYNAME=displayName
+export REVA_LDAP_SCHEMA_CN=cn
+export REVA_FRONTEND_URL=http://localhost:9140 # needed because the proxy is not started
+export REVA_DATAGATEWAY_URL=http://localhost:9140/data # needed because the proxy is not started
+```
+
+Then you need to start the ocis-reva services
+```
+bin/ocis-reva frontend & \
+bin/ocis-reva gateway & \
+bin/ocis-reva auth-basic & \
+bin/ocis-reva auth-bearer & \
+bin/ocis-reva sharing & \
+bin/ocis-reva storage-home & \
+bin/ocis-reva storage-home-data & \
+bin/ocis-reva storage-oc & \
+bin/ocis-reva storage-oc-data & \
+bin/ocis-reva users &
+```
+
+### Run the API acceptance tests
+
+In the ownCloud 10 core repo run
+
+```
+make test-acceptance-api \
+TEST_SERVER_URL=http://localhost:9140 \
+TEST_EXTERNAL_USER_BACKENDS=true \
+TEST_OCIS=true \
+OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
+BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS&&~@preview-extension-required' \
+SKELETON_DIR=apps/testing/data/apiSkeleton
+```
+
+Make sure to adjust the settings `TEST_SERVER_URL`,`OCIS_REVA_DATA_ROOT` and `SKELETON_DIR` according to your environment.
+
+This will run all tests that are relevant to OCIS.
+
+To run a single test add `BEHAT_FEATURE=<feature file>` and specify the path to the feature file and an optional line number. For example: `BEHAT_FEATURE='tests/acceptance/features/apiWebdavUpload1/uploadFile.feature:12'`
+
+### use existing tests for BDD
+
+As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
+Every scenario that does not work in OCIS with OC storage, is listed in `tests/acceptance/expected-failures-on-OC-storage.txt` with a link to the related issue.
+Similarly, scenarios that do not work in OCIS with EOS storage are listed in `tests/acceptance/expected-failures-on-EOS-storage.txt`.
+Scenarios from the oC10 API acceptance tests are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
+expected failures. If there are any differences then the CI pipeline fails.
+
+Additionally, some issues have scenarios that demonstrate the current buggy behaviour in ocis(reva).
+Those scenarios are in this ocis-reva repository in `tests/acceptance/features/apiOcisSpecific`.
+Have a look into the [documentation](https://doc.owncloud.com/server/developer_manual/testing/acceptance-tests.html#writing-scenarios-for-bugs) to understand why we are writing those tests.
+Also, ocis behaves partly differently with EOS-Storage and OC-Storage. There are scenarios that do not work in OCIS when run on EOS-storage, but works when on OC-Storage, and vice-versa. For those kind of scenarios, ` @skipOnOcis-EOS-Storage` and `@skipOnOcis-OC-Storage` tags are used. For instance, for a scenario that fails on EOS-Storage but passes on OC-Storage, we use `@skipOnOcis-EOS-Storage` tag to let it run on OC-Storage, where it works as expected, instead of skipping the test completely.
+
+If you want to work on a specific issue
+
+1.  adjust the core commit id to the latest commit in core so that CI will run the latest test code and scenarios from core.
+    For that change `coreCommit` in the `config` section:
+
+        config = {
+          'apiTests': {
+            'coreBranch': 'master',
+            'coreCommit': 'a06b1bd5ba8e5244bfaf7fa04f441961e6fb0daa',
+            'numberOfParts': 2
+          }
+        }
+
+2.  locally run each of the tests marked with that issue in the expected failures file:
+
+    E.g.:
+    ```
+    make test-acceptance-api \
+    TEST_SERVER_URL=http://localhost:9140 \
+    TEST_EXTERNAL_USER_BACKENDS=true \
+    TEST_OCIS=true \
+    OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
+    BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
+    ```
+
+3.  the tests will fail, try to understand how and why they are failing
+4.  fix the code
+5.  go back to 2. and repeat till the tests are passing.
+6.  remove those tests from the expected failures file.
+7.  run each of the local tests that were demonstrating the **buggy** behavior. They should fail.
+8.  delete each of the local tests that were demonstrating the **buggy** behavior.
+9.  make a PR that has the fixed code, relevant lines removed from the expected failures file and bug demonstration tests deleted.
+
+    If the changes also affect the `ocis` repository make sure the changes get ported over there.
+    That will need the fixed code in `ocis-reva` to be applied to `ocis` along with the test-related changes.
+
+### Notes
+- in a normal case the test-code cleans up users after the test-run, but if a test-run is interrupted (e.g. by CTRL+C) users might have been left on the LDAP server. In that case rerunning the tests requires wiping the users in the ldap server, otherwise the tests will fail when trying to populate the users. This can be done by simply running `docker stop docker-slapd && docker rm docker-slapd` and [restarting the LDAP server container](#run-a-ldap-server-in-a-docker-container)
+- the tests usually create users in the OU `TestUsers` with usernames specified in the feature file. If not defined in the feature file, most users have the password `123456`, defined by `regularUserPassword` in `behat.yml`, but other passwords are also used, see [`\FeatureContext::getPasswordForUser()`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L386) for mapping and [`\FeatureContext::__construct`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L1668) for the password definitions.
@@ -0,0 +1,19 @@
+---
+title: "Updating reva"
+date: 2020-05-22T00:00:00+00:00
+weight: 50
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: updating.md
+---
+
+{{< toc >}}
+
+## Updating reva
+
+1. Run `go get github.com/cs3org/reva@master`
+2. Create a changelog entry containing changes that were done in [reva](https://github.com/cs3org/reva/commits/master)
+3. Create a Pull Request to ocis-reva master with those changes
+4. If test issues appear, you might need to [adjust the tests]({{< ref "testing.md" >}})
+5. After the PR is merged, consider doing a [release of the storage submodule]({{< ref "releasing.md" >}})
+
@@ -0,0 +1,52 @@
+---
+title: "Users"
+date: 2020-01-16T00:00:00+00:00
+weight: 35
+geekdocRepo: https://github.com/owncloud/ocis
+geekdocEditPath: edit/master/docs/extensions/storage
+geekdocFilePath: users.md
+---
+
+### Demo driver
+
+This is a simple user driver for testing. It contains three users:
+```
+einstein:relativity
+marie:radioactivty
+richard:superfluidity
+```
+In order to use the `demo` driver you need to export the relevant environment variable:
+```
+export STORAGE_USERS_DRIVER=demo
+```
+
+### JSON driver
+
+In order to switch from the `ldap` driver to JSON based users you need to export the relevant environment variables:
+```
+export STORAGE_USERS_DRIVER=json
+export STORAGE_USERS_JSON=/path/to/users.json
+```
+
+For the format of the users.json have a look at the [reva examples](https://github.com/cs3org/reva/blob/master/examples/separate/users.demo.json)
+
+### LDAP driver
+
+This is the default user driver.
+
+If the below defaults don't match your environment change them accordingly:
+```
+export STORAGE_LDAP_HOSTNAME=localhost
+export STORAGE_LDAP_PORT=9126
+export STORAGE_LDAP_BASE_DN='dc=example,dc=org'
+export STORAGE_LDAP_USERFILTER='(&(objectclass=posixAccount)(cn=%s))'
+export STORAGE_LDAP_GROUPFILTER='(&(objectclass=posixGroup)(cn=%s))'
+export STORAGE_LDAP_BIND_DN='cn=reva,ou=sysusers,dc=example,dc=org'
+export STORAGE_LDAP_BIND_PASSWORD=reva
+export STORAGE_LDAP_SCHEMA_UID=uid
+export STORAGE_LDAP_SCHEMA_MAIL=mail
+export STORAGE_LDAP_SCHEMA_DISPLAYNAME=sn
+export STORAGE_LDAP_SCHEMA_CN=cn
+```
+
+Then restart the `bin/storage users` and `bin/storage auth-basic` services for the changes to take effect.