sevenhill/QSfera
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'master' into add_deployment_docs

Browse Source
This commit is contained in:
Jan Müller
2020-10-26 13:57:03 +02:00
committed by GitHub
parent ef9f64e40c f3fb12b3bd
commit ff2e457816
333 changed files with 34634 additions and 9163 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ocis/configuration.md
-232
View File
@@ -1,232 +0,0 @@
---
title: "Configuration"
date: "2020-10-05T21:19:39+0200"
weight: 20
geekdocRepo: https://github.com/owncloud/ocis
geekdocEditPath: edit/master/docs/ocis
geekdocFilePath: configuration.md
---
{{< toc >}}
## Configuration
oCIS Single Binary is not responsible for configuring extensions. Instead, each extension could either be configured by environment variables, cli flags or config files.
Each extension has its dedicated documentation page (e.g. https://owncloud.github.io/extensions/ocis_proxy/configuration) which lists all possible configurations. Config files and environment variables are picked up if you use the `./bin/ocis server` command within the oCIS single binary. Command line flags must be set explicitly on the extensions subcommands.
### Configuration using config files
Out of the box extensions will attempt to read configuration details from:
```console
/etc/ocis
$HOME/.ocis
./config
```
For this configuration to be picked up, have a look at your extension `root` command and look for which default config name it has assigned. *i.e: ocis-proxy reads `proxy.json | yaml | toml ...`*.
So far we support the file formats `JSON` and `YAML`, if you want to get a full example configuration just take a look at [our repository](https://github.com/owncloud/ocis/tree/master/config), there you can always see the latest configuration format. These example configurations include all available options and the default values. The configuration file will be automatically loaded if it's placed at `/etc/ocis/ocis.yml`, `${HOME}/.ocis/ocis.yml` or `$(pwd)/config/ocis.yml`.
### Envrionment variables
If you prefer to configure the service with environment variables you can see the available variables below.
### Commandline flags
If you prefer to configure the service with commandline flags you can see the available variables below. Command line flags are only working when calling the subcommand directly.
## Root Command
ownCloud Infinite Scale Stack
Usage: `ocis [global options] command [command options] [arguments...]`
--config-file | $OCIS_CONFIG_FILE
: Path to config file.
--log-level | $OCIS_LOG_LEVEL
: Set logging level. Default: `info`.
--log-pretty | $OCIS_LOG_PRETTY
: Enable pretty logging. Default: `true`.
--log-color | $OCIS_LOG_COLOR
: Enable colored logging. Default: `true`.
## Sub Commands
### ocis kill
Kill an extension by name
Usage: `ocis kill [command options] [arguments...]`
### ocis server
Start fullstack server
Usage: `ocis server [command options] [arguments...]`
--tracing-enabled | $OCIS_TRACING_ENABLED
: Enable sending traces.
--tracing-type | $OCIS_TRACING_TYPE
: Tracing backend type. Default: `jaeger`.
--tracing-endpoint | $OCIS_TRACING_ENDPOINT
: Endpoint for the agent. Default: `localhost:6831`.
--tracing-collector | $OCIS_TRACING_COLLECTOR
: Endpoint for the collector. Default: `http://localhost:14268/api/traces`.
--tracing-service | $OCIS_TRACING_SERVICE
: Service name for tracing. Default: `ocis`.
--debug-addr | $OCIS_DEBUG_ADDR
: Address to bind debug server. Default: `0.0.0.0:9010`.
--debug-token | $OCIS_DEBUG_TOKEN
: Token to grant metrics access.
--debug-pprof | $OCIS_DEBUG_PPROF
: Enable pprof debugging.
--debug-zpages | $OCIS_DEBUG_ZPAGES
: Enable zpages debugging.
--http-addr | $OCIS_HTTP_ADDR
: Address to bind http server. Default: `0.0.0.0:9000`.
--http-root | $OCIS_HTTP_ROOT
: Root path of http server. Default: `/`.
--grpc-addr | $OCIS_GRPC_ADDR
: Address to bind grpc server. Default: `0.0.0.0:9001`.
### ocis run
Runs an extension
Usage: `ocis run [command options] [arguments...]`
### ocis health
Check health status
Usage: `ocis health [command options] [arguments...]`
--debug-addr | $OCIS_DEBUG_ADDR
: Address to debug endpoint. Default: `0.0.0.0:9010`.
### ocis list
Lists running ocis extensions
Usage: `ocis list [command options] [arguments...]`
### List of available Extension subcommands
There are more subcommands to start the individual extensions. Please check the documentation about their usage and options in the dedicated section of the documentation.
#### ocis konnectd
Start konnectd server
#### ocis storage-frontend
Start storage frontend
#### ocis accounts
Start accounts server
#### ocis storage-storage-root
Start storage root storage
#### ocis storage-gateway
Start storage gateway
#### ocis storage-storage-home
Start storage storage service for home mount
#### ocis storage-storage-public-link
Start storage public link storage
#### ocis store
Start a go-micro store
#### ocis glauth
Start glauth server
#### ocis storage-auth-bearer
Start storage auth-bearer service
#### ocis storage-sharing
Start storage sharing service
#### ocis webdav
Start webdav server
#### ocis storage-storage-oc-data
Start storage storage data provider for oc mount
#### ocis thumbnails
Start thumbnails server
#### ocis proxy
Start proxy server
#### ocis settings
Start settings server
#### ocis storage-auth-basic
Start storage auth-basic service
#### ocis storage-storage-metadata
Start storage storage service for metadata mount
#### ocis ocs
Start ocs server
#### ocis storage-storage-eos
Start storage storage service for eos mount
#### ocis storage-storage-eos-data
Start storage storage data provider for eos mount
#### ocis storage-storage-home-data
Start storage storage data provider for home mount
#### ocis storage-storage-oc
Start storage storage service for oc mount
#### ocis storage-users
Start storage users service
#### ocis phoenix
Start phoenix server
docs/ocis/development/testing.md
+7 -12
View File
@@ -45,8 +45,8 @@ Then run the api acceptance tests with the following command:
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
SKELETON_DIR=apps/testing/data/apiSkeleton \
DELETE_USER_DATA_CMD='rm -rf /var/tmp/ocis/storage/users/nodes/root/* /var/tmp/ocis/storage/users/nodes/*-*-*-*' \
BEHAT_FILTER_TAGS='~@notToImplementOnOCIS&&~@toImplementOnOCIS'
```
@@ -59,13 +59,14 @@ To run a single test add `BEHAT_FEATURE=<feature file>`
### use existing tests for BDD
As a lot of scenarios are written for oC10, we can use those tests for Behaviour driven development in ocis.
Every scenario that does not work in OCIS with OC storage, is listed in `tests/acceptance/expected-failures-on-OC-storage.txt` with a link to the related issue.
Every scenario that does not work in OCIS with "owncloud" storage, is listed in `ocis/tests/acceptance/expected-failures-on-OWNCLOUD-storage.txt` with a link to the related issue.
Every scenario that does not work in OCIS with "ocis" storage, is listed in `ocis/tests/acceptance/expected-failures-on-OCIS-storage.txt` with a link to the related issue.
Those scenarios are run in the ordinary acceptance test pipeline in CI. The scenarios that fail are checked against the
expected failures. If there are any differences then the CI pipeline fails.
Similarly, scenarios that do not work in OCIS with EOS storage are listed in `tests/acceptance/expected-failures-on-EOS-storage.txt`.
Similarly, scenarios that do not work in OCIS with EOS storage are listed in `ocis/tests/acceptance/expected-failures-on-EOS-storage.txt`.
Additionally, some issues have scenarios that demonstrate the current buggy behaviour in ocis(reva).
Those scenarios are in this ocis repository in `tests/acceptance/features/apiOcisSpecific`.
Those scenarios are in this ocis repository in `ocis/tests/acceptance/features/apiOcisSpecific`.
Have a look into the [documentation](https://doc.owncloud.com/server/developer_manual/testing/acceptance-tests.html#writing-scenarios-for-bugs) to understand why we are writing those tests.
If you want to work on a specific issue
@@ -77,7 +78,7 @@ If you want to work on a specific issue
'apiTests': {
'coreBranch': 'master',
'coreCommit': 'a06b1bd5ba8e5244bfaf7fa04f441961e6fb0daa',
'numberOfParts': 2
'numberOfParts': 6
}
}
@@ -88,7 +89,7 @@ If you want to work on a specific issue
make test-acceptance-api \
TEST_SERVER_URL=https://localhost:9200 \
TEST_OCIS=true \
OCIS_REVA_DATA_ROOT=/var/tmp/reva/ \
DELETE_USER_DATA_CMD='rm -rf /var/tmp/ocis/storage/users/nodes/root/* /var/tmp/ocis/storage/users/nodes/*-*-*-*' \
BEHAT_FEATURE='tests/acceptance/features/apiComments/comments.feature:123'
```
@@ -99,9 +100,3 @@ If you want to work on a specific issue
7. run each of the local tests that were demonstrating the **buggy** behavior. They should fail.
8. delete each of the local tests that were demonstrating the **buggy** behavior.
9. make a PR that has the fixed code, relevant lines removed from the expected failures file and bug demonstration tests deleted.
If the changes also affect the `ocis-reva` repository make sure the changes get ported over there.
### Notes
- in a normal case the test-code cleans up users after the test-run, but if a test-run is interrupted (e.g. by CTRL+C) users might have been left on the LDAP server. In that case rerunning the tests requires wiping the users in the ldap server, otherwise the tests will fail when trying to populate the users.
- the tests usually create users in the OU `TestUsers` with usernames specified in the feature file. If not defined in the feature file, most users have the password `123456`, defined by `regularUserPassword` in `behat.yml`, but other passwords are also used, see [`\FeatureContext::getPasswordForUser()`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L386) for mapping and [`\FeatureContext::__construct`](https://github.com/owncloud/core/blob/master/tests/acceptance/features/bootstrap/FeatureContext.php#L1668) for the password definitions.
docs/ocis/eos.md
+37 -12
View File
@@ -49,34 +49,59 @@ uid=20000(einstein) gid=30000(users) groups=30000(users),30001(sailing-lovers),3
If the user is not found at first you might need to wait a few more minutes in case the ocis container is still compiling.
{{< /hint >}}
We also need to restart the storage-users service, so it picks up the changed environment. Without a restart it is not able to resolve users from LDAP.
We also need to restart the storage-userprovider service, so it picks up the changed environment. Without a restart it is not able to resolve users from LDAP.
```
docker-compose exec ocis ./bin/ocis kill storage-users
docker-compose exec ocis ./bin/ocis run storage-users
docker-compose exec ocis ./bin/ocis kill storage-userprovider
docker-compose exec ocis ./bin/ocis run storage-userprovider
```
### 3. Home storage
Kill the home storage. By default it uses the `owncloud` storage driver. We need to switch it to the `eoshome` driver and make it use the storage id of the eos storage provider:
Kill the home storage. By default it uses the `ocis` storage driver. We need to switch it to the `eoshome` driver:
```
docker-compose exec ocis ./bin/ocis kill storage-storage-home
docker-compose exec -e STORAGE_STORAGE_HOME_DRIVER=eoshome -e STORAGE_STORAGE_HOME_MOUNT_ID=1284d238-aa92-42ce-bdc4-0b0000009158 ocis ./bin/ocis run storage-storage-home
docker-compose exec ocis ./bin/ocis kill storage-home
docker-compose exec -e STORAGE_HOME_DRIVER=eoshome ocis ./bin/ocis run storage-home
```
### 4. Home data provider
### 4. Users storage
Kill the home data provider. By default it uses the `owncloud` storage driver. We need to switch it to the `eoshome` driver and make it use the storage id of the eos storage provider:
Kill the users storage. By default it uses the `ocis` storage driver. We need to switch it to the `eos` driver:
```
docker-compose exec ocis ./bin/ocis kill storage-storage-home-data
docker-compose exec -e STORAGE_STORAGE_HOME_DATA_DRIVER=eoshome ocis ./bin/ocis run storage-storage-home-data
docker-compose exec ocis ./bin/ocis kill storage-users
docker-compose exec -e STORAGE_USERS_DRIVER=eos ocis ./bin/ocis run storage-users
```
### 5. Metadata storage
First we need to create the metadata root in eos and set an owner:
```
docker-compose exec ocis eos mkdir -p /eos/dockertest/ocis/metadata
docker-compose exec ocis eos chown 2:2 /eos/dockertest/ocis/metadata
```
{{< hint info >}}
The difference between the *home storage* and the *home data provider* are that the former is responsible for metadata changes while the latter is responsible for actual data transfer. The *home storage* uses the cs3 api to manage a folder hierarchy, while the *home data provider* is responsible for moving bytes to and from the storage.
The uid and gid `2` are referencing the user `daemon` inside the ocis container. That user is also configured when restarting the accounts service later. For production systems you should create a dedicated user for the metadata storage.
{{< /hint >}}
Kill the metadata storage. By default it uses the `ocis` storage driver. We need to switch it to the `eos` driver:
```
docker-compose exec ocis ./bin/ocis kill storage-metadata
docker-compose exec -e STORAGE_METADATA_DRIVER=eos -e STORAGE_METADATA_ROOT=/eos/dockertest/ocis/metadata ocis ./bin/ocis run storage-metadata
```
### 6. Accounts service
Kill the accounts service. By default it uses the `ocis` storage driver. We need to switch it to the `eos` driver:
```
docker-compose exec ocis ./bin/ocis kill accounts
docker-compose exec -e ACCOUNTS_SERVICE_USER_USERNAME=daemon -e ACCOUNTS_SERVICE_USER_UID=2 -e ACCOUNTS_SERVICE_USER_GID=2 ocis ./bin/ocis run accounts
```
## Verification
Login with `einstein / relativity`, upload a file to einsteins home and verify the file is there using
@@ -199,7 +224,7 @@ The ocis logs can be accessed using `docker-compose logs ocis`. Add `-f` for fol
1. `docker-compose exec ocis make clean build` to update the binary
2. `docker-compose exec ocis ./bin/ocis kill <service>` to kill the service
3. `docker-compose exec ocis ./bin/ocis run <service>` to start the service. Do not forget to set any env vars, eg.
`docker-compose exec -e STORAGE_STORAGE_EOS_LAYOUT="{{substr 0 1 .Id.OpaqueId}}/{{.Id.OpaqueId}}" -e STORAGE_STORAGE_HOME_DRIVER=eoshome ocis ./bin/ocis run storage-storage-home`
`docker-compose exec -e STORAGE_HOME_DRIVER=eoshome -e STORAGE_DRIVER_EOS_LAYOUT="{{substr 0 1 .Id.OpaqueId}}/{{.Id.OpaqueId}}" ocis ./bin/ocis run storage-home`
### Creation and upload of files does not work