From 2031461349bc948927e7830c149e44cbe67074e2 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Tue, 8 Feb 2022 13:26:48 +0100 Subject: [PATCH 01/24] ADR: Add first draft for readonly external user management --- .../ocis/adr/0003-external-user-management.md | 5 +- ...llow-read-only-external-user-management.md | 96 +++++++++++++++++++ 2 files changed, 99 insertions(+), 2 deletions(-) create mode 100644 docs/ocis/adr/0017-allow-read-only-external-user-management.md diff --git a/docs/ocis/adr/0003-external-user-management.md b/docs/ocis/adr/0003-external-user-management.md index 025c2ec21..7cdb66dda 100644 --- a/docs/ocis/adr/0003-external-user-management.md +++ b/docs/ocis/adr/0003-external-user-management.md @@ -7,9 +7,9 @@ geekdocEditPath: edit/master/docs/ocis/adr geekdocFilePath: 0003-external-user-management.md --- -* Status: accepted +* Status: superseded by [17. Allow read only external User Management]({{< ref "0017-allow-read-only-external-user-management.md" >}}) * Deciders: @butonic, @micbar, @dragotin, @hodyroff, @pmaier1 -* Date: 2020-12-09 +* Date: 2022-02-08 Technical Story: [Skip account-service by talking to CS3 user-api](https://github.com/owncloud/ocis/pull/1020) @@ -112,3 +112,4 @@ We should use an existing LDAP server and make GLAuth a drop in replacement for ## Links * supersedes [ADR-0001]({{< ref "0001-introduce-accounts-service.md" >}}) +* superseded by [17. Allow read only external User Management]({{< ref "0017-allow-read-only-external-user-management.md" >}}) diff --git a/docs/ocis/adr/0017-allow-read-only-external-user-management.md b/docs/ocis/adr/0017-allow-read-only-external-user-management.md new file mode 100644 index 000000000..3660f9599 --- /dev/null +++ b/docs/ocis/adr/0017-allow-read-only-external-user-management.md @@ -0,0 +1,96 @@ +--- +title: "17. Allow read only external User Management" +--- + +* Status: proposed +* Deciders: @butonic, @micbar, @rhafer +* Date: 2022-02-08 + +## Context and Problem Statement + +oCIS needs to be integrated with various external Authentication and Identity Management System. We +settled on Open ID Connect (OIDC) as the central authentication protocol for OCIS. + +OCIS internally relies on a stable and persistent identifier (e.g. a UUID) for accounts in order to +implement permissions and sharing. Unfortunately, some deployments are unable to deliver this kind +of stable identifier for users: + +- In OIDC itself the only stable identifier that is guaranteed to be provided by the IDP is + combination of the sub and iss claims. IDPs can optionally return other claims, but we might not + be able to rely on a specific claim being present. +- When no other services (LDAP, SCIM, ...) is available that could be used look up a user UUID + +Furthermore, there is a difference between users, identities and accounts: A user may have multiple +identities whith which he can authenticate, e.g. his facebook, twitter, microsoft or google +identity. Multiple identities can be linked to an account in ocis, allowing to fall back to another +identity provider should one of them shut down. This also allows migrating from one identity +provider to another. + +There are three cases that require access to users:a + +1. During authentication we neet to build a user object with at least an account uuid (to identify + the account) and the email (for display purposes) +2. When searching for recipients we need to be able to query existing users in the external identity + management system +3. When listing files we need to be able to look up a users display properties (username, email, + avatar...) based on the account uuid + +## Decision Drivers + +* oCIS should be a single binary that can run out of the box without external dependencies like an + LDAP server. +* Time: we want to build a release candidate asap. +* oCIS should be able to be easily integrated with external standard identity mangement systems + +## Considered Options + +* External identity management system is writeable and has all necessary APIs +* External identity management system is read only and provides an interface to query users (e.g. +* IdP is read only and does not provide an API to query users + +## Decision Outcome + +tbd + +### Positive Consequences: + + +### Negative consequences: + + +## Pros and Cons of the Options + +### External identity management system is writeable and has all necessary APIs + +IdP sends all necessary claims: uuid, username, email, displayname, avatar url IdP allows lookup of +display properties by the uuid or email/username + +* Good, because we can fully rely on the external identity management system +* Bad, because we need write access to provision guest accounts + +### External identity management system is read only and provides an interface to query users (e.g. Coporate Active Directy) + +IdP ends sub & iss and mail or username claims, Identity Management System provides Interfaces (e.g. +LDAP) to lookup additional user information. All services use the CS3 API to look up the account for +the given email or username, where CS3 provides backends for LDAP, SCIM, REST ... + +* Good, because we can rely on the external identity management +* Bad, because the Identity managment needs to provide a stable, persistent, non-reussignable user + identifier to identify the account, e.g. owncloudUUID or ms-DS-ConsistencyGuid +* Bad, because we need to implment tools that can change the account id when it did change anyway +* Bad, because we will hammer the identity management system with lookup requests (can mostly be + mitigated with caching) + +### IdP is read only and does not provide an API to query users + +Idp sends sub & iss and mail or username claims. We need to provision an internal account mapping to +look up user properties by account id. + +* Good, because this has very little external requirements +* Bad, because we have to provide the user lookup APIs + +## Links + +* [Link type] [Link to ADR] +* … +* supersedes [3. Use external User Management]({{< ref "0003-external-user-management.md" >}}) From 5a390baaa1bb8a2c35ebeff9f1e9a632bdf2ca3b Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Tue, 8 Feb 2022 15:19:27 +0100 Subject: [PATCH 02/24] Typo fixes and clarifications MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Jörn Friedrich Dreyer --- ...llow-read-only-external-user-management.md | 34 +++++++++++-------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/docs/ocis/adr/0017-allow-read-only-external-user-management.md b/docs/ocis/adr/0017-allow-read-only-external-user-management.md index 3660f9599..a43713bac 100644 --- a/docs/ocis/adr/0017-allow-read-only-external-user-management.md +++ b/docs/ocis/adr/0017-allow-read-only-external-user-management.md @@ -1,5 +1,10 @@ --- title: "17. Allow read only external User Management" +weight: 17 +date: 2022-02-08T10:53:00+01:00 +geekdocRepo: https://github.com/owncloud/ocis +geekdocEditPath: edit/master/docs/ocis/adr +geekdocFilePath: 0017-allow-read-only-external-user-management.md --- * Status: proposed @@ -8,7 +13,7 @@ title: "17. Allow read only external User Management" ## Context and Problem Statement -oCIS needs to be integrated with various external Authentication and Identity Management System. We +oCIS needs to be integrated with various external Authentication and Identity Management Systems. We settled on Open ID Connect (OIDC) as the central authentication protocol for OCIS. OCIS internally relies on a stable and persistent identifier (e.g. a UUID) for accounts in order to @@ -26,27 +31,27 @@ identity. Multiple identities can be linked to an account in ocis, allowing to f identity provider should one of them shut down. This also allows migrating from one identity provider to another. -There are three cases that require access to users:a +There are three cases that require access to users: -1. During authentication we neet to build a user object with at least an account uuid (to identify +1. During authentication we need to build a user object with at least an account UUID (to identify the account) and the email (for display purposes) 2. When searching for recipients we need to be able to query existing users in the external identity management system 3. When listing files we need to be able to look up a users display properties (username, email, - avatar...) based on the account uuid + avatar...) based on the account UUID ## Decision Drivers * oCIS should be a single binary that can run out of the box without external dependencies like an LDAP server. * Time: we want to build a release candidate asap. -* oCIS should be able to be easily integrated with external standard identity mangement systems +* oCIS should be easy to integrate with standard external identity mangement systems ## Considered Options * External identity management system is writeable and has all necessary APIs -* External identity management system is read only and provides an interface to query users (e.g. -* IdP is read only and does not provide an API to query users +* External identity management system is read only and provides an interface to query users +* External identity management system is read only and does NOT provide an API to query users ## Decision Outcome @@ -70,18 +75,17 @@ display properties by the uuid or email/username ### External identity management system is read only and provides an interface to query users (e.g. Coporate Active Directy) -IdP ends sub & iss and mail or username claims, Identity Management System provides Interfaces (e.g. -LDAP) to lookup additional user information. All services use the CS3 API to look up the account for -the given email or username, where CS3 provides backends for LDAP, SCIM, REST ... +IdP sends sub & iss and mail or username claims, Identity Management System provides APIs (e.g. +LDAP, SCIM, REST ...) to lookup additional user information. All services use the CS3 API to look up the account for +the given email or username, where CS3 then uses a backend that relies on the APIs provided by the IdM``` * Good, because we can rely on the external identity management -* Bad, because the Identity managment needs to provide a stable, persistent, non-reussignable user - identifier to identify the account, e.g. owncloudUUID or ms-DS-ConsistencyGuid +* Bad, because the identity managment needs to provide a stable, persistent, non-reassignable user + identifier for an account, e.g. `owncloudUUID` or `ms-DS-ConsistencyGuid` * Bad, because we need to implment tools that can change the account id when it did change anyway -* Bad, because we will hammer the identity management system with lookup requests (can mostly be - mitigated with caching) +* Bad, because without caching we will hammer the identity management system with lookup requests -### IdP is read only and does not provide an API to query users +### External identity management system is read only and does NOT provide an API to query users Idp sends sub & iss and mail or username claims. We need to provision an internal account mapping to look up user properties by account id. From 6a4af40d524a28c7c8ec2c715d7a7ddc817dbaf5 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Tue, 8 Feb 2022 15:20:26 +0100 Subject: [PATCH 03/24] Collect more pros and cons for the different options MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Jörn Friedrich Dreyer --- ...llow-read-only-external-user-management.md | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/docs/ocis/adr/0017-allow-read-only-external-user-management.md b/docs/ocis/adr/0017-allow-read-only-external-user-management.md index a43713bac..4526b294d 100644 --- a/docs/ocis/adr/0017-allow-read-only-external-user-management.md +++ b/docs/ocis/adr/0017-allow-read-only-external-user-management.md @@ -71,15 +71,22 @@ IdP sends all necessary claims: uuid, username, email, displayname, avatar url I display properties by the uuid or email/username * Good, because we can fully rely on the external identity management system -* Bad, because we need write access to provision guest accounts +* Bad, because we need write access to provision guest accounts (very few customers are willing to + provide that) ### External identity management system is read only and provides an interface to query users (e.g. Coporate Active Directy) IdP sends sub & iss and mail or username claims, Identity Management System provides APIs (e.g. -LDAP, SCIM, REST ...) to lookup additional user information. All services use the CS3 API to look up the account for -the given email or username, where CS3 then uses a backend that relies on the APIs provided by the IdM``` +LDAP, SCIM, REST ...) to lookup additional user information. All services use the CS3 API to look up +the account for the given email or username, where CS3 then uses a backend that relies on the APIs +provided by the IdM. * Good, because we can rely on the external identity management +* Good, because ocis services only need to know about the CS3 user provider API, which acts as an + abstraction layer for different identitiy management systems +* Good, because there is only an single source of truth (the external IdM) and we don't need to + implement a synchronization mechanism to maintain an internal user database (we will likely need + some form of caching though, see below) * Bad, because the identity managment needs to provide a stable, persistent, non-reassignable user identifier for an account, e.g. `owncloudUUID` or `ms-DS-ConsistencyGuid` * Bad, because we need to implment tools that can change the account id when it did change anyway @@ -87,11 +94,13 @@ the given email or username, where CS3 then uses a backend that relies on the AP ### External identity management system is read only and does NOT provide an API to query users -Idp sends sub & iss and mail or username claims. We need to provision an internal account mapping to -look up user properties by account id. +Idp sends sub & iss and mail or username claims. We need to provision an internal account mapping +upon first login of a user to be able to look up user properties by account id. * Good, because this has very little external requirements +* Good, because we have accounts fully under our control * Bad, because we have to provide the user lookup APIs +* Bad, because users will only a visible after the first login ## Links From 5c59bf7b59ad4d9c15f33c07d35df589b1ebdd76 Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Tue, 8 Feb 2022 16:05:47 +0100 Subject: [PATCH 04/24] Rearrange Context and Problem Statement a bit --- ...llow-read-only-external-user-management.md | 35 ++++++++++--------- 1 file changed, 18 insertions(+), 17 deletions(-) diff --git a/docs/ocis/adr/0017-allow-read-only-external-user-management.md b/docs/ocis/adr/0017-allow-read-only-external-user-management.md index 4526b294d..f051848b0 100644 --- a/docs/ocis/adr/0017-allow-read-only-external-user-management.md +++ b/docs/ocis/adr/0017-allow-read-only-external-user-management.md @@ -13,33 +13,34 @@ geekdocFilePath: 0017-allow-read-only-external-user-management.md ## Context and Problem Statement -oCIS needs to be integrated with various external Authentication and Identity Management Systems. We -settled on Open ID Connect (OIDC) as the central authentication protocol for OCIS. +oCIS needs to be integrated with various external Authentication and Identity Management Systems. -OCIS internally relies on a stable and persistent identifier (e.g. a UUID) for accounts in order to -implement permissions and sharing. Unfortunately, some deployments are unable to deliver this kind -of stable identifier for users: - -- In OIDC itself the only stable identifier that is guaranteed to be provided by the IDP is - combination of the sub and iss claims. IDPs can optionally return other claims, but we might not - be able to rely on a specific claim being present. -- When no other services (LDAP, SCIM, ...) is available that could be used look up a user UUID - -Furthermore, there is a difference between users, identities and accounts: A user may have multiple +Sidenote: There is a difference between users, identities and accounts: A user may have multiple identities whith which he can authenticate, e.g. his facebook, twitter, microsoft or google identity. Multiple identities can be linked to an account in ocis, allowing to fall back to another identity provider should one of them shut down. This also allows migrating from one identity provider to another. -There are three cases that require access to users: +There are different cases where oCIS requires access to users: -1. During authentication we need to build a user object with at least an account UUID (to identify - the account) and the email (for display purposes) -2. When searching for recipients we need to be able to query existing users in the external identity - management system +1. While we settled on using OpenID Connect (OIDC) as the authentication protocol for oCIS, we + we need to build a user object during authentication with at least an account UUID (to identify + the account) and the email or a name (for display purposes). +2. When searching for share recipients we need to be able to query existing users in the external + identity management system 3. When listing files we need to be able to look up a users display properties (username, email, avatar...) based on the account UUID +oCIS internally relies on a stable and persistent identifier (e.g. a UUID) for accounts in order to +implement permissions and sharing. Unfortunately, some deployments are unable to deliver this kind +of stable identifier for users: + +- In OIDC itself the only stable identifier that is guaranteed to be provided by the IDP is + combination of the sub and iss claims. IDPs can optionally return other claims, but we cannot + rely on a specific claim being present. +- When no other services (LDAP, SCIM, ...) is available that could be used look up a user UUID + + ## Decision Drivers * oCIS should be a single binary that can run out of the box without external dependencies like an From 22ca7e62238625a51672cbdbd7e062288be3451d Mon Sep 17 00:00:00 2001 From: Ralf Haferkamp Date: Thu, 10 Feb 2022 15:54:19 +0100 Subject: [PATCH 05/24] removed the "identity management system is writeable" This ADR is supposed to be about the read-only case. --- ...llow-read-only-external-user-management.md | 37 ++++++++++--------- 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/docs/ocis/adr/0017-allow-read-only-external-user-management.md b/docs/ocis/adr/0017-allow-read-only-external-user-management.md index f051848b0..a6c2c719a 100644 --- a/docs/ocis/adr/0017-allow-read-only-external-user-management.md +++ b/docs/ocis/adr/0017-allow-read-only-external-user-management.md @@ -14,6 +14,9 @@ geekdocFilePath: 0017-allow-read-only-external-user-management.md ## Context and Problem Statement oCIS needs to be integrated with various external Authentication and Identity Management Systems. +Usually oCIS will no have administrative access to such a system and we will not be allowed to +reconfigure it suite our needs (e.g. we will not be able to enhance the schema of an already existing +LDAP Directory). In most case our access will be read-only. Sidenote: There is a difference between users, identities and accounts: A user may have multiple identities whith which he can authenticate, e.g. his facebook, twitter, microsoft or google @@ -50,37 +53,32 @@ of stable identifier for users: ## Considered Options -* External identity management system is writeable and has all necessary APIs -* External identity management system is read only and provides an interface to query users -* External identity management system is read only and does NOT provide an API to query users +There are two case to consider: +* External identity management system provides an OIDC IdP and an interface to query users +* External identity management system provides just an OIDC IdP with no possibility to query users ## Decision Outcome -tbd +It's not really possible single out any of the options for this ADR. In the end we will likely need +to support both scenarios. ### Positive Consequences: +* Very flexible integration with a wide range of external systems ### Negative consequences: +* configuration complexity, high support efforts +* Increasingly complex code in oCIS ## Pros and Cons of the Options -### External identity management system is writeable and has all necessary APIs - -IdP sends all necessary claims: uuid, username, email, displayname, avatar url IdP allows lookup of -display properties by the uuid or email/username - -* Good, because we can fully rely on the external identity management system -* Bad, because we need write access to provision guest accounts (very few customers are willing to - provide that) - ### External identity management system is read only and provides an interface to query users (e.g. Coporate Active Directy) IdP sends sub & iss and mail or username claims, Identity Management System provides APIs (e.g. -LDAP, SCIM, REST ...) to lookup additional user information. All services use the CS3 API to look up -the account for the given email or username, where CS3 then uses a backend that relies on the APIs -provided by the IdM. +LDAP, SCIM, REST ...) to lookup additional user information. All oCIS services use the CS3 API to +look up the account for the given email or username, where CS3 then uses a backend that relies on +the APIs provided by the IdM. * Good, because we can rely on the external identity management * Good, because ocis services only need to know about the CS3 user provider API, which acts as an @@ -95,13 +93,16 @@ provided by the IdM. ### External identity management system is read only and does NOT provide an API to query users -Idp sends sub & iss and mail or username claims. We need to provision an internal account mapping -upon first login of a user to be able to look up user properties by account id. +Idp sends sub & iss and mail or username claims. We need to provision an internal account mapping, +creating a unique ID, upon the first login of a user to be able to look up user properties by account +id. * Good, because this has very little external requirements * Good, because we have accounts fully under our control * Bad, because we have to provide the user lookup APIs * Bad, because users will only a visible after the first login +* Bad, because our internal account mapping might get out of date when user attribute (e.g. name or + mail) change. At least until the next time that user logs in ## Links From 238d2dfdbc19fa4eaeef47c910bebcd17f31fc82 Mon Sep 17 00:00:00 2001 From: David Christofas Date: Fri, 18 Feb 2022 15:44:53 +0100 Subject: [PATCH 06/24] add new notifications service --- notifications/cmd/notifications/main.go | 14 +++++ notifications/pkg/command/health.go | 18 +++++++ notifications/pkg/command/root.go | 64 +++++++++++++++++++++++ notifications/pkg/command/server.go | 47 +++++++++++++++++ notifications/pkg/command/version.go | 19 +++++++ notifications/pkg/config/config.go | 19 +++++++ notifications/pkg/config/debug.go | 9 ++++ notifications/pkg/config/defaultconfig.go | 12 +++++ notifications/pkg/config/log.go | 9 ++++ notifications/pkg/config/parser/parse.go | 40 ++++++++++++++ notifications/pkg/config/service.go | 6 +++ notifications/pkg/logging/logging.go | 17 ++++++ notifications/pkg/service/service.go | 46 ++++++++++++++++ ocis-pkg/config/config.go | 2 + ocis-pkg/config/defaultconfig.go | 2 + ocis/pkg/command/notifications.go | 26 +++++++++ ocis/pkg/runtime/service/service.go | 2 + 17 files changed, 352 insertions(+) create mode 100644 notifications/cmd/notifications/main.go create mode 100644 notifications/pkg/command/health.go create mode 100644 notifications/pkg/command/root.go create mode 100644 notifications/pkg/command/server.go create mode 100644 notifications/pkg/command/version.go create mode 100644 notifications/pkg/config/config.go create mode 100644 notifications/pkg/config/debug.go create mode 100644 notifications/pkg/config/defaultconfig.go create mode 100644 notifications/pkg/config/log.go create mode 100644 notifications/pkg/config/parser/parse.go create mode 100644 notifications/pkg/config/service.go create mode 100644 notifications/pkg/logging/logging.go create mode 100644 notifications/pkg/service/service.go create mode 100644 ocis/pkg/command/notifications.go diff --git a/notifications/cmd/notifications/main.go b/notifications/cmd/notifications/main.go new file mode 100644 index 000000000..6b4323433 --- /dev/null +++ b/notifications/cmd/notifications/main.go @@ -0,0 +1,14 @@ +package main + +import ( + "os" + + "github.com/owncloud/ocis/notifications/pkg/command" + "github.com/owncloud/ocis/notifications/pkg/config" +) + +func main() { + if err := command.Execute(config.DefaultConfig()); err != nil { + os.Exit(1) + } +} diff --git a/notifications/pkg/command/health.go b/notifications/pkg/command/health.go new file mode 100644 index 000000000..7f2adb85b --- /dev/null +++ b/notifications/pkg/command/health.go @@ -0,0 +1,18 @@ +package command + +import ( + "github.com/owncloud/ocis/notifications/pkg/config" + "github.com/urfave/cli/v2" +) + +// Health is the entrypoint for the health command. +func Health(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "health", + Usage: "Check health status", + Action: func(c *cli.Context) error { + // Not implemented + return nil + }, + } +} diff --git a/notifications/pkg/command/root.go b/notifications/pkg/command/root.go new file mode 100644 index 000000000..805d8cec3 --- /dev/null +++ b/notifications/pkg/command/root.go @@ -0,0 +1,64 @@ +package command + +import ( + "context" + "os" + + "github.com/owncloud/ocis/notifications/pkg/config" + "github.com/owncloud/ocis/ocis-pkg/clihelper" + ociscfg "github.com/owncloud/ocis/ocis-pkg/config" + "github.com/thejerf/suture/v4" + "github.com/urfave/cli/v2" +) + +// GetCommands provides all commands for this service +func GetCommands(cfg *config.Config) cli.Commands { + return []*cli.Command{ + // start this service + Server(cfg), + + // interaction with this service + + // infos about this service + Health(cfg), + Version(cfg), + } +} + +// Execute is the entry point for the notifications command. +func Execute(cfg *config.Config) error { + app := clihelper.DefaultApp(&cli.App{ + Name: "notifications", + Usage: "starts notifications service", + Commands: GetCommands(cfg), + }) + + cli.HelpFlag = &cli.BoolFlag{ + Name: "help,h", + Usage: "Show the help", + } + + return app.Run(os.Args) +} + +// SutureService allows for the notifications command to be embedded and supervised by a suture supervisor tree. +type SutureService struct { + cfg *config.Config +} + +// NewSutureService creates a new notifications.SutureService +func NewSutureService(cfg *ociscfg.Config) suture.Service { + cfg.Settings.Commons = cfg.Commons + return SutureService{ + cfg: cfg.Notifications, + } +} + +func (s SutureService) Serve(ctx context.Context) error { + s.cfg.Context = ctx + if err := Execute(s.cfg); err != nil { + return err + } + + return nil +} diff --git a/notifications/pkg/command/server.go b/notifications/pkg/command/server.go new file mode 100644 index 000000000..48ba224aa --- /dev/null +++ b/notifications/pkg/command/server.go @@ -0,0 +1,47 @@ +package command + +import ( + "fmt" + + "github.com/asim/go-micro/plugins/events/nats/v4" + "github.com/cs3org/reva/pkg/events" + "github.com/cs3org/reva/pkg/events/server" + "github.com/owncloud/ocis/notifications/pkg/config" + "github.com/owncloud/ocis/notifications/pkg/config/parser" + "github.com/owncloud/ocis/notifications/pkg/logging" + "github.com/owncloud/ocis/notifications/pkg/service" + "github.com/urfave/cli/v2" +) + +// Server is the entrypoint for the server command. +func Server(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "server", + Usage: fmt.Sprintf("start %s extension without runtime (unsupervised mode)", cfg.Service.Name), + Category: "server", + Before: func(c *cli.Context) error { + return parser.ParseConfig(cfg) + }, + Action: func(c *cli.Context) error { + logger := logging.Configure(cfg.Service.Name, cfg.Log) + + group := "notifications" + + evs := []events.Unmarshaller{ + events.ShareCreated{}, + } + + client, err := server.NewNatsStream(nats.Address("127.0.0.1:4222"), nats.ClusterID("test-cluster")) + if err != nil { + return err + } + evts, err := events.Consume(client, group, evs...) + if err != nil { + return err + } + + svc := service.NewEventsNotifier(evts, logger) + return svc.Run() + }, + } +} diff --git a/notifications/pkg/command/version.go b/notifications/pkg/command/version.go new file mode 100644 index 000000000..f2d47a569 --- /dev/null +++ b/notifications/pkg/command/version.go @@ -0,0 +1,19 @@ +package command + +import ( + "github.com/owncloud/ocis/notifications/pkg/config" + "github.com/urfave/cli/v2" +) + +// Version prints the service versions of all running instances. +func Version(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "version", + Usage: "print the version of this binary and the running extension instances", + Category: "info", + Action: func(c *cli.Context) error { + // not implemented + return nil + }, + } +} diff --git a/notifications/pkg/config/config.go b/notifications/pkg/config/config.go new file mode 100644 index 000000000..254832cd1 --- /dev/null +++ b/notifications/pkg/config/config.go @@ -0,0 +1,19 @@ +package config + +import ( + "context" + + "github.com/owncloud/ocis/ocis-pkg/shared" +) + +// Config combines all available configuration parts. +type Config struct { + *shared.Commons + + Service Service + + Log *Log `ocisConfig:"log"` + Debug Debug `ocisConfig:"debug"` + + Context context.Context +} diff --git a/notifications/pkg/config/debug.go b/notifications/pkg/config/debug.go new file mode 100644 index 000000000..da6d2d590 --- /dev/null +++ b/notifications/pkg/config/debug.go @@ -0,0 +1,9 @@ +package config + +// Debug defines the available debug configuration. +type Debug struct { + Addr string `ocisConfig:"addr" env:"NOTIFICATIONS_DEBUG_ADDR"` + Token string `ocisConfig:"token" env:"NOTIFICATIONS_DEBUG_TOKEN"` + Pprof bool `ocisConfig:"pprof" env:"NOTIFICATIONS_DEBUG_PPROF"` + Zpages bool `ocisConfig:"zpages" env:"NOTIFICATIONS_DEBUG_ZPAGES"` +} diff --git a/notifications/pkg/config/defaultconfig.go b/notifications/pkg/config/defaultconfig.go new file mode 100644 index 000000000..e98e72ff6 --- /dev/null +++ b/notifications/pkg/config/defaultconfig.go @@ -0,0 +1,12 @@ +package config + +// NOTE: Most of this configuration is not needed to keep it as simple as possible +// TODO: Clean up unneeded configuration + +func DefaultConfig() *Config { + return &Config{ + Service: Service{ + Name: "notifications", + }, + } +} diff --git a/notifications/pkg/config/log.go b/notifications/pkg/config/log.go new file mode 100644 index 000000000..ddb4d391f --- /dev/null +++ b/notifications/pkg/config/log.go @@ -0,0 +1,9 @@ +package config + +// Log defines the available log configuration. +type Log struct { + Level string `mapstructure:"level" env:"OCIS_LOG_LEVEL;NOTIFICATIONS_LOG_LEVEL"` + Pretty bool `mapstructure:"pretty" env:"OCIS_LOG_PRETTY;NOTIFICATIONS_LOG_PRETTY"` + Color bool `mapstructure:"color" env:"OCIS_LOG_COLOR;NOTIFICATIONS_LOG_COLOR"` + File string `mapstructure:"file" env:"OCIS_LOG_FILE;NOTIFICATIONS_LOG_FILE"` +} diff --git a/notifications/pkg/config/parser/parse.go b/notifications/pkg/config/parser/parse.go new file mode 100644 index 000000000..5bc4e6e57 --- /dev/null +++ b/notifications/pkg/config/parser/parse.go @@ -0,0 +1,40 @@ +package parser + +import ( + "errors" + + "github.com/owncloud/ocis/notifications/pkg/config" + ociscfg "github.com/owncloud/ocis/ocis-pkg/config" + + "github.com/owncloud/ocis/ocis-pkg/config/envdecode" +) + +// ParseConfig loads accounts configuration from known paths. +func ParseConfig(cfg *config.Config) error { + _, err := ociscfg.BindSourcesToStructs(cfg.Service.Name, cfg) + if err != nil { + return err + } + + // provide with defaults for shared logging, since we need a valid destination address for BindEnv. + if cfg.Log == nil && cfg.Commons != nil && cfg.Commons.Log != nil { + cfg.Log = &config.Log{ + Level: cfg.Commons.Log.Level, + Pretty: cfg.Commons.Log.Pretty, + Color: cfg.Commons.Log.Color, + File: cfg.Commons.Log.File, + } + } else if cfg.Log == nil { + cfg.Log = &config.Log{} + } + + // load all env variables relevant to the config in the current context. + if err := envdecode.Decode(cfg); err != nil { + // no environment variable set for this config is an expected "error" + if !errors.Is(err, envdecode.ErrNoTargetFieldsAreSet) { + return err + } + } + + return nil +} diff --git a/notifications/pkg/config/service.go b/notifications/pkg/config/service.go new file mode 100644 index 000000000..f98aa3d27 --- /dev/null +++ b/notifications/pkg/config/service.go @@ -0,0 +1,6 @@ +package config + +// Service defines the available service configuration. +type Service struct { + Name string +} diff --git a/notifications/pkg/logging/logging.go b/notifications/pkg/logging/logging.go new file mode 100644 index 000000000..039b0451c --- /dev/null +++ b/notifications/pkg/logging/logging.go @@ -0,0 +1,17 @@ +package logging + +import ( + "github.com/owncloud/ocis/notifications/pkg/config" + "github.com/owncloud/ocis/ocis-pkg/log" +) + +// LoggerFromConfig initializes a service-specific logger instance. +func Configure(name string, cfg *config.Log) log.Logger { + return log.NewLogger( + log.Name(name), + log.Level(cfg.Level), + log.Pretty(cfg.Pretty), + log.Color(cfg.Color), + log.File(cfg.File), + ) +} diff --git a/notifications/pkg/service/service.go b/notifications/pkg/service/service.go new file mode 100644 index 000000000..07a46f083 --- /dev/null +++ b/notifications/pkg/service/service.go @@ -0,0 +1,46 @@ +package service + +import ( + "fmt" + "os" + "os/signal" + "syscall" + + "github.com/owncloud/ocis/ocis-pkg/log" +) + +type Service interface { + Run() error +} + +func NewEventsNotifier(events <-chan interface{}, logger log.Logger) Service { + return eventsNotifier{ + logger: logger, + events: events, + signals: make(chan os.Signal, 1), + } +} + +type eventsNotifier struct { + logger log.Logger + events <-chan interface{} + signals chan os.Signal +} + +func (s eventsNotifier) Run() error { + signal.Notify(s.signals, syscall.SIGINT, syscall.SIGTERM) + s.logger.Debug(). + Msg("eventsNotifier started") + for { + select { + case evt := <-s.events: + go func() { + fmt.Println(evt) + }() + case <-s.signals: + s.logger.Debug(). + Msg("eventsNotifier stopped") + return nil + } + } +} diff --git a/ocis-pkg/config/config.go b/ocis-pkg/config/config.go index 76aaeb35f..66707055a 100644 --- a/ocis-pkg/config/config.go +++ b/ocis-pkg/config/config.go @@ -9,6 +9,7 @@ import ( graph "github.com/owncloud/ocis/graph/pkg/config" idp "github.com/owncloud/ocis/idp/pkg/config" nats "github.com/owncloud/ocis/nats/pkg/config" + notifications "github.com/owncloud/ocis/notifications/pkg/config" ocs "github.com/owncloud/ocis/ocs/pkg/config" proxy "github.com/owncloud/ocis/proxy/pkg/config" settings "github.com/owncloud/ocis/settings/pkg/config" @@ -62,6 +63,7 @@ type Config struct { GraphExplorer *graphExplorer.Config `ocisConfig:"graph_explorer"` IDP *idp.Config `ocisConfig:"idp"` Nats *nats.Config `ocisConfig:"nats"` + Notifications *notifications.Config `ocisConfig:"notifications"` OCS *ocs.Config `ocisConfig:"ocs"` Web *web.Config `ocisConfig:"web"` Proxy *proxy.Config `ocisConfig:"proxy"` diff --git a/ocis-pkg/config/defaultconfig.go b/ocis-pkg/config/defaultconfig.go index 53853cbfa..60eea86a6 100644 --- a/ocis-pkg/config/defaultconfig.go +++ b/ocis-pkg/config/defaultconfig.go @@ -7,6 +7,7 @@ import ( graph "github.com/owncloud/ocis/graph/pkg/config" idp "github.com/owncloud/ocis/idp/pkg/config" nats "github.com/owncloud/ocis/nats/pkg/config" + notifications "github.com/owncloud/ocis/notifications/pkg/config" ocs "github.com/owncloud/ocis/ocs/pkg/config" proxy "github.com/owncloud/ocis/proxy/pkg/config" settings "github.com/owncloud/ocis/settings/pkg/config" @@ -31,6 +32,7 @@ func DefaultConfig() *Config { Graph: graph.DefaultConfig(), IDP: idp.DefaultConfig(), Nats: nats.DefaultConfig(), + Notifications: notifications.DefaultConfig(), Proxy: proxy.DefaultConfig(), GraphExplorer: graphExplorer.DefaultConfig(), OCS: ocs.DefaultConfig(), diff --git a/ocis/pkg/command/notifications.go b/ocis/pkg/command/notifications.go new file mode 100644 index 000000000..ec2eb3570 --- /dev/null +++ b/ocis/pkg/command/notifications.go @@ -0,0 +1,26 @@ +package command + +import ( + "github.com/owncloud/ocis/notifications/pkg/command" + "github.com/owncloud/ocis/ocis-pkg/config" + "github.com/owncloud/ocis/ocis-pkg/config/parser" + "github.com/owncloud/ocis/ocis/pkg/register" + "github.com/urfave/cli/v2" +) + +// NatsServerCommand is the entrypoint for the nats server command. +func NotificationsCommand(cfg *config.Config) *cli.Command { + return &cli.Command{ + Name: "notifications", + Usage: "start notifications service", + Category: "extensions", + Before: func(ctx *cli.Context) error { + return parser.ParseConfig(cfg) + }, + Subcommands: command.GetCommands(cfg.Notifications), + } +} + +func init() { + register.AddCommand(NotificationsCommand) +} diff --git a/ocis/pkg/runtime/service/service.go b/ocis/pkg/runtime/service/service.go index d160305ee..fa1a12908 100644 --- a/ocis/pkg/runtime/service/service.go +++ b/ocis/pkg/runtime/service/service.go @@ -24,6 +24,7 @@ import ( graph "github.com/owncloud/ocis/graph/pkg/command" idp "github.com/owncloud/ocis/idp/pkg/command" nats "github.com/owncloud/ocis/nats/pkg/command" + notifications "github.com/owncloud/ocis/notifications/pkg/command" "github.com/owncloud/ocis/ocis-pkg/config" ociscfg "github.com/owncloud/ocis/ocis-pkg/config" "github.com/owncloud/ocis/ocis-pkg/log" @@ -114,6 +115,7 @@ func NewService(options ...Option) (*Service, error) { s.ServicesRegistry["storage-shares"] = storage.NewStorageShares s.ServicesRegistry["storage-public-link"] = storage.NewStoragePublicLink s.ServicesRegistry["storage-appprovider"] = storage.NewAppProvider + s.ServicesRegistry["notifications"] = notifications.NewSutureService // populate delayed services s.Delayed["storage-sharing"] = storage.NewSharing From 666da5ea5a39b72f6e4eb804933505f0c7bd0626 Mon Sep 17 00:00:00 2001 From: David Christofas Date: Fri, 18 Feb 2022 16:36:18 +0100 Subject: [PATCH 07/24] implement first mail sender prototype --- notifications/pkg/channels/channels.go | 51 ++++++++++++++++++++++++++ notifications/pkg/command/server.go | 8 +++- notifications/pkg/service/service.go | 16 ++++++-- 3 files changed, 70 insertions(+), 5 deletions(-) create mode 100644 notifications/pkg/channels/channels.go diff --git a/notifications/pkg/channels/channels.go b/notifications/pkg/channels/channels.go new file mode 100644 index 000000000..335067950 --- /dev/null +++ b/notifications/pkg/channels/channels.go @@ -0,0 +1,51 @@ +// Package channels provides different communication channels to notify users. +package channels + +import ( + "context" + "net/smtp" + + gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1" + "github.com/cs3org/reva/pkg/rgrpc/todo/pool" + "github.com/owncloud/ocis/ocis-pkg/log" +) + +type Channel interface { + SendMessage(receiver, msg string) error +} + +func NewMailChanel(logger log.Logger) (Channel, error) { + gc, err := pool.GetGatewayServiceClient("localhost:9142") + if err != nil { + logger.Error().Err(err).Msg("could not get gateway client") + return nil, err + } + return Mail{ + gatewayClient: gc, + }, nil +} + +type Mail struct { + gatewayClient gateway.GatewayAPIClient +} + +func (m Mail) SendMessage(receiver, msg string) error { + res, err := m.gatewayClient.Authenticate(context.Background(), &gateway.AuthenticateRequest{ + Type: "machine", + ClientId: "userid:" + receiver, + ClientSecret: "change-me-please", + }) + if err != nil { + return err + } + + from := "god" + password := "godisdead" + to := []string{res.User.Mail} + host := "localhost" + port := "1025" + body := []byte(msg) + auth := smtp.PlainAuth("", from, password, host) + + return smtp.SendMail(host+":"+port, auth, from, to, body) +} diff --git a/notifications/pkg/command/server.go b/notifications/pkg/command/server.go index 48ba224aa..af1380591 100644 --- a/notifications/pkg/command/server.go +++ b/notifications/pkg/command/server.go @@ -6,6 +6,7 @@ import ( "github.com/asim/go-micro/plugins/events/nats/v4" "github.com/cs3org/reva/pkg/events" "github.com/cs3org/reva/pkg/events/server" + "github.com/owncloud/ocis/notifications/pkg/channels" "github.com/owncloud/ocis/notifications/pkg/config" "github.com/owncloud/ocis/notifications/pkg/config/parser" "github.com/owncloud/ocis/notifications/pkg/logging" @@ -39,8 +40,11 @@ func Server(cfg *config.Config) *cli.Command { if err != nil { return err } - - svc := service.NewEventsNotifier(evts, logger) + channel, err := channels.NewMailChanel(logger) + if err != nil { + return err + } + svc := service.NewEventsNotifier(evts, channel, logger) return svc.Run() }, } diff --git a/notifications/pkg/service/service.go b/notifications/pkg/service/service.go index 07a46f083..e220bb944 100644 --- a/notifications/pkg/service/service.go +++ b/notifications/pkg/service/service.go @@ -1,11 +1,12 @@ package service import ( - "fmt" "os" "os/signal" "syscall" + "github.com/cs3org/reva/pkg/events" + "github.com/owncloud/ocis/notifications/pkg/channels" "github.com/owncloud/ocis/ocis-pkg/log" ) @@ -13,9 +14,10 @@ type Service interface { Run() error } -func NewEventsNotifier(events <-chan interface{}, logger log.Logger) Service { +func NewEventsNotifier(events <-chan interface{}, channel channels.Channel, logger log.Logger) Service { return eventsNotifier{ logger: logger, + channel: channel, events: events, signals: make(chan os.Signal, 1), } @@ -23,6 +25,7 @@ func NewEventsNotifier(events <-chan interface{}, logger log.Logger) Service { type eventsNotifier struct { logger log.Logger + channel channels.Channel events <-chan interface{} signals chan os.Signal } @@ -35,7 +38,14 @@ func (s eventsNotifier) Run() error { select { case evt := <-s.events: go func() { - fmt.Println(evt) + switch e := evt.(type) { + case events.ShareCreated: + if err := s.channel.SendMessage(e.GranteeUserID.OpaqueId, "You got a share"); err != nil { + s.logger.Error(). + Err(err). + Msg("failed to send a message") + } + } }() case <-s.signals: s.logger.Debug(). From 2e8bc3e8e814d2b38d8f5407a7bd99839ffca466 Mon Sep 17 00:00:00 2001 From: David Christofas Date: Mon, 21 Feb 2022 14:59:16 +0100 Subject: [PATCH 08/24] make notifications service configurable --- notifications/pkg/channels/channels.go | 26 ++++++++++++++--------- notifications/pkg/command/server.go | 9 ++++---- notifications/pkg/config/config.go | 25 ++++++++++++++++++++++ notifications/pkg/config/defaultconfig.go | 15 +++++++++++++ 4 files changed, 60 insertions(+), 15 deletions(-) diff --git a/notifications/pkg/channels/channels.go b/notifications/pkg/channels/channels.go index 335067950..c78068ca2 100644 --- a/notifications/pkg/channels/channels.go +++ b/notifications/pkg/channels/channels.go @@ -7,45 +7,51 @@ import ( gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1" "github.com/cs3org/reva/pkg/rgrpc/todo/pool" + "github.com/owncloud/ocis/notifications/pkg/config" "github.com/owncloud/ocis/ocis-pkg/log" ) +// Channel defines the methods of a communication channel. type Channel interface { + // Todo(c0rby): Do we need a PrepareMessage method? + // Maybe channels need to format the message or will the caller + // of SendMessage do that? + // SendMessage sends a message in a channel specific way. SendMessage(receiver, msg string) error } -func NewMailChanel(logger log.Logger) (Channel, error) { - gc, err := pool.GetGatewayServiceClient("localhost:9142") +// NewMailChannel instantiates a new mail communication channel. +func NewMailChannel(cfg config.Config, logger log.Logger) (Channel, error) { + gc, err := pool.GetGatewayServiceClient(cfg.Notifications.RevaGateway) if err != nil { logger.Error().Err(err).Msg("could not get gateway client") return nil, err } return Mail{ gatewayClient: gc, + conf: cfg, }, nil } +// Mail is the communcation channel for email. type Mail struct { gatewayClient gateway.GatewayAPIClient + conf config.Config } func (m Mail) SendMessage(receiver, msg string) error { + smtpConf := m.conf.Notifications.SMTP res, err := m.gatewayClient.Authenticate(context.Background(), &gateway.AuthenticateRequest{ Type: "machine", ClientId: "userid:" + receiver, - ClientSecret: "change-me-please", + ClientSecret: m.conf.Notifications.MachineAuthSecret, }) if err != nil { return err } - from := "god" - password := "godisdead" to := []string{res.User.Mail} - host := "localhost" - port := "1025" body := []byte(msg) - auth := smtp.PlainAuth("", from, password, host) - - return smtp.SendMail(host+":"+port, auth, from, to, body) + auth := smtp.PlainAuth("", smtpConf.Sender, smtpConf.Password, smtpConf.Host) + return smtp.SendMail(smtpConf.Host+":"+smtpConf.Port, auth, smtpConf.Sender, to, body) } diff --git a/notifications/pkg/command/server.go b/notifications/pkg/command/server.go index af1380591..7f22fa1cb 100644 --- a/notifications/pkg/command/server.go +++ b/notifications/pkg/command/server.go @@ -26,21 +26,20 @@ func Server(cfg *config.Config) *cli.Command { Action: func(c *cli.Context) error { logger := logging.Configure(cfg.Service.Name, cfg.Log) - group := "notifications" - evs := []events.Unmarshaller{ events.ShareCreated{}, } - client, err := server.NewNatsStream(nats.Address("127.0.0.1:4222"), nats.ClusterID("test-cluster")) + evtsCfg := cfg.Notifications.Events + client, err := server.NewNatsStream(nats.Address(evtsCfg.Endpoint), nats.ClusterID(evtsCfg.Cluster)) if err != nil { return err } - evts, err := events.Consume(client, group, evs...) + evts, err := events.Consume(client, evtsCfg.ConsumerGroup, evs...) if err != nil { return err } - channel, err := channels.NewMailChanel(logger) + channel, err := channels.NewMailChannel(*cfg, logger) if err != nil { return err } diff --git a/notifications/pkg/config/config.go b/notifications/pkg/config/config.go index 254832cd1..8ac7da5d8 100644 --- a/notifications/pkg/config/config.go +++ b/notifications/pkg/config/config.go @@ -15,5 +15,30 @@ type Config struct { Log *Log `ocisConfig:"log"` Debug Debug `ocisConfig:"debug"` + Notifications Notifications `ocisConfig:"notifications"` + Context context.Context } + +// Notifications definces the config options for the notifications service. +type Notifications struct { + SMTP SMTP `ocisConfig:"SMTP"` + Events Events `ocisConfig:"events"` + RevaGateway string `ocisConfig:"reva_gateway" env:"REVA_GATEWAY;NOTIFICATIONS_REVA_GATEWAY"` + MachineAuthSecret string `ocisConfig:"machine_auth_api_key" env:"OCIS_MACHINE_AUTH_API_KEY;NOTIFICATIONS_MACHINE_AUTH_API_KEY"` +} + +// SMTP combines the smtp configuration options. +type SMTP struct { + Host string `ocisConfig:"smtp_host" env:"NOTIFICATIONS_SMTP_HOST"` + Port string `ocisConfig:"smtp_port" env:"NOTIFICATIONS_SMTP_PORT"` + Sender string `ocisConfig:"smtp_sender" env:"NOTIFICATIONS_SMTP_SENDER"` + Password string `ocisConfig:"smtp_password" env:"NOTIFICATIONS_SMTP_PASSWORD"` +} + +// Events combines the configuration options for the event bus. +type Events struct { + Endpoint string `ocisConfig:"events_endpoint" env:"NOTIFICATIONS_EVENTS_ENDPOINT"` + Cluster string `ocisConfig:"events_cluster" env:"NOTIFICATIONS_EVENTS_CLUSTER"` + ConsumerGroup string `ocisConfig:"events_group" env:"NOTIFICATIONS_EVENTS_GROUP"` +} diff --git a/notifications/pkg/config/defaultconfig.go b/notifications/pkg/config/defaultconfig.go index e98e72ff6..2d9169f7b 100644 --- a/notifications/pkg/config/defaultconfig.go +++ b/notifications/pkg/config/defaultconfig.go @@ -8,5 +8,20 @@ func DefaultConfig() *Config { Service: Service{ Name: "notifications", }, + Notifications: Notifications{ + SMTP: SMTP{ + Host: "127.0.0.1", + Port: "1025", + Sender: "god@example.com", + Password: "godisdead", + }, + Events: Events{ + Endpoint: "127.0.0.1:4222", + Cluster: "test-cluster", + ConsumerGroup: "notifications", + }, + RevaGateway: "127.0.0.1:9142", + MachineAuthSecret: "change-me-please", + }, } } From e1606159c82436449ad4cc1ffb0cc9126260c4fb Mon Sep 17 00:00:00 2001 From: David Christofas Date: Tue, 22 Feb 2022 13:24:24 +0100 Subject: [PATCH 09/24] add method to send message to groups --- changelog/unreleased/notifications-service.md | 5 ++ notifications/pkg/channels/channels.go | 78 +++++++++++++++---- notifications/pkg/service/service.go | 10 ++- 3 files changed, 77 insertions(+), 16 deletions(-) create mode 100644 changelog/unreleased/notifications-service.md diff --git a/changelog/unreleased/notifications-service.md b/changelog/unreleased/notifications-service.md new file mode 100644 index 000000000..a6bc56932 --- /dev/null +++ b/changelog/unreleased/notifications-service.md @@ -0,0 +1,5 @@ +Enhancement: Implement notifications service + +Implemented the minimal version of the notifications service to be able to notify a user when they received a share. + +https://github.com/owncloud/ocis/pull/3217 diff --git a/notifications/pkg/channels/channels.go b/notifications/pkg/channels/channels.go index c78068ca2..f4e427b01 100644 --- a/notifications/pkg/channels/channels.go +++ b/notifications/pkg/channels/channels.go @@ -6,18 +6,20 @@ import ( "net/smtp" gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1" + groups "github.com/cs3org/go-cs3apis/cs3/identity/group/v1beta1" + rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1" "github.com/cs3org/reva/pkg/rgrpc/todo/pool" "github.com/owncloud/ocis/notifications/pkg/config" "github.com/owncloud/ocis/ocis-pkg/log" + "github.com/pkg/errors" ) // Channel defines the methods of a communication channel. type Channel interface { - // Todo(c0rby): Do we need a PrepareMessage method? - // Maybe channels need to format the message or will the caller - // of SendMessage do that? - // SendMessage sends a message in a channel specific way. - SendMessage(receiver, msg string) error + // SendMessage sends a message to users. + SendMessage(userIDs []string, msg string) error + // SendMessageToGroup sends a message to a group. + SendMessageToGroup(groupdID *groups.GroupId, msg string) error } // NewMailChannel instantiates a new mail communication channel. @@ -30,6 +32,7 @@ func NewMailChannel(cfg config.Config, logger log.Logger) (Channel, error) { return Mail{ gatewayClient: gc, conf: cfg, + logger: logger, }, nil } @@ -37,21 +40,66 @@ func NewMailChannel(cfg config.Config, logger log.Logger) (Channel, error) { type Mail struct { gatewayClient gateway.GatewayAPIClient conf config.Config + logger log.Logger } -func (m Mail) SendMessage(receiver, msg string) error { - smtpConf := m.conf.Notifications.SMTP - res, err := m.gatewayClient.Authenticate(context.Background(), &gateway.AuthenticateRequest{ - Type: "machine", - ClientId: "userid:" + receiver, - ClientSecret: m.conf.Notifications.MachineAuthSecret, - }) +// SendMessage sends a message to all given users. +func (m Mail) SendMessage(userIDs []string, msg string) error { + to, err := m.getReceiverAddresses(userIDs) if err != nil { return err } - - to := []string{res.User.Mail} body := []byte(msg) + + smtpConf := m.conf.Notifications.SMTP auth := smtp.PlainAuth("", smtpConf.Sender, smtpConf.Password, smtpConf.Host) - return smtp.SendMail(smtpConf.Host+":"+smtpConf.Port, auth, smtpConf.Sender, to, body) + if err := smtp.SendMail(smtpConf.Host+":"+smtpConf.Port, auth, smtpConf.Sender, to, body); err != nil { + return errors.Wrap(err, "could not send mail") + } + return nil +} + +// SendMessageToGroup sends a message to all members of the given group. +func (m Mail) SendMessageToGroup(groupID *groups.GroupId, msg string) error { + // TODO We need an authenticated context here... + res, err := m.gatewayClient.GetGroup(context.Background(), &groups.GetGroupRequest{GroupId: groupID}) + if err != nil { + return err + } + if res.Status.Code != rpc.Code_CODE_OK { + return errors.New("could not get group") + } + + members := make([]string, 0, len(res.Group.Members)) + for _, id := range res.Group.Members { + members = append(members, id.OpaqueId) + } + + return m.SendMessage(members, msg) +} + +func (m Mail) getReceiverAddresses(receivers []string) ([]string, error) { + addresses := make([]string, 0, len(receivers)) + for _, id := range receivers { + // Authenticate is too costly but at the moment our only option to get the user. + // We don't have an authenticated context so calling `GetUser` doesn't work. + res, err := m.gatewayClient.Authenticate(context.Background(), &gateway.AuthenticateRequest{ + Type: "machine", + ClientId: "userid:" + id, + ClientSecret: m.conf.Notifications.MachineAuthSecret, + }) + if err != nil { + return nil, err + } + if res.Status.Code != rpc.Code_CODE_OK { + m.logger.Error(). + Interface("status", res.Status). + Str("receiver_id", id). + Msg("could not get user") + continue + } + addresses = append(addresses, res.User.Mail) + } + + return addresses, nil } diff --git a/notifications/pkg/service/service.go b/notifications/pkg/service/service.go index e220bb944..81e4a26b4 100644 --- a/notifications/pkg/service/service.go +++ b/notifications/pkg/service/service.go @@ -40,9 +40,17 @@ func (s eventsNotifier) Run() error { go func() { switch e := evt.(type) { case events.ShareCreated: - if err := s.channel.SendMessage(e.GranteeUserID.OpaqueId, "You got a share"); err != nil { + msg := "You got a share!" + var err error + if e.GranteeUserID != nil { + err = s.channel.SendMessage([]string{e.GranteeUserID.OpaqueId}, msg) + } else if e.GranteeGroupID != nil { + err = s.channel.SendMessageToGroup(e.GranteeGroupID, msg) + } + if err != nil { s.logger.Error(). Err(err). + Str("event", "ShareCreated"). Msg("failed to send a message") } } From caeb2b684def9775b2874b616170a58e5b24e7a2 Mon Sep 17 00:00:00 2001 From: David Christofas Date: Tue, 22 Feb 2022 16:59:13 +0100 Subject: [PATCH 10/24] support thumbnails in spaces --- changelog/unreleased/spaces-thumbnails.md | 5 +++ thumbnails/pkg/service/v0/service.go | 19 +++++++-- thumbnails/pkg/thumbnail/imgsource/cs3.go | 23 +++++++--- webdav/pkg/dav/requests/thumbnail.go | 19 +++++---- webdav/pkg/service/v0/service.go | 52 ++++++++++++++++++++++- 5 files changed, 99 insertions(+), 19 deletions(-) create mode 100644 changelog/unreleased/spaces-thumbnails.md diff --git a/changelog/unreleased/spaces-thumbnails.md b/changelog/unreleased/spaces-thumbnails.md new file mode 100644 index 000000000..7b6817eff --- /dev/null +++ b/changelog/unreleased/spaces-thumbnails.md @@ -0,0 +1,5 @@ +Enhancement: Thumbnails in spaces + +Added support for thumbnails in spaces. + +https://github.com/owncloud/ocis/pull/3219 diff --git a/thumbnails/pkg/service/v0/service.go b/thumbnails/pkg/service/v0/service.go index eb80ef973..b4978e3fb 100644 --- a/thumbnails/pkg/service/v0/service.go +++ b/thumbnails/pkg/service/v0/service.go @@ -217,11 +217,24 @@ func (g Thumbnail) handleWebdavSource(ctx context.Context, req *thumbnailssvc.Ge func (g Thumbnail) stat(path, auth string) (*provider.StatResponse, error) { ctx := metadata.AppendToOutgoingContext(context.Background(), revactx.TokenHeader, auth) - req := &provider.StatRequest{ - Ref: &provider.Reference{ + var ref *provider.Reference + if strings.Contains(path, "!") { + parts := strings.Split(path, "!") + spaceID, path := parts[0], parts[1] + ref = &provider.Reference{ + ResourceId: &provider.ResourceId{ + StorageId: spaceID, + OpaqueId: spaceID, + }, Path: path, - }, + } + } else { + ref = &provider.Reference{ + Path: path, + } } + + req := &provider.StatRequest{Ref: ref} rsp, err := g.cs3Client.Stat(ctx, req) if err != nil { g.logger.Error().Err(err).Str("path", path).Msg("could not stat file") diff --git a/thumbnails/pkg/thumbnail/imgsource/cs3.go b/thumbnails/pkg/thumbnail/imgsource/cs3.go index 9cf3db58a..27b37aa6e 100644 --- a/thumbnails/pkg/thumbnail/imgsource/cs3.go +++ b/thumbnails/pkg/thumbnail/imgsource/cs3.go @@ -6,6 +6,7 @@ import ( "fmt" "io" "net/http" + "strings" gateway "github.com/cs3org/go-cs3apis/cs3/gateway/v1beta1" rpc "github.com/cs3org/go-cs3apis/cs3/rpc/v1beta1" @@ -42,12 +43,24 @@ func (s CS3) Get(ctx context.Context, path string) (io.ReadCloser, error) { if !ok { return nil, errors.New("cs3source: authorization missing") } - ctx = metadata.AppendToOutgoingContext(context.Background(), revactx.TokenHeader, auth) - rsp, err := s.client.InitiateFileDownload(ctx, &provider.InitiateFileDownloadRequest{ - Ref: &provider.Reference{ + var ref *provider.Reference + if strings.Contains(path, "!") { + parts := strings.Split(path, "!") + spaceID, path := parts[0], parts[1] + ref = &provider.Reference{ + ResourceId: &provider.ResourceId{ + StorageId: spaceID, + OpaqueId: spaceID, + }, Path: path, - }, - }) + } + } else { + ref = &provider.Reference{ + Path: path, + } + } + ctx = metadata.AppendToOutgoingContext(context.Background(), revactx.TokenHeader, auth) + rsp, err := s.client.InitiateFileDownload(ctx, &provider.InitiateFileDownloadRequest{Ref: ref}) if err != nil { return nil, err diff --git a/webdav/pkg/dav/requests/thumbnail.go b/webdav/pkg/dav/requests/thumbnail.go index be1e7d43c..48b80ad48 100644 --- a/webdav/pkg/dav/requests/thumbnail.go +++ b/webdav/pkg/dav/requests/thumbnail.go @@ -33,13 +33,13 @@ type ThumbnailRequest struct { Height int32 // In case of a public share the public link token. PublicLinkToken string - // The username from the requested URL - Username string + // The Identifier from the requested URL + Identifier string } // ParseThumbnailRequest extracts all required parameters from a http request. func ParseThumbnailRequest(r *http.Request) (*ThumbnailRequest, error) { - fp, username, err := extractFilePath(r) + fp, id, err := extractFilePath(r) if err != nil { return nil, err } @@ -57,7 +57,7 @@ func ParseThumbnailRequest(r *http.Request) (*ThumbnailRequest, error) { Width: int32(width), Height: int32(height), PublicLinkToken: chi.URLParam(r, "token"), - Username: username, + Identifier: id, }, nil } @@ -68,16 +68,17 @@ func ParseThumbnailRequest(r *http.Request) (*ThumbnailRequest, error) { // User and filepath are dynamic and filepath can contain slashes // So using the URLParam function is not possible. func extractFilePath(r *http.Request) (string, string, error) { - user := chi.URLParam(r, "user") - user, err := url.QueryUnescape(user) + id := chi.URLParam(r, "id") + id, err := url.QueryUnescape(id) if err != nil { return "", "", errors.New("could not unescape user") } - if user != "" { - parts := strings.SplitN(r.URL.Path, user, 2) - return parts[1], user, nil + if id != "" { + parts := strings.SplitN(r.URL.Path, id, 2) + return parts[1], id, nil } + // This is for public links token := chi.URLParam(r, "token") if token != "" { parts := strings.SplitN(r.URL.Path, token, 2) diff --git a/webdav/pkg/service/v0/service.go b/webdav/pkg/service/v0/service.go index 2ab53bd48..a5562171f 100644 --- a/webdav/pkg/service/v0/service.go +++ b/webdav/pkg/service/v0/service.go @@ -66,7 +66,8 @@ func NewService(opts ...Option) (Service, error) { } m.Route(options.Config.HTTP.Root, func(r chi.Router) { - r.Get("/remote.php/dav/files/{user}/*", svc.Thumbnail) + r.Get("/remote.php/dav/spaces/{id}/*", svc.SpacesThumbnail) + r.Get("/remote.php/dav/files/{id}/*", svc.Thumbnail) r.Get("/remote.php/dav/public-files/{token}/*", svc.PublicThumbnail) r.Head("/remote.php/dav/public-files/{token}/*", svc.PublicThumbnailHead) }) @@ -88,6 +89,53 @@ func (g Webdav) ServeHTTP(w http.ResponseWriter, r *http.Request) { g.mux.ServeHTTP(w, r) } +// SpacesThumbnail is the endpoint for retrieving thumbnails inside of spaces. +func (g Webdav) SpacesThumbnail(w http.ResponseWriter, r *http.Request) { + tr, err := requests.ParseThumbnailRequest(r) + if err != nil { + g.log.Error().Err(err).Msg("could not create Request") + renderError(w, r, errBadRequest(err.Error())) + return + } + t := r.Header.Get(TokenHeader) + + fullPath := tr.Identifier + "!" + tr.Filepath + rsp, err := g.thumbnailsClient.GetThumbnail(r.Context(), &thumbnailssvc.GetThumbnailRequest{ + Filepath: strings.TrimLeft(tr.Filepath, "/"), + ThumbnailType: extensionToThumbnailType(strings.TrimLeft(tr.Extension, ".")), + Width: tr.Width, + Height: tr.Height, + Source: &thumbnailssvc.GetThumbnailRequest_Cs3Source{ + Cs3Source: &thumbnailsmsg.CS3Source{ + Path: fullPath, + Authorization: t, + }, + }, + }) + if err != nil { + e := merrors.Parse(err.Error()) + switch e.Code { + case http.StatusNotFound: + // StatusNotFound is expected for unsupported files + renderError(w, r, errNotFound(notFoundMsg(tr.Filename))) + return + case http.StatusBadRequest: + renderError(w, r, errBadRequest(err.Error())) + default: + renderError(w, r, errInternalError(err.Error())) + } + g.log.Error().Err(err).Msg("could not get thumbnail") + return + } + + if len(rsp.Thumbnail) == 0 { + renderError(w, r, errNotFound("")) + return + } + + g.mustRender(w, r, newThumbnailResponse(rsp)) +} + // Thumbnail implements the Service interface. func (g Webdav) Thumbnail(w http.ResponseWriter, r *http.Request) { tr, err := requests.ParseThumbnailRequest(r) @@ -101,7 +149,7 @@ func (g Webdav) Thumbnail(w http.ResponseWriter, r *http.Request) { ctx := metadata.AppendToOutgoingContext(r.Context(), TokenHeader, t) userRes, err := g.revaClient.GetUserByClaim(ctx, &userv1beta1.GetUserByClaimRequest{ Claim: "username", - Value: tr.Username, + Value: tr.Identifier, }) if err != nil || userRes.Status.Code != rpcv1beta1.Code_CODE_OK { g.log.Error().Err(err).Msg("could not get user") From e8702087f388153cb03ea33d17db475a8f34797c Mon Sep 17 00:00:00 2001 From: Florian Schade Date: Wed, 23 Feb 2022 18:46:25 +0100 Subject: [PATCH 11/24] allow requesting capabilities from password protected links --- .../fix-capabilities-for-public-links.md | 7 ++++ proxy/pkg/middleware/basic_auth.go | 22 +++++++++-- proxy/pkg/middleware/basic_auth_test.go | 37 +++++++++++++++++++ 3 files changed, 63 insertions(+), 3 deletions(-) create mode 100644 changelog/unreleased/fix-capabilities-for-public-links.md diff --git a/changelog/unreleased/fix-capabilities-for-public-links.md b/changelog/unreleased/fix-capabilities-for-public-links.md new file mode 100644 index 000000000..c1546a437 --- /dev/null +++ b/changelog/unreleased/fix-capabilities-for-public-links.md @@ -0,0 +1,7 @@ +Bugfix: Capabilities for password protected public links + +Allow password protected public links to request capabilities. + +https://github.com/owncloud/ocis/pull/3229 +https://github.com/owncloud/web/pull/6471 +https://github.com/owncloud/web/issues/5863 diff --git a/proxy/pkg/middleware/basic_auth.go b/proxy/pkg/middleware/basic_auth.go index 6b26dbd0c..64e467497 100644 --- a/proxy/pkg/middleware/basic_auth.go +++ b/proxy/pkg/middleware/basic_auth.go @@ -11,8 +11,6 @@ import ( "github.com/owncloud/ocis/proxy/pkg/webdav" ) -const publicFilesEndpoint = "/remote.php/dav/public-files/" - // BasicAuth provides a middleware to check if BasicAuth is provided func BasicAuth(optionSetters ...Option) func(next http.Handler) http.Handler { options := newOptions(optionSetters...) @@ -111,7 +109,25 @@ type basicAuth struct { func (m basicAuth) isPublicLink(req *http.Request) bool { login, _, ok := req.BasicAuth() - return ok && login == "public" && strings.HasPrefix(req.URL.Path, publicFilesEndpoint) + + if !ok || login != "public" { + return false + } + + publicPaths := []string{ + "/remote.php/dav/public-files/", + "/ocs/v1.php/cloud/capabilities", + } + isPublic := false + + for _, p := range publicPaths { + if strings.HasPrefix(req.URL.Path, p) { + isPublic = true + break + } + } + + return isPublic } // The token auth endpoint uses basic auth for clients, see https://openid.net/specs/openid-connect-basic-1_0.html#TokenRequest diff --git a/proxy/pkg/middleware/basic_auth_test.go b/proxy/pkg/middleware/basic_auth_test.go index fbdfab057..46c7f4828 100644 --- a/proxy/pkg/middleware/basic_auth_test.go +++ b/proxy/pkg/middleware/basic_auth_test.go @@ -1,3 +1,40 @@ package middleware +import ( + "net/http/httptest" + "testing" +) + /**/ + +func TestBasicAuth__isPublicLink(t *testing.T) { + tests := []struct { + url string + username string + expected bool + }{ + {url: "/remote.php/dav/public-files/", username: "", expected: false}, + {url: "/remote.php/dav/public-files/", username: "abc", expected: false}, + {url: "/remote.php/dav/public-files/", username: "private", expected: false}, + {url: "/remote.php/dav/public-files/", username: "public", expected: true}, + {url: "/ocs/v1.php/cloud/capabilities", username: "", expected: false}, + {url: "/ocs/v1.php/cloud/capabilities", username: "abc", expected: false}, + {url: "/ocs/v1.php/cloud/capabilities", username: "private", expected: false}, + {url: "/ocs/v1.php/cloud/capabilities", username: "public", expected: true}, + {url: "/ocs/v1.php/cloud/users/admin", username: "public", expected: false}, + } + ba := basicAuth{} + + for _, tt := range tests { + req := httptest.NewRequest("", tt.url, nil) + + if tt.username != "" { + req.SetBasicAuth(tt.username, "") + } + + result := ba.isPublicLink(req) + if result != tt.expected { + t.Errorf("with %s expected %t got %t", tt.url, tt.expected, result) + } + } +} From f519b4965e7ed64873aa090aa240504543978b84 Mon Sep 17 00:00:00 2001 From: Florian Schade Date: Wed, 23 Feb 2022 19:31:24 +0100 Subject: [PATCH 12/24] add spa fileserver to the web service (#3109) * add spa fileserver to the web service --- .../enhancement-web-cache-control.md | 6 ++ .../enhancement-web-spa-fileserver.md | 6 ++ storage/pkg/command/groups.go | 6 +- web/pkg/assets/server.go | 80 +++++++++++++++++++ web/pkg/service/v0/service.go | 69 +++------------- 5 files changed, 107 insertions(+), 60 deletions(-) create mode 100644 changelog/unreleased/enhancement-web-cache-control.md create mode 100644 changelog/unreleased/enhancement-web-spa-fileserver.md create mode 100644 web/pkg/assets/server.go diff --git a/changelog/unreleased/enhancement-web-cache-control.md b/changelog/unreleased/enhancement-web-cache-control.md new file mode 100644 index 000000000..ff64ff0b9 --- /dev/null +++ b/changelog/unreleased/enhancement-web-cache-control.md @@ -0,0 +1,6 @@ +Enhancement: Re-Enabling web cache control + +We've re-enable browser caching headers (`Expires` and `Last-Modified`) for the web service, this was disabled due to a problem in the fileserver used before. +Since we're now using our own fileserver implementation this works again and is enabled by default. + +https://github.com/owncloud/ocis/pull/3109 diff --git a/changelog/unreleased/enhancement-web-spa-fileserver.md b/changelog/unreleased/enhancement-web-spa-fileserver.md new file mode 100644 index 000000000..dad95f59e --- /dev/null +++ b/changelog/unreleased/enhancement-web-spa-fileserver.md @@ -0,0 +1,6 @@ +Enhancement: Add SPA conform fileserver for web + +We've added an SPA conform fileserver to the web service. +It enables web to use vue's history mode and behaves like nginx try_files. + +https://github.com/owncloud/ocis/pull/3109 diff --git a/storage/pkg/command/groups.go b/storage/pkg/command/groups.go index b71951431..57d1d3899 100644 --- a/storage/pkg/command/groups.go +++ b/storage/pkg/command/groups.go @@ -32,6 +32,7 @@ func Groups(cfg *config.Config) *cli.Command { tracing.Configure(cfg, logger) gr := run.Group{} ctx, cancel := context.WithCancel(context.Background()) + defer cancel() // pre-create folders if cfg.Reva.Groups.Driver == "json" && cfg.Reva.Groups.JSON != "" { @@ -40,9 +41,8 @@ func Groups(cfg *config.Config) *cli.Command { } } - uuid := uuid.Must(uuid.NewV4()) - pidFile := path.Join(os.TempDir(), "revad-"+c.Command.Name+"-"+uuid.String()+".pid") - defer cancel() + cuuid := uuid.Must(uuid.NewV4()) + pidFile := path.Join(os.TempDir(), "revad-"+c.Command.Name+"-"+cuuid.String()+".pid") rcfg := groupsConfigFromStruct(c, cfg) diff --git a/web/pkg/assets/server.go b/web/pkg/assets/server.go new file mode 100644 index 000000000..527134330 --- /dev/null +++ b/web/pkg/assets/server.go @@ -0,0 +1,80 @@ +package assets + +import ( + "bytes" + "golang.org/x/net/html" + "io" + "mime" + "net/http" + "path" + "path/filepath" +) + +type fileServer struct { + root http.FileSystem +} + +func FileServer(root http.FileSystem) http.Handler { + return &fileServer{root} +} + +func (f *fileServer) ServeHTTP(w http.ResponseWriter, r *http.Request) { + upath := path.Clean(path.Join("/", r.URL.Path)) + r.URL.Path = upath + + asset, err := f.root.Open(upath) + if err != nil { + r.URL.Path = "/index.html" + f.ServeHTTP(w, r) + return + } + defer asset.Close() + + s, _ := asset.Stat() + if s.IsDir() { + r.URL.Path = "/index.html" + f.ServeHTTP(w, r) + return + } + + w.Header().Set("Content-Type", mime.TypeByExtension(filepath.Ext(s.Name()))) + + buf := new(bytes.Buffer) + + switch s.Name() { + case "index.html", "oidc-callback.html", "oidc-silent-redirect.html": + _ = withBase(buf, asset, "/") + default: + _, _ = buf.ReadFrom(asset) + } + + _, _ = w.Write(buf.Bytes()) +} + +func withBase(w io.Writer, r io.Reader, base string) error { + doc, _ := html.Parse(r) + var parse func(*html.Node) + parse = func(n *html.Node) { + if n.Type == html.ElementNode && n.Data == "head" { + n.InsertBefore(&html.Node{ + Type: html.ElementNode, + Data: "base", + Attr: []html.Attribute{ + { + Key: "href", + Val: base, + }, + }, + }, n.FirstChild) + + return + } + + for c := n.FirstChild; c != nil; c = c.NextSibling { + parse(c) + } + } + parse(doc) + + return html.Render(w, doc) +} diff --git a/web/pkg/service/v0/service.go b/web/pkg/service/v0/service.go index 49b78d144..d7a92c3ad 100644 --- a/web/pkg/service/v0/service.go +++ b/web/pkg/service/v0/service.go @@ -2,10 +2,12 @@ package svc import ( "encoding/json" + "fmt" "io/ioutil" "net/http" "net/url" "os" + "strconv" "strings" "time" @@ -141,28 +143,19 @@ func (p Web) Static(ttl int) http.HandlerFunc { if !strings.HasSuffix(rootWithSlash, "/") { rootWithSlash = rootWithSlash + "/" } - assets := assets.New( - assets.Logger(p.logger), - assets.Config(p.config), - ) - - notFoundFunc := func(w http.ResponseWriter, r *http.Request) { - // TODO: replace the redirect with a not found page containing a link to the Web UI - http.Redirect(w, r, rootWithSlash, http.StatusTemporaryRedirect) - } static := http.StripPrefix( rootWithSlash, - interceptNotFound( - http.FileServer(assets), - notFoundFunc, + assets.FileServer( + assets.New( + assets.Logger(p.logger), + assets.Config(p.config), + ), ), ) - // TODO: investigate broken caching - https://github.com/owncloud/ocis/issues/1094 - // we don't have a last modification date of the static assets, so we use the service start date - //lastModified := time.Now().UTC().Format(http.TimeFormat) - //expires := time.Now().Add(time.Second * time.Duration(ttl)).UTC().Format(http.TimeFormat) + lastModified := time.Now().UTC().Format(http.TimeFormat) + expires := time.Now().Add(time.Second * time.Duration(ttl)).UTC().Format(http.TimeFormat) return func(w http.ResponseWriter, r *http.Request) { if rootWithSlash != "/" && r.URL.Path == p.config.HTTP.Root { @@ -175,49 +168,11 @@ func (p Web) Static(ttl int) http.HandlerFunc { return } - if r.URL.Path != rootWithSlash && strings.HasSuffix(r.URL.Path, "/") { - notFoundFunc(w, r) - return - } - - // TODO: investigate broken caching - https://github.com/owncloud/ocis/issues/1094 - //w.Header().Set("Cache-Control", fmt.Sprintf("max-age=%s, must-revalidate", strconv.Itoa(ttl))) - //w.Header().Set("Expires", expires) - //w.Header().Set("Last-Modified", lastModified) - w.Header().Set("Cache-Control", "no-cache, no-store, max-age=0, must-revalidate, value") - w.Header().Set("Expires", "Thu, 01 Jan 1970 00:00:00 GMT") - w.Header().Set("Last-Modified", time.Now().UTC().Format(http.TimeFormat)) + w.Header().Set("Cache-Control", fmt.Sprintf("max-age=%s, must-revalidate", strconv.Itoa(ttl))) + w.Header().Set("Expires", expires) + w.Header().Set("Last-Modified", lastModified) w.Header().Set("SameSite", "Strict") static.ServeHTTP(w, r) } } - -func interceptNotFound(h http.Handler, notFoundFunc func(http.ResponseWriter, *http.Request)) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { - notFoundInterceptor := &NotFoundInterceptor{ResponseWriter: w} - h.ServeHTTP(notFoundInterceptor, r) - if notFoundInterceptor.status == http.StatusNotFound { - notFoundFunc(w, r) - } - } -} - -type NotFoundInterceptor struct { - http.ResponseWriter - status int -} - -func (w *NotFoundInterceptor) WriteHeader(status int) { - w.status = status - if status != http.StatusNotFound { - w.ResponseWriter.WriteHeader(status) - } -} - -func (w *NotFoundInterceptor) Write(p []byte) (int, error) { - if w.status != http.StatusNotFound { - return w.ResponseWriter.Write(p) - } - return len(p), nil -} From c871762008ee73db054e5f0ebcbe23af4ff01821 Mon Sep 17 00:00:00 2001 From: Florian Schade Date: Wed, 23 Feb 2022 18:32:08 +0000 Subject: [PATCH 13/24] Automated changelog update [skip ci] --- CHANGELOG.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 752104145..d6a80ab72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ The following sections list the changes for unreleased. * Change - Unify file IDs: [#3185](https://github.com/owncloud/ocis/pull/3185) * Enhancement - Add sorting to list Spaces: [#3200](https://github.com/owncloud/ocis/issues/3200) * Enhancement - Change NATS port: [#3210](https://github.com/owncloud/ocis/pull/3210) +* Enhancement - Re-Enabling web cache control: [#3109](https://github.com/owncloud/ocis/pull/3109) +* Enhancement - Add SPA conform fileserver for web: [#3109](https://github.com/owncloud/ocis/pull/3109) * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) ## Details @@ -56,6 +58,21 @@ The following sections list the changes for unreleased. https://github.com/owncloud/ocis/pull/3210 +* Enhancement - Re-Enabling web cache control: [#3109](https://github.com/owncloud/ocis/pull/3109) + + We've re-enable browser caching headers (`Expires` and `Last-Modified`) for the web + service, this was disabled due to a problem in the fileserver used before. Since we're now using + our own fileserver implementation this works again and is enabled by default. + + https://github.com/owncloud/ocis/pull/3109 + +* Enhancement - Add SPA conform fileserver for web: [#3109](https://github.com/owncloud/ocis/pull/3109) + + We've added an SPA conform fileserver to the web service. It enables web to use vue's history + mode and behaves like nginx try_files. + + https://github.com/owncloud/ocis/pull/3109 + * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) Tags: web From 8098f1e3481c3c63b2bde6bc92633de343697b90 Mon Sep 17 00:00:00 2001 From: Prarup Gurung Date: Tue, 22 Feb 2022 16:19:06 +0545 Subject: [PATCH 14/24] Bump core commit id on ocis --- .drone.env | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.drone.env b/.drone.env index ef0543618..fad168e03 100644 --- a/.drone.env +++ b/.drone.env @@ -1,5 +1,5 @@ # The test runner source for API tests -CORE_COMMITID=a0a4064c77a3576763885f83d57ca4c558348b8c +CORE_COMMITID=bc0748fe0785c0a35564ba95b4ce2778f96052c3 CORE_BRANCH=master # The test runner source for UI tests From a59c031252f0d4673102396f9f0ccf04f1abc5c3 Mon Sep 17 00:00:00 2001 From: Prarup Gurung Date: Wed, 23 Feb 2022 12:03:04 +0545 Subject: [PATCH 15/24] Updated expected failures line numbers after the changes in core tests --- .../expected-failures-API-on-OCIS-storage.md | 146 +++++++++--------- 1 file changed, 73 insertions(+), 73 deletions(-) diff --git a/tests/acceptance/expected-failures-API-on-OCIS-storage.md b/tests/acceptance/expected-failures-API-on-OCIS-storage.md index 7d4a1ea32..2b2eb4055 100644 --- a/tests/acceptance/expected-failures-API-on-OCIS-storage.md +++ b/tests/acceptance/expected-failures-API-on-OCIS-storage.md @@ -5,10 +5,10 @@ The expected failures in this file are from features in the owncloud/core repo. Basic file management like up and download, move, copy, properties, trash, versions and chunking. #### [Getting information about a folder overwritten by a file gives 500 error instead of 404](https://github.com/owncloud/ocis/issues/1239) +- [apiWebdavProperties1/copyFile.feature:286](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L286) - [apiWebdavProperties1/copyFile.feature:287](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L287) -- [apiWebdavProperties1/copyFile.feature:288](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L288) +- [apiWebdavProperties1/copyFile.feature:309](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L309) - [apiWebdavProperties1/copyFile.feature:310](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L310) -- [apiWebdavProperties1/copyFile.feature:311](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L311) #### [Custom dav properties with namespaces are rendered incorrectly](https://github.com/owncloud/ocis/issues/2140) @@ -21,12 +21,12 @@ _ocdav: double check the webdav property parsing when custom namespaces are used - [apiWebdavProperties1/setFileProperties.feature:84](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/setFileProperties.feature#L84) #### [Cannot set custom webDav properties](https://github.com/owncloud/product/issues/264) -- [apiWebdavProperties2/getFileProperties.feature:348](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L348) -- [apiWebdavProperties2/getFileProperties.feature:353](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L353) -- [apiWebdavProperties2/getFileProperties.feature:358](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L358) -- [apiWebdavProperties2/getFileProperties.feature:389](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L389) -- [apiWebdavProperties2/getFileProperties.feature:394](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L394) -- [apiWebdavProperties2/getFileProperties.feature:399](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L399) +- [apiWebdavProperties2/getFileProperties.feature:360](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L360) +- [apiWebdavProperties2/getFileProperties.feature:365](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L365) +- [apiWebdavProperties2/getFileProperties.feature:370](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L370) +- [apiWebdavProperties2/getFileProperties.feature:401](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L401) +- [apiWebdavProperties2/getFileProperties.feature:406](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L406) +- [apiWebdavProperties2/getFileProperties.feature:411](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L411) #### [downloading an old version of a file returns 501](https://github.com/owncloud/ocis/issues/2261) - [apiVersions/fileVersions.feature:426](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiVersions/fileVersions.feature#L426) @@ -651,32 +651,32 @@ _getting and setting quota_ - [apiMain/quota.feature:269](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/quota.feature#L269) - [apiMain/quota.feature:283](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/quota.feature#L283) - [apiMain/quota.feature:289](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/quota.feature#L289) - Scenario Outline: Retrieving folder quota of shared folder with quota when no quota is set for recipient -- [apiWebdavProperties1/getQuota.feature:58](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L58) -- [apiWebdavProperties1/getQuota.feature:59](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L59) -- [apiWebdavProperties1/getQuota.feature:64](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L64) - Scenario Outline: Retrieving folder quota when quota is set and a file was uploaded -- [apiWebdavProperties1/getQuota.feature:77](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L77) -- [apiWebdavProperties1/getQuota.feature:78](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L78) -- [apiWebdavProperties1/getQuota.feature:83](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L83) - Scenario Outline: Retrieving folder quota when quota is set and a file was received -- [apiWebdavProperties1/getQuota.feature:98](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L98) -- [apiWebdavProperties1/getQuota.feature:99](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L99) -- [apiWebdavProperties1/getQuota.feature:104](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L104) Scenario Outline: Retrieving folder quota when no quota is set -- [apiWebdavProperties1/getQuota.feature:17](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L17) - [apiWebdavProperties1/getQuota.feature:18](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L18) -- [apiWebdavProperties1/getQuota.feature:23](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L23) +- [apiWebdavProperties1/getQuota.feature:19](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L19) +- [apiWebdavProperties1/getQuota.feature:24](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L24) Scenario Outline: Retrieving folder quota when quota is set -- [apiWebdavProperties1/getQuota.feature:32](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L32) -- [apiWebdavProperties1/getQuota.feature:33](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L33) -- [apiWebdavProperties1/getQuota.feature:38](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L38) +- [apiWebdavProperties1/getQuota.feature:34](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L34) +- [apiWebdavProperties1/getQuota.feature:35](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L35) +- [apiWebdavProperties1/getQuota.feature:40](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L40) + Scenario Outline: Retrieving folder quota of shared folder with quota when no quota is set for recipient +- [apiWebdavProperties1/getQuota.feature:61](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L61) +- [apiWebdavProperties1/getQuota.feature:62](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L62) +- [apiWebdavProperties1/getQuota.feature:67](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L67) + Scenario Outline: Retrieving folder quota when quota is set and a file was uploaded +- [apiWebdavProperties1/getQuota.feature:81](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L81) +- [apiWebdavProperties1/getQuota.feature:82](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L82) +- [apiWebdavProperties1/getQuota.feature:87](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L87) + Scenario Outline: Retrieving folder quota when quota is set and a file was received +- [apiWebdavProperties1/getQuota.feature:103](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L103) +- [apiWebdavProperties1/getQuota.feature:104](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L104) +- [apiWebdavProperties1/getQuota.feature:109](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/getQuota.feature#L109) #### [Private link support](https://github.com/owncloud/product/issues/201) #### [oc:privatelink property not returned in webdav responses](https://github.com/owncloud/product/issues/262) -- [apiWebdavProperties2/getFileProperties.feature:295](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L295) -- [apiWebdavProperties2/getFileProperties.feature:296](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L296) -- [apiWebdavProperties2/getFileProperties.feature:301](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L301) +- [apiWebdavProperties2/getFileProperties.feature:306](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L306) +- [apiWebdavProperties2/getFileProperties.feature:307](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L307) +- [apiWebdavProperties2/getFileProperties.feature:312](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L312) #### [changing user quota gives ocs status 103 / Cannot set quota](https://github.com/owncloud/product/issues/247) - [apiShareOperationsToShares2/uploadToShare.feature:193](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/uploadToShare.feature#L193) @@ -1197,24 +1197,24 @@ _ocdav: api compatibility, return correct status code_ - [apiWebdavOperations/search.feature:270](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L270) And other missing implementation of favorites -- [apiFavorites/favorites.feature:158](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L158) -- [apiFavorites/favorites.feature:159](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L159) -- [apiFavorites/favorites.feature:183](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L183) -- [apiFavorites/favorites.feature:184](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L184) -- [apiFavorites/favorites.feature:189](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L189) -- [apiFavorites/favorites.feature:216](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L216) -- [apiFavorites/favorites.feature:217](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L217) -- [apiFavorites/favorites.feature:222](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L222) -- [apiFavorites/favoritesSharingToShares.feature:77](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L77) -- [apiFavorites/favoritesSharingToShares.feature:78](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L78) +- [apiFavorites/favorites.feature:162](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L162) +- [apiFavorites/favorites.feature:163](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L163) +- [apiFavorites/favorites.feature:187](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L187) +- [apiFavorites/favorites.feature:188](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L188) +- [apiFavorites/favorites.feature:193](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L193) +- [apiFavorites/favorites.feature:220](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L220) +- [apiFavorites/favorites.feature:221](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L221) +- [apiFavorites/favorites.feature:226](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L226) +- [apiFavorites/favoritesSharingToShares.feature:82](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L82) +- [apiFavorites/favoritesSharingToShares.feature:88](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L88) #### [resource inside Shares dir is not found using the spaces WebDAV API](https://github.com/owncloud/ocis/issues/2968) -- [apiFavorites/favorites.feature:164](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L164) -- [apiFavorites/favoritesSharingToShares.feature:27](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L27) -- [apiFavorites/favoritesSharingToShares.feature:46](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L46) -- [apiFavorites/favoritesSharingToShares.feature:64](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L64) +- [apiFavorites/favorites.feature:168](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L168) +- [apiFavorites/favoritesSharingToShares.feature:28](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L28) +- [apiFavorites/favoritesSharingToShares.feature:48](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L48) +- [apiFavorites/favoritesSharingToShares.feature:67](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L67) - [apiFavorites/favoritesSharingToShares.feature:83](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L83) -- [apiFavorites/favoritesSharingToShares.feature:102](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L102) +- [apiFavorites/favoritesSharingToShares.feature:108](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favoritesSharingToShares.feature#L108) - [apiMain/checksums.feature:203](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/checksums.feature#L203) - [apiShareOperationsToShares2/getWebDAVSharePermissions.feature:49](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/getWebDAVSharePermissions.feature#L49) - [apiShareOperationsToShares2/getWebDAVSharePermissions.feature:75](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/getWebDAVSharePermissions.feature#L75) @@ -1239,18 +1239,18 @@ And other missing implementation of favorites - [apiShareOperationsToShares2/uploadToShare.feature:171](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/uploadToShare.feature#L171) - [apiShareOperationsToShares2/uploadToShare.feature:346](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/uploadToShare.feature#L346) - [apiShareOperationsToShares2/uploadToShare.feature:347](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/uploadToShare.feature#L347) -- [apiWebdavProperties1/copyFile.feature:90](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L90) -- [apiWebdavProperties1/copyFile.feature:117](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L117) -- [apiWebdavProperties1/copyFile.feature:293](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L293) -- [apiWebdavProperties1/copyFile.feature:316](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L316) -- [apiWebdavProperties1/copyFile.feature:344](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L344) -- [apiWebdavProperties1/copyFile.feature:374](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L374) -- [apiWebdavProperties1/copyFile.feature:403](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L403) -- [apiWebdavProperties1/copyFile.feature:432](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L432) -- [apiWebdavProperties1/copyFile.feature:516](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L516) -- [apiWebdavProperties1/copyFile.feature:549](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L549) -- [apiWebdavProperties1/copyFile.feature:581](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L581) -- [apiWebdavProperties1/copyFile.feature:613](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L613) +- [apiWebdavProperties1/copyFile.feature:89](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L89) +- [apiWebdavProperties1/copyFile.feature:116](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L116) +- [apiWebdavProperties1/copyFile.feature:292](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L292) +- [apiWebdavProperties1/copyFile.feature:315](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L315) +- [apiWebdavProperties1/copyFile.feature:343](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L343) +- [apiWebdavProperties1/copyFile.feature:373](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L373) +- [apiWebdavProperties1/copyFile.feature:402](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L402) +- [apiWebdavProperties1/copyFile.feature:431](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L431) +- [apiWebdavProperties1/copyFile.feature:515](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L515) +- [apiWebdavProperties1/copyFile.feature:548](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L548) +- [apiWebdavProperties1/copyFile.feature:580](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L580) +- [apiWebdavProperties1/copyFile.feature:612](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L612) - [apiWebdavProperties1/createFileFolderWhenSharesExist.feature:37](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFileFolderWhenSharesExist.feature#L37) - [apiWebdavProperties1/createFileFolderWhenSharesExist.feature:38](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFileFolderWhenSharesExist.feature#L38) - [apiWebdavProperties1/createFileFolderWhenSharesExist.feature:39](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFileFolderWhenSharesExist.feature#L39) @@ -1559,26 +1559,26 @@ And other missing implementation of favorites - [apiShareOperationsToShares2/shareAccessByID.feature:163](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareOperationsToShares2/shareAccessByID.feature#L163) #### [[OC-storage] share-types field empty for shared file folder in webdav response](https://github.com/owncloud/ocis/issues/2144) -- [apiWebdavProperties2/getFileProperties.feature:207](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L207) -- [apiWebdavProperties2/getFileProperties.feature:208](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L208) -- [apiWebdavProperties2/getFileProperties.feature:213](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L213) +- [apiWebdavProperties2/getFileProperties.feature:215](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L215) +- [apiWebdavProperties2/getFileProperties.feature:216](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L216) +- [apiWebdavProperties2/getFileProperties.feature:221](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L221) #### [Creating a public link with all permissions(31) fails](https://github.com/owncloud/ocis/issues/2145) -- [apiWebdavProperties2/getFileProperties.feature:265](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L265) -- [apiWebdavProperties2/getFileProperties.feature:266](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L266) -- [apiWebdavProperties2/getFileProperties.feature:271](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L271) +- [apiWebdavProperties2/getFileProperties.feature:275](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L275) +- [apiWebdavProperties2/getFileProperties.feature:276](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L276) +- [apiWebdavProperties2/getFileProperties.feature:281](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties2/getFileProperties.feature#L281) #### [Shares to deleted group listed in the response](https://github.com/owncloud/ocis/issues/2441) - [apiShareManagementBasicToShares/createShareToSharesFolder.feature:504](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L504) - [apiShareManagementBasicToShares/createShareToSharesFolder.feature:505](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiShareManagementBasicToShares/createShareToSharesFolder.feature#L505) #### [Trying to copy a file into a readonly share gives HTTP 500 error](https://github.com/owncloud/ocis/issues/2166) +- [apiWebdavProperties1/copyFile.feature:452](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L452) - [apiWebdavProperties1/copyFile.feature:453](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L453) -- [apiWebdavProperties1/copyFile.feature:454](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L454) -- [apiWebdavProperties1/copyFile.feature:459](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L459) +- [apiWebdavProperties1/copyFile.feature:458](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L458) +- [apiWebdavProperties1/copyFile.feature:478](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L478) - [apiWebdavProperties1/copyFile.feature:479](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L479) -- [apiWebdavProperties1/copyFile.feature:480](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L480) -- [apiWebdavProperties1/copyFile.feature:485](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L485) +- [apiWebdavProperties1/copyFile.feature:484](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L484) ### Won't fix Not everything needs to be implemented for ocis. While the oc10 testsuite covers these things we are not looking at them right now. @@ -1598,12 +1598,12 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers - [apiWebdavUpload1/uploadFileToExcludedDirectory.feature:87](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFileToExcludedDirectory.feature#L87) #### [Blacklist files extensions](https://github.com/owncloud/ocis/issues/2177) +- [apiWebdavProperties1/copyFile.feature:132](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L132) - [apiWebdavProperties1/copyFile.feature:133](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L133) -- [apiWebdavProperties1/copyFile.feature:134](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L134) -- [apiWebdavProperties1/copyFile.feature:139](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L139) -- [apiWebdavProperties1/createFolder.feature:92](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L92) -- [apiWebdavProperties1/createFolder.feature:93](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L93) -- [apiWebdavProperties1/createFolder.feature:98](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L98) +- [apiWebdavProperties1/copyFile.feature:138](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/copyFile.feature#L138) +- [apiWebdavProperties1/createFolder.feature:95](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L95) +- [apiWebdavProperties1/createFolder.feature:96](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L96) +- [apiWebdavProperties1/createFolder.feature:101](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavProperties1/createFolder.feature#L101) - [apiWebdavUpload1/uploadFile.feature:181](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFile.feature#L181) - [apiWebdavUpload1/uploadFile.feature:182](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFile.feature#L182) - [apiWebdavUpload1/uploadFile.feature:187](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavUpload1/uploadFile.feature#L187) @@ -1721,9 +1721,9 @@ Not everything needs to be implemented for ocis. While the oc10 testsuite covers - [apiWebdavMove2/moveFileToBlacklistedName.feature:80](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavMove2/moveFileToBlacklistedName.feature#L80) #### [REPORT method on spaces returns an incorrect d:href response](https://github.com/owncloud/ocis/issues/3111) -- [apiFavorites/favorites.feature:117](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L117) -- [apiFavorites/favorites.feature:143](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L143) -- [apiFavorites/favorites.feature:268](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L268) +- [apiFavorites/favorites.feature:121](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L121) +- [apiFavorites/favorites.feature:147](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L147) +- [apiFavorites/favorites.feature:273](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiFavorites/favorites.feature#L273) #### [could not create system tag](https://github.com/owncloud/ocis/issues/3092) - [apiWebdavOperations/search.feature:273](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiWebdavOperations/search.feature#L273) From 15f98223fa92b6ffced6b68c323fd16d99438c79 Mon Sep 17 00:00:00 2001 From: Swikriti Tripathi Date: Thu, 24 Feb 2022 11:04:09 +0545 Subject: [PATCH 16/24] Bump core commit id and update expected to fail --- .drone.env | 2 +- .../expected-failures-API-on-OCIS-storage.md | 14 ++++---------- 2 files changed, 5 insertions(+), 11 deletions(-) diff --git a/.drone.env b/.drone.env index fad168e03..40dac4820 100644 --- a/.drone.env +++ b/.drone.env @@ -1,5 +1,5 @@ # The test runner source for API tests -CORE_COMMITID=bc0748fe0785c0a35564ba95b4ce2778f96052c3 +CORE_COMMITID=87e97eb1a9b4795a72c5dcc94515b5b86b947274 CORE_BRANCH=master # The test runner source for UI tests diff --git a/tests/acceptance/expected-failures-API-on-OCIS-storage.md b/tests/acceptance/expected-failures-API-on-OCIS-storage.md index 2b2eb4055..5b69c0a08 100644 --- a/tests/acceptance/expected-failures-API-on-OCIS-storage.md +++ b/tests/acceptance/expected-failures-API-on-OCIS-storage.md @@ -538,12 +538,6 @@ cannot share a folder with create permission - [apiSharePublicLink2/copyFromPublicLink.feature:182](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/copyFromPublicLink.feature#L182) - [apiSharePublicLink2/updatePublicLinkShare.feature:45](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L45) - [apiSharePublicLink2/updatePublicLinkShare.feature:46](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L46) -- [apiSharePublicLink2/updatePublicLinkShare.feature:118](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L118) -- [apiSharePublicLink2/updatePublicLinkShare.feature:119](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L119) -- [apiSharePublicLink2/updatePublicLinkShare.feature:120](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L120) -- [apiSharePublicLink2/updatePublicLinkShare.feature:121](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L121) -- [apiSharePublicLink2/updatePublicLinkShare.feature:126](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L126) -- [apiSharePublicLink2/updatePublicLinkShare.feature:127](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L127) #### [OCIS share permissions not enforced](https://github.com/owncloud/product/issues/270) @@ -554,10 +548,10 @@ cannot share a folder with create permission #### [Adding public upload to a read only shared folder as a recipient is allowed ](https://github.com/owncloud/ocis/issues/2164) -- [apiSharePublicLink2/updatePublicLinkShare.feature:342](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L342) -- [apiSharePublicLink2/updatePublicLinkShare.feature:343](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L343) -- [apiSharePublicLink2/updatePublicLinkShare.feature:402](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L402) -- [apiSharePublicLink2/updatePublicLinkShare.feature:403](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L403) +- [apiSharePublicLink2/updatePublicLinkShare.feature:338](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L338) +- [apiSharePublicLink2/updatePublicLinkShare.feature:339](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L339) +- [apiSharePublicLink2/updatePublicLinkShare.feature:398](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L398) +- [apiSharePublicLink2/updatePublicLinkShare.feature:399](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiSharePublicLink2/updatePublicLinkShare.feature#L399) #### [Upload-only shares must not overwrite but create a separate file](https://github.com/owncloud/ocis-reva/issues/286) From 4bc85c969b7cfe7aa9a1acca2d34e2cfc6b516ec Mon Sep 17 00:00:00 2001 From: David Christofas Date: Thu, 24 Feb 2022 08:44:07 +0000 Subject: [PATCH 17/24] Automated changelog update [skip ci] --- CHANGELOG.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index d6a80ab72..37e6df89b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ The following sections list the changes for unreleased. ## Summary +* Bugfix - Capabilities for password protected public links: [#3229](https://github.com/owncloud/ocis/pull/3229) * Bugfix - Make events settings configurable: [#3214](https://github.com/owncloud/ocis/pull/3214) * Bugfix - Align storage metadata GPRC bind port with other variable names: [#3169](https://github.com/owncloud/ocis/pull/3169) * Change - Unify file IDs: [#3185](https://github.com/owncloud/ocis/pull/3185) @@ -17,6 +18,14 @@ The following sections list the changes for unreleased. ## Details +* Bugfix - Capabilities for password protected public links: [#3229](https://github.com/owncloud/ocis/pull/3229) + + Allow password protected public links to request capabilities. + + https://github.com/owncloud/web/issues/5863 + https://github.com/owncloud/ocis/pull/3229 + https://github.com/owncloud/web/pull/6471 + * Bugfix - Make events settings configurable: [#3214](https://github.com/owncloud/ocis/pull/3214) We've fixed the hardcoded events settings to be configurable. From 5dcc0e3b6cbab10af445a9f5ac078fb3c76829ba Mon Sep 17 00:00:00 2001 From: David Christofas Date: Thu, 24 Feb 2022 08:54:23 +0000 Subject: [PATCH 18/24] Automated changelog update [skip ci] --- CHANGELOG.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 37e6df89b..fdf7ee331 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ The following sections list the changes for unreleased. * Enhancement - Change NATS port: [#3210](https://github.com/owncloud/ocis/pull/3210) * Enhancement - Re-Enabling web cache control: [#3109](https://github.com/owncloud/ocis/pull/3109) * Enhancement - Add SPA conform fileserver for web: [#3109](https://github.com/owncloud/ocis/pull/3109) +* Enhancement - Implement notifications service: [#3217](https://github.com/owncloud/ocis/pull/3217) * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) ## Details @@ -82,6 +83,13 @@ The following sections list the changes for unreleased. https://github.com/owncloud/ocis/pull/3109 +* Enhancement - Implement notifications service: [#3217](https://github.com/owncloud/ocis/pull/3217) + + Implemented the minimal version of the notifications service to be able to notify a user when + they received a share. + + https://github.com/owncloud/ocis/pull/3217 + * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) Tags: web From 0df3fdddcc4f44186930e2517b316795194a3789 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rn=20Friedrich=20Dreyer?= Date: Thu, 24 Feb 2022 09:55:04 +0000 Subject: [PATCH 19/24] Automated changelog update [skip ci] --- CHANGELOG.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index fdf7ee331..6bb281314 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ The following sections list the changes for unreleased. * Enhancement - Re-Enabling web cache control: [#3109](https://github.com/owncloud/ocis/pull/3109) * Enhancement - Add SPA conform fileserver for web: [#3109](https://github.com/owncloud/ocis/pull/3109) * Enhancement - Implement notifications service: [#3217](https://github.com/owncloud/ocis/pull/3217) +* Enhancement - Thumbnails in spaces: [#3219](https://github.com/owncloud/ocis/pull/3219) * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) ## Details @@ -90,6 +91,12 @@ The following sections list the changes for unreleased. https://github.com/owncloud/ocis/pull/3217 +* Enhancement - Thumbnails in spaces: [#3219](https://github.com/owncloud/ocis/pull/3219) + + Added support for thumbnails in spaces. + + https://github.com/owncloud/ocis/pull/3219 + * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) Tags: web From 7d511eabb94c683d12da9b7c0a9c3431d318e891 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=B6rn=20Friedrich=20Dreyer?= Date: Thu, 24 Feb 2022 12:44:18 +0100 Subject: [PATCH 20/24] Apply suggestions from code review Co-authored-by: Michael Barz --- .../0017-allow-read-only-external-user-management.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/ocis/adr/0017-allow-read-only-external-user-management.md b/docs/ocis/adr/0017-allow-read-only-external-user-management.md index a6c2c719a..879171b37 100644 --- a/docs/ocis/adr/0017-allow-read-only-external-user-management.md +++ b/docs/ocis/adr/0017-allow-read-only-external-user-management.md @@ -14,12 +14,12 @@ geekdocFilePath: 0017-allow-read-only-external-user-management.md ## Context and Problem Statement oCIS needs to be integrated with various external Authentication and Identity Management Systems. -Usually oCIS will no have administrative access to such a system and we will not be allowed to -reconfigure it suite our needs (e.g. we will not be able to enhance the schema of an already existing -LDAP Directory). In most case our access will be read-only. +Usually oCIS will have no administrative access to such a system and we will not be allowed to +reconfigure it to suit our needs (e.g. we will not be able to enhance the schema of an already existing +LDAP Directory). In most of the cases our access will be read-only. Sidenote: There is a difference between users, identities and accounts: A user may have multiple -identities whith which he can authenticate, e.g. his facebook, twitter, microsoft or google +identities which he can authenticate with, e.g. his facebook, twitter, microsoft or google identity. Multiple identities can be linked to an account in ocis, allowing to fall back to another identity provider should one of them shut down. This also allows migrating from one identity provider to another. @@ -41,7 +41,7 @@ of stable identifier for users: - In OIDC itself the only stable identifier that is guaranteed to be provided by the IDP is combination of the sub and iss claims. IDPs can optionally return other claims, but we cannot rely on a specific claim being present. -- When no other services (LDAP, SCIM, ...) is available that could be used look up a user UUID +- When no other services (LDAP, SCIM, ...) are available that could be used look up a user UUID ## Decision Drivers @@ -83,7 +83,7 @@ the APIs provided by the IdM. * Good, because we can rely on the external identity management * Good, because ocis services only need to know about the CS3 user provider API, which acts as an abstraction layer for different identitiy management systems -* Good, because there is only an single source of truth (the external IdM) and we don't need to +* Good, because there is only a single source of truth (the external IdM) and we don't need to implement a synchronization mechanism to maintain an internal user database (we will likely need some form of caching though, see below) * Bad, because the identity managment needs to provide a stable, persistent, non-reassignable user From e0a09ccbd76dacbd20b8d83336b8fc3bc693b6df Mon Sep 17 00:00:00 2001 From: Michael Barz Date: Thu, 24 Feb 2022 12:50:31 +0100 Subject: [PATCH 21/24] update reva --- changelog/unreleased/update-reva.md | 5 +++++ go.mod | 2 +- go.sum | 2 ++ 3 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 changelog/unreleased/update-reva.md diff --git a/changelog/unreleased/update-reva.md b/changelog/unreleased/update-reva.md new file mode 100644 index 000000000..18612669c --- /dev/null +++ b/changelog/unreleased/update-reva.md @@ -0,0 +1,5 @@ +Enhancement: Update reva to v1.16.1-0.20220224114254-d4ffaf323e70 + +We updated reva to the latest edge branch. + +https://github.com/owncloud/ocis/pull/3231 diff --git a/go.mod b/go.mod index 7b9425c09..29110efad 100644 --- a/go.mod +++ b/go.mod @@ -21,7 +21,7 @@ require ( github.com/blevesearch/bleve/v2 v2.3.0 github.com/coreos/go-oidc/v3 v3.1.0 github.com/cs3org/go-cs3apis v0.0.0-20220126114148-64c025ccdd19 - github.com/cs3org/reva v1.16.1-0.20220221135950-8cc813ecfc4c + github.com/cs3org/reva v1.16.1-0.20220224114254-d4ffaf323e70 github.com/disintegration/imaging v1.6.2 github.com/glauth/glauth/v2 v2.0.0-20211021011345-ef3151c28733 github.com/go-chi/chi/v5 v5.0.7 diff --git a/go.sum b/go.sum index 4643f0bee..e5078b9c6 100644 --- a/go.sum +++ b/go.sum @@ -341,6 +341,8 @@ github.com/cs3org/go-cs3apis v0.0.0-20220126114148-64c025ccdd19 h1:1jqPH58jCxvba github.com/cs3org/go-cs3apis v0.0.0-20220126114148-64c025ccdd19/go.mod h1:UXha4TguuB52H14EMoSsCqDj7k8a/t7g4gVP+bgY5LY= github.com/cs3org/reva v1.16.1-0.20220221135950-8cc813ecfc4c h1:52qF91Pu/jUVVWgZo4kl2GaQT5SPCdV0b5s6SQG2jrM= github.com/cs3org/reva v1.16.1-0.20220221135950-8cc813ecfc4c/go.mod h1:M+Nausi93NaMDoinwiUX/Ekgrp/RrYv715FG5KrIarY= +github.com/cs3org/reva v1.16.1-0.20220224114254-d4ffaf323e70 h1:uL5Tm7N1nJiKh0hJKX33fGl64nJw5xXp0xguyPHoICQ= +github.com/cs3org/reva v1.16.1-0.20220224114254-d4ffaf323e70/go.mod h1:M+Nausi93NaMDoinwiUX/Ekgrp/RrYv715FG5KrIarY= github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8 h1:Z9lwXumT5ACSmJ7WGnFl+OMLLjpz5uR2fyz7dC255FI= github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8/go.mod h1:4abs/jPXcmJzYoYGF91JF9Uq9s/KL5n1jvFDix8KcqY= github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4= From f210ac47a8cee43257a16871ea2ba112c761df09 Mon Sep 17 00:00:00 2001 From: Michael Barz Date: Thu, 24 Feb 2022 11:51:17 +0000 Subject: [PATCH 22/24] Automated changelog update [skip ci] --- CHANGELOG.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6bb281314..cd4875db0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ The following sections list the changes for unreleased. * Enhancement - Add SPA conform fileserver for web: [#3109](https://github.com/owncloud/ocis/pull/3109) * Enhancement - Implement notifications service: [#3217](https://github.com/owncloud/ocis/pull/3217) * Enhancement - Thumbnails in spaces: [#3219](https://github.com/owncloud/ocis/pull/3219) +* Enhancement - Update reva to v1.16.1-0.20220224114254-d4ffaf323e70: [#3231](https://github.com/owncloud/ocis/pull/3231) * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) ## Details @@ -97,6 +98,12 @@ The following sections list the changes for unreleased. https://github.com/owncloud/ocis/pull/3219 +* Enhancement - Update reva to v1.16.1-0.20220224114254-d4ffaf323e70: [#3231](https://github.com/owncloud/ocis/pull/3231) + + We updated reva to the latest edge branch. + + https://github.com/owncloud/ocis/pull/3231 + * Enhancement - Update ownCloud Web to v5.1.0: [#3202](https://github.com/owncloud/ocis/pull/3202) Tags: web From 6956d0f8715acee07c6aaf6119a2d145b4b9089c Mon Sep 17 00:00:00 2001 From: Michael Barz Date: Thu, 24 Feb 2022 15:03:02 +0100 Subject: [PATCH 23/24] [full-ci] update reva (#3231) * update reva * adjust expected failures --- changelog/unreleased/update-reva.md | 2 +- go.mod | 6 +++--- go.sum | 6 ++---- tests/acceptance/expected-failures-API-on-OCIS-storage.md | 4 ---- tests/acceptance/expected-failures-webUI-on-OCIS-storage.md | 3 --- 5 files changed, 6 insertions(+), 15 deletions(-) diff --git a/changelog/unreleased/update-reva.md b/changelog/unreleased/update-reva.md index 18612669c..2e5e87224 100644 --- a/changelog/unreleased/update-reva.md +++ b/changelog/unreleased/update-reva.md @@ -1,5 +1,5 @@ Enhancement: Update reva to v1.16.1-0.20220224114254-d4ffaf323e70 -We updated reva to the latest edge branch. +We updated reva to the latest edge branch. Changelog will follow. https://github.com/owncloud/ocis/pull/3231 diff --git a/go.mod b/go.mod index 29110efad..49f837a59 100644 --- a/go.mod +++ b/go.mod @@ -7,6 +7,7 @@ require ( github.com/GeertJohan/yubigo v0.0.0-20190917122436-175bc097e60e github.com/ReneKroon/ttlcache/v2 v2.11.0 github.com/asim/go-micro/plugins/client/grpc/v4 v4.0.0-20220118152736-9e0be6c85d75 + github.com/asim/go-micro/plugins/events/nats/v4 v4.0.0-20220118152736-9e0be6c85d75 github.com/asim/go-micro/plugins/logger/zerolog/v4 v4.0.0-20220118152736-9e0be6c85d75 github.com/asim/go-micro/plugins/registry/consul/v4 v4.0.0-20220118152736-9e0be6c85d75 github.com/asim/go-micro/plugins/registry/etcd/v4 v4.0.0-20220118152736-9e0be6c85d75 @@ -21,7 +22,7 @@ require ( github.com/blevesearch/bleve/v2 v2.3.0 github.com/coreos/go-oidc/v3 v3.1.0 github.com/cs3org/go-cs3apis v0.0.0-20220126114148-64c025ccdd19 - github.com/cs3org/reva v1.16.1-0.20220224114254-d4ffaf323e70 + github.com/cs3org/reva v1.16.1-0.20220224115936-e17c65a48a5a github.com/disintegration/imaging v1.6.2 github.com/glauth/glauth/v2 v2.0.0-20211021011345-ef3151c28733 github.com/go-chi/chi/v5 v5.0.7 @@ -69,6 +70,7 @@ require ( go.opentelemetry.io/otel/trace v1.4.1 golang.org/x/crypto v0.0.0-20220112180741-5e0467b6c7ce golang.org/x/image v0.0.0-20211028202545-6944b10bf410 + golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8 google.golang.org/genproto v0.0.0-20220118154757-00ab72f36ad5 google.golang.org/grpc v1.44.0 @@ -91,7 +93,6 @@ require ( github.com/acomagu/bufpipe v1.0.3 // indirect github.com/armon/go-metrics v0.3.10 // indirect github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect - github.com/asim/go-micro/plugins/events/nats/v4 v4.0.0-20220118152736-9e0be6c85d75 // indirect github.com/aws/aws-sdk-go v1.42.39 // indirect github.com/beevik/etree v1.1.0 // indirect github.com/beorn7/perks v1.0.1 // indirect @@ -249,7 +250,6 @@ require ( go.uber.org/multierr v1.7.0 // indirect go.uber.org/zap v1.19.1 // indirect golang.org/x/mod v0.5.1 // indirect - golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd // indirect golang.org/x/sync v0.0.0-20210220032951-036812b2e83c // indirect golang.org/x/sys v0.0.0-20220114195835-da31bd327af9 // indirect golang.org/x/text v0.3.7 // indirect diff --git a/go.sum b/go.sum index e5078b9c6..4557a5b88 100644 --- a/go.sum +++ b/go.sum @@ -339,10 +339,8 @@ github.com/crewjam/saml v0.4.5/go.mod h1:qCJQpUtZte9R1ZjUBcW8qtCNlinbO363ooNl02S github.com/cs3org/cato v0.0.0-20200828125504-e418fc54dd5e/go.mod h1:XJEZ3/EQuI3BXTp/6DUzFr850vlxq11I6satRtz0YQ4= github.com/cs3org/go-cs3apis v0.0.0-20220126114148-64c025ccdd19 h1:1jqPH58jCxvbaJ9WLIJ7W2/m622bWS6ChptzljSG6IQ= github.com/cs3org/go-cs3apis v0.0.0-20220126114148-64c025ccdd19/go.mod h1:UXha4TguuB52H14EMoSsCqDj7k8a/t7g4gVP+bgY5LY= -github.com/cs3org/reva v1.16.1-0.20220221135950-8cc813ecfc4c h1:52qF91Pu/jUVVWgZo4kl2GaQT5SPCdV0b5s6SQG2jrM= -github.com/cs3org/reva v1.16.1-0.20220221135950-8cc813ecfc4c/go.mod h1:M+Nausi93NaMDoinwiUX/Ekgrp/RrYv715FG5KrIarY= -github.com/cs3org/reva v1.16.1-0.20220224114254-d4ffaf323e70 h1:uL5Tm7N1nJiKh0hJKX33fGl64nJw5xXp0xguyPHoICQ= -github.com/cs3org/reva v1.16.1-0.20220224114254-d4ffaf323e70/go.mod h1:M+Nausi93NaMDoinwiUX/Ekgrp/RrYv715FG5KrIarY= +github.com/cs3org/reva v1.16.1-0.20220224115936-e17c65a48a5a h1:LgwfJ4N+ekLRFOK2vwVKbgtnQYYcrNiMtu5kekJvJ54= +github.com/cs3org/reva v1.16.1-0.20220224115936-e17c65a48a5a/go.mod h1:M+Nausi93NaMDoinwiUX/Ekgrp/RrYv715FG5KrIarY= github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8 h1:Z9lwXumT5ACSmJ7WGnFl+OMLLjpz5uR2fyz7dC255FI= github.com/cubewise-code/go-mime v0.0.0-20200519001935-8c5762b177d8/go.mod h1:4abs/jPXcmJzYoYGF91JF9Uq9s/KL5n1jvFDix8KcqY= github.com/cyberdelia/templates v0.0.0-20141128023046-ca7fffd4298c/go.mod h1:GyV+0YP4qX0UQ7r2MoYZ+AvYDp12OF5yg4q8rGnyNh4= diff --git a/tests/acceptance/expected-failures-API-on-OCIS-storage.md b/tests/acceptance/expected-failures-API-on-OCIS-storage.md index 5b69c0a08..180a071e7 100644 --- a/tests/acceptance/expected-failures-API-on-OCIS-storage.md +++ b/tests/acceptance/expected-failures-API-on-OCIS-storage.md @@ -1098,10 +1098,6 @@ API, search, favorites, config, capabilities, not existing endpoints, CORS and o #### [no robots.txt available](https://github.com/owncloud/ocis/issues/1314) - [apiMain/main.feature:5](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/main.feature#L5) Scenario: robots.txt file should be accessible -#### [Different version, edition and productname in status request and capabilities request](https://github.com/owncloud/ocis/issues/2174) -- [apiMain/status.feature:5](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiMain/status.feature#L5) Scenario: Status.php is correct -#### [ocs config endpoint only accessible by authorized users](https://github.com/owncloud/ocis/issues/1338) - #### [Ability to return error messages in Webdav response bodies](https://github.com/owncloud/ocis/issues/1293) - [apiAuthOcs/ocsDELETEAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsDELETEAuth.feature#L10) Scenario: send DELETE requests to OCS endpoints as admin with wrong password - [apiAuthOcs/ocsGETAuth.feature:10](https://github.com/owncloud/core/blob/master/tests/acceptance/features/apiAuthOcs/ocsGETAuth.feature#L10) Scenario: using OCS anonymously diff --git a/tests/acceptance/expected-failures-webUI-on-OCIS-storage.md b/tests/acceptance/expected-failures-webUI-on-OCIS-storage.md index cb8afc016..0bcc91b84 100644 --- a/tests/acceptance/expected-failures-webUI-on-OCIS-storage.md +++ b/tests/acceptance/expected-failures-webUI-on-OCIS-storage.md @@ -491,9 +491,6 @@ Other free text and markdown formatting can be used elsewhere in the document if - [webUISharingInternalGroups/shareWithGroups.feature:77](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalGroups/shareWithGroups.feature#L77) - [webUISharingInternalUsers/shareWithUsers.feature:57](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingInternalUsers/shareWithUsers.feature#L57) -### [shares are not listed with full paths](https://github.com/owncloud/ocis/issues/2462) -- [webUISharingPublicBasic/publicLinkCreate.feature:88](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUISharingPublicBasic/publicLinkCreate.feature#L88) - ### [web config update is not properly reflected after the ocis start](https://github.com/owncloud/ocis/issues/2944) - [webUIFiles/breadcrumb.feature:50](https://github.com/owncloud/web/blob/master/tests/acceptance/features/webUIFiles/breadcrumb.feature#L50) From 83282b8144fe12f3f81a0700bf14fbe7ad41646c Mon Sep 17 00:00:00 2001 From: Michael Barz Date: Thu, 24 Feb 2022 14:03:31 +0000 Subject: [PATCH 24/24] Automated changelog update [skip ci] --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cd4875db0..b9bd9aa61 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -100,7 +100,7 @@ The following sections list the changes for unreleased. * Enhancement - Update reva to v1.16.1-0.20220224114254-d4ffaf323e70: [#3231](https://github.com/owncloud/ocis/pull/3231) - We updated reva to the latest edge branch. + We updated reva to the latest edge branch. Changelog will follow. https://github.com/owncloud/ocis/pull/3231