Commit Graph
86 Commits
Author SHA1 Message Date
Andre Duffeck 1aeffb08b7 Automated changelog update [skip ci] 2023-12-20 08:33:38 +00:00
Andre DuffeckandGitHub 2284fe388d Also log the traceid alongside the request id (#8023)
* Also log the traceid alongside the request id

* Add changelog
2023-12-20 09:32:57 +01:00
Andre Duffeck 9b0e88f311 Automated changelog update [skip ci] 2023-12-19 15:05:12 +00:00
Andre DuffeckandGitHub fe20ea27e2 Merge pull request #8017 from aduffeck/fix-empty-traceids
Use a on-sampling instead of noop traceprovider to get proper traceids
2023-12-19 16:04:35 +01:00
Andre DuffeckandGitHub d85aaab2d1 Merge pull request #8001 from aduffeck/sync-gowebdav
Sync gowebdav client with reva edge
2023-12-18 15:29:34 +01:00
Andre Duffeck 2f5742aa80 Automated changelog update [skip ci] 2023-12-18 09:06:12 +00:00
Andre DuffeckandGitHub 5749348b11 [full-ci] Ocm tus (#7998)
* Add service account configuration for the ocm service

* Bump reva

* Add changelog
2023-12-18 10:05:31 +01:00
Andre Duffeck 2922fc3b82 Automated changelog update [skip ci] 2023-12-14 13:50:19 +00:00
Andre DuffeckandGitHub 15a7f2dea6 Add the ocm subcommand to ocis (#7576)
* Add the ocm subcommand to ocis

* Add changelog
2023-12-14 14:49:41 +01:00
Andre Duffeck d3f79dec0c Automated changelog update [skip ci] 2023-12-14 13:37:10 +00:00
Andre DuffeckandGitHub fd09d29b5c Merge pull request #7973 from aduffeck/ocm-access
[full-ci] Ocm access
2023-12-14 14:36:31 +01:00
Andre DuffeckandGitHub e6ffd35e30 Merge pull request #7874 from aduffeck/retry-pp
[full-ci] Retry postprocessing steps
2023-12-12 14:56:40 +01:00
1858912408 Apply suggestions from code review
Co-authored-by: Martin <github@diemattels.at>
2023-12-12 14:08:21 +01:00
Andre DuffeckandGitHub 9be4a1fb22 Merge pull request #7843 from aduffeck/fix-wrapper-lockup
Read from stderr and stdout concurrently to prevent buffers from filling
2023-11-29 13:11:30 +01:00
7bb67d7e41 [full-ci] Bump reva (#7675)
* Bump reva

* Adapt expected failures

* update MOVE files between shares tests

---------

Co-authored-by: Saw-jan <saw.jan.grg3e@gmail.com>
2023-11-09 15:26:16 +01:00
Andre DuffeckandGitHub 5291978a42 Merge pull request #7667 from aduffeck/list-ocm-shares
Expose variable to enable/disable the listing of ocm shares
2023-11-08 10:23:41 +01:00
701feef965 Update services/frontend/pkg/config/config.go
Co-authored-by: Martin <github@diemattels.at>
2023-11-07 16:11:20 +01:00
Andre DuffeckandGitHub 524fff00a3 Merge pull request #7620 from aduffeck/ocm-docs
Ocm docs
2023-11-06 09:23:39 +01:00
208501a688 Apply headline capitalisation suggestions from code review
Co-authored-by: Martin <github@diemattels.at>
2023-11-06 08:29:55 +01:00
Andre DuffeckandGitHub f5f11e0bd2 Merge pull request #7580 from aduffeck/ocm-env-vars
Streamline ocm env vars
2023-10-25 09:41:11 +02:00
Andre DuffeckandGitHub e474e7bf05 Merge pull request #7564 from aduffeck/ocm2
[full-ci] Wire ocm services
2023-10-24 14:12:09 +02:00
Andre DuffeckandGitHub 96c877727d Merge pull request #7556 from owncloud/skipTestMovingBetweenSpace
[test-only] skip tests moving between shares jail
2023-10-23 09:59:59 +02:00
Andre DuffeckandGitHub 5a76586678 Merge pull request #7463 from aduffeck/bump-reva
Bump reva to pull in the ocm changes in reva edge.
2023-10-11 14:17:47 +02:00
Andre Duffeck 701673c2e1 Automated changelog update [skip ci] 2023-09-22 07:41:35 +00:00
Andre DuffeckandGitHub d5f73ba68d Do not reset share state (#7322)
* Do not reset received share state to pending if there is a state already

* Add changelog
2023-09-22 09:41:07 +02:00
Andre DuffeckandGitHub 2a0b553a41 Merge pull request #6991 from aduffeck/fix-subcommand
Remove subcommand from rebuild-jsoncs3-indexes that doesn't belong there
2023-08-09 08:51:07 +02:00
Andre Duffeck 62ffe1e18f Automated changelog update [skip ci] 2023-08-08 11:12:51 +00:00
Andre DuffeckandGitHub 52750bd615 Fail on errors (#6986)
* Return an error when something went wrong

* Add changelog
2023-08-08 13:12:03 +02:00
Andre Duffeck 2a50219282 Automated changelog update [skip ci] 2023-08-07 11:13:53 +00:00
Andre DuffeckandGitHub db6c98abf4 Merge pull request #6971 from aduffeck/rebuild-jsoncs3-index
Rebuild jsoncs3 index
2023-08-07 13:13:19 +02:00
Andre DuffeckandGitHub 5b26de71d4 Merge pull request #6824 from aduffeck/propagators
Make the storage-users propagator configurable
2023-07-31 11:04:01 +02:00
38c04b5517 Update services/storage-users/pkg/config/config.go
Co-authored-by: Martin <github@diemattels.at>
2023-07-31 10:26:19 +02:00
f17e56febc Update services/storage-users/pkg/config/config.go
Co-authored-by: Martin <github@diemattels.at>
2023-07-31 10:26:19 +02:00
Andre DuffeckandGitHub 3662ff2514 Merge pull request #6904 from aduffeck/rollback-migrations2
[full-ci] Add support for running/rolling back decomposedfs migrations
2023-07-31 09:03:46 +02:00
Andre DuffeckandGitHub 55426e7d70 Merge pull request #6785 from aduffeck/bump-reva
Bump reva to pull in the fix for the space index data structure
2023-07-13 09:49:11 +02:00
Andre DuffeckandGitHub 20252409ba Handle events asynchronously (#6584) 2023-06-22 09:23:49 +02:00
Andre Duffeck 938a2a006f Automated changelog update [skip ci] 2023-06-20 06:40:31 +00:00
Andre DuffeckandGitHub c81f423357 Merge pull request #6512 from aduffeck/fix-case-insensitivity
Always make Name and Tags queries case-insensitive
2023-06-20 08:39:50 +02:00
c0665975b3 Do not try to fulltext-index large files. (#6395)
* Do not try to extract the content of large files.

Both tika and ocis do not handle large files/archives very well.

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

---------

Co-authored-by: Martin <github@diemattels.at>
2023-05-27 19:03:58 +02:00
Andre DuffeckandGitHub 8da7d15279 Merge pull request #6388 from aduffeck/fix-internal-bleve-queries
Do not mess with non-lowercased fields
2023-05-25 11:48:28 +02:00
Andre DuffeckandGitHub 7b48fbbbad Merge pull request #6371 from aduffeck/improve-complex-search
Improve complex search
2023-05-24 14:39:20 +02:00
Andre DuffeckandGitHub 798f3d53dc Merge pull request #6356 from aduffeck/reduce-search-index-size
Do not include full-text and tag data in the composite field
2023-05-22 09:51:20 +02:00
Andre Duffeck be4cc7749e Automated changelog update [skip ci] 2023-05-12 10:12:28 +00:00
Andre DuffeckandGitHub 32014badd7 Merge pull request #6268 from aduffeck/improve-search
Improve search
2023-05-12 12:11:38 +02:00
Andre Duffeck cc4b8a8ec0 Automated changelog update [skip ci] 2023-05-11 09:20:12 +00:00
Andre DuffeckandGitHub c4a64ba69b Merge pull request #6271 from aduffeck/fix-test-expectations
[full-ci] Fix test expectations
2023-05-11 11:18:09 +02:00
Andre DuffeckandGitHub 64db3dd10e Merge pull request #6185 from aduffeck/gather-space-info-in-parallel
Speed up me/drives by gathering space information in parallel
2023-04-28 11:54:03 +02:00
129489203b Cleanup and improve the caching config (#6148)
* Cleanup and improve the caching config

* bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* disable stat cache

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>

* Bump reva

* Linter fixes

---------

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-04-27 12:35:30 +02:00
Andre DuffeckandGitHub be569b7c59 [full-ci] More cache cleanup (#6134)
* Make env var names consistent with the others

* Remove unused OCS cache

* Use the same cache database for shared caches

* Bump reva
2023-04-25 15:46:59 +02:00
77bb3d8bcd [full-ci] Refactor stores (#6019)
* Streamline the store implementation with and into reva

* Adapt to the cache/store refactoring in reva

* Streamline config options and their env vars

* Apply suggestions from code review

Co-authored-by: Martin <github@diemattels.at>

* Use the same database for all stores

* Bump reva

* Configure stat and filemetadata cache separately

* Fix default config

---------

Co-authored-by: Martin <github@diemattels.at>
2023-04-24 15:13:35 +02:00
Andre Duffeck 2479527cf6 Automated changelog update [skip ci] 2023-04-03 12:52:04 +00:00
Andre DuffeckandGitHub 668a37059b Merge pull request #5997 from aduffeck/fix-oidc-cache
Fix OIDC cache
2023-04-03 14:51:14 +02:00
0bc471007e Make the number of concurrent goroutines in decomposedfs configurable (#5887)
* Make the number of concurrent goroutines in decomposedfs configurable

* Update services/storage-users/pkg/config/config.go

Co-authored-by: Martin <github@diemattels.at>

* Update services/storage-users/pkg/config/config.go

Co-authored-by: Martin <github@diemattels.at>

---------

Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Martin <github@diemattels.at>
2023-03-24 10:49:51 +01:00
Andre Duffeck ebbc388b6d Automated changelog update [skip ci] 2023-03-17 11:31:38 +00:00
Andre DuffeckandGitHub 7bd735f615 Add command for inspecting and manipulating node metadata (#5858)
* Add command for inspecting and manipulating node metadata

* Add changelog

* Bump reva
2023-03-17 12:31:07 +01:00
06e2fee99a enable messagepack backend (#5762)
Switch to dev branch temporarily

bump commit



bump reva commit



bump reva to 0fc4fd83



bump reva to 144d26e7



enable treesize accounting



bump reva

bump reva

bump reva

bump reva

bump reva

bump reva

bump reva

bump reva

bump reva

bump reva

bump reva to 5fd530ba4f87



use messagepack bin format



bump reva to 5d216811275c



Bump reva

Bump reva

Bump reva

bump reva

Signed-off-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Michael Barz <mbarz@owncloud.com>
2023-03-13 22:06:48 +01:00
8b68830201 [full-ci] Improve ini backend, support redis sentinel caches (#5737)
* Configure the file metadata cache for storage-users

* Also configure the cache for the system storage

* Update services/storage-system/pkg/config/config.go

Co-authored-by: Martin <github@diemattels.at>

* Improve documentation

* Bump reva to pull in the latest ini backend changes

* Fix missing comment

* Update services/storage-users/README.md

Co-authored-by: Martin <github@diemattels.at>

* Update services/storage-system/README.md

Co-authored-by: Martin <github@diemattels.at>

* Tweak docs

* Apply suggestions from code review

Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
Co-authored-by: Martin <github@diemattels.at>

---------

Co-authored-by: Martin <github@diemattels.at>
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2023-03-06 15:04:19 +01:00
Andre DuffeckandGitHub 0dc6eaf466 Graph test coverage (#5136)
* Add unit tests for users.go

* Do not choke on broken spaces

* Generate mocks for the RoleService

* Increase test coverage

* Increase test coverage
2022-11-29 10:22:40 +01:00
Andre DuffeckandGitHub e07c7bc808 Graph test coverage drives (#5141)
* Add GetSingleDrive to the service interface

* Increase test coverage for the drives endpoint

* Remove dead code

* Add missing methods to the graph service interface

* Use the existing permission client of the graph service

* Increase test coverage
2022-11-29 10:21:25 +01:00
Andre DuffeckandGitHub 7c2961e18a Graph test coverage (#5098)
* First iteration of a groups test

* Do not render a result in case of an error

* Generate mocks for identity.Backend

* Export listResponse (-> ListResponse)

* Add unit tests for GetGroups

* Finish GetGroups unit tests

* Increase test coverage

* Increase test coverage

* Increase test coverage

* Add missing mocks

* Fix return codes

* Fix error messages
2022-11-23 14:01:29 +01:00
1b764d2142 Handle more events (#5071)
* Handle SpaceShared events

* Adapt to changed ID in the SpaceShared event

* Handle the SpaceUnshared events

* Handle the SpaceUpdated event

* update reva to latest edge

* fix typo

* Disable cyclomatic complexity check for StartAuditLogger

Co-authored-by: Michael Barz <mbarz@owncloud.com>
2022-11-23 14:00:29 +01:00
Andre Duffeck 2ee2f7181c Automated changelog update [skip ci] 2022-11-15 15:53:36 +00:00
Andre DuffeckandGitHub 60b979d1d4 Find spaces (#5044)
* Use the node name for the name field in the index instead of the path

* Add changelog

* Fix tests

* Adapt expected failures
2022-11-15 16:52:55 +01:00
Andre Duffeck 4efe8fd43b Automated changelog update [skip ci] 2022-11-10 16:09:00 +00:00
Andre DuffeckandGitHub a0762e248e Filter hidden files (#5018)
* Filter hidden files from search results

* Add changelog

* Do not filter hidden files by default

* Set the hidden fields when converting fields to an entity

* Fix test
2022-11-10 17:05:43 +01:00
Andre Duffeck a5b2bb13ee Automated changelog update [skip ci] 2022-11-08 14:21:18 +00:00
Andre DuffeckandGitHub 925790d64b Dont reindex twice (#5001)
* Only reindex a space once at a time

* Add changelog
2022-11-08 15:20:38 +01:00
Andre Duffeck c260f6594d Automated changelog update [skip ci] 2022-10-14 15:03:38 +00:00
Andre DuffeckandGitHub 5e220920f0 [full-ci] Rescan spaces (#4777)
* Increase the timeout for indexing spaces

* Allow for making queries that are not constrained to a RootID

* Use nano precision for the Mtime

* Add a SpaceDebouncer

The debouncer can be used to delay operations on spaces until things
have settled down to avoid doing the same operation multiple times.

* Do not index subtrees until they have changed (i.e. the mtime differs)

* Also pass a user to the space debouncer func

* Trigger a rescan of the according space when an event is received

* Improve wording/logging

* Add changelog

* Get rid of superfluous GetUserByClaim call

* Fix tests

* Fix reindexing triggered by shares by using the event's SpaceOwner

* Bump reva to pull in the space owner events changes

* Fix changelog

* Fix linter issues

* Fall back to the executant if no owner was received from the event

* Bump reva and go-cs3apis

* Fix go.sum

* Bump core

* Adapt expected failures

* Tweak debounce settings
2022-10-14 17:03:05 +02:00
Andre Duffeck 6ca8a04311 Automated changelog update [skip ci] 2022-09-27 08:42:02 +00:00
Andre DuffeckandGitHub eaba396394 Fix unfindable entities (#4651)
* Fix unfindable entities from shares/publicshares

Directories or empty files weren't added to the search index properly
because they were indexed relative to the share instead of the space
root.

Fixes https://github.com/owncloud/ocis/issues/4489

* Add changelog

* Fix tests

* Reuse existing authorized context
2022-09-27 10:39:42 +02:00
Andre DuffeckandGitHub 68b17eb68d Search dav spaces (#4661)
* Add a Ref option to SearchRequest, e.g. for limiting the search to a space

* Enable REPORT requests for /dav/spaces URLs

* Limit the search to the according space in case of /dav/spaces searches

* Add changelog

* Adapt expected failures

* Comment exported functions
2022-09-27 10:38:15 +02:00
291e191490 Update services/sharing/pkg/revaconfig/config.go
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-09-09 15:33:31 +02:00
8bdfe5f6d1 Update ocis/pkg/command/migrate.go
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-09-09 15:33:20 +02:00
a107924a4c Update services/sharing/pkg/config/config.go
Co-authored-by: Jörn Friedrich Dreyer <jfd@butonic.de>
2022-09-09 15:25:06 +02:00
Andre Duffeck 26f15c2fa9 Automated changelog update [skip ci] 2022-08-02 07:19:16 +00:00
Andre DuffeckandGitHub 9fc485c95f Merge pull request #4308 from aduffeck/fix-search-in-received-shares
Fix search in received shares
2022-08-02 09:11:32 +02:00
Andre Duffeck 3f80c8831d Automated changelog update [skip ci] 2022-07-13 11:59:59 +00:00
Andre DuffeckandGitHub f990ef91b6 Merge pull request #4189 from aduffeck/add-total-to-search
Search: Return the number of total matches
2022-07-13 13:58:06 +02:00
Andre Duffeck f739d18807 Automated changelog update [skip ci] 2022-07-07 06:47:44 +00:00
Andre DuffeckandGitHub edb8e85fb9 Merge pull request #4118 from aduffeck/webdav-search
Wire up the search service at the old webdav endpoint
2022-07-07 08:47:19 +02:00
Andre Duffeck ee97594742 Automated changelog update [skip ci] 2022-07-06 06:47:34 +00:00
Andre DuffeckandGitHub 24fbb4a389 Merge pull request #4094 from aduffeck/polish-search
Polish search
2022-07-06 08:44:43 +02:00
Andre DuffeckandGitHub 0e7de0faf3 Configure the provider id of the shares storage provider (#3723) 2022-05-13 10:34:02 +02:00
Andre DuffeckandGitHub d540c4405d [full-ci] Fix searching shares (#3668)
* Be more robust when limiting the search to a directory in the space

* Add more debug logs

* Fix searching in shares

* Delay indexing by a second so that everything has time to settle

Indexing immediately when the event arrives sometimes causes issues
trying to stat the changed resource.

* Pick up the machine auth secret from the default config

* Adapt to the resourceid refactoring in reva

* Fix unit tests
2022-05-04 13:15:28 +02:00
Andre DuffeckandGitHub ed417ea1c9 Add support for the cs3 metadata storage share manager (#3290)
* Add support for the cs3 share manager

* Name the cs3 storage config options generically so they can be reused

* Set default value for CS3ServiceUserIdp
2022-03-21 10:19:48 +01:00