André Duffeck
71c0a469b9
Reduce default batch size to prevent memory issues with large documents
2026-03-25 14:27:37 +01:00
André Duffeck
428f69416f
Commit batches when the limit is reached while iterating over children
2026-03-25 14:27:37 +01:00
André Duffeck
dea306247b
Do not remove stopwords by default
...
Keeping the stop words leads to slightly bigger indexes but fixes
chopped up highlights of search results and phrase accuracy during
search.
2026-03-25 09:41:23 +01:00
André Duffeck
8a83eea742
Limit the highlighter to two fragments
2026-03-25 08:41:48 +01:00
André Duffeck and Florian Schade
4fa5198501
Improve highlight support in osu
...
Co-authored-by: Florian Schade <f.schade@icloud.com >
2026-03-25 08:41:48 +01:00
André Duffeck
a6dd9b9e18
Use the fast vector highlighter for highlighting search results
2026-03-25 08:41:48 +01:00
André Duffeck
9e93f29ffe
Introduce opensearch index v2
...
The new index allows for faster highlighing uses the fvh highlighter and
searching for favorites.
2026-03-25 08:41:48 +01:00
André Duffeck
7ad037d96b
Add the favorite flag to REPORT results
2026-03-23 15:35:23 +01:00
André Duffeck
10bc14130e
Do not send back the full content in the search response
2026-03-23 13:52:09 +01:00
André Duffeck
a54273b8f6
Adapt expected failures until the tests have been adapted
2026-03-13 09:38:58 +01:00
André Duffeck
44549379ca
Bump reva
2026-03-13 09:38:57 +01:00
André Duffeck
cd0831aa10
We no longer manage favorites via arbitrary metadata
2026-03-13 09:38:28 +01:00
André Duffeck
bfba4ec671
Render the followed drive item after following it according to the spec
2026-03-13 09:38:28 +01:00
André Duffeck
1452d30bc6
Bump libre-graph-api-go
2026-03-13 09:38:28 +01:00
André Duffeck
d214a7535c
Make the search index pick up changes to favorites
2026-03-13 09:38:28 +01:00
André Duffeck
58db915775
Emit events when favorites are added/removed
2026-03-13 09:38:28 +01:00
André Duffeck
3402390b69
Add endpoints for following/unfollowing drive items to the graph API
2026-03-13 09:38:28 +01:00
André Duffeck
ce5ec1b3dc
Add support for favorites to the search service
2026-03-13 09:38:28 +01:00
André Duffeck
e51ee6d229
Adapt to cs3api changes
2026-03-06 14:59:30 +01:00
André Duffeck
2146e970ee
Bump reva
2026-03-06 14:59:29 +01:00
André Duffeck
2043dee745
Bump opencloud version
2026-02-16 11:02:50 +01:00
André Duffeck
97da8655b7
Bump reva
2026-02-16 10:53:20 +01:00
André Duffeck and Ralf Haferkamp
ae3582afa3
Bump reva
2026-02-11 10:05:53 +01:00
André Duffeck
e2f6a68810
Do not ever set a TTL for the ID cache. It's not supposed to expire.
2026-01-28 12:52:24 +01:00
André Duffeck
a93769ae9b
Bump reva to pull in the latest fixes and improvements
2026-01-21 08:43:08 +01:00
André Duffeck
ef3c0da0cb
Assing the opencloudUser to the users so they get a personal space
2026-01-20 15:49:13 +01:00
André Duffeck
80601fc0dc
Do not try to enable multi tenancy for the system storage
2026-01-20 15:49:13 +01:00
André Duffeck and fschade
f9e43497ae
Remove items from the index when they are purged from the trashbin
...
Also purge all deleted items when the whole trashbin is purged.
2025-09-03 11:18:37 +02:00
André Duffeck and fschade
71bde21a65
Improve logging
2025-09-03 11:18:21 +02:00
André Duffeck and fschade
398534761a
Implement purging items from the index
2025-09-03 11:18:21 +02:00
André Duffeck and fschade
acd39b5ce8
Purge items from the index recursively
2025-09-03 11:18:19 +02:00
André Duffeck and fschade
1003734b45
Use batches when restoring or moving items
2025-08-28 15:50:44 +02:00
André Duffeck and fschade
9f9e03794d
Do not intertwine different batch operations
2025-08-28 15:50:44 +02:00
André Duffeck
cc6681be3b
Drop the "InProgress" metric
...
It didn't really make sense in multi-node setups because items could be
started and finished on different nodes.
2025-08-07 20:33:05 +02:00
André Duffeck and Christian Richter
0f09cdd8ec
bump reva
2025-08-05 17:12:58 +02:00
André Duffeck and Christian Richter
b9f48edd87
Try to find users by username or id, just like the ldap backend does
2025-08-05 17:10:59 +02:00
André Duffeck and Christian Richter
926a2c2080
Filter users by tenant, add tenant ids to demo users
2025-08-05 17:10:59 +02:00
André Duffeck
7c59e57d43
Make batch size configurable
2025-08-04 16:14:56 +02:00
André Duffeck
7a7d148dcf
Fix tests
2025-08-04 15:39:27 +02:00
André Duffeck
3c8e2dacfd
Improve indexing performance using batches
2025-08-04 15:15:54 +02:00
André Duffeck
1afc1331af
Do not run the timout func if the work func has run
2025-08-01 10:03:56 +02:00
André Duffeck
4bafe2e611
Make sure to register prometheus collectors only once
2025-07-31 08:55:10 +02:00
André Duffeck
b4cc781832
Do not try to log metrics when we failed to get the consumer info
...
Fixes #1285
2025-07-29 14:19:00 +02:00
André Duffeck
484f8bc66b
Make "remote.php" part of thumbnail URLs optional
2025-07-22 10:43:35 +02:00
André Duffeck
4c2b723ab0
Implement $expand=thumbnails for sharedwithme
2025-07-22 10:43:00 +02:00
André Duffeck
683b979bdd
Remove unused code
2025-07-22 10:42:46 +02:00
André Duffeck
a6cdbc710d
Fix thumbnail URLs for sharedbyme
2025-07-22 10:42:22 +02:00
André Duffeck
3bd7a61e46
Document the metrics exposed in search and postprocessing
2025-07-18 08:50:03 +02:00
André Duffeck
386e322050
Fix metric
2025-07-18 08:27:06 +02:00
André Duffeck
dbd6a4db71
Fix tests
2025-07-17 16:00:11 +02:00
André Duffeck
8adf425b2d
Bump reva
2025-07-17 15:23:57 +02:00
André Duffeck
4704cedd3b
Add metrics for the postprocessing duration
2025-07-17 15:23:57 +02:00
André Duffeck
791b4df173
Collect metrics about search and index durations
2025-07-17 15:23:57 +02:00
André Duffeck
273c0ed270
Expose nats metris for the postprocessing service
2025-07-17 15:23:57 +02:00
André Duffeck
400b9a5d30
Expose nats metrics of the search service
2025-07-17 14:33:23 +02:00
André Duffeck
de7b659e3d
Expose the MaxAckPending and AckWait settings for postprocessing
2025-07-17 09:16:14 +02:00
André Duffeck
ad8ac8ee25
Only ack postprocessing events after persisting
2025-07-17 09:16:14 +02:00
André Duffeck
aa9f74bcb4
Regenerate mocks
2025-07-16 10:29:34 +02:00
André Duffeck
e92f7c9086
Fix tests
2025-07-16 10:29:34 +02:00
André Duffeck
4998e1e09f
Inject a stream into HandleEvents to increase testability
2025-07-16 10:29:34 +02:00
André Duffeck
63ab8f789e
Bump reva
2025-07-16 10:29:34 +02:00
André Duffeck
e21fe7a4fe
Expose the MaxAckPending and AckWait settings for the search service
2025-07-15 09:57:19 +02:00
André Duffeck
608af81913
Switch to the raw nats consumer instead of the go-micro events
...
This allows more direct control of the nats events. As a first step
we'll now acknowledge events manually, but in the future we'll add
metrics etc. to gain more insight into the queues.
2025-07-15 09:57:19 +02:00
André Duffeck
5c6eaf9b96
Only remove obsolete IDs from the index
2025-06-27 11:19:53 +02:00
André Duffeck
e8a1609bde
get hostname of S3 server from var. not secret
2025-06-26 09:01:26 +02:00
André Duffeck
adc09da4c7
Fix typo
2025-06-26 08:50:49 +02:00
André Duffeck
c0e09f76c8
Add a "posixfs consistency" command
...
This command checks posixfs storages for inconsistencies and fixes them.
2025-06-26 08:50:49 +02:00
André Duffeck
dc108d84ea
Bump reva to pull in the latest fixes
2025-06-18 10:37:09 +02:00
André Duffeck
bf7b80b7cb
Bump reva to pull in the latest fixes
2025-06-17 10:20:07 +02:00
André Duffeck
8908f5e8ff
Explicitly stop the watcher after reading from it
2025-06-05 15:24:00 +02:00
André Duffeck
c9a704d446
Continue when encountering keys that aren't relevant
2025-06-05 15:24:00 +02:00
André Duffeck
ac8b58233e
Explicitly stop the watcher after reading from it
2025-06-05 15:24:00 +02:00
André Duffeck
e9201d65a0
Fix comments
2025-06-05 15:24:00 +02:00
André Duffeck
888d0c8d99
Bump reva
2025-06-05 15:23:58 +02:00
André Duffeck
beb7920b50
Improve logging, fix typo
2025-06-05 15:20:20 +02:00
André Duffeck
02b37cc17c
Migrate existing activities to the new batch format
2025-06-05 15:20:20 +02:00
André Duffeck
33b7fa79f9
Streamline config, fix tests
2025-06-05 15:20:20 +02:00
André Duffeck
f9d4a038b3
Switch activitylog servie to a native nats store
...
Also split the list of activities in different chunks, one per buffered
write. That allows for scaling the service horizontally and also helps
reduce the load caused by the write operations.
2025-06-05 15:20:20 +02:00
André Duffeck
3ed30f58bd
Fix purging revisions from decomposeds3 blobstores
2025-05-30 14:14:20 +02:00
André Duffeck
c473515ec1
Do not reuse resource infos from previous iterations
2025-05-16 10:24:43 +02:00
André Duffeck
7feaeeab6c
Bump reva
2025-05-16 10:24:43 +02:00
André Duffeck
daa650123c
Do not retrieve resources twice
2025-05-16 10:24:43 +02:00
André Duffeck
4b5c45254e
Fix invalidating the parent id cache for path based references
2025-05-16 10:24:43 +02:00
André Duffeck
b9301b7ac1
Fix tests
2025-05-16 10:24:43 +02:00
André Duffeck
cbac5f1555
Switch to a different msgpack lib
...
The current one has a problem handling time.Time{} structs
2025-05-16 10:24:43 +02:00
André Duffeck
66e79044b6
Log error when failing to unmarshal messagepack
2025-05-16 10:24:43 +02:00
André Duffeck
77aa702d23
Bump reva to pull in the latest changes
2025-05-16 10:24:43 +02:00
André Duffeck
400fc0b4c2
Do not reset the timer when new activities are queued
2025-05-16 10:24:43 +02:00
André Duffeck
c618c7436a
Make the write buffer duration configurable
2025-05-16 10:24:43 +02:00
André Duffeck
b3017dc774
Use the parent ID from the event if possible
2025-05-16 08:34:26 +02:00
André Duffeck
e602ea40a7
Switch to msgpack lists
2025-05-16 08:34:26 +02:00
André Duffeck
e826b778c7
Add a ttl cache for parent ids
2025-05-16 08:34:26 +02:00
André Duffeck
4a98077e20
Add tests for debounced activity storage
2025-05-16 08:34:26 +02:00
André Duffeck
3b6de607ff
Buffer activities for writing to reduce load
2025-05-16 08:34:26 +02:00
André Duffeck
94a973f8a3
Add tracing for activitylog
2025-05-16 08:34:26 +02:00
André Duffeck
f8c89b1c3d
Add deprecation notice for STORAGE_USERS_POSIX_WATCH_FOLDER_KAFKA_BROKERS
2025-04-16 11:09:30 +02:00
André Duffeck
8c9e05d1f0
Expose more config vars for the posix fs watchers
2025-04-16 11:08:03 +02:00
André Duffeck
91fd396c86
Add env var to make the inotify stats frequency configurable
2025-04-15 09:24:55 +02:00
André Duffeck
b0f3ab6252
Simplify item-trashed SSEs. Also fixes it for coll. posix fs.
2025-04-03 15:13:11 +02:00
André Duffeck
09818a9d7e
Adapt command to changed signature
2025-04-01 15:04:53 +02:00
André Duffeck