André Duffeck
228adbe177
Reduce default batch size to prevent memory issues with large documents
2026-03-25 18:32:57 +01:00
André Duffeck
de1120210e
Commit batches when the limit is reached while iterating over children
2026-03-25 18:32:54 +01:00
André Duffeck
312eacd46a
Bump latest tag
2026-02-05 15:35:05 +01:00
André Duffeck
3c906e1dae
Bump reva
2026-02-05 15:34:11 +01:00
André Duffeck
052ee8910d
Bump reva
2026-02-05 11:45:04 +01:00
André Duffeck and fschade
f9e43497ae
Remove items from the index when they are purged from the trashbin
...
Also purge all deleted items when the whole trashbin is purged.
2025-09-03 11:18:37 +02:00
André Duffeck and fschade
71bde21a65
Improve logging
2025-09-03 11:18:21 +02:00
André Duffeck and fschade
398534761a
Implement purging items from the index
2025-09-03 11:18:21 +02:00
André Duffeck and fschade
acd39b5ce8
Purge items from the index recursively
2025-09-03 11:18:19 +02:00
André Duffeck and fschade
1003734b45
Use batches when restoring or moving items
2025-08-28 15:50:44 +02:00
André Duffeck and fschade
9f9e03794d
Do not intertwine different batch operations
2025-08-28 15:50:44 +02:00
André Duffeck
cc6681be3b
Drop the "InProgress" metric
...
It didn't really make sense in multi-node setups because items could be
started and finished on different nodes.
2025-08-07 20:33:05 +02:00
André Duffeck and Christian Richter
0f09cdd8ec
bump reva
2025-08-05 17:12:58 +02:00
André Duffeck and Christian Richter
b9f48edd87
Try to find users by username or id, just like the ldap backend does
2025-08-05 17:10:59 +02:00
André Duffeck and Christian Richter
926a2c2080
Filter users by tenant, add tenant ids to demo users
2025-08-05 17:10:59 +02:00
André Duffeck
7c59e57d43
Make batch size configurable
2025-08-04 16:14:56 +02:00
André Duffeck
7a7d148dcf
Fix tests
2025-08-04 15:39:27 +02:00
André Duffeck
3c8e2dacfd
Improve indexing performance using batches
2025-08-04 15:15:54 +02:00
André Duffeck
1afc1331af
Do not run the timout func if the work func has run
2025-08-01 10:03:56 +02:00
André Duffeck
4bafe2e611
Make sure to register prometheus collectors only once
2025-07-31 08:55:10 +02:00
André Duffeck
b4cc781832
Do not try to log metrics when we failed to get the consumer info
...
Fixes #1285
2025-07-29 14:19:00 +02:00
André Duffeck
484f8bc66b
Make "remote.php" part of thumbnail URLs optional
2025-07-22 10:43:35 +02:00
André Duffeck
4c2b723ab0
Implement $expand=thumbnails for sharedwithme
2025-07-22 10:43:00 +02:00
André Duffeck
683b979bdd
Remove unused code
2025-07-22 10:42:46 +02:00
André Duffeck
a6cdbc710d
Fix thumbnail URLs for sharedbyme
2025-07-22 10:42:22 +02:00
André Duffeck
3bd7a61e46
Document the metrics exposed in search and postprocessing
2025-07-18 08:50:03 +02:00
André Duffeck
386e322050
Fix metric
2025-07-18 08:27:06 +02:00
André Duffeck
dbd6a4db71
Fix tests
2025-07-17 16:00:11 +02:00
André Duffeck
8adf425b2d
Bump reva
2025-07-17 15:23:57 +02:00
André Duffeck
4704cedd3b
Add metrics for the postprocessing duration
2025-07-17 15:23:57 +02:00
André Duffeck
791b4df173
Collect metrics about search and index durations
2025-07-17 15:23:57 +02:00
André Duffeck
273c0ed270
Expose nats metris for the postprocessing service
2025-07-17 15:23:57 +02:00
André Duffeck
400b9a5d30
Expose nats metrics of the search service
2025-07-17 14:33:23 +02:00
André Duffeck
de7b659e3d
Expose the MaxAckPending and AckWait settings for postprocessing
2025-07-17 09:16:14 +02:00
André Duffeck
ad8ac8ee25
Only ack postprocessing events after persisting
2025-07-17 09:16:14 +02:00
André Duffeck
aa9f74bcb4
Regenerate mocks
2025-07-16 10:29:34 +02:00
André Duffeck
e92f7c9086
Fix tests
2025-07-16 10:29:34 +02:00
André Duffeck
4998e1e09f
Inject a stream into HandleEvents to increase testability
2025-07-16 10:29:34 +02:00
André Duffeck
63ab8f789e
Bump reva
2025-07-16 10:29:34 +02:00
André Duffeck
e21fe7a4fe
Expose the MaxAckPending and AckWait settings for the search service
2025-07-15 09:57:19 +02:00
André Duffeck
608af81913
Switch to the raw nats consumer instead of the go-micro events
...
This allows more direct control of the nats events. As a first step
we'll now acknowledge events manually, but in the future we'll add
metrics etc. to gain more insight into the queues.
2025-07-15 09:57:19 +02:00
André Duffeck
5c6eaf9b96
Only remove obsolete IDs from the index
2025-06-27 11:19:53 +02:00
André Duffeck
e8a1609bde
get hostname of S3 server from var. not secret
2025-06-26 09:01:26 +02:00
André Duffeck
adc09da4c7
Fix typo
2025-06-26 08:50:49 +02:00
André Duffeck
c0e09f76c8
Add a "posixfs consistency" command
...
This command checks posixfs storages for inconsistencies and fixes them.
2025-06-26 08:50:49 +02:00
André Duffeck
dc108d84ea
Bump reva to pull in the latest fixes
2025-06-18 10:37:09 +02:00
André Duffeck
bf7b80b7cb
Bump reva to pull in the latest fixes
2025-06-17 10:20:07 +02:00
André Duffeck
8908f5e8ff
Explicitly stop the watcher after reading from it
2025-06-05 15:24:00 +02:00
André Duffeck
c9a704d446
Continue when encountering keys that aren't relevant
2025-06-05 15:24:00 +02:00
André Duffeck
ac8b58233e
Explicitly stop the watcher after reading from it
2025-06-05 15:24:00 +02:00
André Duffeck
e9201d65a0
Fix comments
2025-06-05 15:24:00 +02:00
André Duffeck
888d0c8d99
Bump reva
2025-06-05 15:23:58 +02:00
André Duffeck
beb7920b50
Improve logging, fix typo
2025-06-05 15:20:20 +02:00
André Duffeck
02b37cc17c
Migrate existing activities to the new batch format
2025-06-05 15:20:20 +02:00
André Duffeck
33b7fa79f9
Streamline config, fix tests
2025-06-05 15:20:20 +02:00
André Duffeck
f9d4a038b3
Switch activitylog servie to a native nats store
...
Also split the list of activities in different chunks, one per buffered
write. That allows for scaling the service horizontally and also helps
reduce the load caused by the write operations.
2025-06-05 15:20:20 +02:00
André Duffeck
3ed30f58bd
Fix purging revisions from decomposeds3 blobstores
2025-05-30 14:14:20 +02:00
André Duffeck
c473515ec1
Do not reuse resource infos from previous iterations
2025-05-16 10:24:43 +02:00
André Duffeck
7feaeeab6c
Bump reva
2025-05-16 10:24:43 +02:00
André Duffeck
daa650123c
Do not retrieve resources twice
2025-05-16 10:24:43 +02:00
André Duffeck
4b5c45254e
Fix invalidating the parent id cache for path based references
2025-05-16 10:24:43 +02:00
André Duffeck
b9301b7ac1
Fix tests
2025-05-16 10:24:43 +02:00
André Duffeck
cbac5f1555
Switch to a different msgpack lib
...
The current one has a problem handling time.Time{} structs
2025-05-16 10:24:43 +02:00
André Duffeck
66e79044b6
Log error when failing to unmarshal messagepack
2025-05-16 10:24:43 +02:00
André Duffeck
77aa702d23
Bump reva to pull in the latest changes
2025-05-16 10:24:43 +02:00
André Duffeck
400fc0b4c2
Do not reset the timer when new activities are queued
2025-05-16 10:24:43 +02:00
André Duffeck
c618c7436a
Make the write buffer duration configurable
2025-05-16 10:24:43 +02:00
André Duffeck
b3017dc774
Use the parent ID from the event if possible
2025-05-16 08:34:26 +02:00
André Duffeck
e602ea40a7
Switch to msgpack lists
2025-05-16 08:34:26 +02:00
André Duffeck
e826b778c7
Add a ttl cache for parent ids
2025-05-16 08:34:26 +02:00
André Duffeck
4a98077e20
Add tests for debounced activity storage
2025-05-16 08:34:26 +02:00
André Duffeck
3b6de607ff
Buffer activities for writing to reduce load
2025-05-16 08:34:26 +02:00
André Duffeck
94a973f8a3
Add tracing for activitylog
2025-05-16 08:34:26 +02:00
André Duffeck
f8c89b1c3d
Add deprecation notice for STORAGE_USERS_POSIX_WATCH_FOLDER_KAFKA_BROKERS
2025-04-16 11:09:30 +02:00
André Duffeck
8c9e05d1f0
Expose more config vars for the posix fs watchers
2025-04-16 11:08:03 +02:00
André Duffeck
91fd396c86
Add env var to make the inotify stats frequency configurable
2025-04-15 09:24:55 +02:00
André Duffeck
b0f3ab6252
Simplify item-trashed SSEs. Also fixes it for coll. posix fs.
2025-04-03 15:13:11 +02:00
André Duffeck
09818a9d7e
Adapt command to changed signature
2025-04-01 15:04:53 +02:00
André Duffeck
054c87b3fd
Enable scan/watch in the storageprovider only
2025-04-01 14:46:20 +02:00
André Duffeck
d18549e3fa
Adapt test, the tokens API returns the ids as the tokens now
2025-03-26 19:30:02 +01:00
André Duffeck
82e63866f8
Bump reva
2025-03-26 19:00:49 +01:00
André Duffeck
e37bedda1c
Fix permission errors when uploading a new instance logo
...
Fixes #475
2025-03-26 09:20:50 +01:00
André Duffeck and Ralf Haferkamp
2344092b81
Bump reva
2025-03-25 17:52:20 +01:00
André Duffeck
8b7f12126f
Do not cache when there was an error gathering the data
...
Fixes #452
2025-03-24 15:12:30 +01:00
André Duffeck
ac54b446a5
Adapt expectation on number of versions of same-mtime uploads
2025-03-24 10:01:35 +01:00
André Duffeck
073da849e0
Fix backup command to understand all versions
2025-03-24 09:48:26 +01:00
André Duffeck
c2ce9a136d
Bump reva
2025-03-24 08:47:22 +01:00
André Duffeck
467fc2af0c
Adapt to auth-app changes
2025-03-24 08:47:22 +01:00
André Duffeck
73c179e447
Adapt expected failures
2025-03-21 15:10:10 +01:00
André Duffeck
ac8351bd3b
Adapt test expectations, now that .mlock files are properly removed
2025-03-21 15:10:06 +01:00
André Duffeck
00d49804cf
Bump reva to pull in the latest fixes
2025-03-21 12:29:25 +01:00
André Duffeck
fe2e4b6c23
Bump reva
2025-03-20 12:45:46 +01:00
André Duffeck
d58f066376
Re-enable mtime related tests
2025-03-20 10:22:58 +01:00
André Duffeck
073e902626
Bump web in CI
2025-03-20 08:43:33 +01:00
André Duffeck
4569064400
Adapt expected failures
2025-03-19 18:24:03 +01:00
André Duffeck
5aa51dcfda
Use a separate expected failures file for posix
2025-03-19 18:24:03 +01:00
André Duffeck and Ralf Haferkamp
31f24a65b3
Bump reva
2025-03-19 17:41:47 +01:00
André Duffeck and Ralf Haferkamp
05c2b39281
Always run CLI tests with the decomposed storage driver
2025-03-19 17:41:47 +01:00
André Duffeck
a701f973de
Run ci with posix by default, allow for switching to decomposed
2025-03-19 12:25:28 +01:00
André Duffeck
94e668b194
Sort env vars
2025-03-19 12:23:28 +01:00
André Duffeck