Commit Graph
4 Commits
Author SHA1 Message Date
MertandGitHub 23f1f87dd0 feat(ml): optimized model inference (#31750)
* add immich_model

* graph per shape

* decouple ocr from ort

* prepared graphs

* use f16 for accelerators

* preload all shapes

* warm-up pass after preload

* keep precision for fp32 graphs

* ensure padding for text recognition

* optimize postprocessing for fp16 outputs

* handle raw ctc logits

* use f16 output for rknn

* mmap token embedding

* read device metadata
2026-09-25 14:56:44 -04:00
MertandGitHub 2b1b43a7e4 feat(ml): composable ml (#9973)
* modularize model classes

* various fixes

* expose port

* change response

* round coordinates

* simplify preload

* update server

* simplify interface

simplify

* update tests

* composable endpoint

* cleanup

fixes

remove unnecessary interface

support text input, cleanup

* ew camelcase

* update server

server fixes

fix typing

* ml fixes

update locustfile

fixes

* cleaner response

* better repo response

* update tests

formatting and typing

rename

* undo compose change

* linting

fix type

actually fix typing

* stricter typing

fix detection-only response

no need for defaultdict

* update spec file

update api

linting

* update e2e

* unnecessary dimension

* remove commented code

* remove duplicate code

* remove unused imports

* add batch dim
2024-06-07 03:09:47 +00:00
MertandGitHub 0c4df216d7 feat(ml): improve test coverage (#7041)
* update e2e

* tokenizer tests

* more tests, remove unnecessary code

* fix e2e setting

* add tests for loading model

* update workflow

* fixed test
2024-02-11 17:58:56 -05:00
MertandGitHub 258b98c262 fix(ml): load models in separate threads (#4034)
* load models in thread

* set clip mode logs to debug level

* updated tests

* made fixtures slightly less ugly

* moved responses to json file

* formatting
2023-09-09 16:02:44 +07:00