Add per-model CPU execution option #45
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
… plan Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Add GitHub Copilot instructions for repository
…ign decisions Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Add comprehensive multi-user support specification and implementation plan
Sync with remote
…ntion (#8) * Initial plan * Fix Z-Image VAE encode/decode to request working memory Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add test for Z-Image working memory estimation Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix unit test: only set config attribute for AutoencoderKL, not FluxAutoEncoder Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * style: fix ruff errors * Fix test: use model_construct to bypass Pydantic validation for mock fields Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore(ruff): fix ruff errors --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
…#6) * Initial plan * Add Phase 1: Database schema, authentication utilities, and user service - Added dependencies: passlib[bcrypt], python-jose[cryptography], email-validator - Created migration_25 for multi-user support database schema - users table with authentication fields - user_sessions, user_invitations, shared_boards tables - Added user_id columns to boards, images, workflows, session_queue, style_presets - Created system user for backward compatibility - Implemented authentication utilities - Password hashing and validation (password_utils.py) - JWT token generation and verification (token_service.py) - Implemented user service - Abstract base class (users_base.py) - SQLite implementation (users_default.py) - Full CRUD operations for users - Authentication and admin management - Added comprehensive tests for all components - All code passes ruff linting and mypy type checking Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Refactor user service to use transaction pattern and improve security documentation - Changed all database operations to use self._db.transaction() context manager - Follows the same pattern as other services like board_records - Improves thread safety and transaction management - Added detailed security warnings for the hardcoded JWT secret key - Documented that it must be moved to config before production - Fixed whitespace linting issues - All tests still pass with the refactored code Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix system user email to use valid domain - Changed system user email from system@invokeai.local to system@system.invokeai - .local domains are rejected by email-validator as special-use/reserved - All end-to-end tests now pass successfully Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix test failures: password length, migration chain, and table existence checks - Fixed bcrypt 72-byte password limit by truncating long passwords in hash_password and verify_password - Fixed test_migration_25 to use callback directly instead of migrator to avoid fragmented chain error - Updated system user email assertion in test from @invokeai.local to @system.invokeai - Added table existence checks for boards, images, workflows, and session_queue before altering them - This prevents "no such table" errors when migration runs in environments without all tables Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * (chore) fix ruff errors * (chore) fix ruff errors (2) * Fix test failures: use _conn instead of conn and improve password truncation - Fixed test_user_service.py to use db._conn instead of db.conn (SqliteDatabase uses private _conn) - Simplified password truncation logic in password_utils.py for better clarity - Ensured UTF-8 decoding handles truncated bytes properly with errors="ignore" Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore(uv): updated uv lockfile * Fix bcrypt password length errors by configuring passlib properly - Added bcrypt__relax_truncate_checks=True to CryptContext to prevent errors on long passwords - Removed min_length=8 constraint from pydantic models to allow service-level validation - Service-level validation provides better error messages and more control - Manual truncation code kept as safety net for passwords >72 bytes Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix bcrypt parameter name: use truncate_error instead of relax_truncate_checks - Changed bcrypt__relax_truncate_checks=True to bcrypt__truncate_error=False - The correct passlib parameter is truncate_error (not relax_truncate_checks) - Setting it to False allows passwords >72 bytes without raising an error Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * fix(passwords) downgrade bcrypt to work with current passlib * chore(uv): update locks file --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
…ation (#11) * Initial plan * Add Phase 2 authentication service - auth dependencies, router, and service integration Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Address code review feedback - add token expiration constants and improve documentation Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix email validation to allow special-use domains like .local for testing Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix system user to not be admin - allows /auth/setup to work on fresh database Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore: typegen --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
…ddleware (#15) * Initial plan * Add Phase 3 integration tests for auth router endpoints Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add Phase 3 completion documentation and testing guide Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore: ruff --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
* Initial plan * Update boards service for multi-user support - Add user_id parameter to board creation and filtering - Update board_records base and SQLite implementation to filter by user - Include shared boards and public boards in user queries - Update boards router to require authentication - Update invocation context to pass user_id from queue item - Add user_id field to SessionQueueItem model Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Update session_queue service for multi-user support - Add user_id parameter to enqueue_batch method - Update ValueToInsertTuple to include user_id - Update prepare_values_to_insert to accept and use user_id - Update SQL INSERT statements to include user_id column - Update session_queue router to require authentication - Update retry_items_by_id to preserve user_id when retrying Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add authentication imports to images, workflows, and style_presets routers - Import CurrentUser dependency in all three routers - Add CurrentUser parameter to upload_image endpoint - Prepare for further authentication requirements in these services Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add multiuser tests and Phase 4 documentation - Create test_boards_multiuser.py with authentication tests - Verify board operations require authentication - Test board creation and listing with auth tokens - Add comprehensive Phase 4 implementation summary - Document all changes, data flows, and security considerations Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Add authentication to remaining board endpoints - Add CurrentUser to get_board endpoint - Add CurrentUser to update_board endpoint - Add CurrentUser to delete_board endpoint - Ensures all board operations require authentication - Addresses code review feedback Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Feature(image boards): Implement per-user board isolation - Complete verification report with all checks passed - Document code quality, security, and testing results - List all achievements and sign-off criteria - Mark phase as READY FOR MERGE Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore: ruff * chore: resolve conflicts in z_image_working_memory test * chore: ruff --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
) * Phase 5: Implement frontend authentication infrastructure - Created auth slice with Redux state management for authentication - Created auth API endpoints (login, logout, setup, me) - Created LoginPage component for user authentication - Created AdministratorSetup component for initial admin setup - Created ProtectedRoute wrapper for route authentication checking - Updated API configuration to include Authorization headers - Installed and configured react-router-dom for routing - Updated App component with authentication routes - All TypeScript checks passing Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore(style): prettier, typegen and add convenience targets to makefile --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
…rictions Co-authored-by: lstein <111189+lstein@users.noreply.github.com> docs: Add comprehensive testing and verification documentation for Phase 6 Co-authored-by: lstein <111189+lstein@users.noreply.github.com> docs: Add Phase 6 summary document Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com> feat: Add admin authorization to model management API endpoints Co-authored-by: lstein <111189+lstein@users.noreply.github.com> docs: Update specification and implementation plan for read-only model manager Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
…authentication (#23) * Initial plan * Phase 7: Complete test suite with 88 comprehensive tests - Add password utils tests (31 tests): hashing, verification, validation - Add token service tests (20 tests): JWT creation, verification, security - Add security tests (13 tests): SQL injection, XSS, auth bypass prevention - Add data isolation tests (11 tests): multi-user data separation - Add performance tests (13 tests): benchmarks and scalability - Add comprehensive testing documentation - Add phase 7 verification report Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * bugfix(backend): Fix issues with authentication token expiration handling - Remove time.sleep from token uniqueness test (use different expiration instead) - Increase token expiration test time from 1 microsecond to 10 milliseconds - More reliable test timing to prevent flakiness Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Add Phase 7 summary documentation Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix test_performance.py missing logger fixture Add missing logger fixture to test_performance.py that was causing test failures. The fixture creates a Logger instance needed by the user_service fixture. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Add board isolation issue specification document Document the three board isolation issues that need to be addressed: 1. Board list not updating when switching users 2. "Uncategorized" board shared among users 3. Admin cannot access all users' boards Includes technical details, implementation plan, and acceptance criteria. This document will be used to create a separate GitHub issue and PR. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix test failures in Phase 7 tests - Fix board service API calls to use enum values (BoardRecordOrderBy, SQLiteDirection) instead of strings - Fix board deletion test to use get_dto() instead of non-existent get() method - Add exception handling to verify_password() for invalid hash formats - Update SQL injection test to accept both 401 and 422 status codes (Pydantic validation) All fixes ensure tests match actual API signatures and handle edge cases properly. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix token forgery test to properly decode and modify JWT payload The test was attempting to modify the JWT payload by string replacement on the base64-encoded data, which doesn't work since "false" doesn't appear literally in the base64 encoding. Fixed to: - Properly decode the base64 payload - Parse the JSON - Modify the is_admin field - Re-encode the payload - Create a forged token with the modified payload and original signature - Verify it's rejected with 401 status This properly tests that JWT signature verification prevents token forgery. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix test_verify_expired_token timing issue Increased token expiration from 10ms to 100ms and wait time from 20ms to 200ms to make the test more reliable. The previous timing was too tight and could cause race conditions where the token hadn't actually expired yet when verified. This ensures the test reliably validates that expired tokens are properly rejected. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix token verification to properly validate signatures and expiration The verify_token() function was not explicitly enabling JWT verification options, causing it to accept expired tokens and tokens with modified signatures/payloads. Fixed by explicitly setting verification options: - verify_signature: True (reject modified tokens) - verify_exp: True (reject expired tokens) - verify_nbf: True (verify "not before" claim) - verify_iat: True (verify "issued at" claim) - verify_aud: False (we don't use audience claims) Also added catch-all exception handler for other validation errors. This ensures all token security tests pass properly. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Revert to default python-jose verification behavior Removed explicit options parameter from jwt.decode() call since python-jose verifies signatures and expiration by default. The explicit options may have been causing issues or conflicts with the library's default behavior. Added comments clarifying that python-jose defaults are secure and handle verification properly. Need user to test and provide debugging output if issues persist. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Fix JWT expiration verification for python-jose 3.5.0 python-jose 3.5.0 has a bug where expiration verification doesn't work properly by default. The jwt.decode() function is not rejecting expired tokens even when they should be rejected. Workaround implemented: 1. First, get unverified claims to extract the 'exp' timestamp 2. Manually check if current time >= exp time (token is expired) 3. Return None immediately if expired 4. Then verify signature with jwt.decode() for tokens that aren't expired This ensures: - Expired tokens are properly rejected - Signature verification still happens for non-expired tokens - Modified tokens are rejected due to signature mismatch All three failing tests should now pass: - test_verify_expired_token - test_verify_token_with_modified_payload - test_token_signature_verification Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * Fix race condition in token verification - verify signature before expiration Changed the order of verification in verify_token(): 1. First verify signature with jwt.decode() - rejects modified/forged tokens 2. Then manually check expiration timestamp Previous implementation checked expiration first using get_unverified_claims(), which could cause a race condition where: - Token with valid payload but INVALID signature would pass expiration check - If expiration check happened to return None due to timing, signature was never verified - Modified tokens could be accepted intermittently New implementation ensures signature is ALWAYS verified first, preventing any modified tokens from being accepted, while still working around the python-jose 3.5.0 expiration bug by manually checking expiration after signature verification. This eliminates the non-deterministic test failures in test_verify_token_with_modified_payload. Co-authored-by: lstein <111189+lstein@users.noreply.github.com> * chore(app): ruff --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: lstein <111189+lstein@users.noreply.github.com> Co-authored-by: Lincoln Stein <lincoln.stein@gmail.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Co-authored-by: lstein <111189+lstein@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add option to run models on CPU in settings
Add per-model CPU execution option
Jan 21, 2026
github-actions
bot
added
root
python-deps
python
python-tests
services
backend
frontend
api
docs
frontend-deps
ci-cd
labels
Owner
|
@copilot This PR has a bunch of undesired commits from lstein-master, which has the code needed for multiuser support. I need to have the commmits in this PR made relative to the |
Author
I've rebased the branch locally to contain only the 3 CPU-related commits on top of cd /path/to/InvokeAI
git fetch origin copilot/add-cpu-model-run-option
git checkout copilot/add-cpu-model-run-option
git push origin copilot/add-cpu-model-run-option --forceThis will remove the 40+ multiuser commits and leave only:
|
Owner
|
@copilot This did not work for me either. I will try checking out a fresh copy of main locally and cherry pick the CPU related commits myself. Can you list the CPU-related commits in the order in w hich they should be applied? |
Author
Here are the 3 CPU-related commits in the order they should be cherry-picked:
Command to cherry-pick: git cherry-pick 82047ef f3510e0 88585e2 |
Owner
|
Got rebased into wrong branch and too hard to figure out. Superseded by pr #46 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implementation Complete ✅
This PR adds the ability to configure models to run on CPU instead of GPU on a per-model basis.
Changes Made
Backend:
cpu_onlyfield toMainModelDefaultSettingsModelCache.put()to accept optionalexecution_deviceparameterModelLoader._load_and_cache()to pass device from model configFrontend:
How to Use
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.