Settings & Administration
This section guides you through managing your personal settings, organization administration, billing management, AI configuration, repository integration, and security features. Each area is designed to give you control over your experience and, for administrators, the ability to manage your organization's setup and integrations.
Organization Management (Admin)
Administrators have access to comprehensive tools for managing organizations, members, billing, and enterprise features.
Organization Settings
- Organization Profile: Manage organization name, subdomain (yourcompany.testworthy.us), and basic settings
- Custom Subdomain: Each organization gets its own isolated workspace with custom URL
- Data Isolation: Complete separation of data between organizations for security and compliance
- Organization Status: Control active/inactive status and manage organization lifecycle
Member Management & Roles
- Create Users: Administrators can create new users directly with email, name, password, role, and optional credit quota allocation
- Role Assignment: Assign roles during user creation or modify existing user roles
- Access Control: Manage member access with role-based permissions
- Credit Allocation: When creating or editing users, allocate monthly credit quota and purchased credits from the organization pool
- Member Removal: Delete users with automatic project reassignment and cascade cleanup
Available Roles
- Platform Admin: System-wide administrative access across all organizations (can only be assigned by other platform admins)
- Tenant Owner: Full ownership and control of the organization, including billing, user management, and credit distribution. Cannot demote the last tenant owner.
- Project Manager: Project management, user coordination, API key management, and integration configuration
- QA: Full testing capabilities with test creation, execution, and AI feature access
- Developer: Test artifact CRUD access (same capabilities as QA)
- Guest: Read-only access across the organization
User Creation Constraints
- Email must be unique across the platform
- Password must be 8+ characters with uppercase, digit, and special character
- User count cannot exceed the organization's subscription tier limit (
max_users) - Credit allocation must fit within the organization's available pool
- User creation is blocked when subscription status is
pending; allowed duringcanceledgrace period (3 days)
Organization Quotas & Limits
- User Limits: Maximum number of users determined by subscription tier (or custom override via
custom_max_users) - Credit Quotas: Organization-level AI credit pool distributed across members by tenant owner
- Purchased Credits: Organization-level purchased credit balance, allocatable to individual users
Billing & Subscription Management (Admin)
Comprehensive billing management with transparent pricing and flexible subscription options.
Subscription Plans
- Trial: Free tier for up to 3 users, 350 AI credits/month — all features included
- Starter ($49/month): Up to 10 users, 3,500 AI credits/month — all features included
- Standard ($99/month): Up to 20 users, 9,000 AI credits/month — all features included
- Professional ($199/month): Unlimited users, 12,000 AI credits/month — all features included
All plans include the full feature set: AI test generation, GitHub/GitLab/Bitbucket integration, Jira integration, automation script export, and email support.
All paid plans support both monthly and yearly billing cycles via Stripe, with a 20% yearly discount available.
Billing Management
Subscription Lifecycle
- Plan Selection: Choose and upgrade/downgrade between subscription tiers
- Billing Cycle: Select monthly or yearly billing with automatic proration on plan changes
- Payment Methods: Manage credit cards and payment information through Stripe
- Invoicing: Access detailed invoices with PDF download capability via Stripe-hosted invoice URLs
- Payment History: View paginated payment history with transaction details (subscription payments, credit purchases, plan upgrades)
Subscription Controls
- Upgrade/Downgrade: Change plans via Stripe subscription update with automatic proration handling
- Cancellation: Cancel subscription with grace period—service continues until the current billing period ends (
cancel_at_period_end) - Resume Subscription: Undo cancellation before the end of the billing period
- Revert to Trial: Downgrade a pending unpaid subscription back to the free trial tier
- Checkout Flow: Stripe Checkout sessions handle secure payment collection with redirect-based flow
Credit Economy & AI Usage
Dual-Level Credit Architecture
- Organization Pool: Main credit quota inherited from the subscription tier (350–12,000 credits/month)
- User Allocation: Tenant owners can distribute the organization pool across individual users via the Manage Allocations modal
- Purchased Credits: One-time credit top-ups that expire after 365 days (FIFO deduction, oldest first)
- Priority Deduction: Monthly quota is consumed first; purchased credits are used only after monthly quota is exhausted
Credit Quotas
- Monthly Allocation: Each subscription tier includes a monthly AI credit quota (Trial: 350, Starter: 3,500, Standard: 9,000, Professional: 12,000)
- Usage Tracking: Monitor credit consumption across all AI operations with per-user attribution
- Quota Reset: Automatic monthly reset on the 1st of each month
- Purchased Credits: Buy additional credit packages via Stripe for heavy AI usage; these expire after 365 days
- Multi-User Allocation: Tenant owners can adjust individual user quotas within the organization pool using the Manage Allocations modal
- Quota Scaling: When the subscription tier changes,
scale_user_quotas()proportionally adjusts all user quotas
Usage Analytics
- Operation Breakdown: Detailed tracking by AI operation type:
project_import— AI-powered codebase analysis and project generationchunk_processing— Processing code chunks during importheal_test_case— AI test case improvement and healingaggregate_chunks— Aggregating analysis resultsgithub_integration— Pull request analysis and test recommendationsgenerate_automation_script— Playwright/Selenium/Cypress script generation
- AI Provider & Model Tracking: Each operation records the AI provider (Vertex AI, Anthropic via Vertex AI, Llama) and specific model used
- Cost Estimation: Estimated USD cost calculated per operation based on input/output token counts
- Historical Usage: Full audit trail via the
UserTokenUsagetable with timestamps for reporting - Billing Integration: Credit purchases appear as payment records alongside subscription payments
Billing Portal Access
- Self-Service Portal: Direct access to Stripe customer portal for payment management
- Invoice Management: Download invoices and manage billing information
- Payment Method Updates: Update credit cards and billing addresses
- Subscription History: View complete subscription and payment history
Subscription Status Banners
The platform displays prominent banners across all pages to alert users about subscription issues:
| Status | Severity | Message | Action |
|---|---|---|---|
| Canceled | Critical (Red) | Subscription has been canceled | Resubscribe |
| Past Due | Warning (Yellow) | Payment is overdue | Update Payment |
| Cancellation Scheduled | Notice (Orange) | Subscription will cancel at period end | Resume Subscription |
| Trialing | Info (Blue) | Currently on trial period | Upgrade Plan |
| Pending | Warning (Yellow) | Subscription payment is pending | Go to Credits & Billing |
Banners are dismissible per-status and reset when the subscription status changes. All CTAs navigate to the Credits & Billing page.
Model Pricing Management (Platform Admin)
Platform administrators can configure AI model pricing through a dedicated management page.
Accessing Model Pricing
Navigate to Model Pricing in the sidebar (visible to Platform Admins only).
Configuration
- Per-Model Pricing: Configure input token cost, output token cost, and credit charge for each AI model
- CRUD Operations: Create, read, update, and delete model pricing entries
- Cost Transparency: Model pricing determines the credit cost of each AI operation based on actual token consumption
AI Provider Configuration
Configure which AI provider and model the platform uses for your AI-powered operations. Each user can set their own preferred provider and model.
How AI Providers Work
AI provider API keys are configured at the platform level via environment variables by the system administrator. Individual users do not need to supply their own API keys — the platform manages credentials centrally.
Supported Providers:
| Provider | Key Source | Notes |
|---|---|---|
| Vertex AI | Google service account credentials (GOOGLE_APPLICATION_CREDENTIALS) | Default provider; Gemini models via GCP |
| Anthropic (via Vertex AI) | Google service account credentials | Claude models routed through GCP |
| Llama (via Vertex AI) | Google service account credentials | Llama models routed through GCP |
| Llama (via OpenRouter) | OPENROUTER_API_KEY environment variable | Llama models via OpenRouter |
Available Models
Models are curated per provider and defined in the platform frontend. Available models include:
- Vertex AI: Gemini 3.1 Pro Preview, Gemini 3.1 Flash Lite Preview, Gemini 2.5 Pro, Gemini 2.5 Flash
- Anthropic (via Vertex AI): Claude 4.5 Haiku, Claude 4.5 Sonnet
- Llama (via Vertex AI): Llama 4 Maverick 17B, Llama 4 Scout 17B
- Llama (via OpenRouter): Llama 4 Maverick, Llama 4 Scout
Model availability may be updated as new models are released.
Per-Operation Override
When performing specific AI operations (e.g., generating automation scripts or processing code imports), you can override the provider and model for that single operation without changing your default settings.
Prompt Templates
AI prompts are managed as a static configuration file (ai_prompts.json) by platform administrators. Prompts are defined for each operation type:
- chunk_processing — Code analysis and test case generation from uploaded codebases
- aggregate_deduplication — Merging and deduplicating AI-generated test results
- test_case_healing — AI-powered test case improvement suggestions
- script_generation — Automation script generation (Playwright/Selenium/Cypress)
- github_integration — Pull request analysis and test recommendations
Prompt templates use variable substitution (e.g., {project_name}, {chunk_content}, {test_case_json}) and are not editable through the UI.
Repository Integration
Connect your code repositories for automated pull request analysis and test recommendations.
Supported Platforms
- GitHub: Full integration with GitHub.com and GitHub Enterprise
- GitLab: Support for GitLab.com and self-hosted GitLab instances
- Bitbucket: Full integration with Bitbucket Cloud
Repository Setup
Connection Process
- Access Tokens: Generate personal access tokens or OAuth tokens for repository access
- Repository Selection: Choose specific repositories to integrate with your projects
- Webhook Configuration: Automatic webhook setup for real-time event processing
- Permissions Setup: Configure appropriate access permissions for code analysis
- Branch Filtering: Select which branches trigger automated analysis
Security Configuration
- Token Encryption: Repository access tokens are encrypted and securely stored
- Webhook Secrets: Secure webhook validation with automatically generated secrets
- Access Scope: Minimum required permissions for repository integration
- Audit Logging: Complete logging of all repository integration activities
Pull Request Analysis
Automated Analysis Features
- Change Detection: Automatic analysis of code changes in pull requests
- Impact Assessment: AI-powered evaluation of testing impact from code changes
- Test Recommendations: Intelligent suggestions for new or updated test cases
- Confidence Scoring: AI confidence ratings for each recommendation
- Auto-Creation: Optional automatic creation of recommended test cases
Analysis Configuration
- Trigger Rules: Configure which PR events trigger analysis (opened, synchronized, etc.)
- File Patterns: Include or exclude specific file types from analysis
- Analysis Depth: Control how thoroughly PRs are analyzed for performance optimization
Integration Management
- Connection Status: Monitor active repository connections and health
- Usage Analytics: Track integration usage and performance metrics
- Error Handling: Automatic retry and error reporting for failed operations
- Integration Logs: Detailed logging for troubleshooting integration issues
Security & Authentication
Enterprise-grade security features for protecting your testing data and maintaining compliance.
Authentication Methods
- Email/Password: Standard authentication with strong password policies
- OTP Verification: Two-factor authentication via email for enhanced security
- JWT Tokens: Secure session management with automatic token refresh
Password Security
- Password Policies: Configurable password strength requirements (8+ characters, uppercase, digit, special character)
- Password Reset: Secure password reset via OTP verification
Security Monitoring
- Audit Logs: Comprehensive logging of all security-related events
- Activity Tracking: Monitor user actions across the platform
Compliance Features
- Data Encryption: Sensitive data encrypted at rest (API tokens, credentials)
- Activity Audit: Complete audit trail for all user actions
Email Notifications
Testworthy sends automated email notifications for key platform events:
| Notification Type | Description |
|---|---|
| Welcome Email | Sent to new users upon account creation |
| OTP Verification | One-time password for two-factor authentication |
| Password Reset | Secure password reset link via email |
| Credit Warning | Alert when credit quota approaches exhaustion |
| Trial Ending | Notification before trial period expires |
| Payment Receipt | Confirmation after successful payment |
| New User Notification | Admin alert when a new user is created |
| Scheduled Report | Automated report delivery (daily, weekly, or monthly) |
User Profile Management
Personalize your profile settings and manage your account preferences.
Updating Your Personal Profile
- Access your profile settings by clicking on your avatar or profile icon in the application header
- Update your display name and email address
- Save your updates to ensure your profile information is current
Account Security
- Change Password: Update your password (must meet policy: 8+ characters, uppercase, digit, special character)
- Theme Preferences: Switch between light and dark themes with automatic persistence
Personal AI & Credit Settings
- AI Provider & Model: Select your preferred AI provider and model from the AI Settings page (see AI Provider Configuration)
- Credit Usage Overview: View 4 key metrics — monthly quota, used this month, remaining, and reset date
- Usage Progress Bar: Visual indicator showing current quota utilization
- Operation Breakdown: See credit consumption per AI operation type (project import, chunk processing, test healing, script generation, GitHub integration)
- Lifetime Statistics: Track total credits consumed across all time
- Org Pool View (Tenant Owner): See the organization's total monthly allocation, allocated credits across users, available pool, and purchased credit balance
By following these instructions, you can fully manage your organization settings, billing, AI configuration, repository integrations, and security features, ensuring the platform is tailored to your organization's needs and compliance requirements. If you encounter any issues or need further assistance, refer to the help documentation or contact your administrator.