This directory contains the testing suite for the Webscout project.

These tests use mocks to simulate API responses and are safe to run in CI environments.

• tests/providers/test_discovery.py: Verifies that all providers are importable and inherit from the base Provider class.
• tests/providers/test_all_mocked.py: Tests the ask method of all providers using generic mocks to ensure no crashes.
• tests/providers/test_<provider>.py: Specific unit tests for major providers (OpenAI, Groq, Gemini, etc.).

uv run pytest tests/providers/

The tests/live_test.py script provides a CLI for performing live tests against real AI providers.

• List all available providers and their models.
• Test a specific provider and model.
• Test all providers at once.
• Test all models of a specific provider.
• Support for API keys via CLI or JSON file.

List all providers:

uv run python tests/live_test.py --list

Test a specific provider:

uv run python tests/live_test.py --provider OpenAI --api-key YOUR_KEY

Test all models of a provider:

uv run python tests/live_test.py --provider Groq --api-key YOUR_KEY --test-models

Test all providers (live):

uv run python tests/live_test.py --test-all --api-keys-file keys.json

{
    "OpenAI": "sk-...",
    "Groq": "gsk_...",
    "default": "some-generic-key"
}

• tests/providers/utils.py: Contains FakeResp, a mock response object for testing.