Four pre-existing flakes on main that gate this branch's workflow even
though they're unrelated to the reasoning_effort_grid suite:
1. tests/local_testing/test_completion.py::test_completion_fireworks_ai
2. tests/local_testing/test_completion_cost.py::test_completion_cost_fireworks_ai[fireworks_ai/llama-v3p3-70b-instruct]
3. tests/llm_translation/test_fireworks_ai_translation.py::test_document_inlining_example[False]
The Fireworks-hosted `llama-v3p3-70b-instruct` deployment is currently
returning 404 "Model not found, inaccessible, and/or not deployed".
These tests pass when the model is deployed; the issue is upstream
capacity, not our code path. Wrap the live call in a try/except that
pytest.skip's on litellm.NotFoundError so a Fireworks deployment hiccup
no longer fails CI for unrelated PRs.
4. tests/llm_translation/test_gemini.py::test_gemini_image_size_limit_exceeded
The test fetches the 32MB "Blue Marble 2002" image from Wikimedia to
exercise the 50MB image-size cap. CI runners share an IP pool with
noisy traffic, so Wikimedia routinely returns HTTP 429. The size-limit
check never gets a chance to fire. Catch the 429 BadRequestError and
pytest.skip in that case.
None of these belong on this PR conceptually, but they're included per
request to unblock the workflow before morning.
Mateo Wang
2b00ea9ee4