Ready-made infrastructure for AI pipelines
Your run says 1,000 succeeded.
Thirty of them didn't.
They returned a value. Nothing raised, nothing retried, the exit code was 0 — and the answer was wrong. You find out from a customer, two weeks later, and the smallest thing you can re-run is the whole batch.
Papayya grades every item, keeps the input that produced it, and lets you re-drive the one that broke. Queue, retries, per-item state, per-tenant cost — the scaffolding you were going to write around your pipeline anyway, already built.
Here are the thirty.
sample data · the interaction is the product's
One run of tag_tickets. A thousand tickets, four tenants. The run exited 0. 30 of them did not work — and 21 of those finished cleanly and returned a wrong answer, so nothing raised and nothing was retried. Filter it. Re-drive one.
| Item | Tenant | Ran | Worked | Cost | |
|---|---|---|---|---|---|
| ticket_8168openai.rate_limit | acme | completed | degraded | $0.0021 | |
| ticket_8187openai.rate_limit | initech | completed | degraded | $0.0020 | |
| ticket_8209tool.invalid_input | umbrella | failed | failed | $0.0021 | |
| ticket_8268openai.rate_limit | initech | completed | degraded | $0.0025 | |
| ticket_8269tool.invalid_input | acme | failed | failed | $0.0020 | |
| ticket_8357empty_response | umbrella | completed | degraded | $0.0025 | |
| ticket_8368openai.rate_limit | initech | completed | degraded | $0.0019 | |
| ticket_8402provider.timeout | globex | failed | failed | $0.0025 | |
| ticket_8417empty_response | globex | completed | degraded | $0.0022 | |
| ticket_8431tool.invalid_input | initech | failed | failed | $0.0021 | |
| ticket_8451empty_response | globex | completed | degraded | $0.0021 | |
| ticket_8459tool.invalid_input | initech | failed | failed | $0.0019 | |
| ticket_8466schema_violation | globex | completed | degraded | $0.0019 | |
| ticket_8474schema_violation | acme | completed | degraded | $0.0019 | |
| ticket_8510openai.rate_limit | globex | completed | degraded | $0.0018 | |
| ticket_8576provider.timeout | globex | failed | failed | $0.0024 | |
| ticket_8587schema_violation | umbrella | completed | degraded | $0.0026 | |
| ticket_8621schema_violation | globex | completed | degraded | $0.0023 | |
| ticket_8694empty_response | initech | completed | degraded | $0.0024 | |
| ticket_8735schema_violation | globex | completed | degraded | $0.0019 | |
| ticket_8810tool.invalid_input | globex | failed | failed | $0.0019 | |
| ticket_8860openai.rate_limit | initech | completed | degraded | $0.0025 | |
| ticket_8938provider.timeout | globex | failed | failed | $0.0021 | |
| ticket_8942openai.rate_limit | acme | completed | degraded | $0.0024 | |
| ticket_8977empty_response | acme | completed | degraded | $0.0022 | |
| ticket_8991schema_violation | globex | completed | degraded | $0.0020 | |
| ticket_9011schema_violation | umbrella | completed | degraded | $0.0023 | |
| ticket_9037empty_response | umbrella | completed | degraded | $0.0018 | |
| ticket_9051provider.timeout | initech | failed | failed | $0.0025 | |
| ticket_9056openai.rate_limit | initech | completed | degraded | $0.0026 |
A 150-page report. Page 73 came back blank.
This is a real call, and the numbers are measured rather than imagined. What changes is not the speed — it is whether there is anything smaller than the document to ask about.
| The pipeline they have | On Papayya | |
|---|---|---|
| What can you name? | The document. 150 pages arrive as one row, so there is no page 73 to point at. | Page 73. Every page is an item with its own id, input, verdict and cost. |
| How do you fix it? | Re-run the document. All 150 pages execute again — 300 page executions to change one. | Re-drive page 73. The 149 steps that already worked are reused, not recomputed. |
| What does that cost? | Two days in the queue, and the other 149 pages written downstream a second time. | 40 seconds. $0.002. Nothing downstream moves that did not need to. |
| How did you know? | A customer told you. The run reported success. | The item is graded on the way through, so page 73 was in the work list before the call. |
150 pages · 300 page executions to fix one · vs 1 executed, 149 reused · 40s · $0.002