Question/Proposal: Option to end sending requests gracefully and retain successful requests

[mychiffonn]

Use case:
Some APIs, like DeepSeek R1 and OpenAI, can get busy and don't respond to requests after some time. I want an option to gracefully stop sending requests AND also retain data of successful requests.

I'm looking through .cache/curator and I only see an .arrow file associated with a successful request load. It would be nice if:

• We can download data from curator viewer or
• Data of successful requests are added to a database as they're done, instead of waiting on a long request chain

If this feature is somewhat available, how should I use it? Thanks!

[madiator]

Thanks for this request. We will add this feature!

[RyanMarten]

@kartik4949 The simplest thing I am thinking we can do is have the user CTL+C out of the program if they want to stop generating, and then in the code add a flag to the curator.LLM which signifies no new requests to be sent + load the successful requests from the cache.

[pascalwhoop]

does curator cache online calls at all? My .cache file only contains jsonl files from what seems to be my batch trials. Note I am mostly using gemini/litellm

[madiator]

It should! If you rerun the same call do you see it returns quickly from cached files or not? Sent via Superhuman Android ( ***@***.*** )

…

On Mon, May 26 2025 at 10:23 AM, Pascal Bro < ***@***.*** > wrote: *pascalwhoop* left a comment (bespokelabsai/curator#618) ( #618 (comment) ) does curator cache online calls at all? My .cache file only contains jsonl files from what seems to be my batch trials. Note I am mostly using gemini/litellm — Reply to this email directly, view it on GitHub ( #618 (comment) ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AAIJX2ZN2PIO2GMH4EUVY6T3ANEY3AVCNFSM6AAAAAB2N7L5J6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSMJQGMYTKMRUGQ ). You are receiving this because you commented. Message ID: <bespokelabsai/curator/issues/618/2910315244 @ github. com>