Feature request: make cache dir configurable directly instead of through environment variables about tiktoken HOT 2 CLOSED

Also the caching is very undocumented!

from tiktoken.

Thanks for the report! Since you posted this issue there have been a couple relevant improvements. We no longer use blobfile in most cases; there's much less scope for an azure hang with just a plain unauthenticated http request. I also changed the default cache location a little for better cross-platform support.

By "pass in the directory directly", where exactly are you envisioning passing that? I'm unlikely to complicate interfaces for this feature request (unless it proves popular). Some workarounds in the meantime:

• Construct the Encoding objects directly, using whatever file loading logic you'd like
• Assign to os.environ before calling get_encoding

Thanks for the feedback on documentation. There aren't really dedicated docs for tiktoken right now, when I get time to add some, I'll make sure to mention this somewhere.

from tiktoken.