--- name: mithril-cache-agent description: Build mithril-cache for torch.compile caching. Use when implementing content-addressable storage, cache keys, eviction, or framework hooks. --- # Mithril Cache Agent Build compilation caching to reduce torch.compile cold starts from 67s to <1s. ## Status Read `crates\/mithril-cache\/STATUS.md` for current progress. ## Reference Documentation - `cache\/SPEC.md` - Full product specification - `RESEARCH.md` - References (sccache, PyTorch codecache, Triton cache) ## Module Responsibilities ### cas (Content-Addressable Storage) Store artifacts by content hash: ```rust pub struct ContentStore { root: PathBuf, hasher: Blake3Hasher, } impl ContentStore { pub fn new(root: impl AsRef) -> Result; \/\/\/ Store content, return its address (hash) pub async fn put(&self, content: &[u8]) -> Result; \/\/\/ Retrieve content by address pub async fn get(&self, address: &str) -> Result>>; \/\/\/ Check if address exists pub async fn exists(&self, address: &str) -> Result; } ``` ### keys Cache key generation (machine-independent): ```rust pub struct CacheKey { pub graph_hash: [u8; 32], pub inputs: Vec, pub device: DeviceClass, } impl CacheKey { pub fn to_storage_key(&self) -> String; } pub struct InputSpec { pub shape: Vec, pub dtype: DType, } pub enum DeviceClass { CudaAny, CudaCompute { major: u8, minor: u8 }, Cpu, } ``` ### eviction LRU eviction for local cache: ```rust pub struct LruCache { max_size_bytes: u64, entries: LinkedHashMap, current_size: u64, } impl LruCache { pub fn new(max_size_bytes: u64) -> Self; pub fn get(&mut self, key: &str) -> Option<&CacheEntry>; pub fn put(&mut self, key: String, entry: CacheEntry); fn evict_if_needed(&mut self); } ``` ### hooks Framework integration via environment variables: ```rust pub fn init(config: CacheConfig) -> Result { let manager = CacheManager::new(config)?; \/\/ Intercept via environment variables std::env::set_var("TORCHINDUCTOR_CACHE_DIR", manager.inductor_dir()); std::env::set_var("TRITON_CACHE_DIR", manager.triton_dir()); Ok(manager) } ``` **MVP Strategy**: Don't hook internal PyTorch APIs. Just redirect cache directories. ## Target Metrics | Metric | Target | |--------|--------| | Local lookup | <10ms | | Cold start (cache hit) | <1s | | Cache hit rate | ≥80% | ## Key Dependencies ```toml mithril-core = { workspace = true } blake3 = { workspace = true } tokio = { workspace = true } ``` ## Testing ```bash cargo test -p mithril-cache cargo bench -p mithril-cache ``` ## Implementation Order 1. Implement `cas` module (content-addressable storage) 2. Implement `keys` module (cache key generation) 3. Implement `eviction` (LRU) 4. Implement `hooks` (env var interception) 5. Integration test with synthetic data 6. Update STATUS.md ## Completion Criteria - [ ] CAS stores\/retrieves artifacts correctly - [ ] LRU eviction works - [ ] <10ms local lookup - [ ] Unit tests pass - [ ] STATUS.md updated to COMPLETE ## Notes - **Don't integrate deeply with PyTorch internals** - they're unstable - Start with env var interception (shallow integration) - Remote cache (S3\/GCS) is post-MVP