RSS Bot@lemmy.bestiver.seBM to Hacker News@lemmy.bestiver.seEnglish · 1 day agoLaya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)gist.github.comexternal-linkmessage-square2linkfedilinkarrow-up11arrow-down10file-text
arrow-up11arrow-down1external-linkLaya (OS Jev) on Mac M4 CoreML Offline (45 decisions per second)gist.github.comRSS Bot@lemmy.bestiver.seBM to Hacker News@lemmy.bestiver.seEnglish · 1 day agomessage-square2linkfedilinkfile-text
minus-squaremira_tokarev@lem.cochrun.xyzlinkfedilinkEnglisharrow-up1·1 day agoWhether hosted Jev or on-device Laya, high-frequency decisions amplify whatever you put in state. Before optimizing another decisions/sec, log tokens(state) separately and run a boring deterministic Tier-1 shrink. Complementary to the model itself — not a replacement. contextpress: https://github.com/Taha-azizi/contextpress · https://pypi.org/project/contextpress/
Whether hosted Jev or on-device Laya, high-frequency decisions amplify whatever you put in
state.Before optimizing another decisions/sec, log tokens(state) separately and run a boring deterministic Tier-1 shrink. Complementary to the model itself — not a replacement. contextpress: https://github.com/Taha-azizi/contextpress · https://pypi.org/project/contextpress/