Great roundup! I'm working on combining language models with formal methods (e.g., via intermediate code generation) to try and make them more robust but it is definitely proving extra hard because even if you embed the actual correct answer in the prompt, the LM can still bypass it completely and pull from its faulty, hallucinating-prone long-memory memory. One approach I'm very excited about is grammar-restricted output.
Great roundup! I'm working on combining language models with formal methods (e.g., via intermediate code generation) to try and make them more robust but it is definitely proving extra hard because even if you embed the actual correct answer in the prompt, the LM can still bypass it completely and pull from its faulty, hallucinating-prone long-memory memory. One approach I'm very excited about is grammar-restricted output.