"As a Language Model": Chat Template Switches LLM Self-Referential Voice

Researchers have proposed a method to allow language models to refer to themselves in a more natural way. They created a chat template that enables a self-referential voice, allowing the model to discuss its own capabilities, limitations, and even its own responses. The approach is based on a combination of techniques, including knowledge graph embeddings and language model pre-training. The researchers tested the method on several language models, including BERT and RoBERTa, and observed improved performance in self-referential tasks. The proposed method is seen as a step towards more human-like language understanding and generation in language models.