Show HN: Mini-AGI – Dynamic continual learning model trained on 8GB VRAM

The author has developed a dynamic continual learning model, dubbed 'Mini-AGI,' that can be trained on 8GB of VRAM. The model is designed to be efficient and allows for full control over the data it sees during training, without the need for large amounts of disk space or expensive hardware. The author used AI tools to develop the project and hopes to explore two new ideas: using a modular expert (MoE) approach with dynamically added and pruned experts, and training on a single continuous stream of data to reduce VRAM requirements. The model is currently training on a 7.8B character corpus and is expected to take a couple of weeks to complete.