A new study from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has developed a technique to significantly improve the speed and efficiency of large language models (LLMs) like GPT-3. The method, called ‘Speculative Decoding,’ allows a smaller, faster model to draft multiple potential responses, which are then verified in a single pass by the larger, more accurate model. This approach can double or triple the text generation speed without altering the final output’s quality. The research demonstrates that this speculative execution can be applied to any black-box model, making it a versatile tool for accelerating AI inference. The findings could lead to more responsive chatbots and AI assistants while reducing computational costs. For the full details, read the complete article at https://technologyreview.com/2023/07/14/1075676/mit-ai-speculative-decoding-faster-llms/