What happens when we give LLMs access to thousands of deep learning models? — Recently, we have witnessed a rise of foundation models to popularity within deep learning research. Pre-trained large language models (LLMs) have led to a new paradigm, in which a single model can be used — with surprising success — to solve many different problems. Despite the popularity of generic LLMs…

Using tools makes us more capable as humans. Is the same true of LLMs? — As we learn more about them, large language models (LLMs) become increasingly interesting. These models can solve a variety of complex tasks accurately. At the same time, however, they struggle with certain functionality that we, as humans, consider basic! For example, LLMs commonly make arithmetic mistakes, lack access to current…

LLMs can write code, but what if they can execute programs? — Although Large Language Models (LLMs) are used for a variety of applications, they have typically struggled to solve reasoning-based tasks. This issue was significantly diminished with the advent of prompting techniques like Chain of Thought and Least-to-Most prompting. At a high level, these techniques encourage reasoning behavior in LLMs by…

Simple strategies for getting the most out of any language model… — Anyone who has worked with large language models (LLMs) will know that prompt engineering is an informal and difficult process. Small changes to a prompt can cause massive changes to the model’s output, it is difficult (or even impossible in some cases) to know the impact that changing a prompt…

What to do when few-shot learning isn’t enough… — The popularization of large language models (LLMs) has completely shifted how we solve problems as humans. In prior years, solving any task (e.g., reformatting a document or classifying a sentence) with a computer would require a program (i.e., a set of commands precisely written according to some programming language) to…

Tips and tricks for successful prompting with LLMs… — Due to their text-to-text format, large language models (LLMs) are capable of solving a wide variety of tasks with a single model. Such a capability was originally demonstrated via zero and few-shot learning with models like GPT-2 and GPT-3 [5, 6]. …

A practical and simple approach for “reasoning” with LLMs — The success of large language models (LLMs) stems from our ability to pre-train (using a language modeling objective) decoder-only transformer models across massive textual corpora. Given that we pre-train sufficiently large models, LLMs are incredibly capable few-shot learners. In other words, this means that we can solve a variety of…

How LLaMA is making open-source cool again — Despite recent advances in large language models (LLMs), many of the most powerful models are only accessible via paid APIs and trained using large amounts of proprietary data, thus limiting the research community from accessing or reproducing such models. This trend raises serious concerns about whether LLMs will be mostly…

High-performing language models that are open-source… — For years, the deep learning community has embraced openness and transparency, leading to massive open-source projects like HuggingFace. Many of the most profound ideas in deep learning (e.g., transformers [2], self-supervised learning, etc.) are openly available online, either via public code repositories or Arxiv. Although open-source has been the norm…