A massive breakthrough in scene representation — As we have seen with methods like DeepSDF [2] and SRNs [4], encoding 3D objects and scenes within the weights of a feed-forward neural network is a memory-efficient, implicit representation of 3D data that is both accurate and high-resolution. However, the approaches we have seen so far are not quite…

How to render 3D scenes on a smart phone — By now, we should know that deep learning is a great way to represent 3D scenes and generate new renderings of these scenes from arbitrary viewpoints. The problem with the approaches we have seen so far (e.g., ONets and SRNs [2, 3], however, is that they require many images of…

Modeling complex 3D scenes at infinite resolution — We have seen recently (e.g., with DeepSDF and ONets [3, 5]) how 3D geometries can be represented with a neural net. But, these approaches have some limitations, such as requiring access to ground truth, 3D geometries for training and inference. Plus, what if we want to represent an entire scene…

Representing 3D space with learnable functions — The problem of 3D reconstruction aims to produce a high-resolution object representation given a noisy view of the object (e.g., partial point cloud, 2D image, etc.) as input. Recently, deep neural networks have become a popular approach for 3D reconstruction, as they can encode prior information that helps to handle…

Simple neural networks can capture complex 3D geometries — Prior research in computer graphics and 3D computer vision has proposed numerous approaches for representing 3D shapes. Such methods are useful for: storing memory-efficient representations of known shapes generating new shapes fixing/reconstructing shapes based on limited or noisy data Beyond classical approaches, deep learning — or, more specifically, generative neural…

Simple techniques for creating better, domain-specific LLMs — Large language models (LLMs) are incredibly-useful, task-agnostic foundation models. But, how much can we actually accomplish with a generic model? These models are adept at solving common natural language benchmarks that we see within the deep learning literature. But, using LLMs practically usually requires that the model be taught new…

Can bigger models solve all of our problems? — Within this overview, we will take a look at the generation of large language models (LLMs) that came after GPT-3 [7]. The incredible results of GPT-3 demonstrated clearly that increasing the size of language models (LMs) is very beneficial. The question is, however, when does this trend plateau? …

Understanding why LLMs like GPT-3 work so well… — Language models (LMs) are incredibly generic–they take text as input and produce text as output. Recent research has revealed that this generic text-to-text structure can be exploited to solve a variety of tasks without task-specific adaptation (i.e., no fine-tuning or architectural modifications) by using prompting techniques to perform accurate zero…

How smaller language models inspired modern breakthroughs — Language models like GPT-3 [7] have revolutionized modern deep learning applications for NLP, leading to widespread publicity and recognition. Interestingly, however, most of the technical novelty of GPT-3 was inherited from its predecessors GPT and GPT-2 [1, 2]. As such, a working understanding of GPT and GPT-2 is useful for…