Currently best small LLM model with very large context window?

I'm looking for a small LLM model, around 1-3B, that has very large context window. Preferably very proficient in NLP.

6 Best LLMs (2024): Large Language Models Compared - eWEEK

GPT-4 is an advanced API-based LLM that you can access for as low as $20 per month. And it's remarkably easy to use via the mobile and web ...

List of the Best 21 Large Language Models (LLMs) (September 2024)

From compact models like Phi-2 and Alpaca 7B to cutting-edge architectures like Jamba and DBRX, the field of LLMs is pushing the boundaries of what's possible ...

Most powerful LLMs (Large Language Models) - Codingscape

The most powerful large language models (LLMs) from OpenAI GPT-4o to Claude 3.5 Sonnet drive a multi-trillion dollar AI industry. Here are the best LLMs.

Why larger LLM context windows are all the rage - IBM Research

Larger context windows give language models more background to consider as they generate a response, leading to more coherent and relevant answers.

The best open source large language model - Baseten

Best small LLM under 7 billion parameters: Phi 3 Mini · Excellent output quality rivals 7B LLMs from just a few months ago. · 128k-token context window variant ...

LLMs with largest context windows - Codingscape

Benefit – LLMs with large context windows can read and analyze long documents without needing to split them into smaller sections. This makes summarization, ...

8 Top Open-Source LLMs for 2024 and Their Uses - DataCamp

If the Falcon 40B already impressed the open-source LLM community (it ranked #1 on Hugging Face's leaderboard for open-source large language models), the new ...

100M Token Context Windows - Magic.dev

Research update on ultra-long context models, our partnership with Google Cloud, and new funding. ... There are currently two ways for AI models ...

Tiny but mighty: The Phi-3 small language models with big potential

Large language models (LLMs) have created exciting new opportunities to be more productive and creative using AI. But their size means they can ...

How to choose the right LLM for your needs | TechTarget

While performance benchmarks and context window size cover some LLM capabilities, organizations also must evaluate other model features, such as ...

Xnhyacinth/Awesome-LLM-Long-Context-Modeling - GitHub

Must-read papers for LLM-based Long Context Modeling. Thanks for all the great contributors on GitHub! ⚡ . Contents. 1. Survey Papers; 2 ...

LLM Prompt Best Practices for Large Context Windows - Winder.AI

A larger window would enable an LLM to process more extensive information, which is crucial for tasks requiring in-context learning.

Long context window models vs. RAG | by Jm - Medium

The most relevant chunks are passed on to the LLM. Depending on the data structure and type (text, images, tables) different calculations of ...

How Gradient created an open LLM with a million-token context ...

The race to create open-source models with long context windows can reshuffle the LLM market and unlock applications that are not possible with ...

Do Enormous LLM Context Windows Spell the End of RAG?

By increasing the amount of text LLMs can process at one time, these extended contextual windows enhance the model's ability to understand more ...

Introducing Meta Llama 3: The most capable openly available LLM ...

Our pretrained model also establishes a new state-of-the-art for LLM models at those scales. *Please see evaluation details for setting and ...

LLM Context Windows: Basics, Examples & Prompting Best Practices

In large language models (LLMs), the context window is crucial for capturing the nuances of a conversation or a text. It ensures that the model remains relevant ...

Long Context RAG Performance of LLMs | Databricks Blog

Taking longer context lengths to the extreme, there is even a debate about whether long context language models will eventually subsume RAG ...

The Secret Sauce behind 100K context window in LLMs

Moreover, the inability to cache transformers makes the use of large context windows quite costly, as all previous messages must be sent with ...