Bland.ai, the telephone agent of the future, Magic and the 100 million token LLM of context

100M Token Context Windows

Magic, in collaboration with Google Cloud, has launched an AI model capable of processing impressive contexts, up to 100 million tokens.

This advancement could revolutionize code synthesis and other applications. Their LTM-2-mini model, trained on hashes, offers advantages in terms of cost and memory compared to other models. Magic has also introduced HashHop, an evaluation metric that eliminates implicit prompts, providing more accurate results.

Two new supercomputers powered by NVIDIA are also planned on Google Cloud.

Magic has recently raised $465 million in new funding.

100M Token Context Windows — Magic

Research update on ultra-long context models, our partnership with Google Cloud, and new funding.

https://magic.dev/blog/100m-token-context-windows

How to Achieve Near Human-Level Performance in Chunking for RAGs

Retrieval-Augmented Generation (RAG) applications are becoming increasingly important in the field of artificial intelligence.

A crucial aspect for proper functioning is the chunking process, which involves dividing text into smaller fragments. A recent article published on Towards Data Science explores an innovative technique called "agentive chunking," which promises to significantly improve the performance of RAG systems.

Traditionally, the most common chunking techniques include recursive character splitting and semantic splitting. The former uses a sliding window approach with a fixed token length, while the latter divides the text based on significant changes between consecutive sentences.

However, both have limitations. Recursive character splitting doesn't guarantee that a theme is entirely contained within a single window, risking context fragmentation. On the other hand, semantic splitting, while better capturing themes, fails to connect similar concepts that might be found in distant parts of the text.

This is where agentive chunking comes into play. This innovative technique aims to overcome the limitations of existing methodologies, offering a more sophisticated solution for text division.

The agentive approach considers not only the surface structure of the text but also the semantic relationships between different parts of the document. It uses advanced algorithms to analyze content and identify thematic connections, even between seemingly unrelated sections. This allows for the creation of more coherent and informative chunks, which can significantly improve the quality of results in RAG applications.

The potential advantages of this technique are numerous:

Better context preservation: Created chunks maintain greater thematic coherence, reducing the loss of crucial information.

More accurate retrieval: The superior quality of chunks translates into more precise retrieval of relevant information.

Generation of more pertinent responses: With richer and more accurate context, language models can generate more appropriate and informative responses.

Computational efficiency: Despite the increased complexity of the chunking algorithm, the agentive approach can lead to a reduction in the total number of chunks, thus optimizing the use of computational resources.

However, it's important to note that implementing agentive chunking requires greater algorithmic complexity and may need additional computational resources in the preprocessing phase.

Developers will need to carefully evaluate the trade-off between performance improvement and increased computational costs.

In conclusion, agentive chunking represents a promising evolution in text processing techniques for RAG applications.

As research in this field continues, we are likely to see further refinements and optimizations of this technique. Developers and researchers in the AI field should keep an eye on these developments, as they could lead to significant improvements in the performance of RAG-based systems.