Comprehensive Introduction about Large Language Models (LLMs)

Large language Models [LLMs] 

refer to powerful artificial intelligence models that are trained on vast amounts of text data to understand and generate human-like language. These models are part of a broader category of artificial intelligence known as natural language processing (NLP). LLMs are characterized by their ability to process and generate text on a large scale, exhibiting a wide range of language-related tasks such as language translation, text summarization, question answering, and more.

One notable example of a Large Language Model is GPT-3 (Generative Pre-trained Transformer 3), developed by OpenAI. GPT-3 is one of the largest and most advanced language models to date, with 175 billion parameters. Parameters, in this context, are the internal variables that the model learns during training.

Some other examples of Large Language Models include:

GPT-2 (Generative Pre-trained Transformer 2): The predecessor to GPT-3, GPT-2 also gained attention for its ability to generate coherent and contextually relevant text. It has 1.5 billion parameters.

BERT (Bidirectional Encoder Representations from Transformers): Developed by Google, BERT is another influential language model. Unlike traditional models that read text in a left-to-right or right-to-left fashion, BERT reads text bidirectionally, considering the context of each word based on the surrounding words.

XLNet: XLNet is a transformer-based language model that combines ideas from autoregressive models (like GPT) and autoencoder models (like BERT). It addresses some limitations of both approaches.

T5 (Text-to-Text Transfer Transformer): T5, developed by Google AI, treats all NLP tasks as converting input text to output text, making it a versatile model for various natural language processing tasks.

RoBERTa (Robustly optimized BERT approach): RoBERTa is a variant of BERT that modifies key hyperparameters and removes the next-sentence prediction objective, leading to improved performance on certain tasks.

ERNIE (Enhanced Representation through kNowledge Integration): Developed by Baidu, ERNIE incorporates world knowledge into pre-training through knowledge masks, phrase-level masking, and entity-level masking to improve language understanding.

ALBERT (A Lite BERT): ALBERT is a variation of BERT designed to reduce the number of parameters while maintaining or improving performance. It achieves this by using parameter-sharing techniques and factorized embedding parameterization.

DistilBERT: This model, developed by Hugging Face, is a distilled version of BERT designed for faster inference and reduced memory requirements while maintaining most of BERT's performance.

Electra: Electra is an alternative pre-training method that focuses on training a model to distinguish between "real" and "fake" input tokens, where some tokens in the input sequence are replaced with incorrect ones.

ERNIE 2.0 (Enhanced Representation through kNowledge Integration 2.0): An improved version of ERNIE, it incorporates knowledge graph information and enhances the model's ability to understand entities and their relationships.

CTRL (Controlled Text Generation): CTRL is designed to allow users to guide the generation of text by conditioning the model on specific control codes. This enables users to influence the style and content of the generated text.

UniLM (Unified Language Model): Developed by Microsoft Research, UniLM is a model designed for multiple NLP tasks, including language modeling, text classification, and sequence-to-sequence tasks, by unifying different pre-training objectives.

ERNIE-M (Multilingual ERNIE): An extension of ERNIE for multilingual support, ERNIE-M is trained on a diverse set of languages to handle tasks in multiple languages.

Flair: Flair is an open-source natural language processing library that combines contextual embeddings with traditional word embeddings to capture the contextual information of words in a sentence.

XLM-R (Cross-lingual Language Model - RoBERTa): This model is an extension of RoBERTa that is pre-trained on a large corpus of data from multiple languages, making it effective for cross-lingual tasks.

ERNIE-Turbo: Another variant of ERNIE, ERNIE-Turbo incorporates a two-step learning strategy to better capture the hierarchical structure of language and improve performance on various tasks.

Megatron-LM: Developed by NVIDIA, Megatron-LM is a large-scale language model based on the transformer architecture. It is known for its parallelism and scalability, allowing training on very large datasets.

ProphetNet: Developed by Microsoft, ProphetNet is a model designed for sequence-to-sequence tasks. It introduces a novel masked n-gram language model objective for pre-training.

ERNIE 3.0: An evolution of the ERNIE series, ERNIE 3.0 incorporates heterogeneous graph learning to better capture the relationships between entities and their attributes.

LUKE (Language Understanding with Knowledge-based Embeddings): LUKE is a model that integrates world knowledge into pre-trained language representations by leveraging information from knowledge bases like Wikidata.

BART (BART Ain't a Random Transformer): Developed by Facebook AI, BART is a denoising autoencoder for pre-training sequence-to-sequence models. It is used for various tasks, including text summarization and language generation.

UniLMv2 (Unified Language Model v2): A successor to UniLM, UniLMv2 is a versatile model designed to handle a wide range of NLP tasks through a unified pre-training framework.

ERNIE-Gram (Enhanced Language Representation with Informative Entities): Another iteration of ERNIE, ERNIE-Gram incorporates an entity-grammar pre-training task to enhance the model's understanding of grammar and entities.

DeBERTa (Decoding-enhanced BERT with Disentangled Attention): DeBERTa improves upon BERT by introducing disentangled attention mechanisms, allowing the model to focus on different aspects of the input text independently.

TinyBERT: As the name suggests, TinyBERT is a smaller and more efficient version of BERT designed for deployment on resource-constrained devices.

ALBERT-Lite: An even lighter version of ALBERT, ALBERT-Lite further reduces the number of parameters for efficient deployment in low-resource environments.

MiniLM: MiniLM is a compact version of BERT that retains most of BERT's performance while significantly reducing the model size, making it more suitable for deployment in memory-constrained environments.

Longformer: Longformer is designed to handle long documents by introducing a novel attention mechanism that allows the model to efficiently process sequences of thousands of tokens.

ERNIE 1.0 (Enhanced Representation through kNowledge Integration 1.0): The original version of the ERNIE model, which laid the foundation for subsequent ERNIE iterations.

CamemBERT: Developed by Facebook AI for French language understanding, CamemBERT is a BERT-based model pre-trained on a large French corpus.

Funnel-Transformer: Funnel-Transformer is designed to handle long-range dependencies in sequences by utilizing a funnel-like architecture. It has been used for various NLP tasks, including language modeling and text classification.

BERTje: BERTje is a Dutch-language variant of BERT, pre-trained on a large Dutch corpus. It is designed to understand and generate Dutch text.

Roberta: (Robustly optimized BERT approach): Similar to RoBERTa, Roberta is another variant of BERT that modifies training techniques to improve performance on a range of NLP tasks.

XLNet-Mini: A smaller version of XLNet, XLNet-Mini is designed to be more lightweight while retaining some of the benefits of the original model.

XLM-RoBERTa-Large: An extension of XLM-R, XLM-RoBERTa-Large is a large-scale multilingual model trained on a diverse range of languages.

CTRL-UNI: An extension of CTRL, CTRL-UNI is designed for controlled text generation, allowing users to influence the style and content of generated text.

 DistilGPT: DistilGPT is a smaller version of GPT-2, developed by OpenAI. It is designed to be more computationally efficient while maintaining reasonable performance on various language tasks.

ERNIE 2.0-Base: A baseline version of ERNIE 2.0, this model incorporates knowledge graph information and enhances the model's understanding of entities and their relationships.

These models are typically pre-trained on a massive corpus of diverse text data and then fine-tuned for specific tasks or domains. Their large size and capacity for understanding contextual relationships in language make them powerful tools for a wide range of natural language processing applications.

Architecture of LLM

The architecture of Large Language Models (LLMs) is typically based on the transformer architecture, which was introduced in the paper "Attention is All You Need" by Vaswani et al. in 2017. The transformer architecture has become the foundation for many state-of-the-art language models due to its effectiveness in handling sequential data and capturing long-range dependencies.

Here are the key components of the transformer architecture, which is commonly used in LLMs:

Encoder-Decoder Structure: The transformer architecture is built on an encoder-decoder structure. However, for language modelling tasks, especially autoregressive models like GPT (Generative Pre-trained Transformer), only the decoder part is used.

Attention Mechanism: The attention mechanism is a fundamental part of the transformer architecture. It allows the model to focus on different parts of the input sequence when making predictions. The self-attention mechanism in particular enables the model to weigh the importance of different words in the input sequence for each word in the output sequence.

Multi-Head Attention: To capture different aspects of the input sequence, multiple attention heads are used in parallel. Each head operates on a different linear projection of the input, providing the model with the ability to attend to different patterns.

Positional Encoding: Transformers do not inherently understand the order of elements in a sequence since they process input in parallel. Positional encoding is added to the input embeddings to provide information about the relative or absolute position of the tokens in the sequence.

Feedforward Neural Network: After the attention mechanism, the model typically has a feedforward neural network for each position in the sequence, allowing it to capture complex patterns in the data.

Layer Normalization and Residual Connections: Each sub-layer (like attention and feedforward layers) is followed by layer normalization and a residual connection, helping with training stability and the flow of information through the network.

Large Language Models, such as GPT-3, can have a massive number of parameters, which contributes to their ability to capture intricate patterns and nuances in language. These models are usually pre-trained on vast amounts of text data and fine-tuned for specific tasks, enabling them to perform well across a range of natural language processing applications.

Key Takeaways

LLMs are artificial neural networks (mainly transformers) and are (pre) trained using self-supervised learning and semi-supervised learning.

Self-Supervised Learning (SSL): Unlocking Knowledge from Unlabeled Data

Self-Supervised Learning (SSL) stands as a groundbreaking paradigm in machine learning, leveraging unlabeled data to train models without the need for explicit external labels. The core idea is to design tasks within the learning framework that inherently generate supervision signals from the data itself. This approach enables models to learn rich and meaningful representations, contributing to improved performance on downstream tasks.

Types of Self-Supervised Learning:

Auto associative self-supervised learning

Auto associative self-supervised learning is a specific category of self-supervised learning where a neural network is trained to reproduce or reconstruct its own input data. In other words, the model is tasked with learning a representation of the data that captures its essential features or structure, allowing it to regenerate the original input.

The term "auto associative" comes from the fact that the model is essentially associating the input data with itself. This is often achieved using autoencoders, which are a type of neural network architecture used for representation learning. Autoencoders consist of an encoder network that maps the input data to a lower-dimensional representation (latent space), and a decoder network that reconstructs the input data from this representation.

The training process involves presenting the model with input data and requiring it to reconstruct the same data as closely as possible. The loss function used during training typically penalizes the difference between the original input and the reconstructed output. By minimizing this reconstruction error, the autoencoder learns a meaningful representation of the data in its latent space.

Contrastive self-supervised learning

For a binary classification task, training data can be divided into positive examples and negative examples. Positive examples are those that match the target. For example, if you're learning to identify birds, the positive training data are those pictures that contain birds. Negative examples are those that do not. Contrastive self-supervised learning uses both positive and negative examples. Contrastive learning's loss function minimizes the distance between positive samples while maximizing the distance between negative samples.

Non-contrastive self-supervised learning

Non-contrastive self-supervised learning (NCSSL) uses only positive examples. Counterintuitively, NCSSL converges on a useful local minimum rather than reaching a trivial solution, with zero loss. For the example of binary classification, it would trivially learn to classify each example as positive. Effective NCSSL requires an extra predictor on the online side that does not back-propagate on the target side.

Semi-supervised Learning

Weak supervision, also called semi-supervised learning, is a paradigm in machine learning, the relevance and notability of which increased with the advent of large language models due to large amount of data required to train them. It is characterized by using a combination of a small amount of human-labelled data (exclusively used in more expensive and time-consuming supervised learning paradigm), followed by a large amount of unlabeled data (used exclusively in unsupervised learning paradigm). In other words, the desired output values are provided only for a subset of the training data. The remaining data is unlabeled or imprecisely labelled. Intuitively, it can be seen as an exam and labelled data as sample problems that the teacher solves for the class as an aid in solving another set of problems. In the transductive setting, these unsolved problems act as exam questions. In the inductive setting, they become practice problems of the sort that will make up the exam. Technically, it could be viewed as performing clustering and then labelling the clusters with the labelled data, pushing the decision boundary away from high-density regions, or learning an underlying one-dimensional manifold where the data reside.

Training and architecture details

Let's delve into the training and architecture details of Large Language Models (LLMs) with a focus on several key aspects:

1. Reinforcement Learning from Human Feedback (RLHF):

Training Approach:

Pre-training: LLMs are initially pre-trained on a large corpus of data using unsupervised learning. This helps the model learn the intricacies of language and syntax.

Fine-tuning with Reinforcement Learning (RL): RLHF involves fine-tuning the model using reinforcement learning, where human-generated feedback is used to refine the model's output. The model is guided by a reward signal based on the quality of its responses.

2. Instruction Tuning:

Training Approach:

Supervised Fine-Tuning: After pre-training, LLMs can undergo supervised fine-tuning using datasets with human-crafted instructions. The model learns to follow specific instructions provided in the training data.

Reinforcement Learning: Instruction tuning often involves reinforcement learning, where the model receives feedback on how well it adheres to given instructions. This process refines the model's behaviour.

3. Mixture of Experts:

Architecture:

Expert Modules: LLMs may have a "mixture of experts" architecture where different components or modules specialize in specific tasks. Each expert focuses on a particular aspect of the input data.

Gating Mechanism: A gating mechanism determines which expert or combination of experts to activate based on the input. This allows the model to dynamically choose the most relevant component for a given task.

4. Prompt Engineering:

Training Approach:

Prompt Design: Prompt engineering involves designing effective prompts or queries to guide the model's behaviour. Crafting well-phrased prompts is crucial for eliciting desired responses from the LLM.

Iterative Refinement: Engineers iteratively refine prompts based on the model's output, human evaluations, or reinforcement learning. This process helps to improve the model's performance over time.

5. Attention Mechanism:

Architecture:

Self-Attention: LLMs, based on transformer architectures, utilize attention mechanisms. Self-attention allows the model to weigh the importance of different words in a sequence when making predictions.

Multi-Head Attention: Attention is often applied across multiple heads or attention mechanisms, enabling the model to capture different types of relationships and dependencies within the input.

6. Context Window:

Architecture:

Contextual Understanding: LLMs, especially in tasks like language modelling, benefit from a large context window. A context window refers to the number of preceding words or tokens considered when predicting the next word.

Long-Range Dependencies: A larger context window helps the model capture long-range dependencies and understand the broader context of the input sequence.

These training and architecture details highlight the sophistication and versatility of Large Language Models, incorporating reinforcement learning, specialized expert modules, prompt engineering, attention mechanisms, and context-aware architectures to achieve impressive natural language understanding and generation capabilities.

Properties

The following four hyper-parameters characterize an LLM:

* cost of (pre-)training (�C ),

*size of the artificial Neural network itself, such as number of parameters �N  (i.e. amount of neurons in its layers, amount of weights between them and biases),

*size of its (pre-)training dataset (i.e. number of tokens in corpus, �D ),

*performance after (pre-)training.

They are related by simple Statistical Laws called "scaling laws". One particular scaling law ("Chinchilla Scaling") for LLM autoregressively trained for one epoch, with a log-log learning rate schedule, states that:

where the variables are:

C is the cost of training the model, in floating point operations per second [ FLOPs].

�N  is the number of parameters in the model.

�D  is the number of tokens in the training set.

�L  is the average negative log-likelihood loss per token (nats/token), achieved by the trained LLM on the test dataset.

and the statistical hyper-parameters are:

Co = 6, meaning that it costs 6 FLOPs per parameter to train on one token. Note that training cost is much higher than inference cost, where it costs 1 to 2 FLOPs per parameter to infer on one token.

alpha = 0.34, beta = 0.28 , A = 406.4, B = 410.7, Lo = 1.69

Evaluation

Perplexity is a metric commonly used to evaluate the performance of language models, including Large Language Models (LLMs). It is a measure of how well a probability distribution or probability model predicts a sample.

In the context of LLMs, perplexity is often used in language modelling tasks, where the goal is to predict the probability distribution of the next word in a sequence given the context of previous words. Lower perplexity values indicate better performance.

Here's a brief explanation of perplexity:

Definition: Perplexity is a measure of how well a probability distribution or probability model predicts a sample. It is calculated as 2 to the power of the entropy, where entropy is a measure of uncertainty or disorder in a set of probabilities.

The most commonly used measure of a language model's performance is its perplexity on a given text corpus. Perplexity is a measure of how well a model is able to predict the contents of a dataset; the higher the likelihood the model assigns to the dataset, the lower the perplexity. Mathematically, perplexity is defined as the exponential of the average negative log likelihood per token:

log⁡(Perplexity)=−1�∑�=1�log⁡(��(token�|context for token�))

here N  is the number of tokens in the text corpus, and "context for token i " depends on the specific type of LLM used. If the LLM is autoregressive, then "context for token i " is the segment of text appearing before token i. If the LLM is masked, then "context for token i" is the segment of text surrounding token i.

Because language models may overfit to their training data, models are usually evaluated by their perplexity on a test set of unseen data. This presents particular challenges for the evaluation of large language models. As they are trained on increasingly large corpora of text largely scraped from the web, it becomes increasingly likely that models' training data inadvertently includes portions of any given test set.

 Language Modelling:

In the context of language modelling, the perplexity of a model is calculated based on its ability to predict the next word in a sequence. A lower perplexity indicates that the model is more certain and accurate in its predictions.

Formula:

The perplexity (PP) is calculated as 2 to the power of the cross-entropy (H) of the model's predictions. Mathematically, it can be expressed as: PP =  

Interpretation:

A perplexity of 1 would mean that the model perfectly predicts the next word, while higher values indicate increasing uncertainty and worse performance.

Evaluation:

During training, language models are optimized to minimize perplexity. In the evaluation phase, perplexity is used to assess how well the model generalizes to unseen data. A lower perplexity on a test dataset suggests better language understanding and generation capabilities.

Relation to Probability:

Perplexity is inversely related to the probability assigned by the model to the actual outcomes. A lower perplexity corresponds to a higher probability assigned by the model to the true outcomes.

In summary, perplexity is a valuable metric for evaluating the quality of language models, including LLMs. It provides a quantitative measure of how well the model captures the underlying patterns and structure of the language in the data it was trained on and how well it generalizes to new, unseen sequences.

Conclusion

In conclusion, Large Language Models (LLMs) represent a transformative breakthrough in the field of natural language processing and artificial intelligence. These models, built on advanced transformer architectures, have demonstrated unprecedented capabilities in understanding, generating, and manipulating human-like language. Key aspects and conclusions about LLMs include:

Unprecedented Scale: LLMs are characterized by their massive scale, often comprising tens or hundreds of billions of parameters. This scale contributes to their ability to capture complex linguistic patterns and nuances.

Pre-training Paradigm: LLMs follow a pre-training paradigm, where models are initially trained on large, diverse datasets through unsupervised learning tasks. This pre-training phase equips the models with a deep understanding of language structures.

Fine-Tuning and Adaptability: After pre-training, LLMs can be fine-tuned for specific tasks or domains, making them adaptable to a wide range of applications. This adaptability is crucial for achieving state-of-the-art performance in various natural language processing tasks.

Effective Transfer Learning: LLMs showcase the power of transfer learning. The representations learned during pre-training can be transferred to downstream tasks, allowing for efficient training on smaller datasets and generalization across diverse domains.

Natural Language Generation: LLMs excel in natural language generation tasks, producing coherent and contextually relevant text. This proficiency has applications in content creation, chatbots, summarization, and other language generation tasks.

Challenges and Ethical Considerations: Despite their remarkable capabilities, LLMs also pose challenges and ethical considerations. Concerns include biases present in training data, potential misuse, and the environmental impact of training such large models.

Interdisciplinary Impact: LLMs have made a significant impact across various disciplines, from advancing the field of linguistics to improving user interactions in applications, enhancing accessibility, and aiding in information retrieval.

Ongoing Research and Innovation: The field of LLMs is dynamic, with ongoing research and innovation. Researchers continuously explore new architectures, training strategies, and ways to address limitations, ensuring the evolution of these models.

In essence, Large Language Models have reshaped the landscape of natural language understanding and generation, paving the way for more sophisticated and context-aware applications. While their capabilities are extraordinary, ongoing efforts are essential to address challenges and ensure responsible and ethical deployment in diverse real-world scenarios.

Top of Form

 Inspired by : LLMs Wiki

------------------------------------------------------@@@ Happy Learning @@-------------------------------------------------------