In machine learning, what does the term "model training" mean?
Model training is the process of finding the optimal values for the model parameters that minimize the error between the model predictions and the actual output.This is done by using a learning algorithm that iteratively updates the parameters based on the input features and the output1.Reference:Oracle Cloud Infrastructure Documentation
What is the primary goal of machine learning?
Machine learning is a branch of artificial intelligence that enables computers to learn from data and experience without being explicitly programmed.Machine learning algorithms can adapt to new data and situations and improve their performance over time2.Reference:Artificial Intelligence (AI) | Oracle
What role do tokens play in Large Language Models (LLMs)?
Tokens are the basic units of text representation in large language models. They can be words, subwords, characters, or symbols. Tokens are used to encode the input text into numerical vectors that can be processed by the model's neural network.Tokens also determine the vocabulary size and the maximum sequence length of the model3.Reference:Oracle Cloud Infrastructure 2023 AI Foundations Associate | Oracle University
How do Large Language Models (LLMs) handle the trade-off between model size, data quality, data size and performance?
Large language models are trained on massive amounts of data to capture the complexity and diversity of natural language. Larger model sizes mean more parameters, which enable the model to learn more patterns and nuances from the data. Larger models also tend to generalize better to new tasks and domains. However, larger models also require more computational resources, data quality, and data size to train and deploy.Therefore, large language models handle the trade-off by prioritizing larger model sizes to achieve better performance, while using various techniques to optimize the training and inference efficiency4.Reference:Artificial Intelligence (AI) | Oracle
What is the purpose of Attention Mechanism in Transformer architecture?
The attention mechanism in the Transformer architecture is a technique that allows the model to focus on the most relevant parts of the input and output sequences. It computes a weighted sum of the input or output embeddings, where the weights indicate how much each word contributes to the representation of the current word.The attention mechanism helps the model capture the long-range dependencies and the semantic relationships between words in a sequence12.Reference:The Transformer Attention Mechanism - MachineLearningMastery.com,Attention Mechanism in the Transformers Model - Baeldung
