Top 51 Generative AI Interview Questions and Answers

Generative AI refers to AI systems that create new content — text, images, audio, code, or video — rather than just analyzing or classifying existing data.

• Learns patterns from large training datasets, then generates novel outputs that resemble that data.
• Examples: ChatGPT/Claude (text), DALL-E/Midjourney (images), GitHub Copilot (code).

They solve fundamentally different problems:

• Discriminative models: learn to distinguish between classes (e.g., spam vs. not spam) — they model P(label | input).
• Generative models: learn the underlying data distribution to create new samples — modeling P(input) or P(input | condition).
• A spam classifier is discriminative; a model that writes a new email from scratch is generative.

A Large Language Model is a neural network (typically Transformer-based) trained on massive amounts of text to understand and generate human-like language.

• "Large" refers to both the size of the training data and the number of parameters (often billions).
• Examples: GPT-4, Claude, Gemini, Llama.

The Transformer is the neural network architecture underlying most modern LLMs, introduced in the 2017 paper "Attention Is All You Need."

• Relies entirely on the self-attention mechanism instead of recurrence (RNNs) or convolution, allowing it to process all tokens in a sequence in parallel.
• This parallelism made training on much larger datasets practical, driving the current generation of LLMs.

Self-Attention allows a model to weigh the relevance of every other token in a sequence when processing a given token.

• Computes Query, Key, and Value vectors for each token, then uses their dot-products to determine how much attention each token should pay to every other token.
• Enables the model to capture long-range dependencies and context far better than earlier sequential architectures.

A Token is the basic unit of text an LLM processes — often a word, part of a word, or punctuation mark.

• Text is broken into tokens via a tokenizer before being fed into the model (e.g., "unhappiness" might split into "un", "happi", "ness").
• Model context limits and API pricing are typically measured in tokens, not characters or words.

The Context Window is the maximum number of tokens (input + output combined) an LLM can process in a single request.

• Everything the model "remembers" during a conversation must fit within this window — older content gets truncated once the limit is exceeded.
• Modern models range from a few thousand to over a million tokens, affecting how much conversation history or document content can be included.

Prompt Engineering is the practice of carefully crafting input text to guide an LLM toward better, more reliable outputs.

• Techniques include giving clear instructions, providing examples (few-shot prompting), specifying output format, and breaking complex tasks into steps.
• Critical for getting consistent, high-quality results without modifying the underlying model.

These describe how many examples are given in the prompt before asking the model to perform a task:

• Zero-shot: no examples given — the model relies purely on its pretrained knowledge and instructions.
• One-shot: exactly one example is provided.
• Few-shot: multiple (typically 2-5+) examples are provided to demonstrate the desired pattern or format.

Fine-tuning further trains a pretrained model on a smaller, task-specific dataset to specialize its behavior.

• Adjusts the model's weights based on new examples, unlike prompting which doesn't change the model at all.
• Useful for adapting a general-purpose model to a specific domain, tone, or task format — but is more expensive and time-consuming than prompt engineering.

RAG combines an LLM with an external knowledge retrieval step to ground responses in up-to-date or domain-specific information.

• At query time, relevant documents are retrieved (typically via vector similarity search) and injected into the prompt as context.
• Reduces hallucination and allows the model to answer questions about information it was never trained on, without needing to fine-tune.

A Vector Embedding is a numerical representation of text (or images, audio) as a list of floating-point numbers in high-dimensional space.

• Semantically similar content produces embeddings that are close together in that space (measured via cosine similarity or Euclidean distance).
• Foundational to semantic search, RAG, recommendation systems, and clustering.

A Vector Database (e.g., Pinecone, Weaviate, Milvus, pgvector) stores and efficiently searches embeddings for similarity.

• Enables semantic search: finding documents by meaning rather than exact keyword match.
• Core infrastructure for RAG pipelines, where relevant context must be retrieved quickly from potentially millions of embedded documents.

Hallucination occurs when an LLM generates confident-sounding output that is factually incorrect or entirely fabricated.

• Happens because LLMs generate the statistically most likely next tokens, not verified facts — they have no built-in mechanism to "know" they're wrong.
• Mitigated (not eliminated) through RAG, fact-checking pipelines, lower temperature settings, and careful prompting.

Temperature controls the randomness of an LLM's output by scaling the probability distribution over possible next tokens.

• Low temperature (near 0): more deterministic, focused, and repetitive output — picks the most likely tokens.
• High temperature (closer to 1+): more random, creative, and varied output — increases the chance of selecting less likely tokens.

Top-p sampling selects the next token from the smallest set of candidates whose cumulative probability exceeds a threshold p.

• E.g., top_p = 0.9 considers only the most likely tokens that together make up 90% of the probability mass, ignoring the unlikely "long tail."
• Often used alongside or instead of temperature to control output diversity while avoiding very low-probability, nonsensical tokens.

Both limit which tokens can be sampled, but with different criteria:

• Top-k: restricts sampling to the k most likely next tokens, regardless of their actual probabilities.
• Top-p: restricts sampling to the smallest set of tokens whose combined probability reaches p — dynamically adapts based on how confident the model is.

A System Prompt is a special instruction given to an LLM before the user's actual message, setting its behavior, persona, or constraints for the entire conversation.

System: You are a helpful coding assistant. Always respond with concise code examples.

• Typically has higher priority than user messages and isn't usually shown to the end user.

Chain-of-Thought prompting encourages an LLM to reason step-by-step before giving a final answer, rather than jumping straight to a conclusion.

Q: If a train travels 60 mph for 2.5 hours, how far does it go?
A: Let's think step by step. Distance = speed × time = 60 × 2.5 = 150 miles.

• Significantly improves accuracy on multi-step reasoning tasks like math or logic problems.

Function (or Tool) Calling lets an LLM invoke external functions/APIs to fetch real-time data or perform actions it can't do on its own (like arithmetic, web search, or database queries).

• The model outputs a structured request (function name + arguments); the calling application executes it and returns the result back to the model.
• Foundational to building AI agents that can interact with real systems, not just generate text.

An AI Agent is a system built around an LLM that can autonomously plan, use tools, and take multi-step actions to accomplish a goal, rather than just responding to a single prompt.

• Typically involves a loop: the LLM decides an action (e.g., call a tool), observes the result, and decides the next action, repeating until the goal is achieved.
• Examples: coding assistants that can read files and run commands, or research agents that browse the web autonomously.

Both are Transformer-based, but designed for different purposes:

• BERT (encoder-only): processes the entire input bidirectionally, ideal for understanding tasks like classification or extracting answers from text.
• GPT (decoder-only): generates text autoregressively, predicting the next token based only on previous tokens — ideal for open-ended text generation.

Multimodal AI models can process and/or generate multiple types of data — text, images, audio, video — within a single model.

• Example: a model that can accept an image and a text question, then generate a text answer about the image.
• Increasingly common in modern LLMs (e.g., GPT-4V, Gemini), enabling richer, more flexible applications.

Quantization reduces the numerical precision of a model's weights (e.g., from 32-bit floats to 8-bit or 4-bit integers).

• Significantly reduces model size and memory/compute requirements, enabling models to run on smaller hardware (even consumer laptops or phones).
• Trades a small amount of accuracy/quality for large gains in efficiency and speed.

Both are training stages, but with different goals:

• Pretraining: the model learns general language patterns from a massive, mostly unlabeled text corpus (predicting the next token).
• Instruction Tuning: the pretrained model is further trained on examples of instructions paired with desired responses, teaching it to follow directions and be helpful, rather than just completing text.

RLHF further trains a model based on human preferences between different possible responses.

• Humans rank/rate multiple model outputs; this feedback trains a reward model, which is then used to fine-tune the LLM via reinforcement learning.
• A key technique behind making models like ChatGPT more helpful, harmless, and aligned with human expectations, beyond just raw next-token prediction.

Alignment refers to ensuring an AI model's outputs match human values, intentions, and safety expectations.

• Addresses issues like avoiding harmful content, following instructions faithfully, and being honest about uncertainty.
• Achieved through techniques like RLHF, instruction tuning, and constitutional AI approaches, rather than raw capability improvements alone.

A Foundation Model is a large model pretrained on broad data, designed to be adapted (via fine-tuning or prompting) to many different downstream tasks.

• Examples: GPT-4, Claude, Llama — general-purpose bases rather than models built for one narrow task.
• The term emphasizes that these models serve as a "foundation" upon which many specialized applications are built.

They differ in accessibility and control:

• Open-source (e.g., Llama, Mistral): weights are publicly available, can be self-hosted, fine-tuned, and inspected freely.
• Closed-source (e.g., GPT-4, Claude): accessed only via API, with the underlying weights and training details kept private by the provider.
• Trade-offs include cost, control, data privacy, and typically raw capability (closed models are often, though not always, more capable).

Prompt Injection is an attack where malicious instructions are embedded in input data (like a webpage or document) that an LLM processes, tricking it into ignoring its original instructions.

• Especially dangerous in agentic systems that can take real actions (e.g., sending emails, running code) based on LLM output.
• Mitigations include strict input/output validation, sandboxing tool access, and treating untrusted content as data rather than instructions.

Both are outputs of language models, but serve very different purposes:

• Embeddings: a fixed-size numeric vector representing the meaning of text, used for similarity search/comparison.
• Completions: generated text continuing or responding to a prompt, used for conversational or content-generation tasks.
• Many providers offer separate, cheaper, specialized models just for generating embeddings.

Cosine Similarity measures the angle between two vectors, indicating how similar their direction is (regardless of magnitude).

• Ranges from -1 (opposite) to 1 (identical direction); used to compare embedding vectors.
• Core to semantic search: finding the stored document embeddings closest in meaning to a query's embedding.

Chunking splits large documents into smaller pieces before generating embeddings, since embedding models and context windows have size limits.

• Chunk size affects retrieval quality: too large loses precision, too small loses context.
• Common strategies include fixed-size chunks with overlap, or splitting along natural boundaries (paragraphs, sections).

They retrieve results using different matching logic:

• Keyword search: matches exact words/phrases (e.g., traditional full-text search) — misses synonyms or rephrased queries.
• Semantic search: matches based on meaning using embeddings, so "car" and "automobile" can both retrieve relevant results even without exact keyword overlap.

Chat-based LLM APIs structure conversations into roles:

• System: sets overall behavior/instructions for the model (usually invisible to the end user).
• User: represents the human's input/questions.
• Assistant: represents the model's previous responses, included so the model has conversational context/memory.

Distillation trains a smaller "student" model to mimic the outputs of a larger "teacher" model.

• Produces a much smaller, faster model that retains much of the teacher's capability for specific tasks.
• Used to create efficient models that can run cheaply at scale, or on resource-constrained devices.

Both measure performance, but from different angles:

• Latency: time taken to get a response for a single request (important for interactive chat UX).
• Throughput: total number of requests/tokens processed per unit time across many requests (important for batch processing or high-traffic systems).
• Optimizing for one can sometimes trade off against the other (e.g., batching improves throughput but may increase per-request latency).

Streaming sends the model's output incrementally, token by token, as it's generated, instead of waiting for the entire response to complete.

• Significantly improves perceived responsiveness in chat interfaces, since users see text appearing immediately.
• Implemented via server-sent events (SSE) or WebSocket connections in most LLM APIs.

A Guardrail is a safety mechanism that constrains an LLM's input or output to prevent harmful, off-topic, or policy-violating behavior.

• Can be implemented via system prompts, output filtering/classification, or dedicated moderation models run before/after the main LLM call.
• Essential for production applications exposed to untrusted user input.

They achieve customization very differently:

• Prompting: no model changes — behavior is guided purely through the instructions/context given at inference time. Fast, cheap, reversible.
• Fine-tuning: actually updates the model's weights using labeled examples — more powerful for deeply ingrained behavior changes, but requires data, compute, and retraining for updates.

Several complementary techniques help reduce hallucination beyond RAG:

• Lower temperature: makes output more deterministic and less speculative.
• Explicit uncertainty instructions: prompting the model to say "I don't know" when unsure, rather than guessing.
• Citation requirements: asking the model to cite sources for claims, making fabrications easier to spot.
• Output verification: having a second pass (human or automated) fact-check generated content before it's used.

In self-attention, each token is projected into three vectors:

• Query (Q): represents what the current token is "looking for."
• Key (K): represents what each token "offers" to be matched against.
• Value (V): the actual content passed along, weighted by how well Query matches Key.
• The attention score is computed from the Query-Key dot product, then used to weight the Values that get combined into the output.

They differ in control, cost structure, and operational burden:

• API-based (e.g., OpenAI, Anthropic APIs): pay-per-use, no infrastructure to manage, always up to date, but data leaves your environment and costs scale with usage.
• Self-hosted (e.g., running Llama on your own GPUs): full control over data and customization, potentially cheaper at very high volume, but requires significant infrastructure and ML ops expertise.

Retrieval typically combines an embedding model and a vector database:

1. The user's query is converted into an embedding.
2. The vector database finds the k most similar stored document chunks (via cosine similarity or similar metric).
3. Those chunks are inserted into the LLM's prompt as context before generating a response.

RAG systems can fail in several characteristic ways:

• Poor chunking: retrieved chunks lack enough context to be useful.
• Irrelevant retrieval: the vector search returns semantically "close" but actually unhelpful documents.
• Context overload: too many retrieved chunks crowd out the model's ability to focus on the most relevant information.
• The model can still hallucinate even with correct retrieved context, if it doesn't properly ground its answer in it.

They're trained and optimized for different outputs:

• Embedding Model: outputs a fixed-length numeric vector representing meaning — used for search/comparison, not for generating readable text.
• Generative Model: outputs human-readable text (or other media) as a continuation/response to a prompt.
• Many providers offer both as separate, differently-priced API endpoints.

Few-shot Learning describes a model's ability to learn a new task from only a handful of examples, rather than requiring a large labeled dataset.

• Large pretrained models exhibit strong few-shot (and even zero-shot) capabilities because of the broad knowledge learned during pretraining.
• In LLMs specifically, this often manifests as in-context learning — providing a few examples directly in the prompt.

Both come from the same architecture, but differ in later training stages:

• Base model: only pretrained on raw text prediction — completes text but doesn't reliably follow instructions or hold a conversation.
• Chat/Instruct model: further fine-tuned (often with RLHF/instruction tuning) specifically to follow instructions and engage in helpful dialogue.

Context Stuffing refers to cramming excessive, often irrelevant, information into an LLM's prompt/context window.

• Can dilute the model's attention across too much text, making it harder to identify what's actually relevant ("needle in a haystack" problem).
• Best practice is to retrieve and include only the most relevant, well-curated context rather than maximizing volume.

An Eval is a systematic test suite used to measure an LLM application's quality, accuracy, or safety against a defined set of criteria or example cases.

• Can be automated (comparing output against expected answers, or using another LLM as a judge) or human-reviewed.
• Essential for catching regressions when changing prompts, models, or RAG pipelines before deploying to production.

They differ in how strictly the output format is controlled:

• Free-form: the model generates natural, unstructured prose.
• Structured Output: the model is constrained (via JSON schema, function calling, or grammar-based decoding) to produce output in a specific, machine-parseable format.
• Structured output is essential when LLM responses need to be reliably consumed by other software, rather than just read by a human.