Generative AI Engineer interview questions

A generative model learns a data distribution it can sample from, while a discriminative model learns to predict a label or boundary from an input.

• A diffusion model can create a new image resembling its training distribution.
• A discriminative image classifier maps an existing image to labels such as cat or dog.
• Some learned representations can support both tasks, but generation and classification use different objectives and outputs.

A latent representation is a learned compact description of data in a space where the model can manipulate useful features.

• Nearby latent points often decode to outputs with similar high-level properties, although this is not guaranteed for every model.
• A latent can retain shape and composition while omitting some pixel-level detail.
• Sampling or editing latents is cheaper than operating on every pixel when the latent tensor is much smaller.

An autoencoder learns to reconstruct its input through an encoder, a bottleneck representation, and a decoder.

• The encoder maps an image or other asset into a lower-dimensional code.
• The decoder uses that code to reconstruct an output close to the original input.
• A reconstruction loss trains both parts, and the bottleneck forces the code to retain useful information instead of copying every value directly.

A variational autoencoder learns a distribution for each input's latent code and trains with an ELBO that balances reconstruction against a regular latent space.

• The encoder predicts parameters such as a mean and variance rather than one fixed code.
• The reconstruction term rewards latents that let the decoder reproduce the input.
• The KL term keeps the learned posterior near a chosen prior, usually a standard normal, so new latents can be sampled.
• The ELBO is optimized because the exact data likelihood involves an intractable integral over possible latents.

The reparameterization trick makes stochastic latent sampling compatible with gradient-based training.

• The encoder predicts a mean and scale, then sampling is written as z equals mean plus scale times random epsilon.
• Epsilon comes from a fixed distribution, so randomness is separated from the learned encoder outputs.
• Gradients can then flow through the mean and scale into the encoder during backpropagation.

A GAN trains a generator to create synthetic samples and a discriminator to distinguish real samples from generated ones.

• The generator maps random noise, and sometimes conditioning, into an image or other asset.
• The discriminator receives real and generated examples and outputs a real-versus-fake judgment.
• Training alternates updates so the discriminator supplies a learned training signal to the generator.

Adversarial loss rewards the discriminator for separating real from generated samples and rewards the generator for fooling the discriminator.

• The two objectives form a game rather than a fixed pixel-by-pixel target for the generator.
• In practice, the non-saturating generator loss is often used because it gives stronger gradients early in training.
• If one network becomes much stronger, the other can receive poor gradients, so update balance and learning rates matter.

Mode collapse occurs when a generator produces only a narrow set of outputs even though the real data contains much more variety.

• I would generate many samples from different noise inputs under fixed settings and look for repeated identities, poses, layouts, or textures.
• Feature-space distances, clustering, and coverage or recall metrics can reveal low diversity within the generated set and missing regions of a real validation distribution.
• Sharp individual images do not rule out collapse, and nearest training-neighbor checks address memorization rather than generated-sample diversity.

The forward diffusion process gradually adds known noise to clean training data until it becomes close to random noise.

• At each timestep, a small amount of Gaussian noise is added according to a predefined schedule.
• A closed-form relation can produce a noisy sample at a chosen timestep directly during training.
• The forward process is fixed rather than learned; it creates supervised noisy and clean pairs for training the denoiser.

Reverse denoising starts from noise and repeatedly uses a trained model to estimate a less noisy sample.

• The denoiser receives the current noisy latent or image together with its timestep.
• A scheduler converts the model prediction into the next sample using the selected sampling rule.
• Repeating this path toward timestep zero produces a structured result from the learned data distribution.

A noise schedule defines how much signal is removed or noise is added across diffusion timesteps.

• It can be expressed through beta, alpha, or sigma values depending on the formulation and scheduler.
• Early and late timesteps have different signal-to-noise ratios, which changes what the denoiser must recover.
• Training and inference schedules must be compatible, because an arbitrary mismatch can reduce sample quality.

Timestep embeddings tell the denoiser how noisy the current sample is so it can apply the appropriate transformation.

• The same network sees inputs ranging from almost clean to almost pure noise.
• A numeric timestep is converted into a vector, often with sinusoidal features followed by learned layers.
• That vector conditions blocks throughout the denoiser instead of being treated as an image pixel.

Epsilon prediction asks the model for the noise in the current sample, while x0 prediction asks for an estimate of the clean sample directly.

• Given the timestep and schedule, either prediction can be converted into the other mathematically.
• The parameterization changes the target scale and how training emphasizes different noise levels.
• The pipeline scheduler must interpret the model output using the parameterization on which the checkpoint was trained.

Classifier-free guidance strengthens conditioning by combining a conditional denoiser prediction with an unconditional prediction from the same model.

• During training, conditioning is sometimes dropped so the model learns both behaviors.
• At inference, the difference between conditional and unconditional predictions is scaled and added to the unconditional result.
• A higher guidance scale can improve prompt adherence but may reduce diversity or create oversaturated artifacts.

More sampling steps usually increase latency linearly, while quality gains often level off after enough steps for the chosen model and scheduler.

• Each step normally requires another denoiser evaluation, which is the expensive part of image generation.
• Too few steps can leave composition or fine details underdeveloped unless the model was trained or distilled for short sampling.
• I would compare candidate step counts on a fixed prompt and seed set because the best count is scheduler and checkpoint specific.

Latent diffusion performs the denoising process in a compressed learned space instead of directly over full-resolution pixels.

• An encoder converts an image into a smaller spatial latent tensor before diffusion training or image-conditioned inference.
• The denoiser processes fewer values, reducing memory and compute compared with pixel-space diffusion.
• A decoder converts the final clean latent back into an image, so compression quality still affects the output.

The VAE translates between pixel images and the latent space in which the diffusion model operates.

• Its encoder is used for training images and for tasks such as img2img or inpainting that begin with an image.
• Its decoder turns the final denoised latent into visible pixels.
• The VAE is not the main denoiser, and reconstruction errors can appear as color shifts or lost fine detail even when denoising succeeds.

A U-Net denoiser transforms a noisy image or latent into the prediction required for one reverse diffusion step.

• Its downsampling path captures broad spatial context, while its upsampling path restores spatial resolution.
• Skip connections carry local detail between matching resolutions.
• Timestep and text conditioning are injected into its blocks so the prediction reflects both noise level and prompt.

Cross-attention lets image or latent features retrieve relevant information from text representations while denoising.

• Queries come from current image features, while keys and values come from encoded prompt tokens.
• Different spatial positions can attend to different words, helping connect objects and attributes to regions.
• Cross-attention improves conditioning but does not guarantee correct counting, binding, or layout.

The text encoder converts tokenized prompt text into contextual vectors that condition the image denoiser.

• The tokenizer first maps text to the vocabulary and sequence format expected by that encoder.
• The encoder represents each token in context, so the same word can produce different vectors in different prompts.
• Many pipelines keep the text encoder frozen during inference and most adapter training, while the denoiser consumes its outputs.