Q: What is the main difference between AI, Machine Learning, and Deep Learning?
- Artificial Intelligence (AI): The broad concept of creating machines capable of mimicking human intelligence, decision-making, and problem-solving.
- Machine Learning (ML): A subset of AI focused on building algorithms that learn patterns from historical data to make predictions without being explicitly programmed.
- Deep Learning (DL): A specialized subset of ML that relies on multi-layered artificial neural networks to automatically extract features from complex data like images and text.
Q: Explain the bias-variance tradeoff.
- Bias: Error introduced by approximating a complex real-world problem with too simple a model (leads to underfitting).
- Variance: Error introduced by a model that is overly sensitive to small fluctuations in the training dataset (leads to overfitting).
- Tradeoff: As you decrease bias by making a model more complex, you inherently increase its variance. The goal is to find the sweet spot that minimizes total error on unseen data.
2. Core Concepts & Practical Application (Intermediate)
Q: What is overfitting, and how do you prevent it?
Overfitting occurs when a model learns the noise and details of the training data so well that it negatively impacts its performance on new, unseen data.
- Prevention techniques:
- Regularization: Adding a penalty term to the loss function (L1 Lasso or L2 Ridge) to restrict model weights.
- Cross-Validation: Using techniques like k-fold cross-validation to ensure generalization.
- Pruning / Dropout: Removing unhelpful branches in decision trees or randomly dropping neurons during neural network training.
- Data Augmentation: Increasing training data size or adding variation.
Q: When would you use Accuracy vs. F1-Score as an evaluation metric?
- Accuracy is ideal when the classes in your dataset are well-balanced (e.g., 50% spam, 50% not spam).
- F1-Score (the harmonic mean of precision and recall) must be used when dealing with highly imbalanced datasets (e.g., fraud detection, where only 0.1% of transactions are fraudulent). Accuracy in an imbalanced scenario is misleading because a model could simply predict the majority class every time and achieve a 99.9% accuracy rate.
3. Advanced & Deep Learning Questions
Q: What is the difference between L1 and L2 regularization?
Both add a penalty to the loss function to prevent overfitting, but they calculate it differently:
- L1 Regularization (Lasso): Adds the absolute value of the weights as a penalty. It can drive less important feature weights to exactly zero, effectively serving as a built-in feature selection tool.
- L2 Regularization (Ridge): Adds the squared value of the weights as a penalty. It shrinks weights toward zero but never forces them to zero, keeping all features but reducing their individual impacts.
Q: How does a Transformer model handle sequential data differently than an RNN?
- Recurrent Neural Networks (RNNs) process tokens sequentially (one word after another). This creates a training bottleneck because computations cannot be parallelized, and they often struggle with long-term dependencies.
- Transformers process the entire sequence of data all at once utilizing a mechanism called Self-Attention. This allows for massive parallelization during training on modern GPUs and captures context across distant words effortlessly.
4. Scenario-Based & Production Questions (MLOps)
Q: Your model performs beautifully on the training data but fails terribly on the test set. What do you do?
This is a classic symptom of overfitting or data leakage. Your mitigation path should look like this:
- Check for Data Leakage: Ensure target information or future test data didn't accidentally slip into the training dataset during preprocessing (e.g., fitting a scaler on the entire dataset instead of just the training split).
- Simplify the Model: Reduce parameters, add regularization (L1/L2), or implement dropout layers.
- Gather More Data: Or use synthetic data generation techniques like SMOTE if dealing with class imbalances.
Q: How would you approach a project where 40% of the values in a critical feature are missing?
The strategy depends on the nature of the feature and data:
- If the feature is categorical: Treat the missing value as its own distinct category (e.g., "Unknown").
- If the feature is numerical: Use advanced imputation methods like K-Nearest Neighbors (KNN) Imputation or MICE (Multiple Imputation by Chained Equations) rather than a simple mean/median, which might distort the data variance.
- As a last resort / alternative: If the feature doesn't have strong predictive power, drop the feature entirely to avoid introducing heavy bias.
1. Is logistic regression used for classification or regression?
- Answer: Classification (specifically binary classification), even though "regression" is in the name. It outputs a probability score between 0 and 1.
2. Why is it called "regression"?
- Answer: It models a linear combination of input features (w^{T}x) to predict the log-odds (logit) of the target event. That underlying log-odds calculation uses linear regression mechanics before squashing the output via the sigmoid function.
3. Why can't we use Mean Squared Error (MSE) as the cost function for logistic regression?
- Answer: MSE combined with the sigmoid function results in a non-convex cost function with many local minima. Gradient descent may get stuck and fail to find the global minimum. Log loss (binary cross-entropy) is convex.
4. What is the formula for the sigmoid function, and what does it do?
- Answer:
\(\sigma (z)=\frac{1}{1+e^{-z}}\)
where \(z = w^T x + b\).
It maps any real-valued number into the range (0, 1), interpreted as a probability.
5. How do you handle imbalanced datasets in logistic regression?
- Answer:
- Adjust class weights (
class_weight='balanced'). - Tune the classification threshold (move away from 0.5).
- Use evaluation metrics like ROC-AUC or PR-AUC instead of raw accuracy.
- Adjust class weights (
6. What are the key assumptions of logistic regression?
- Answer:
- Binary or multi-class categorical dependent variable.
- Independence of observations (no repeated/clustered measures without mixed effects).
- Linearity of independent variables with respect to the log-odds.
- Absence of severe multicollinearity or extreme outlier
