Events

Two day FDP on GPU Programming & Distributed Deep Learning with Pytorch

Day 1: Advanced Topics in Deep Learning with PyTorch (6 hours)

Session 1: Distributed Training with Multiple GPUs (2 hours)

• Challenges in multi-GPU training
• Data parallelism vs Model parallelism
• Implementing distributed training with PyTorch

Session 2: Model Optimization Techniques (2 hours)

• Quantization and reducing model size
• Pruning and sparsity techniques
• Mixed-precision training for faster convergence

Session 3: : Custom Loss Functionsand Metrics (1 hour)

• Implementing custom loss functions
• Defining custom evaluation metrics
• Fine-tuning model performance

Day 2: Scaling and Optimization (6 hours)

Session 4: Multi-CPU and Distributed Computing (2 hours)

• Scaling to multiple CPUs
• Distributed computing with PyTorch
• Managing data and communication overhead

Session 5: Efficient Data Loading and Augmentation (1 hour)

• Optimizing dataloading pipelines
• Data augmentation for improved model generalization
• Using PyTorch’s dataloading utilities

Session 6: Hyperparameter Tuning and AutoML (2 hours)

• Hyperparameter optimization strategies
• Tools like Optunaand HyperOpt
• Automated machine learning (AutoML) pipelines

Session 7: Workshop Conclusion and Q&A (1 hour)

• Recap of advanced topics covered

Open Q&A session for participants