Notes

• ALF Knowlege Base
  • Reading the source code of ALF
  • Debugging using VScode
  • Training scheme
  • Algorithm
  • TimeStep
  • DataTransformers
  • Environment
  • ParallelAlfEnvironment and ThreadEnvironment
  • Snapshot
  • Differences with the Tensorflow version of ALF
• PyTorch Notes
  • 1. Transform / Bijector
  • 2. Distribution / Probability
  • 3. Optimizer
  • 4. Inplace Operator
  • 5. Constructing tensor from numpy float scalar
  • 6. Type promotion
• Asynchronous training guide
  • Background
  • Difference with IMPALA
  • Throughput EPS
  • Optimize EPS
  • Experiments
  • Results analysis
  • Different batch size
  • Conclusion
• Estimating the derivative of an expectation
  • Scenario 1: Derivative of an entropy
  • Scenario 2: SAC discrete actor training with a CriticNetwork
• How to preview this website locally
  • Troubleshooting
• How to write docstrings (or RST files)
  • Raw docstring
  • Math
  • Code segment
  • Class init docstring
  • Arguments
  • Returns
  • Raises
  • Notes and warnings
  • More examples
• Fetch Environments with SAC + Action Repeat
  • FetchReach
  • FetchPush
  • FetchPickAndPlace
  • FetchSlide
• Soft Actor-Critic with hybrid action types
  • Actor training loss
  • Different entropy coefficients
• Computing probabilities of a transformed distribution
  • Background
  • What is the essential difference between detach and no detach of \(y\) for PG?
  • PyTorch’s sample() and rsample()
  • Action transformations as environment wrappers
• Hybrid and Offline RL Training
  • 1. RL Training Modes in ALF: Online, Offline and Hybrid
  • 2. Hybrid RL Training
  • 3. Offline Buffer Structure
  • 4. Offline Buffer Generation
  • 5. How to Implement New Offline and Hybrid RL Algorithms