Benchmarking

In this section we have documentation on the tools/techniques/models/existing frameworks/etc… we are using for benchmarking

• Evaluation plan
  • Orca approach
  • Proposed approach
• Existing frameworks
  • Fairseq MoE
  • DeepSpeed-MoE
  • FasterTransformer MoE
• MoE architectures
  • G-Shard
  • Fairseq’s 15B-parameters LM MoE model