CONVERSION

Here is how to convert a GPT2 model generated outside of transformers

• Megatron-LM-generated model:

  Use convert_megatron_gpt2_checkpoint.py

• big-science fork of Megatron-Deepspeed-generated model:

  Use the instructions here. This approach uses a set of scripts that require the use of this particular fork of Megatron-Deepspeed.