MTOP (All Birds with One Stone: Multi-task Text Classification for Efficient Inference with One Forward Pass)

Note: This project is a work in progress; please stay tuned.

MTOP is a text encoder multi-task method that can conduct one forward pass to make predictions for all tasks. We propose prompt-based modules tailored for the multi-task setting and a conditional pooler for flexible task representations, and initialization from single task models for effective knowledge transfer. Our proposed approach gets superior performance on news tasks and the GLUE benchmark. We also release a multi-task news dataset.