ContextQMD
Back to Mistral Rs

Auto-mapping

docs/src/content/docs/reference/python/automap.md

0.8.23747 B
Original Source

TextAutoMapParams

Auto-mapping parameters for a text model. These affect automatic device mapping but are not a hard limit.

FieldTypeDefault
max_seq_lenint4 * 1024
max_batch_sizeint1

MultimodalAutoMapParams

Auto-mapping parameters for a multimodal model. These affect automatic device mapping but are not a hard limit.

FieldTypeDefault
max_seq_lenint4 * 1024
max_batch_sizeint1
max_num_imagesint1
max_image_lengthint1024

<small>Generated from mistralrs-pyo3/mistralrs.pyi.</small>