You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like for there to be support for Vision Encoder Decoder models, such as TrOCR, with the BetterTransformer tool.
Motivation
I have a TrOCR model I want to speed up inference for (reducing latency). I've tried every quantization option I could, and none of them are speeding up my inference. If BetterTransformers could support the VisionEncoderDecoder model type, that would make my life a whole lot easier!!
Your contribution
I'm not very knowledgeable about open source contribution, especially on such a big project with so many complicated parts, so I don't know if I could really help with this.
The text was updated successfully, but these errors were encountered:
Feature request
I would like for there to be support for Vision Encoder Decoder models, such as TrOCR, with the BetterTransformer tool.
Motivation
I have a TrOCR model I want to speed up inference for (reducing latency). I've tried every quantization option I could, and none of them are speeding up my inference. If BetterTransformers could support the VisionEncoderDecoder model type, that would make my life a whole lot easier!!
Your contribution
I'm not very knowledgeable about open source contribution, especially on such a big project with so many complicated parts, so I don't know if I could really help with this.
The text was updated successfully, but these errors were encountered: