Language Models are not good in numeric mathematical understanding? Happy to share our #ICLR2024 Spotlight paper: "SNIP: Symbolic-Numeric Integrated Pre-training", A multi-modal CLIP-like model for math! (https://lnkd.in/eYbttTfA) A collaboration with Parshin Shojaee, Chandan Reddy, and Amir Barati Farimani SNIP takes large-scale synthetic data of mathematical functions and their numeric data observations as two modalities, and learns powerful representations with the symmetric contrastive learning. SNIP representations show informative patterns for the non-trivial task of cross-modal property prediction between symbolic and numeric domains. We have also tested SNIP for more complex tasks such Symbolic Regression (discovery of governing math equations from data) and observe that SNIP representations provide great landscape for the search of optimal mathematical functions. We really hope this model can be applied to other tasks in science and spark more research on numeric-symbolic understanding in LLMs. We release our code & models publicly: 🔗Paper: https://lnkd.in/eYbttTfA 👨💻Code & Model: https://lnkd.in/efDR5dtg We have also open-soruced the code of testing SNIP on Symbolic Regression (SR) benchmarks. 👨💻SR Code & Model: https://lnkd.in/e4wWhPAz
Nice work! It’s obvious to me why numeric encoder is useful but do you think if there is any real application for the symbolic encoder as well? For example, do you think we can potentially give an incomplete equation to the symbolic encoder and expect the numeric encoder to build the relevant dataset?
Great work!!
Congratulations !!
Great work! Congrats Kazem!
Congrats Kazem!
Congratulations Kazem! 🎉
Great work, boss!
Bioengineering | Mechanical Engineering | Computational Medicine | Bioinformatics | CFD Modeling | Biofluid Dynamics
7moCongratulations Kazem 👏