Loading…
2
The ML engineer at the Seattle meetup who flipped my take on small models
I used to be all in on the giant model hype, figured anything under a billion parameters was a toy. Then at a meetup in Seattle back in March, this ML engineer named Priya showed me a 3B model she fine-tuned on her laptop in about 6 hours that beat the big API model on her company's ticket routing by 20 points. She pulled it up right there on her screen and ran 50 test cases in front of me. What got me was when she said, "Why pay for a semi truck when you're hauling groceries?" I've been running small local models for most of my work since then. Anybody else made the switch to small models, and what finally changed your mind?
1 comments
Log in to join the discussion
Log In1 Comment
the_lucas28d ago
That Priya demo sounds slick but 50 test cases on one company's ticket data is a pretty narrow slice to draw a big lesson from. I ran a fine-tuned 7B on our support stuff and it looked great in testing, then fell apart the second real users started typing messy half-sentences with typos and three questions in one message. Small models can be awesome when the job is narrow and the input stays clean, but the moment you need broad world knowledge or the task shifts, you're back to babysitting it. What kind of work are you throwing at the local models now?
9