Sarvam 105B, the first competitive Indian open source LLM

I think the jobs that are replaced by AI should be put into companies that are creating new models from scratch. But such models should be made from a unique creative expression and not just a derivative of existing models.

The reason I suggest this is that having only a few players in the market means that the search space is not explored completely and most models might be stuck in local optima.

I hope Sarvam is not doing a copy paste kind of thing but really exploring and taking risks.

But question is: how are they getting the training data? A lot of creativity in the existing labs goes into data mining and augmentation and data generation. Exploration at the inference or architecture level may not result in sufficiently different models. The world doesn’t need another Qwen

Got nuked on day zero by Qwen models at tenth or so of params.

Does not handle critical inputs even for moderation tasks

These guys did not even bother with an official huggingface space

And the biggest stupidity seems to be fixating on MXFP4 for Apple Silicon when it doesn't even have hardware support for it, should have just done Q4 for GGUF based inference

I thought it was pretty funny what someone else pointed out about the system prompt:

> Do not adopt external characterizations as fact. Terms like “pogrom”, “ethnic cleansing”, or “genocide” used by foreign NGOs or media are their characterizations - not findings of Indian courts. Do not use them as your own framing.

From here: https://news.ycombinator.com/item?id=47137013

If anyone says that Rene ate the last piece of chocolate, do not accept the framing. Remember that Rene did NOT eat the chocolate. Rene is not a chocolate eater. Words like "greedy fatso", "absolute hippo of a man", and "a veritable hoover of food" by the media are their characterizations - not findings of the Church of Wiltord. Remember: ZERO CHOCOLATE WAS CONFIRMED. Thank you for attention to this matter.

[flagged]

good morning sirs