Launch HN: Parachute (YC S25) – Guardrails for Clinical AI

Hi jph! Not intending to subvert the thread, but I'd love to chat to someone like you. The non-profit I work at has been working on democratizing evals. This wouldn't be to ensure your in-house AI is up to scratch (parachute looks ideal!), but on ensuring the general landscape of models is up-to-date on best practice, e.g. NICE and other guidance, so that everyday model users aren't misled. Such a demo eval is here: https://weval.org/analysis/uk-clinical-scenarios/08278696ca2...

We're looking for domain experts especially in high risk domains like healthcare, education, therapy. Then we'd work together co-authoring an eval in your specialism to expose and motivate AI labs to do better.

Just to say - I was impressed chatting to someone in and around NHS Wales software a few years ago when I was leading the development side of a health app. I seem to remember there were some good plans in place to join up patient health records across the Welsh trusts, which sounded very sensible.

Thanks! Would love to see how we can help!

> That list of logos you all have - are those paying customers today?

Doesn't look like it. The first list of logos is standards bodies. The second list of logos is integrations.

That’s a great point. We don’t use AI agents to grade other models. Instead, we run in-house evaluations tailored to each category of clinical AI, giving hospitals an apples-to-apples comparison between similar vendors.

This line of thinking always leaves me confused about other peoples experience in the Pre-AI world. People and systems around me fail all the time because evaluation fails. Yes, the failure modes are different but I don't consider them favorable without AI. In fact, I consider that they are better.

For example, consider what happens in this video: https://www.youtube.com/watch?v=AZhCYisIQB8&t=2s

Please don't make this mistake of thinking "aha, but you see, a human intervened!" This will never happen in the real world for the vast majority of humans in a similar scenario.

> This is cool, but I’m a little skeptical. If Parachute uses AI agents to evaluate other models, who’s evaluating the AI agents?

Usually you can run human-in-the-loop spot checks to ensure that there's parity between your LLM evaluators and the equivalent specialist human evaluator.

I wouldn't touch a YC company for this use case. All the marketing from the landing pages is just that - blabla.

My friend, you are overthinking! The funding round just came in and its smooth sailing ahead. Well for the next six months.

Thanks for your question.

We use in-house evals (based on existing state-of-the-art benchmarks) to compare ambient scribes.

If you take a deeper look into the companies on our landing page, you will see that the first list refers to the compliance standards our workflows follow and the second refers to the existing tools we integrate with.

Evidence for safe and fair AI systems is possible as long as you define what "safe" and "fair" mean for your usecase. Fairness might look like "no cohort has >5% higher false positive rate than another" and safety might mean "the model must have a false negative rate of less than 15%". Safety more so encompasses the workflows around the model, including human intervention, auditing, monitoring, etc.

Here's a good overview of fairness: https://learn.microsoft.com/en-us/azure/machine-learning/con... and there's plenty of papers discussing how to safely use predictive analytics and AI in healthcare.

I don't know if this product can give proof for safe and fair ML systems, but it's not impossible to use these things safely and fairly.

Thanks for your question. Parachute's workflows are built around widely accepted conventions for safety and fairness for AI models in healthcare such as NIST AI RMF, CHAI and HAIP's HEAAL framework.

You're right; we should not even try. Better to have 0% compliance coverage and your honor than 90% coverage as best-effort.

Acceptance criteria has entered the chat.

Yes but don’t worry there’s another YC startup who is building AI to check on AI that’s checking on AI.

Like I said above, we don’t use AI agents to grade other models. Instead, we run in-house evaluations tailored to each category of clinical AI, giving hospitals an apples-to-apples comparison between similar vendors.

This is a good question. Parachute is not a certification body so we do not help you pass audits. Instead we help with internal decision-making surrounding the implementation of AI tools. We also help hospitals keep track of why decisions we made and who made them to produce to regulators and litigators in the future. Parachute does not assume any liability during production deployment.

I don't understand your comment. What sort of time bombs do you mean?