---
format: "aidr-story-markdown/v1"
id: "663a5e2e30d0e11bd12d5b654d2564403a6592294f3283b8857d3a4205a620eb"
canonical_url: "https://aidr.today/663a5e2e?lang=en"
title: "Preference Model Open-Sources Karotte RL Framework Used in 1M Runs"
lang: "en"
requested_lang: "en"
available_langs: ["en","vi"]
translation_fallback: null
fallback_fields: []
published_at: "2026-10-07T19:08:09.000Z"
category: "Open Source"
topics: ["open-source"]
source_urls: ["https://marketbrief.now/ai/preference-model-open-sources-karotte-rl-framework-used-in-1m-runs-c9b5657a","https://huggingnews.com/ai/preference-model-open-sources-karotte-rl-framework-used-in-1m-runs-c9b5657a"]
summary: "Preference Model has open-sourced Karotte, a reinforcement learning framework it developed over the last year and tested across more than 1M evaluation runs, with a16z announcing an investment in the company to support infrastructure for capable and aligned models."
---

# Preference Model Open\-Sources Karotte RL Framework Used in 1M Runs

> [Open the canonical story](<https://aidr.today/663a5e2e?lang=en>)

**Published:** 2026-10-07T19:08:09.000Z
**Category:** Open Source
**Topics:** open\-source

## Summary

Preference Model has open\-sourced Karotte, a reinforcement learning framework it developed over the last year and tested across more than 1M evaluation runs, with a16z announcing an investment in the company to support infrastructure for capable and aligned models\.

## Sources

- [Story source](<https://marketbrief.now/ai/preference-model-open-sources-karotte-rl-framework-used-in-1m-runs-c9b5657a>)
- [Story source](<https://huggingnews.com/ai/preference-model-open-sources-karotte-rl-framework-used-in-1m-runs-c9b5657a>)