Senior/Lead Research Scientist, Model Evaluations
Thomson Reuters
Job Description
Thomson Reuters Labs builds and ships custom models for professional workflows requiring fiduciary-grade AI. The Thomson LLM family was developed by training leading open-weight models on decades of authoritative Thomson Reuters content, with thousands of hours of subject-matter-expert input. Owning the model layer is how we earn trust in the most regulated, highest-stakes professional workflows.
See the detailed technical report for more information on the latest model release:
The evaluation team focuses on providing signal for model development and adoption using a combination of benchmarks and targeted evaluations to assess performance on highly complex legal and professional workflows. The team works with legal experts to adapt their expertise into benchmarks and with product leaders to ensure model capabilities are aligned with real, user-facing flows in professional work.
As a research scientist on the team, you would be working with researchers to design, develop, optimize and scale the evaluation suite as Thomson scales to newer releases and product integrations. You’ll work in cutting edge areas of AI development, such as multi-agent systems and decision models to keep the Thomson model suite at the frontier of specialized professional LLMs.
Key Responsibilities
- Build and operate the evaluation suite that runs Thomson evaluations. Leverage our existing suite to identify and understand the strengths and weakness of Thomson. Create new evaluations to meet the gaps in our coverage, based on a combination of product, external, and aspirational use cases for the model.
- Lead cross-functional teams in the creation of evaluation assets. Use your technical expertise to guide teams of scientists, engineers, and subject-matter experts working on evaluations and benchmarks. Mentor junior scientists in developing their own research and technical skills.
- Inform the technical direction of Thomson. Propose opportunities for improvements on topics such as more robust and reliable evaluation, new opportunities for applying LLMs, and optimizations on existing assets. Draw on evaluation results to inform our training objectives. Provide recommendations to senior leadership on technical trade-offs in model development.
- Conduct fundamental research. Develop new ideas in model evaluation across science of evaluations, new benchmarks, and safety and alignment. Publish research findings in top AI/ML/NLP conferences on a regular cadence. Integrate new research findings into model development and product pipelines.
Required Qualifications
- Hands-on research experience in evaluations – with examples of designing, building and releasing benchmarks for frontier models or agents, including research publications in top conferences such as NeurIPS/ICML/ICLR/*ACL.
- Experience working closely with researchers and cross-functional stakeholders to translate ideas into practical evaluations.
- Hands-on Python coding experience implementing benchmarks, agents or other systems built around LLMs.
- Familiarity with current developments in LLMs, with an interest in staying up to date with the evolving frontier.
- A PhD in Computer Science or a related field, or equivalent practical experience, plus industry experience working on post-training or evaluation. Professional experience will be considered as a factor for levelling between Senior and Lead.
Preferred Qualifications
- Experience working with specialized domain experts in areas such as law, tax, or similar professions is especially desirable.
- Familiarity with AI-assisted coding and with leveraging coding agents at scale to accelerate work.
- Experience in experimental design and best practices for working with human data.
Even if you don’t have all the above requirements, we encourage you to apply to explore a mutual fit!
#LI-FP3
What’s in it For You?
- Hybrid Work Model: We’ve adopted a flexible hybrid working environment for our office-based roles while delivering a seamless experience that is digitally and physically connected.
- Flexibility & Work-Life Balance: Flex My Way is a set of supportive workplace policies designed to help manage personal and professional responsibilities, whether caring for family, giving back to the community, or finding time to refresh and reset. This builds upon our flexible work arrangements, including work from anywhere for up to 8 weeks per year, empowering employees to achieve a better work-life balance.
- Career Development and Growth: By fostering a culture of continuous learning and skill development, we prepare our talent to tackle tomorrow’s challenges and deliver real-world solutions. Our Grow My Way programming and skills-first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future.
- Industry Competitive Benefits: We offer comprehensive benefit plans to include flexible vacation, two company-wide Mental Health Days off, access to the Headspace app, retirement savings, tuition reimbursement, employee incentive programs, and resources for mental, physical, and financial wellbeing.
- Culture: Globally recognized, award-winning reputation for inclusion and belonging, flexibility, work-life balance, and more. We live by our values: Obsess over our Customers, Compete to Win, Challenge (Y)our Thinking, Act Fast / Learn Fast, and Stronger Together.
- Social Impact: Make an impact in your community with our Social Impact Institute. We offer employees two paid volunteer days off annually and opportunities to get involved with pro-bono consulting projects and Environmental, Social, and Governance (ESG) initiatives.
- Making a Real-World Impact: We are one of the few companies globally that helps its customers pursue justice, truth, and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world.
DISCLAIMER
The above information in this description has been designed to indicate the general nature and level of work performed by employees within this classification. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities, and qualifications required of employees assigned to this job.
About Us
Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news.
We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound exciting? Join us and help shape the industries that move society forward.
As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace.
We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here.
Learn more on how to protect yourself from fraudulent job postings here.
More information about Thomson Reuters can be found on thomsonreuters.com
- ...curious and rigorous researcher who wants to push... ...large language models (LLMs) learn to... ...Labs is seeking a Lead Research Scientist to drive research... ...efficiency, and evaluation with domain experts... ...as lead or senior author at top-tier... ...You? Hybrid Work Model: We’ve adopted a...SeniorVollzeitPostdoc Stelle
- ...Lead Applied Scientist, Document Understanding This position is based in either... ...solve publishing-quality research problems with immediate production... ...workflows Design robust evaluation frameworks for document... ...it For You? Hybrid Work Model: We’ve adopted a flexible...EmpfohlenVollzeitSoforteinstellung
- ...projects and promising early-stage research from academia together under one... ...and unleash innovation. This model allows science and small teams of experts to lead the way. We build bridges to groundbreaking... ...Comfortably influencing senior stakeholders across medical, clinical...SeniorVollzeitSchichtarbeit
- ...Zug Institute for Blockchain Research at the University of Lucerne The University of... ...proposals they advocate in response. 3.Evaluate public support for different welfare state... ...text analysis, NLP, or large language models for social science research) is a plus...EmpfohlenPostdoc StelleFestanstellungFlexible Arbeitszeit
- ...You act as a technical lead for a cross-functional software team developing integrated solutions for molecular diagnostics systems.... ...testing and continuous integration Act as a mentor and role model guiding team members in technical development Collaborate with...SeniorTemporärVollzeit
- ...unmet need. As a global leader in myostatin biology, a... ...: The Director/Senior Director, EMEA Health Economics and Outcomes Research (HEOR) Leader, reporting... ...Medical Affairs, will lead the development and execution... ...and cost-effectiveness model adaptations, evidence...SeniorVollzeit
- ...assets. We are looking for a Manager Research Analyst to join our research team. In this... ...Diligence & Manager Research Support and help lead in-depth due diligence on quantitative... ...capacity, and historical performance Evaluate managers' quantitative research process,...
- ...Lead Software Engineer, AI What if the AI agents you architect empower... ...tools that let a small, senior group move at a pace well beyond... ...agent observability and evaluation practices (tracing, automated... ...in it For You? Hybrid Work Model: We’ve adopted a flexible hybrid...SeniorVollzeitSchichtarbeit
- ...where muscle atrophy remains a critical unmet need. As a global leader in myostatin biology, a field focused on proteins that regulate... ...of Position: The Patient Advocacy & Corporate Communications Lead, EMEA will be responsible for developing and executing...SeniorVollzeit
- ...projects and promising early-stage research from academia together under one... ...and unleash innovation. This model allows science and small teams of experts to lead the way. We build bridges to groundbreaking... ...Support the review, evaluation and compliant administration of...SeniorVollzeitSchichtarbeit
- ...Clinical Development Lead (ECDL) to join... ...Oncology Research and... ...and Clinical Scientists who will design... ...stakeholders, including senior management,... ...Committees in the evaluation and assessment... ...a physician scientist in the pharmaceutical... ..., PK/ PD modeling and simulation...SeniorVollzeitRemote job
- ...real world. The Senior Medical Director,... ...senior physician leader within the clinical... ...Clinical Development Lead (CDL), the Senior... ...closely with the Senior Scientist and cross-... ...judgement, safety evaluation, and medical interpretation... ..., Clinical Research, Clinical Trial Designs...SeniorVollzeit
- ...by early 2029 . The Senior Medical Director, Oncology... ...Clinical Development Lead (CDL), Oncology... ...responsible for clinical research activities pertaining... ...collaborate with physicians, scientists, regulatory... ...Investigators in the evaluation and assessment of publications...SeniorVollzeit
- ...projects and promising early-stage research from academia together under one... ...and unleash innovation. This model allows science and small teams of experts to lead the way. We build bridges to groundbreaking... ...executive-level presentations to senior stakeholders Confident and...SeniorVollzeit
- ...™ – The Human Data Science Company™ – is a leading global provider of advanced analytics, technology solutions and clinical research services to the life sciences and healthcare... ...Research & Competitive Intelligence Lead qualitative market research and competitive...VollzeitSoforteinstellung
- ...planners, designers, engineers, scientists, digital innovators, program... ...Job Description The Senior Counsel, Artificial Intelligence & Data Privacy will lead and advise on global data protection... ...including rights to training data, model and dataset licenses, open-...SeniorVollzeitÖffentlicher DienstRemote job
- ...As a Senior Product Security Engineer, you will join our team of talented... .... ~ You will support threat modeling sessions and secure code... ...in it For You? Hybrid Work Model: We’ve adopted a flexible hybrid... ...tools and knowledge to grow, lead, and thrive in an AI-enabled...SeniorVollzeit
- ...Leadership, the VP, Medicine Development Leader (MDL) – Cancer Vaccines will strategically lead the optimal global development of... ...of contact and spokesperson to senior management and senior boards (e.g... ...successfully across functions – research, commercial, regulatory, global...SeniorVollzeitInterim
- ...Senior Paid Media Specialist - BYD & DENZA... ...team. BYD is a leading, high-tech multinational... ...of BYD, a global leader in electric... ...campaign delivery and evaluate performance.... ...with Swiss media research and audience measurement... ...environment with short lead times, and keeps...SeniorVollzeit
- ...Global CRM & Salesforce Lead (m/w/d) Pensum: 100 % Anstellung: Festanstellung Start: Nach Vereinbarung Ihre nächste Herausforderung Für unseren Kunden, ein international agierendes Unternehmen, suchen wir eine management- und technologieorientierte Führungspersönlichkeit...Festanstellung
- ...Investmententscheidungen Aufbereitung von Investmentanalysen, Präsentationen und Entscheidungsgrundlagen für das Investmentteam Austausch mit externen Research-Anbieter, Asset Manager und weiteren Marktteilnehmer Eigenständige Investmentanalysen mit Fokus auf Aktien und Multi-Asset...SeniorKurzfristige Beschäftigung
- ...Senior Manager leading insights generation, forecasting, and competitive... ...Responsibilities Lead insights and business-... ...Develop forecasting models and assess business... ...intelligence operating model and lead monitoring across... ...Market research budget management and...Senior
- ...Johnson & Johnson seeks a Senior Supply Chain Business Intelligence Analyst to support... ...Bachelor's degree in Supply Chain, Operations Research, Engineering, Computer Science, or... ...experience Advanced SQL skills for data modeling, complex queries, and performance tuning...Senior
- ...develop customer-facing applications and platform capabilities as a senior full stack engineer, working across frontend, backend and... ...Swiss work permit required Conditions Workload: 100 % Work model: Hybrid Option of a permanent position after the assignment if...SeniorInterimIm Frühling
- ...Global Life Science Vertical Lead – SI Buildings 80-100% ID de Puesto 526341 Publicado... ...for pharmaceutical, biotechnology, research, laboratory, and advanced medtech production... ...contribute authentically. Flexible working models and remote working are a reality for us...VollzeitRemote jobFlexible Arbeitszeit
- ...international buyers and traders Act as a good resource to other leaders in the biofuels industry with good networking skills with... ...~ Basic risk analysis and management, hedging, economic evaluation, RED regulation, financial, credit and accounting knowledge in...SeniorVollzeit
- ...Johnson & Johnson seeks a Sales Lead to drive intracardiac echocardiography expansion across West EMEA in electrophysiology. The role combines commercial strategy, account leadership, and cross-functional collaboration to build market adoption and revenue growth while...
- ...professional for the position of Policy and Public Affairs Regional Lead – EU & UK of our Group to work alongside our dynamic team.... ...their business implications, and provide clear, timely advice to senior management and relevant functions. ~ External representation:...SeniorVollzeitÖffentlicher Dienst
- ...remains a critical unmet need. As a global leader in myostatin biology, a field focused on... ...Position: Scholar Rock is seeking a senior medical affairs professional to advance... ...expertise with a strong track record of leading high-impact medical initiatives and engaging...SeniorVollzeit
- ...Infrastructure und Gebäudeautomation suchen wir einen erfahrenen Senior HVAC Engineer – Smart Buildings & Building Automation zur... ...semantische Gebäudemodelle Anbindung von Web of Things (WoT) Thing Models zur Optimierung der Geräteintegration Einsatz von KI für...SeniorTemporärSoforteinstellung
Wollen Sie mehr Stellenangebote erhalten?
Abonnieren Sie und erhalten Sie ähnliche Stellenangebote wie Senior/Lead Research Scientist, Model Evaluations. Seien Sie der Erste, der sich bewirbt!
