The AI Alignment Forum

MARCH 5, 2025

Published on March 5, 2025 10:56 PM GMT In collaboration with Scale AI, we are releasing MASK (Model Alignment between Statements and Knowledge) , a benchmark with over 1000 scenarios specifically designed to measure AI honesty. Even when aware of the truth, they often choose to lie in many scenarios in our benchmark.

Benchmark 59

Benchmark Measure System Evaluation 59

Beth's Blog: How Nonprofits Can Use Social Media

JUNE 23, 2011

I remember thinking to myself at the time, well if one networked nonprofit can do this, couldn’t a network of networked nonprofits use real-time collaborative benchmarking data sharing for learning? Number of followers has become a benchmark stat like the number of hits on our web pages used to be.

Benchmark 95

Benchmark Network Learning Museum 95

Google Research AI blog

FEBRUARY 17, 2023

In light of this data scarcity, we position FRMT as a benchmark for few-shot translation, measuring an MT model’s ability to translate into regional varieties when given no more than 100 labeled examples of each language variety. While human evaluation is the best way to be sure of model quality, it is often slow and expensive.

Benchmark 117

Benchmark Awareness Evaluation Language 117

Care2

JUNE 21, 2013

Next Thursday, we''re holding a webinar that will dive into the key findings from the 2013 Blackbaud Online Marketing Benchmark Study – to indicate where your organization''s performance stacks up in relation to your peers, as well as what you can be doing better to speed your journey to online success. How Do You Measure Up?

Benchmark 56

Benchmark Measure Blackbaud America 56

The Verge

SEPTEMBER 16, 2021

Cardiologists are looking forward to the future of blood pressure tech — but the field still needs to catch up It’s been over two years since Samsung first announced that its Galaxy Watch would be able to measure people’s blood pressure. Blood pressure measurement is something we need to do a lot more a lot of,” she says.

Measure 98

Measure San Francisco South Korea Metrics 98

TechCrunch

JANUARY 4, 2022

The fact that startups have become more expensive, measured by comparing their valuations to their revenues, is reasonable. The Bessemer Cloud Index (trackable as the $WCLD ETF ), a basket of public software stocks, has lost ample ground thus far in 2022 (the following chart has a five-day range, so mind the dates): Image Credits: YCharts.

Public 92

Public Flash Marketing Software 92

Beth's Blog: How Nonprofits Can Use Social Media

OCTOBER 5, 2011

A few hours ago, it went public – at least on your FB page for the world to see, but it isn’t yet visible to page administrators in the insights program. The next action was an uncontrollable desire to compare my “People Talking About” with my peers, realizing that benchmarking of meaningless data is well, meaningless.

Metrics 114

Metrics Facebook People Measure 114

Nonprofit Tech for Good

JULY 5, 2020

Video engagement on social media is the measurable action that viewers take after watching. We studied 778 nonprofit organizations’ video posts on Facebook, Twitter and Instagram from January 1, 2019 – December 31, 2019 to establish benchmarks, identify trends, and uncover insights. What Is Video Engagement and Why Is It Important?

Video 321

Video Share Social Nonprofit 321

TechCrunch

FEBRUARY 14, 2022

ARR is probably the most critical metric for a startup — it’s what investors look at to predict future growth and even measure relationship changes within your subscription base. D’Onofrio brings substantial bona fides to the table to help new founders build a firm intellectual understanding of recurring revenue and its measurement.

Benchmark 88

Benchmark Metrics Measure Statistics 88

DataRobot

APRIL 21, 2022

Leaderboards for popular benchmarks are full of examples of Red AI where improvements are often the result of scaling processing power. Here, model size is measured by the amount of floating-point operations. How can we shift from Red AI that is inefficient and unavailable to the public to efficient and democratic Green AI?

Green 145

Green Environment Metrics Measure 145

Socialbrite

OCTOBER 31, 2021

So, how do you measure your video marketing efforts? Measuring your video marketing. To know how to measure the effectiveness of your video marketing , you must first decide what the goal of the marketing is so you know what you’re looking for. Measuring the engagement of your videos. Measuring brand awareness.

Measure 59

Measure Video Marketing Guide 59

Beth's Blog: How Nonprofits Can Use Social Media

MARCH 2, 2015

A few months ago, I facilitated a mini-innovation lab on measuring impact for grantees of the Google Nonprofit program at the Impact Hub. The group included a mix of Google grantees as well as measurement professionals. Create a roadmap for measuring your financial + social impact. Europe and Latin America. Report Out.

Measure 50

Measure Impact Program Nonprofit 50

The Verge

SEPTEMBER 17, 2020

That all tracks with the 1185G7’s synthetic-benchmark performance. That handily beats Ice Lake , and as you can see on 3DMark’s public leaderboard , it also beats the 4800U (which tops out at 1,450). In productivity benchmarks, the 1185G7 wins, but it doesn’t quite run away with the contest the way it did on the graphics front.

Test 136

Test Laptop Benchmark Integration 136

Candid

JANUARY 4, 2023

Candid saw an opportunity to relieve this data-request burden while also providing a standardized benchmark measurement for the entire sector: enter Demographics via Candid. These disorganized efforts take valuable staff time away from mission-critical work. Introducing Demographics via Candid.

Demographics 111

Demographics Practice Fund Data 111

Beth's Blog: How Nonprofits Can Use Social Media

SEPTEMBER 6, 2012

Mission Measurement (@MissionMeasure) September 5, 2012. Is your organization making use of public data for organizational strategy decisions? We may need to start wearing these here! RT @ kanter what every #datanerd needs in their wardrobe – I Love Data T-shirt bit.ly/QlhMt4. We may need to start wearing these here!

Attention 100

Attention Data Fundraising Nonprofit 100

Beth's Blog: How Nonprofits Can Use Social Media

JANUARY 6, 2009

Inspiration for How To Think Like A Social Media Marketing Genius Presentation by Beth Kanter For the past two years, I've been doing an annual Blog benchmarking process that attempts to do a ROI analysis. The most valuable part of the blog benchmarking process is the reflection process and linking insights to making improvements.

ROI 50

ROI Metrics Measure Content 50

TechCrunch

NOVEMBER 23, 2021

According to OpenView’s annual Financial & Operating Benchmarks report , only 13% of nearly 600 companies surveyed named “burning too much cash” as one of their top three concerns, compared to 30% last year. Nowhere is this more apparent, the authors claim, than when you look at public B2B SaaS companies.

Change 85

Change Benchmark Metrics Chart 85

TechCrunch

NOVEMBER 19, 2020

“We’ve built a platform that enables hyperlocal measurement. We measure all the greenhouse gases as well as regulated air pollutants. The interconnected crises of climate change, public health and environmental justice urgently require lasting solutions,” said Herzl, in a statement.

Fund 99

Fund San Diego Develop Tech 99

Fast Company Tech

MARCH 18, 2025

The o1 model rose quickly to the top of the rankings in common benchmark tests, and soon Google DeepMind , Anthropic , DeepSeek and others were training their models for real-time reasoning. The model generates PhD-level answers to complex questions, landing it on top of performance benchmark rankings.

Companies 107

Companies Model Training Train 107

Beth's Blog: How Nonprofits Can Use Social Media

APRIL 3, 2009

My autographed copy of Brian Solis 's (and Deirdre Breakenridge's) new book, Putting the Public Back in Public Relationships: How Social Media is Reinventing the Aging Business of PR just arrived in today's mail. As KD Paine said on our SXSW ROI Poetry Slam , "You have engagement fully measured, now calculate the resulting treasure!".

Public 50

Public Metrics Social Media ROI 50

Candid

MAY 16, 2024

Based on data from 59,550 public charities that shared demographic data between July 2019 and January 2024, 47% of all staff identify as white, 24% as Black/African American, 19% as Hispanic/Latino, 6% as Asian American/Pacific Islander, 1% as Native American, and 4% as multiracial.

Demographics 119

Demographics Nonprofit Leadership Data 119

DataRobot

FEBRUARY 10, 2021

Over the holidays, I used DataRobot to reproduce a few machine learning benchmarks. The examples I share in this post come from the world’s best data scientists and affect hundreds of peer-reviewed research publications. The SARCOS dataset is a widely used benchmark dataset in machine learning. SARCOS Dataset Failure.

Model 98

Model Benchmark Metrics Training 98

TechCrunch

NOVEMBER 4, 2021

That’s the case OpenView, a Boston-based VC firm, makes in its annual Financial and Operating Benchmarks survey. The report, though, isn’t just about measuring how many companies adopt a trend that OpenView has been cheering for. ”

Companies 100

Companies API Model Boston 100

Tech Soup

MAY 30, 2013

Nicole McGougan is a public relations associate at Blackbaud. The 2013 Online Marketing Benchmark Study. Don't miss our most popular webinar series of the year, where we'll reveal the results of the 2013 Blackbaud Online Marketing Benchmark Study. To get where you want to be, you need to know where you are today.

Benchmark 40

Benchmark Studies Online Marketing 40

TechCrunch

DECEMBER 3, 2020

Pave , a San Francisco-based startup that recently graduated from YC Combinator is aiming to end the pay and equity gap with a software tool it developed to make it easier to track, measure, and communicate how and what they pay their employees. Is the industry ready to be benchmarked? Pave’s Total Rewards Portal for employees.

Raise 86

Raise San Francisco Benchmark Wyoming 86

TechCrunch

APRIL 17, 2023

There have been several startup attempts to measure staff morale, some more halting that others. But now a Bristol-based, UK startup hopes to make waves in the ‘workplace measurement’ space, and it plans to use the rocket fuel of Generative AI. Officevibe appeared around the same kind’ve era. Loopin has now raised a £1.6

Platform 81

Platform Raise Measure Benchmark 81

Google Research AI blog

AUGUST 9, 2023

Consequently, academic benchmarks report strong model accuracy, but these same models do poorly when used for complex real-world applications. We list five requirements for a good document understanding benchmark, based on the kinds of real-world documents for which document understanding models are frequently used.

Template 76

Template Benchmark Evaluation Sample 76

TechCrunch

MARCH 16, 2023

The models are trained on usage metadata from customers, public cloud pricing and inventory information, and undisclosed “other signals” that cloud providers make available. Cast AI’s optimization dashboard, showing real-time data across public cloud instances.

Companies 98

Companies Raise Help Public 98

Beth's Blog: How Nonprofits Can Use Social Media

DECEMBER 6, 2013

With the digital communications landscape evolving at light speed we wanted to track challenges, leadership attitudes, measurement strategies and 2014 investment plans. The survey is currently in the field but initial responses show broad adoption and rising confidence and sophisticated use of online and social tools.

Social Media 108

Social Media Arts Social Media 108

Beth's Blog: How Nonprofits Can Use Social Media

JUNE 16, 2011

SMART Objectives are specific, measurable, attainable, relevant, and timely objectives. Benchmarking comparing your organization’s past performance to itself or doing a formal or informal analysis of peer organizations can help. It also helps to break down your goal into monthly or quarterly benchmarks.

Facebook 102

Facebook Network Nonprofit Social Media 102

TechCrunch

APRIL 25, 2021

The company says it has outperformed benchmarks since launching in 2017. At the end of March, its portfolios outperformed their same-risk benchmarks (proxied by MSCI World Equity Index and FTSE World Government Bond Index), with annualized returns ranging from 16.5% (for the highest-risk portfolio) to 4% (the lowest-risk portfolio).

India 100

India United Arab Emirates Singapore Thailand 100

Amy Sample Ward

JUNE 6, 2012

After all, building and maintaining a meaningful social media presence requires a significant investment of time — and it’s not always easy to measure the return on that investment. They are not exclusive – you can mix and match. I will define the terms below and discuss mixing and matching below."

Web 117

Web Social Media San Francisco Search 117

The Verge

MARCH 28, 2022

For the next few weeks, we’re showing you how to run various commonly used benchmarks on your PC. But you don’t have to rely on us — you can easily run these benchmarks yourself on your own PC. Geekbench CPU measures your processor’s performance, simulating both single-core and multi-core usage scenarios. Click Run benchmark.

Phone 69

Phone Ubuntu Benchmark License 69

Beth's Blog: How Nonprofits Can Use Social Media

JULY 7, 2011

I asked this on Google+ as a public thread so I could share it more broadly. Paying attention to measurement benchmarks will be interesting here. But it’s also like starting over – I mean, what to measure? Ah, an opportunity for reflection! What goals to set? What’s even going on here? Dan Michel.

Google 95

Google Org Skype Facebook 95

Beth's Blog: How Nonprofits Can Use Social Media

AUGUST 3, 2011

Tracking what content is utilized, and the services rendered provide a means of measuring our success in achieving that mission. Institutional objectives lead to Goals, which need Key Performance Indicators (KPIs) to measure. These become the metrics & KPIs behind our work. What are your KPIs?

Social Media 97

Social Media Media Social Flickr 97

Google Research AI blog

FEBRUARY 24, 2023

In “ Spotlight: Mobile UI Understanding using Vision-Language Models with a Focus ”, accepted for publication at ICLR 2023 , we present a vision-only approach that aims to achieve general UI understanding completely from raw pixels. In this experiment, we compare Spotlight with several benchmark models.

Language 122

Language Foundation Widget Attention 122

TechCrunch

NOVEMBER 16, 2020

That’s a dramatic shift in urban transportation by any measure, and it prompts a simple but important question: How did we get here? The report laid the groundwork for cooperative safety measures to be taken by both operators and cities to ensure that not only riders and pedestrians but all road users are protected.

New York 118

New York New York City Share Los Angeles 118