site-engagement-metrics

Site Engagement Metrics

—

by

Dan Petrovic
in

To access the feature in Chrome visit: chrome://site-engagement/

Google Site Engagement Metrics Framework plays a crucial role in assessing and analyzing user engagement with websites. This framework leverages detailed metrics, such as user interactions and engagement scores, to provide insights into browsing behavior. Here’s a breakdown of how this system works, based on the Site Engagement Metrics implementation.

Core Concepts in Site Engagement Metrics

  1. Base Metrics Tracked
    Chromium tracks site engagement through various key metrics:
    • Total Origins Engaged: The number of distinct domains (or origins) that a user has interacted with meaningfully.
    • Mean and Median Engagement: The average and median engagement scores across all tracked origins.
    • Engagement Score: A per-origin score reflecting user interaction levels, such as clicks, time spent, and other behaviors.
    • Engagement Type: Specific actions categorized by type (e.g., notifications, shortcuts, or advanced interactions).
  2. Histograms for Data Collection
    Data is recorded using UMA (User Metrics Analysis) histograms, enabling Chromium to log and analyze these engagement metrics for internal or experimental purposes. Examples of these histograms include:
    • Origins Engaged Histogram: Tracks the number of domains with user interaction.
    • Mean and Median Engagement Histograms: Focus on understanding overall engagement distribution.
    • Engagement Type Histogram: Logs user activity by specific engagement types.

How Metrics Are Recorded

Chromium uses a combination of pre-defined histograms and specialized functions to record and process engagement data. Here are some key functions within the framework:

  • Recording Total Origins
    The RecordTotalOriginsEngaged function logs the number of unique origins a user has interacted with, using the kTotalOriginsHistogram.
  • Tracking Scores
    Functions like RecordMeanEngagement and RecordMedianEngagement log average and median engagement scores across all domains. These scores help measure overall user engagement with the web.
  • Engagement by Details
    The RecordEngagementScores function iterates over a list of site engagement details and logs individual scores to the kEngagementScoreHistogram.
  • Categorized Engagement
    The RecordEngagement function logs the type of engagement, using an enumeration to distinguish between different types (e.g., notification points or shortcut launches).

Site Engagement Parameters and Values

  • Max Points Per Day: 15
  • Decay Period (in hours): 2
  • Decay Points: 0
  • Decay Proportion: 0.984
  • Score Cleanup Threshold: 0.5
  • Navigation Points: 1.5
  • User Input Points: 0.6
  • Visible Media Playing Points: 0.06
  • Hidden Media Playing Points: 0.01
  • Web App Installed Points: 5
  • First Daily Engagement Points: 1.5
  • Bootstrap Points: 24
  • Medium Engagement Boundary: 15
  • High Engagement Boundary: 50
  • Max Decays Per Score: 4
  • Last Engagement Grace Period (in hours): 1
  • Notification Interaction Points: 1

components/site_engagement/content/site_engagement_score.cc

// Copyright 2016 The Chromium Authors
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include "components/site_engagement/content/site_engagement_score.h"

#include <algorithm>
#include <array>
#include <cmath>
#include <utility>

#include "base/metrics/field_trial_params.h"
#include "base/no_destructor.h"
#include "base/strings/string_number_conversions.h"
#include "base/time/clock.h"
#include "base/time/time.h"
#include "base/values.h"
#include "components/content_settings/core/browser/host_content_settings_map.h"
#include "components/content_settings/core/common/content_settings.h"
#include "components/content_settings/core/common/content_settings_types.h"
#include "components/content_settings/core/common/content_settings_utils.h"
#include "components/site_engagement/content/engagement_type.h"
#include "components/site_engagement/content/site_engagement_metrics.h"
#include "third_party/blink/public/mojom/site_engagement/site_engagement.mojom.h"

namespace site_engagement {

namespace {

// Delta within which to consider scores equal.
constexpr double kScoreDelta = 0.001;

// Delta within which to consider internal time values equal. Internal time
// values are in microseconds, so this delta comes out at one second.
constexpr double kTimeDelta = 1000000;

// Number of days after the last launch of an origin from an installed shortcut
// for which WEB_APP_INSTALLED_POINTS will be added to the engagement score.
constexpr int kMaxDaysSinceShortcutLaunch = 10;

bool DoublesConsideredDifferent(double value1, double value2, double delta) {
  double abs_difference = fabs(value1 - value2);
  return abs_difference > delta;
}

base::Value::Dict GetSiteEngagementScoreDictForSettings(
    const HostContentSettingsMap* settings,
    const GURL& origin_url) {
  if (!settings)
    return base::Value::Dict();

  base::Value value = settings->GetWebsiteSetting(
      origin_url, origin_url, ContentSettingsType::SITE_ENGAGEMENT, nullptr);
  if (!value.is_dict())
    return base::Value::Dict();

  return std::move(value).TakeDict();
}

}  // namespace

const double SiteEngagementScore::kMaxPoints = 100;

const char SiteEngagementScore::kRawScoreKey[] = "rawScore";
const char SiteEngagementScore::kPointsAddedTodayKey[] = "pointsAddedToday";
const char SiteEngagementScore::kLastEngagementTimeKey[] = "lastEngagementTime";
const char SiteEngagementScore::kLastShortcutLaunchTimeKey[] =
    "lastShortcutLaunchTime";

// static
SiteEngagementScore::ParamValues& SiteEngagementScore::GetParamValues() {
  static base::NoDestructor<ParamValues> param_values([]() {
    SiteEngagementScore::ParamValues param_values;
    param_values[MAX_POINTS_PER_DAY] = {"max_points_per_day", 15};
    param_values[DECAY_PERIOD_IN_HOURS] = {"decay_period_in_hours", 2};
    param_values[DECAY_POINTS] = {"decay_points", 0};
    param_values[DECAY_PROPORTION] = {"decay_proportion", 0.984};
    param_values[SCORE_CLEANUP_THRESHOLD] = {"score_cleanup_threshold", 0.5};
    param_values[NAVIGATION_POINTS] = {"navigation_points", 1.5};
    param_values[USER_INPUT_POINTS] = {"user_input_points", 0.6};
    param_values[VISIBLE_MEDIA_POINTS] = {"visible_media_playing_points", 0.06};
    param_values[HIDDEN_MEDIA_POINTS] = {"hidden_media_playing_points", 0.01};
    param_values[WEB_APP_INSTALLED_POINTS] = {"web_app_installed_points", 5};
    param_values[FIRST_DAILY_ENGAGEMENT] = {"first_daily_engagement_points",
                                            1.5};
    param_values[BOOTSTRAP_POINTS] = {"bootstrap_points", 24};
    param_values[MEDIUM_ENGAGEMENT_BOUNDARY] = {"medium_engagement_boundary",
                                                15};
    param_values[HIGH_ENGAGEMENT_BOUNDARY] = {"high_engagement_boundary", 50};
    param_values[MAX_DECAYS_PER_SCORE] = {"max_decays_per_score", 4};
    param_values[LAST_ENGAGEMENT_GRACE_PERIOD_IN_HOURS] = {
        "last_engagement_grace_period_in_hours", 1};
    param_values[NOTIFICATION_INTERACTION_POINTS] = {
        "notification_interaction_points", 1};
    return param_values;
  }());
  return *param_values;
}

double SiteEngagementScore::GetMaxPointsPerDay() {
  return GetParamValues()[MAX_POINTS_PER_DAY].second;
}

double SiteEngagementScore::GetDecayPeriodInHours() {
  return GetParamValues()[DECAY_PERIOD_IN_HOURS].second;
}

double SiteEngagementScore::GetDecayPoints() {
  return GetParamValues()[DECAY_POINTS].second;
}

double SiteEngagementScore::GetDecayProportion() {
  return GetParamValues()[DECAY_PROPORTION].second;
}

double SiteEngagementScore::GetScoreCleanupThreshold() {
  return GetParamValues()[SCORE_CLEANUP_THRESHOLD].second;
}

double SiteEngagementScore::GetNavigationPoints() {
  return GetParamValues()[NAVIGATION_POINTS].second;
}

double SiteEngagementScore::GetUserInputPoints() {
  return GetParamValues()[USER_INPUT_POINTS].second;
}

double SiteEngagementScore::GetVisibleMediaPoints() {
  return GetParamValues()[VISIBLE_MEDIA_POINTS].second;
}

double SiteEngagementScore::GetHiddenMediaPoints() {
  return GetParamValues()[HIDDEN_MEDIA_POINTS].second;
}

double SiteEngagementScore::GetWebAppInstalledPoints() {
  return GetParamValues()[WEB_APP_INSTALLED_POINTS].second;
}

double SiteEngagementScore::GetFirstDailyEngagementPoints() {
  return GetParamValues()[FIRST_DAILY_ENGAGEMENT].second;
}

double SiteEngagementScore::GetBootstrapPoints() {
  return GetParamValues()[BOOTSTRAP_POINTS].second;
}

double SiteEngagementScore::GetMediumEngagementBoundary() {
  return GetParamValues()[MEDIUM_ENGAGEMENT_BOUNDARY].second;
}

double SiteEngagementScore::GetHighEngagementBoundary() {
  return GetParamValues()[HIGH_ENGAGEMENT_BOUNDARY].second;
}

double SiteEngagementScore::GetMaxDecaysPerScore() {
  return GetParamValues()[MAX_DECAYS_PER_SCORE].second;
}

double SiteEngagementScore::GetLastEngagementGracePeriodInHours() {
  return GetParamValues()[LAST_ENGAGEMENT_GRACE_PERIOD_IN_HOURS].second;
}

double SiteEngagementScore::GetNotificationInteractionPoints() {
  return GetParamValues()[NOTIFICATION_INTERACTION_POINTS].second;
}

void SiteEngagementScore::SetParamValuesForTesting() {
  GetParamValues()[MAX_POINTS_PER_DAY].second = 5;
  GetParamValues()[DECAY_PERIOD_IN_HOURS].second = 7 * 24;
  GetParamValues()[DECAY_POINTS].second = 5;
  GetParamValues()[NAVIGATION_POINTS].second = 0.5;
  GetParamValues()[USER_INPUT_POINTS].second = 0.05;
  GetParamValues()[VISIBLE_MEDIA_POINTS].second = 0.02;
  GetParamValues()[HIDDEN_MEDIA_POINTS].second = 0.01;
  GetParamValues()[WEB_APP_INSTALLED_POINTS].second = 5;
  GetParamValues()[BOOTSTRAP_POINTS].second = 8;
  GetParamValues()[MEDIUM_ENGAGEMENT_BOUNDARY].second = 5;
  GetParamValues()[HIGH_ENGAGEMENT_BOUNDARY].second = 50;
  GetParamValues()[MAX_DECAYS_PER_SCORE].second = 1;
  GetParamValues()[LAST_ENGAGEMENT_GRACE_PERIOD_IN_HOURS].second = 72;
  GetParamValues()[NOTIFICATION_INTERACTION_POINTS].second = 1;

  // This is set to values that avoid interference with tests and are set when
  // testing these features.
  GetParamValues()[FIRST_DAILY_ENGAGEMENT].second = 0;
  GetParamValues()[DECAY_PROPORTION].second = 1;
  GetParamValues()[SCORE_CLEANUP_THRESHOLD].second = 0;
}
// static
void SiteEngagementScore::UpdateFromVariations(const char* param_name) {
  std::array<double, MAX_VARIATION> param_vals;

  for (int i = 0; i < MAX_VARIATION; ++i) {
    std::string param_string =
        base::GetFieldTrialParamValue(param_name, GetParamValues()[i].first);

    // Bail out if we didn't get a param string for the key, or if we couldn't
    // convert the param string to a double, or if we get a negative value.
    if (param_string.empty() ||
        !base::StringToDouble(param_string, &param_vals[i]) ||
        param_vals[i] < 0) {
      return;
    }
  }

  // Once we're sure everything is valid, assign the variation to the param
  // values array.
  for (int i = 0; i < MAX_VARIATION; ++i)
    SiteEngagementScore::GetParamValues()[i].second = param_vals[i];
}

SiteEngagementScore::SiteEngagementScore(base::Clock* clock,
                                         const GURL& origin,
                                         HostContentSettingsMap* settings)
    : SiteEngagementScore(
          clock,
          origin,
          GetSiteEngagementScoreDictForSettings(settings, origin)) {
  settings_map_ = settings;
}

SiteEngagementScore::SiteEngagementScore(SiteEngagementScore&& other) = default;

SiteEngagementScore::~SiteEngagementScore() = default;

SiteEngagementScore& SiteEngagementScore::operator=(
    SiteEngagementScore&& other) = default;

void SiteEngagementScore::AddPoints(double points) {
  DCHECK_NE(0, points);

  // As the score is about to be updated, commit any decay that has happened
  // since the last update.
  raw_score_ = DecayedScore();

  base::Time now = clock_->Now();
  if (!last_engagement_time_.is_null() &&
      now.LocalMidnight() != last_engagement_time_.LocalMidnight()) {
    points_added_today_ = 0;
  }

  if (points_added_today_ == 0) {
    // Award bonus engagement for the first engagement of the day for a site.
    points += GetFirstDailyEngagementPoints();
    SiteEngagementMetrics::RecordEngagement(
        EngagementType::kFirstDailyEngagement);
  }

  double to_add = std::min(kMaxPoints - raw_score_,
                           GetMaxPointsPerDay() - points_added_today_);
  to_add = std::min(to_add, points);

  points_added_today_ += to_add;
  raw_score_ += to_add;

  last_engagement_time_ = now;
}

double SiteEngagementScore::GetTotalScore() const {
  return std::min(DecayedScore() + BonusIfShortcutLaunched(), kMaxPoints);
}

mojom::SiteEngagementDetails SiteEngagementScore::GetDetails() const {
  mojom::SiteEngagementDetails engagement;
  engagement.origin = origin_;
  engagement.base_score = DecayedScore();
  engagement.installed_bonus = BonusIfShortcutLaunched();
  engagement.total_score = GetTotalScore();
  return engagement;
}

void SiteEngagementScore::Commit() {
  DCHECK(settings_map_);
  DCHECK(score_dict_);
  if (!UpdateScoreDict(*score_dict_))
    return;

  settings_map_->SetWebsiteSettingDefaultScope(
      origin_, GURL(), ContentSettingsType::SITE_ENGAGEMENT,
      base::Value(std::move(*score_dict_)));
}

blink::mojom::EngagementLevel SiteEngagementScore::GetEngagementLevel() const {
  DCHECK_LT(GetMediumEngagementBoundary(), GetHighEngagementBoundary());

  double score = GetTotalScore();
  if (score == 0)
    return blink::mojom::EngagementLevel::NONE;

  if (score < 1)
    return blink::mojom::EngagementLevel::MINIMAL;

  if (score < GetMediumEngagementBoundary())
    return blink::mojom::EngagementLevel::LOW;

  if (score < GetHighEngagementBoundary())
    return blink::mojom::EngagementLevel::MEDIUM;

  if (score < SiteEngagementScore::kMaxPoints)
    return blink::mojom::EngagementLevel::HIGH;

  return blink::mojom::EngagementLevel::MAX;
}

bool SiteEngagementScore::MaxPointsPerDayAdded() const {
  if (!last_engagement_time_.is_null() &&
      clock_->Now().LocalMidnight() != last_engagement_time_.LocalMidnight()) {
    return false;
  }

  return points_added_today_ == GetMaxPointsPerDay();
}

void SiteEngagementScore::Reset(double points,
                                const base::Time last_engagement_time) {
  raw_score_ = points;
  points_added_today_ = 0;

  // This must be set in order to prevent the score from decaying when read.
  last_engagement_time_ = last_engagement_time;
}

void SiteEngagementScore::SetLastEngagementTime(const base::Time& time) {
  if (!last_engagement_time_.is_null() &&
      time.LocalMidnight() != last_engagement_time_.LocalMidnight()) {
    points_added_today_ = 0;
  }
  last_engagement_time_ = time;
}

bool SiteEngagementScore::UpdateScoreDict(base::Value::Dict& score_dict) {
  double raw_score_orig = score_dict.FindDouble(kRawScoreKey).value_or(0);
  double points_added_today_orig =
      score_dict.FindDouble(kPointsAddedTodayKey).value_or(0);
  double last_engagement_time_internal_orig =
      score_dict.FindDouble(kLastEngagementTimeKey).value_or(0);
  double last_shortcut_launch_time_internal_orig =
      score_dict.FindDouble(kLastShortcutLaunchTimeKey).value_or(0);

  bool changed =
      DoublesConsideredDifferent(raw_score_orig, raw_score_, kScoreDelta) ||
      DoublesConsideredDifferent(points_added_today_orig, points_added_today_,
                                 kScoreDelta) ||
      DoublesConsideredDifferent(last_engagement_time_internal_orig,
                                 last_engagement_time_.ToInternalValue(),
                                 kTimeDelta) ||
      DoublesConsideredDifferent(last_shortcut_launch_time_internal_orig,
                                 last_shortcut_launch_time_.ToInternalValue(),
                                 kTimeDelta);

  if (!changed)
    return false;

  score_dict.Set(kRawScoreKey, raw_score_);
  score_dict.Set(kPointsAddedTodayKey, points_added_today_);
  score_dict.Set(kLastEngagementTimeKey,
                 static_cast<double>(last_engagement_time_.ToInternalValue()));
  score_dict.Set(
      kLastShortcutLaunchTimeKey,
      static_cast<double>(last_shortcut_launch_time_.ToInternalValue()));

  return true;
}

SiteEngagementScore::SiteEngagementScore(
    base::Clock* clock,
    const GURL& origin,
    std::optional<base::Value::Dict> score_dict)
    : clock_(clock),
      raw_score_(0),
      points_added_today_(0),
      last_engagement_time_(),
      last_shortcut_launch_time_(),
      score_dict_(std::move(score_dict)),
      origin_(origin),
      settings_map_(nullptr) {
  if (!score_dict_)
    return;

  raw_score_ = score_dict_->FindDouble(kRawScoreKey).value_or(0);
  points_added_today_ =
      score_dict_->FindDouble(kPointsAddedTodayKey).value_or(0);

  std::optional<double> maybe_last_engagement_time =
      score_dict_->FindDouble(kLastEngagementTimeKey);
  if (maybe_last_engagement_time.has_value())
    last_engagement_time_ =
        base::Time::FromInternalValue(maybe_last_engagement_time.value());

  std::optional<double> maybe_last_shortcut_launch_time =
      score_dict_->FindDouble(kLastShortcutLaunchTimeKey);
  if (maybe_last_shortcut_launch_time.has_value())
    last_shortcut_launch_time_ =
        base::Time::FromInternalValue(maybe_last_shortcut_launch_time.value());
}

double SiteEngagementScore::DecayedScore() const {
  // Note that users can change their clock, so from this system's perspective
  // time can go backwards. If that does happen and the system detects that the
  // current day is earlier than the last engagement, no decay (or growth) is
  // applied.
  int hours_since_engagement =
      (clock_->Now() - last_engagement_time_).InHours();
  if (hours_since_engagement < 0)
    return raw_score_;

  int periods = hours_since_engagement / GetDecayPeriodInHours();
  return std::max(0.0, raw_score_ * pow(GetDecayProportion(), periods) -
                           periods * GetDecayPoints());
}

double SiteEngagementScore::BonusIfShortcutLaunched() const {
  int days_since_shortcut_launch =
      (clock_->Now() - last_shortcut_launch_time_).InDays();
  if (days_since_shortcut_launch <= kMaxDaysSinceShortcutLaunch)
    return GetWebAppInstalledPoints();
  return 0;
}

}  // namespace site_engagement

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *