Temporal Portability of Numeric User Metadata on Twitter

Abstract

Numeric user metadata in social media are often reused over time. However, their reusability may depend on what an analysis needs to preserve. We introduce temporal portability as an analytical perspective for assessing the cross-time reuse of user features and feature-based rules. Specifically, we ask how well relevant properties are preserved when features and rules defined at a source time point are reused at a target time point. We used quarterly data on user features obtained directly from or derived from Japanese-language tweets in Twitter's 1% sample stream from 2020-Q1 to 2022-Q3. Each quarter included approximately 10.1--11.0 million unique users. We evaluated 13 numeric user features in terms of feature distributions, same-user relative ranks, selection rates, and selected-user membership. Across quarters, feature distributions changed and, for many features, same-user relative ranks were less well preserved at longer quarter lags. Reusing source-quarter thresholds also produced selection-rate drift. Target-quarter recalibration nearly matched source-quarter selection rates. However, membership turnover persisted and increased at longer quarter lags. Our results show that temporal portability should be assessed in terms of the property that an analysis needs to preserve.