LAUSR

While web‐based indicators for scientific impact – so‐called altmetrics – have an increasing uptake as means for research evaluation, many questions regarding their actual meanings remain unanswered. In this article we analyse the data from a survey about researchers' use of 107 online actions that underlie potential altmetrics to discover whether certain types of altmetrics (1) better reflect the judgments of researchers from certain career stages and (2) more reliably capture positive judgments than others. We apply variance analyses to reveal significant differences between the frequencies with which early‐stage researchers and professors perform various actions whose occurrences are counted as impact metrics (e.g., downloading, liking, or sharing of scientific publications). The findings imply varying degrees of representation of these groups in respective altmetrics. Moreover, by investigating how commonly various types of actions are used to express positive judgments, we disclose how reliably metrics can be used as proxies for positive impact. Our findings contribute to a more comprehensive understanding of the meaning of different web‐based metrics for scholarly impact and provide a basis for evidence‐based guidelines on how to use and interpret them.