Tracking impact

Universities are increasingly attuned to the ‘impact’ agenda – seeking to measure and quantify the impact of our work on society.

The Australian Research Council defines ‘research impact’ as ‘the contribution that research makes to the economy, society, environment or culture, beyond the contribution to academic research.’

In many fields, ‘impact’ is measured through citation counts. This is problematic for multiple reasons: self-citations can ‘game’ citation counts (and are included in most citation measures); and a highly cited article could be cited a lot because it is so bad! So use citation data with caution.

Also, we need to contextualise any impact data in the context of our field; so comparison tools like Dimensions and Publish or Perish can be very useful, if we use them to consider performance in our field specifically.

There are a number of practical tools that can assist with measuring impact:

• Web of Science and Scopus: both measure citations, but tend to do badly in capturing law/legal data.
• Google Scholar: Google Scholar can provide your H-index for articles captured by Google Scholar. Setting up a Google Scholar profile can help to publicise your work, and track your citations. You can also set up an email alert for new citations.
• Publish or Perish: this is a useful tool for comparing, eg, Google Scholar data across those in your field. 
• Dimensions: Dimensions can offer a field citation ratio, comparing the citations of your work to other works in the field of the same age.
• Altmetrics: captures mentions of your work across social media, like Twitter/X and LinkedIn. 
• Google search: this I find most useful! Search your work; search your name. See where your work has been used. Try limiting the search to, eg, site:.gov.au for Australian government references; site:.gov.uk for UK government references; or to site:.ac.uk for references in UK universities and research repositories. This helped me capture, eg, citations in Parliament, university subject guides, and government reports and inquiries.
• Case searches: in law, citations by judges matter. Try searching case databases (e.g. Lexis, Austlii, Westlaw) for your name and/or the title of your work.

Because citation indexes do so poorly with legal scholarship, another option is to create your own H-index, by searching law databases. One way to do this is to use Zotero, and to set up folders for each of your outputs: add each citing source to the relevant output folder.

Academia.edu purports to offer lots of citations to your work if you pay for a premium subscription. I trialled this, but found the results were far inferior to a simple Google search.

I also trialled Overton, which purports to measure impact in policy documents – again, there was nothing there that hadn’t emerged through a Google search.

Another way of measuring and tracking impact is to ask people directly! Perhaps they can write a statement explaining why your work has been useful/impactful.

It is a great exercise, as you finish a large piece of work, to write up an impact case study, thinking about how your work has influenced the field and society. The impact case study for my DECRA grant is available at ageworks.info