Image: Tools. Credit: Pixabay. CC0
A major goal of fundamental blue skies discovery research is to understand the particular protein molecules that are involved in the behaviour of cells and organisms.
And a critical step in translational drug discovery is to pinpoint which proteins are faulty or misbehaving and causal in a given disease, such as cancer or neurological conditions – and could therefore be targeted with drugs to treat that disease.
High-quality chemical probes are key reagents in developing an understanding of protein behaviour in health and disease. These are highly selective small-molecule reagents, usually inhibitors, that alter the function of a target protein in a specific way – generally blocking its activity.
Using suitable high-quality chemical probes enables us to study the activity of particular individual proteins, and to determine their functional role in healthy or diseased cells – giving us a better understanding of the biology and disease pathology. But as regular followers of my blog will be aware, being very careful in choosing and using chemical probes correctly is crucial in both basic research and drug discovery – and the Chemical Probes Portal, hosted by the ICR, aims to help researchers do just that.
Chemical probes are complementary to genetic technologies such as CRISPR and RNA interference, and they have the additional advantage of a greater degree of control of concentration- and time-response relationships as well as more directly mimicking the effects of future potential drug candidates.
However, with both biological and chemical tools alike, great care must be taken to maximise the desired on-target effect and minimise or mitigate any off-target effects on other proteins.
As discussed in a recent ‘opinion’ article that I co-authored with postdoctoral fellow Marco Licciardello, we have seen a real step-change in the quality and quantity of small-molecule tools over the last ten years, as well as in the range of proteins that they are able to target.
Of note is that high-quality chemical probes have been produced for proteins that were previously considered ‘undruggable’ because they lacked a traditional binding pocket – leading directly not only to the use of such probes in the lab, but also to new drugs currently in clinical trials and even approved – as in the case of the highly oncogenic mutant KRAS protein.
Using probes appropriately
Despite the increased development and availability of high-quality chemical probes, these new tools are not always used in the best way in biological experiments, meaning that the findings obtained with them are not as robust as we would like – and may be questionable or wrong. Anecdotally, the chemical biology and drug discovery community has been well aware for many years that many published papers do not satisfy consensus best-practice guidelines for the use of chemical probes. However, to date there has been no systematic analysis of the scale of this problem.
New results, recently published in Nature Communications by Lenka Munoz and colleagues at the University of Sydney, Australia, now do provide, for the first time, a systematic and quantitative analysis that provides us with solid evidence of the markedly suboptimal use of even top-quality chemical probes in biomedical research.
The authors of the new study reviewed the use of eight high-quality chemical probes – published in the peer-reviewed literature and receiving high, i.e. 3 or 4 star, ratings on the Chemical Probes Portal – to assess whether their use was consistent with consensus guidelines that have been developed and endorsed by the expert chemical biology and drug discovery community.
The key elements of best practice they looked at were: 1) to use chemical probes at the recommended concentration or lower; 2) to apply alongside them a paired target-inactive (or much less active) control compound having a chemical structure closely matched to the active probe; and 3) in addition, to use a different probe with a distinct chemical structure (different chemotype or backbone) which targets the same protein, known as an orthogonal active probe.
The purpose of the consensus guidelines, which have been in place for up to ten to fifteen years, is to increase the likelihood that the effects of the chemical probes are occurring through the claimed target of interest and to reduce the likelihood of effects due to any confounding off-targets. Even high-quality chemical probes inhibit a certain number of off-target proteins. This has been referred to as them having an ‘active nightlife’.
A comprehensive predictive study of the number of drug–protein interactions across the proteome has illustrated the scale of the off-target challenge. The risk of off-target effects is mitigated by the inclusion of the paired inactive control and the orthogonal active probe. A proviso is that both of these control compounds must be profiled as extensively as the main chemical probe itself to ensure that they behave as they should.
In addition, use of chemical probes at the correct concentration is essential because the risk of off-target effects goes up at higher concentrations.
Using the eight selected high-quality chemical probes, which are recommended for particular target proteins that are involved in epigenetic regulation or that are protein kinases, the authors checked systematically for compliance with the above guideline criteria in 662 peer-reviewed primary research publications citing them.
Generally bad news, but some good news
Although past experience suggested to me, and I’m sure other chemical probe experts, that compliance with consensus guidelines would not be high, it was nevertheless surprising, and perhaps disheartening, that the study revealed that only as few as 4% of the research papers complied with all three criteria: using the correct concentration range and including the two types of control compounds.
When used incorrectly, chemical probes will not produce the robust results that are needed to have confidence in the finding, especially due to the risk of off-target effects.
However, there is some good news. First of all, it is pleasing to see that high-quality chemical probes are being selected for use, at least in these cases, as distinct from the lower-quality, poorly selective tools that have commonly been used in the past (and unfortunately are often still used today). Furthermore, the authors noticed only a small number of studies which used compounds previously referred on the Chemical Probes Portal as ‘Historical Compounds’, now renamed as ‘Unsuitables’. This is a collection of compounds that are either deeply flawed and behave as ‘frequent hitters’ across numerous targets, or alternatively have been superseded by better quality probes. Although this use of higher quality probes may be the case in the recent study, there is in fact continued use of such problematic nuisance compounds and out-of-date probes in the wider literature.
It was also somewhat reassuring that only 22% of the 662 publications analysed had used chemical probes at concentrations higher than the recommended level, although the authors themselves admitted that they were somewhat generous in how this determined. The majority of the 662 publications assessed (78%) used appropriate concentrations of compounds in their studies. Nevertheless, the 22% use of too high a concentration remains a concern.
In addition, somewhat over half (58%) of the 662 publications employed an orthogonal active chemical probe as a control – which is good, although the choice of orthogonal probe was not always ideal. Of course, this also means that 42% of the papers did not use a second chemotype probe at all. A biologist would not rely on only a single RNA interference reagent to attribute a function to a protein.
Even more worryingly, there was no matched inactive control in 92% of the assessed publications despite these controls being available. The overall situation more generally is likely to be even worse, since for most chemical probes there is no inactive control available; moreover, of those that have been published, few are sold by vendors, and apparently where they are sold the uptake is low. Note that inactive controls are provided free of charge for chemical probes developed by the Structural Genomics Consortium-led Donated Chemical Probes initiative in association with several pharmaceutical companies and Boehringer Ingelheim’s opnMe project – to both of which considerable credit is due.
An additional comment to make is that the recent study did not look at whether information was available on target engagement and target modulation, which are also considered essential in a high-quality chemical probe. Whenever a chemical probe is used, it is important to show that the probe is indeed binding to the intended target in the cell or organism used, and that the function of the protein is modulated as desired. This was outside the scope of the study and is more demanding to analyse, but is something that needs to be looked at in future. Without data showing that the chemical probe binds and modulates the target is cells, it is not possible to be confident about linking any observed effects on the phenotype to the perturbation of the protein target. Although not studied in detail in the paper, Lenka Munoz has told me that compliance with this was very low as well.
The Munoz group study also explored the downstream impact of the lack of best practice by assessing the number of citations that the 662 publications that they assessed have subsequently received to date. They observed that more than 2,500 papers have cited the publications using excessive probe concentrations; almost 7,750 papers have referenced the publications lacking inactive controls; while over 7,000 papers have cited the publications that had not included a second orthogonal active control probe. Clearly the impact of suboptimal chemical probe use could be far-reaching.
I should emphasise that incomplete compliance with the full set of best-practice guidance does not necessarily mean that the conclusions drawn from a study are incorrect. And I have not always been able to comply in my own work, for example because the desired controls were not available. However, it does mean that there is greater risk of off-target effects. Munoz and colleagues prefer to refer to studies that fall short of the full compliance package as ‘incomplete’, i.e. awaiting further validation.
Using the Chemical Probes Portal
How do we address this inadequate use of chemical tool compounds in biomedical research? One solution is that biologists wanting to use chemical probes can consult the non-profit Chemical Probes Portal – of which I am the Director and which is hosted by the ICR with the main funding coming from Wellcome.
The Portal is a free, online resource that provides an expert assessment and advice on hundreds of chemical probes and rates their quality for use in biological research.
As mentioned above, the eight chemical probes selected for the recent study were all selected as they are rated as 3 or 4 star on the Portal, indicating that they are highly recommended by the experts who review for the Portal.
The Portal Information Centre also contains a range of advice on chemical probes, including the consensus criteria for high-quality chemical probes of a range of types; recommended concentration ranges for use of chemical probes in cellular experiments; and recommendations for inactive and orthogonal active probes. There is also information on target engagement and modulation and also the more demanding criteria for use of chemical probes in rodent models where appropriate – these include consideration of dose and pharmacokinetic/pharmacodynamic relationships.
It is important to stress that the recent study was concerned with use of probes for cellular studies and the demands on using chemical probes in, for example, mouse models are even tougher.
House of cards – and solutions to changing behaviour
The results of the recent study, summarised above, were presented by Lenka Munoz in a recent webinar event hosted by Target 2035 in association with the Chemical Probes Portal, which I chaired and is available to watch on demand. Lenka Munoz likened the current situation to a ‘house of cards’ in which chemical probes are a crucial part of the construction of an argument linking the function of a given protein to a phenotype, such as a biological pathway, process or disease state. Using chemical probes incorrectly removes one of the underpinning cards in the construction and this can then cause the collapse of the biological logic. The consequences of this can include the wrong annotation of the function of protein in cells and diseases, and also incorrect validation of a potential drug target, contributing to the high failure rate of drugs in the clinic.
Ultimately, it is crucial that we spread the word about choosing and using chemical probes beyond the chemical biology and drug discovery community, which is very familiar with the challenges and solutions, and ensure that we share information much more widely about their appropriate use.
The publication in Nature Communications of a recent conversation between senior editor Katarzyna Marcinkiewicz, Cheryl Arrowsmith and myself was designed to increase awareness in the biomedical research community. Hopefully, the publication of the article by the Munoz team in the same general science journal will also help to raise awareness of the issues and solutions.
Lenka Munoz and colleagues highlighted in their valuable study that for each of the eleven papers which complied fully with the assessed probe criteria, there was a chemical biologist on the author list. Certainly, collaborating with experts in chemical probes, or at least talking to them, is highly recommended before selecting and using these reagents.
It can be very helpful for biologists seeking to use chemical probes to use resources like the Chemical Probes Portal and others. Also included in the Nature Communications paper from the Munoz group is a flow chart/decision tree, which is usefully a simpler version of one that my colleagues and I published previously in our review on published resources for chemical probes.
In addition, I encourage reviewers of grants and submitted publications, and also journal editors and reviewers, to use resources like the Chemical Probes Portal so as to catch the problem at the source and encourage researchers to gather more complete data – using probes at the right concentration and with adequate use of controls. Munoz and colleagues also provide a valuable checklist for journal editors to use in assessing papers that include the application of chemical probes.
The future can be bright
It is great news that high-quality chemical probes are now being produced by more groups around the world, and with the Target 2035 initiative the aspiration is to achieve total proteome coverage with tool compounds over the next decade.
The stage is set for a bright future in the use of chemical probes in basic research and drug discovery. We now need all the players to learn the rules to make sure that we gather robust data at the outset that will in turn lead to the best outcomes in the long term.
comments powered by