PHM-Exch> WHO’s Study on Substandard and Falsified Medical Products: Questionable Data and Analysis.

[Claudio Schuftan]

From: "K.M. Gopakumar" <kumargopakm at gmail.com>

*TWN Info Service on Health Issues (Nov17/09)*


* 28 November 2017 Third World Network*


http://www.twn.my/title2/health.info/2017/hi171109.htm


*WHO’s Study on Substandard and Falsified Medical Products: Questionable
Data and Analysis.*

Geneva, 28th November (Third World Network): There are significant concerns
that WHO’s study on public health and socio economic impact of substandard
and falsified medical products to be released during a press conference
today afternoon in Geneva, is based on questionable data, lacking in
rigorous analysis.

A virtual press conference (VPC) is being organised at CET 15to launch a
report on the Global Surveillance and Monitoring System (GSMS) and a study
on substandard and falsified medical products.


The first report compiles 1500 real cases from all WHO regions, giving
estimates on the products most affected and regional prevalence.  The GSMS
was established in 2013, with financial support from the European Union and
some of its Member States.  The cases however were reported to the GSMS
between 2013 and 2016 and during this time, the GSMS was using the
following categories to capture the data from various member states:
falsified medical products, suspected falsified medical products,
substandard medical products, suspected substandard medical products,
intentionally manufactured substandard medical products, diverted medical
products, stolen medical products and unlicensed medical products. However
the Secretariat has never provided breakdown of the data captured by the
GSMS, according to the terms used by GSMS.


In 2016, the World Health Assembly determined that compromised medical
products should be categorized as “substandard” and “falsified” and adopted
working definitions on the same. Given this development, it is rather
concerning that the GSMS cases are automatically categorized as being
“substandard” or “falsified”.


The second report is titled ““Public health and socioeconomic impact of
substandard and falsified products”. This report is the brainchild of, and
supported by the US.


The Member State Mechanism (MSM) has however decided that it would not
formally adopt the study.

Discussions with several member states familiar with the process leading to
the study revealed that many have concerns with the findings of the Study
especially the estimation of prevalence of substandard and falsified
medical products in the low and middle-income countries using an
unscientific methodology.


TWN has learned that the study examined 100 published papers and one
publicly available database between 1st  January 2007 to 31st December 2016
covering 48,218 samples. These samples are grouped into low income
countries (11156), middle income countries ( 36884)  and high income
countries ( 178). Based on the number of failed samples in low-income
countries (1166) and middle income countries (3906), the report finds the
aggregate failure rate of tested samples of substandard and falsified
medicines in low and middle-income countries is approximately 10.5%. The
study then asserts that as the market size for low and middle income
countries is nearly $300 billion, and given the 10.5% failure rate, the
amount countries spend on substandard and falsified and medicines is $30
billion.


The methodology used by the Study is rather flawed.  First, the Study is
based on published papers that use heterogeneous terms and definitions. The
terms and definitions also differ from the working definitions of
“substandard” and “falsified” adopted by the 2017 World Health Assembly.


Secondly there are significant divergences in terms of methodology and
sample size of the various papers relied on by the study, varying from 10
samples to 15,000 per study. The findings of the papers are thus not
comparable. Thirdly, the samples substantially belong to two types of
medicines i.e. antimalarial (18764) and antibiotics and anti-infectives
(12375).

These flaws calls into question the data and analysis presented in the
Study and its extrapolation of the failure rate to the total medicines
market.


The study gives the impression that 10.5% products valued at USD 30 billion
sold in the low and middle-income countries are substandard and falsified.
The compilation of the 48218 samples from 100 published papers with a
divergent number of samples and different parameters (definition) for the
categorisation of substandard and falsified  medical products cannot be
used to measure the  prevalence of substandard medicines in the entire
pharmaceutical market of low and mididle income countries.

The inherent limitations of data led to the experts commissioned to develop
the study to reject the idea of calculating the economic value of spending
on substandard and falsified medicines. The first draft report submitted to
the Steering Committee meeting in March 2016 did not contain any figures
with regard to the purchase of the substandard and falsified medicine. The
first draft report estimated the prevalence  based on  the sample size
combining “*data from 107 published field studies that met quality criteria
for inclusion, totalling almost 73,000 samples. The mean prevalence rate of
substandard medicines obtained from pooling the samples – the proportion of
sampled medicines that failed laboratory testing based on specified
reference standards - was 8.7%”.*


Further it stated, “*Most of the studies were conducted in Africa and Asia,
and many focused on anti-malarial and antibiotic medicines. Because of this
sampling bias, it is not possible to use the 8.7% figure as a global
estimate of the prevalence of substandard medicines or as an estimate for
all therapeutic categories*”.


Then it concluded:  “*Empirical data on the public health and socioeconomic
impact of substandard and falsified medicines is also very limited,
particularly from studies where it is possible to track how the authors of
published papers have developed their estimates. All that can be said at
the moment is that the consequences of substandard medicines for public
health, and for patient and health system costs, can be large in certain
cases but as yet the data do not allow a global estimate of the cost that
would be considered methodologically sound*”.


The finding of the first version was changed without any valid explanation
in the second draft circulated in November 2016.

The second draft estimated the economic impact of substandard and falsified
medicine to the value of USD 31.1 billion.  The second draft states: “*The
approach that is therefore adopted is ascribing the 10.5% ?6.2 prevalence
rate to the pharmaceutical market of low-income countries ($135.2 billion),
which produces a projected economic impact of $14.2 billion (with a 95%
confidence interval of ?8.4 billion) in low income countries as a result of
substandard and falsified medicines. For middle-income countries, the data
used is based on a subset of field studies that sample the quality of a
basket of medicines; 14 such field studies examine various therapeutic
categories, selected on the basis of both availability and essentiality.
These studies not only cover a wide range of therapeutic products, but 13
of the 14 studies were conducted in at least one middle-income country. The
prevalence figure from this subset of 14 field studies is 6.8% ?5.6. If we
apply this figure to the market size of middle-income countries ($249.2
billion), the estimated cost would be $16.9 billion (with a 95% confidence
interval of ?13.9 billion). Combining these estimates, we arrive at a
potential economic impact of $31.1 billion to LMICs*.”

The third draft provided in March 2017 for comments by the expert advisory
group and Member States continues the same approach and estimated the cost
to be 28.6 billion.


“*The approach that has therefore been adopted is ascribing the mean
prevalence rate (across 2196 samples) of 10.8% to the pharmaceutical market
of LICs ($112.4 billion), which produces a 2197 projected substandard and
falsified medicines economic impact of $12.1 billion in LICs. For 2198
MICs, the data used is based on a subset of field studies that sample the
quality of a basket of 2199 medicines; 24 such field studies examine
various therapeutic categories, selected on the basis 2200 of both
availability and essentiality. These studies not only cover a wide range of
therapeutic products, but 22 of the 24 studies (92%) included at least one
MICs. The prevalence figure from this subset of 22 field studies is 6.8%.
If this prevalence figure is applied to the market size of MIC ($242.9
billion), the estimated cost would be $16.5 billion. Combining these two
estimates yields a projected economic impact of $28.6 billion to LMICs due
to substandard and falsified medicines”.*


The final version of the study, according to sources, avoids all
explanations, but continues using the flawed methodology.

The study also does not provide a break down of numbers that are
substandard and falsified from the total of 4072 failed samples.

[The 2017 World Health Assembly adopted working definitions of substandard
and falsified medicines. The former category generally refers to “out of
specification” medicines produced by licensed manufactures, while the
latter category would generally include activities of unlicensed
manufacturers. ]


The study also overlooks the nuances involved in the health implications of
substandard medicines.  Generally, substandard medicines are grouped into
minor, major and critical deviations. Major and critical deviations from
the specifications are usually of a concern from a public health
perspective, requiring regulatory actions. Given the study fails to provide
such details, one cannot conclude that all the failed samples pose a threat
to health.  For instance a study published by WHO titled

“Survey of the Quality of Anti-Tuberculosis Medicines in Circulating in
Newly Independent States of the Former Soviet Union” (
http://apps.who.int/prequal/info_applicants/qclabs/monitoring_documents/
TBQuality-Survey_Nov2011.pdf) concludes:

 *“A survey of quality of selected anti-TB medicines in six NIS identified
11.3% samples failing to meet the specifications set for this survey.
Focusing only on extreme deviations as defined in this report, which are
likely associated with direct health implications, the failure rate reached
1.0%. No sample was suspected to be of spurious, falsely-labelled,
falsified or counterfeit product*”


The Secretariat is proposing to widely disseminate this study as part of
the prioritised activities of MSM for the 2018-19 biennium. The proposed
prioritised activities F states:  “Disseminate and use the study on the
public health and socioeconomic impact of substandard and falsified medical
products and the report on the WHO Global Surveillance and Monitoring
System for substandard and falsified medical products as a tool to increase
awareness and advocate at the highest policy levels nationally, regionally
and globally for support and resources to prevent, detect and respond to
substandard and falsified medical products”.


A major concern with this action is the wide dissemination of a study
rooted in unscientific flawed methodology and misleading information, which
does not reflect the on-the ground reality and results in unwarranted
policy response.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://phm.phmovement.org/pipermail/phm-exchange-phmovement.org/attachments/20171130/9ce3fdce/attachment-0002.html>