The Delphi technique has been used for meeting several challenges, including [1, 2, 3]:

  • Clarifying and exploring objectives, indicators, criteria, measures and options while avoiding social pressures and psychological distractions;
  • Exploring perceptions associated with the impact of potential options or measures with the aim of obtaining the pros and cons arguments associated with their implementation;
  • Collecting information and experts perceptions to select drivers for building scenarios;
  • Gathering evidence and preferences for weighting criteria and building value functions;
  • Forecasting by exploring what is likely to happen, what may change, and what effects can be anticipated;
  • Planning for resource allocation by exploring stakeholder’s priorities, motivations and values.

Delphi has been applied in a large number of domains [3] including business, economics, finance, health care, sales and strategic planning.

Welphi Portfolio

Real-life cases where Welphi has been used for


Euro-Healthy

Euro-Healthy Research project Building a composite index to measure population health on the European regions. Welphi was used in three distinct Delphi processes: (1) for selecting indicators, (2) for defining the shapes of the indicators’ value functions and (3) for weighting the indicators.

Anicare, Erasmus +

Anicare ERASMUS+ European project aiming to implement an innovative pedagogical approach to support farm animal welfare teaching through peer to peer debates. It targets to promote critical thinking and inspire a behavioral change while respecting farmers’ emancipation, within a wider perception of animal welfare.

Fórum Brasileiro de Mudança do Clima

Brazilian Forum for Climate Change Defining and prioritizing measures to be implemented by the Brazilian government with the objective of reducing carbon emissions, consulting actors of the society and of the Brazilian Government.

Enerphi

ENERPHI Promoting the Welphi platform to support students in teaching participatory web-Delphi processes for participatory processes in Energy decision-making contexts.

Impact HTA

Impact HTA research project looking at new and improved methods across ten thematic areas aiming at understanding variations in costs and health outcomes within and across countries, and integrating clinical and economic data from different sources to improve methods in economic evaluation in the context of HTA and health system performance measurement.

Rede Social de Lisboa

Lisbon Forum of the Social Action Network Selecting the objectives of the Social Development Plan (2016-2019) of the municipality of Lisbon, Portugal.

Welphi has been used as a tool for conducting Delphi processes in severall areas. You can see a list of some of the publications made by our clients below:

Reference papers of applications using Welphi:

  • Alvarenga, A., Bana e Costa, C., Borrell, C., Ferreira, P., Freitas, Â., Freitas, L., Oliveira, M., Rodrigues, T., Santana, P., Lopes Santos, M. and Vieira, A., Scenarios for population health inequalities in 2030 in Europe: the EURO-HEALTHY project experience. 2019, International Journal for Equity in Health. 18(100).
  • Freitas, Â., Santana, P., Oliveira, M., Almendra, R., Bana e Costa, J. and Bana e Costa, C., Indicators for evaluating European population health: a Delphi selection process. 2018, BMC Public Health 18(557).
  • Vieira, A., Oliveira, M. and Bana e Costa, C., Enhancing knowledge construction processes within multicriteria decision analysis: The Collaborative Value Modelling framework. 2019, Omega 94
  • Hermelijn S, Kersten C, Mullassery D On behalf of the CONNECT study consortium COS development group, On behalf of the CONNECT study consortium COS development group, et al Development of a core outcome set for congenital pulmonary airway malformations: study protocol of an international Delphi survey BMJ Open 2021;11:e044544. doi: 10.1136/bmjopen-2020-044544
  • Veeranki, Sreenivas P., Priti Pednekar, Marlon Graf, Rifat Tuly, Michael Recht, Katharine Batt. A Delphi consensus approach for difficult-to-treat patients with severe hemophilia A without inhibitors. Journal of blood medicine. 2021; 12: 1–16
  • Graf M, Tuly R, Harley C, Pednekar P, Batt K. Understanding the evolution of coverage policies for prophylaxis treatments of hemophilia A without inhibitors: a payer Delphi panel. J Manag Care Spec Pharm. 2021 2:1-13. doi: 10.18553/jmcp.2021.20600. Epub ahead of print. PMID: 33843253.
  • Chaves Loureiro C, Branco Ferreira M, Ferreira J, Lima R, Marques J, Sokolova A, Tonin FS, Duarte Ramos F. Reducing oral corticosteroids in severe asthma (ROSA Project): a nationwide Portuguese consensus. Pulmonology. 2020 Nov 8:S2531-0437(20)30217-8. doi: 10.1016/j.pulmoe.2020.10.002. Epub ahead of print. PMID: 33177008.
  • Cousins, G., Durand, L., Boland, F. et al. Development of quality indicators for the continued and safe delivery of Opioid Agonist Treatment (OAT), throughout and beyond COVID-19, using a Delphi Consensus technique. 2021, HRB Open Res 4:90
  • Fontanari, M. and Traskevich, A. Consensus and diversity regarding overtourism: the Delphi-study and derived assumptions for the post-COVID-19 time. 2021, International Journal of Tourism Policy 11(2): 161-187.
  • Spohn, K., Hiller, M., Hering, R., Kaltwasser, A., Sellmann, T., & Schroeder, S. (2024). Delphi Survey: Demand-orientated equipment on intensive and intermediate care units in Germany. ANASTHESIOLOGIE & INTENSIVMEDIZIN, 65, 15–26. https://doi.org/10.19224/ai2024.015
  • Juhrmann, M. L., Clayton, J. M., Butow, P. N., Simpson, P., Boughey, M., & Makeham, M. (2024). Development of a palliative paramedicine framework to standardise best practice: A Delphi study. Palliative Medicine. https://doi.org/10.1177/02692163241234004
  • Palones, E., Plaza, V., Crespo-Lessmann, A., García-Rivero, J. L., Cisneros, C., Almonacid-Sánchez, C., & Pérez-de Llano, L. (2024). Methodology Employed to Develop the Airway Mucus Secretion Test (T-SEC): A Questionnaire to Evaluate Airway Mucus Hypersecretion in Patients With Asthma. Archivos de Bronconeumologia, 60(3), 186-190–190. https://doi.org/10.1016/j.arbres.2024.01.002
  • Miranda, R., Silverio, R., Baptista, F. M., & Oliveira, M. D. (2024). Unlocking Continuous Improvement in Heart Failure Remote Monitoring: A Participatory Approach to Unveil Value Dimensions and Performance Indicators. TELEMEDICINE AND E-HEALTH. https://doi.org/10.1089/tmj.2023.0560
  • van Alphen, A.M.I.A., Krijkamp, E.M., Gravesteijn, B.Y. et al. (2024). Surgical prioritization based on decision model outcomes is not sensitive to differences between the health-related quality of life values estimates of physicians and citizens. Quality of Life Research 33, 529–539. https://doi.org/10.1007/s11136-023-03544-5
  • Moussa, A., Bennett, M., Bokhari, L., Rees, H., Bhoyrul, B., Chitreddy, V., Eisman, S., Sharma, P., Sinclair, R., Wall, D., Meah, N., York, K., Asfour, L., Abraham, L. S., Asz-Sigall, D., Basmanav, F. B., Betz, R. C., Bergfeld, W., Blume-Peytavi, U., … Zhou, C. (2024). The Alopecia Areata Severity and Morbidity Index (ASAMI) Study: Results from a Global Expert Consensus Exercise on Determinants of Alopecia Areata Severity. JAMA Dermatology. https://doi.org/10.1001/jamadermatol.2023.5869
  • Rodríguez-Monforte, M., Fernández-Jané, C., Bracha, M., Bartoszewska, A., Kozakiewicz, M., Leclerc, M., Nimani, E., Soanvaara, P., Jarvinen, S., Van Sherpenseel, M., van der Velde, M., Alves-Lopes, A., Handgraaf, M., Grüneberg, C., & Carrillo-Alvarez, E. (2024). Defining a competency framework for health and social professionals to promote healthy aging throughout the lifespan: an international Delphi study. Advances in Health Sciences Education : Theory and Practice. https://doi.org/10.1007/s10459-024-10316-4
  • Main, C., Haig, M., Chavez, D., & Kanavos, P. (2024). Assessing the Value of Provider-Facing Digital Health Technologies Used in Chronic Disease Management: Toward a Value Framework Based on Multistakeholder Perceptions. Medical Decision Making, 44(1), 28-41–41. https://doi.org/10.1177/0272989X231206803
  • Freitas, L., Vieira, A.C.L., Oliveira, M.D. et al. Which value aspects are relevant for the evaluation of medical devices? Exploring stakeholders’ views through a Web-Delphi process. BMC Health Serv Res 23, 593 (2023). https://doi.org/10.1186/s12913-023-09550-0
  • Gravesteijn, B.Y., van Hof, K.S., Krijkamp, E. et al. Minimizing population health loss due to scarcity in OR capacity: validation of quality of life input. BMC Med Res Methodol 23, 31 (2023). https://doi.org/10.1186/s12874-022-01818-z
  • Deering, T. F., Piccini, J. P., Graf, M., Chou, J. W., Wilson, R., Land, N., McKindley, D. S., Singh, C. M., & Blomström-Lundqvist, C. (2023). Expert Consensus on Comprehensive Early Rhythm Control in Addition to Guideline-Based Care for Atrial Fibrillation: A Modified Delphi Survey. The American Journal of Cardiology, 207, 328–335. https://doi.org/10.1016/j.amjcard.2023.08.004
  • Haig, M., Main, C., Chávez, D., & Kanavos, P. (2023). A Value Framework to Assess Patient-Facing Digital Health Technologies That Aim to Improve Chronic Disease Management: A Delphi Approach. Value in Health, 26(10), 1474-1484. https://doi.org/10.1016/j.jval.2023.06.008
  • Crespo-Lessmann, A., Marqués-Espi, J. A., Dominguez-Ortega, J., Perez de Llano, L., Blanco-Aparicio, M., Santiñá, M., Palop-Cervera, M., Álvarez, F. J., & Fraj, J. (2023). Quality indicators in the rational management of severe asthma: A Spanish multidisciplinary consensus. Journal of Healthcare Quality Research, 38(5), 277–283. https://doi.org/10.1016/j.jhqr.2023.03.003
  • Gouldthorpe, C., Ancoli-Israel, S., Cash, E., Innominato, P., Jakobsen, G., Lévi, F., Miaskowski, C., Parganiha, A., Pati, A. K., Pereira, D., Revell, V., Zeitzer, J. M., & Davies, A. (2023). International e-Delphi Consensus Recommendations for the Assessment and Diagnosis of Circadian rest–Activity Rhythm Disorders (CARDs) in Patients with Cancer. Cancers, 15(15), 3784. https://doi.org/10.3390/cancers15153784
  • Pednekar, P., Graf, M., Tuly, R., Batt, K., & Wang, C. (2023). Gaining consensus around patient risk groups and prognostic profiles to guide CMV management among patients with solid organ transplant: Insights from a Delphi panel with SOT experts. Clinical Transplantation, 37(8). https://doi.org/10.1111/ctr.14905
  • Bana e Costa, C. A., Oliveira, M. D., Vieira, A. C. L., Freitas, L., Rodrigues, T. C., Bana e Costa, J., Freitas, A., & Santana, P. (2023). Collaborative development of composite indices from qualitative value judgements: The EURO-HEALTHY Population Health Index model. European Journal of Operational Research, 305(1), 475–492. https://doi.org/10.1016/j.ejor.2022.05.037
  • Teunissen, N. M., Brendel, J., Heurn, L. W. E. V., Wijnen, R., Eaton, S., & Ure, B. (2023). Selection of Quality Indicators to Evaluate Quality of Care for Patients with Esophageal Atresia Using a Delphi Method. European Journal of Pediatric Surgery. https://doi.org/10.1055/s-0043-1777100
  • Toro, C., Felmingham, B., Jessop, S., Celermajer, D. S., Kotecha, R. S., Govender, D., Terese Hanna, D. M., O’Connor, M., Manudhane, R., Ayer, J., O’Sullivan, J., Sullivan, M., Costello, B., La Gerche, A., Walwyn, T., Horvath, L., Mateos, M. K., Fulbright, J., Jadhav, M., … Conyers, R. (2022). Cardio-Oncology Recommendations for Pediatric Oncology Patients: An Australian and New Zealand Delphi Consensus. JACC: Advances, 1(5). https://doi.org/10.1016/j.jacadv.2022.100155
  • van Alphen, A.M.I.A., van Hof, K.S., Gravesteijn, B.Y. et al. (2022). Minimising population health loss in times of scarce surgical capacity: a modelling study for surgical procedures performed in nonacademic hospitals. BMC Health Services Researc, 22, 1456. https://doi.org/10.1186/s12913-022-08854-x
  • Stenkjaer, R. L., Weis, J., Herling, S. F., Egerod, I., Ista, E., van Dijk, M., Kudchadkar, S. R., & Ramelet, A.S. (2022). Development of a non-pharmacologic delirium management bundle in paediatric intensive care units. Nursing in Critical Care, 27(6), 867-876. https://doi.org/10.1111/nicc.12809
  • Angelidou, M., Politis, C., Panori, A., Bakratsas, T., & Fellnhofer, K. (2022). Emerging smart city, transport and energy trends in urban settings: Results of a pan-European foresight exercise with 120 experts. Technological Forecasting & Social Change, 183. https://doi.org/10.1016/j.techfore.2022.121915
  • Ignatenko, E., Ribeiro, M., & Oliveira, M.D. (2022). Informing the Design of Data Visualization Tools to Monitor the COVID-19 Pandemic in Portugal: A Web-Delphi Participatory Approach. International Journal of Environmental Research and Public Health, 19(17):11012. https://doi.org/10.3390/ijerph191711012
  • Blagu, D., Szabo, D., Dragomir, D., Neamțu, C., & Popescu, D. (2022). Offering Carbon Smart Options through Product Development to Meet Customer Expectations. Sustainability, 14(16):9913. https://doi.org/10.3390/su14169913
  • Kersten, C.M, Hermelijn, S.M., Mullassery, D., Muthialu, N., Cobanoglu, N., Gartner, S., Bagolan, P., Mesas Burgos, C., Sgrò, A., Heyman, S. et al. (2022). The Management of Asymptomatic Congenital Pulmonary Airway Malformation: Results of a European Delphi Survey. Children, 9(8). https://doi.org/10.3390/children9081153
  • Mastrorilli, G., Fiorentino, F., Tucci, C., Lombardi, G., Aghemo, A., & Colombo, G.L. (2022). Cost Analysis of Platelet Transfusion in Italy for Patients with Chronic Liver Disease and Associated Thrombocytopenia Undergoing Elective Procedures. ClinicoEconomics and Outcomes Research, 14, 205-220, DOI: 10.2147/CEOR.S354470
  • Salgado, M., Vieira, A.C.L., Torres, A., & Oliveira, M.D. (2020). Selecting Indicators to Monitor and Assess Environmental Health in a Portuguese Urban Setting: A Participatory Approach. International Journal of Environmental Research and Public Health, 17(22). https://doi.org/10.3390/ijerph17228597
  • Bana e Costa, C.A., Vieira, A.C.L., Nóbrega, M., Quintino, A., Oliveira, M.D., & Bana e Costa, J.C. (2019). Collaborative Value Modelling in corporate contexts with MACBETH. Procedia Computer Science, 162, 786–794. https://doi.org/10.1016/j.procs.2019.12.051
  • Presentations of applications using Welphi:

  • Graf, Marlon, Pamela Baker, Meaghan Roach, Jacquelyn W. Chou, and John M. Kane. Assessing expert consensus around progression and mitigation of cognitive decline following onset of schizophrenia disease: A Delphi method study. American Psychiatric Association (APA) Annual Meeting (virtual), May 2021.
  • Graf, Marlon, Carolyn Harley, Rifat Tuly, and Katharine Batt. Assessing Expert Consensus around Current and Future Coverage Practices for Prophylaxis Treatments of Hemophilia A without Inhibitors: A Delphi Method Study. Academy of Managed Care Pharmacy (AMCP) Annual Meeting (virtual), April 2021.
  • Freitas, A, João Pedro Pêgo, António Augusto de Sousa, José Fernando Oliveira. Promoting Faculty pedagogical development: the strategy for a public engineering Higher Education institution. CISPEE 2021 4th International Conference of the Portuguese Society for Engineering Education, June 2021.
  • Sources:

  • [1] Linstone, H.A. and M. Turoff, The Delphi method: Techniques and applications. 2002, Reading: Addison-Wesley.
  • [2] Gupta, U.G. and R.E. Clarke, Theory and applications of the Delphi technique: A bibliography (1975–1994). Technological forecasting and social change, 1996. 53(2): 185-211.
  • [3] Warner, L.A., Using the Delphi Technique to Achieve Consensus: A Tool for Guiding Extension Programs. 2014, Agricultural Education and Communication Department, UF/IFAS Extension.