Survey of biomedical scientists (July-August 2023) - Individual repllies
see also www.cellcomm.org/survey-planning
Q2. Which are the advantages and disadvantages of promoting the development of AI systems that are more transparent, and where the biomedical community participates in the control?
Advantage - ensure that the training is accurate and acknowledges gaps in knowledge.
Advantages: hopefully less biased, more well-rounded. Disadvantages: issues in making it truly accessible to all and transparent, regulated over time
More accurate output of AI, more likelihood biologists and biomedical scientists will benefit from AI.
transparency is essential.
Transparency is incredibly important. It can increase trust, increase performance, and decrease falsifications for monetary value. I don't know how the biomedical community would participate in the control, but I don't have a clear idea of what that would actually look like.
Give to AI validated data from scientists not devoted to industry.
Better understanding of the goals and expectations that each person in the community can expect from oneself and collaborators. Knowing the ground rules levels the playing field for everyone.
Advantage: To be used to better assess big data and provide predictions & patterns. Disadvantage: deception.
Someone taking profit of it.
Issue is confidentiality.
Peer review by trusted practitioners--Good. openness is always good.
Promoting transparency in AI development and involving the biomedical community can yield numerous benefits in terms of ethical AI, trust, and bias mitigation. Finding the right balance is essential to harness the full potential of AI in the biomedical domain while safeguarding patient rights and ensuring ethical use.
The advantage is to improve the accuracy of knowledge provided by the AI system. Disadvantage is that training AI with feedbacks from human will lead to out of control of AI systems, which will control human someday.
1) Explicit disclosure of AI use in research, including details, in publications and other forms of dissemination of knowledge; 2) develop universal guidelines and criteria to establish transparency
very powerful technology that should be overseen by a non-partisan community from various contingencies of biomedicine; I do not see disadvantages.
A: AI systems could reduce human factor (i.e., errors, mis- and under interpretation of results, and perhaps see trends that are not discovered with conventional analytic methods or procedures). D: we know little so far if AI-based analyses are biased in a different, unexpected ways.
advantages: may contribute to the development of new treatment for neurological and neurodegenerative human diseases. Disadvantages: lacking of control and transparency of AI would be a high risk affecting human health.
Really any aspect of the biomedical sciences has good and bad aspects and thus the field in general is going to be benefited by transparency. How the scientific community does this is very complicated and most likely is application dependent.
There are only advantages to more transparent AI systems: understanding of potential biases, awareness of overtraining, and understanding of limitations of clinical translation.
It will limit the spread of misinformation by AI chatbots.
AI systems that impact patient care directly or indirectly must be evaluated for training set bias, or unintentional discrimination may result. The identification of such weaknesses will allow for better focus on remediating these data gaps.
Transparency is always good. I think that there have to be reality checks on AI. Computers are very powerful, but when they get it "wrong" they get it very wrong. I envision that at some point an AI system will come up with an important statement that will be accepted as "fact" that is incorrect. However, we will not realize this (certainly at first) because the calculations to check the AI system are too complicated for any person. I don't want our field or our society to go way off track based on a "fact" that AI comes up with. For example, it might evaluate all of the analytes in the blood of every person who has ever been hospitalized with anything and conclude that a certain concentration of a certain metabolite was a great predictor of someone becoming psychotic. Do we test those people and guard them with suspicion?
advantages are ability to more easily map out and account for issues of implicit bias, boundary conditions on assertions or conclusions offered by the AI systems, and ability to understand gaps or over/under-representation of data used to train/tune the AI system.
When accessing knowledge we need to know if it is generated by humans or AI.
clearing out misinformation
We can be participants to establishing the rules of transparency, rather than the recipients of rules made by others outside the biomedical community.
needed for validation of AI claims.
useful for intervention development and health messaging
This should be part of a national and international discussion.
"Disadvantages: 1. Any limitations imposed by the control of the biomedical community may bias and limit the output of a complex and self-learning system like AI; 2. AI control and transparency should be provided by AI experts, not by the biomedical community (everyone should do their own thing)."
Advantage: more control at every step is desirable; disadvantage: the biomedical community is diverse and without general community, policy, and legal governance I would not trust one community more than another. This has been obvious even in a pandemic where one new virus appears. Society had to deal with pathogens since its beginnings, and still many biomedical scientists did not even follow the basic guidelines of how research and recommendations should be done.
understanding how it works and what its limitations are can only be helpful. not participating in seeing how it can be applied to our work can only lead to disaster later.
Transparency should promote algorithms less susceptible to bias.
1.Any system must be applicable equally to academia and industry, as well as for profit and non-profit organizations.
The advantages are obvious, as you noted in the introduction. However, examples of the disadvantages are as follows: conspiracy theories and dishonesty; jealous/unethical competition among individuals, groups and countries; and fraud.
The hope would be to reduce bias and improve accuracy of AI systems; the fear is that control is essentially impossible at this point.
Advantages: I think AI could be developed as a tool to generate a more unbiased, comprehensive literature summary that would greatly aid in the review process of grants and manuscripts. Disadvantages of transparency is that it may be hard for those without computational backgrounds to understand AI development. So part of transparency needs to be plain language communication of capabilities.
pro: understanding the ramifications of training data sets and potential biases etc. con: could land up limiting power of system.
Reproducibility, external validity
AI is still pretty much GIGO [garbage in - garbage out] as the training and testing sets are still so few and small. Replication is near impossible. Biomed must participate to keep the training and outputs honest and relevant.
advantages could be increased accuracy of AI predictions, ability to manage larger datasets, ability to integrate knowledge across fields or sources, learning about new fields more quickly. disadvantages could be incorporation of misinformation from bad sources.
There are no downsides to transparency. It is essential.
Keep human in the loop is a key for AI development. In healthcare, development of AI system can improve efficiency, reduce burden, likely improve quality of care, or find new treatment. However, all AI systems potentially have bias. Transparency allows identification of biases, improve AI systems or model accuracy, reduce potential unexpected risks, and earns public trust. Biomedical community should actively engage with public and industry, maximize the benefit, protect individual privacy and promote equality, and reduce the potential risk of harm.
I'm not sure where the biomedical community participates in the control of any of these methods. We need some agreement on somewhat straightforward things like writing grant applications (is it OK for AI to shorten your text) to complex ideas of how graduate students and other trainees can meaningfully embrace the technology but still "learn how to be scientists". There is such inherent biases in the current systems which are extraordinarily dangerous in propagating stereotypes that need to be addressed immediately.
To be honest, I'm not sure, but I am definitely fearful of where AI can go awry, especially if control and stopgaps are not in place.
Already studies started to state that big data are analyzed by AI to generate a conclusion. The reviewers and readers have no clue what has been done to generate the results. Transparency and describing the methods used are critical.
You can increase productivity, but you can also be led in wrong directions.
AI has tremendous potential, so it needs to be developed. But it also needs to be controlled so that misinformation is not created and propagated. The biomedical community needs to be part of the conversation. I don't see any disadvantages to that.
Choice of Database in use.
transparency, avoiding bias, etc
Obvious ethical societal political importance
ChatGPT answers are great!
Pro: shows helpful non "under the light bulb" findings. Con: introduces bias to the algorithm...
more reliable results
A critical problem with AI is that it derives its product from all available data without necessarily showing attribution. This raises questions of intellectual property, as well as interfering with critical assessments of the AI product. Part of the desired transparency in AI systems is attribution of sources.
Safety, carefulness, time to be thoughtful.
AI tools can be useful to biomedical scientists in that they can collect and process information as an aid to scientists. But if used as a substitute for careful analysis, or used to generate content that is presented as coming from an investigator, AI has the potential to promote fraud. Not sure exactly what you mean by a transparent AI system. It seems like the transparency that's needed involves the user, who should make it clear that the work product involved the use of AI tools.
More transparency in the development of AI for biomedical use is imperative as currently tech companies are developing algorithms but there is no way to understand what the algorithms are supposed to do and what data is being used and how for AI development. Only by participating in this development can the biomedical community have any understanding of its use, where it is doing well and where it needs improvement.
Accurate, verifiable data - when I tried Chat-GPT, it issued a list of solid-appearing references to publications that did not exist!
They can be a major driver of new research ideas. But they can also pose threats to privacy, data development & research integrity, & plagiarism.
Seriously, of course
Health delivery to the community in general. Advantages are numerous; disadvantages are few in number. Can AI promote cost saving in health care delivery??!
Full transparency is necessary for scientific progress.
Control of AI systems by experts may promote reliable and accurate information.
AI is part of the future, and we must learn to use it in a way that does not harm people, but unnecessary regulation should be avoided.
there should be a national debate about this. it is possible to better diagnosis of human diseases.
AI can provide insights into complex systems that are difficult for humans to understand. But
there are some concerns over the trustworthiness of these systems: do they provide accurate representations of real-world biological system. AI systems should be used to guide decisions but humans should always be the decision makers. Humans should be the central controller not machines.
Perceived security with the disadvantage that AI systems can swiftly become difficult to monitor.
+s: More likely to be clinically useful; less likely to be based on biased data; more trustworthy for patients and physicians. -s: The opposites of the +s.
I honestly don't think this goes far enough. The research, ethics, and community at large should have a say in how cutting-edge AI is developed and what guardrails are in place
Advantages: -may provide more scientific insight into problems, disadvantages: - limit utilization. Transparent systems are required by clinicians as an excuse for not using them. As an engineer, I don't care how I get the right diagnosis.
only advantages in managing development of AI......the above question could have been rewritten by AI into a more concise form.
Integration of really large biomedical datasets.
they will be more accurate and useful.
Transparency in developing the tool.
I see mostly advantages. Disadvantages could come from AI training using patient health data.
Advantage: people feel better. Disadvantage: biomedical practitioners have no deep knowledge of AI systems, so will not have informed input regarding their function.
Advantages include disclosing who has access to the information gathered and processed by AI (transparency) and that the information can be edited based on accuracy rather than on plurality (control). Disadvantage is similar to Wikipedia, Twitter in its original iteration, and the federal government mostly under the Biden administration in that who and how decisions to edit may actually squelch and hinder the open dialogue and hypothesis testing that is the foundation of the scientific method.
Open source, and annotated, is the way to go, so that all users can understand how AI is being used for their projects.
Hopefully AI will allow us to give better care to our patients and facilitate many of our procedures.
I see only advantages in developing AI systems that are more transparent.
Advantage: the user will know what happens with the data! Right now, nobody is sure of what 's going on. Disadvantages: I do not see any. Where: this needs a serious ethical discussion, with specialists of both fields. The more people involved, the better.
advantage: reproducibility, accessibility. disadvantage: could slow down the development.
adv: reduce inconsistent results.
The present AI tools used in the HC [Health Care] are biased and we do not know how they arrive to their conclusions.
1. Better understanding of how decisions are made by AI. 2. Better control over private industry profiting at the expense of researchers 3. Open source remains open.
Insure professional standards and up-to-date guidance compliance.
Real understanding and progress only come from understanding the principles, not by regurgitating and reproducing old facts. We should not allow AI to pretend to explain biology and pretend that it is science.
understand value and potential risks.
Advantages: A more accurate and useful AI. Disadvantages: time taken from the biomedical community to participate in shaping the AI model.
I do not see any disadvantages.
Advantages: Better understanding of the output of AI. Disadvantages: Gatekeepers to the control can introduce bias. Biomedical community can participate in the input and feedback levels.
It is critical for maintaining the integrity of the scientific process to control AI as it is known to make up data and even quote incorrect, nonexistent references.
developing AI based on reliable/repeatable data that is peer-reviewed rather than exclusive AI self-training will ensure that data used to further develop AI is valid and validated.
Transparent, open systems will be easier to test for biases and correct.
AI will never replace human mind.
Adv: allows for ethical development that can advance science; disadv: may slow creative solutions.
I don't know enough about this specific topic to offer informed advantages or disadvantages.
Scientists may not understand how these work. Unless regulated, makers of AI systems won't want to bother engaging the scientific community in critical dialog.
Advantages: 1. Trust and Acceptance: Transparency in AI systems increases public and healthcare professionals' confidence in these technologies. 2. Explainability: Transparency allows understanding how decisions are made in AI systems. 3. Detection and Correction of Biases: With greater transparency, it becomes easier to identify potential biases and prejudices in data or AI models used in the biomedical field. Disadvantages: 1. Complexity and Performance: Transparency can imply that algorithms are more complex and less efficient in terms of time and computational resources. 2. Protection of Sensitive Data: Transparency may make handling and protecting sensitive and private medical data more challenging. 3. Limitations on Innovation: Transparency could restrict innovation, as some novel and highly effective AI approaches may be less explainable.
Ensure proper use of the systems and that the systems are trained on good input/data. Also, scientists need to know when their data and input are being used by AI systems for training, etc.
While AI will provide knowledge for better choice, fabrication of data may be a trick.
reliability and correctness of data
A major advantage is the avoidance of bias and improvements in the ability to judge the reliability of inferences and generated data.
It is important to understand the limitations for AI. For example, a well-known problem with chatgpt is that is "hallucinates" and makes up false information. Therefore, the tool needs to be more transparent when it does not really know. Some sort of arbitrary confidence calculation would be helpful for the human user to interpret the AI response.
AI has many downsides, like privacy issues, deepfakes, and existential risk. These should all be discussed by as wide a group as possible.
Haven't thought about it.
Could be extremely helpful in writing background info for manuscripts and possibly assist in analysis of large data sets. Both a positive and negative associated and needs to be closely regulated with specific procedures outlined for use and referencing the use.
transparency will be essential to judge the quality of AI results. There is much bias in AI models, for example bias towards privileged populations, as available date is dominated by results from those people. Involvement of the biological community might help alleviate this bias, although I am not overly optimistic.
helpful for students and faculty
transparency is an integral part of the academic enterprise.
Curated content from verified sources, synthesis/suggestions always with references
improved standardization; more objective measurement of performance metrics; easier improvement of code
No advantages. Disadvantage: Wrong dissemination of knowledge.
Advantages are numerous and obvious. Only disadvantage I can think of - if the methods developed to promote and oversee AI by the biomedical community are used by "bad actors" (possibly themselves AI) to better their manipulation of systems and/or obfuscate their origins.
Development of AI system will help in understanding cell - cell communication, more transparent AI system for biomedical community is required.
I think it will be robust and helpful.
the main problems are (1) the AI uses whatever it can find, even is false, and (2) the AI uses only what it can find, even if incomplete.
Advantages: build trust, control misuse. Disadvantages: potential to slow advancement of the field.
Transparency promotes trust.
It's clear that AI will be used more often in the future, so it's important to establish guidelines for its use upfront.
Parsing large amounts of data, collating and filtering data to find correlations, particularly across ethnicities, geographic locations and other demographics. The biomedical community at this stage needs to work hard to curate data and evaluate the output from AI. In my experience, not all of the results make biological sense, or even any sense at times, so we use it to define sub cohorts for better analysis, cut offs for stratifying, etc. However, for using AI in medicine, it's useful for diagnostication in conditions that have well defined parameter ranges, normative vs disease or specific to prediction of sequelae, etc. on the basis of clinical features, biomarkers, etc. The AI must have certain built in checkpoints for data accuracy at the input level and quality assessment of the output. Therefore, throughout, transparency is critical.
I don't have the knowledge to sensibly answer this question. I am sure controls are needed urgently.
I only see advantages. scientists have a more objective view of things and are trained to spot -and minimize or control - bias.
Adv. More protection from bad actors
Biologist and medical scientists must keep pace with the developing technologies and make its use in best possible way in teaching and research while adhering to ethical requisites.
Scientists are increasingly dealing with larger and larger datasets, and certain AI algorithms are better suited for analyzing these datasets. However, the scientists need to understand better why ML clusters data in particular ways. Do not accept the answer "I don't know what's driving these associations". Always ask.
It helps to understand and discuss everything in the name of science.
transparency in itself is a benefit and a goal in terms of verifying sources.
Acceleration of discovery but only if results verification embedded.
AI systems are changing humanity and how humans function in all domains. An important part of this change is the "guidance" that AI analysis will provide. If we don't understand who/what is in control of the systems that guide us we cannot understand how "decisions" are made and for whose benefit. They will manipulate us. We must understand and minimize this.
Transparency is the key. The problem will be to design monitoring and editing systems which increase accuracy while remaining open and transparent.
Biomedical community should not control the AI.
Advantages- statements supported by data, according to scientific principles.
it is not clear to me how the community would "participate in the control" for training these systems.
Application in patient therapeutics, patient diagnosis, infectious agent identification in the clinical labs.
AI will greatly expedite key learnings.
Not sure- so early but it is clear that it is powerful and but lacks accuracy in text. It will be interesting to see where it goes with hard science.
pro. easier mining of large datasets and across datasets. con. how to gauge trustworthiness/accuracy of ai generated results
Advantages only.
I see only advantages in making the AI system more transparent and having the biomes community be involved in that process.
For any powerful system, having control is a necessary advantage.
Advantage: To ensure that AI training sets do not disadvantage marginalized groups. Advantage: To foster collaboration and sharing of information in development of expert systems and algorithms. Advantage: To ensure that ethical standards are developed and maintained in the use of these systems. Advantage: to ensure that discoveries are not paywalled and monetized to the detriment of scientific progress.
helping to draft statements, avoiding student plagiarism, avoiding reproducing data and facial images, adding to scientific discourse.
We should aim to promote transparent AI development. The only disadvantage to transparency is regulation that is overly burdensome to progress in development.
we should pay attention to the training set, which are biased against the new
There must always be a community involvement in this. It is too powerful a tool to be randomly regulated. It always needs outside validation and regulation and then repeat both.
Advantages - it could help us use the technology productively and stave off negative impacts; Disadvantages, it may take time away from our research.
Need to understand the data.
Advantages - speed of search; Disadvantages - inaccurate data is often provided by these searches. For instance, I was quoted made up references by ChatGPT. Very worrisome trend!
ensure accuracy of training input
Adv: trustworthiness, robustness; Dis: slower development.
Advantage: better than letting focus stay on uncontrolled AI. Disadvantage: adds legitimacy to broad (and oft unreliable) AI efforts
AI systems are intrinsically NOT transparent, even by their creators.
It is key that all aspects of AI development and implementation be transparent. AI uses public and private information to train on and ownership of training items (faces, scientific papers, data) be discussed. Same with implementation, where even thornier ethical issues will arise.
In my opinion there is not a single positive point of AI regarding education or in any aspect. Rather it is minimizing creative abilities of person using it.
I don't see how we can use such systems without understanding how they get their information and from whom.
Control helps in that the human aspect is not lost. Disadvantage is that in may cause slowing down