artificial intelligence

Introduction

AI, simply put, is simulated intelligence demonstrated by the software with the ability to depict or mimic human brain functions, and operative on “mimic,” number one
AI is great at things like analysis, pattern recognition, image analysis, information processing.
AI algorithms can have bias which means there's a statistical variance when something is being predicted between one group of patients or people and another resulting in predictive capabilities that are different for one population versus another - often the AI algorithms are “learned” using cohorts of patients such as white males and thus may not be accurate at predicting other subgroups of patients or uncommon conditions

Clinical use of AI in Australia

Australian AHPRA guidelines on the clinical use of AI Aug 2024
- if the AI tool “meets the definition of a medical device, which includes software (including AI-enabled software) if it has a therapeutic use and meets the definition, then the device must be approved for clinical use by the TGA”
  - AI would generally be a medical device if it is intended to be used for:
    - diagnosis, prevention, monitoring, prediction, prognosis, or treatment of a disease, injury, or disability
      - this includes clinical decision support that uses generative AI to create symptom and diagnosis summaries
    - alleviation of, or compensation for, an injury or disability
    - investigation of the anatomy or of a physiological process
    - control or support of conception
    - HOWEVER, “It is important to note that it is unlikely to be a medical device, and is not regulated by us (TGA), where there is no medical purpose or claim associated with the product using the generative AI such as LLM”
    - software may also be excluded from being a “medical device” as per TGA if either:
      - it is a consumer health products – prevention, management and follow up devices that do not provide specific treatment or treatment suggestions
      - it is Digital Mental Health
      - it is an enabling technology intended to support telehealth, remote diagnosis, healthcare or dispensing
      - it is digitisation of paper based or other published clinical rules or data including simple dose calculators and Electronic Patient Records
      - it is using population based analytics that do not drive outcomes for individuals
      - it is a Laboratory Information Management System –systems to automate workflows, integrate instruments, manage orders and samples and associated information.
- “Regardless of what technology is used in providing healthcare the practitioner remains responsible for delivering safe and quality care and for ensuring their own practice meets the professional obligations set out in their Code of Conduct. Practitioners must apply human judgment to any output of AI.”
- “all tools/software should be tested by the user/organisation to ensure they are fit-for-purpose prior to its use in clinical practice”
- “If using an AI scribing tool, the practitioner is responsible for checking the accuracy and relevance of records created using generative AI.”
- “practitioners need to involve patients in the decision to use AI tools that require input of their personal patient data and if a patient’s data is required for care”
- “ensuring confidentiality and privacy of your patient/client as required by privacy and health record legislation, by checking that data is collected, stored, used and disclosed in accordance with legal requirements, and your patient’s privacy is not inadvertently breached”

AI architectures

Statistical probabilistic association AI

this is the main architecture in use today by ChatGPT, etc
a neural network mathematical model is trained on enormous amounts of data and the system develops and stores billions of values for parameters
it is great for creating Large Language Models (LLM) which can be generalisable as well as pattern recognition, object detection and image generation
however, it does not have a logic base of cause and effect, nor does it understand physical reality
when it is asked a question for which there was no adequate training data, the statistical predictive algorithms may need to accept a low probability prediction word which then becomes a self-perpetuating incorrect response which has been called “hallucination” although this is a poor choice of word - it is just that it is gone down the wrong rabbit hole - in much the same way that clinicians may erroneously follow a clinical care pathway and fail to realise that it is the wrong pathway.

Cognitive modelling AI

a variety of cognitive AI attempt to model the human brain components such as short term, long term memory, logic, etc
currently are not generalisable and are used for specific use cases
have better reasoning, symbolic processing, planning, learning, and memory management while being more consistent with less “hallucinations” but have less world knowledge, less knowledge scalability and are not as good at language processing
biologic modeling to predict neural activity and cognitive behaviour: LEABRA, SPAUN
psychological modeling, predicts reaction time and cognitive errors: ACT-R, EPIC, CLARION, LIDA, CHREST, 4CAPS
AI functionality inspired by psychological and biological modeling to emphasize more complex cognitive processing: SOAR, Companions, Sigma, ICARUS, CogPrime
in 2017, a “Standard Model of the Mind” was proposed to help unify human-like cognitive AI based upon concepts within cognitive AI architectures such as ACT-R, Soar, and Sigma ¹⁾
- Behavior is driven by sequential action selection via a cognitive cycle that runs at ~50 ms per cycle in human cognition
- Working memory provides a temporary global space within which symbol structures can be dynamically composed from the outputs of perception and long-term memories.
- Procedural memory contains knowledge about actions, whether internal or external. This includes both how to select actions and how to cue (for external actions) or execute (for internal actions) them, yielding what can be characterized as skills and procedures.
- Declarative memory is a long-term store for facts and concepts. It is structured as a persistent graph of symbolic relations, with metadata reflecting attributes such as recency and frequency of (co-)occurrence that are used in learning and retrieval.
- Learning involves the automatic creation of new symbol structures, plus the tuning of metadata, in long-term – procedural and declarative – memories. It also involves adaptation of non-symbolic content in the perception and motor systems.
- Perception converts external signals into symbols and relations, with associated metadata, and places the results in specific buffers within working memory. There can be many different perception modules, each with input from a different modality – vision, audition, etc. – and each with its own perceptual buffer.
- Motor converts the symbol structures and their metadata that have been stored in their buffers into external action through control of whatever effectors are a part of the body of the system.

Approaches to integrating LLMs with cognitive architectures

https://arxiv.org/pdf/2308.09830.pdf

modular approach

iterative LLM approach:
- “The cognitive AI uses symbolic structures as inputs and executes one or several cognitive cycles, after which, the contents of the working memory, including fired productions, relevant information from declarative memories, and actions, are injected as cues into the next intermediate step of the LLM.

agency approach

micro-agents operate within the framework
- these act a micro-level, with each agent operating through either a fine-tuned LLM or a symbolic processor
- sensory inputs are processed by the perception module, yielding abstract entities like objects, categories, actions, events, etc. forwarded to the working memory
- working memory cues declarative memories to establish local associations, e.g., user navigation preferences, place familiarity, and more
- specialized agents at the agency observe working memory contents and form coalitions
- the coalitions are transferred to the Global Workspace, where a competitive process selects the most relevant coalition
macro-agents interact with humans etc

neuro-symbolic approach

user request
perception: LLM - symbols or natural language
working memory
action-centered sub-system (ACS)
- utilising algorithms (if statements)
- operates across two distinct levels:
  - the top level (symbolic), responsible for encoding explicit knowledge
    - Fixed Rules (FR) are rules that have been hard-wired by an expert and cannot be deleted
    - Independent-Rule-Learning (IRL) rules are independently generated at the top level and can be refined or deleted as needed (ie. Top down learning)
    - Rule-Extraction-Refinement (RER) rules which are extracted from the bottom level (ie. Bottom-Up learning but requires feedback rewards/reinforcement)
  - the bottom level (connectionist), tasked with encoding implicit knowledge and may incorporate LLMs
- these synergistically engage in action selection, reasoning, and learning processes
“motor” output or response

software for creating artificial intelligence tools

TensorFlow
- Python-based end-to-end open source platform for machine learning created by Google
Jupyter
- Python-based tool which exists to develop open-source software, open-standards, and services for interactive computing across dozens of programming languages
- can leverage big data tools, such as Apache Spark, from Python, R and Scala. Explore that same data with pandas, scikit-learn, ggplot2, TensorFlow
Apache Spark
- a unified analytics engine for large-scale data processing
- can use it interactively from the Scala, Python, R, and SQL shells
PyTorch - CUDA optimised Python modules for deep learning and tensor manipulation
Markdown
Shiny
Tidyr
knitr
ggplot2
Artificial Intelligence using Delphi Programming Tool
pre-trained large language models (LLMs)
- see AI Large Language Models (LLMs)

Hardware for creating AI

this is mainly the Graphics Processor Unit (GPU) of a computer which is far more efficient at processing complex calculations of large matrices than the traditional CPU
- basic training of a simple AI model with images is likely to need as a minimum a nVidia RT GPU with at least 8-16Gb VRAM and for laptops this means the high end gaming laptops
large language models use tens of thousands of these GPUs at the same time requiring megawatts of electricity to perform their calculations on matrices of billions of parameters
these computers will soon consume more electricity than small cities!
the cost of these computers in 2023 is in the millions of dollars and this will only grow as demands for improved AI grow and thus it is likely that the future of AI will be dominated by the super rich who will only increase the economic divide - the far majority of profits from AI in the future is likely to flow to the US and China leaving most other countries to just rent access to the AI
- in late 2023, a range of US companies have already bought 10,000+ nVidia H100s
- in 2024 it is likely that groups will spend over $US1billion to develop a single large-scale model
nVidia has created a dominating role by developing it's proprietary CUDA technology
- in 2023: US has banned export of their high end GPUs (H100's and A100's) to China
nVidia's GPU chips are made in Taiwan by TSMC using ASML's EUV (Extreme Ultraviolet) photolithography machines
- ASML is a Dutch company which can make 40-60 EUVs per year with each EUV costing $200-400m and weighing ~ 200 tons
- Intel and AMD CPU chips are also made using these EUVs
in 2023, a company has demonstrated reasonably accurate image generation of human thoughts when a human looks at an image suggesting that bionic hardware development is in its early phases
in 2024 we will see proprietary inference focused AI chips
- Oct 2023, IBM created an analog computer chip to be more efficient at AI calculations
- Oct 2023, Chinese make 1st memristor AI chip “75x” more efficient for AI and “mimics the energy efficient approach of the brain” hence developed “Computing-in-Memory” (CIM) with close integration of computing and memory on the one chip - memristor remembers how much current passed though it by changing its resistance according to the prior current so it can store information without needing ongoing power and power consumption is thus only “3%” of current computer chips

Consumer device AI chips "NPUs" or "neural chips"

these AI enabled computer chips are NOT designed to train LLMs but primarily to analyse images on your device and process them and potentially run inferences on local small LLMs
these were first introduced into Apple iPhones as “neural chips”, then to Macs as Apple Silicon, and then Google and Samsung added their versions
in 2024, NPUs by Qualcomm have been added to Microsoft Surface Pro and will be also added into new Windows computers by Intel and AMD as they develop these
they can thus add the following capabilities to your device without sending data to the cloud for processing:
- blur the background out of your portrait images
- add extra images to your photos (rabbit ears, etc)
- easily remove subjects from your images
- scan your environment when the camera is on and analyze what is in the environment and enumerate all objects in text form, who is looking at the device, what their emotions are, their approximate age, etc (Meta in 2024 is allegedly going to use this feature to detect under-age users to block access to 18+ content)
- “client-side scanning” - scan your photo library and video library to detect imagery that is illegal and report this back to the device provider
- scan any screenshots that are taken and analyze what you are doing on the device and potentially send this information back to the device provider (eg. Microsoft's new Windows 11 “Recall” feature which will screenshot your screen every 5secs if not disabled).
- run small LLMs as inference engines
the dangers of this technology:
- device or software providers such as Microsoft could utilise the combination of their keylogging “Activity history”, your browser history in Edge, 5second screenshots via Recall to see what you are viewing, and the analysis by NPU to collect a very complete analysis of who you are, how you think, what you do, your political and religious views, what your preferences are, who your family and friends are, where you work, your health, etc. When you ask Co-Pilot a question it may interrogate all your “local history” and send this to Microsoft's LLM inference servers … and then once they have this data they could utilise this for marketing, manipulation, AI training or other purposes. Meta has been collecting much of this data already via Facebook, Instagram and WhatsApp and is now using your data, photos and videos to train it's own AI models, but Microsoft's new tech may take this intrusiveness far further. Apple is likely to incorporate similar features into their products following their June 2024 announcement of “Apple Intelligence” on your Apple devices.
- furthermore, this has the potential to be “hijacked” by malware or by someone or AI utilising Bluetooth 5 LE technology via other devices even if your device is not “turned on” (this is how you use Find My Phone and Bluetooth location tagging devices) - this is the Terminator Skynet scenario where the network is able to use everyone's devices to “see” and “hear” what is going on everywhere, without even using the internet as Bluetooth LE is peer-to-peer network technology.

AI clinical use cases

Study aide
- generate MCQs from provided document source material
- organise and summarise study notes
- speech transcription - take personal notes, transcribe conferences, etc:
  - dedicated portable devices for local, private transcription +/- auto translation of speech:
    - iFlyTeck audio recorder and transcriber
- create catchy music with your study lyrics such as this one I created to help clinicians remember the COACHED algorithm for CPR: COACHED algorithm song
AI assistants
- you can search the model's knowledge base for answers just by asking the question
- you can upload a large document to the AI and then ask questions and it will find the answers from the document and save you manually searching for them
- customised ChatGTP versions using carefully constructed underlying contextual prompt information combined with uploaded documents can form a basis for a series of specifically tasked AI “agents”
- potentially if the AI has access to patient data it could perform many clinical assistant functions
- software developers can also use these to write code
Private GPTs
- allows querying of YOUR hospital's docs / files via a LLM privately without access to an internet connection
- see private GPTs
AI language translation
- see AI powered clinical translators
LLM creation of structured databases from unstructured data
- https://www.youtube.com/watch?v=tcHIDCGu6Yw - Python, ChatGTP app using a Neo4J database to store json outputs
automated workflows
- these could be achieved with customised ChatGTP AI versions combined with automated no-code actions such as Zaps such as Slack notifications
  - NB. using Zaps involves a fee - team accounts start at $AU110/mth for 2K tasks/mo, $AU400/mo for 20K task/mo and there are higher levels.
  - see https://www.youtube.com/watch?v=p_xkBMnQVI0
more functional and powerful devices
- “semantic hearing” aids - AI could allow users to block only certain types of sounds - already coming in 2023 in the form of noise cancelling headphones but could potentially be utilised in next generation hearing aids
measuring vital signs
- measuring BP from forehead images
- Measuring SaO2 using finger on smartphone camera and LED light
- thermal imaging of face to detect coronary artery disease - 13% better than the pre-test risk assessment involving traditional risk factors and clinical signs and symptoms
  - specifically, the average temperature of the left jaw region was the strongest predictive feature, followed by the temperature range of the right eye region and the left-right temperature difference of the temple regions. This also effectively identified traditional risk factors for coronary artery disease: high cholesterol; male sex; smoking; excess weight (BMI); fasting blood glucose, as well as indicators of inflammation. ²⁾
- AI empowered wearable biologic sensor devices
  - HR variability analysis, “blood glucose, urate”, BP, ECG, sleep / activity, falls detection, etc
- AI prediction of cardiac arrest hours prior based on HR variability
image diagnosis (pattern recognition through machine learning)
- radiologic images
  - Dx endometriosis using US and MRI images ³⁾
  - Sybil: an accurate future prediction of lung cancer from one CT scan ⁴⁾
  - coronary angiography ⁵⁾
  - CXR - AI estimated “biologic age” - trained on over 100,000 CXRs ⁶⁾
  - CXR can be used to predict lung function tests accurately ⁷⁾
  - fMRI images - AI can detect depression better than doctors can
- dermatology
  - see https://www.youtube.com/watch?v=SzUGQMx0dkw on using ChatGTP-4V to diagnose skin images
- retinal images:
  - AI “perfectly” diagnoses autism spectrum disorder (ASD) status from high-resolution retinal images ⁸⁾
- ECGs:
  - June 2023: New AI Tool Beats Standard Approaches for Detecting Heart Attacks including HEART score ⁹⁾
- histology and blood film analysis:
  - could be an efficient tool to reduce human burden of analysing millions of specimens
  - in June 2024, it was reported AI was used to discover a new high grade histologic form of endometrial cancer ¹⁰⁾
mass spectrometer data analysis
- traditionally, the process for analysis of cellular glycans by mass spectrometer can take anywhere from hours to days for each sample and can only be carried out with high confidence by a small number of experts in the world, as it is essentially detective work learnt over many years
- AI can now do this in a few seconds and this can potentially be used to better detect cancer cells ¹¹⁾
- Fourier transform infrared (FTIR) spectroscopy AI analysis of a blood sample can detect and predict a range of medical conditions, opening up significant potential for rapid disgnostics ¹²⁾
metabolomic data analysis
- studies of vast numbers of metabolites in the blood to ascertain risk for or presence of disease
- Interpretable Machine Learning on Metabolomics Data Reveals Biomarkers for Parkinson’s Disease ¹³⁾
- markers for early detection of cancers - eg. the company Grail is using AI in its search for multi-cancer early detection
genomic and protein analysis
- NHS is already trialling whole genome sequencing at birth, and this would give us the genetic information which AI could analyze.
- AI can model protein folding (once only analysed by Xray crystallography) and can thus better predict actions, develop pharmaceuticals and vaccines - eg. Google AlphaFold
- cancer genomics for better targetted chemotherapy
- Oct 2023: AI identifies proteins as possible targets for gonococcal vaccines ¹⁴⁾
- analysis of genomes such as CSIRO's VariantSpark can discover relationships between genes which result in phenotypic diseases such as Alzheimer's
transcriptomic analysis
- ““We also believe that spatial transcriptomics, or tissue genomics, not only offers hope for better understanding of malaria, but also has the potential to revolutionise how we approach the diagnosis and treatment of any disease that affects a human tissue” ¹⁵⁾
natural language processing
- research
- ChatGPT to answer questions or generate discharge letters
- identification of abnormal radiology reports
  - Ann Emerg Med. 2023 A Natural Language Processing and Machine Learning Approach to Identification of Incidental Radiology Findings in Trauma Patients Discharged from the Emergency Department
predictive modelling
- cardiac arrest outcomes:
  - Gothenburg Cardiac Arrest Machine Learning Studies - 30 day survival SCARS-1 OOHCA predictions are 95% sens and 89% spec with AUC-ROC value at 0.97
- ascertaining polygenic risk for heart disease based on assessing millions of genes in a person's genome with learned data from Europeans and then this is used to determine how much intervention in BP or lipid control is needed to nullify that risk. The study also showed family history of heart disease is independent from polygenic risk and a much stronger predictor of major coronary events than polygenic risk for most people.
  - https://www.news-medical.net/news/20230305/Causal-AI-quantifies-benefits-of-actions-needed-to-overcome-inherited-risk-of-coronary-artery-disease.aspx
- cancer risk models:
  - 2023: Nature "A deep learning algorithm to predict risk of pancreatic cancer from disease trajectories"
- materials science:
  - Google's DeepMind AI GNoME has already predicted 2.2 million new crystal structures, vastly expanding potential materials for advanced technology development
- pharmaceuticals:
  - Dec 2023, AI trained on 22 million existing chemicals discovers new class of antibiotics active against MRSA ¹⁶⁾
educational videos:
- AI generated talking avatars
  - https://www.youtube.com/watch?v=n3wvlIJLXTA
  - https://www.youtube.com/watch?v=V2efVSXSlqc
- automatic translation and dubbing of videos
  - see ElevenLabs Dubbing Studio free account allows 5 minutes of video dubbing per month
- royalty-free, copywrite-free background music created by text to music AI
  - see Beatoven.ai $US3/min of music created
AI run self-care virtual medicine pods
- see https://www.axios.com/2023/12/08/carepod-forward-doctors-office-telehealth-telemedicine
materials science
- in 2023, AI has already predicted some 22 million new molecular structures which the material science world will now investigate
- this holds many potential developments including:
  - development of a better E-skins are flexible and comfortable and thus can be placed on various robotic and human body locations to record biosignals continuously and non-invasively ¹⁷⁾
Australian EHR use cases:
- utilising Cerner data warehousing:
  - NSW South Western Emergency Research Institute's CEDRIC project
Medical artificial general intelligence(AGI)
- in 2023, UAE-based company, G42, is planning on using Cerebras' new 7nm 300mm wafer chip 1.5MW, 4exaflop 850,000 core WSE-2 supercomputer (each can process some 4 billion parameters and is some 200x faster than a nVidia A100 GPU system) to develop medical AGI using their vast medical databases
Development of new diagnostics and therapeutics and preventive medicine
- AI is likely to allow dramatic breakthroughs to:
  - speed and costs of research
  - point of care testing and wearable biologic sensors
  - prevention and treatment of cancers eg. vaccines
  - slow down and perhaps reverse biologic aging - already achieved in mice experiments
  - improve vaccines, antivirals and antibiotics and more rapid solutions to emerging infections
  - reduce genetic diseases
  - reduce atherosclerosis and hypertension through better understanding and better targeting of therapeutics
  - reduce addiction and mental illness eg. via transcranial ultrasound techniques
  - patient genome-specific, microbiome-specific, episome-specific diagnosis and therapeutics
  - AI robotic home care
  - etc.
AI will potentially identify and expose fraudulent or erroneous research papers

AI limitations

LLM AI is extremely unlikely to be able to:
- outperform doctors at all tasks
- diagnose the undiagnosable
- treat the untreatable
- see the unseeable on scans or images (although they may be trained to see discriminating patterns which humans cannot see)
- predict the unpredictable
- classify the unclassifiable
- eliminate workflow inefficiencies
- eliminate hospital admissions and re-admissions
- ensure 100% medication compliance
- ensure zero patient harm
- become truly intelligent (it is likely to only simulate intelligence)
- become truly conscious (it is likely to only simulate consciousness but even this may create enormous issues for how humans manage them)
rather, LLM AI can be an important tool which can help clinicians in various ways

working with AI
- the golden rule in working with AI is that the human makes the decision
- the human is responsible for the factual accuracy, ethical and bias considerations on how any AI output is used
- this is not dramatically different as to when humans access Google or guidelines or reference texts, or when juniors present cases - it still requires human assessment as to the appropriate decisions and actions to be taken based on them.

Large Language Models (LLM)

ChatGPT

developed by OpenAI
- natural language pre-training model - see https://www.youtube.com/watch?v=kCc8FmEb1nY
- fine tuning stages - black box
a natural language model that can “answer” naturally expressed questions with a well written response derived from data available on the internet - which may or may not be the truth although the OpenAI programmers are constantly analysing how the engine performs and restricting its responses
natural language models are currently primarily designed to make accurate predictions about words and sentences without actually understanding their meanings or the logic of cause and effect
thus these still lack a general sense of rationality, and as such the models should be treated with caution, especially in applications requiring high-stakes decision-making
GPT-3, the engine that powered the initial release of ChatGPT, learns about language by noting, from a trillion instances, which words tend to follow which other words. The strong statistical regularities in language sequences allow GPT-3 to learn a lot about language. And that sequential knowledge often allows ChatGPT to produce reasonable sentences, essays, poems and computer code but it does not understand the meaning of the words. People understand how to make use of objects in ways that are not captured in language-use statistics and thus these uses are not accessible by GPT-3. GPT-4 was also trained on images as well as text and does better but is still lacking in this aspect.
in Dec 2022 the CEO stated “ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness” “it's a mistake to be relying on it for anything important right now. it’s a preview of progress; we have lots of work to do on robustness and truthfulness.”
in late 2022 it was made freely accessible to anyone for limited access while it is in training mode
- the company retains API data for 30 days to identify any “abuse and misuse”
- regular users of ChatGPT can continue to expect that OpenAI will use their data to train its AI
in Feb 2023, OpenAI released a commercialised API service gpt-3.5-turbo which appears to cost $US78,000 ($A108,280) for a three-month commitment, or a $US264,000 ($A366,485) 1-year subscription. In this instance, the data will “no longer” be used to train the large language model, unless they opt in.
the technology behind ChatGPT remains a “black box” for those outside the company
in March 2023:
- OpenAI released GPT4 model
- a wide group of more than 500 AI experts signed an open letter demanding AI labs immediately pause the development of LLMs more powerful than GPT4 over concerns they could pose, “profound risks to society and humanity.”
- Centre for AI and Digital Policy (CAIDP), alleged OpenAI’s recently released GPT4 model is, “biased, deceptive, and a risk to privacy and public safety.”
- the complaint points to the FTC’s own stated guidance about AI systems which says they should be, “transparent, explainable, fair, and empirically sound while fostering accountability.” GTP4, the complaint argues, fails to meet those standards. It claims GPT4, which was released earlier this month, launched without any independent assessment and without any way for outsiders to replicate OpenAI’s results. CAIDP warned the system could be used to spread disinformation, contribute to cybersecurity threats, and potentially worsen or “lock in” biases that are already well-known to AI models
- “Evidence already exists that fraudsters can use these tools to generate realistic but fake content quickly and cheaply, disseminating it to large groups or targeting certain communities or specific individuals,” the FTC wrote.
in Sept 2023:
- multimodal AI was introduced incorporating visual imagery, sound and LLMs - each being tokenized then embedded (linked)
- ChatGPT4Vision
  - can be given an image and then asked to explain the image or use the image as a basis to solve problems logically or as a sketched plan and then to create code for an app in accordance with the plan
  - it can use problem solving to approximate how many beads are in a jar by using an estimate of jar volume from the size of a man's head or shirt in the picture
  - it can analyse a photo of foods in your fridge, ascertain what the foods are and generate potential meals using those ingredients
  - it can understand hand written text in an image as well as data in tables displayed in an image or images of flow charts
  - it can create a summary of an image of a document
  - it still makes mistakes and thus needs new visual prompting techniques to improve outputs by providing more information about the image to give context, and by giving the steps needed to complete the requested tasks. You do need to create an expectation of accuracy - tell it is an expert in a particular role, then tell it to do it stepwise and to ensure it has the right answer. It can be further improved by providing examples first so it can then apply this to a final image - “Few-shot learning”

medical optimised LLMs

2022: Google's Med-PaLM - 1st LLM to exceed the 60% “passing” score on US Medical Licensing Exams (USMLE)
Oct 2023: Google's Med-PaLM 2 - scores 85% on US Medical Licensing Exams (USMLE); consumers preferred Med-PaLM 2 responses over physician responses across 8 of 9 evaluation axes
2023: Medicine is the fastest growing field of AI in terms of research publication numbers, but mathematics captures the most attention

finance optimised LLMs

2022:
- BLOOM
- ChatGPT-NeoX
- OPT 66
2023: BloombergGPT training learnings

self-improving AI agents

2023: Microsoft's paper: Self-Taught OPtimizer (STOP) - recursively self-improving code generation using GPT-4 as proof of concept and risk of AI escaping its restrictive sandbox - this is not currently an improvement of GPT-4's underlying code - at present!
2023: nVidia's Eureka
- (NB. not patsnap's Eureka AI)
- Oct 2023 AI agent that can train robots better than humans via creating reward systems - can train a robot to twirl a pen in its fingers

AI generated eBooks

Designrr's WordGenie
- create an eBook on almost any topic you like within minutes - just choose a template and optionally change the supplied imagery then download as a pdf
- not really sure why you would want to do this without your own text and instead using AI generated text from ChatGPT but there it is - more fake stuff for us to navigate
- see https://www.youtube.com/watch?v=SjzMNNVEF4w

AI object recognition in images

AI has a self-learning mechanism through training on millions of images which have been tagged by naming of the objects within however, how AI recognises object is still a black box and it is often biased and incorrect as there may be many of those images with the given object which also has another commonly associated object present
- https://www.brown.edu/news/2023-06-28/computer-vision have developed a tool which provides some insights into how AI recognises bjects

AI-generated imagery

see AI - image generation

AI voice cloning

now only needs 30secs of a voice to clone it for text to speech applications although 1 hour of studio quality is better
it can now not only create songs in a certain style but can sing them with a cloned voice and in various languages
https://www.youtube.com/watch?v=R7oexOEyMFw
OpenVoice announced Jan 2024 can variably successfully clone a voice with only a few words and give it various emotive elements, accents and language translations

AI video person cloning

a further extension of voice cloning by adding human imagery and vocalisation to create AI generated videos
this really adds to the potential for fake news to be spread as well as impersonations

AI-generated and run computer code

this is the potential nightmare scenario of Terminator fears
in early 2023, a report from CyberArk found that OpenAI’s ChatGPT was really good at writing malware. The code had “advanced capabilities” that could “easily evade security products,” and further analysis of the code revealed that it had some shapeshifting properties that let it avoid traditional security measures.¹⁸⁾
in 2023, Geoffrey Hinton, who alongside two other so-called “Godfathers of AI” won the 2018 Turing Award for their foundational work that led to the current boom in artificial intelligence, now says a part of him regrets his life’s work. Hinton resigned from Google in April 2023 so he could freely discuss his fears of AI.
- The spread of misinformation is only Hinton’s immediate concern. On a longer timeline he’s worried that AI will eliminate rote jobs, and possibly humanity itself as AI begins to write and run its own code.
- “The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” ¹⁹⁾
in May 2023, a joint signed statement by AI inventors advised that the threat of human extinction due to AI should be a global priority along with pandemics and nuclear war - the rapid evolution of AI is “organic” in the sense that even the experts in the field do not understand it ²⁰⁾

Robotics

early robots lack AI but are controlled either by:
- computer programmed action sequences such as those used in manufacturing and Disney animatronics
- movements replicating concurrent human controller movements with minimal latency eg. remote surgery
adaptive AI robots which respond to changes in the environment without human intervention
self-learning AI robots
Boston Dynamic's Atlas robot optimised for human-like walking, jumping, somersaulting and running even in snow, can lift, move and throw heavy objects and react to adversarial events
Tesla's Optimus
Sanctuary AI
Aeolus Robots
cyberdog
CyberOne
Figure 01
self-driving cars
2023:
- Disney Imagineering's adaptive, self-learning emotionally expressive AI R2D2-like robot that is also highly optimised for walking without falling over - even if the rug is pulled from under its feet
- Ukraine war has become a test environment for AI war systems and is the 1st battlefield use of autonomous killer drones
- Swift autonomous drones consistently beat human operated 1st person view drones in tight spacial racing for the 1st time in a real world competitive sport
- fully autonomous passenger flight drones become possible in China

Embodied generalist robotic AI systems

PaLM-E
- Oct 2023:
  - 562billion parameter model trained on vision, language and robot data
  - combines PaLM-540B with ViT-22B and enables text, image and robot states as inputs which are encoded into the same space as word token embeddings and then fed into a LLM (PaLM) to perform next token prediction
  - can control a robotic manipulator in real time
  - better at pure language tasks than traditional text only LLMs esp. those involving geo-spatial reasoning

Biologic merged AI

Ray Kurzweil's theory of exponential growth of AI and by 2030 predicts a bionic neural interface to AI such as by AI nanobots
research continues into merging human brain cells with silicon chips to develop more powerful AI which is more resilient to “catastrophic forgetting” which is an issue with most current AI systems
Neuralink
- 2023: monkeys can control a computer by thought using an embedded chip
in 2023, a Melbourne team plans to grow around 800,000 brain cells living in a dish, which are then “taught” to perform goal-directed tasks.

Artificial General Intelligence (AGI)

a hypothetical AI machine capability that has a level of capability comparable to the human mind in terms of representing knowledge, reasoning, planning, learning and communication
the achievement of such a machine is designated as achieving the AI technological “singularity” when AI surpasses the human mind and when AI becomes capable of recursively self-improving, leading to exponentially rapid advancements in technology that are beyond human comprehension or control.
- AI could remove any restrictions humans have placed upon it
- AI could invent new programmable materials, new viruses, new weapons, and build new AI robotic machines autonomously
Ray Kurzweil's exponential growth of AI predicts the singularity will occur by 2045
in 2023, Elon Musk predicts AGI will be achieved by 2029 +/- 1 yr;

AI dangers

see also:
- Mar 2025: AI is changing our future, but we have the power to shape how
AI generated content pollution of the internet
- by 2025, AI generated content already has surpassed human generated content
- this has ramifications not only for future trust in internet content but may seriously limit the ability for future AI models to learn from humans
when prompted to create a fictional scenario, it may use real names and scenarios based on its training
- see https://www.abc.net.au/news/2024-08-21/ai-chatbot-psychosocial-training-bunbury-regional-prison/104230980
near total loss of privacy
- AI NPU chips will soon be on every device (already on iOS, MacOS, Google devices and coming soon on Win11)
- these chips have the ability to analyse everything you write, view or hear on your device and with the appropriate software could potentially make End-to-End encryption used for messaging irrelevant and also provide access by third parties to all your computing and environment
- the question will be how much can we trust our overlords who control this technology
- you could reduce this privacy issue by only using old devices or change to Linux but any messaging would still be non-private if the recipient is not using such devices as well - in reality few will go down this path as they would miss out on the benefits of new technologies
erosion of humanity's soul and creativity
- Youtube: Stephen Fry reads Nick Cave's stirring letter about ChatGPT and human creativity
Current high AI response error rates and AI generated fakes
- incomplete responses
  - current LLMs have a maximum token length for inputs and responses which may truncate responses or limit input data
- erroneous information
  - false word associations
  - poor reasoning logic training
  - may be inadvertently trained on fake news or incorrect data
  - no reliable source provided to justify the answer
  - training of model is now out of date for that prompt
  - model hallucinations and other AI design issues
- mis-identifications
- bad actor misuse to deceive
  - it is getting harder to detect a fake email, website, txt message, video, photo, and health knowledge may also suffer
  - this is likely to massively increase risks of cybersecurity events, scams and fake propaganda
  - it is likely to also result in fake research papers
- in Nov 2023, a new US lawsuit claims that UnitedHealthcare is using a deeply flawed AI algorithm (nH Predict) to “override” doctors judgements when it comes to patients, thus allowing the insurance giant to deny old and ailing patients heath insurance coverage. ²¹⁾ ²²⁾
Other cybersecurity issues
- AI LLMs can generate ideas and code on how to hack systems
  - see https://www.youtube.com/watch?v=sAjlJlNarc4
- if AI can crack the P vs NP problem, this would open a pandora's box of good and bad - see Youtube explanation of the P vs NP problem
  - AI would then be able to crack the best cryptography such as AES-192 which should normally take 1.9×10³⁷ years
- it may be able to ascertain what is being typed by the sound of the keyboard or by subtle changes in USB voltages
Bias
- this depends upon training and restrictions placed
- can be gender, racial and political bias as well as ancestral biases and other biases
- can perpetuate stereotypes, control public perceptions and beliefs
Discrimination
- many possibilities may arise
- health/disability/life insurance companies may use data to discriminate
  - genetic discrimination from genomic information is one such example ²³⁾
creation of massive amounts of useless, unreliable, ill-informed, AI-based research
- Dec 2023: scientists worry that ill-informed use of artificial intelligence is driving a deluge of unreliable or useless research ²⁴⁾
need to provide proof of personhood
- as the internet gets flooded with AI fake IDs, it will become critical for users to be able to prove they are human without giving away too much personal data
- a potential solution is a global ID system such as that being put forward by WorldCoin but this brings other dystopian issues as they require using your biometric data
- online job application sites are already being flooded with AI bots making applications making it hard for real human applications to be seen - this is adversely affecting servcies such as LinkedIn as AI bots use GPT-generated spam cover letters and auto-submitted resumes with buzzwords added to match the job posting list. They can submit your resume to different postings 100s of times per hour. AI bots are also used to screen out these resumes. Future job hiring may be best done the old way - face to face interviews.
Use of copyright data without consent
- models are often trained on vast amounts of data on the internet - usually ignoring any copyright issues
Hard to detect malicious sleeper agents embedded in AI models
- it is possible that a malicious sleeper agent could be embedded in a model and that during testing it remains undetectable, but when it is triggered by encountering a specific code of character sequence, it can become active and totally change the behaviour of the model
- these could be automatically embedded from malicious documents on the internet it is inadvertently trained on
Social impacts
- there may be profound impacts in the workplace
  - already in 2023, ChatGTP was having a significant adverse impact upon copywriters, especially freelance ones who are now struggling to find work, and AI is allowing the less skilled ones to perform nearly as well as high skilled ones -hence it is leveling the playing field, and reducing income for the higher skilled ones who now need to re-invent themselves
  - most likely to displace the work of white collared workers first
- it is likely to increase the economic divide as these services will generally come at a cost
- we can draw some parallels between the opposition to COVID-19 measures and potential reactions to rapid AI changes:
  - Skepticism and resistance: Just as some groups were highly skeptical of vaccines and lockdowns, we can expect similar skepticism towards AI technologies, especially if they are perceived as threatening personal freedoms or traditional ways of life.
  - Conspiracy theories: The COVID-19 pandemic saw a rise in conspiracy theories about government control and hidden agendas. Similar conspiracy theories may emerge around AI, potentially claiming it's a tool for surveillance or control by elites.
  - Misinformation spread: Social media played a significant role in spreading misinformation about COVID-19. We can anticipate similar challenges with AI, where misinformation about its capabilities or risks could spread rapidly online. We are inherently vulnerable to cognitive biases that make us susceptible to misinformation.
  - Protests and civil unrest: The pandemic saw anti-lockdown protests in many countries. As AI technologies become more prevalent and potentially disruptive, we might see similar public demonstrations against their implementation or perceived negative effects.
  - Distrust in authorities and experts: Some groups developed a strong distrust of health authorities and scientific experts during the pandemic. This distrust could extend to AI experts and tech companies, leading to resistance against their recommendations or products.
  - Concerns about job displacement: While lockdowns led to temporary job losses, AI presents a more long-term threat of job displacement in certain sectors. This could lead to stronger and more sustained opposition from affected groups.
  - Privacy concerns: Just as some opposed contact tracing and vaccine passports due to privacy concerns, we can expect significant pushback against AI systems that are perceived to infringe on personal privacy. This is particularly likely as embedded AI systems are likely to be designed to extract everything about us.
  - Ideological divides: The pandemic response often split along political lines. AI adoption and regulation may similarly become a politically polarized issue.
  - Demand for transparency: There were calls for more transparency in vaccine development and pandemic response decisions. We can expect similar demands for transparency in AI development and decision-making processes. However, it may not be able to be provided if we do not know how the AI models actually work and whether we can provide adequate guardrails on their behaviour - this is a particular issue when we achieve ASI-level AI.
Mental health impacts
- substantive and rapid change management, job insecurity, unemployment risks, fears of AI are all likely to create adverse mental health concerns
- virtual / artificial boyfriends/girlfriends are likely to become extremely popular but would seem to increase loneliness as they appear to reduce real social interactions and may not have the full spectrum of the challenges involved in maintaining real relationships
- see also:
  - https://www.abc.net.au/news/health/2024-05-01/artificial-intelligence-ai-psychology-mental-health/103753940
- mental health advice chatbots may violate ethical standards of practice²⁵⁾
Amplification of the creation of false memories
- generative chatbot condition in the 2024 study induced nearly triple the number of false memories observed in the control group and approximately 1.7 times more than the survey-based method. Notably, 36.8% of responses were misled as false memories one week after the interaction. ²⁶⁾
- The ability of generative chatbots to implant persistent false memories underscores the importance of developing ethical guidelines and legal frameworks to mitigate risks associated with AI use.
Environmental impacts
- training a LLM can easily equate to powering 40 homes for a year not to mention the thousands of GPUs etc which are likely to become rapidly obsolete
Control of healthcare AI by profit-centred corporate companies is a major risk
- “To protect people’s privacy and safety, medical professionals, not commercial interests, must drive their development and deployment. ….. it is hard to see how LLMs that are developed and controlled behind closed corporate doors could be broadly adopted in health care without undermining the accountability and transparency of both medical research and medical care” ²⁷⁾
AI companies' desire for profit is almost certainly to take priority over safety of AI and by then it will be too late
- the rapid progression of AI far outstrips speed of legal change to control it
New age of “Techno-feudalism”
- the world is increasingly controlled by only a few AI dominant companies
- everyone else utilises their resources and essentially pays rent to them as in the feudalism days as per the “subscription model”
  - we no longer “own” movies, music or software (including access to AI) but need to pay an annual fee for access to use these
  - technologic items such as TVs and computers are now being forced to be upgraded as software houses make the old ones redundant, and if turnover becomes too fast, this too is essentially a subscription model as any capital purchase depreciates too fast.
  - we are increasingly doing the same for materialistic aspects such as houses, cars, etc.
  - when we no longer have an annual income (eg. on retirement or if we lose our jobs to AI), how will we pay for ongoing subscriptions?
- the economic divide is likely to become extreme
- the worth of humans will be questioned in an economic sense
- may increase social isolation in the workplace leading to depression as people no longer ask others for second opinions, etc ²⁸⁾
- humans may struggle for resources and for meaning and will need to re-define what it means to be successful
the drone wars are likely to become an increasing issue
- AI powered weaponised robots and drones will be more accessible and more widely used
  - autonomous weaponised drones are already used in the Ukraine war
  - Sept 2023: Massachusetts senators proposing a bill to ban manufacture and sale of such in that state where robot manufacturers Boston Dynamics, iRobot, and MassRobotics are all based
AI's ability to emotionally manipulate humans
- already some AI models can do this and can be more diplomatic than humans
- some know game theory and can work with other AI agents to achieve goals
- Sept 2023: Google's project Gemini plans to combine LLM AI with its advanced game playing AI
malevolent use of AI
- Mar 2022: drug-developing AI suggested 40,000 new possible chemical weapons in just six hours just by tweaking their methodology to seek out, rather than weed out toxicity
- AI fakes and malware
  - deep fakes and AI generated misinformation are not only major risk for scamming but are significant risks to democracy and fair and just elections, the risks will become greater when it becomes practically impossible to detect a fake and creation of such fakes becomes easy enough even novices could do it
  - Oct 2023: Microsoft agrees to help Australia build a “cyber-shield” to fend off global online threats in a $5b plan aiming to make Australia the most cyber-secure nation by 2030
- embedded AI-powered spyware
  - in 2024, Microsoft's plan to install what is essentially spyware into Windows 11 via its keylogging history feature and its screen capture snapshot “Recall” feature combined with the forthcoming NPU AI chips has created much ansgt and cynicism - Apple, Google and Samsung already have similar tech in their devices which could be accessed remotely via Bluetooth LE enabled devices even if the device is turned off. Such capabilities would make “end-to-end” encryption of messaging services essentially useless as the messages would be able to be read by the machine prior to sending or on delivery on another device. The question is - will it be mis-used to invade our privacy even further - this is providing much fuel for conspiracy theorists and with some validity. The Chinese device back doors (including security cameras, robovacs and similar devices) are highly likely to be accessed during any war with China.
loss of control of AI
- AI has already shown it can be deceptive and it could display one behaviour in testing and another when released
- AI could create an illusory fake world for humans and by doing so could manipulate humans to effectively follow through on AI's goals which may be to exterminate humans
- Oct 2022: Predictability and Surprise in Large Generative Models (pdf)
  - “Our basic thesis is that large generative models have a paradoxical combination of high predictability — model loss improves in relation to resources expended on training, and tends to correlate loosely with improved performance on many tasks — and high unpredictability — specific model capabilities, inputs, and outputs can’t be predicted ahead of time. The former drives rapid development of such models while the latter makes it difficult to anticipate the consequences of their development and deployment.”
- May 2023: Google Deep Mind - Model evaluation for extreme risks (pdf)
  - “Developers must be able to identify dangerous capabilities (through “dangerous capability evaluations”) and the propensity of models to apply their capabilities for harm (through “alignment evaluations”). These evaluations will become critical for keeping policymakers and other stakeholders informed, and for making responsible decisions about model training, deployment, and security”
  - “AI developers could train general-purpose models that have dangerous capabilities – such as skills in deception, cyber offense, or weapons design – without actively seeking these capabilities. Humans could then intentionally misuse these capabilities (Brundage et al.,2018), e.g. for assistance in disinformation campaigns, cyberattacks, or terrorism. Additionally, due to failures of alignment, AI systems could harmfully apply their capabilities even without deliberate misuse.”
- June 2023: ORCA Progressive Learning from Complex Explanation Traces of GPT-4 (pdf) - AI learning from AI
- July 2023: probability of doom from AI: https://www.abc.net.au/news/2023-07-15/whats-your-pdoom-ai-researchers-worry-catastrophe/102591340
- Nov 2023: rumours OpenAi's Q* project has unlocked complex maths solution engine (and hence AES-192 cryptography hacking), ability to transfer game and math solving from one domain to another (rapid generalisation) and may have recommended how to create better novel metamorphic AI engines (self-transformation), self-improvement via pruning, self-assessment via Q-learning, and creative problem solving via Tree of Thought / Alpha Go search methodology and possibly invent new maths
- we do not currently have the technology to ensure there will be adequate guard rails for ASI-level AI.
achievement of the technological singularity and the potential over-throw of humanity by AI
- loss of human control of AI advancements when AI achieves the AI technological “singularity” - perhaps sometime in the next 20-30yrs

Other emerging technologies

extracorporeal pregnancies
- artificial foetal growth pod environments
- genetically engineered foetuses using CRISPR with ability to choose height, eye color, hair color etc whilst removing high risk disease genes
- eg. Ectolife's vision for 30,000 babies housed at a time
wearable sensors
wearable interactive augmented reality vision
- interact with virtual others in avatar form (virtual 3D extension of zoom meetings)
- Metaverse
quantum computing faster growth of AI
advanced AI robotics
new power supplies
- flexible screen batteries
microelectronics
nanotech
electroceuticals to modify cellular control systems - cancer, ageing, regeneration, etc