AI + LLM reading list for public servants

It’s clear to me we are in the midst of an AI hype-cycle and I’m skeptical of claims companies are making which directly serve their valuations. Throwing tech into our problems will not solve them.

But this isn’t to say that there isn’t something interesting going on, there is. Machine learning, data science techniques, large language models and all the other stuff being labelled as ‘AI’ are something public servants need to keep a watching brief on, and carefully experiment with. To that end I’ve been sharing some reading I’ve found helpful with colleagues, and I’ve brought all that into one place here.

Benedict Evans is, to my mind, one of the best commentators and analysts out there and his latest essay is extremely helpful for thinking things through: AI and the automation of work.

I really enjoyed a session on Large Language Models (LLMs) at our away day. So many great discussions delving into the philosophy of knowledge. Two recent articles fed into my thinking for the event. Firstly this one in MIT Technology Review exploring why asking an LLM to complete a test like the legal bar exam, is a flawed approach to understanding LLM’s capabilities. Secondly Simon Willinson posted the transcript of his recent sort of ‘year in LLMs’ talk which really helps to focus on the strengths and weaknesses of the current state of the art.

Innovation professor Ethan Mollick writes of his concerns about what the “write this for me” button powered by LLMs means for productivity and meaning. I don’t know if I agree but it’s thought provoking. I have heard that HMRC are already receiving LLM-created letters seeking to reduce people’s taxes with false understandings of tax law. But it still adds to their workload.

Here’s a report of a prototype powered by GPT-4 that lets you draw software that then gets coded for you. What the quality of that code is, I don’t know. But an interesting possible future for our work?

I’m sceptical of the productivity claims being made for Large Language Models (LLMs), but constantly searching for new analysis and insight into this field. Manchester University’s Professor Richard Jones has written a fascinating blog post on this topic, featuring a really interesting example on the use of AI in protein folding for pharmaceuticals. Definitely worth a read.

Large Language Models like GPT are all the hype rage at the moment, so if you want to really understand how they work then Stephen Wolfram has written an epic explanation. Or the simpler version I’ve seen online is “it’s just spicy autocorrect.” Whatever you think of the hype, here’s NCSC’s guidance on their use in government.

How the natural language interface of LLMs makes securing them so hard, enter the world of ‘prompt engineering’.

Max Roser is one of the lead members of “Our World in Data” a wonderful online data resource. I recently read his article from December 2020 Artificial intelligence is transforming our world — it is on all of us to make sure that it goes well and it’s as timely as ever.

Milton Mueller, a Professor at the Georgia Institute of Technology, The Basic Fallacy Underlying the AI Panic, is a punchy argument against fears that “AI will take over”.