This section is devoted to theoretical and practical aspects of Artificial Intelligence
1. Using Large Language Models (LLMs) such as ChatGPT and Claude to do serious mathematics gives rise to all sorts of problems. On the one hand these models can provide some astonishing insights even though they actually do not perform logical deductions like a theorem prover. Yet they can also generate some preposterous results. You have to be extremely vigilant with these models and if you let your guard down you can be caught out.
Being a long time Mathematica user I was interested in seeing how Notebook Assistant (NA) worked within Mathematica. It is essentially ChatGPT and is able to produce Mathematica code within the Mathematica environment. I tested it out on what I thought was a problem it could nail simply because the issue I put to it was so ubiquitous it would have been in the original training set and is referred to in many contexts. What I asked Notebook Assistant to do is to generate Mathematica code for a discrete Fast Fourier Transform to multiply two large numbers. Well it demonstrated that it was not up to it and it took a lot of work to get it to understand some Mathematica code that I knew worked. I then tested Claude and Deepseek on the same problem.
All the responses are set out and it is interesting to note that the free version of Deepseek actually produced an astonishing 82 page stream of consciousness.
While you can get some useful work out of these models you actually have to be relatively sophisticated to be aware of their limitations. It is a bit like giving a loaded AK 47 to a young child.
The full story can be found here
2. I recently published an article on how ChatGPT as it is employed in Mathematica's Notebook Assistant could not without much effort produce Mathematica code for a discrete Fast Fourier Transform: https://gotohaggstrom.com/Mathematicas%20Notebook%20Assistant%20(ChatGPT).pdf
I actually thought before the experiment that ChatGPT could produce the code simply because the concept is so ubiquitous and would have been in the original training.
I was wrong and a more recent test on something so preposterously banal makes me even more worried about the reliability of these models.
The article can be downloaded here
1. Using Large Language Models (LLMs) such as ChatGPT and Claude to do serious mathematics gives rise to all sorts of problems. On the one hand these models can provide some astonishing insights even though they actually do not perform logical deductions like a theorem prover. Yet they can also generate some preposterous results. You have to be extremely vigilant with these models and if you let your guard down you can be caught out.
Being a long time Mathematica user I was interested in seeing how Notebook Assistant (NA) worked within Mathematica. It is essentially ChatGPT and is able to produce Mathematica code within the Mathematica environment. I tested it out on what I thought was a problem it could nail simply because the issue I put to it was so ubiquitous it would have been in the original training set and is referred to in many contexts. What I asked Notebook Assistant to do is to generate Mathematica code for a discrete Fast Fourier Transform to multiply two large numbers. Well it demonstrated that it was not up to it and it took a lot of work to get it to understand some Mathematica code that I knew worked. I then tested Claude and Deepseek on the same problem.
All the responses are set out and it is interesting to note that the free version of Deepseek actually produced an astonishing 82 page stream of consciousness.
While you can get some useful work out of these models you actually have to be relatively sophisticated to be aware of their limitations. It is a bit like giving a loaded AK 47 to a young child.
The full story can be found here
2. I recently published an article on how ChatGPT as it is employed in Mathematica's Notebook Assistant could not without much effort produce Mathematica code for a discrete Fast Fourier Transform: https://gotohaggstrom.com/Mathematicas%20Notebook%20Assistant%20(ChatGPT).pdf
I actually thought before the experiment that ChatGPT could produce the code simply because the concept is so ubiquitous and would have been in the original training.
I was wrong and a more recent test on something so preposterously banal makes me even more worried about the reliability of these models.
The article can be downloaded here