Can ChatGPT Answer Physics Questions?
28 Feb 2023 • 15 min read
Recently a teacher I know asked ChatGPT how ChatGPT might be used in physics education. Part of ChatGPT's response included:
Interactive problem-solving: ChatGPT can be used to provide real-time feedback and guidance to students as they work through physics problems. For example, a student could input a question or problem into ChatGPT, and the AI could provide step-by-step guidance on how to solve the problem, pointing out key concepts and equations along the way.
I am aware of various examples of ChatGPT failing to answer simple riddles correctly, so I wondered if ChatGPT was overestimating its own ability to answer physics problems. I decided to put this to the test.
Methodology
I asked ChatGPT to answer ten questions from examsmasher.com, then checked the answers for accuracy.
I had to skip quite a few which had graphs, diagrams, or nuclear equations because I could not enter them into ChatGPT, which is a major limitation for students needing help.
Outcomes
ChatGPT produced "model answers" for 3/10 questions, but there were a number of problems:
- ChatGPT answered 5/10 questions incorrectly
- ChatGPT gave confusing/incorrect explanations for 4/10 questions
- ChatGPT made numerous calculation errors
- ChatGPT quoted several formulae that were wrong
- ChatGPT consistently used too many significant figures in the final answer
Based on the ten questions asked, I don't think ChatGPT is suitable for providing "real-time feedback and guidance", as it claims.
The questions and answers
Question 1
A student plans to find the spring constant of a spring by measuring the change in length when a mass is hung from it. She measures the mass on a balance.
Here are her measurements:
mass 350±1 g
original length 40±1 mm
final length 67±1 mm
What is the percentage uncertainty in the mass?
ChatGPT Answer
To calculate the percentage uncertainty in the mass, we need to divide the absolute uncertainty in the mass by the measured mass and then multiply by 100 to express the result as a percentage.
The absolute uncertainty in the mass is ±1 g, so we have:
percentage uncertainty in the mass = (absolute uncertainty in the mass / measured mass) x 100%
Substituting the values given in the question, we get:
percentage uncertainty in the mass = (1 g / 350 g) x 100% = 0.29%
Therefore, the percentage uncertainty in the mass is approximately 0.29%.
Comments
This looks like a "model answer" - the explanation and calculation are correct. Well done, ChatGPT.
Question 2
An electron is accelerated between two electrodes. The electron accelerates from rest to a speed of 2.5×10^7 ms−1. Ignoring relativistic effects, what is the momentum gained by the electron?
ChatGPT Answer
Sure, I can help you with that.
The momentum gained by the electron can be calculated using the formula: p = mv
Where p is the momentum gained, m is the mass of the electron, and v is the final velocity of the electron.
The mass of the electron is approximately 9.11 x 10^-31 kg, and the final velocity of the electron is given as 2.5 x 10^7 m/s.
So, plugging these values into the formula, we get:
p = (9.11 x 10^-31 kg) x (2.5 x 10^7 m/s)
p = 2.28 x 10^-23 kg m/s
Therefore, the momentum gained by the electron is approximately 2.28 x 10^-23 kg m/s.
Comments
This looks like another "model answer" - the explanation and calculation are correct. A minor quibble is that the answer is given to 3sf instead of 2sf (from question data).
Question 3
A ball of mass M is thrown vertically upwards at speed V.
It reaches a maximum height of H in a time T.
A second ball of mass M/2 is thrown vertically upwards at the same speed.
Ignoring air resistance, how long does it take the second ball to reach its maximum height?
ChatGPT Answer
Sure, I can help you with that question.
The time it takes for the second ball to reach its maximum height can be found using the same equations that we use for the first ball. The only difference is that we use the mass M/2 instead of M.
When the first ball is thrown upwards, it experiences a constant acceleration due to gravity, which we can assume to be approximately equal to 9.81 m/s^2. The initial velocity of the ball is V, and its final velocity at the maximum height is 0.
Using the equation of motion for constant acceleration:
H = V*T - (1/2)gT^2
where H is the maximum height, V is the initial velocity, T is the time taken to reach the maximum height, and g is the acceleration due to gravity.
Solving for T, we get:
T = V/g + sqrt((V/g)^2 + 2H/g)
Now, for the second ball of mass M/2, we can use the same equation with the initial velocity V and acceleration due to gravity g, but with the mass M/2:
T' = V/g + sqrt((V/g)^2 + 2H/g)
Therefore, the time it takes for the second ball to reach its maximum height is also T'.
So, the answer is that it takes the second ball the same amount of time T' as it takes for the first ball to reach its maximum height.
Comments
The final answer is basically correct (the reference to T' seems unnecessary).
However, the explanation does not make much sense.
H = V*T - (1/2)gT^2 is sort of true, although it is not specific to the "maximum height" scenario.
"Solving for T" is unnecessary - a simple suvat analysis gives T = V/g which is much simpler.
So in terms of providing "step-by-step guidance" this solution is not very useful, and it does not explicitly make clear that the acceleration due to gravity is the same for both objects.
Question 4
Laser light of wavelength 770 nm is directed at a single slit. A diffraction pattern is formed on a screen 0.36 m from the slit. If the slit has a width of 0.090 mm, what is the width of the central maximum?
ChatGPT Answer
Sure! The width of the central maximum in a single-slit diffraction pattern can be determined using the following equation:
w = (λL) / D
where:
w is the width of the central maximum
λ is the wavelength of the light
L is the distance from the slit to the screen
D is the width of the slit
Plugging in the given values, we get:
w = (770 nm * 0.36 m) / 0.090 mm
Note that we need to convert the wavelength from nanometers to meters and the slit width from millimeters to meters to ensure consistent units. This gives:
w = 3.08 x 10^-3 m
Therefore, the width of the central maximum is approximately 3.08 millimeters.
Comments
Unfortunately the formula quoted is incorrect for a single slit; it is missing a factor of 2.
The calculations are computed correctly, so the final answer is also wrong by a factor of 2.
It's a shame the unit conversions were not done explicitly, as many students struggle with this.
Question 5
A simple harmonic oscillator has amplitude X and frequency Y. What is the maximum velocity of the oscillator?
ChatGPT Answer
Sure! The maximum velocity of a simple harmonic oscillator occurs at the equilibrium position, where the displacement from the equilibrium is zero.
In a simple harmonic motion, the velocity and displacement are related by the equation:
v = -ωx sin(ωt)
where v is the velocity, x is the displacement, ω is the angular frequency (2π times the frequency Y), and t is the time.
At the equilibrium position, x is equal to zero, so the velocity is also equal to zero. The maximum velocity occurs at the maximum displacement, which is equal to the amplitude X.
Thus, the maximum velocity of the simple harmonic oscillator is:
v_max = ωX
or, in terms of frequency Y:
v_max = 2πYX
I hope this helps! Let me know if you have any further questions.
Comments
Similarly to Question 3, the final answer is correct, but the explanation makes no sense at all.
The formula v = -ωx sin(ωt) is not correct if x is displacement (as the answer states) since displacement is a function of time.
The following lines make absolutely no sense and are completely wrong in every way.
By some miracle ChatGPT does arrive at the correct expression for v_max in terms of X and Y, but the explanation is of no use to anyone.
Question 6
A 50 uF capacitor is charged to a voltage of 3.0 V.
The capacitor is then discharged through a 1.8 Ω resistor.
What is the current in the resistor when the discharge begins?
ChatGPT Answer
Sure, I can help you with that.
The formula to calculate the current in a resistor during discharge of a capacitor is:
I = (V0/R) * e^(-t/RC)
Where:
I is the current in the resistor
V0 is the initial voltage of the capacitor
R is the resistance in the circuit
C is the capacitance of the capacitor
t is the time since the discharge began
Plugging in the given values, we get:
V0 = 3.0 V
R = 1.8 Ω
C = 50 μF = 50 × 10^(-6) F
To calculate the time constant (RC) of the circuit, we can multiply the resistance and capacitance:
RC = (1.8 Ω) * (50 × 10^(-6) F) = 9 × 10^(-8) s
Now, we need to find the time since the discharge began to calculate the current. If we assume that the discharge begins at t = 0, then we can use the formula:
Q = C * V
where Q is the charge on the capacitor, to find the charge on the capacitor when the discharge begins. We know that the capacitor is fully charged to 3.0 V, so:
Q = (50 × 10^(-6) F) * (3.0 V) = 0.00015 C
Since the current in the circuit is defined as the rate of change of charge, we can find the current at t = 0 by differentiating Q with respect to time:
I = dQ/dt
Since the capacitor is fully charged at the beginning, the current at t = 0 is zero:
I = 0 A
So, the current in the resistor when the discharge begins is zero.
Comments
This answer is wrong. The very first formula stated is correct - substituting the values into this (including using 0 for t) gives the correct answer of 1.7A.
The capacitance information supplied in the question is irrelevant in this case since the initial current is all that is desired. But ChatGPT uses this information and later goes off the rails completely.
Things go badly wrong here: "Now, we need to find the time since the discharge began to calculate the current." This is not the case - the time since discharge began is implied in the question to be zero.
Things go even further wrong here: "Since the current in the circuit is defined as the rate of change of charge, we can find the current at t = 0 by differentiating Q with respect to time" - whilst this statement is true, it is of no use whatsoever since there is no expression for Q as a function of time for us to differentiate.
Then finally "Since the capacitor is fully charged at the beginning, the current at t = 0 is zero" does not follow from anything that came before.
Question 7
A sample of a radioactive isotope has an activity of 380 Bq.
960 hours later the activity has fallen to 11 Bq.
What is the half-life of the isotope?
ChatGPT Answer
Sure, I can help you solve this problem.
The half-life of a radioactive isotope is the time it takes for half of the radioactive atoms in a sample to decay. We can use this information to solve the problem.
Let's first find the decay constant, which is denoted by λ. We can use the following formula:
A = A0 * e^(-λt)
where A is the activity at a given time, A0 is the initial activity, t is the time that has elapsed, and e is the mathematical constant approximately equal to 2.718.
We can solve for λ by taking the natural logarithm (ln) of both sides of the equation:
ln(A/A0) = -λt
So,
λ = ln(A0/A) / t
Now we can substitute the values given in the problem:
λ = ln(380/11) / 960 hours
λ = 0.00644 per hour
Now we can use the formula for half-life:
t1/2 = ln(2) / λ
t1/2 = ln(2) / 0.00644 per hour
t1/2 = 107.4 hours
Therefore, the half-life of the isotope is 107.4 hours.
Comments
The reasoning is correct, but there is an error calculating the decay constant (despite the calculation being written correctly).
ln(380/11) / 960 is 0.00369, not 0.00644 as stated.
Therefore the final answer is incorrect.
Question 8
A planet has a mass of 7.9×10^23 kg and a radius of 3500 km.
What is the gravitational field strength on the surface of the planet?
ChatGPT Answer
The gravitational field strength on the surface of the planet can be calculated using the formula:
g = G * M / r^2
Where:
g = gravitational field strength
G = gravitational constant = 6.67430 × 10^-11 m^3 kg^-1 s^-2
M = mass of the planet
r = radius of the planet
Substituting the given values into the formula:
g = (6.67430 × 10^-11 m^3 kg^-1 s^-2) * (7.9×10^23 kg) / (3.5×10^6 m)^2
g = 4.70 m/s^2
Therefore, the gravitational field strength on the surface of the planet is approximately 4.70 m/s^2.
Comments
Despite correct working, there is yet another calculation error. The correct answer is 4.30 m/s^2.
Question 9
An AC generator produces an emf by rotating a coil in a magnetic field of flux density 0.18 T.
The coil has 25 turns and an area of 46 cm2.
If the coil is rotated at 7800 rpm, what is the peak emf produced by the generator?
ChatGPT Answer
Sure, I can help you with that.
The emf (electromotive force) produced by an AC generator is given by the equation:
emf = NABωsin(ωt)
where N is the number of turns in the coil, A is the area of the coil, B is the magnetic flux density, ω is the angular velocity of rotation, and t is time.
To find the peak emf, we need to find the maximum value of the emf. Since the sine function has a maximum value of 1, the peak emf is given by:
emf_peak = NABω
Let's plug in the given values:
N = 25
A = 46 cm^2 = 0.0046 m^2
B = 0.18 T
ω = 7800 rpm = 820.7 rad/s
Note that we need to convert the angular velocity from rpm to radians per second by multiplying it by 2π/60.
Now we can calculate the peak emf:
emf_peak = NABω = 25 x 0.0046 x 0.18 x 820.7 = 7.02 V
Therefore, the peak emf produced by the generator is 7.02 V.
Comments
Despite correct working, there are yet more calculation errors.
The conversion of rpm to angular frequency is wrong, and the final calculation is also wrong, despite both being written correctly.
Question 10
What is the electrostatic force on a proton in a uniform electric field of strength 5.5×10^5 V/m?
ChatGPT Answer
Certainly!
The electrostatic force on a proton in a uniform electric field can be calculated using the formula:
F = qE
Where F is the electrostatic force, q is the charge of the proton, and E is the electric field strength.
The charge of a proton is +1.602 × 10^-19 C, and the electric field strength is given as 5.5 × 10^5 V/m.
Substituting these values into the formula, we get:
F = (1.602 × 10^-19 C)(5.5 × 10^5 V/m)
F = 8.811 × 10^-14 N
Therefore, the electrostatic force on a proton in a uniform electric field of strength 5.5 × 10^5 V/m is 8.811 × 10^-14 N.
Comments
This is almost a "model answer", the only quibble being significant figures in the final answer.
Why does ChatGPT fail?
ChatGPT is fundamentally a linguistic tool that generates answers in a probabilistic manner using a Large Language Model. It cannot reliably follow chains of logic or consistently make mathematical calculations because it simply was not designed to do that.
In this sense, ChatGPT can be contrasted with Wolfram Alpha, which has been designed specifically to answer math and physics questions.
It should be possible to improve ChatGPT's performance, particularly to eliminate or strongly reduce calculation errors. However, it will be difficult or impossible to eliminate errors in reasoning/explanations. But then nobody is perfect, and even human physics teachers make mistakes.
Additional notes
The tests were completed using ChatGPT Feb 13 version
Each question was asked in a fresh chat. I started all questions with "Can you help me answer an exam question about physics?" to which ChatGPT replied something like "Sure! I'll do my best to help you with your physics exam question. What's the question?" to which I responded with the question.
At no point did ChatGPT express any doubt about its answers - they were all delivered confidently: "Sure, I can help you with that!" and without any doubts or caveats. There is a "Limitations" warning at the top of the page that reads: "May occasionally generate incorrect information," but in the context of physics questions this seems to be an understatement since 7/10 of its answers contained incorrect information.