Thursday, July 06, 2023

Math hallucinations of Chat GPT

A client asked me yesterday to calculate the Net Present Value (NPV) of the alimony payments he owed to his wife. After reminding myself about the details of the NPV calculation, I realized he actually just needed a present value calculation rather then the more complex NPV one, since alimony payments are even and NPV is for uneven cashflows.

While I was trying to sort that stuff out my client had asked his question of Chat GPT, which had of course produced an answer. The problem is that the answer it produced, while sounding authoritative, actually made no sense whatsoever. I can't even figure out exactly what mistake it made to produce one component number of its bastard summation. What it did to was make me run the calculation first in Excel, then using my financial calculator, then using some online financial calculators, just to make sure that I wasn't screwed up in my thinking. And thus I squandered north of an hour of the work day, at the tail end of the day.

There were, no doubt, some beneficial sides to this whole interaction. I had to refamiliarize myself with some basic concepts and toolkits around the time value of money. I got this nice blog post. And my fear of the robots showing up to steal my livelihood, never a big concern, receded further into the rearview. But it was a pain in the ass.



*BTW, I put "Chat GPT math hallucination" into my preferred search engine and found this article which suggests that generative AIs are bullshitting rather than hallucinating. I think that's more or less right in this case. It just threw some math bullshit out there on the assumption that the reader was too stupid or lazy to do the math.

No comments: