Of course we can, we can simply speak out what steps are we doing to add numbers and it will correlate pretty much exactly to the actual actions done. Can tell to a similar query that first I add 5+6 and get 11, then I write down 1 and carry 1 to the next decimal space. Then I add 5+6 plus 1 I carried, then get 12 and write 121 as a final answer. That's the description of the real addition process I did in my head.
LLMs on the other hand will do some pretty unconventional stuff, like estimating the closest numbers not exactly matching, then evaluating probability spread of their low-precision sums, create some lookup tables, then do some dances combining this and that, making a higher precision estimations in a sequence, and eventually arriving to a single result. But no LLM will reply with these procedure steps to the simple "explain how you did it" query, simply because it is way less probable answer, therefore it won't be chosen.
LLMs on the other hand will do some pretty unconventional stuff, like estimating the closest numbers not exactly matching, then evaluating probability spread of their low-precision sums, create some lookup tables, then do some dances combining this and that, making a higher precision estimations in a sequence, and eventually arriving to a single result. But no LLM will reply with these procedure steps to the simple "explain how you did it" query, simply because it is way less probable answer, therefore it won't be chosen.