Batteries should be at 12.6 - 12.7 after charging and sitting for 24 hours or so. One way batteries can be bad is that they have shorting cell, and no amount of fuse pulling, wire testing, parasitic loss search, etc will tell you about that. If you fully charge your batteries, leave them disconnected and then check 24 hours later to see if both are at the voltage same level OR just take them to a shop (Walmart for instance) and ask them to test the batteries. Most of the Walmart people know that they have to disconnect the batteries prior to testing, but some don't. You may have to 'remind' them about that, assuming you don't remove the batteries yourself. You should not be at 12.3 and dropping. Something's wrong.
I put new batteries in my truck prior to driving from Kansas to Arizona and noticed on the trip that the alternator charge rate was always about 15 1/2 volts, and I knew that shouldn't happen based on the conditions at the time. I thought the problem was with one of the battery ground cables and I made a repair for that. Still had the constant charging problem. I would charge the batteries (both connected in the truck) overnight and the truck would start very quickly, alternator charge rate when running was 14 or so, which is where it should have been. . So I thought I was done. The next day, the charge rate was back to 15 1/2. So charged both batteries again, and then I disconnected one battery, started the truck, charging rate was 14, then disconnected 1st battery and the 2nd battery was very low, something like 10 volts. Turned out to be a shorted cell. So when the batteries were together, the bad battery dragged down the good one. When separated, you could tell which was bad. Replaced that battery and everything is A-OK, charging rate 14 or so.