The couple of times I've tried to fine-tune a text model I've consistently gotten way worse results (I'm not good at it), so I've just stopped trying...