Supervised Finetuning (SFT) can be a game changer for the humanities because it allows us to adapt powerful general-purpose models to specialized domains. Importantly, this approach doesn’t need massive amounts of data. Well-curated, small datasets can be used to tailor large open-weights models to particular historical periods, languages, or cultural contexts. However, significant errors remain, as knowledge from the model's original training data inevitably leaks into the results, and we need strategies to identify and reduce these types of errors. Part of the solution is purely technical and draws from available methods in the machine learning toolkit. For example, typical approaches for extracting more signal from limited data include hyperparameter optimization, active learning, and data augmentation. For reliability estimation, recent methods include metamorphic relation (MR) testing, which validates model outputs by checking whether they satisfy invariant properties when inputs are transformed. However, a narrow engineering perspective can lead to fundamental blind spots, as Lauren Klein, Joy Buolamwini, Kate Crawford, and others have demonstrated. Critical practices, from devising interactive artworks to closely reading datasets, should also inform the ways machine learning models are developed and evaluated. In this spirit, my research group designed a series of exploratory probes as an integral part of our model finetuning pipeline. In subsequent experiments, these probes helped us identify failure modes not surfaced by other methods, enabling the creation of better synthetic data, and the design of more robust tests. This approach led to measurable improvements in model performance and reliability. The main case studies in this talk come from research on Southeast Asian historical newspapers, but these methods can be generalized across humanities domains where cultural reliability is crucial.