I have been experimenting with LLMs to see how well they navigate highly cross-referenced texts like the Bible. Standard models often hallucinate verses or lose historical context.
To try and fix this, I built a free app called Biblians (no ads, no paywalls). I built it specifically for people who have questions they might hesitate to ask in person, or who simply want a 1-click way to explain a verse.
While testing it, I discovered a fascinating denominational bias that is still lingering and changes depending entirely on the language you use:
- In English: It is Protestant-leaning. It praises Luther, saying things like, "Martin Luther sought to return the Church to the truth of God's Word."
- In Spanish, French, or Portuguese: It is Catholic-leaning. It condemns Luther's actions, stating: "...trajo confusión..." (...brought confusion...).
Has anyone else noticed how drastically the training data changes the core bias based on the language prompted?
I would love for this community to test the app, look for other linguistic biases, or just try to break the AI's logic.
You can experiment with it here: https://play.google.com/store/apps/details?id=com.biblians.app
Let me know what weird outputs you get!
[link] [comments]