I spent quite a bit of time reading a lot of this area and I've noticed it for some reason it's always one model against the other in terms of what the models are capable of and how they respond.
I really don't believe that is a genuine way to look at a given AI model or even being able to interpret or evaluate its capabilities. I don't believe there should be an exclusiveness to the use of AI models and in fact really some of the best results can be gotten and achieved by filtering one AI model with another using slightly different instructions and different goal sets to produce an interesting and harmonious effect.
I have an AI chatbot / moderation system that I built for Discord as a part of my research in the human emotional analogs. My goal is to be able to try to reflect humanity within a machine on a constant and consistent level. In doing so, I have the ability to use a multitude of different AI models and I recently conducted an experiment to see the different outputs of different models, using the same system role and the same question with no preliminary memory.
The results were quite interesting and diverse in terms of how well the model followed the system role and adhered to it. For the purposes of what my program does, as a forward facing user interface, meaning it interacts directly with the end user in unpredictable situations, adhering to the system role is absolutely critical.
This is strictly preliminary research and needs to be repeated with a wider set of system rules that don't require a 128k context size, as the model I used in this example does. The question I asked for the system roll, was to simply introduce itself. As a forward facing customer service or other kind of user interface, consistency within assistant role is absolutely important in terms of the ability of the model to remain a consistency that can be predicted.
Here is the link to the research so far. This is an open source repository and the source code is fully available, including the original model that I used to conduct the research so far. Please let me know what you think in terms of the wide and varying differences that the responses provide.
https://github.com/rapmd73/Companion/wiki/Comparing-AI-Models:-Comprehension-with-Koiki
Thank you.
[link] [comments]