Medical SLM Model Output based on Graph Dictionary, 85% to 100% token success, 0.002 loss, 1.01 perplexity and all of this based on only 500 PubMed dataset samples and 85% weight on graph dictionary vector embeddings, These are simply results of 20 epochs of MLM training next I will run a CLM training on these saved checkpoints my goal is 95% to 100% success ratio based on graph vector embeddings and a tinny sample set
2025-03-02 07:48:36,128 - INFO - Epoch 20/20, Batch 1/50, Loss: 0.0121, Perplexity: 1.0122
2025-03-02 07:49:19,040 - INFO - Epoch 20/20, Batch 10/50, Loss: 0.0062, Perplexity: 1.0062
2025-03-02 07:50:06,699 - INFO - Epoch 20/20, Batch 20/50, Loss: 0.0138, Perplexity: 1.0139
2025-03-02 07:50:54,655 - INFO - Epoch 20/20, Batch 30/50, Loss: 0.0200, Perplexity: 1.0202
2025-03-02 07:51:42,764 - INFO - Epoch 20/20, Batch 40/50, Loss: 0.0248, Perplexity: 1.0251
2025-03-02 07:52:30,767 - INFO - Epoch 20/20, Batch 50/50, Loss: 0.0121, Perplexity: 1.0122
2025-03-02 07:52:30,768 - INFO - Epoch 20 completed. Training - Average Loss: 0.0138, Average Perplexity: 1.0139
2025-03-02 07:52:52,360 - INFO - Validation - Perplexity: 1.3930
2025-03-02 07:52:54,095 - INFO -
--- Validation Examples - Full Sentence Prediction ---
2025-03-02 07:52:54,096 - INFO - Example 1:
2025-03-02 07:52:54,097 - INFO - Question: Prognosis of well differentiated small hepatocellular carcinoma--is well differentiated hepatocellular carcinoma clinically early cancer?
2025-03-02 07:52:54,098 - INFO - Original: W-d HCCs were clinically demonstrated not to be early cancer, because there was no significant diffe...
2025-03-02 07:52:54,099 - INFO - Predicted: w - d hccs were clinically demonstrated not to be early cancer, because there was no significant dif...
2025-03-02 07:52:54,100 - INFO - Token-level accuracy: 100.00%
2025-03-02 07:52:54,100 - INFO - --------------------------------------------------
2025-03-02 07:52:54,101 - INFO - Example 2:
2025-03-02 07:52:54,101 - INFO - Question: Profiling quality of care: Is there a role for peer review?
2025-03-02 07:52:54,102 - INFO - Original: For conditions with a well-developed quality of care evidence base, such as hypertension and diabete...
2025-03-02 07:52:54,103 - INFO - Predicted: for conditions with a well - developed quality of care evidence severe, such as hypertension and dia...
2025-03-02 07:52:54,104 - INFO - Token-level accuracy: 86.14%
2025-03-02 07:52:54,105 - INFO - --------------------------------------------------
2025-03-02 07:52:54,106 - INFO - Example 3:
2025-03-02 07:52:54,107 - INFO - Question: Does automatic transmission improve driving behavior in older drivers?
2025-03-02 07:52:54,107 - INFO - Original: Switching to automatic transmission may be recommended for older drivers as a means to maintain safe...
2025-03-02 07:52:54,108 - INFO - Predicted: ##ptic to supportedem may be recommended for olderaneous as a means to maintain safe driving and the...
2025-03-02 07:52:54,109 - INFO - Token-level accuracy: 82.14%
2025-03-02 07:52:54,109 - INFO - --------------------------------------------------
2025-03-02 07:52:54,110 - INFO - Example 4:
2025-03-02 07:52:54,111 - INFO - Question: Does angiotensin-converting enzyme-1 (ACE-1) gene polymorphism lead to chronic kidney disease among hypertensive patients?
2025-03-02 07:52:54,111 - INFO - Original: It is concluded that ACE-DD genotype may be a risk factor for the causation and development of chron...
2025-03-02 07:52:54,111 - INFO - Predicted: it is propose that ace - parallels genotype may be a risk factor for the ca thetion and of chronic k...
2025-03-02 07:52:54,113 - INFO - Token-level accuracy: 87.88%
2025-03-02 07:52:54,114 - INFO - --------------------------------------------------
2025-03-02 07:52:54,115 - INFO - Example 5:
2025-03-02 07:52:54,116 - INFO - Question: Can transcranial direct current stimulation be useful in differentiating unresponsive wakefulness syndrome from minimally conscious state patients?
2025-03-02 07:52:54,116 - INFO - Original: a-tDCS could be useful in identifying residual connectivity markers in clinically-defined UWS, who m...
2025-03-02 07:52:54,117 - INFO - Predicted: a - tdcs could be useful in identifying residualhip markers in clinically - defined uses, who may la...
2025-03-02 07:52:54,118 - INFO - Token-level accuracy: 89.74%
2025-03-02 07:52:54,119 - INFO - --------------------------------------------------
2025-03-02 07:52:54,120 - INFO - Example 6:
2025-03-02 07:52:54,121 - INFO - Question: Does timing of initial surfactant treatment make a difference in rates of chronic lung disease or mortality in premature infants?
2025-03-02 07:52:54,121 - INFO - Original: Early surfactant administration is associated with shorter duration of ventilation but does not appe...
2025-03-02 07:52:54,122 - INFO - Predicted: early cellactant administration is associated with shorter duration of ventilation but does not appe...
2025-03-02 07:52:54,123 - INFO - Token-level accuracy: 95.89%
2025-03-02 07:52:54,124 - INFO - --------------------------------------------------
2025-03-02 07:52:54,125 - INFO - Example 7:
2025-03-02 07:52:54,125 - INFO - Question: Department of Transportation vs self-reported data on motor vehicle collisions and driving convictions for stroke survivors: do they agree?
2025-03-02 07:52:54,126 - INFO - Original: In our population of stroke survivors, self-reports of motor vehicle collisions and driving convicti...
2025-03-02 07:52:54,126 - INFO - Predicted: in our population severe stroke survivors, self - difference of motor preparation focused and drivin...
2025-03-02 07:52:54,128 - INFO - Token-level accuracy: 84.31%
2025-03-02 07:52:54,128 - INFO - --------------------------------------------------
2025-03-02 07:52:54,129 - INFO - Example 8:
2025-03-02 07:52:54,130 - INFO - Question: Are performance measurement systems useful?
2025-03-02 07:52:54,130 - INFO - Original: This study contributes to the literature investigating the design and implementation of a non-financ...
2025-03-02 07:52:54,131 - INFO - Predicted: this study contributes to the literature italy the " and implementation of a non - financial measure...
2025-03-02 07:52:54,133 - INFO - Token-level accuracy: 86.21%
2025-03-02 07:52:54,133 - INFO - --------------------------------------------------
2025-03-02 07:52:54,161 - INFO - Training completed. Best perplexity achieved: 1.3339
[link] [comments]