Loading weights:   0%|          | 0/473 [00:00<?, ?it/s]Loading weights:  43%|████▎     | 203/473 [00:00<00:00, 2029.40it/s]Loading weights: 100%|██████████| 473/473 [00:00<00:00, 4195.10it/s]
/root/public/unsloth/lib/python3.11/site-packages/peft/config.py:165: UserWarning: Unexpected keyword arguments ['alora_invocation_tokens', 'arrow_config', 'ensure_weight_tying', 'lora_ga_config', 'peft_version', 'qalora_group_size', 'target_parameters', 'use_bdlora', 'use_qalora'] for class LoraConfig, these are ignored. This probably means that you're loading a configuration file that was saved using a higher version of the library and additional parameters have been introduced since. It is highly recommended to upgrade the PEFT version before continuing (e.g. by running `pip install -U peft`).
  warnings.warn(
[transformers] `causal_conv1d_fn` is falling back to its reference PyTorch implementation because `causal_conv1d` is not installed. This is correct but much slower; install `causal_conv1d` for the optimized kernel.
[transformers] `chunk_gated_delta_rule` is falling back to its reference PyTorch implementation because `flash-linear-attention` is not installed. This is correct but much slower; install `flash-linear-attention` for the optimized kernel.
[transformers] `causal_conv1d_update` is falling back to its reference PyTorch implementation because `causal_conv1d` is not installed. This is correct but much slower; install `causal_conv1d` for the optimized kernel.
[transformers] `fused_recurrent_gated_delta_rule` is falling back to its reference PyTorch implementation because `flash-linear-attention` is not installed. This is correct but much slower; install `flash-linear-attention` for the optimized kernel.
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/old_component_comparison', 'inferred': 16, 'total': 1496, 'seconds': 3}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/old_component_comparison', 'inferred': 272, 'total': 1496, 'seconds': 30}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/old_component_comparison', 'inferred': 528, 'total': 1496, 'seconds': 60}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/old_component_comparison', 'inferred': 784, 'total': 1496, 'seconds': 90}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/old_component_comparison', 'inferred': 1040, 'total': 1496, 'seconds': 115}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/old_component_comparison', 'inferred': 1296, 'total': 1496, 'seconds': 142}
Loading weights:   0%|          | 0/473 [00:00<?, ?it/s]Loading weights:  42%|████▏     | 201/473 [00:00<00:00, 2009.29it/s]Loading weights: 100%|██████████| 473/473 [00:00<00:00, 3923.63it/s]
[transformers] `use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
{'phase': 'component_branch', 'epoch': 1, 'samples': 200, 'total': 4840, 'steps': 25, 'seconds': 24, 'loss': 0.7061}
{'phase': 'component_branch', 'epoch': 1, 'samples': 400, 'total': 4840, 'steps': 50, 'seconds': 48, 'loss': 0.1795}
{'phase': 'component_branch', 'epoch': 1, 'samples': 600, 'total': 4840, 'steps': 75, 'seconds': 73, 'loss': 0.1361}
{'phase': 'component_branch', 'epoch': 1, 'samples': 800, 'total': 4840, 'steps': 100, 'seconds': 97, 'loss': 0.1133}
{'phase': 'component_branch', 'epoch': 1, 'samples': 1000, 'total': 4840, 'steps': 125, 'seconds': 121, 'loss': 0.106}
{'phase': 'component_branch', 'epoch': 1, 'samples': 1200, 'total': 4840, 'steps': 150, 'seconds': 145, 'loss': 0.0957}
{'phase': 'component_branch', 'epoch': 1, 'samples': 1400, 'total': 4840, 'steps': 175, 'seconds': 169, 'loss': 0.0712}
{'phase': 'component_branch', 'epoch': 1, 'samples': 1600, 'total': 4840, 'steps': 200, 'seconds': 194, 'loss': 0.0761}
{'phase': 'component_branch', 'epoch': 1, 'samples': 1800, 'total': 4840, 'steps': 225, 'seconds': 218, 'loss': 0.0769}
{'phase': 'component_branch', 'epoch': 1, 'samples': 2000, 'total': 4840, 'steps': 250, 'seconds': 243, 'loss': 0.0681}
{'phase': 'component_branch', 'epoch': 1, 'samples': 2200, 'total': 4840, 'steps': 275, 'seconds': 267, 'loss': 0.0654}
{'phase': 'component_branch', 'epoch': 1, 'samples': 2400, 'total': 4840, 'steps': 300, 'seconds': 291, 'loss': 0.0535}
{'phase': 'component_branch', 'epoch': 1, 'samples': 2600, 'total': 4840, 'steps': 325, 'seconds': 315, 'loss': 0.0545}
{'phase': 'component_branch', 'epoch': 1, 'samples': 2800, 'total': 4840, 'steps': 350, 'seconds': 339, 'loss': 0.0387}
{'phase': 'component_branch', 'epoch': 1, 'samples': 3000, 'total': 4840, 'steps': 375, 'seconds': 364, 'loss': 0.0397}
{'phase': 'component_branch', 'epoch': 1, 'samples': 3200, 'total': 4840, 'steps': 400, 'seconds': 388, 'loss': 0.0421}
{'phase': 'component_branch', 'epoch': 1, 'samples': 3400, 'total': 4840, 'steps': 425, 'seconds': 412, 'loss': 0.044}
{'phase': 'component_branch', 'epoch': 1, 'samples': 3600, 'total': 4840, 'steps': 450, 'seconds': 436, 'loss': 0.0407}
{'phase': 'component_branch', 'epoch': 1, 'samples': 3800, 'total': 4840, 'steps': 475, 'seconds': 461, 'loss': 0.0316}
{'phase': 'component_branch', 'epoch': 1, 'samples': 4000, 'total': 4840, 'steps': 500, 'seconds': 485, 'loss': 0.0419}
{'phase': 'component_branch', 'epoch': 1, 'samples': 4200, 'total': 4840, 'steps': 525, 'seconds': 510, 'loss': 0.0372}
{'phase': 'component_branch', 'epoch': 1, 'samples': 4400, 'total': 4840, 'steps': 550, 'seconds': 534, 'loss': 0.0322}
{'phase': 'component_branch', 'epoch': 1, 'samples': 4600, 'total': 4840, 'steps': 575, 'seconds': 558, 'loss': 0.0322}
{'phase': 'component_branch', 'epoch': 1, 'samples': 4800, 'total': 4840, 'steps': 600, 'seconds': 582, 'loss': 0.0322}
{'phase': 'component_branch', 'epoch': 2, 'samples': 200, 'total': 4840, 'steps': 630, 'seconds': 612, 'loss': 0.026}
{'phase': 'component_branch', 'epoch': 2, 'samples': 400, 'total': 4840, 'steps': 655, 'seconds': 636, 'loss': 0.0297}
{'phase': 'component_branch', 'epoch': 2, 'samples': 600, 'total': 4840, 'steps': 680, 'seconds': 660, 'loss': 0.0271}
{'phase': 'component_branch', 'epoch': 2, 'samples': 800, 'total': 4840, 'steps': 705, 'seconds': 684, 'loss': 0.0224}
{'phase': 'component_branch', 'epoch': 2, 'samples': 1000, 'total': 4840, 'steps': 730, 'seconds': 708, 'loss': 0.0258}
{'phase': 'component_branch', 'epoch': 2, 'samples': 1200, 'total': 4840, 'steps': 755, 'seconds': 733, 'loss': 0.029}
{'phase': 'component_branch', 'epoch': 2, 'samples': 1400, 'total': 4840, 'steps': 780, 'seconds': 757, 'loss': 0.0282}
{'phase': 'component_branch', 'epoch': 2, 'samples': 1600, 'total': 4840, 'steps': 805, 'seconds': 781, 'loss': 0.0168}
{'phase': 'component_branch', 'epoch': 2, 'samples': 1800, 'total': 4840, 'steps': 830, 'seconds': 806, 'loss': 0.0184}
{'phase': 'component_branch', 'epoch': 2, 'samples': 2000, 'total': 4840, 'steps': 855, 'seconds': 830, 'loss': 0.0187}
{'phase': 'component_branch', 'epoch': 2, 'samples': 2200, 'total': 4840, 'steps': 880, 'seconds': 854, 'loss': 0.0174}
{'phase': 'component_branch', 'epoch': 2, 'samples': 2400, 'total': 4840, 'steps': 905, 'seconds': 878, 'loss': 0.0273}
{'phase': 'component_branch', 'epoch': 2, 'samples': 2600, 'total': 4840, 'steps': 930, 'seconds': 903, 'loss': 0.019}
{'phase': 'component_branch', 'epoch': 2, 'samples': 2800, 'total': 4840, 'steps': 955, 'seconds': 927, 'loss': 0.0234}
{'phase': 'component_branch', 'epoch': 2, 'samples': 3000, 'total': 4840, 'steps': 980, 'seconds': 951, 'loss': 0.015}
{'phase': 'component_branch', 'epoch': 2, 'samples': 3200, 'total': 4840, 'steps': 1005, 'seconds': 976, 'loss': 0.0172}
{'phase': 'component_branch', 'epoch': 2, 'samples': 3400, 'total': 4840, 'steps': 1030, 'seconds': 1001, 'loss': 0.0284}
{'phase': 'component_branch', 'epoch': 2, 'samples': 3600, 'total': 4840, 'steps': 1055, 'seconds': 1025, 'loss': 0.0199}
{'phase': 'component_branch', 'epoch': 2, 'samples': 3800, 'total': 4840, 'steps': 1080, 'seconds': 1049, 'loss': 0.0273}
{'phase': 'component_branch', 'epoch': 2, 'samples': 4000, 'total': 4840, 'steps': 1105, 'seconds': 1074, 'loss': 0.0185}
{'phase': 'component_branch', 'epoch': 2, 'samples': 4200, 'total': 4840, 'steps': 1130, 'seconds': 1099, 'loss': 0.0203}
{'phase': 'component_branch', 'epoch': 2, 'samples': 4400, 'total': 4840, 'steps': 1155, 'seconds': 1123, 'loss': 0.0162}
{'phase': 'component_branch', 'epoch': 2, 'samples': 4600, 'total': 4840, 'steps': 1180, 'seconds': 1148, 'loss': 0.0115}
{'phase': 'component_branch', 'epoch': 2, 'samples': 4800, 'total': 4840, 'steps': 1205, 'seconds': 1173, 'loss': 0.0185}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/component_branch', 'inferred': 16, 'total': 1496, 'seconds': 2}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/component_branch', 'inferred': 272, 'total': 1496, 'seconds': 41}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/component_branch', 'inferred': 528, 'total': 1496, 'seconds': 83}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/component_branch', 'inferred': 784, 'total': 1496, 'seconds': 122}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/component_branch', 'inferred': 1040, 'total': 1496, 'seconds': 166}
{'evaluation': '/root/private/project/pallet_detection_v2/outputs/pallet_description_round4_training_v2/vlm_run_v1/component_branch', 'inferred': 1296, 'total': 1496, 'seconds': 205}
{'phase': 'round4_vlm_holdout', 'targets': 1496, 'strict_complete_JSON_valid': 175, 'strict_complete_JSON_fraction': 0.11697860962566844, 'attribute_exact': {'支腿': {'correct': 162, 'labelled': 1496, 'accuracy': 0.10828877005347594}, '横梁': {'correct': 162, 'labelled': 1496, 'accuracy': 0.10828877005347594}, '材质': {'correct': 165, 'labelled': 1490, 'accuracy': 0.11073825503355705}}, 'token_limit_hits': 0, 'human_observation_reference_targets': 272, 'observation_correctness_not_inferred_from_JSON_validity': True, 'no_normal_negative_labels_available': True, 'semantic_observation_generalization_certified': False}
Loading weights:   0%|          | 0/473 [00:00<?, ?it/s]Loading weights:  48%|████▊     | 227/473 [00:00<00:00, 2269.52it/s]Loading weights: 100%|██████████| 473/473 [00:00<00:00, 3746.53it/s]
{'phase': 'full_pool', 'epoch': 1, 'samples': 200, 'total': 6336, 'steps': 25, 'seconds': 24, 'loss': 0.1988}
{'phase': 'full_pool', 'epoch': 1, 'samples': 400, 'total': 6336, 'steps': 50, 'seconds': 49, 'loss': 0.0907}
{'phase': 'full_pool', 'epoch': 1, 'samples': 600, 'total': 6336, 'steps': 75, 'seconds': 73, 'loss': 0.0652}
{'phase': 'full_pool', 'epoch': 1, 'samples': 800, 'total': 6336, 'steps': 100, 'seconds': 98, 'loss': 0.0671}
{'phase': 'full_pool', 'epoch': 1, 'samples': 1000, 'total': 6336, 'steps': 125, 'seconds': 122, 'loss': 0.0461}
{'phase': 'full_pool', 'epoch': 1, 'samples': 1200, 'total': 6336, 'steps': 150, 'seconds': 147, 'loss': 0.0397}
{'phase': 'full_pool', 'epoch': 1, 'samples': 1400, 'total': 6336, 'steps': 175, 'seconds': 171, 'loss': 0.0391}
{'phase': 'full_pool', 'epoch': 1, 'samples': 1600, 'total': 6336, 'steps': 200, 'seconds': 195, 'loss': 0.0385}
{'phase': 'full_pool', 'epoch': 1, 'samples': 1800, 'total': 6336, 'steps': 225, 'seconds': 220, 'loss': 0.0466}
{'phase': 'full_pool', 'epoch': 1, 'samples': 2000, 'total': 6336, 'steps': 250, 'seconds': 245, 'loss': 0.0412}
{'phase': 'full_pool', 'epoch': 1, 'samples': 2200, 'total': 6336, 'steps': 275, 'seconds': 269, 'loss': 0.0431}
{'phase': 'full_pool', 'epoch': 1, 'samples': 2400, 'total': 6336, 'steps': 300, 'seconds': 294, 'loss': 0.0354}
{'phase': 'full_pool', 'epoch': 1, 'samples': 2600, 'total': 6336, 'steps': 325, 'seconds': 318, 'loss': 0.0293}
{'phase': 'full_pool', 'epoch': 1, 'samples': 2800, 'total': 6336, 'steps': 350, 'seconds': 342, 'loss': 0.0352}
{'phase': 'full_pool', 'epoch': 1, 'samples': 3000, 'total': 6336, 'steps': 375, 'seconds': 367, 'loss': 0.0457}
{'phase': 'full_pool', 'epoch': 1, 'samples': 3200, 'total': 6336, 'steps': 400, 'seconds': 392, 'loss': 0.045}
{'phase': 'full_pool', 'epoch': 1, 'samples': 3400, 'total': 6336, 'steps': 425, 'seconds': 416, 'loss': 0.0337}
{'phase': 'full_pool', 'epoch': 1, 'samples': 3600, 'total': 6336, 'steps': 450, 'seconds': 440, 'loss': 0.0308}
{'phase': 'full_pool', 'epoch': 1, 'samples': 3800, 'total': 6336, 'steps': 475, 'seconds': 465, 'loss': 0.0227}
{'phase': 'full_pool', 'epoch': 1, 'samples': 4000, 'total': 6336, 'steps': 500, 'seconds': 489, 'loss': 0.0235}
{'phase': 'full_pool', 'epoch': 1, 'samples': 4200, 'total': 6336, 'steps': 525, 'seconds': 514, 'loss': 0.0243}
{'phase': 'full_pool', 'epoch': 1, 'samples': 4400, 'total': 6336, 'steps': 550, 'seconds': 538, 'loss': 0.0244}
{'phase': 'full_pool', 'epoch': 1, 'samples': 4600, 'total': 6336, 'steps': 575, 'seconds': 562, 'loss': 0.0367}
{'phase': 'full_pool', 'epoch': 1, 'samples': 4800, 'total': 6336, 'steps': 600, 'seconds': 587, 'loss': 0.034}
{'phase': 'full_pool', 'epoch': 1, 'samples': 5000, 'total': 6336, 'steps': 625, 'seconds': 612, 'loss': 0.0236}
{'phase': 'full_pool', 'epoch': 1, 'samples': 5200, 'total': 6336, 'steps': 650, 'seconds': 636, 'loss': 0.0264}
{'phase': 'full_pool', 'epoch': 1, 'samples': 5400, 'total': 6336, 'steps': 675, 'seconds': 660, 'loss': 0.0286}
{'phase': 'full_pool', 'epoch': 1, 'samples': 5600, 'total': 6336, 'steps': 700, 'seconds': 684, 'loss': 0.0195}
{'phase': 'full_pool', 'epoch': 1, 'samples': 5800, 'total': 6336, 'steps': 725, 'seconds': 709, 'loss': 0.0285}
{'phase': 'full_pool', 'epoch': 1, 'samples': 6000, 'total': 6336, 'steps': 750, 'seconds': 733, 'loss': 0.0267}
{'phase': 'full_pool', 'epoch': 1, 'samples': 6200, 'total': 6336, 'steps': 775, 'seconds': 758, 'loss': 0.0294}
{'phase': 'full_pool', 'epoch': 2, 'samples': 200, 'total': 6336, 'steps': 817, 'seconds': 799, 'loss': 0.0216}
{'phase': 'full_pool', 'epoch': 2, 'samples': 400, 'total': 6336, 'steps': 842, 'seconds': 823, 'loss': 0.0211}
{'phase': 'full_pool', 'epoch': 2, 'samples': 600, 'total': 6336, 'steps': 867, 'seconds': 848, 'loss': 0.0206}
{'phase': 'full_pool', 'epoch': 2, 'samples': 800, 'total': 6336, 'steps': 892, 'seconds': 872, 'loss': 0.022}
{'phase': 'full_pool', 'epoch': 2, 'samples': 1000, 'total': 6336, 'steps': 917, 'seconds': 897, 'loss': 0.0143}
{'phase': 'full_pool', 'epoch': 2, 'samples': 1200, 'total': 6336, 'steps': 942, 'seconds': 921, 'loss': 0.0177}
{'phase': 'full_pool', 'epoch': 2, 'samples': 1400, 'total': 6336, 'steps': 967, 'seconds': 946, 'loss': 0.014}
{'phase': 'full_pool', 'epoch': 2, 'samples': 1600, 'total': 6336, 'steps': 992, 'seconds': 970, 'loss': 0.0192}
{'phase': 'full_pool', 'epoch': 2, 'samples': 1800, 'total': 6336, 'steps': 1017, 'seconds': 995, 'loss': 0.0203}
{'phase': 'full_pool', 'epoch': 2, 'samples': 2000, 'total': 6336, 'steps': 1042, 'seconds': 1019, 'loss': 0.0179}
{'phase': 'full_pool', 'epoch': 2, 'samples': 2200, 'total': 6336, 'steps': 1067, 'seconds': 1044, 'loss': 0.0185}
{'phase': 'full_pool', 'epoch': 2, 'samples': 2400, 'total': 6336, 'steps': 1092, 'seconds': 1068, 'loss': 0.0174}
{'phase': 'full_pool', 'epoch': 2, 'samples': 2600, 'total': 6336, 'steps': 1117, 'seconds': 1093, 'loss': 0.0243}
{'phase': 'full_pool', 'epoch': 2, 'samples': 2800, 'total': 6336, 'steps': 1142, 'seconds': 1117, 'loss': 0.0151}
{'phase': 'full_pool', 'epoch': 2, 'samples': 3000, 'total': 6336, 'steps': 1167, 'seconds': 1142, 'loss': 0.0133}
{'phase': 'full_pool', 'epoch': 2, 'samples': 3200, 'total': 6336, 'steps': 1192, 'seconds': 1166, 'loss': 0.0167}
{'phase': 'full_pool', 'epoch': 2, 'samples': 3400, 'total': 6336, 'steps': 1217, 'seconds': 1191, 'loss': 0.0197}
{'phase': 'full_pool', 'epoch': 2, 'samples': 3600, 'total': 6336, 'steps': 1242, 'seconds': 1215, 'loss': 0.0204}
{'phase': 'full_pool', 'epoch': 2, 'samples': 3800, 'total': 6336, 'steps': 1267, 'seconds': 1239, 'loss': 0.024}
{'phase': 'full_pool', 'epoch': 2, 'samples': 4000, 'total': 6336, 'steps': 1292, 'seconds': 1264, 'loss': 0.0171}
{'phase': 'full_pool', 'epoch': 2, 'samples': 4200, 'total': 6336, 'steps': 1317, 'seconds': 1289, 'loss': 0.0157}
{'phase': 'full_pool', 'epoch': 2, 'samples': 4400, 'total': 6336, 'steps': 1342, 'seconds': 1314, 'loss': 0.0188}
{'phase': 'full_pool', 'epoch': 2, 'samples': 4600, 'total': 6336, 'steps': 1367, 'seconds': 1338, 'loss': 0.0166}
{'phase': 'full_pool', 'epoch': 2, 'samples': 4800, 'total': 6336, 'steps': 1392, 'seconds': 1362, 'loss': 0.0212}
{'phase': 'full_pool', 'epoch': 2, 'samples': 5000, 'total': 6336, 'steps': 1417, 'seconds': 1387, 'loss': 0.0171}
{'phase': 'full_pool', 'epoch': 2, 'samples': 5200, 'total': 6336, 'steps': 1442, 'seconds': 1411, 'loss': 0.0141}
{'phase': 'full_pool', 'epoch': 2, 'samples': 5400, 'total': 6336, 'steps': 1467, 'seconds': 1436, 'loss': 0.012}
{'phase': 'full_pool', 'epoch': 2, 'samples': 5600, 'total': 6336, 'steps': 1492, 'seconds': 1461, 'loss': 0.014}
{'phase': 'full_pool', 'epoch': 2, 'samples': 5800, 'total': 6336, 'steps': 1517, 'seconds': 1485, 'loss': 0.0214}
{'phase': 'full_pool', 'epoch': 2, 'samples': 6000, 'total': 6336, 'steps': 1542, 'seconds': 1510, 'loss': 0.0209}
{'phase': 'full_pool', 'epoch': 2, 'samples': 6200, 'total': 6336, 'steps': 1567, 'seconds': 1534, 'loss': 0.017}
