Model Card: Qwen2-math-Adv-ULtra-UK

Model Description

Qwen2-math-Adv-ULtra-UK is a highly specialized fine-tuned large language model designed for solving complex mathematical problems. It is built upon the ujjwal52/Qwen2-math-Adv-Ultra-UK base model, further enhancing its capabilities for precise and robust mathematical reasoning, particularly in advanced and challenging scenarios.

This model was fine-tuned using the ujjwal52/Maths-complex-200-COT-Uk dataset, which provides a collection of complex mathematical problems with Chain-of-Thought (COT) explanations. This training methodology enables the model to not only provide correct answers but also to articulate the reasoning process, making it highly suitable for educational applications, research, and any task requiring interpretable mathematical solutions.

Why Choose This Model? (What it does great & why it's better)

1. Advanced Mathematical Reasoning: Qwen2-math-Adv-ULtra-UK excels in handling complex and nuanced mathematical queries. Its specialized fine-tuning on a dataset of challenging problems ensures robust performance across a range of advanced mathematical concepts.

2. Superior Chain-of-Thought (COT) Generation: Thanks to its training on the COT-enabled dataset, this model can generate detailed, logical, step-by-step solutions. This is invaluable for users who need to understand how an answer was derived, fostering deeper understanding and trust in its results.

3. Efficiency with QLoRA: Fine-tuned using QLoRA with 4-bit quantization, this model achieves impressive performance while maintaining a relatively small memory footprint. This makes it accessible for deployment in environments with limited resources without sacrificing mathematical prowess.

4. Versatile Applications: Ideal for automated math problem solvers, intelligent tutoring systems, content generation for advanced math education, and assisting researchers with complex calculations requiring detailed explanations.

Training Details

Base Model

• ujjwal52/Qwen2-math-Adv-Ultra-UK

Dataset

• ujjwal52/Maths-complex-200-COT-Uk: A specialized dataset containing complex mathematical problems with detailed Chain-of-Thought (COT) explanations.

How to Use

This model can be easily loaded and used for text generation tasks, particularly for complex mathematical problem-solving. Here's how you can use it with the transformers library:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Define the model ID on Hugging Face
model_id = "ujjwal52/Qwen2-math-Adv-ULtra-UK"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

# Load model (make sure to specify the correct dtype and device_map for your setup)
# For the merged model, float16 is usually appropriate.
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,  # Use the dtype the merged model was saved in
    device_map="auto",           # Automatically maps model to available devices (e.g., GPU)
)

# Create a text generation pipeline
prompt = "A circle is inscribed in a square with side length $s$. Inside this circle, a smaller square is inscribed such that its vertices touch the circle. A line segment is drawn from the center of the circle to one vertex of the inner square, creating a right triangle with the center of the circle, the vertex of the inner square, and the point where the radius intersects the edge of the inner square. If the area of the outer square is $A$, find the sum of the squares of the lengths of the sides of the triangle. Express your answer in terms of $A$."
pipe = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=1500)

# Format the prompt according to the Qwen2 chat template
chat_template_prompt = f"<s>[INST] {prompt} [/INST]"

# Generate text
result = pipe(chat_template_prompt)
print(result[0]['generated_text'])

# Another example:
prompt_2 = "If $f(x) = \frac{3x-2}{x-2}$, what is the value of $f(-2) +f(-1)+f(0)$? Express your answer as a common fraction."
chat_template_prompt_2 = f"<s>[INST] {prompt_2} [/INST]"
result_2 = pipe(chat_template_prompt_2)
print(result_2[0]['generated_text'])

Limitations and Biases

• Mathematical Scope: While highly proficient in algebraic, functional, and complex problem-solving, the model's performance might vary in highly specialized or abstract mathematical fields (e.g., advanced theoretical physics, niche combinatorial problems) not directly covered by its training data.
• Problem Novelty: The model performs exceptionally well on problems similar in structure and complexity to its training data. Extremely novel or open-ended mathematical research questions might still require human-in-the-loop validation.
• Hallucination Risk: As with all large language models, there is an inherent risk of generating plausible-sounding but mathematically incorrect information. Critical mathematical results should always be independently verified.
• Language and Context Bias: The model's responses and problem interpretations are influenced by the language and framing present in the ujjwal52/Maths-complex-200-COT-Uk dataset. Users should be aware of potential biases in problem presentation or solution methodologies.

License

This model is based on ujjwal52/Qwen2-math-Adv-Ultra-UK, which in turn derives from Qwen/Qwen2-Math-1.5B. Please refer to the Qwen2-Math-1.5B model page for its exact licensing terms. The fine-tuned weights of Qwen2-math-Adv-ULtra-UK inherit the licensing terms of its base models.

Citation

If you utilize Qwen2-math-Adv-ULtra-UK in your research, applications, or publications, please consider citing the original Qwen2 work, the base model, and the dataset used for fine-tuning:

@misc{qwen2,
    title={Qwen2: A New Series of Large Language Models},
    author={The Qwen Team},
    year={2024},
    publisher={Hugging Face},
    url={https://huggingface.co/Qwen/Qwen2-Math-1.5B}
}

@misc{qwen2_math_adv_ultra_uk,
    title={Qwen2-math-Adv-Ultra-UK: An Advanced Mathematical Reasoning Model},
    author={ujjwal52},
    year={2024},
    publisher={Hugging Face},
    url={https://huggingface.co/ujjwal52/Qwen2-math-Adv-Ultra-UK}
}

@misc{maths_complex_200_cot_uk,
    title={Maths-complex-200-COT-Uk Dataset: A Collection for Chain-of-Thought Math Problem Solving},
    author={ujjwal52},
    year={2024},
    publisher={Hugging Face},
    url={https://huggingface.co/datasets/ujjwal52/Maths-complex-200-COT-Uk}
}