Apertus-SEA-LION-v4-8B

Last updated: 2026-02-05

SEA-LION is a collection of Large Language Models (LLMs) which have been pretrained and instruct-tuned for the Southeast Asia (SEA) region.

Introduction

SEA-LION stands for Southeast Asian Languages In One Network.

Apertus-SEA-LION-v4-8B-IT is a 8-billion parameter model built upon the Apertus-8B-Instruct architecture. To ensure domain adaptation for the region, the model underwent rigorous post-training on a curated dataset of approximately 8.54 million instruction-text pairs.

This extensive post-training instills multilingual and multicultural fluency, covering key SEA languages such as Burmese, Malay, Tagalog and Tamil. The dataset also includes 347,000 tool-calling instruction-text pairs to impart these capabilities, in addition to linguistic fluency.

Apertus-SEA-LION-v4-8B-IT is designed as a fully open model; to align with this core philosophy, we have released the datasets used for post-training, as well as the evaluation codes and datasets used to evaluate the model.

These resources can be accessed via the link below.

• Open post-training datasets we used.

• SEA-HELM Evaluation codes and datasets

Model Details

Model Description

SEA-LION stands for Southeast Asian Languages In One Network.

We performed post-training in English and SEA languages on Apertus-8B-Instruct-2509, a decoder model using the Apertus architecture, and post-training to create Apertus-SEA-LION-v4-8B-IT.

For tokenization, the model employs the default tokenizer used in Apertus-8B-Instruct-2509.

• Developed by: AI Products Pillar, AI Singapore

• Funded by: Singapore NRF

• Shared by: AI Products Pillar, AI Singapore

• Model type: Decoder

• Context length: 65k

• Language(s): fine-tuned on English, Burmese, Tagalog, Malay and Tamil

• License: Apache-2.0

• Finetuned from model: Apertus-8B-Instruct

Model Sources

• Repository: https://huggingface.co/collections/aisingapore/sea-lion-v4

Uses

Out-of-Scope Use

The model has not been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claims, damages, or other liabilities arising from the use of the released weights and codes.

Bias, Risks, and Limitations

The models were not tested for robustness against adversarial prompting. It is important for users to be aware that our models exhibit certain limitations that warrant consideration. Like many LLMs, the models can hallucinate and occasionally generates irrelevant content, introducing fictional elements that are not grounded in the provided context. Users should also exercise caution in interpreting and validating the model's responses due to the potential inconsistencies.

How to Get Started with the Model

Use the code below to get started with the model with 🤗 Transformers libraries.

pip install transformers>=4.56.0

# The code is adopted from Apertus example
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "aisingapore/Apertus-SEA-LION-v4-8B-IT"
device = "cuda"  # for GPU usage or "cpu" for CPU usage

# load the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
).to(device)

# prepare the model input
prompt = "Explain the concept of 'Hari Raya Puasa' in simple terms."
messages_think = [
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages_think,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt", add_special_tokens=False).to(model.device)

# Generate the output
generated_ids = model.generate(**model_inputs, max_new_tokens=32768)

# Get and decode the output
output_ids = generated_ids[0][len(model_inputs.input_ids[0]) :]
print(tokenizer.decode(output_ids, skip_special_tokens=True))

Tool Calling

The prompt in the example is in Malay and translates to “Please help me find a 4-room flat near Tampines, budget under $500,000. I also want to know the estimated monthly loan payment.”

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained('aisingapore/gemma-sealion-4b-v4-cand5', trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "aisingapore/gemma-sealion-4b-v4-cand5",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Tolong carikan flat 4-bilik dekat Tampines, bajet bawah $500,000. Nak tahu juga berapa anggaran pinjaman bulanan."}
]

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_hdb_listings",
            "description": "Search for HDB flats available for sale",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "Town or area name"},
                    "flat_type": {"type": "string", "description": "Flat type e.g. 3-room, 4-room, 5-room"},
                    "max_price": {"type": "number", "description": "Maximum price in SGD"}
                },
                "required": ["location", "flat_type"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate_mortgage",
            "description": "Calculate estimated monthly mortgage payment",
            "parameters": {
                "type": "object",
                "properties": {
                    "loan_amount": {"type": "number", "description": "Loan amount in SGD"},
                    "interest_rate": {"type": "number", "description": "Annual interest rate as percentage"},
                    "loan_tenure_years": {"type": "integer", "description": "Loan period in years"}
                },
                "required": ["loan_amount"]
            }
        }
    }
]

input_ids = tokenizer.apply_chat_template(
    messages,
    tools=tools,
    return_tensors="pt",
    add_generation_prompt=True
).to(model.device)

generated_ids = model.generate(
    input_ids,
    max_new_tokens=512,
    do_sample=False,
)

response = tokenizer.decode(
    generated_ids[0][input_ids.shape[1]:],
    skip_special_tokens=False,
).replace("", "").strip()

print(response)

Training Details

Training Data

The instruction fine-tuning text dataset comprises of a collection of OSS & synthetic data. The datasets used for post-training can be accessed via the link below.

Datasets for Instruction Fine Tuning:

• 🤗aisingapore/SEA-Instruct-2602

Datasets for Tool-calling:

• 🤗allenai/Dolci-Instruct-SFT-Tool-Use

• 🤗Agent-Ark/Toucan-1.5M

Datasets for Reinforcement Learning:

• 🤗nvidia/Nemotron-Cascade-RL-Instruction-Following

• 🤗openai/gsm8k

Training Procedure

Training Hyperparameters

• Training regime: Our post-training workflow consists of instruction fine-tuning and model merging.

• Training hyperparameters: The following hyperparameters were used during training:

Category	Hyperparameter	Value
Optimization	Optimizer	ADAMW_TORCH_FUSED (β1=0.9, β2=0.999, ε=1e-08)
Batch Size	Train Batch Size (per device)	1
	Eval Batch Size (per device)	1
Hardware	Distributed Type	multi-GPU
	Number of Devices	64
Schedule	LR Scheduler Type	constant_with_warmup
	LR Scheduler Warmup Steps	269
Other	Training Steps	5397
	Seed	42

Evaluation

Testing Data, Factors & Metrics

We evaluated Apertus-SEA-LION-v4-8B-IT on general language capabilities and LLM-specific capabilities using SEA-HELM.

Results

For details on Apertus-SEA-LION-v4-8B-IT performance, please refer to the Leaderboard results on SEA-HELM.

Download the Models

The Apertus-SEA-LION-v4-8B models are available for download via the 🤗 HuggingFace Apertus-SEA-LION-v4-8B-IT repository. You can also explore more models in the same collection at 🤗 HuggingFace SEA-LION v4 Collection.

More Information

This is the repository for the commercial instruction-tuned model. The models have not been aligned for safety. Developers and users should perform their own safety fine-tuning and related security measures. In no event shall the authors be held liable for any claims, damages, or other liabilities arising from the use of the released weights and codes.

For more info, please contact us at SEA-LION Inquiry Form or [email protected]