Fine-tuning GLiNER

Experiment note: fine-tune GLiNER for ticket classification — the work use case. GLiNER’s schema-as-prompt architecture (entity types as text, zero-shot) is exactly the shape for support-ticket triage: labels are free text, new categories need no retraining to try, and a fine-tune pins the ones that matter. The hypothesis to test: a small fine-tuned GLiNER beats both zero-shot GLiNER and an LLM classifier on the org’s ticket taxonomy — at a fraction of the inference cost.

How the pieces fit:

• GLiNER2 (OSS) — the model family + gliner2 Python API with a TrainingDataset/InputExample path built in (pip install gliner2[train]); GLiNER2-PII is the existing proof that fine-tuned variants (0.3B) match much larger models (PII redaction).
• GLiNER2.5-Decide — the newest family member (Fastino), and the direct subject of the open-weight decision model announcement: 340M encoder, Apache-2.0, schema of typed questions (single/multi/ordered answers, instructions, examples, cross-question rules) → constrained joint decoding with probabilities, confidence, feasibility metadata. Latency: 38-47 ms p50 across T4/L4/V100/A100, 167 ms on 48-vCPU CPU — air-gappable. Fine-tuning: full or LoRA. The Fast Decisions suite (17 datasets, 5,100 examples) is itself the ticket-classification-shaped benchmark: support intent, ticket routing, email triage, agent handoff, product feedback among the tasks — GLiNER2.5-Decide leads at 60.1% avg, strongest exactly on support intent (75.3%). @george_onx demo thread quoted in the capture.
• **The <$10claim\ast \ast :[@DhruvAtreja1](https://x.com/DhruvAtreja1/status/2103531555884880036)—"Youcanfine-tuneyourOWNJevfor<$10 on your OWN data” via the Fastino GLiNER agent skill (hosted APIs for dataset prep, fine-tuning, eval, inference; npx skills add https://api.fastino.ai --skill fastino-gliner). Same architecture as Decision models — a fine-tuned GLiNER is a cheap private Jev.
• On-device: @JackdeS11 converted it to LiteRT (fp16): identical answers to fp32 on 361/361 test requests, 66-70 ms per request at 128 tokens on a Galaxy S26 GPU (175 ms at 256). Card + Python/Android sample: litert-community/GLiNER2.5-Decide-LiteRT. The triage-classifier-could-run-on-a-phone datapoint — relevant if the ticket classifier ever needs to go fully offline/edge.
• Fine-tuning mechanics live in LLM fine-tuning/SFT; the data-curation rules there (quality over volume, a few hundred clean examples, seed synthetic with real examples and filter aggressively) apply directly to ticket labels.

Experiment plan (ticket classification)

1. Taxonomy + data: export a labeled sample from the ticketing system (Kayako/Jira queues); labels = the triage categories. Few hundred to few thousand clean examples per the Fastino data rules; hand-correct mislabels — the model learns dataset proportions.
2. Baselines: zero-shot GLiNER2 with the label list as the schema; an LLM classify call (the current default everyone compares against).
3. Fine-tune: gliner2[train] locally on the 3090 (0.3B model — QLoRA-class memory, see LoRA) or the Fastino hosted skill for the <$10 run — do both, compare cost/quality.
4. Eval: Agent tests rules — error analysis first, binary pass/fail per ticket, task-specific checks not generic benchmarks; production monitoring after deploy. Precision on “exact set, nothing else” matters for triage (a wrong queue is a lost ticket).
5. Deploy shape: local llama-swap (OSS)-adjacent service or the Fastino endpoint, fronting the triage flow; ~100 ms per ticket.

Open questions

• Is classification-as-NER the right frame (single-label from a schema) or does ticket triage need multi-label? GLiNER handles both via entity spans — test.
• Polish-language tickets: GLiNER is multilingual but the fine-tune data mix matters (the PII redaction Polish benchmark is the cautionary tale — Presidio’s Polish weakness is exactly what AnonimiNER fixed with a PL-trained encoder).
• Where does the labeled data come from — historical assignments are noisy proxies for ground truth.

Sources (captured 2026-09-26): clippings/x-2103531555884880036.md (tweet + quoted @george_onx GLiNER2.5-Decide demo thread) · clippings/docs-fastino-ai-concepts-gliner-agent-skill.md (agent-skill docs).

GLiNER2.5 unlimited span length: the extraction unlock (2026-10-08)

george_onx (143❤; Fastino): GLiNER2.5 now extracts spans of any length — most extraction models score spans only up to a fixed maximum width; GLiNER2.5 “predicts where an entity begins and ends instead” (boundary prediction, not window scoring). Unlock: full addresses, product descriptions, hypotheses + supporting evidence, long quotes, entire legal clauses. The before/after: “it used to be that only decoder-based LLMs had a chance at longer spans, and even then, plagued by hallucinations — now deterministic information extraction for longer spans from a local SLM.” Artifacts: gliner2.5-multi-v1 weights, unlimited-span Space (runs locally), agent.fastino.ai API. For the experiment note: this widens the ticket-triage lane from labels to extracts — pull the full problem statement, reproduction steps, or affected-config block as a span, deterministically, beside the classification. Same thesis as Small language models’s win-a-job pattern and the Agent knowledge systems compass extraction: structured output from cheap local models, not LLM regeneration.