LLM experimentation · UPC / 2025–26
Aspect-Based Sentiment Analysis with LLMs.
A careful comparison of small-model strategies
A restaurant review can praise the food and criticize the service in the same sentence. Together with Máximo Meya, I explored how small open-source language models could extract sentiment for individual aspects across English and Spanish reviews, and how different ways of adapting a model affect the task.
Zero- / few-shotStructured generationSynthetic dataPEFT / LoRAMacro F1
A more structured prediction
Instead of assigning one sentiment to a whole review, the model predicts polarity for the aspect categories that are actually mentioned, such as FOOD#QUALITY or SERVICE#GENERAL. Outputs follow a shared structured schema, making prompt-based approaches and model adaptation comparable at the prediction and evaluation levels.
Four approaches, one framework
The project implements zero-shot prompting, few-shot prompting with examples, synthetic labeled data generation and LoRA-based parameter-efficient fine-tuning. Shared data handling, output parsing and evaluation scripts make it possible to study the effect of changing one experimental choice without changing the task itself.
- Structured zero-shot outputs and prompt/decoding sweeps
- Few-shot selection and synthetic review generation
- LoRA training and inference with common evaluation utilities
An example of measured behavior
In the published zero-shot v12 sweep, greedy decoding reached a mean per-example F1 of 72.65% on the project evaluation set, with 75.15% for English and 71.52% for Spanish. That row represents one decoding configuration and one run; it is not an aggregate comparison of all four strategies. Other sampling settings in the same sweep performed worse, illustrating that decoding choices affect a structured extraction task.
What made the comparison useful
For a constrained task, a more elaborate method does not automatically produce a better system. The interesting questions are when examples improve prompting, whether synthetic labels help, and when adapting the weights is worth its cost. The repository records the methods and experiments, while the published sweep supports only the specific zero-shot result cited here.
UPC MGL-GIA 2025/26 group project with Máximo Meya. The quoted F1 is mean per-example F1 from a published zero-shot sweep, not a cross-method benchmark or a held-out production estimate.