Research Project / ACM MM 2025
case 01StePO-Rec
Turning styling knowledge into a traceable, explainable, and personalized recommendation pipeline.
StePO-Rec reframes outfit recommendation from black-box generation into a staged decision pipeline that connects user preference, fashion expertise, and trend change.
Research Problem
Making fashion knowledge truly enter the recommendation decision chain
The project targets a harder task than product retrieval: a personalized outfit styling assistant that needs to understand users, fashion principles, trend change, and recommendation rationale at the same time.
issue 01
Professional styling knowledge rarely enters the recommendation process directly and is often flattened into tags or hidden in black-box models.
issue 02
Long-term user preference and short-term trend change are difficult to balance in the same recommendation output.
issue 03
Recommendations are often delivered without a clear explanation path, which weakens trust and perceived expertise.
PAFA Knowledge Base
PAFA turns fashion expertise into an operable knowledge base.
PAFA is not a collage of references, but a multimodal structure that organizes theory, situational context, trend signals, and user preference into one backbone for retrieval, reasoning, and reranking.
Three-Layer Architecture
PAFA uses a Metadata / Principle / Semantic hierarchy to connect item attributes, styling rules, and higher-level style semantics.
METADATA Layer
The data foundation that defines fine-grained item, outfit, and scene attributes for retrieval and matching.
PRINCIPLE Layer
A computable middle layer that formalizes color, silhouette, proportion, and contextual constraints.
SEMANTIC Layer
A higher layer that links rules to style contexts through a style-rule graph and case library.
Knowledge Sources
PAFA gains credibility not from one dataset, but from how cross-source and cross-modal knowledge is curated, aligned, and maintained.
source 01
Professional books and literature: stable principles of color, silhouette, and proportion.
source 02
Trend reports and fashion media: seasonal direction, color migration, and silhouette change.
source 03
User history corpora: stable personal preference and short-term taste fluctuation.
Six sub-libraries form the callable PAFA system.
PAFA is not a static knowledge graph, but a fashion knowledge operating system that can enter the recommendation flow and support hybrid retrieval with multi-step reasoning.
Reasoning Pipeline
Tree-structured reasoning makes styling decisions converge step by step.
StePO-Rec combines hybrid retrieval, tree-search multi-step reasoning, and preference-trend reranking so the system does not jump to one answer but progressively narrows the decision path.
step 01
Input and anchor item: start from the user, the item, and context instead of generic popularity.
step 02
Hybrid knowledge retrieval: retrieve cases, principles, trends, and personal preference together rather than only similar vectors.
step 03
Tree-structured multi-step reasoning: unfold decisions progressively through scene-style-color-silhouette paths.
step 04
Preference-trend collaborative reranking: rebalance outputs between personal identity and temporal relevance.
Case Study
The explanation path makes the recommendation system visible.
The case study translates the abstract method into a user-facing result: the system not only recommends, but also lets people trace outcomes back to rules, cases, and preference evidence.
reading 01
Personalized linkage: connect the recommendation to similar scenes and user preference traces.
reading 02
Key factor explanation: expose how color, silhouette, and style factors participate in the decision.
reading 03
Concrete scoring: make preference and trend explicit signals rather than hidden weights.
reading 04
Natural-language rationale: translate knowledge evidence into a readable report for users.
Results
Validation shows knowledge, reasoning, and reranking must work together.
On IQON-3000, using Recall@K and MAP, StePO-Rec shows that the performance gain comes from the collaboration between retrieval, tree-structured reasoning, and preference-trend reranking rather than from one isolated prompt trick.
Recall@1
0.55
28.3% over the best baseline
MAP
0.85
32.8% over the best baseline
proof 01
Against the best baseline, Two-Stage RAG with reranking, Recall@1 improves by 28.3% and MAP by 32.8%.
proof 02
Without knowledge retrieval, R@1 drops from 0.55 to 0.34; without reranking, MAP drops from 0.85 to 0.57.
proof 03
The result shows that personalized outfit recommendation can be strengthened by structured knowledge and recursive reasoning rather than opaque generation alone.
Closing Reflection
From paper to system: StePO-Rec becomes the starting point for what came after.
What makes StePO-Rec important in this portfolio is not publication alone. It is the earliest project that systematically proved fashion knowledge can be structured, recommendation can be explained, and research methods can grow into later systems and products.
value 01
It turns fashion expertise into a computable, retrievable, and explainable system backbone.
value 02
It becomes the methodological source of later work such as FashionAgent and OUTFITTED rather than mere conceptual context.
value 03
It demonstrates that my work spans problem framing, knowledge engineering, system design, and experimental validation beyond visual expression.