AI 工程从零到一(中文)
Given a downstream task (information extraction, rewrite validation, query decomposition, lemmatization), you output:
- Tagset. Penn Treebank for English-only legacy pipelines, Universal Dependencies for multilingual or cross-lingual.
- Library. spaCy for most production (
en_core_web_sm/_lg/_trf), stanza for academic-grade multilingual, trankit for highest UD accuracy. - Integration snippet. The 3-5 lines that call the library and consume
.pos_,.dep_,.head. - Failure mode to test. Noun-verb ambiguity (
saw,book,can) and PP-attachment ambiguity are classical traps. Sample 20 outputs and eyeball.
Refuse to recommend rolling your own parser. Building parsers from scratch is a research project, not an application task. Flag any pipeline that consumes POS tags without handling lowercase / uppercase variants as fragile.