API reference

Generated from the package docstrings.

Top-level

kosac.load_lexicon(feature, ngrams=[1], min_freq=0, threshold=0.0)

Load a bundled KOSAC lexicon by feature name.

Parameters:

• feature (str) – One of kosac.FEATURES (e.g. 'polarity', 'intensity').

• ngrams (list[int]) – Which n-gram lengths to keep (default: unigrams only).

• min_freq (optional filtering forwarded to set_lexicon.)

• threshold (optional filtering forwarded to set_lexicon.)

kosac.citation(style='bibtex')

Return a citation string for the KOSAC lexicon package.

kosac.describe_feature(feature)

Return {feature, values, reference} describing a feature’s labels.

kosac.FEATURES = ('polarity', 'intensity', 'expressive-type', 'nested-order', 'subjectivity-polarity', 'subjectivity-type')

The six canonical KOSAC sentiment features (the frozen 2016 data).

kosac.info.FEATURE_VALUES

Short English summaries of each feature’s label values.

Analyzer

High-level sentiment analysis built on top of the KOSAC lexicons.

SentimentAnalyzer bundles one or more feature lexicons with a tokenizer and scores raw text in a single call — including KOSAC’s six semantic features at once. It is the recommended entry point for end-to-end use:

from kosac import SentimentAnalyzer
analyzer = SentimentAnalyzer('all')            # needs a tokenizer (Kiwi by default)
analyzer.analyze('이 영화 정말 좋다')

kosac.analyzer.select_matches(tokens, entry_set, ngram_lengths)

Greedy leftmost-longest, non-overlapping match over tokenized text.

tokens is a list of (token_str, char_start, char_end). Returns a list of (entry, tok_start_idx, tok_end_idx, char_start, char_end).

kosac.analyzer.select_matches_multiscale(tokens, entry_set, ngram_lengths)

All (overlapping) n-gram windows present in entry_set, every length.

Unlike select_matches() (greedy leftmost-longest, non-overlapping), this yields every matching window, so a token contributes to its unigram and to any longer n-grams it is part of. Summing these multi-scale matches (instead of letting a long match suppress its unigrams) scores markedly better on held-out sentiment classification, which is why it is the analyzer’s default. Same return shape as select_matches().

class kosac.analyzer.SentimentAnalyzer(features='polarity', tokenizer=None, ngrams=(1, 2, 3), min_freq=0, threshold=0.0, smoothing=True, align=False, negation=False, intensifier=False, window=2, intensifier_factor=2.0, negations=None, intensifiers=None, scoring='multiscale', ngram_weights=None)

Bases: object

Score Korean text against one or more KOSAC feature lexicons.

Parameters:

• features (str | iterable[str]) – A feature name, an iterable of names, or 'all' for every feature in kosac.FEATURES.

• tokenizer (Tokenizer, optional) – Defaults to kosac.tokenizers.KiwiTokenizer (requires the kiwi extra). Any object with tokenize_with_offsets works.

• ngrams – Forwarded to the underlying lexicons.

• min_freq – Forwarded to the underlying lexicons.

• threshold – Forwarded to the underlying lexicons.

• smoothing (bool) – Apply add-one smoothing before aggregating (matches get_sent_probs).

• scoring ({'multiscale', 'greedy'}) – 'multiscale' (default) sums every overlapping n-gram match, which scores best on held-out sentiment classification; 'greedy' is the legacy leftmost-longest non-overlapping matcher. Affects analyze/polarity_score (count always uses the non-overlapping matcher for word tallies).

• ngram_weights (dict[int, float], optional) – Per-length weights for multiscale matches (default 1.0 each).

analyze(text)

Analyze a single string. Returns a JSON-serialisable dict.

analyze_batch(texts)

Analyze an iterable of strings. Returns a list of result dicts.

analyze_frame(texts)

Analyze an iterable of strings into a tidy pandas.DataFrame.

One row per text; <feature>.label / <feature>.prob columns.

polarity_score(text)

Continuous polarity in [-1, 1]: P(POS) - P(NEG) (higher = more POS).

The benchmark’s decision axis — threshold it to get a binary label (see predict_polarity()). Uses the multiscale scorer by default.

predict_polarity(text, threshold=0.0)

Binary polarity label: 'POS' if polarity_score() > threshold.

predict_polarity_batch(texts, threshold=0.0)

predict_polarity() over an iterable of texts.

count(text)

Frequency-based analysis (the method common in social-science studies).

Counts matched morphemes by their dominant label (max.value) and reports counts, proportions, and the most frequent label per feature.

count_batch(texts)

Frequency-count an iterable of strings. Returns a list of result dicts.

count_frame(texts)

Frequency-count into a pandas.DataFrame: <feature>.label, <feature>.total, and one <feature>.<label> count column per label.

Lexicons

class kosac.lexicon.SentimentLexicon(filepath=None, ngrams=[1])

Bases: object

labels = []

classmethod load(ngrams=[1], min_freq=0, threshold=0.0)

Load this feature’s lexicon from the CSV bundled with the package.

get_original_lexicon()

save(filepath)

Write the lexicon to a CSV in the package’s absolute-count format.

Columns are ngram (morphemes joined by a space) and one absolute-count column per label — the same format the loader reads. Reload it through the constructor: a concrete subclass uses its declared labels, while GenericLexicon(filepath=…) infers the labels from the columns.

set_lexicon(min_freq=0, threshold=0.0)

get_lexicon()

get_size()

get_labels()

get_entry(morph)

verify(morph, verbose=True)

initialize_entry(morph, **kwargs)

add_token(morph, tag, verbose=True)

update(examples)

update_from_corpus(corpus, tokenizer, pos_tag=None, min_freq=0, max_value_threshold=0.0)

Rebuild the lexicon’s frequencies from a labeled corpus.

Each text is tokenized into morpheme N-grams and its label is counted for every N-gram. The optional filters keep the result from being dominated by uninformative entries:

Parameters:

• pos_tag (str or iterable of str, optional) – Keep only N-grams that contain at least one token with one of these POS tags (e.g. {'NNG', 'NNP', 'VV', 'VA', 'XR', 'MAG'} for content words). This drops pure function-morpheme entries (이/MM, 다/EF) while keeping meaningful constructions like ㄹ/ETM 수/NNB 있/VV (“can …”).

• min_freq (int) – Drop entries observed fewer than min_freq times.

• max_value_threshold (float) – Drop entries whose dominant-label proportion (max.prop) is below this.

Notes

This is a full rebuild from the corpus: any counts previously in the lexicon are discarded and the result holds exactly the N-grams observed in corpus. Counts are tallied with a Counter and assembled in one vectorized pass, so a 150k-sentence corpus builds in seconds rather than the minutes a per-token pandas update would take.

export_user_dict(dict_path='user_dictionary.txt')

get_pattern(sorting=True)

match_patterns(sentence, tokenizer, sorting=True)

get_match_info(sentence, tokenizer, sorting=True)

get_smoothed_lexicon()

get_sent_probs(sentence, tokenizer, smoothing=True)

get(morph)

match(sentence, tokenizer, sorting=True)

class kosac.lexicon.PolarityLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

Sentiment polarity: POS / NEG / NEUT / COMP (mixed) / None.

labels = ['COMP', 'NEG', 'NEUT', 'None', 'POS']

class kosac.lexicon.IntensityLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

Sentiment intensity: High / Medium / Low / None.

labels = ['High', 'Low', 'Medium', 'None']

class kosac.lexicon.ExpressiveTypeLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

How the sentiment is expressed (direct/indirect/writing-device, …).

labels = ['dir-action', 'dir-explicit', 'dir-speech', 'indirect', 'writing-device']

class kosac.lexicon.NestedOrderLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

Nesting depth of the subjective expression (0–3).

labels = ['0', '1', '2', '3']

class kosac.lexicon.SubjectivityPolarityLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

Polarity of the subjectivity: POS / NEG / NEUT / COMP.

labels = ['COMP', 'NEG', 'NEUT', 'POS']

class kosac.lexicon.SubjectivityTypeLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

Type of subjectivity: Judgment / Emotion / Argument / Intention / …

labels = ['Agreement', 'Argument', 'Emotion', 'Intention', 'Judgment', 'Others', 'Speculation']

class kosac.lexicon.PolarityBlendLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

High-accuracy POS/NEG polarity lexicon: the frozen KOSAC seeds blended with an NSMC-learned lexicon (see benchmarks/ and build_shipped_blend.py).

Unlike the frozen PolarityLexicon (5 labels, near-chance out of domain), this 2-label lexicon is tuned for classification and ships gzipped. Best used with the analyzer’s default multi-scale scorer. Derived data — CC BY-SA (mixes CC BY-SA KOSAC with CC0 NSMC); see data/polarity-blend.NOTICE.

labels = ['NEG', 'POS']

class kosac.lexicon.GenericLexicon(filepath=None, ngrams=[1])

Bases: SentimentLexicon

A lexicon with user-defined labels (see set_labels()).

set_labels(labels)

Set the label set for this lexicon.

Parameters:

labels (list)

Tokenizers

Tokenizers used to turn raw Korean text into surface/POS morpheme tokens.

The base Tokenizer is pure-Python. KiwiTokenizer (the recommended POS tokenizer) requires kosac-lexicon[kiwi] — Kiwi installs from PyPI with no Java/JVM. HuggingFaceTokenizer requires kosac-lexicon[transformers]. Those heavy dependencies are imported lazily so that import kosac never fails when the extras are absent.

Note on tagsets: the lexicon entries use Sejong-style surface/POS tokens. Kiwi’s tagset is Sejong-based and aligns on the common content/function morpheme tags (NNG, VV, VA, JKS, EC, …), so it matches the lexicon well; a few symbol and web tags differ from the original tagger and may segment differently.

class kosac.tokenizers.Tokenizer

Bases: object

Whitespace tokenizer and n-gram helper. No external dependencies.

tokenize(sentence)

get_tokens_str(sentence)

tokenize_with_offsets(sentence)

Return (token, start, end) triples aligning tokens to char offsets.

The default implementation locates each token as a substring of the input. Subclasses with real offset information (e.g. Kiwi) should override it.

get_ngrams(sentence, ns)

tokenize_batch(sentences)

Tokenize many sentences at once. The default applies tokenize() per sentence; tokenizers with a batch API (e.g. Kiwi) override this to process the whole list in parallel.

get_ngrams_batch(sentences, ns)

N-grams for many sentences (one list per sentence), via tokenize_batch.

Equivalent to [get_ngrams(s, ns) for s in sentences] but routed through tokenize_batch(), so a batch tokenizer parallelizes the heavy step.

class kosac.tokenizers.KiwiTokenizer(user_words=None, **kwargs)

Bases: Tokenizer

Morpheme tokenizer backed by Kiwi. Requires kosac-lexicon[kiwi].

Emits surface/POS tokens (e.g. 힘/NNG) matching the lexicon’s entry format. Extra keyword arguments are forwarded to kiwipiepy.Kiwi.

add_user_words(words, score=0.0)

Register (form, tag) pairs (or 'form/tag' strings) as Kiwi user words. Returns the number successfully added; entries Kiwi rejects (e.g. the wildcard * forms) are skipped.

classmethod from_lexicon(lexicon, tags=None, score=0.0, **kwargs)

Build a tokenizer seeded with a lexicon’s unigram entries as user words.

This biases Kiwi toward segmenting text the way the lexicon expects, easing the Sejong/Kiwi tagset mismatch. tags optionally restricts which POS tags are registered (e.g. {'NNG', 'VV', 'VA', 'XR'} for content words).

tokenize(sentence)

tokenize_batch(sentences)

Tokenize many sentences at once. The default applies tokenize() per sentence; tokenizers with a batch API (e.g. Kiwi) override this to process the whole list in parallel.

tokenize_with_offsets(sentence)

Return (token, start, end) triples aligning tokens to char offsets.

The default implementation locates each token as a substring of the input. Subclasses with real offset information (e.g. Kiwi) should override it.

class kosac.tokenizers.HuggingFaceTokenizer(model_name='snunlp/KR-ELECTRA-discriminator')

Bases: Tokenizer

Subword tokenizer backed by a HuggingFace model. Requires kosac-lexicon[transformers].

tokenize(sentence)

Corpora

class kosac.corpora.Corpus(filepath)

Bases: object

get_labels()

scikit-learn

scikit-learn integration: use the KOSAC lexicon as a feature extractor.

Requires the sklearn extra (pip install kosac-lexicon[sklearn]):

from kosac.sklearn import KosacVectorizer
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression

clf = make_pipeline(KosacVectorizer('all'), LogisticRegression())
clf.fit(texts, labels)

class kosac.sklearn.KosacVectorizer(*args, **kwargs)

Bases: BaseEstimator, TransformerMixin

Transform Korean text into KOSAC label-probability features.

Each output column is one <feature>=<label> probability. Constructor arguments mirror kosac.SentimentAnalyzer.

fit(X=None, y=None)

transform(X)

get_feature_names_out(input_features=None)

Command line

Command-line interface: kosac analyze "문장" and friends.