SpanRuler

The span ruler lets you add spans to Doc.spans and/or Doc.ents using token-based rules or exact phrase matches. For usage examples, see the docs on rule-based span matching.

Assigned Attributes {id="assigned-attributes"}

Matches will be saved to Doc.spans[spans_key] as a SpanGroup and/or to Doc.ents, where the annotation is saved in the Token.ent_type and Token.ent_iob fields.

Location	Value
`Doc.spans[spans_key]`	The annotated spans. ~~SpanGroup~~
`Doc.ents`	The annotated spans. ~~Tuple[Span]~~
`Token.ent_iob`	An enum encoding of the IOB part of the named entity tag. ~~int~~
`Token.ent_iob_`	The IOB part of the named entity tag. ~~str~~
`Token.ent_type`	The label part of the named entity tag (hash). ~~int~~
`Token.ent_type_`	The label part of the named entity tag. ~~str~~

Config and implementation {id="config"}

The default config is defined by the pipeline component factory and describes how the component should be configured. You can override its settings via the config argument on nlp.add_pipe or in your config.cfg.

Example

python

config = {
   "spans_key": "my_spans",
   "validate": True,
   "overwrite": False,
}
nlp.add_pipe("span_ruler", config=config)

Setting	Description
`spans_key`	The spans key to save the spans under. If `None`, no spans are saved. Defaults to `"ruler"`. ~~Optional[str]~~
`spans_filter`	The optional method to filter spans before they are assigned to doc.spans. Defaults to `None`. ~~Optional[Callable[[Iterable[Span], Iterable[Span]], List[Span]]]~~
`annotate_ents`	Whether to save spans to doc.ents. Defaults to `False`. ~~bool~~
`ents_filter`	The method to filter spans before they are assigned to doc.ents. Defaults to `util.filter_chain_spans`. ~~Callable[[Iterable[Span], Iterable[Span]], List[Span]]~~
`phrase_matcher_attr`	Token attribute to match on, passed to the internal `PhraseMatcher` as `attr`. Defaults to `None`. ~~Optional[Union[int, str]]~~
`matcher_fuzzy_compare` <Tag variant="new">3.5</Tag>	The fuzzy comparison method, passed on to the internal `Matcher`. Defaults to `spacy.matcher.levenshtein.levenshtein_compare`. ~~Callable~~
`validate`	Whether patterns should be validated, passed to `Matcher` and `PhraseMatcher` as `validate`. Defaults to `False`. ~~bool~~
`overwrite`	Whether to remove any existing spans under `Doc.spans[spans key]` if `spans_key` is set, or to remove any ents under `Doc.ents` if `annotate_ents` is set. Defaults to `True`. ~~bool~~
`scorer`	The scoring method. Defaults to `Scorer.score_spans` for `Doc.spans[spans_key]` with overlapping spans allowed. ~~Optional[Callable]~~

python

%%GITHUB_SPACY/spacy/pipeline/span_ruler.py

SpanRuler.init {id="init",tag="method"}

Initialize the span ruler. If patterns are supplied here, they need to be a list of dictionaries with a "label" and "pattern" key. A pattern can either be a token pattern (list) or a phrase pattern (string). For example: {"label": "ORG", "pattern": "Apple"}.

Example

python

# Construction via add_pipe
ruler = nlp.add_pipe("span_ruler")

# Construction from class
from spacy.pipeline import SpanRuler
ruler = SpanRuler(nlp, overwrite=True)

Name	Description
`nlp`	The shared nlp object to pass the vocab to the matchers and process phrase patterns. ~~Language~~
`name`	Instance name of the current pipeline component. Typically passed in automatically from the factory when the component is added. Used to disable the current span ruler while creating phrase patterns with the nlp object. ~~str~~
keyword-only
`spans_key`	The spans key to save the spans under. If `None`, no spans are saved. Defaults to `"ruler"`. ~~Optional[str]~~
`spans_filter`	The optional method to filter spans before they are assigned to doc.spans. Defaults to `None`. ~~Optional[Callable[[Iterable[Span], Iterable[Span]], List[Span]]]~~
`annotate_ents`	Whether to save spans to doc.ents. Defaults to `False`. ~~bool~~
`ents_filter`	The method to filter spans before they are assigned to doc.ents. Defaults to `util.filter_chain_spans`. ~~Callable[[Iterable[Span], Iterable[Span]], List[Span]]~~
`phrase_matcher_attr`	Token attribute to match on, passed to the internal PhraseMatcher as `attr`. Defaults to `None`. ~~Optional[Union[int, str]]~~
`matcher_fuzzy_compare` <Tag variant="new">3.5</Tag>	The fuzzy comparison method, passed on to the internal `Matcher`. Defaults to `spacy.matcher.levenshtein.levenshtein_compare`. ~~Callable~~
`validate`	Whether patterns should be validated, passed to Matcher and PhraseMatcher as `validate`. Defaults to `False`. ~~bool~~
`overwrite`	Whether to remove any existing spans under `Doc.spans[spans key]` if `spans_key` is set, or to remove any ents under `Doc.ents` if `annotate_ents` is set. Defaults to `True`. ~~bool~~
`scorer`	The scoring method. Defaults to `Scorer.score_spans` for `Doc.spans[spans_key]` with overlapping spans allowed. ~~Optional[Callable]~~

SpanRuler.initialize {id="initialize",tag="method"}

Initialize the component with data and used before training to load in rules from a pattern file. This method is typically called by Language.initialize and lets you customize arguments it receives via the [initialize.components] block in the config. Any existing patterns are removed on initialization.

Example

python

span_ruler = nlp.add_pipe("span_ruler")
span_ruler.initialize(lambda: [], nlp=nlp, patterns=patterns)

ini

### config.cfg
[initialize.components.span_ruler]

[initialize.components.span_ruler.patterns]
@readers = "srsly.read_jsonl.v1"
path = "corpus/span_ruler_patterns.jsonl"

Name	Description
`get_examples`	Function that returns gold-standard annotations in the form of `Example` objects. Not used by the `SpanRuler`. ~~Callable[[], Iterable[Example]]~~
keyword-only
`nlp`	The current `nlp` object. Defaults to `None`. ~~Optional[Language]~~
`patterns`	The list of patterns. Defaults to `None`. ~~Optional[Sequence[Dict[str, Union[str, List[Dict[str, Any]]]]]]~~

SpanRuler.len {id="len",tag="method"}

The number of all patterns added to the span ruler.

Example

python

ruler = nlp.add_pipe("span_ruler")
assert len(ruler) == 0
ruler.add_patterns([{"label": "ORG", "pattern": "Apple"}])
assert len(ruler) == 1

Name	Description
RETURNS	The number of patterns. ~~int~~

SpanRuler.contains {id="contains",tag="method"}

Whether a label is present in the patterns.

Example

python

ruler = nlp.add_pipe("span_ruler")
ruler.add_patterns([{"label": "ORG", "pattern": "Apple"}])
assert "ORG" in ruler
assert not "PERSON" in ruler

Name	Description
`label`	The label to check. ~~str~~
RETURNS	Whether the span ruler contains the label. ~~bool~~

SpanRuler.call {id="call",tag="method"}

Find matches in the Doc and add them to doc.spans[span_key] and/or doc.ents. Typically, this happens automatically after the component has been added to the pipeline using nlp.add_pipe. If the span ruler was initialized with overwrite=True, existing spans and entities will be removed.

Example

python

ruler = nlp.add_pipe("span_ruler")
ruler.add_patterns([{"label": "ORG", "pattern": "Apple"}])

doc = nlp("A text about Apple.")
spans = [(span.text, span.label_) for span in doc.spans["ruler"]]
assert spans == [("Apple", "ORG")]

Name	Description
`doc`	The `Doc` object to process, e.g. the `Doc` in the pipeline. ~~Doc~~
RETURNS	The modified `Doc` with added spans/entities. ~~Doc~~

SpanRuler.add_patterns {id="add_patterns",tag="method"}

Add patterns to the span ruler. A pattern can either be a token pattern (list of dicts) or a phrase pattern (string). For more details, see the usage guide on rule-based matching.

Example

python

patterns = [
    {"label": "ORG", "pattern": "Apple"},
    {"label": "GPE", "pattern": [{"lower": "san"}, {"lower": "francisco"}]}
]
ruler = nlp.add_pipe("span_ruler")
ruler.add_patterns(patterns)

Name	Description
`patterns`	The patterns to add. ~~List[Dict[str, Union[str, List[dict]]]]~~

SpanRuler.remove {id="remove",tag="method"}

Remove patterns by label from the span ruler. A ValueError is raised if the label does not exist in any patterns.

Example

python

patterns = [{"label": "ORG", "pattern": "Apple", "id": "apple"}]
ruler = nlp.add_pipe("span_ruler")
ruler.add_patterns(patterns)
ruler.remove("ORG")

Name	Description
`label`	The label of the pattern rule. ~~str~~

SpanRuler.remove_by_id {id="remove_by_id",tag="method"}

Remove patterns by ID from the span ruler. A ValueError is raised if the ID does not exist in any patterns.

Example

python

patterns = [{"label": "ORG", "pattern": "Apple", "id": "apple"}]
ruler = nlp.add_pipe("span_ruler")
ruler.add_patterns(patterns)
ruler.remove_by_id("apple")

Name	Description
`pattern_id`	The ID of the pattern rule. ~~str~~

SpanRuler.clear {id="clear",tag="method"}

Remove all patterns the span ruler.

Example

python

patterns = [{"label": "ORG", "pattern": "Apple", "id": "apple"}]
ruler = nlp.add_pipe("span_ruler")
ruler.add_patterns(patterns)
ruler.clear()

SpanRuler.to_disk {id="to_disk",tag="method"}

Save the span ruler patterns to a directory. The patterns will be saved as newline-delimited JSON (JSONL).

Example

python

ruler = nlp.add_pipe("span_ruler")
ruler.to_disk("/path/to/span_ruler")

Name	Description
`path`	A path to a directory, which will be created if it doesn't exist. Paths may be either strings or `Path`-like objects. ~~Union[str, Path]~~

SpanRuler.from_disk {id="from_disk",tag="method"}

Load the span ruler from a path.

Example

python

ruler = nlp.add_pipe("span_ruler")
ruler.from_disk("/path/to/span_ruler")

Name	Description
`path`	A path to a directory. Paths may be either strings or `Path`-like objects. ~~Union[str, Path]~~
RETURNS	The modified `SpanRuler` object. ~~SpanRuler~~

SpanRuler.to_bytes {id="to_bytes",tag="method"}

Serialize the span ruler to a bytestring.

Example

python

ruler = nlp.add_pipe("span_ruler")
ruler_bytes = ruler.to_bytes()

Name	Description
RETURNS	The serialized patterns. ~~bytes~~

SpanRuler.from_bytes {id="from_bytes",tag="method"}

Load the pipe from a bytestring. Modifies the object in place and returns it.

Example

python

ruler_bytes = ruler.to_bytes()
ruler = nlp.add_pipe("span_ruler")
ruler.from_bytes(ruler_bytes)

Name	Description
`bytes_data`	The bytestring to load. ~~bytes~~
RETURNS	The modified `SpanRuler` object. ~~SpanRuler~~

SpanRuler.labels {id="labels",tag="property"}

All labels present in the match patterns.

Name	Description
RETURNS	The string labels. ~~Tuple[str, ...]~~

SpanRuler.ids {id="ids",tag="property"}

All IDs present in the id property of the match patterns.

Name	Description
RETURNS	The string IDs. ~~Tuple[str, ...]~~

SpanRuler.patterns {id="patterns",tag="property"}

All patterns that were added to the span ruler.

Name	Description
RETURNS	The original patterns, one dictionary per pattern. ~~List[Dict[str, Union[str, dict]]]~~

Attributes {id="attributes"}

Name	Description
`key`	The spans key that spans are saved under. ~~Optional[str]~~
`matcher`	The underlying matcher used to process token patterns. ~~Matcher~~
`phrase_matcher`	The underlying phrase matcher used to process phrase patterns. ~~PhraseMatcher~~

Assigned Attributes {id="assigned-attributes"}

Config and implementation {id="config"}

Example

SpanRuler.__init__ {id="init",tag="method"}

Example

SpanRuler.initialize {id="initialize",tag="method"}

Example

SpanRuler.__len__ {id="len",tag="method"}

Example

SpanRuler.__contains__ {id="contains",tag="method"}

Example

SpanRuler.__call__ {id="call",tag="method"}

Example

SpanRuler.add_patterns {id="add_patterns",tag="method"}

Example

SpanRuler.remove {id="remove",tag="method"}

Example

SpanRuler.remove_by_id {id="remove_by_id",tag="method"}

Example

SpanRuler.clear {id="clear",tag="method"}

Example

SpanRuler.to_disk {id="to_disk",tag="method"}

Example

SpanRuler.from_disk {id="from_disk",tag="method"}

Example

SpanRuler.to_bytes {id="to_bytes",tag="method"}

Example

SpanRuler.from_bytes {id="from_bytes",tag="method"}

Example

SpanRuler.labels {id="labels",tag="property"}

SpanRuler.ids {id="ids",tag="property"}

SpanRuler.patterns {id="patterns",tag="property"}

Attributes {id="attributes"}

SpanRuler.init {id="init",tag="method"}

SpanRuler.len {id="len",tag="method"}

SpanRuler.contains {id="contains",tag="method"}

SpanRuler.call {id="call",tag="method"}