Data Format

Input Format

RESTful Input

Definition

To make a RESTful call, one needs to send a json HTTP POST request to the server, which contains at least a text field or a tokens field. The input to RESTful API is very flexible. It can be one of the following 3 formats:

1. It can be a document of raw str filled into text. The server will split it into sentences.
2. It can be a list of sentences, each sentence is a raw str, filled into text.
3. It can be a list of tokenized sentences, each sentence is a list of str typed tokens, filled into tokens.

Additionally, fine-grained controls are performed with the arguments defined in 
:meth:`hanlp_restful.HanLPClient.parse`.

Examples

curl -X 'POST' \
  'https://hanlp.hankcs.com/api/parse' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "language": "zh",
  "text": "HanLP为生产环境带来次世代最先进的多语种NLP技术。晓美焰来到北京参观自然语义科技公司。"
}'

Model Input

```{seealso}
We mostly follow the conventional file format of each NLP task instead of re-inventing them. Thus, we use `.tsv` for tagging and 
`.conllu` for parsing etc. For more details, refer to [datasets](https://hanlp.hankcs.com/docs/api/hanlp/datasets/index.html).   
```

The input format to models is specified per model and per task. Generally speaking, if a model has no tokenizer built in, then its input is a sentence in list[str] form (a list of tokens), or multiple such sentences nested in a list.

If a model has a tokenizer built in, each sentence is in str form. Additionally, you can use skip_tasks='tok*' to ask the model to use your tokenized inputs instead of tokenizing them, in which case, each of your sentence needs to be in list[str] form, as if there was no tokenizer.

For any model, its input is of sentence level, which means you have to split a document into sentences beforehand. 
You may want to try :class:`~hanlp.components.eos.ngram.NgramSentenceBoundaryDetector` for sentence splitting.

Output Format

The outputs of both :class:`~hanlp_restful.HanLPClient` and 
:class:`~hanlp.components.mtl.multi_task_learning.MultiTaskLearning` are unified as the same 
:class:`~hanlp_common.document.Document` format.

For example, the following RESTful codes will output such an instance.

:tags: [output_scroll]
from hanlp_restful import HanLPClient
HanLP = HanLPClient('https://hanlp.hankcs.com/api', auth=None)  # Fill in your auth
print(HanLP('2021年HanLPv2.1为生产环境带来次世代最先进的多语种NLP技术。晓美焰来到北京立方庭参观自然语义科技公司。'))

The outputs above is represented as a json dictionary where each key is a task name and its value is the output of the corresponding task. For each output, if it's a nested list then it contains multiple sentences otherwise it's just one single sentence.

We make the following naming convention of NLP tasks, each consists of 3 letters.

```{seealso}
Each NLP task can exploit multiple datasets with their annotations, see our [annotations](annotations/index) for details.
```

Naming Convention

When there are multiple models performing the same task, their keys are appended with a secondary identifier. For example, tok/fine and tok/corase means a fine-grained tokenization model and a coarse-grained one respectively.