Link Search Menu Expand Document Documentation Menu
文档

UAX URL 电子邮件分词器

除了常规文本之外，uax_url_email 分词器还设计用于处理 URL、电子邮件地址和域名。它基于 Unicode 文本分段算法（UAX #29），能够正确地对包括 URL 和电子邮件地址在内的复杂文本进行分词。

使用示例

以下示例请求创建了一个名为 my_index 的新索引，并使用 uax_url_email 分词器配置了一个分析器

PUT /my_index
{
  "settings": {
    "analysis": {
      "tokenizer": {
        "uax_url_email_tokenizer": {
          "type": "uax_url_email"
        }
      },
      "analyzer": {
        "my_uax_analyzer": {
          "type": "custom",
          "tokenizer": "uax_url_email_tokenizer"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "content": {
        "type": "text",
        "analyzer": "my_uax_analyzer"
      }
    }
  }
}

生成的词元

使用以下请求检查使用该分析器生成的词元

POST /my_index/_analyze
{
  "analyzer": "my_uax_analyzer",
  "text": "Contact us at support@example.com or visit https://example.com for details."
}

响应包含生成的词元

{
  "tokens": [
    {"token": "Contact","start_offset": 0,"end_offset": 7,"type": "<ALPHANUM>","position": 0},
    {"token": "us","start_offset": 8,"end_offset": 10,"type": "<ALPHANUM>","position": 1},
    {"token": "at","start_offset": 11,"end_offset": 13,"type": "<ALPHANUM>","position": 2},
    {"token": "support@example.com","start_offset": 14,"end_offset": 33,"type": "<EMAIL>","position": 3},
    {"token": "or","start_offset": 34,"end_offset": 36,"type": "<ALPHANUM>","position": 4},
    {"token": "visit","start_offset": 37,"end_offset": 42,"type": "<ALPHANUM>","position": 5},
    {"token": "https://example.com","start_offset": 43,"end_offset": 62,"type": "<URL>","position": 6},
    {"token": "for","start_offset": 63,"end_offset": 66,"type": "<ALPHANUM>","position": 7},
    {"token": "details","start_offset": 67,"end_offset": 74,"type": "<ALPHANUM>","position": 8}
  ]
}

参数

uax_url_email 分词器可以通过以下参数进行配置。

参数 必需/可选 数据类型 描述
max_token_length 可选 整数 设置生成词元的最大长度。如果超出此长度，词元将按 max_token_length 中配置的长度拆分为多个词元。默认值为 255。
剩余 350 字符

有问题？。

想做贡献？ 或 。