Link Search Menu Expand Document Documentation Menu
文档

CJK 分析器

内置的 cjk 分析器可以使用以下命令应用于文本字段

PUT /cjk-index
{
  "mappings": {
    "properties": {
      "content": {
        "type": "text",
        "analyzer": "cjk"
      }
    }
  }
}

词干排除

您可以使用以下命令将 stem_exclusion 与此语言分析器一起使用

PUT index_with_stem_exclusion_cjk_analyzer
{
  "settings": {
    "analysis": {
      "analyzer": {
        "stem_exclusion_cjk_analyzer": {
          "type": "cjk",
          "stem_exclusion": ["example", "words"]
        }
      }
    }
  }
}

CJK 分析器内部

cjk 分析器由以下组件构建而成

  • 分词器：standard

  • 词元过滤器

    • cjk_width
    • lowercase
    • cjk_bigram
    • stop（类似于英文）

自定义 CJK 分析器

您可以使用以下命令创建自定义 CJK 分析器

PUT /cjk-index
{
  "settings": {
    "analysis": {
      "filter": {
        "english_stop": {
          "type":       "stop",
          "stopwords":  [ 
            "a", "and", "are", "as", "at", "be", "but", "by", "for",
            "if", "in", "into", "is", "it", "no", "not", "of", "on",
            "or", "s", "such", "t", "that", "the", "their", "then",
            "there", "these", "they", "this", "to", "was", "will",
            "with", "www"
          ]
        }
      },
      "analyzer": {
        "cjk_custom_analyzer": {
          "tokenizer": "standard",
          "filter": [
            "cjk_width",
            "lowercase",
            "cjk_bigram",
            "english_stop"
          ]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "content": {
        "type": "text",
        "analyzer": "cjk_custom_analyzer"
      }
    }
  }
}

生成的词元

使用以下请求检查使用该分析器生成的词元

POST /cjk-index/_analyze
{
  "field": "content",
  "text": "学生们在中国、日本和韩国的大学学习。123456"
}

响应包含生成的词元

{
  "tokens": [
    {"token": "学生","start_offset": 0,"end_offset": 2,"type": "<DOUBLE>","position": 0},
    {"token": "生们","start_offset": 1,"end_offset": 3,"type": "<DOUBLE>","position": 1},
    {"token": "们在","start_offset": 2,"end_offset": 4,"type": "<DOUBLE>","position": 2},
    {"token": "在中","start_offset": 3,"end_offset": 5,"type": "<DOUBLE>","position": 3},
    {"token": "中国","start_offset": 4,"end_offset": 6,"type": "<DOUBLE>","position": 4},
    {"token": "日本","start_offset": 7,"end_offset": 9,"type": "<DOUBLE>","position": 5},
    {"token": "本和","start_offset": 8,"end_offset": 10,"type": "<DOUBLE>","position": 6},
    {"token": "和韩","start_offset": 9,"end_offset": 11,"type": "<DOUBLE>","position": 7},
    {"token": "韩国","start_offset": 10,"end_offset": 12,"type": "<DOUBLE>","position": 8},
    {"token": "国的","start_offset": 11,"end_offset": 13,"type": "<DOUBLE>","position": 9},
    {"token": "的大","start_offset": 12,"end_offset": 14,"type": "<DOUBLE>","position": 10},
    {"token": "大学","start_offset": 13,"end_offset": 15,"type": "<DOUBLE>","position": 11},
    {"token": "学学","start_offset": 14,"end_offset": 16,"type": "<DOUBLE>","position": 12},
    {"token": "学习","start_offset": 15,"end_offset": 17,"type": "<DOUBLE>","position": 13},
    {"token": "123456","start_offset": 18,"end_offset": 24,"type": "<NUM>","position": 14}
  ]
}
剩余 350 字符

有问题？。

想贡献？或。